19 Commits

Author SHA1 Message Date
Michael
4791e94a73
feat(gateway): add /health/ready readiness probe backed by the database (#5166)
* feat(gateway): add /health/ready readiness probe backed by the database

## Why

GET /health only proves the process is up: it returns 200 even when the persistence engine cannot reach the database. Orchestrators already treat it as a readiness gate (docker-compose.yaml marks the gateway service healthy and nginx depends_on service_healthy), so a DB outage or a still-migrating Postgres leaves the stack 'healthy' while every request fails.

## What changed

- New GET /health/ready endpoint: bounded SELECT 1 against the existing persistence engine (deerflow.persistence.engine.get_engine) with a 2s timeout.

- Response is 200 {'status': 'ready', 'database': 'ok'} when reachable, 503 {'status': 'degraded', 'database': 'unreachable'} when the probe fails, and 200 ready with database=not_configured for backend=memory (nothing to probe).

- GET /health is unchanged (pure liveness), and /health/ready is public through the existing /health auth whitelist.

- docker-compose.yaml gateway healthcheck now polls /health/ready so service_healthy reflects database reachability.

- Documented both endpoints in backend/app/gateway/AGENTS.md.

## Surface area

- [x] Backend API - new GET /health/ready endpoint under backend/app/gateway

- [x] Sandbox / Docker - gateway healthcheck in docker/docker-compose.yaml now gates on readiness

- [ ] Frontend UI / Agents / Skills / Dependencies

- [x] Default behavior change - existing /health unchanged; the prod compose healthcheck is stricter (503 while the database is unreachable)

## Validation

- New unit tests in backend/tests/test_gateway_health.py cover ok / unreachable / not_configured probe results and the 200/503 payload mapping (6 passed).

- app.gateway.app imports cleanly and registers both /health and /health/ready.

- ruff check + ruff format clean.

## AI assistance

**Tool(s) used:** Codex (coding agent)

**How you used it:** design, implementation, and unit tests produced with AI assistance; reviewed before commit.

- [ ] I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.

* fix(helm): point the gateway readiness probe at /health/ready

## Why

Review on #5166 (willem-bd, P1): the chart still probed /health for readiness,
so Kubernetes marked the pod ready and routed traffic while the database was
unreachable - exactly the failure mode /health/ready was added to catch.

## What changed

- deploy/helm/deer-flow/templates/gateway-deployment.yaml: readinessProbe
  httpGet.path now hits /health/ready (DB-backed, 503 while the database is
  unreachable). The liveness probe stays on /health.

## Verification

- One-line path change inside the existing readinessProbe block; git diff
  confirms only the readiness path changed (liveness untouched).

## AI assistance

**Tool(s) used:** Codex (coding agent)

**How you used it:** implemented the reviewer-requested probe path change; reviewed before commit.

- [ ] I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.

* fix(gateway): readiness probe also checks the effective checkpointer/Store backend

## Why

Follow-up review on #5166 (willem-bd, P1): get_engine() only represents the
ORM backend selected by `database:`. The legacy `checkpointer:` section takes
precedence for the LangGraph checkpointer and Store, so a split configuration
(a local SQLite/memory `database:` with `checkpointer.type: postgres`) could
report 200 while the PostgreSQL backend agent runs depend on was down.

## What changed

- GET /health/ready now probes both persistence halves: the ORM engine behind
  `database:` (unchanged) and the effective LangGraph checkpointer/Store
  backend resolved with the runtime's own rule (legacy `checkpointer:` config,
  otherwise derived from `database:`), for memory/sqlite/postgres backends.

- The payload gains a `checkpointer` field with the same
  ok / not_configured / unreachable vocabulary as `database`; 503 degraded is
  returned when either probe is unreachable.

- Probes are bounded by the existing 2s timeout: sqlite via aiosqlite SELECT 1
  on the resolved path, postgres via a bounded psycopg AsyncConnection SELECT 1
  on the DSN with the configured search_path. A missing driver for a configured
  backend degrades readiness (the runtime could not run either).

- Documented the two-probe semantics in the endpoint docstring and
  backend/app/gateway/AGENTS.md.

## Verification

- New tests: healthy ORM engine + unreachable legacy checkpointer backend ->
  503 degraded with database: ok / checkpointer: unreachable; checkpointer
  probe mapping for memory/sqlite(postgres missing-driver) backends; existing
  payload tests now pin the checkpointer field.

- cd backend && python -m pytest tests/test_gateway_health.py: 11 passed.

- app.gateway.app imports cleanly; ruff check + ruff format clean.

## AI assistance

**Tool(s) used:** Codex (coding agent)

**How you used it:** design, implementation, and unit tests produced with AI assistance; reviewed before commit.

- [ ] I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.

* fix(gateway): bound /health/ready to one deadline and probe the startup checkpointer snapshot

## Why

Second round of review on #5166 (zhfeng P1/P2, willem-bd P1/P1/P2). Three
correctness issues remained in the readiness endpoint:

- The database and checkpointer probes ran sequentially, each allowed 2s, so a
  healthy response could take almost 4s - past Kubernetes' 1s default
  readinessProbe timeout and inside Docker's 3s client timeout. A slow but
  healthy backend could make every replica unready.

- The checkpointer probe re-resolved process-wide, hot-reloaded configuration
  per request, while app.state.checkpointer/store are built once from the
  startup_config snapshot in langgraph_runtime(). After a live config edit the
  endpoint could probe a backend the running gateway does not use, and a
  resolution failure was swallowed into None -> not_configured -> 200.

- The SQLite probe opened the path with aiosqlite.connect(), which creates the
  file when missing: a deleted checkpoint database was silently resurrected as
  an empty file and reported ok instead of surfacing the outage.

## What changed

- backend/app/gateway/health.py: the two probes now run concurrently beneath a
  single endpoint-wide deadline (_READINESS_DEADLINE_SECONDS=3.0) so a healthy
  response completes within one probe window (~2s), never the sum of both.
  A probe that overruns the deadline degrades the endpoint instead of hanging.

- langgraph_runtime() now records the checkpointer/Store config resolved from
  the same startup_config snapshot its checkpointer/store singletons are built
  from (app.state.checkpointer_config); /health/ready probes that snapshot and
  never re-resolves hot-reloaded config. resolve_checkpointer_config() returns
  None on resolution failure and the endpoint fails closed (503, checkpointer:
  unreachable) instead of reporting not_configured.

- The SQLite probe opens disk-backed paths with the non-creating mode=rw URI
  flag, so a missing database file stays missing and yields unreachable;
  in-memory forms (:memory:, file:...mode=memory) have nothing external to
  probe and report not_configured like the memory backend.

- Orchestrator timeouts now sit above the endpoint bound: Helm readinessProbe
  gains timeoutSeconds: 5 (Kubernetes default is 1s) and the docker-compose
  gateway healthcheck client timeout moves from 3s to 5s.

## Verification

- New regression tests: concurrent probes keep total elapsed time within one
  probe window; a probe ignoring its budget trips the endpoint deadline to 503;
  missing SQLite file stays absent and yields unreachable; in-memory SQLite
  forms map to not_configured; missing startup snapshot / config resolution
  failure fail closed to 503; resolve_checkpointer_config() raising is covered.

- cd backend && python -m pytest tests/test_gateway_health.py: 21 passed;
  tests/test_gateway_docs_toggle.py and lifespan/shutdown gateway suites pass.

- ruff check + ruff format clean on all changed files.

## AI assistance

**Tool(s) used:** Codex (coding agent)

**How you used it:** implemented the reviewer-requested concurrency/deadline, startup-snapshot probing, fail-closed resolution, and non-creating SQLite probe; reviewed before commit.

- [ ] I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.

* fix(gateway): serialize connection-opening readiness probes behind a strict gate

## Why

Review on #5166 (willem-bd, P1): every request to /health/ready opened a new
PostgreSQL connection in _probe_postgres_backend, outside both the ORM pool and
the runtime checkpointer pool. The route is public through the /health auth
prefix and nginx proxies /health/*, so concurrent unauthenticated requests
could create an unbounded number of connections (each held for up to two
seconds), exhaust PostgreSQL max_connections, and take down both normal
traffic and the readiness probe itself.

## What changed

- backend/app/gateway/health.py: connection-opening checkpointer probes
  (sqlite connect, postgres AsyncConnection.connect) now run inside a strict
  per-process gate - an asyncio.Lock cached per running event loop - so at
  most one probe connection can be in flight per worker process. Requests
  that queue behind the gate are still shed by the existing endpoint-wide
  deadline, so a flood cannot pile up new connections or open files.

- Memory and unknown-backend decisions stay outside the gate; payload and
  probe semantics are unchanged. The serialization is documented in the
  module docstring and backend/app/gateway/AGENTS.md.

## Verification

- New regression test: 8 concurrent readiness_payload() requests against an
  instrumented sqlite probe assert the maximum number of in-flight probe
  connections is 1 while every request still returns 200.

- cd backend && python -m pytest tests/test_gateway_health.py: 22 passed;
  tests/test_gateway_docs_toggle.py and tests/test_gateway_lifespan_shutdown.py
  also pass on the merged main head.

- ruff check + ruff format clean on all changed files.

## AI assistance

**Tool(s) used:** Codex (coding agent)

**How you used it:** implemented the reviewer-requested strict concurrency bound for the public readiness probe; reviewed before commit.

- [ ] I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.
2026-09-04 23:26:53 +08:00
Aari
9e0fbd60fa
fix(sandbox): isolate concurrent subagent shell sessions (#5134)
* fix(sandbox): isolate concurrent subagent shell sessions

* fix(sandbox): make execution acquire idempotent

* fix(sandbox): close execution lifecycle gaps

* fix(sandbox): serialize retained client lifecycle

* fix(sandbox): close remaining client lifecycle gaps

* fix(sandbox): unwind failed client lookup

* fix(sandbox): protect internal lease identities

* fix(sandbox): make cancellation reconciliation durable

* fix(sandbox): fence cancelled workers and IM uploads
2026-09-02 21:05:23 +08:00
Aari
340bff1107
feat(mcp): manage servers from Settings (#5022)
* feat(mcp): manage servers from settings

* fix(mcp): make settings updates targeted

* fix(mcp): reject ambiguous masked array edits

* fix(mcp): honor targeted server field deletions

* fix(mcp): preserve OAuth extension secrets

* fix(mcp): validate config before persistence

* fix(mcp): preserve environment placeholders

* fix(mcp): harden targeted configuration routes

* docs: keep gateway guidance within budget

* fix(mcp): protect per-tool override secrets

* fix(mcp): keep disabled edits structurally safe
2026-09-01 23:24:49 +08:00
rayhpeng
cd35363a05
fix(history): early user messages vanish or jump mid-run when pagination and context compaction overlap (#4696)
* fix(history): stop dropping user messages that fall outside the loaded page window

Two independent paths made a user's own message disappear from a long thread
(#4666, #4508, #4363). Both are reproduced by a real two-round run: once the
thread passes the 50-row `/messages/page` window AND context compaction fires,
the two sources of truth stop overlapping at the head.

1. Middleware-answered tool results never reached the event store. A middleware
   that short-circuits a tool call (e.g. ReadBeforeWriteMiddleware's blocked
   write) returns a user-visible ToolMessage, but LangChain never emits
   `on_tool_end`, so RunJournal never persisted it — the user saw it during the
   run and it vanished on reload. RunJournal already reconciles final-output
   tool messages, but only for an `ask_clarification` allowlist. The allowlist
   is removed; scope stays bounded by the three conditions that actually matter
   (visible, this run's lead agent, not already persisted), so subagent results
   still stay in their own step feed.

2. mergeMessages discarded the checkpoint prefix before the first shared anchor.
   #4065 correctly established that a summarization-rescued early message must
   not be appended to the tail, and suppressed it instead. That suppression is
   what deletes the message when the first history page no longer reaches back
   to it. It is now woven in before the first shared anchor — the one position
   both the checkpoint and seq-sorted history agree on — so #4065's invariant
   (never the tail) still holds. A collapsed unloaded gap is recoverable by
   paging; a dropped message is not.

Verified against real captured payloads from the reproducing run: the first user
message returns to the transcript. Its exact position is still approximate —
after compaction the live window carries too few anchors to place it precisely,
which only seq-based ordering can close.

Backend: 10809 passed (baseline 10808; same 15 pre-existing failures in
browser/crawler community tools). Frontend: 986 passed, typecheck + eslint clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(events): look up a persisted message's seq by identity

Groundwork for placing checkpoint messages in the seq-ordered thread feed
(#4666). A checkpoint carries no seq of its own and loses messages to
summarization, so once the feed's 50-row page window no longer reaches back to a
surviving old message, a client has nothing to place it by. The seq already
exists in run_events keyed by the message id — this exposes it without paging
the whole feed.

`message_identity` is the backend half of the identity rule the frontend applies
in `hooks.ts::messageIdentity`: a ToolMessage is keyed by `tool_call_id`, and
DynamicContextMiddleware's `X` / `X__user` human copies collapse to one identity.
The two halves must stay in sync — a mismatch is silent, degrading placement
rather than raising.

`get_message_seqs` is implemented for all three stores. Misses are absent from
the result rather than an error, so callers degrade to their own placement rule;
the earliest seq wins when one identity resolves to several rows, so a
re-persisted message keeps the position it first occupied. The DB store decodes
rows in Python because `content` is a TEXT column holding a JSON string, not a
JSON column — the identity fields cannot be projected in SQL.

Nothing consumes this yet; no behavior change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(runtime): carry each persisted message's feed seq on values frames

Attaches `additional_kwargs.deerflow_seq` to messages in a root `values` frame
that the thread feed already holds, so a client can place a message the
checkpoint kept but its loaded history page window no longer reaches (#4666).
Nothing is written back to the checkpoint: the seq is added when the frame is
serialized and belongs to that frame only.

Cost is bounded to frames introducing identities the run has not resolved yet.
Messages this run produces are not in the feed while streaming, so they are
looked up once, recorded as misses, and never retried — in a real run the only
frame that pays for a query is the one where compaction brings older messages
back into view. Measured on a reproducing two-round run: 1 lookup across 25
values frames.

The stamper is built once per run rather than per `_stream_once`, or a goal
continuation would discard the resolved seqs. Subgraph frames are not stamped:
a subagent's snapshot is not part of this thread's feed ordering. A lookup
failure logs and leaves the frame unstamped rather than failing it — placement
is an enhancement and clients fall back to their own ordering rule.

Frontend does not read the field yet; no behavior change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(gateway): strip the server-owned message seq from untrusted input

`deerflow_seq` is display metadata the Gateway attaches when it serializes a
values frame. A client replaying messages (regenerate / edit-and-rerun) would
otherwise write it into the checkpoint, where it becomes wrong the moment the
thread is forked — a branch re-seeds its feed and reassigns seq (#4380).

Joins the existing server-owned key set, so it follows the same trusted-internal
rule as the dynamic-context and view-image markers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(frontend): place a checkpoint message by its feed seq, not its nearest anchor

Completes #4666. Weaving a compaction-rescued message before the first shared
anchor keeps it in the transcript, but not in the right place: after compaction
the live window carries too few anchors, and the nearest one can sit deep inside
the loaded page window — measured at row 25 of 50 on a reproducing run, which is
why the first user turn rendered mid-transcript instead of at the head.

Both sides now carry the backend's thread-global seq. `buildVisibleHistoryMessages`
copies each row's `seq` onto the message (same shape as the existing `run_id`),
and the Gateway stamps it onto `values` frame messages it has already persisted.
A live message whose seq is below the loaded window's lower bound is placed ahead
of everything on screen rather than before the nearest anchor. A message with no
seq — still streaming, so not in the feed yet — keeps the weaving path, since the
tail is already its correct position.

Verified against the captured payloads of the reproducing run: the first user
message goes from absent, to #13 (behind the second question), to #0.

Frontend: 988 passed, typecheck + eslint clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(frontend): place a pre-window checkpoint message even when no anchor is shared

Also #4666. Placing a compaction-rescued message by its feed seq was gated on
reaching a shared anchor, because the split ran inside the anchor walk. When the
loaded page and the live checkpoint share no identity at all, that walk never
runs and the message fell through to `[...canonical, ...live]` — appended after
the entire window, the one arrangement #4065 proved wrong, with its seq known
the whole time.

That is not a corner case. Open an old, already-summarized conversation and send
a message: the page on screen is the newest rows from before that turn, while
the checkpoint holds the rescued first user turn plus steps of the new run that
are not in the feed yet. On a reproducing run the two sides shared zero anchors
and the user's own first question rendered at row 50 of 50 — the reported
"first message jumps to the bottom".

Split `beforeWindow` out of `live` before walking anchors, walk `liveInWindow`,
and use it for the no-anchor branch as well, so a message routed ahead of the
window is not re-appended at the tail by dedup.

Measured on captured payloads of a reproducing run (real gateway, real
compaction), first user message position:

  no shared anchor:  row 50 -> row 0, seq order monotonic again
  shared anchors:    row 0 -> row 0 (unchanged)
  paged to the top:  row 0 -> row 0 (unchanged)

Regression test verified red-green: reverting the fix fails it with the message
rendered after the window.

Frontend: 989 passed, eslint + tsc clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(gateway): stamp the message feed seq on checkpoint reads, not only on stream frames

Completes #4666. `_MessageSeqStamper` sits on the streaming publish path, so a
client that joins a live run learns where a summarization-rescued turn belongs
while a client that merely opens the conversation does not — and opening is the
common case. `GET /threads/{id}/state` and `POST /threads/{id}/history` returned
the checkpoint with no seq at all, so the merge fell back to the nearest shared
anchor, which after summarization sits deep inside the loaded page.

Reproduced in a browser against a real gateway, on a thread that had already
compacted: the user's first question rendered at row 320 of 389, behind the
newest question instead of at the head. Both reads showed 0 of 13 messages
carrying a seq. That is the reported symptom, still present after the streaming
fix.

Add `stamp_messages_with_seq`, the request-scoped counterpart of the stamper:
everything a checkpoint still holds is already persisted, so one batched lookup
resolves the whole list and there is nothing to retry later. Resolve the store
through `_optional_run_event_store` rather than `get_run_event_store`, because
seq is placement metadata — a deployment without a feed must still be able to
read a thread.

After the fix, on the same thread in the same browser: 13 of 13 messages carry a
seq and the first question renders at the head, ahead of the newest one.

Backend: ruff clean, 326 passed across the touched suites.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(harness): move the injected-user-id suffix helpers to utils.messages to break an import cycle

message_identity imported strip_injected_user_message_id_suffix from the
dynamic-context middleware, closing a cycle (middleware -> deerflow.runtime
-> worker -> events -> middleware) that only stayed hidden while an earlier
import happened to break it. Define INJECTED_USER_MESSAGE_ID_SUFFIX and the
strip helper in deerflow.utils.messages and re-export them from the
middleware so existing importers keep working.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(docs): improve formatting and clarity in AGENTS.md and message-merge.test.ts

* perf(events): stop the seq scan once every wanted identity is resolved

Rows past the last wanted seq can only be re-persisted copies that
already lose the earliest-seq-wins tiebreak, so all three stores now
break out of the scan (and the db store out of its per-row JSON
decoding) once found covers wanted. Matters most for /state and
/history reads of long threads, where this lookup runs with no run
cache and a typically tiny wanted set.

Raised by review on #4696.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(events): share the seq-stamping expression between the two stampers

The walrus-plus-merge expression was duplicated verbatim between
stamp_messages_with_seq and _MessageSeqStamper.stamp — two counterparts
of one rule where silent divergence is the likely failure mode if only
one side is edited. Both now call attach_message_seq next to
MESSAGE_SEQ_KEY in message_identity.py. The trailing
isinstance(message, Mapping) guard was unreachable (a non-Mapping entry
already got identity = None) and is gone with the extraction.

Raised by review on #4696.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(events): seq stamping survives launch paths without user context

The db store's get_message_seqs defaults to user_id=AUTO, which raises
when no user is in the contextvar — the first strict-AUTO read ever
called from the worker context. On a launch path that never inherits
the auth context (e.g. a null-owner scheduled task), stamp()'s except
clause swallowed that into a per-frame warning and silently disabled
seq stamping for exactly the background runs that need it.

The stamper now soft-resolves the user id once at build time — the
same rule as the worker's write paths beside it (unset -> no filter)
— and passes it explicitly. jsonl/memory stores gain the same
user_id kwarg the base list_messages contract already carries.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* perf(events): SQL-prefilter the message seq lookup's candidate rows

get_message_seqs scanned and JSON-decoded every message row of the
thread: the early exit never fires when a wanted identity is absent
from the feed (a message still streaming, or checkpoint-only), and
/state / /history reads want the newest messages, so the ascending
scan traversed essentially the whole feed — with the content column
carrying full tool outputs, that is heavy I/O plus N JSON parses on
exactly the long threads this lookup exists for.

A LIKE prefilter now keeps that cost in SQL: only rows containing a
wanted raw id as a substring are fetched and decoded. False positives
are re-checked by message_identity; LIKE wildcards are escaped; an id
json.dumps would escape (breaking the verbatim-substring guarantee)
falls the whole set back to the full scan rather than silently
missing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(agents): sink runtime mechanism docs below the gateway guidance budget

Merging main pushed backend/app/gateway/AGENTS.md past its 40KB soft
budget (main had left 81 bytes of headroom). Per the nearest-file rule,
move the mechanism detail of the message-seq stamping and run-delivery
receipt sections — both owned by runtime/ code — into
packages/harness/deerflow/runtime/AGENTS.md, leaving the gateway file
the REST-surface summary and a pointer. The seq section also documents
the stamper's build-time soft user-id resolution and the db store's SQL
prefilter from the review follow-ups.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(agents): sink durable-MCP task detail below the backend guidance budget

Merging main pushed backend/AGENTS.md past its 24KB module soft budget
(main itself is at 24762 after #4848 — this branch adds zero net bytes
to the file). Per the nearest-file rule, move the two durable-MCP task
runtime bullets' mechanism detail into
packages/harness/deerflow/mcp/AGENTS.md, leaving summaries and
pointers; this also restores ~2KB of headroom so the next merge does
not trip the same wire.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(events): re-ask a message-seq miss once the feed advances

The run-scoped stamper cached lookup misses for the whole run. A message
this run produces reaches a values frame before RunJournal flushes it, so
its first lookup legitimately misses — and the journal persists it moments
later, giving it a feed seq the stamper never asks for again. A long run
that afterwards rolls past the history page and compacts then carries that
message unstamped, back to the approximate anchor placement this stamper
exists to replace (#4666). A transient store error had the same permanent
effect, since the except clause degrades to an empty result.

A miss is now provisional while a hit stays final: RunJournal counts its
successful event-store writes as `feed_generation`, and the stamper re-asks
a missed identity only once that counter moves. Retrying is therefore
bounded by feed writes rather than by frames — the per-frame query the
run-scoped cache was built to avoid — and a failed lookup costs one
generation instead of the run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-01 22:04:17 +08:00
Ryker_Feng
8d8ca506ba
feat(artifacts): download run files as zip (#5117)
* feat(artifacts): download run files as zip

* fix(artifacts): address archive review feedback

* fix(artifacts): gate unavailable archive downloads

* fix(artifacts): verify archive availability

* fix(artifacts): harden archive consistency

* fix(artifacts): reject archive path aliases
2026-09-01 22:01:21 +08:00
Hyeonsang Cho
9146bfa03d
feature(gateway): issue request trace ids unconditionally (#5119)
* refactor(gateway): issue request trace ids unconditionally

The request trace id was gated behind logging.enhance.enabled at every
entry point, so downstream code had to keep asking whether one existed:
a header-provenance flag in its own ContextVar, a precedence resolver,
and three-level carrier fallbacks at each consumer.

Bind one unconditionally instead. TraceMiddleware covers Gateway HTTP;
ensure_trace_context covers the entry points that never touch ASGI --
scheduled occurrences, MCP task notification runs, IM channel messages,
and the embedded client -- each scoped to one unit of work so a
long-lived worker task cannot leak one occurrence's id into the next.
The ContextVar becomes the only source; the response header, runtime
context, run metadata and log records are derived outputs.

Consumers now use ensure_trace_id() or resolve_trace_id(*carriers) and
drop their presence guards. Removed: resolve_deerflow_trace_id, the
header-provenance flag and its three helpers, set/reset_current_trace_id,
is_trace_correlation_enabled and its gateway alias.

BREAKING CHANGE: every Gateway HTTP response now carries X-Trace-Id and
it cannot be turned off; logging.enhance.enabled controls log output
only. Installations on the default enabled: false will start seeing the
header. No config keys were added or removed.

* fix(gateway): stop persisting a caller-supplied trace id on the run record

body.metadata forks two ways: through build_run_config into the live run
config, which the run worker restamps, and through create_or_reject into
the run record that the runs API echoes verbatim. Only the first was
covered, so a client sending metadata.deerflow_trace_id made the most
durable and most visible surface of a run disagree with the X-Trace-Id
and the log lines the same request produced -- a correlation id that
does not match the logs is worse than none.

Stamp the server-issued id once at the trust boundary so both forks
receive it, preserving the caller's own metadata keys. Close the same
gap on config.context, which reaches the runtime context by a separate
path: _build_runtime_context no longer merges server-owned keys from the
caller, and _install_runtime_context assigns rather than setdefaults.

A thread's metadata is no longer seeded with the run-scoped id of
whichever run created it -- one thread spans many runs and as many
trace ids.

Found by driving a real run through the Gateway and reading the run back
from the runs API; every unit test built its metadata by hand and so
could not see it.

* fix(gateway): expose X-Trace-Id to split-origin browser clients

X-Trace-Id is not on the CORS safelist, so a browser client served from
a separate origin could not read it -- and those are exactly the clients
that cannot read the Gateway's logs either, leaving them with nothing to
quote in a bug report. Same-origin nginx deployments were unaffected,
which is why this stayed hidden.

Add it to CORS_EXPOSED_HEADERS beside Content-Location, referencing
TRACE_ID_HEADER rather than repeating the literal.

* fix(gateway): keep X-Trace-Id on unhandled-exception 500s

Starlette's ServerErrorMiddleware sits outside every user middleware and
emits unhandled-exception 500s through the raw send, so those responses
never pass TraceMiddleware's header-writing wrapper. The 500 for a server
bug is exactly the response a user most needs to correlate with a log line,
and it was the one response that shipped without the id.

TraceMiddleware now tracks whether http.response.start has been sent. On an
exception with no response started it emits its own plain 500 carrying the
header, then re-raises: the outer ServerErrorMiddleware sees the response
already started and only re-raises too, so the server's exception logging is
untouched. An exception mid-stream keeps propagating unchanged — a second
response start cannot be sent, and the already-written header stands.

The trace id is printable ASCII by construction (normalize_trace_id /
generate_trace_id), which is what makes the raw latin-1 header encoding
safe.

* fix(gateway): strip the forged trace id from the persisted request echo

The run-record fix stopped a forged metadata.deerflow_trace_id on the
authoritative metadata surface, but the raw request echo still carried one:
create_or_reject persists body.config verbatim as runs.kwargs_json, which
the runs API serves back. A client posting config.context.deerflow_trace_id
therefore still got its forged value stored and echoed on one API surface
while the header, logs, run metadata, and checkpoint all carried the real
id — the id is ignored as input there, so echoing it back only manufactures
disagreement.

Two changes close it. redact_config_secrets — already the shared scrub for
that echo, applied at admission and again at serve time, so historical
records are covered too — now also drops deerflow_trace_id from
config.metadata and config.context. And build_run_config now merges run
metadata onto a copy of the caller's config["metadata"] instead of updating
it in place: the nested values of the request config are reference copies,
so the in-place merge was writing the server-stamped key through into
body.config, contaminating the "what the client sent" record before it was
persisted (and incidentally masking the forged-value echo on the metadata
container).

The regression test posts a forged id through body.metadata,
config.metadata, and config.context at once and reads the kwargs echo back
off the run record, failing if either leak returns.

* docs(harness): record the trace-echo scrub, 500 fallback, and accepted retry divergence

The trace section of the harness AGENTS.md now covers the two fixes that
close the derived-output rule (the kwargs-echo scrub in
redact_config_secrets plus build_run_config's copy merge, and
TraceMiddleware's own 500 for unhandled exceptions), and CHANGELOG gains
their Fixed entries.

It also writes down the one accepted divergence: a crash-recovered
scheduled launch reuses the durable run through its idempotency key, and
start_run returns early on idempotency_reused without restamping — so the
run record keeps the first attempt's deerflow_trace_id while the retry's
own log lines carry the freshly minted id of its ensure_trace_context
binding. The divergence is confined to the crash-recovery window and is
accepted rather than fixed: restamping on reuse would rewrite a persisted
record for a run that already exists, which is worse than two ids that each
correlate their own attempt's logs. Written down so the next reader of the
scheduler recovery path does not diagnose it as a bug.

* docs(config): align the logging.enhance schema note with the unconditional trace id

The config-module AGENTS.md still described logging.enhance as the gate for
the Gateway X-Trace-Id header and Langfuse deerflow_trace_id. That model is
gone: ids are issued unconditionally and this block decides log output only.
Left as-is, the stale wording invites an agent to "restore" a header gate it
believes was lost. Reworded to match the sibling AGENTS.md files and
config.example.yaml, with a pointer to the Request Trace Context section
that owns the full model.

* docs(changelog): link the trace entries to #5119

The five new entries pointed at the ([#XXXX]) placeholder with no reference
definition, rendering as literal text instead of a link — and RELEASING.md
step 2 relies on those references when the section becomes release notes.
All five now point at #5119, with the definition appended to the reference
block.

* refactor(harness): rename _stream_without_trace_context to _stream_turn

The name asserted the opposite of what the method now does. It was accurate
while logging.enhance.enabled could route stream() around the trace scope;
with the gate gone it is the only stream implementation left, and it binds
the id itself via ensure_trace_id(). Private, so the rename touches only the
definition and the one stream() call site.

* docs(harness): fit the trace-context guidance inside the AGENTS.md chain budget

The expanded Request Trace Context section pushed the effective AGENTS.md
chain for agents/middlewares to 99,815 bytes, past the 98,304 hard limit
scripts/check_agent_guidance.py enforces in CI (AG002). Compressed the
section from 7,359 to 4592 bytes with no facts removed: the entry-point
table, the derived-output rule and its enforcement points, the accepted
scheduled-retry divergence, the two resolution helpers, the stream()
binding rationale, the log-output-only gate, the CORS listing, the 500
fallback, and the test map all remain.

Sized against the merge, not just the branch: current main grew the same
chain by ~724 bytes, so the check was verified on the merged tree as well
(97,772 bytes; branch tree 97,048).

* fix(gateway): declare content-length on the fallback 500

The pre-response 500 declared content-type but no content-length, leaving
the framing to the ASGI server: chunked on HTTP/1.1, close-delimited on
HTTP/1.0 — the one wire difference from the ServerErrorMiddleware response
it replaces, which sends content-length: 21. The explicit header keeps the
fallback byte-identical to what clients saw before.

* docs(readme): drop the trace-correlation condition from the translations

The zh/ja/fr/ru Langfuse sections still said metadata.deerflow_trace_id
matches X-Trace-Id "when request trace correlation is enabled". The id now
always matches and that condition no longer exists, so each bullet states
the unconditional match and that logging.enhance.enabled only controls
whether the id is printed into logs — the one piece of the feature a user
can still configure.

* test(gateway): pin TraceMiddleware wiring through create_app()

Every X-Trace-Id test exercised a hand-built four-route app, so the real
stack's add_middleware(TraceMiddleware) line was pinned by nothing: deleting
it — or short-circuiting above it — passed CI while silently dropping both
the response header and the ambient id the run-record stamp and enhanced log
records derive from. One case now drives /health through create_app() and
asserts the inbound id round-trips; mutation-checked by removing the wiring
line, which fails exactly this test.

* docs(gateway): note the fallback 500 is CORS-opaque

The pre-response 500 is emitted outside CORSMiddleware — the exception has
already unwound past it — so it carries no Access-Control-Allow-Origin and
a split-origin browser client cannot read the id on this one response,
unchanged from the ServerErrorMiddleware 500 it replaces. Documented on the
class and in the CHANGELOG entry rather than fixed: replicating the origin
allowlist outside CORSMiddleware would let the two policies drift.

* fix(harness): keep abandoned-stream cleanup inside the trace binding

stream() binds the turn's id around each next(inner) and resets it before
yielding, but the finally's inner.close() ran after that binding was gone.
Abandoning the stream therefore drove the inner LangGraph generator's
GeneratorExit/finally path with no trace id — or an unrelated ambient one
from whichever context ran the close — so cancellation and finalization
logs and callbacks did not correlate with the turn they belong to.

inner.close() is now wrapped in a local bind/reset of the same turn id. The
token is set and reset in the same frame, never across a yield, so the
per-step cross-context safety is preserved even when GC closes the
generator from another Context — pinned by the existing copy_context close
test, which now exercises this path. The regression test records the id
from the inner generator's finally and fails without the binding.

* test(harness): teach the worker-trace fake about RunManager.cleanup

Upstream #5112 (bound gateway memory after terminal runs) added a
run_manager.cleanup(run_id) call to run_agent's finalization, so the
merge-commit CI run failed all five worker-trace-binding tests with
AttributeError on this PR's _FakeRunManager. The fake gains the same no-op
shape as its other methods.

* docs(gateway): bring the gateway AGENTS.md back under its soft budget

Upstream #5092 grew backend/app/gateway/AGENTS.md to 40,966 bytes, 6 over
the 40,960 soft budget that
test_agent_guidance_check.py::test_repository_guidance_stays_below_soft_budgets_and_avoids_doc_indexes
enforces — its Unit Tests run on main was cancelled by push concurrency, so
main is currently red on that test and every PR merge-run inherits the
failure. Two whitespace/wording trims in the row #5092 touched (a doubled
space, and "its configured `context_window`" → "its `context_window`")
bring the file to 40,953 with no content change.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-01 16:49:39 +08:00
Willem Jiang
56a7185f30
fix(ci): fix the ci check of gateway AGENTS.md (#5130)
* fix(ci): fix the ci check of gateway AGENTS.md

* fix(ci): fix the ci check of gateway AGENTS.md
2026-09-01 16:34:45 +08:00
Sunshine
a956bbc030
fix(runs): reject cancel actions on GET stream joins (#5092)
* fix(runs): reject cancel actions on GET stream joins

stream_existing_run is registered for both GET and POST, and its
?action=interrupt|rollback branch cancels the run. The CSRF middleware
exempts GET, so a session-authenticated browser could be forced
cross-site (img/script/top-level navigation) into
GET /api/threads/{id}/runs/{run_id}/stream?action=interrupt|rollback —
a state-changing GET that bypasses the CSRF protection guarding the
POST variant. Introduced with the dual registration in #1403.

The handler's docstring already documents cancel-then-stream as
POST-only (the LangGraph SDK's joinStream/useStream stop button uses
POST); enforce it: GET with an action answers 405, action-less GET
joins and POST cancel-then-stream are unchanged.

Regression drives the real router: GET+action is 405 with the run left
running, plain GET join still streams, POST+action still cancels.

* fix(runs): scope the 405 detail to the action requirement

"GET is a read-only stream join" overstates the current main: on a
locally-owned run with the default on_disconnect=cancel, a GET join's
disconnect can still trigger cancellation. That observer-disconnect
vector is closed by #5041; the detail here should only claim what this
guard enforces.

* fix(runs): harden GET stream action rejection

* fix(runs): align stream schema with method contract

* test(runs): pin GET stream action 405 through the production stack

Review follow-up (defence-in-depth): the GET-action suite drove bare
FastAPI() apps, so nothing pinned that a session-authenticated
cross-site GET reaches the route gate at all once CSRF exempts the
safe method. test_pat_auth.py already assembles the production
middleware order (AuthMiddleware inner, CSRFMiddleware outer), so its
mirror app now registers the real _reject_get_stream_action
dependency on a GET join route.

The new case pins the end-to-end premise: an authenticated GET
?action=interrupt is answered 405 + Allow: POST by the production
route dependency, while the same unauthenticated GET dies at
AuthMiddleware's 401 before any route logic runs.

Validation: focused suites (test_pat_auth, test_stream_get_action,
test_csrf_middleware) — 62 passed; ruff check + format clean; the new
case errors on the pre-fix baseline (guard absent), confirming the
pin.
2026-09-01 15:51:37 +08:00
PeaceMaker-best
72ba661b84
feat(skills): install local skill archives (#5039)
* feat(skills): install local skill archives

Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>

* fix(skills): enforce upload limits before parsing

* fix(nginx): scope skill upload limit to upload route

* fix(nginx): harden skill upload proxy handling

* fix(skills): improve archive upload feedback

* fix(skills): address upload review polish

---------

Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>
Co-authored-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>
2026-08-31 15:22:45 +08:00
Sunshine
bf740ffa90
feat(auth): add personal access tokens for programmatic API access (#5041)
* feat(auth): add personal access tokens for programmatic API access (#4849)

Backend-first implementation of the PAT contract from #4849: show-once
dfp_ tokens bound to their owning user (AUTH_SOURCE_PAT,
is_internal=false), digest-only storage (migration 0017), strict
credential precedence (invalid Bearer is a 401, never cookie fallback),
CSRF double-submit skipped only for Bearer requests while
auth-endpoint origin checks still run, scopes intersecting the authz
route permissions, session-auth-only PAT management and password
changes, and throttled best-effort last_used_at stamps.

* fix(auth): harden PAT scope boundary and schema parity from adversarial review

Independent review of the initial draft found: (1) scopes only constrained
the threads/runs permission axis while admin routes treated a PAT as its
(possibly admin) owner — is_admin_user now rejects PAT callers outright
since no scope grants admin capability; (2) the model declared a column
UNIQUE constraint while migration 0017 created a named unique index, so
downgrade failed on create_all-bootstrapped DBs — both now use the named
unique index; (3) auth-disabled mode is an operator override and now stays
ahead of the Bearer check so a stray Authorization header cannot 401 an
E2E sandbox; plus wiring the previously-unused constants, bounding the
last_used_at stamp cache, and four new tests (middleware-level expiry,
expires_in_days, admin-capability rejection with session control, and the
auth-disabled precedence).

* docs(api): document personal access tokens for programmatic API access

* fix(auth): close PAT security boundaries from review (default-deny routes, extension admin suppression)

P1-1: scope intersection only constrains @require_permission routes, so
undecorated mutation routes (DELETE /api/memory, POST /api/agents, Lark
credential switching, channel config) accepted a PAT holding a single read
scope. AuthMiddleware now enforces a default-deny route policy in
auth/pat.py: PAT requests are admitted only to the thread/run lifecycle
routes the v1 scopes govern; everything else answers 403 regardless of
scopes. Session-cookie callers are unaffected.

P1-2: the extension principal resolver projected is_admin/roles from the
raw system_role, so an admin-owned PAT passed
deerflow_extension_api.require_admin on contributed routes despite the
documented no-admin guarantee. The projection is now PAT-aware and
suppresses every admin signal for PAT callers, mirroring
deps.is_admin_user.

Both fixes carry regression tests (route outside policy 403 + session
control; production resolver admin suppression), and API.md documents the
default-deny boundary.

* fix(auth): enforce PAT scopes on stateless run entry and harden decorator

Follow-up hardening from an independent audit of the P1 fixes:

- POST /api/runs/stream and /api/runs/wait were the only allowlisted run
  entrypoints without @require_permission, so a threads:read-only PAT
  could still start runs (same bug class as P1-1, now closed): both now
  carry @require_permission("runs", "create"). POST /api/threads and
  POST /api/threads/search gain threads:write / threads:read for the
  same reason. Authorization-disabled deployments see no change (the
  permission set resolves to all permissions).
- require_permission now binds the wrapped signature to locate a
  positionally-passed request before injecting the test stub, fixing
  'got multiple values for argument' on direct positional unit-test
  calls.
- API.md: the intro PAT example used GET /api/models, which the new
  default-deny policy 403s — replaced with GET /api/threads; the
  default-deny route list now spells out method sets.

Regression test: threads:read-only PAT is 403 on the decorated stateless
entry while a runs:create PAT passes.

* fix(auth): address review P2s (empty Authorization header, PAT name trimming, API example)

- CSRFMiddleware treats an explicitly empty Authorization header as
  present (is None), so an invalid credential always reaches
  AuthMiddleware's uniform 401 instead of a CSRF 403 that varies by
  method/CSRF state. Regression: empty-header request dies at auth.
- PATCreateRequest strips the name and rejects whitespace-only values
  before token generation; created names are stored trimmed.
- API.md intro PAT example now uses the implemented
  POST /api/threads/search endpoint (GET /api/threads does not exist).
- AGENTS.md trimmed back under the guidance soft budget after the
  upstream merge.

* fix(auth): tighten PAT route policy to implemented methods only

The allowlist admitted GET /api/threads, a method no router implements.
Pre-authorizing a dead method weakens the default-deny boundary: a
future GET collection route added without a permission decorator would
become PAT-reachable without an explicit policy change. Restrict the
rule to POST, fix the stale GET description in API.md's PAT
constraints, and document the default-deny boundary accurately in the
gateway AGENTS.md guidance (only the threads/runs allowlist is
PAT-reachable; every other authenticated route 403s PAT callers).

Audited every remaining rule against the mounted routers: all other
method+path entries map to real routes. Regression:
test_pat_policy_does_not_pre_authorize_unimplemented_methods.

* test(auth): guarantee the negative digest test mutates the token

token[:-1] + "X" is identical to the original whenever the generated
token already ends in X (1/62), making the negative digest assertion
fail intermittently. Choose the replacement character based on the
existing tail so the mutated token always differs.

* fix(auth): require runs:cancel for cancel-then-stream requests

stream_existing_run is gated at runs:read so action-less stream joins
work with read-only credentials, but its ?action=interrupt|rollback
branch cancels the run — a separate permission. A runs:read-only PAT
passed both the PAT route policy and the route decorator and could
interrupt or roll back an active run, bypassing the runs:cancel scope.

Decorators cannot express query-parameter-conditional permissions, so
the check lives in require_cancel_permission_when_action(), applied at
the top of the handler. Regression drives the real helper through the
production middleware: runs:read-only PAT + action is 403, the same
token joins action-less, runs:read+cancel passes, session control
unaffected.

* docs(changelog): add the PAT feature entry

* docs(readme): add personal access tokens section

Repo documentation-update policy requires user-facing features to
update README.md in the same changeset; the PAT feature previously
touched only backend/docs/API.md and the gateway AGENTS.md.

* fix(auth): require runs:cancel for mutating multitask strategies

All five run-creation entrypoints were gated only by runs:create, but
RunCreateRequest.multitask_strategy accepts interrupt/rollback and
start_run forwards it to create_or_reject, which terminates an
already-active run. A runs:create-only PAT could therefore kill an
existing run through a create request, bypassing runs:cancel.

Decorators cannot express body-parameter-conditional permissions, and
per-route checks leave the same hole for the next entrypoint, so the
gate lives in start_run itself — the single choke point every
run-creation path (HTTP routes and internal launchers) flows through.
Regenerate launches pass multitask_strategy="reject" and are
unaffected; requests without a stamped auth context (internal/test
compositions) skip the gate.

The check is the shared authz.require_cancel_permission_if primitive;
require_cancel_permission_when_action now delegates to it, so every
request dimension that carries cancel capability (query action, body
strategy) flows through one gate.

Regression drives the real middleware stack: runs:create-only PAT +
interrupt/rollback is 403 with the exact detail, reject (explicit and
default) stays available, runs:create+cancel passes, session control
unaffected; a source anchor pins the gate inside start_run.

* fix(runs): keep observer joins from applying creator cancel-on-disconnect

sse_consumer's finally block applied the record's on_disconnect=cancel
policy on ANY consumer's disconnect. The join surfaces (GET /join and
the action-less GET/POST stream join) feed it the existing RunRecord,
so anyone with thread read access — including a runs:read-only PAT —
could cancel a locally-owned running run simply by closing the SSE
connection, without runs:cancel. The policy expresses the creator's
intent for their own connection; an observer's disconnect must never
be read as that intent.

sse_consumer gains apply_on_disconnect (default True). The two join
surfaces pass False; the creating endpoints (thread-scoped and
stateless create-and-stream) keep the creator semantics unchanged.
wait_for_run_completion needs no change: its callers are creator-side
or post-explicit-cancel paths only.

Regression exercises a real generator close — the same machinery
Starlette drives on client disconnect — against the production
sse_consumer: creator stream disconnect cancels, observer join
disconnect does not; a wiring anchor pins both join call sites and the
creator defaults. API.md documents the cancel-capability constraint
(this fix plus the action/strategy gates) in PAT Constraints.

* test(auth): pin the multitask gate behaviorally; state wait invariant

Independent adversarial review of the round-5 fixes found the P1-a
regression only mirror-pinned: the source anchor could be satisfied by
a comment, and deleting the gate from start_run would not fail the
suite. This drives the production start_run directly — a create-only
auth context gets 403 with the exact detail for interrupt, and a
reject request with no cancel permission at all proceeds past the gate
(never a permission 403).

Also documents wait_for_run_completion's creator-side invariant
(every caller is the creating endpoint or post-explicit-cancel) so a
future observer wiring thinks twice before reusing it — the one-caller-
away variant of the observer-disconnect P1.

* docs(changelog): correct the PAT entry's digest and route-policy description

The entry said HMAC digests (the implementation stores SHA-256 digests,
as documented in API.md and pinned by the repository tests) and claimed
the route policy admits 'implemented stateless endpoints' (it admits
the thread/run lifecycle routes, narrowing further by scopes). Also
notes the cancel-capability gate now covering action and multitask
strategies.

* fix(auth): enumerate the PAT runs route policy per implemented subroute

The runs subtree rule was a GET|POST /runs(/.*)? wildcard — it
pre-authorized every current and future subroute under /runs, including
methods the router never implemented (e.g. GET /runs/stream), which is
the same latent default-deny weakening the threads collection rule was
tightened for: a future route added under /runs would become
PAT-reachable without an explicit policy change.

The wildcard is replaced with six segment-precise rules covering exactly
the 14 implemented method+path combinations; the {run_id} slot
necessarily matches any single segment, so the POST-only collection
names (stream, wait, regenerate, edit-regenerate) are excluded from the
GET run-id rule via negative lookahead — no dead method stays
pre-authorized. Behavior for implemented routes is unchanged.

test_pat_runs_policy_admits_exactly_the_mounted_routes derives the
expected set from the mounted thread_runs router instead of a
hand-maintained list: every implemented GET/POST route under /runs must
be admitted, routes in this router outside the subtree stay denied, and
representative unimplemented neighbors are denied — so adding a route
under /runs now fails CI until it is explicitly allowlisted, and a
removed route leaves a dead rule visible. API.md's PAT constraints list
the enumerated routes and drops a feedback mention that belonged to the
stateless /api/runs axis.

* docs(migration): add the 0017 renumbering coordination note to 0017

The PR's migration-coordination comment states each migration file
carries the note; the file did not. Adds it: numbering was generated
against main head 0016 alongside #5078 and #4843; whoever merges first
keeps the slot, the others renumber on rebase (revision/down_revision
plus the bootstrap head assertions).

* fix(auth): pad base62 tokens to a fixed 43-char width

int.from_bytes discards leading zero bytes, so the unpadded encoder
returned a variable-length body — empty for all-zero input, and shorter
than 40 characters for any draw below 62**39 (~1 in 14.5M), leaving
test_generate_pat_token_format probabilistically flaky and the token
body without stable width (review round 6, P3).

_base62 now left-pads with "0" to _base62_width(len(data)) — the exact
integer digit count (62^43 > 2^256 > 62^42, so 43 for 32 bytes). The
format test asserts the exact fixed width instead of a probabilistic
floor, and a new unit test pins the all-zero, leading-zero-byte, and
max-value edges deterministically.
2026-08-29 23:50:45 +08:00
rayhpeng
ed336ec3dd
fix(gateway): enforce run-create authorization on stateless endpoints (#5030)
* fix(gateway): enforce authz on stateless runs

* fix(gateway): guard scheduled run creation
2026-08-27 14:59:31 +08:00
Xinmin Zeng
943d148e5e
feat(threads): distinguish branched conversations (#4983)
* feat(threads): number branched conversation titles

* feat(frontend): show branch lineage in recent chats

* fix(threads): allocate unique branch suffixes

* fix(threads): preserve suffix and filter semantics
2026-08-25 08:29:18 +08:00
Aari
ff0a6768c2
feat(subagents): add unified capacity and durable batch execution (#4998)
* feat(subagents): add capacity controls and durable batches

* fix(helm): sync subagent config schema version

* fix(subagents): preserve batch history without worker

* fix(subagents): support explicit factory runtimes

* fix: address durable batch review findings
2026-08-25 07:49:38 +08:00
Aari
1aa813ddb3
feat: add managed subagents and delegation scopes (#4887)
* feat: manage and scope subagents

* fix: address subagent review feedback

* fix: address managed subagent review feedback

* fix: harden subagent settings semantics

* fix: harden managed subagent cache invalidation

* fix: reuse assembled lead agent inputs

* fix: migrate managed subagent definitions

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-08-24 11:04:23 +08:00
Aari
5ffc2d3e27
feat(mcp): complete durable task notifications and chat UI (#4833)
* feat(mcp): add reliable task notifications and cancellation

* feat(mcp): add background task chat UI

* fix(mcp): hide and sanitize task notification prompts

* fix(mcp): sanitize projected task names

* fix(mcp): harden task notifications and details

* fix(mcp): harden task lifecycle recovery

* fix(mcp): gate task UI and isolate cancellations

* test: scope plain-text response locator

* fix(mcp): align task notification boundaries

* fix(mcp): bound task delivery retries

* fix background task notification races
2026-08-22 16:53:32 +08:00
Mason Zhou
a181c3398b
fix: support Studio file-based app loading (#4838)
* fix: support Studio file-based app loading

* docs: clarify Studio loader invariant
2026-08-16 23:54:06 +08:00
Mason Zhou
432c09f6b0
fix: restore standalone LangGraph Studio compatibility (#4760)
* fix: restore standalone LangGraph Studio compatibility

* fix: secure standalone Studio assistant ownership

* fix: harden Studio provenance reconciliation

* fix: repair Studio persistence before runtime startup

* fix: harden standalone Studio compatibility
2026-08-15 21:20:34 +08:00
Aari
47b258ebd7
feat(mcp): add ordinary durable task driver (#4690)
* feat(mcp): add durable task runtime foundation

* fix(chart): sync embedded config version

* fix(mcp): isolate task polls during shutdown

* feat(mcp): track consecutive poll errors on mcp_tasks

poll_attempt_count grows on every claim (successful polls included), so it
cannot drive a failure backoff without misjudging normal long tasks. Add
consecutive_poll_error_count: incremented when a claim is released after a
poll error, reset to zero by any applied snapshot. The backoff/terminal
policy that consumes it lands with the first concrete driver.

* fix(mcp): harden durable task lifecycle

* feat(mcp): add ordinary durable task driver

* test(mcp): address durable task review feedback

* fix(mcp): preserve submit tool descriptions

* fix(mcp): bound remote task calls

* fix(mcp): bound persisted task payloads

* fix(mcp): preserve task tool error details

* fix(mcp): enforce durable task boundaries

* test(mcp): cover task config snapshot lifecycle

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-08-15 14:26:38 +08:00
Ryker_Feng
ccff5f5ce7
docs: govern agent guidance size (#4799)
* docs: govern agent guidance size

* refactor: split agent guidance by code scope

* Clarify virtual path handling in AGENTS.md

Updated the translation section to clarify the role of `LocalSandboxProvider` and the handling of virtual paths in the tool layer.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-08-13 21:49:04 +08:00