* feat(artifacts): preview CSV and TSV files as bounded tables
* chore: keep preview screenshots out of the PR file diff
* fix(artifacts): detect record newlines outside quoted fields
* test(auth): include project permissions in me contract expectations
* feat(projects): project workspaces with scoped chats and thread membership
Backend:
- projects table model and migration; fail-closed ProjectRepository with
ownership checks, CRUD/archive/restore/delete router, and atomic thread
move between projects
- threads_meta.project_id column exposed as reserved deerflow_project_id
metadata; project-aware thread create/search with pagination bounds and
membership echoed in create responses
- first-run admission assigns the project only at genuine first run, seeded
at write time and dropped when invalid; serialized against project
deletion and thread assignment
- branch creation inherits the source thread's project membership (an
archived/deleted project degrades the branch to unassigned instead of
failing the request)
Frontend:
- projects data layer, thread move API, and sidebar projects section with
flat/grouped modes, archived-project threads, and stable virtual-list
offsets
- project detail page with project-scoped new chat
(/workspace/chats/new?project=) and paginated thread list
- move-to-project thread menu, new-project dialog, archived-project gates
- project-scoped new chats pre-create the thread with membership before the
first submit or /goal set, so runs never proceed outside the project
- goal-set preparation is fenced against conversation switches: a stale
continuation is dropped instead of saving the goal or launching the
abandoned submission on the newly opened conversation
- project thread lists join thread lifecycle invalidations (stop, pin) so
an open project page never keeps stale titles, recency, or pagination
* fix(chats): keep archive undo toast when the sidebar row unmounts
The archive success toast was fired from per-mutate callbacks passed to
mutation.mutate. React Query drops those handlers when the observer
component unmounts before the mutation settles; archiving the open chat
removes its sidebar row mid-flight, so the undo toast never appeared and
the e2e archive-undo test timed out waiting for it.
Move the success/error handlers to the mutation level (useArchiveThread
options, same pattern as useMoveThreadToProject) where callbacks are
delivered even after the originating row unmounts.
* fix(projects): pin project thread listing contract and exclude archived chats
GET /api/projects/{id}/threads returned the thread store row verbatim
(list[dict], no response_model): user_id/assistant_id leaked, any future
ThreadMetaRow column would auto-leak, and the OpenAPI schema was empty.
Return a narrow ProjectThreadResponse (the exact fields ProjectThread
declares) with the same metadata secret redaction the surrounding thread
endpoints get from _MetadataRedactingResponse.
The listing also ran search() without the archived filter, so a retired
chat rendered as a normal row on the project page while the sidebar hid
it. Search archived=False to mirror the sidebar's archived:false lists;
restore stays on the global Archived tab.
Both regressions pinned by new router tests: wire-shape allowlist and
archived-member exclusion.
* docs(migrations): record the 0019/0020 chain against the bootstrap reservation
The tree now chains 0018 -> 0019_projects -> 0020_threads_meta_project_id,
so migrations/AGENTS.md was stale twice over: the revision index stopped at
0018 and the rolling-forward section still claimed the tree 'deliberately
remains at 0018'.
Document the new head and record the intentional numeric-prefix reuse of
0019: 0019_projects is in-chain while 0019_thread_incarnations stays the
reserved, allowlisted out-of-tree rollout id. The owning rollout revision
must re-parent onto this tree's head when it merges so alembic never sees
two heads off 0018; bootstrap.py now cross-references that note next to
_FORWARD_COMPATIBLE_REVISION.
* fix(chats): invalidate project thread lists on archive/restore
useArchiveThread refreshed the infinite sidebar cache, threads/search and
the per-thread metadata cache but not the project-scoped list
([...PROJECTS_QUERY_KEY, 'threads', id]) this PR adds — the one thread
mutation not wired to that key, after usePinThread, useRenameThread,
useDeleteThread, useMoveThreadToProject and invalidateStoppedThreadCaches.
An archive from a sidebar row while a project page is open therefore left
the archived chat rendered as a normal row until remount (and undo left it
missing). Invalidate the prefix in the mutation-level success handler.
Regression test asserts the project-list prefix is invalidated on success.
* fix(projects): fetch project discovery only in grouped sidebar mode
RecentChatList mounted two useProjects queries per sidebar render, but
knownProjectIds is consumed only by the grouped-mode exclusion filter; in
the default flat mode every page load paid two GET /api/projects?status=
round trips for data nothing read. Gate both queries on grouped mode —
GroupedProjectList fetches the same keys when the toggle is on and
TanStack dedupes the observers.
Also set retry: false on useProject: a deleted or foreign project 404s
deterministically, and the page renders a dedicated not-found state for
it, so the default 1s/2s/4s retry backoff kept deep links in 'loading'
for ~7s before that state appeared. Matches useThreadMetadata /
useThreadTokenUsage.
* fix(threads): fail closed on project-scoped create in memory mode
MemoryThreadMetaStore.create accepted project_id and silently ignored it,
making memory mode the one membership path that fails open: POST
/api/threads with a project id returned 200 and the run started
unassigned, violating the invariant that a run never proceeds outside the
selected project (the SQL store raises ProjectNotAssignableError inside
the insert transaction for the same request).
Raise ProjectNotAssignableError whenever project_id is present so the
router's existing 404 mapping applies, the frontend keeps the composer
text for a retry, and memory mode behaves exactly like SQL mode.
set_project already reports rejection; create now matches it.
Store-level test (raises, nothing persisted, project filter stays empty,
unscoped creates still work) plus a router-level test asserting the 404
and that no row is left behind.
* fix(projects): window the project page thread list
ProjectThreadsSection rendered every loaded page as a plain Link row, so a
long-lived project accumulated unbounded DOM on the page's scroll surface:
each load-more appended another 100 rows and every formatTimeAgo tick
re-rendered the whole list.
Reuse VirtualThreadList (now generic over any row shape with a
thread_id), pointing its scroll parent at this page's ScrollArea viewport
via the shared [data-slot="scroll-area-viewport"] selector used by
/workspace/chats; under the 60-row threshold it falls back to the plain
render, so small projects are unchanged.
* fix(projects): restore row dividers and pin them with a render test
The row class template literal concatenated transition-colors directly
with the conditional border-b token, so non-final rows rendered the
invalid class 'transition-colorsborder-b' and lost both the divider and
the transition. Compose the row classes with cn() and a boolean guard
instead.
The section moved out of page.tsx into a testable component so the row
markup finally has coverage: a DOM test asserts every row except the
final data row carries border-b (index-based, not last: — correct under
virtualization where the last mounted row is not the last data row), and
the untitled fallback plus load-more button render for a partial page.
* fix(projects): validate forward schemas and fence membership reads
* refactor(gateway): issue request trace ids unconditionally
The request trace id was gated behind logging.enhance.enabled at every
entry point, so downstream code had to keep asking whether one existed:
a header-provenance flag in its own ContextVar, a precedence resolver,
and three-level carrier fallbacks at each consumer.
Bind one unconditionally instead. TraceMiddleware covers Gateway HTTP;
ensure_trace_context covers the entry points that never touch ASGI --
scheduled occurrences, MCP task notification runs, IM channel messages,
and the embedded client -- each scoped to one unit of work so a
long-lived worker task cannot leak one occurrence's id into the next.
The ContextVar becomes the only source; the response header, runtime
context, run metadata and log records are derived outputs.
Consumers now use ensure_trace_id() or resolve_trace_id(*carriers) and
drop their presence guards. Removed: resolve_deerflow_trace_id, the
header-provenance flag and its three helpers, set/reset_current_trace_id,
is_trace_correlation_enabled and its gateway alias.
BREAKING CHANGE: every Gateway HTTP response now carries X-Trace-Id and
it cannot be turned off; logging.enhance.enabled controls log output
only. Installations on the default enabled: false will start seeing the
header. No config keys were added or removed.
* fix(gateway): stop persisting a caller-supplied trace id on the run record
body.metadata forks two ways: through build_run_config into the live run
config, which the run worker restamps, and through create_or_reject into
the run record that the runs API echoes verbatim. Only the first was
covered, so a client sending metadata.deerflow_trace_id made the most
durable and most visible surface of a run disagree with the X-Trace-Id
and the log lines the same request produced -- a correlation id that
does not match the logs is worse than none.
Stamp the server-issued id once at the trust boundary so both forks
receive it, preserving the caller's own metadata keys. Close the same
gap on config.context, which reaches the runtime context by a separate
path: _build_runtime_context no longer merges server-owned keys from the
caller, and _install_runtime_context assigns rather than setdefaults.
A thread's metadata is no longer seeded with the run-scoped id of
whichever run created it -- one thread spans many runs and as many
trace ids.
Found by driving a real run through the Gateway and reading the run back
from the runs API; every unit test built its metadata by hand and so
could not see it.
* fix(gateway): expose X-Trace-Id to split-origin browser clients
X-Trace-Id is not on the CORS safelist, so a browser client served from
a separate origin could not read it -- and those are exactly the clients
that cannot read the Gateway's logs either, leaving them with nothing to
quote in a bug report. Same-origin nginx deployments were unaffected,
which is why this stayed hidden.
Add it to CORS_EXPOSED_HEADERS beside Content-Location, referencing
TRACE_ID_HEADER rather than repeating the literal.
* fix(gateway): keep X-Trace-Id on unhandled-exception 500s
Starlette's ServerErrorMiddleware sits outside every user middleware and
emits unhandled-exception 500s through the raw send, so those responses
never pass TraceMiddleware's header-writing wrapper. The 500 for a server
bug is exactly the response a user most needs to correlate with a log line,
and it was the one response that shipped without the id.
TraceMiddleware now tracks whether http.response.start has been sent. On an
exception with no response started it emits its own plain 500 carrying the
header, then re-raises: the outer ServerErrorMiddleware sees the response
already started and only re-raises too, so the server's exception logging is
untouched. An exception mid-stream keeps propagating unchanged — a second
response start cannot be sent, and the already-written header stands.
The trace id is printable ASCII by construction (normalize_trace_id /
generate_trace_id), which is what makes the raw latin-1 header encoding
safe.
* fix(gateway): strip the forged trace id from the persisted request echo
The run-record fix stopped a forged metadata.deerflow_trace_id on the
authoritative metadata surface, but the raw request echo still carried one:
create_or_reject persists body.config verbatim as runs.kwargs_json, which
the runs API serves back. A client posting config.context.deerflow_trace_id
therefore still got its forged value stored and echoed on one API surface
while the header, logs, run metadata, and checkpoint all carried the real
id — the id is ignored as input there, so echoing it back only manufactures
disagreement.
Two changes close it. redact_config_secrets — already the shared scrub for
that echo, applied at admission and again at serve time, so historical
records are covered too — now also drops deerflow_trace_id from
config.metadata and config.context. And build_run_config now merges run
metadata onto a copy of the caller's config["metadata"] instead of updating
it in place: the nested values of the request config are reference copies,
so the in-place merge was writing the server-stamped key through into
body.config, contaminating the "what the client sent" record before it was
persisted (and incidentally masking the forged-value echo on the metadata
container).
The regression test posts a forged id through body.metadata,
config.metadata, and config.context at once and reads the kwargs echo back
off the run record, failing if either leak returns.
* docs(harness): record the trace-echo scrub, 500 fallback, and accepted retry divergence
The trace section of the harness AGENTS.md now covers the two fixes that
close the derived-output rule (the kwargs-echo scrub in
redact_config_secrets plus build_run_config's copy merge, and
TraceMiddleware's own 500 for unhandled exceptions), and CHANGELOG gains
their Fixed entries.
It also writes down the one accepted divergence: a crash-recovered
scheduled launch reuses the durable run through its idempotency key, and
start_run returns early on idempotency_reused without restamping — so the
run record keeps the first attempt's deerflow_trace_id while the retry's
own log lines carry the freshly minted id of its ensure_trace_context
binding. The divergence is confined to the crash-recovery window and is
accepted rather than fixed: restamping on reuse would rewrite a persisted
record for a run that already exists, which is worse than two ids that each
correlate their own attempt's logs. Written down so the next reader of the
scheduler recovery path does not diagnose it as a bug.
* docs(config): align the logging.enhance schema note with the unconditional trace id
The config-module AGENTS.md still described logging.enhance as the gate for
the Gateway X-Trace-Id header and Langfuse deerflow_trace_id. That model is
gone: ids are issued unconditionally and this block decides log output only.
Left as-is, the stale wording invites an agent to "restore" a header gate it
believes was lost. Reworded to match the sibling AGENTS.md files and
config.example.yaml, with a pointer to the Request Trace Context section
that owns the full model.
* docs(changelog): link the trace entries to #5119
The five new entries pointed at the ([#XXXX]) placeholder with no reference
definition, rendering as literal text instead of a link — and RELEASING.md
step 2 relies on those references when the section becomes release notes.
All five now point at #5119, with the definition appended to the reference
block.
* refactor(harness): rename _stream_without_trace_context to _stream_turn
The name asserted the opposite of what the method now does. It was accurate
while logging.enhance.enabled could route stream() around the trace scope;
with the gate gone it is the only stream implementation left, and it binds
the id itself via ensure_trace_id(). Private, so the rename touches only the
definition and the one stream() call site.
* docs(harness): fit the trace-context guidance inside the AGENTS.md chain budget
The expanded Request Trace Context section pushed the effective AGENTS.md
chain for agents/middlewares to 99,815 bytes, past the 98,304 hard limit
scripts/check_agent_guidance.py enforces in CI (AG002). Compressed the
section from 7,359 to 4592 bytes with no facts removed: the entry-point
table, the derived-output rule and its enforcement points, the accepted
scheduled-retry divergence, the two resolution helpers, the stream()
binding rationale, the log-output-only gate, the CORS listing, the 500
fallback, and the test map all remain.
Sized against the merge, not just the branch: current main grew the same
chain by ~724 bytes, so the check was verified on the merged tree as well
(97,772 bytes; branch tree 97,048).
* fix(gateway): declare content-length on the fallback 500
The pre-response 500 declared content-type but no content-length, leaving
the framing to the ASGI server: chunked on HTTP/1.1, close-delimited on
HTTP/1.0 — the one wire difference from the ServerErrorMiddleware response
it replaces, which sends content-length: 21. The explicit header keeps the
fallback byte-identical to what clients saw before.
* docs(readme): drop the trace-correlation condition from the translations
The zh/ja/fr/ru Langfuse sections still said metadata.deerflow_trace_id
matches X-Trace-Id "when request trace correlation is enabled". The id now
always matches and that condition no longer exists, so each bullet states
the unconditional match and that logging.enhance.enabled only controls
whether the id is printed into logs — the one piece of the feature a user
can still configure.
* test(gateway): pin TraceMiddleware wiring through create_app()
Every X-Trace-Id test exercised a hand-built four-route app, so the real
stack's add_middleware(TraceMiddleware) line was pinned by nothing: deleting
it — or short-circuiting above it — passed CI while silently dropping both
the response header and the ambient id the run-record stamp and enhanced log
records derive from. One case now drives /health through create_app() and
asserts the inbound id round-trips; mutation-checked by removing the wiring
line, which fails exactly this test.
* docs(gateway): note the fallback 500 is CORS-opaque
The pre-response 500 is emitted outside CORSMiddleware — the exception has
already unwound past it — so it carries no Access-Control-Allow-Origin and
a split-origin browser client cannot read the id on this one response,
unchanged from the ServerErrorMiddleware 500 it replaces. Documented on the
class and in the CHANGELOG entry rather than fixed: replicating the origin
allowlist outside CORSMiddleware would let the two policies drift.
* fix(harness): keep abandoned-stream cleanup inside the trace binding
stream() binds the turn's id around each next(inner) and resets it before
yielding, but the finally's inner.close() ran after that binding was gone.
Abandoning the stream therefore drove the inner LangGraph generator's
GeneratorExit/finally path with no trace id — or an unrelated ambient one
from whichever context ran the close — so cancellation and finalization
logs and callbacks did not correlate with the turn they belong to.
inner.close() is now wrapped in a local bind/reset of the same turn id. The
token is set and reset in the same frame, never across a yield, so the
per-step cross-context safety is preserved even when GC closes the
generator from another Context — pinned by the existing copy_context close
test, which now exercises this path. The regression test records the id
from the inner generator's finally and fails without the binding.
* test(harness): teach the worker-trace fake about RunManager.cleanup
Upstream #5112 (bound gateway memory after terminal runs) added a
run_manager.cleanup(run_id) call to run_agent's finalization, so the
merge-commit CI run failed all five worker-trace-binding tests with
AttributeError on this PR's _FakeRunManager. The fake gains the same no-op
shape as its other methods.
* docs(gateway): bring the gateway AGENTS.md back under its soft budget
Upstream #5092 grew backend/app/gateway/AGENTS.md to 40,966 bytes, 6 over
the 40,960 soft budget that
test_agent_guidance_check.py::test_repository_guidance_stays_below_soft_budgets_and_avoids_doc_indexes
enforces — its Unit Tests run on main was cancelled by push concurrency, so
main is currently red on that test and every PR merge-run inherits the
failure. Two whitespace/wording trims in the row #5092 touched (a doubled
space, and "its configured `context_window`" → "its `context_window`")
bring the file to 40,953 with no content change.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(runs): reject cancel actions on GET stream joins
stream_existing_run is registered for both GET and POST, and its
?action=interrupt|rollback branch cancels the run. The CSRF middleware
exempts GET, so a session-authenticated browser could be forced
cross-site (img/script/top-level navigation) into
GET /api/threads/{id}/runs/{run_id}/stream?action=interrupt|rollback —
a state-changing GET that bypasses the CSRF protection guarding the
POST variant. Introduced with the dual registration in #1403.
The handler's docstring already documents cancel-then-stream as
POST-only (the LangGraph SDK's joinStream/useStream stop button uses
POST); enforce it: GET with an action answers 405, action-less GET
joins and POST cancel-then-stream are unchanged.
Regression drives the real router: GET+action is 405 with the run left
running, plain GET join still streams, POST+action still cancels.
* fix(runs): scope the 405 detail to the action requirement
"GET is a read-only stream join" overstates the current main: on a
locally-owned run with the default on_disconnect=cancel, a GET join's
disconnect can still trigger cancellation. That observer-disconnect
vector is closed by #5041; the detail here should only claim what this
guard enforces.
* fix(runs): harden GET stream action rejection
* fix(runs): align stream schema with method contract
* test(runs): pin GET stream action 405 through the production stack
Review follow-up (defence-in-depth): the GET-action suite drove bare
FastAPI() apps, so nothing pinned that a session-authenticated
cross-site GET reaches the route gate at all once CSRF exempts the
safe method. test_pat_auth.py already assembles the production
middleware order (AuthMiddleware inner, CSRFMiddleware outer), so its
mirror app now registers the real _reject_get_stream_action
dependency on a GET join route.
The new case pins the end-to-end premise: an authenticated GET
?action=interrupt is answered 405 + Allow: POST by the production
route dependency, while the same unauthenticated GET dies at
AuthMiddleware's 401 before any route logic runs.
Validation: focused suites (test_pat_auth, test_stream_get_action,
test_csrf_middleware) — 62 passed; ruff check + format clean; the new
case errors on the pre-fix baseline (guard absent), confirming the
pin.
* fix(test): exclude blocking I/O suite from make test
Keep make test-blocking-io as the dedicated suite owner.
Add regression coverage for the Makefile contract.
Refs #5088
* test: pin blocking I/O workflow ownership
Document both targets required for full offline validation.
Keep the dedicated workflow and Makefile target under contract coverage.
Refs #5088
* docs(test): align blocking-I/O test guidance
* feat(auth): add personal access tokens for programmatic API access (#4849)
Backend-first implementation of the PAT contract from #4849: show-once
dfp_ tokens bound to their owning user (AUTH_SOURCE_PAT,
is_internal=false), digest-only storage (migration 0017), strict
credential precedence (invalid Bearer is a 401, never cookie fallback),
CSRF double-submit skipped only for Bearer requests while
auth-endpoint origin checks still run, scopes intersecting the authz
route permissions, session-auth-only PAT management and password
changes, and throttled best-effort last_used_at stamps.
* fix(auth): harden PAT scope boundary and schema parity from adversarial review
Independent review of the initial draft found: (1) scopes only constrained
the threads/runs permission axis while admin routes treated a PAT as its
(possibly admin) owner — is_admin_user now rejects PAT callers outright
since no scope grants admin capability; (2) the model declared a column
UNIQUE constraint while migration 0017 created a named unique index, so
downgrade failed on create_all-bootstrapped DBs — both now use the named
unique index; (3) auth-disabled mode is an operator override and now stays
ahead of the Bearer check so a stray Authorization header cannot 401 an
E2E sandbox; plus wiring the previously-unused constants, bounding the
last_used_at stamp cache, and four new tests (middleware-level expiry,
expires_in_days, admin-capability rejection with session control, and the
auth-disabled precedence).
* docs(api): document personal access tokens for programmatic API access
* fix(auth): close PAT security boundaries from review (default-deny routes, extension admin suppression)
P1-1: scope intersection only constrains @require_permission routes, so
undecorated mutation routes (DELETE /api/memory, POST /api/agents, Lark
credential switching, channel config) accepted a PAT holding a single read
scope. AuthMiddleware now enforces a default-deny route policy in
auth/pat.py: PAT requests are admitted only to the thread/run lifecycle
routes the v1 scopes govern; everything else answers 403 regardless of
scopes. Session-cookie callers are unaffected.
P1-2: the extension principal resolver projected is_admin/roles from the
raw system_role, so an admin-owned PAT passed
deerflow_extension_api.require_admin on contributed routes despite the
documented no-admin guarantee. The projection is now PAT-aware and
suppresses every admin signal for PAT callers, mirroring
deps.is_admin_user.
Both fixes carry regression tests (route outside policy 403 + session
control; production resolver admin suppression), and API.md documents the
default-deny boundary.
* fix(auth): enforce PAT scopes on stateless run entry and harden decorator
Follow-up hardening from an independent audit of the P1 fixes:
- POST /api/runs/stream and /api/runs/wait were the only allowlisted run
entrypoints without @require_permission, so a threads:read-only PAT
could still start runs (same bug class as P1-1, now closed): both now
carry @require_permission("runs", "create"). POST /api/threads and
POST /api/threads/search gain threads:write / threads:read for the
same reason. Authorization-disabled deployments see no change (the
permission set resolves to all permissions).
- require_permission now binds the wrapped signature to locate a
positionally-passed request before injecting the test stub, fixing
'got multiple values for argument' on direct positional unit-test
calls.
- API.md: the intro PAT example used GET /api/models, which the new
default-deny policy 403s — replaced with GET /api/threads; the
default-deny route list now spells out method sets.
Regression test: threads:read-only PAT is 403 on the decorated stateless
entry while a runs:create PAT passes.
* fix(auth): address review P2s (empty Authorization header, PAT name trimming, API example)
- CSRFMiddleware treats an explicitly empty Authorization header as
present (is None), so an invalid credential always reaches
AuthMiddleware's uniform 401 instead of a CSRF 403 that varies by
method/CSRF state. Regression: empty-header request dies at auth.
- PATCreateRequest strips the name and rejects whitespace-only values
before token generation; created names are stored trimmed.
- API.md intro PAT example now uses the implemented
POST /api/threads/search endpoint (GET /api/threads does not exist).
- AGENTS.md trimmed back under the guidance soft budget after the
upstream merge.
* fix(auth): tighten PAT route policy to implemented methods only
The allowlist admitted GET /api/threads, a method no router implements.
Pre-authorizing a dead method weakens the default-deny boundary: a
future GET collection route added without a permission decorator would
become PAT-reachable without an explicit policy change. Restrict the
rule to POST, fix the stale GET description in API.md's PAT
constraints, and document the default-deny boundary accurately in the
gateway AGENTS.md guidance (only the threads/runs allowlist is
PAT-reachable; every other authenticated route 403s PAT callers).
Audited every remaining rule against the mounted routers: all other
method+path entries map to real routes. Regression:
test_pat_policy_does_not_pre_authorize_unimplemented_methods.
* test(auth): guarantee the negative digest test mutates the token
token[:-1] + "X" is identical to the original whenever the generated
token already ends in X (1/62), making the negative digest assertion
fail intermittently. Choose the replacement character based on the
existing tail so the mutated token always differs.
* fix(auth): require runs:cancel for cancel-then-stream requests
stream_existing_run is gated at runs:read so action-less stream joins
work with read-only credentials, but its ?action=interrupt|rollback
branch cancels the run — a separate permission. A runs:read-only PAT
passed both the PAT route policy and the route decorator and could
interrupt or roll back an active run, bypassing the runs:cancel scope.
Decorators cannot express query-parameter-conditional permissions, so
the check lives in require_cancel_permission_when_action(), applied at
the top of the handler. Regression drives the real helper through the
production middleware: runs:read-only PAT + action is 403, the same
token joins action-less, runs:read+cancel passes, session control
unaffected.
* docs(changelog): add the PAT feature entry
* docs(readme): add personal access tokens section
Repo documentation-update policy requires user-facing features to
update README.md in the same changeset; the PAT feature previously
touched only backend/docs/API.md and the gateway AGENTS.md.
* fix(auth): require runs:cancel for mutating multitask strategies
All five run-creation entrypoints were gated only by runs:create, but
RunCreateRequest.multitask_strategy accepts interrupt/rollback and
start_run forwards it to create_or_reject, which terminates an
already-active run. A runs:create-only PAT could therefore kill an
existing run through a create request, bypassing runs:cancel.
Decorators cannot express body-parameter-conditional permissions, and
per-route checks leave the same hole for the next entrypoint, so the
gate lives in start_run itself — the single choke point every
run-creation path (HTTP routes and internal launchers) flows through.
Regenerate launches pass multitask_strategy="reject" and are
unaffected; requests without a stamped auth context (internal/test
compositions) skip the gate.
The check is the shared authz.require_cancel_permission_if primitive;
require_cancel_permission_when_action now delegates to it, so every
request dimension that carries cancel capability (query action, body
strategy) flows through one gate.
Regression drives the real middleware stack: runs:create-only PAT +
interrupt/rollback is 403 with the exact detail, reject (explicit and
default) stays available, runs:create+cancel passes, session control
unaffected; a source anchor pins the gate inside start_run.
* fix(runs): keep observer joins from applying creator cancel-on-disconnect
sse_consumer's finally block applied the record's on_disconnect=cancel
policy on ANY consumer's disconnect. The join surfaces (GET /join and
the action-less GET/POST stream join) feed it the existing RunRecord,
so anyone with thread read access — including a runs:read-only PAT —
could cancel a locally-owned running run simply by closing the SSE
connection, without runs:cancel. The policy expresses the creator's
intent for their own connection; an observer's disconnect must never
be read as that intent.
sse_consumer gains apply_on_disconnect (default True). The two join
surfaces pass False; the creating endpoints (thread-scoped and
stateless create-and-stream) keep the creator semantics unchanged.
wait_for_run_completion needs no change: its callers are creator-side
or post-explicit-cancel paths only.
Regression exercises a real generator close — the same machinery
Starlette drives on client disconnect — against the production
sse_consumer: creator stream disconnect cancels, observer join
disconnect does not; a wiring anchor pins both join call sites and the
creator defaults. API.md documents the cancel-capability constraint
(this fix plus the action/strategy gates) in PAT Constraints.
* test(auth): pin the multitask gate behaviorally; state wait invariant
Independent adversarial review of the round-5 fixes found the P1-a
regression only mirror-pinned: the source anchor could be satisfied by
a comment, and deleting the gate from start_run would not fail the
suite. This drives the production start_run directly — a create-only
auth context gets 403 with the exact detail for interrupt, and a
reject request with no cancel permission at all proceeds past the gate
(never a permission 403).
Also documents wait_for_run_completion's creator-side invariant
(every caller is the creating endpoint or post-explicit-cancel) so a
future observer wiring thinks twice before reusing it — the one-caller-
away variant of the observer-disconnect P1.
* docs(changelog): correct the PAT entry's digest and route-policy description
The entry said HMAC digests (the implementation stores SHA-256 digests,
as documented in API.md and pinned by the repository tests) and claimed
the route policy admits 'implemented stateless endpoints' (it admits
the thread/run lifecycle routes, narrowing further by scopes). Also
notes the cancel-capability gate now covering action and multitask
strategies.
* fix(auth): enumerate the PAT runs route policy per implemented subroute
The runs subtree rule was a GET|POST /runs(/.*)? wildcard — it
pre-authorized every current and future subroute under /runs, including
methods the router never implemented (e.g. GET /runs/stream), which is
the same latent default-deny weakening the threads collection rule was
tightened for: a future route added under /runs would become
PAT-reachable without an explicit policy change.
The wildcard is replaced with six segment-precise rules covering exactly
the 14 implemented method+path combinations; the {run_id} slot
necessarily matches any single segment, so the POST-only collection
names (stream, wait, regenerate, edit-regenerate) are excluded from the
GET run-id rule via negative lookahead — no dead method stays
pre-authorized. Behavior for implemented routes is unchanged.
test_pat_runs_policy_admits_exactly_the_mounted_routes derives the
expected set from the mounted thread_runs router instead of a
hand-maintained list: every implemented GET/POST route under /runs must
be admitted, routes in this router outside the subtree stay denied, and
representative unimplemented neighbors are denied — so adding a route
under /runs now fails CI until it is explicitly allowlisted, and a
removed route leaves a dead rule visible. API.md's PAT constraints list
the enumerated routes and drops a feedback mention that belonged to the
stateless /api/runs axis.
* docs(migration): add the 0017 renumbering coordination note to 0017
The PR's migration-coordination comment states each migration file
carries the note; the file did not. Adds it: numbering was generated
against main head 0016 alongside #5078 and #4843; whoever merges first
keeps the slot, the others renumber on rebase (revision/down_revision
plus the bootstrap head assertions).
* fix(auth): pad base62 tokens to a fixed 43-char width
int.from_bytes discards leading zero bytes, so the unpadded encoder
returned a variable-length body — empty for all-zero input, and shorter
than 40 characters for any draw below 62**39 (~1 in 14.5M), leaving
test_generate_pat_token_format probabilistically flaky and the token
body without stable width (review round 6, P3).
_base62 now left-pads with "0" to _base62_width(len(data)) — the exact
integer digit count (62^43 > 2^256 > 62^42, so 43 for 32 bytes). The
format test asserts the exact fixed width instead of a probabilistic
floor, and a new unit test pins the all-zero, leading-zero-byte, and
max-value edges deterministically.
Recipes and scripts ran sibling shell scripts bare (./scripts/x.sh), so
any checkout that lost the executable bit -- zip/tarball download,
core.fileMode=false, non-POSIX filesystem -- failed with:
make: ./scripts/docker.sh: Permission denied
make: *** [Makefile:181: docker-start] Error 127
The tracked modes are already 100755, so chmod cannot fix it. Name the
interpreter instead: the POSIX branch of RUN_SHELL_SCRIPT (renamed from
RUN_WITH_GIT_BASH) now expands to $(BASH) rather than nothing, and the
five script-to-script call sites are prefixed with bash.
Fixes#2903
Co-authored-by: zaoshangduziteng <309590849+zaoshangduziteng@users.noreply.github.com>
* feat(scripts): support skipping frontend build on make start
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(scripts): validate skip-frontend-build before stop_all and format test
---------
Co-authored-by: PoetryLin <PoetryLin@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(skills): accept portable frontmatter forms
* fix(skills): normalize portable tool names
* Safely preserve parenthesized portable skill tool patterns
Portable Agent Skills declarations such as Bash(tvly *) contain spaces inside a command pattern. Keep those patterns as single literal entries while preserving exact names from the existing YAML-list form, so skill loading no longer fragments valid metadata or rewrites mixed-case MCP tools.
Constraint: DeerFlow's current skill policy matches exact tool names and does not inspect Bash arguments
Constraint: Agent Skills scalar syntax uses whitespace-separated entries with parenthesized command patterns
Rejected: raw.split() | fragments Bash(tvly *) into unrelated tool names
Rejected: normalize YAML-list entries | breaks case-sensitive MCP/runtime tool names
Rejected: map Bash(...) to bash | broadens command-scoped declarations into unrestricted shell access
Confidence: high
Scope-risk: narrow
Reversibility: clean
Directive: Keep Bash(...) entries literal and inactive until DeerFlow has an explicit command-pattern authorization model
Tested: 175 focused parser, validation, installer, review, loader, and tool-policy tests; Ruff check and format; compileall; git diff --check
Not-tested: Full backend suite stopped at pre-existing Windows mode assertion test_runtime_config_store_file_is_owner_only
Related: #4912
* Preserve exact custom tool names in portable skill parsing
Portable scalar frontmatter needs alias normalization for known DeerFlow-compatible names, but generic case conversion corrupts MCP and custom tool identifiers. The tokenizer also treated quoted or escaped parentheses as structural delimiters, rejecting valid command patterns. Preserve unknown names and parse quoted or escaped patterns without broadening Bash(...) into bash.
Constraint: Runtime skill policy uses exact tool-name matching
Constraint: Parenthesized patterns remain literal because argument-level authorization is not implemented
Rejected: Generic CamelCase-to-snake_case for every scalar | rewrites custom/MCP names
Rejected: Map Bash(...) to bash | broadens command-scoped declarations into unrestricted shell access
Confidence: high
Scope-risk: narrow
Reversibility: clean
Directive: Add an explicit alias before supporting another portable tool name; keep command-pattern authorization separate
Tested: 225 skills tests passed, 1 skipped; Ruff check; Ruff format --check; compileall; git diff --check
Not-tested: Full backend suite remains affected by unrelated Windows permissions/path and missing Lark CLI tests
Related: #4984; #4912
* Preserve case-sensitive exact tool authorities
Case-folding a scalar declaration before alias lookup can turn literal write into write_file, substituting a different runtime authority. Keep exact portable spellings as aliases and preserve lowercase, custom, and MCP names; strengthen activation coverage for spaced Bash patterns and command fragments.
Constraint: Runtime skill policy uses exact tool-name matching
Constraint: Bash(...) remains literal and inactive because command-pattern authorization is not implemented
Rejected: Case-insensitive alias lookup | maps lowercase runtime tools onto built-in authorities
Rejected: Broaden the parser into command-pattern authorization | outside this PR's scope
Confidence: high
Scope-risk: narrow
Reversibility: clean
Directive: Add aliases only for documented portable spellings; preserve all other scalar names verbatim
Tested: 226 skills tests passed, 1 skipped; Ruff check; Ruff format --check; compileall; git diff --check
Not-tested: Full backend suite remains affected by unrelated Windows permissions/path and missing Lark CLI tests; GitNexus index refresh remains stale
Related: #4984; #5016297602
* Support portable Glob and Grep skill aliases
Portable Agent Skills commonly declare Glob and Grep, but DeerFlow exposes the runtime tools as glob and grep. Add explicit exact-spelling aliases and activation coverage so imported skills retain search-tool access without broad normalization.
Constraint: Runtime skill policy uses exact tool-name matching
Constraint: Alias conversion is limited to documented portable spellings
Rejected: Case-fold all scalar names | can substitute custom or MCP authorities
Rejected: Map arbitrary names by convention | breaks exact runtime compatibility
Confidence: high
Scope-risk: narrow
Reversibility: clean
Directive: Keep the alias table explicit and preserve unknown scalar names verbatim
Tested: 228 skills tests passed, 1 skipped; Ruff check; Ruff format --check; compileall; git diff --check
Not-tested: Full backend suite has unrelated environment failures on Windows; GitNexus index reports stale line mappings
Related: #4984; #5026257899
---------
Co-authored-by: kriptoburak <kriptoburak@users.noreply.github.com>
* Preserve Windows CLI compatibility for local sandbox commands
MSYS path conversion must remain disabled for DeerFlow virtual paths, but applying a blanket environment override to every POSIX command breaks host-native CLI shims on Windows. Limit MSYS argument-conversion exclusions to safe non-root virtual path prefixes, omit values that would broaden the exclusion pattern, and document the contract.
Constraint: Preserve the virtual-path protection introduced by #2765/#2766
Rejected: Disable MSYS conversion for every command | breaks Windows CLI shims
Rejected: Toggle blanket conversion only for commands containing virtual paths | host CLIs can receive virtual-path arguments and still need normal conversion for their own paths
Confidence: high
Scope-risk: moderate
Reversibility: clean
Directive: Keep regression coverage for virtual-path arguments, root mounts, and host-native CLI launchers
Tested: test_local_sandbox_encoding.py (12 passed); related sandbox suite (197 passed, 8 skipped, 7 failures matching origin/main); ruff check; ruff format --check; git diff --check; direct LocalSandbox CLI and virtual-path smoke tests
Not-tested: Full offline suite completion; stopped at 6% after unrelated Windows and optional-runtime failures
Related: #2765
Related: #2766
* Keep MSYS regression tests portable across CI operating systems
The Windows-shell environment tests patched os.name to nt while mounting Windows-specific paths. On Linux and macOS, pathlib then attempted to construct WindowsPath during command resolution or output masking, so the backend merge gate failed before exercising the environment contract. Stub the exclusion boundary in execute-command tests and retain mapping-specific filtering coverage in the helper test.
Constraint: Backend unit tests run on Linux, while the behavior under test is Windows-only
Rejected: Skip the tests outside Windows | would remove CI coverage of the environment contract
Rejected: Patch pathlib internals | couples tests to implementation details and hides the platform boundary
Confidence: high
Scope-risk: narrow
Reversibility: clean
Directive: Keep OS-specific subprocess assertions independent from host-path resolution
Tested: test_local_sandbox_encoding.py (12 passed); ruff check; ruff format --check; git diff --check
Not-tested: Linux runner execution locally because Docker Desktop is unavailable and WSL cannot access this linked worktree
Related: #5003
Related: https://github.com/bytedance/deer-flow/pullrequestreview-5013380238
* fix(docker): create compose env files and keep Windows compose paths relative
Windows Docker reports a generic file-not-found when env_file targets are missing, or when compose paths are doubled. Make docker-start copy .env examples and invoke compose with filenames relative to docker/.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(docker): make dev compose env files optional and repair test fixture
Address review feedback on #4956.
[P1] prepare_compose_env aborted before the mocked COMPOSE_CMD in
test_compose_commands_set_deer_flow_root_before_compose, because the
temp root had no compose file or .env examples. Seed them in the
fixture so the preflight reaches the mock.
[P2] .env is gitignored, so a fresh clone has none and a direct
`docker compose -f docker/docker-compose-dev.yaml up --build` aborts on
Windows before scripts/docker.sh can help. Mark the dev env_file entries
`required: false` so a missing .env is not fatal, and document that
direct Compose must be run from the repository root.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(docker): declare Compose 2.24 floor and keep non-start commands read-only
Address the second review round on #4956.
- Document Compose >= 2.24 (CONTRIBUTING, README, compose header) and fail
early from make docker-start with an actionable message; probe both
`docker compose` and the hyphenated `docker-compose` binary.
- Document DEER_FLOW_ROOT for direct Compose callers (bash + PowerShell);
leave the variable without a $PWD fallback because PowerShell/cmd do not
export it.
- Split prepare_compose_env: compose_preflight is shared and read-only;
ensure_env_files runs only from start.
- Expand tests for version boundaries, hyphenated fallback, env-file
creation, and read-only stop/logs/restart behavior.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(docker): reuse the probed Compose binary for wrapper operations
The version probe could accept a standalone docker-compose install while
COMPOSE_CMD stayed hardcoded to `docker compose`, so preflight passed and
start/logs/stop/restart then failed. Keep the selected executable in
COMPOSE_BIN (array), refresh COMPOSE_CMD from it in the current shell, and
extend the fallback test through an actual stop invocation.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix skill moderation parsing for Responses API content blocks
Normalize LangChain Responses API text blocks before parsing the security moderation decision, while preserving the existing fail-closed behavior for unavailable or invalid moderation results. Add regression coverage for mixed content blocks and document the compatibility boundary.
Constraint: Responses API AIMessage content is list-shaped while Chat Completions content is string-shaped
Rejected: Disable security scanning | would weaken the skill write safety boundary
Confidence: high
Scope-risk: narrow
Reversibility: clean
Directive: Keep moderation parsing provider-format tolerant without including reasoning or tool blocks in the decision payload
Tested: 27 security scanner tests; ruff check; ruff format check
Not-tested: Live moderation request against the configured external endpoint
* Reuse shared LLM response text normalization
Route skill moderation responses through the existing provider-format normalizer so only text and output_text blocks participate in JSON parsing. Strengthen regression coverage with reasoning and tool blocks that contain misleading text fields.\n\nConstraint: Responses API content is shared across multiple harness consumers\nRejected: Keep a private normalizer | duplicated provider-shape policy diverges and can reintroduce reasoning-block contamination\nConfidence: high\nScope-risk: narrow\nReversibility: clean\nDirective: Extend the shared normalizer when a new provider content shape is verified; do not add divergent local parsers\nTested: 118 related backend tests; regression test red against the previous parser; Ruff check and format check\nNot-tested: Live GitHub CLA status refresh
* Restore trusted external skill package loading
Skill discovery follows one-level package-directory symlinks, but activation path validation rejected the resolved external path. Restore that compatibility for configured custom-skill category roots while keeping file-level symlinks and deeper escapes blocked. Add regression coverage for local and user-scoped storage plus slash activation, and document the boundary.
Constraint: Existing skill discovery follows directory symlinks and operator-managed external packages must remain loadable
Rejected: Allow arbitrary resolved paths | would weaken the skill path trust boundary
Confidence: high
Scope-risk: moderate
Directive: Keep the final SKILL.md file symlink-free and preserve one-level category-root validation
Tested: 79 targeted skill storage, loader, slash activation, and user-scoped tests passed; Ruff check and format check passed; GitNexus staged change detection reported low risk
Not-tested: Real symlink activation on this Windows host lacks SeCreateSymbolicLinkPrivilege and is skipped
Related: Skill projection copies sources into sandbox-visible views
* Exercise real filesystem symlink boundaries in skill storage tests
Replace global Path.resolve/is_symlink mocks with real directory and file symlinks, preserving the Windows privilege skip. Add regression coverage for deeper custom-root escapes and symlinks under non-custom categories so the one-level allowance remains explicit.
Constraint: Symlink creation requires SeCreateSymbolicLinkPrivilege on some Windows runners
Rejected: Keep global path-method mocks | they validate the mock behavior rather than filesystem semantics
Confidence: high
Scope-risk: narrow
Reversibility: clean
Directive: Keep security-boundary tests on real filesystem primitives; skip only when the runner lacks symlink privilege
Tested: 76 targeted loader/storage/slash tests; Ruff check; Ruff format check
Not-tested: Windows symlink-enabled execution on this host
Related: #4936
* Pin the actual nested symlink escape boundary
Place the second symlink below a real custom package directory so the test reaches the one-level relative-parent guard instead of returning early on a non-symlink parent. Keep the public-category rejection coverage unchanged.
Constraint: The security boundary depends on both symlink depth and category root
Rejected: Link the outer package directory directly | the parent is not a symlink at validation time, so the depth guard is never evaluated
Confidence: high
Scope-risk: narrow
Reversibility: clean
Directive: Keep this regression tied to the exact relative_parent.parts depth check
Tested: Targeted storage, loader, and slash suites; GitNexus staged detection
Not-tested: Symlink-enabled execution on this Windows host
Related: #4936
* Make the nested symlink regression reach the depth guard
The test now validates the SKILL.md directly through the nested symlink, so the symlink is the immediate parent and the relative-parent depth check is executed.
Constraint: Windows test execution may skip when symlink privilege is unavailable
Rejected: Keep the extra nested path segment | it bypasses the symlink-depth guard through an early return
Confidence: high
Scope-risk: narrow
Reversibility: clean
Directive: Mutation tests must fail when the depth restriction is removed
Tested: Targeted test (skipped on this Windows host without symlink privilege); Ruff check and format check
Not-tested: Real symlink execution on Windows; Linux CI will exercise the case
Related: #4936
* Keep sandbox projections fresh for linked external skill packages
The storage layer intentionally accepts one-level custom package-directory symlinks, but projection freshness previously hashed only the link inode. Follow the permitted target tree during custom and legacy source-signature scans so edits to SKILL.md, scripts, references, or assets trigger a rebuild before sandbox use.
Constraint: Preserve the existing one-level custom/legacy symlink boundary and do not follow public, integration, nested, or file symlinks
Rejected: Invalidate projections only from /api/skills/reload | sandbox acquisition must also detect edits made directly in external targets
Confidence: high
Scope-risk: narrow
Reversibility: clean
Directive: Keep target-tree traversal limited to the storage paths that explicitly permit external package-directory links
Tested: 77 projection, user-scoped storage, and lifecycle tests passed; Ruff check and format check passed; git diff --check passed; GitNexus staged detection reported low risk
Not-tested: Real external symlink execution on this Windows host without SeCreateSymbolicLinkPrivilege; existing tests skip that platform limitation
Sandbox is an execution environment, not a named resource: multiple tools
(bash, read_file, write_file, glob, grep, ...) depend on it, all funneled
through ensure_sandbox_initialized / ensure_sandbox_initialized_async. Gate
the single acquisition entry point (single source of truth) instead of
maintaining a sandbox-tool-name set in middleware:
- authorize_sandbox_execution helper (authz/sandbox_authz.py) checks
authorize("sandbox", "execute", target="*") — a binary judgment
(can this role use the sandbox at all); RBAC allow:"*"/true permits,
allow:[]/false denies.
- lazy path: ensure_sandbox_initialized (+ async) calls the gate before
provider.acquire.
- eager path: SandboxMiddleware.before_agent / abefore_agent call the gate
before _acquire_sandbox.
- deny raises SandboxAuthorizationError (SandboxError subclass) which
propagates through tool execution as a friendly ToolMessage (RFC §9:
'not a crash').
- authorization.enabled: false is a no-op everywhere; provider errors
follow fail_closed (deny) / fail_open (allow).
12 tests in tests/test_sandbox_authorization.py cover disabled/allow/deny/
deny-via-bool/no-policy-unrestricted/provider-error-fail-closed/open/
internal-caller + ensure_sandbox_initialized deny (never acquires) and
allow (acquires) integration paths.
Adds scheduler.recursion_limit to config.yaml (default 1000, clamped by
max_recursion_limit) so scheduled background runs can use a different
recursion limit than the web UI. The value is read at dispatch time, so
a YAML edit applies to the next scheduled run without a Gateway restart.
Also logs a warning when the resolver falls back to the default or
clamps the configured value.
* feat(harness): add deterministic tool receipts with model-visible ledger
Stamp an immutable per-call fact record (tool name, status, args/output
hashes, byte count, timestamp) onto every tool result via a new
ToolReceiptMiddleware, and inject the derived receipt ledger (r1..rN)
into the model context so subagent reports can cite executed actions.
- tool_receipt.py: receipt core (make/extract/render), newest-first
budget eviction, ids derived from the append-only message stream
- ToolReceiptMiddleware: stamps ToolMessages directly or inside
Command-wrapped results; hidden ledger injection mirrors
DurableContextMiddleware; sits between ToolProgress and
ToolErrorHandling with a build-time ordering guard
- config: new verification section (receipts on, judge off), config
version 32 -> 33 with example/helm/docs updates
* feat(harness): split receipt rendering from stamping; address PR review
Review fixes (PR #4659):
- output_sha256 now uses sort_keys=True for structured content, matching
the order-invariant args fingerprint
- stamping failures log at warning (silent ledger gaps would corrupt
citations); tool execution remains never blocked
- _insert_after_leading_system_messages extracted to shared public
message_utils.insert_after_leading_system_messages; both middlewares
depend on it instead of a private cross-module helper
- code comments in English
RFC #4651 revision-2 alignment:
- receipts_render_mode config ('always' | 'delegation_only'): subagent
chains always render the ledger (citations are produced there); the
lead chain renders only while processing subagent results, removing
the always-on token tax from ordinary turns
- receipts gain bounded args_preview/output_preview (<=200 chars, tail
for output) so later typed claim bindings (tests_passed) can anchor
to a specific recorded execution
* docs(harness): state receipt freshness caveat and vocabulary layering in module docstring
* merge: upstream/main — resolve AGENTS.md split, bump config_version to 34, drop unused receipt previews
- backend/AGENTS.md: take upstream's slimmed root guidance (#4799); move the
ToolReceiptMiddleware chain entry into agents/middlewares/AGENTS.md and the
verification.* hot-reload mention into config/AGENTS.md
- config.example.yaml + helm values/README: config_version 33 -> 34 so existing
v33 configs get the outdated-config prompt (review: willem-bd)
- tool_receipt.py: drop args_preview/output_preview — no Layer 1 consumer reads
them; re-add with the Layer 2 claim-binding consumer (review: willem-bd)
* docs(harness): cover receipt id renumbering after compaction in module docstring
Positional display ids are stable only while history is append-only;
compaction drops ToolMessages and the survivors renumber, so Layer 2
citation verification must resolve [rN] against the ledger as of the
citing turn (review: willem-bd, doc-only).
* chore(config): bump config_version to 35
main reached 34 via #4780 without the verification section; publishing
the new schema at the same number would silently skip the outdated-config
prompt for configs synced from main in that window (review: willem-bd).
* fix(skills): restore errno import dropped upstream in #4830
upstream/main adf6c422 uses errno.ENOTDIR in the drift guard but removed
the import, so the PR merge ref fails lint-backend (F821).
* fix(harness): harden tool receipts against forgery and turn-scope delegation_only
Address willem-bd's pre-merge review on #4659:
1. Untrusted receipt metadata: the gateway now strips the server-owned
deerflow_tool_receipt key from external input messages; stamping always
overwrites any tool-supplied value instead of preserving it; and
extract_tool_receipts validates persisted receipt shapes (required typed
fields, unknown keys ignored) so malformed entries are skipped instead of
crashing render or passing as runtime-stamped evidence.
2. delegation_only no longer sticks on: _should_render now scopes the
subagent_status scan to the current turn (messages after the latest
genuine user message), so an old completed delegation stops rendering the
ledger on later ordinary turns. The genuine-user predicate moves to
message_utils.is_genuine_user_message, shared with input sanitization.
* fix(harness): stamp receipts outside short-circuiting tool middlewares
Address willem-bd's review on #4659: ToolReceiptMiddleware was registered
inside Guardrail/SandboxAudit/ReadBeforeWrite/ToolProgress, each of which
can return a ToolMessage without invoking its handler — blocked calls
(e.g. a read-before-write-denied write_file) never got a receipt, silently
gapping the ledger on a default-enabled path. SandboxAudit additionally
rebuilds medium-risk results, dropping an inner stamp.
ToolReceiptMiddleware is now the outermost wrap_tool_call layer in the
runtime tail. Normal results still carry deerflow_tool_meta (stamped by
ToolErrorHandling on the inner return path); short-circuit messages
self-stamp meta or fall back to message.status. The new invariant is
declared as ordering constraints in deerflow.extensions.ordering, with
composed-chain regression tests for a blocked write and a warn-rebuilt
bash result.
* feat(extensions): let an out-of-tree extension observe what the agent did
DeerFlow's extension system can contribute middleware, services and routes,
but an extension cannot answer basic questions about a run without reaching
into host internals. Several of the facts it would need are destroyed by the
operations that produce them:
* The middleware chain injects and rewrites a lot of context — date
reminders, recalled memory, compaction summaries, durable-context data,
image payloads, activated skill bodies. Downstream, none of it is
attributable: at the model-call boundary an injected HumanMessage is
indistinguishable from the user's own, and anything wanting to tell them
apart has to pattern-match prompt wording, which breaks on the next copy
edit.
* Two runs of "the same agent" are only comparable if the chain enforced the
same limits, prompts and thresholds. Recovering that from outside means
reading private attributes and guessing which of them change behaviour — a
guess that rots silently as middlewares gain fields.
* The lead-agent factory resolves a model after runtime overrides, renders a
prompt, filters tools through authorization and composes a stack, all
inside one synchronous call, and none of it survives: a middleware sees its
neighbours but not the prompt, the run worker sees a graph but not what
went into it.
* Summarization is destructive by design. N messages leave the context and
one summary enters it; afterwards only the summary exists, so "which
messages became this?" is not reconstructible.
This adds seven neutral facilities so those facts are recorded where they are
still true, and releases the contract package as 0.2.0.
Message provenance
Producers stamp `deerflow_content_kind` / `deerflow_producer_kind` onto the
messages they inject or rewrite. Stamping is unconditional — a fact whose
presence depends on whether an observer is installed is not a fact — and the
keys are server-owned, so provenance cannot be forged from a request.
Middleware self-description
Twelve middlewares declare their own behaviour-affecting parameters through
a duck-typed `release_policy_parameters()`. Long text is hashed rather than
embedded: a declaration is an identity, not a copy of the prompt.
Agent assembly descriptor
`assemble_lead_agent()` returns the graph plus a descriptor whose fingerprint
answers "did anything about this agent change between these two runs?".
`make_lead_agent()` keeps its graph-only signature — it is the LangGraph
Server ABI declared in langgraph.json. Tools and skills are sorted before
hashing because their assembly order is incidental; middlewares are not,
because stack order decides what wraps what. Host build identity is reported
but excluded from the fingerprint, so a redeploy does not invalidate every
agent's identity.
Context compaction observation
Summarization emits the content hashes of the messages it is about to remove
joined to the summary that replaced them. Content is the only identity
available at that seam: the summary does not become a message, and what later
projects it into a request renders it bounded and escaped rather than
verbatim.
Neutral policy, transform and MCP-source facts
Guardrail decisions are published to runtime context under a `__`-prefixed
key; result-rewriting middlewares append a declared, ordered transform trail;
MCP tools carry their credential-free logical origin.
Extension route identity
Contributed routes are session-authenticated and cannot opt out, but
"logged in" and "administrator" are different questions. Extensions get a
neutral projection of the caller rather than the host's auth context, and
`require_admin` fails closed when identity cannot be determined.
Extension-owned tables
An extension that persists data owns its own MetaData and migration chain, so
its tables are absent from Base.metadata and `alembic revision --autogenerate`
proposes dropping them. Extensions declare a table prefix, which is rejected
at registration if it would shadow a host table.
The contract package stays dependency-free and imports no host code; every new
Protocol method has a default so later additions remain additive. The loader's
pre-1.0 rule requires an exact major.minor match, so extensions written against
0.1 are now refused at startup with an actionable install hint rather than
loading into a host that implements a different surface.
uv.lock records the contract package's new version, so `uv sync --locked` still
resolves on a fresh checkout.
* fix(backend): sort gateway service imports
* fix(mcp): exclude internal temp files from workspace changes
* fix(mcp): address review — shared tmp subdir constant, any-depth docs, nested test
- Export MCP_TMP_SUBDIR from constants.py and import it in both stdio
launch paths (tools.py, task_tool_caller.py) so the "/tmp" suffix is
composed once.
- Document that the .mcp exclusion matches by directory name at any
depth (like .git/node_modules) in README.md and mcp/AGENTS.md —
subagent work dirs below the workspace root get their own .mcp/tmp.
- Pin the any-depth semantic in test_workspace_changes.py with a nested
workspace/project/.mcp assertion.
* docs(mcp): correct any-depth exclusion rationale; re-home tmp pinning comment
The previous commit justified the any-depth `.mcp` exclusion with subagent
work dirs sitting below the workspace root — a mechanism that doesn't
exist: subagents share the parent's thread_id and both stdio launch paths
resolve sandbox_work_dir(thread_id), so `.mcp/tmp` is always pinned at the
workspace root. Reword mcp/AGENTS.md and the test comment to the real
justification (consistency with the other reserved dir names, robustness
against a server creating a relative `.mcp` from another cwd).
Also move the orphaned "pinning the process temp dir" rationale from
tools.py to constants.py next to MCP_TMP_SUBDIR, where both importers see it.
The Star History charts in the README files no longer render because the underlying chart service relies on GitHub stargazer data that is currently restricted. This switches the charts to a working alternative that needs no API token, updating the English, Simplified Chinese, Japanese, French, and Russian READMEs at the same time.
Co-authored-by: OctoBored <212877535+OctoBored@users.noreply.github.com>
* fix(sandbox): bound aggregate E2B mount upload work
* fix(sandbox): preserve mount guards on upload failure
* fix(sandbox): cover mount preflight with deadline
* refactor(sandbox): clarify mount deadline checks
* refactor(sanbox): deduplicate mount deadline reason
* fix(sandbox): evaluate mount deadline reason lazily
* feat: integrate MiniMax Code as an ACP agent
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
