* feat(memory): add cognitiveStyle for collaboration preferences
Extend cross-session memory with user.cognitiveStyle and cognitive facts,
backward-compatible normalization, prompt injection as Thinking Style, and
settings UI. Add design notes and unit tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(memory): normalize legacy memory imports on frontend
Add normalizeMemoryPayload aligned with backend normalize_memory_data,
use it for Settings import and API reads, and add unit/regression tests.
Document schema-evolution steps and sync FACT_EXTRACTION_PROMPT cognitive category.
* fix(memory): harden legacy memory normalization
* fix(memory): keep import envelope validation strict
* fix(memory): keep unknown fields through frontend import normalization
normalizeMemoryPayload rebuilt the memory document from a section whitelist,
so every key it did not name was dropped. Because readMemoryResponse routes
all memory API responses through it, the loss showed up on read, export, and
import alike: revision vanished, and any future backend field would silently
break lossless export/import.
The Gateway models are extra="allow" and deermem normalizes additively, which
left the frontend as the only lossy hop in the chain. Spread before overriding
at each nesting level so normalization only validates and defaults
contract-owned fields, leaving the envelope check as strict as before.
Covered by a unit test on the strict import path, two backend tests for
extension preservation and caller immutability, and an E2E round-trip that
asserts on the captured import request body.
* docs(memory): state the normalization invariant that actually holds
The docstring cited `display` as a backend-driven field alongside
`revision`, but #4546 was closed unmerged and no such field exists in the
contract. It also promised lossless export/import, which the Gateway's
`extra="ignore"` models preclude for any undeclared field.
Replace both with the invariant the code is really defending: the frontend
must not be narrower than the Gateway contract, using the declared
top-level `revision` as the concrete example.
* fix(memory): preserve legacy migration and reject lossy imports
* fix(memory): align legacy fact normalization defaults
* test(memory): use authenticated app for import regression
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(gateway): keep custom-skill rollback filesystem IO off the event loop
Closes#5747. Three blocking sites remained on the Gateway event loop in
the rollback route and its shared response reader, all offloaded through
asyncio.to_thread following the #5729 pattern:
- storage.validate_skill_markdown_content() creates a temp dir and writes
SKILL.md into it (site 1)
- the post-scan read of the file being replaced (site 2, named by #5729's
test docstring)
- _read_custom_skill_response(): load_skills() walks every skill directory
and read_custom_skill() opens SKILL.md — O(installed skills) loop time
paid per accepted rollback and per GET /skills/custom/{name} (site 3)
A blocking_io regression drives the full accepted-rollback path (past the
security scan) under the strict gate; the two pre-existing cases only
covered the pre-scan branches.
* fix(gateway): keep the skill element type in the response-parts closure
Review nit on #5751: tuple[object, ...] erased the element type for the
_skill_to_response call; Skill is already imported in this module.
* fix(authz): cover create/run/memory/agent routes with permission checks; scope USER.md per user
Route permissions: the AuthorizationProvider model only applied to routes
carrying @require_permission, so POST /api/threads, /api/threads/search,
POST /api/runs/stream and /api/runs/wait (runs:create), every /api/memory
route and the custom-agent routes ran with authentication only — a
provider configured to deny threads:write/runs:create/... could not
enforce those decisions. Add the missing decorators (threads:write,
threads:read, runs:create, memory:read/write, agents:read/write) and
register the new permission names in authz.Permissions. No owner checks
are added: per-user data scoping stays in the repository layer.
User profile: GET/PUT /api/user-profile read and wrote a single global
{base_dir}/USER.md, so in a multi-user deployment any authenticated user
could overwrite the prompt context injected for everyone else (and read
it). Scope the file to the caller's bucket
({base_dir}/users/{user_id}/USER.md) like user-scoped skills; no other
consumer of the old global path exists in the tree (verified by grep and
the updated tests). test_put_user_profile asserted the wrong effective
user under the autouse conftest user fixture; fixed to test-user-autouse.
* fix(authz): bind positional args in require_permission; migrate legacy USER.md
Review follow-up on #4989 (willem-bd):
- require_permission's wrapper only looked for request in keyword
arguments, so direct positional calls like
create_thread(body, request) made the injected keyword stub collide
with the positional request (TypeError: multiple values). The wrapper
now binds the wrapped signature via inspect.signature().bind() and
honors a positionally-passed request (and thread_id) instead of
assuming kwargs; the test-stub injection only fires when request is
absent everywhere. The two positional callers in
tests/test_threads_router.py pass again (8/8 channel tests).
- migrate_user_isolation.py now claims the legacy global USER.md for
--user-id (default 'default') like the other unowned legacy
artifacts; without it, upgrading installs with an existing profile
would read content: null and later strand the old file beside the
new per-user one. Conflict handling mirrors migrate_memory (rename to
USER.legacy.md).
* style(scripts): keep migrate_user_isolation help within the line budget
* test(authz): give internal-request stubs realistic auth fields
The create_thread permission wrapper added in this PR authenticates
the direct calls in the internal-owner tests; their SimpleNamespace
requests had state.user but no auth_source and no cookies, so
get_current_user_from_request fell through to request.cookies.get and
raised AttributeError (verified introduced by this branch: both tests
pass on the base commit).
The stubs now carry cookies={} and
state.auth_source=AUTH_SOURCE_INTERNAL, which is exactly what
AuthMiddleware stamps on real internal requests, so state.user is
honored without the JWT path. All 80 tests in the file pass.
* fix(pat): keep memory/agent permissions out of PAT scopes by design
The route permissions exist (they guard the memory/agent routers) but PATs
govern the thread/run lifecycle only: _PAT_ROUTE_RULES default-denies those
routers for PAT callers regardless of scopes. The alignment invariant
becomes a subset check plus a pinned exclusion, and pat.py documents that
opening these scopes is a product decision requiring three synchronized
changes.
* test/docs: cover migrate_user_profile; drop the unclaimed USER.md injection wording
Five-scenario test class mirroring the sibling migration steps (move,
conflict-rename, noop, both under dry-run). The GET/PUT descriptions no
longer say USER.md is 'injected into agents' — nothing at this head consumes
it for prompts; the routes are storage/retrieval only. The memory AGENTS.md
migration pointer now enumerates skills/ and the global USER.md.
* docs(paths): align the USER.md layout comment with the injection disclaimer
* test(authz): align effective permission contracts
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
The Gateway's staged upload commit links the `.part` file to its final name and
then unlinks the staged name. The conversion path duplicates a descriptor on the
staged inode *before* that commit (deliberately: it is what keeps conversion
reading the bytes this request wrote rather than whatever the committed name
points at afterwards), so the staged file still has an open handle when the
commit tries to remove it.
POSIX allows that; Windows does not. Uploading a convertible document
(`uploads.auto_convert_documents: true`, a PDF/DOCX/PPTX/XLSX upload) on a
Windows host therefore failed with a 500:
Failed to upload report.pdf: [WinError 32] The process cannot access the
file because it is being used by another process
Split publishing from removing the staged name: `_commit_upload_temp_no_overwrite`
now takes `unlink_staged`, and the ingestion service keeps ownership of the
staged path whenever it still holds the conversion descriptor, removing it once
that descriptor is released. The same best-effort removal replaces the
`os.unlink` in `_abort_upload_temp`, where an abandoned duplication worker can
hold the inode open and turn a cancellation into a secondary permission error.
Both removals are best-effort: a staged name that cannot be removed yet is
already hidden from every upload listing, and the Gateway sweeps leftovers on
startup.
Verified on Windows (Python 3.12):
- the new regression test reproduces the sharing violation portably (it pins the
staged name while the descriptor lives) and fails on main with the exact 500
above; it passes here
- `pytest tests/test_uploads_router.py tests/test_project_documents_promotion.py
tests/blocking_io/test_project_documents_promotion.py` -> 88 passed
(16 of those failed on main on this host)
- `ruff check` / `ruff format --check` clean
Co-authored-by: Shxiao101 <Shxiao101@users.noreply.github.com>
* fix(skills): rollback reads its history off the event loop
rollback_custom_skill constructed the user-scoped storage, probed whether the
skill exists and parsed custom/.history/<name>.jsonl inline, while the sibling
get_custom_skill_history two lines above already offloads exactly those three
steps and documents them as "blocking filesystem IO that must stay off the
event loop". The history file grows one entry per edit carrying full previous
and new content, so a rollback request parsed the whole edit history on the
Gateway loop; the strict Blockbuster gate catches the construction as a blocking
os.getcwd() through SkillsConfig.get_skills_path() -> project_root().
Offload the three steps through the same worker-thread closure and anchor both
pre-scan branches under tests/blocking_io/.
* test(skills): address rollback anchor review feedback
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(channels): fix Telegram inbound file download and sandbox readability
- Shut down the download Bot on the Telegram loop (Bot.shutdown(), which
closes its HTTPX clients) before the loop stops, instead of the
non-existent Bot.session.close().
- Grant group/other read on channel-downloaded uploads so the non-root
AIO/Docker sandbox process can read the root-written 0o600 file.
- Apply the sandbox permission change with os.fchmod on a descriptor
opened with O_NOFOLLOW (validated as a regular file via fstat), bound
to the validated upload inode, so a symlink swapped in after lstat
cannot redirect the chmod to a target outside the uploads directory.
The open also uses O_NONBLOCK so a sandbox-swapped FIFO cannot block the
read-only open before the regular-file check (matching the existing
open_upload_file_no_symlink convention).
Centralized in a shared apply_upload_sandbox_permits helper reused by
the channel inbound path and the HTTP upload readable/writable helpers.
- Surface the download failure cause chain in logs with the Bot API URL
masked: the configured token is redacted and both URL forms are
collapsed, covering the file download URL (/file/bot<token>/...) and
the method URLs (/bot<token>/getMe, /bot<token>/getFile).
- Migrate the existing receive_file tests onto the download Bot and add
coverage for _get_download_bot (loop-bound creation + caching, cleanup
on init failure), download-bot routing over the application bot,
real-Bot shutdown closing both HTTPX clients, receive_file
timeout-containment, masked cause-chain logging (file and method URL
forms), and inbound-file sandbox perms (including the swap-after-lstat
symlink regression).
* fix(uploads): surface sandbox permission failures
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(uploads): stop deleting a converted companion we cannot prove we wrote
Conversion names a document's Markdown companion after the document's
stem and falls back to a _N suffix when that name is already taken, so
the .md beside a document may belong to another document sharing the
stem, or to the user. Delete removed it anyway: uploading a.docx and
a.pdf produces a.md and a_1.md, and deleting a.pdf destroyed a.docx's
companion while orphaning a.pdf's own.
Delete now removes only the file it was asked to remove. The companion
stays listed and can be deleted on its own. Orphans are the cost of not
guessing; issue #5672 covers giving companions a provable owner, which
is what a safe cleanup needs, along with the two related readers that
still guess (the outline injected for a document and the agent's file
listing).
convertible_extensions loses its last use and is dropped from the
signature and both call sites. The gateway router keeps importing
CONVERTIBLE_EXTENSIONS for the ingestion bridge and now declares it in
__all__, where that module documents its re-exports.
* docs(changelog): note that delete keeps the converted markdown (#5673)
* feat(channels): add WeChat QR login and binding recovery
* fix(channels): enforce single-worker WeChat QR login and preserve bot ID
Reject QR login endpoints when multiple Gateway workers are configured, while keeping manual token setup available.
Preserve the configured bot ID when the provider omits it or returns an empty value. Add regression coverage for worker guards and credential persistence.
* fix(channels): sync WeChat completion state on provider updates
Show the connected step when refreshed provider data confirms the binding, so the dialog no longer waits indefinitely after polling is cancelled.
Add regression tests for provider updates during pending poll and binding requests, including late responses and expiry.
* fix(channels): preserve WeChat pairing codes across waits and redirects
---------
Co-authored-by: YxinMiracle <“939157765@qq.com”>
* fix(events): serialize DB deletion with thread writers
* fix(runs): delete thread history without dropping reservations
* fix(feedback): support owner-scoped thread cleanup
* fix(threads): clean persisted records on deletion
* fix(threads): correct the feedback cleanup rationale
* test(runs): drop the wall-clock probe from the in-flight delete test
* docs: record the thread-delete and event-store fence contracts
* fix(threads): preserve legacy event-store delete compatibility
* fix(gateway): preserve owner isolation when thread metadata is missing
Follow-up to the #5448 review P1 (post-merge finding): owner_check=True
also authorizes threads whose meta row is missing (legacy compatibility)
or NULL-owner (shared/pre-auth data). _run_scope_user_id returned None
for every trusted internal caller, which dropped the only remaining
per-user filter on those threads and let an internal caller acting for
owner A read owner B's persisted runs.
_run_scope_user_id now takes the thread_id and consults the thread meta
store: when an existing meta row establishes ownership, the authorized
thread's runs are still read unfiltered (merged #5448 semantics,
including owner-header-less internal callers); when the meta row is
missing or NULL-owner, the filter falls back to the acting owner's raw
stamp (the exact value start_run writes) — or the synthetic "default"
identity without an owner header — so cross-user runs stay hidden.
Isolation coverage uses the real MemoryThreadMetaStore with no metadata
row (and a NULL-owner row) plus another user's persisted run: /runs and
/runs/page must be empty and /runs/{run_id} must 404 for internal
callers, while an established-ownership thread keeps the unfiltered
read.
* fix(gateway): gate run-scoped sub-resource reads for internal callers
Review follow-up on #5484: the P1 owner-isolation class remained
reachable through run-scoped sibling reads that apply no per-user filter
at all — /runs/{run_id}/messages, /events, /join, /stream and
/workspace-changes query by (thread_id, run_id) directly, so on
missing/NULL-owner threads an internal caller acting for owner A could
still read owner B's run content by id (verified 200 at the previous
head).
- Extract _thread_ownership_established (shared meta-row check) and add
_require_run_visible_to_scope: for internal callers on threads without
established ownership, the run's own user_id stamp must match the
acting owner's raw value (or the legacy "default" stamp) or the read
404s. Established-ownership threads and every non-internal caller keep
their existing thread-scoped semantics.
- Wire the gate into join, stream, messages, events and
workspace-changes; reword the now-stale messages comment to track the
new scoping semantics.
Regression tests: sub-resource reads 404 for a mismatched internal
owner while the matching owner reads them normally, and the owner-less
fallback branch (synthetic "default" filter on missing-meta threads) is
pinned. Red confirmed against the pre-gate head.
* fix(gateway): gate cancel and artifact archive for internal callers
Review follow-up on #5484 round 2: POST /cancel resolved runs unscoped
(require_existing=True only closes the missing-meta case — NULL-owner
meta rows still pass), so an internal caller acting for a different
owner could interrupt another owner's active run on a shared thread
while /join and /stream were already gated. The archive manifest and
download pair likewise leaked the other owner's delivered-file count
and a 200-vs-409 delivery oracle on NULL-owner threads (missing-meta
threads were already denied by require_existing=True).
All three routes now call _require_run_visible_to_scope; its docstring
records the extended coverage. NULL-owner-thread regression tests pin:
a mismatched internal owner gets 404 from cancel, manifest and archive
download, while the acting owner reaches the real conflict path (409 on
a terminal run) and reads the manifest (file_count 2).
* fix(gateway): tolerate state-less request stand-ins in the scope helpers
The new owner-isolation gate and _run_scope_user_id read request.state
directly, which crashed the FakeRequest-based unit suites for the run
events, workspace-changes and scope endpoints (backend-unit-tests shards
1/2/4 on #5484). Read the state object defensively first: a request
without state is simply not an internal caller, so those paths keep
their pre-gate semantics.
* fix(gateway): scope the thread token-usage aggregate by owner
Review follow-up on #5484 round 4: GET /{thread_id}/token-usage called
aggregate_tokens_by_thread(thread_id) with no user filter at all, so on
missing/NULL-owner threads an internal caller acting for owner A read
owner B's spend, model names, run count and (with include_active=true)
live activity; the NULL-owner variant reached browser sessions too.
build_context_usage's latest-model lookup was unfiltered as well.
aggregate_tokens_by_thread gains an optional user_id (mirroring
list_by_thread: explicit None = unfiltered, AUTO resolves the contextvar)
in the memory store, the SQL repository and the store base;
build_context_usage/_resolve_thread_model_name thread the scope through
the latest-run lookup; the token-usage endpoint passes
_run_scope_user_id's value. Established-ownership threads aggregate
unfiltered as before; shared/missing-meta threads narrow to the acting
identity. Stale helper-test comment reworded after the #5482 merge
adaptation.
* test(gateway): pin the unfiltered aggregate on established-ownership threads
Review follow-up on #5484 round 5: the established-ownership branch of
the token-usage scoping (store receives user_id=None) was the only
unpinned half of the contract — the round-4 call-assertions never set
app.state.thread_store, so their None came from the user-less stand-in
path. test_token_usage_unfiltered_on_established_ownership_for_
internal_callers seeds an established meta row plus runs stamped by two
different identities and asserts the totals fold (166 = 111 + 55);
together with the isolation tests it now catches both failure modes
(always-stamp narrowing and always-None leak).
* feat(authz): filter per-caller skill visibility on the listing surfaces (#4063 Phase 4)
GET /api/skills, GET /api/skills/custom, and GET /api/skills/{name} now
filter the user-scoped catalog through filter_resources(principal,
"skill", ...) — mirroring list_models. Anonymous callers are unfiltered;
provider errors follow authorization.fail_closed (fail-closed -> empty
listing / 404, fail-open -> full listing). An invisible skill on the
detail surface returns the standard 404 so the endpoint cannot become an
existence oracle the filtered list closed. Management endpoints stay
require_admin_user-gated; runtime activation is #4541's layer.
resolve_skill_authorization joins resolve_model_authorization as a thin
sibling over a shared _resolve_route_scoped_authorization core.
* docs(authz): reflect per-caller skill visibility in OpenAPI metadata and implementation notes (#5489)
Address the two non-blocking review findings on #5489:
- The three user-facing GET routes (/skills, /skills/custom,
/skills/{name}) now say in their /docs-visible descriptions that
authorization filters the response (hidden skills 404 on detail).
- Add the dated Phase 4 decision-log entry to the authorization
implementation notes, per the convention of every prior merged
authz PR: listing-visibility semantics, the 404-vs-403
existence-oracle rationale, anonymous-caller behavior, and the
#4541 rebase reconciliation points (config.example.yaml roles
comment + this file's decision log).
* docs(authz): move route guidance into Gateway module guide
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
The message edit/regenerate prepare chain resolves source runs through
_resolve_run_id_for_message, _require_successful_source_run and
_find_interrupted_target_run_id, which filtered by the authorization
identity (get_current_user) — the same conflation fixed for the runs and
messages read endpoints in #5448, left out of scope there. For trusted
internal callers the authorization identity never matches the raw
owner-stamped run rows, so regenerate/edit-regenerate prepare fails with
409 on threads the caller is authorized on (#5482).
The three helpers now resolve their filter id through _run_scope_user_id
as well; both prepare endpoints keep their owner_check=True
authorization and browser/API sessions keep the per-user filter.
Regression tests extend test_thread_runs_internal_scope.py to the helper
fallback paths: internal callers resolve raw-owner-stamped runs
(including the interrupted-run and status-409 paths), browser sessions
keep the per-user filter and 409 on cross-user runs.
* feat(projects): Projects MVP Phase 2 — instructions, document shelf, promotion, trash
Implements docs/superpowers/specs/2026-09-12-projects-mvp-phase2-design.md
(issue #5160, tracker #5129) in the slice order of the spec's §16.
Slices:
- A: ProjectsConfig + write-time 422 UTF-8 byte cap; PROJECT_CONTEXT_KEY
admission pinning (both server-owned sets + worker hoist); latest-only
request-scoped <project> block via DynamicContextMiddleware
wrap_model_call/awrap_model_call (idempotent reassembly, reserved ID
prefix + marker + provenance, never persisted); journal audit
fingerprints; Instructions tab.
- B: ProjectDocumentRow + migration 0023; ProjectDocumentRepository with
locked check-and-set; hash-qualified immutable shelf storage with
Paths helpers; upload/list/content/delete-to-trash routes; project
delete trashes the shelf in-transaction; request-scoped bounded
<documents> index with honest count/shown + actionable overflow note;
list_project_documents/read_project_document tools registered only on
pinned runs; PAT allowlist + drift guards; blocking-IO anchors.
- C: shared thread-upload ingestion service (uploads router refactored to
parity); POST from-thread with provenance; attach-to-thread with
lock-staged copy (archived source allowed); read-only thread-files
view with per-group truncation reporting.
- D: restore (restored/merged/not_found/no_target/content_missing; no
file moves), purge (continuous row lock across unlink/delete/commit,
retryable on FS errors), retention sweep (lazy + startup, 24h orphan
guard, row-side reconciliation never deletes).
- E: Documents tab (shelf + conversation-files browser, provenance,
archived banner, content-missing rows), /workspace/trash route,
sidebar entry, composer attach handoff, i18n (en-US/zh-CN), e2e mocks
+ specs.
Review hardening folded in (10 rounds, all with tests):
- force active shelf content (HTML/XML family) to download; nosniff on
artifact + content responses; unified unsandboxed-iframe PDF preview
(fixes the pre-existing Chromium sandbox blank in the artifact viewer)
- scope document trash to the URL project under the document lock
- atomic no-overwrite filename reservation for ALL ingestion (seeded
claims + os.link commit with suffix retry; same-name re-upload now
unique-names instead of replacing); hidden staging only, no visible
placeholders; lease cleanup on setup failure
- serialize conversion under the document lock with post-lock active
revalidation; drain locked filesystem work on cancellation; preserve
bytes when an insert's commit state is uncertain (including trashed
rows)
- original-integrity checks before serving text or cached conversions;
content_missing surfaced in list responses (UI reads the flag, no
409-probe); downloads always serve original bytes
- bounded streaming document reads with cached char counts; shelf limits
declared in middleware release identity
- thread-root confinement for from-thread sources; config fallback
rejects fractional/infinite values; composer counts staged
attachments; pending attachments persist until submission or removal;
in-flight instruction/rename edits survive save refetches; shelf and
trash pagination; conversation-file and thread-files pages stay
subscribed to refetches
Docs: README/README_zh, backend API.md/ARCHITECTURE.md, AGENTS.md
contracts, config.example.yaml projects block.
Review follow-ups (head b4807477 → this revision):
- The trash retention sweep is split so repeated lazy triggers stay
bounded: the indexed expiry purge still runs on every trigger
(GET /api/trash/documents, POST /api/trash/purge) while the
O(all rows + all files) reconciliation is throttled to one run per
user per 15 minutes (process-local, per-user window). The startup
sweep now runs as a background task instead of blocking gateway
readiness, and shutdown awaits it (bounded).
- The export scrub (stripInternalMarkers) is fence- and indentation-aware
like the render path, so a pasted, fenced <project>/<documents> snippet
survives markdown export while real injected blocks (never fenced) are
still removed. Fence regexes moved to a dependency-free leaf module to
avoid the messages↔streamdown import cycle.
- The artifact viewer's PDF iframe no longer carries an added title
attribute (the upstream e2e contract locates it via :not([title])), and
the upstream artifact-preview spec now pins the new contract: PDFs
render unsandboxed, images keep sandbox="".
* fix(projects): round-2 review — cancel an overrun trash sweep, restore the PDF frame title
- Shutdown cancelled only the shield around the background startup sweep,
so an all-users reconciliation that outlived the 5s budget kept walking
rows and files while the document repo and DB engine were disposed
underneath it. The wait now lives in `_shutdown_startup_trash_sweep`,
which cancels the task and drains it before worker exit: the shield
keeps the wait bounded, the cancel makes it final (CancelledError lands
at the sweep's next await, and `_run_startup_trash_sweep` only catches
`Exception`, so nothing swallows it).
- The browser-preview iframe lost `title={getFileName(filepath)}` in the
previous fix round, leaving the PDF frame without an accessible name
while its siblings keep theirs. Restore it (WCAG frame titles), assert
it in the DOM test, and anchor the e2e on `iframe[title="report.pdf"]`
instead of `iframe:not([title])`.
* fix(projects): round-3 review — report the sweep's late finish, not a phantom cancel
`Task.cancel()` returns False when the sweep already finished inside the
window between the deadline firing and the cancel, so the shutdown log
claimed a cancellation that never happened. Branch on that outcome: the
warning stays for a real cancel, a late finish is logged at info, and both
paths still reap the task before worker exit.
* fix(projects): round-4 review — make Empty trash delete what it confirms
`POST /api/trash/purge` only ran the retention sweep, and the sweep's
candidate selection is age-gated, so a freshly trashed document survived
"Empty trash" even though the confirmation promises that every listed
document is permanently deleted. With one trashed row the route answered
`{"purged": 0}` and left it in place; `GET /api/trash/documents` sweeps
expired rows before listing, so the visible rows were normally ineligible
for the action by construction.
Empty trash now drives `purge_all_trashed`: the caller's trashed rows
(`list_all_trashed`, no age filter) each go through the same guarded,
row-locked `purge` as the single-document delete — bytes first, then the
row, in one transaction — so a row restored mid-flight is skipped instead of
force-deleted, and an unlink failure rolls that row back and answers 500 with
a retryable message. Retention expiry stays where it was: the sweep's
`purge_candidates` is now the only age-gated selection, and the lazy
retention sweep still runs on the listing and at startup.
Tests: the router suite replaces the retention-gated expectation with the
reviewer's repro (fresh row purged, bytes unlinked, shelf and other users'
trash untouched, a failing unlink stays retryable and 500); a blocking-I/O
anchor drives the new entry point through the offload; the mocked e2e covers
the action end to end; a new real-backend spec performs it against the real
gateway and re-reads `GET /api/trash/documents`. README, API, ARCHITECTURE
and the phase-2 design docs (en+zh) state the age-independent contract.
* fix(gateway): scope runs read endpoints by data identity, not authorization identity
Trusted internal callers are authorized as a synthetic internal user
(id "default", or the make_safe_user_id-normalized owner with an owner
header), while start_run stamps run rows with the raw trusted-owner value.
list_runs / get_run / list_runs_page filtered by the authorization identity,
so the store-side user filter never matched and internal callers always saw
an empty runs list (or 404) for threads they are authorized to read.
The three read endpoints now resolve their filter id through
_run_scope_user_id: internal-role callers skip the per-user filter (thread
visibility is already authorized by owner_check=True), and browser/API
sessions keep the existing per-user filter unchanged.
Regression tests cover both identities across the three endpoints (list,
keyset page, single get) with a MemoryRunStore seeded with mixed-owner rows;
without the fix the four internal-caller cases fail while the
browser-session isolation case passes.
* fix(gateway): route the message read endpoints through the same data-identity scoping
Review follow-up on #5448: list_thread_messages and list_thread_messages_page
resolved get_current_user and passed it as the data filter to the event-store
scan, hidden-run lookups, turn-duration injection and the feedback queries —
the same authorization-vs-data identity conflation fixed for the runs
endpoints, leaving the #5437 empty-read symptom in place for lossy owner
values. Both endpoints now resolve their filter id through
_run_scope_user_id as well.
Regression tests extend to the two message endpoints, asserting the resolved
filter identity at the runs-store and feedback-repo boundaries (None for
internal callers, the session user id for browser sessions).
* fix(feedback): deterministic per-run collapse for unfiltered feedback reads
Review follow-up on #5448: with _run_scope_user_id returning None for
internal callers, the feedback lookups now receive an explicit-None user id,
which skips the user_id WHERE in FeedbackRepository. On shared/NULL-owner
threads several browser users can hold feedback on the same run, and
list_by_thread_grouped / list_by_run_ids collapsed rows per run_id via a
dict comprehension over unordered results — the feedback attached to the
last AI message would be an arbitrary user's row.
Both methods now order by created_at ASC with feedback_id as the tie-break,
so the collapse deterministically keeps the most recently created feedback.
_run_scope_user_id's docstring now documents that the resolved id also
scopes feedback and event-store reads, not just run rows.
Regression test seeds multi-user feedback on one run and asserts the
collapse outcome is stable across repeated unfiltered reads.
* docs(feedback): the collapse keeps the most recently written feedback
created_at is refreshed on upsert, so the surviving row per run is the
most recently written (created or updated), not the most recently
created — align both docstrings with the ordering key's actual semantics.
* feat(gateway): accept conversation references in run context and report the capability
LangGraph SDK clients build a fixed run body and drop unknown top-level
fields, so they cannot send the conversation_references field from #5399.
RunCreateRequest now lifts context.conversation_references into the
top-level field before validation, so it keeps the same bounds and error
locations, and drops it from context, so it never reaches the merged run
context or the checkpointed configurable. Sending both is a 422.
GET /api/features reports conversation_references {enabled, max_references}
with the same "tool is configured" predicate as run admission, so a client
can hide an entry point on deployments without the tool.
Related to #5398.
* fix(gateway): report the field type error for a malformed top-level reference list
A malformed top-level conversation_references sent alongside a context
list now fails with the field's own type error instead of the conflict
message. The tool-configured predicate reads tool.use directly, and the
features test doubles carry that attribute like every real ToolConfig.
* fix(gateway): treat every list-like top-level reference value as a conflict
Pydantic's lax mode coerces tuples, sets, frozensets and deques into the
list[str] field, so a direct Python caller passing one of those together
with context.conversation_references now reports the conflict instead of
slipping both grants through. Unreachable over HTTP, where JSON has no
such types.
* fix(gateway): ask pydantic whether a top-level reference value is list-like
Enumerating list-like types cannot track pydantic's lax acceptance set
(generators, UserList, dict key views also coerce into list[str]). The
conflict guard now validates the top-level value with a TypeAdapter for
list[Any]: whatever pydantic would coerce reports the conflict when it is
non-empty, and whatever it rejects still surfaces the field's own type
error. Regression tests cover deque, UserList, dict keys and a generator,
plus rejected scalars.
* fix(gateway): probe top-level references with the field's own annotation
The conflict guard now validates the top-level value with the exact item
annotation the field uses, so its acceptance set is the field's rather than
a superset: an item the field rejects (an empty string, a non-string, the
ints of a range or dict view) surfaces the field's own item error instead
of a conflict. The annotation is shared through one alias so the two cannot
drift.
* fix(gateway): materialise a one-shot iterator before probing top-level references
The item-validating probe could consume a generator while collecting an
item error, after which the field re-validated the exhausted iterator,
coerced it to [] and let the request through with the key still in
context. Iterators are now read once into a list that both the probe and
the field validate, so a bad item is reported at its index and a valid
generator is kept.
* fix(gateway): materialise every once-walkable iterable before probing references
Pydantic coerces any iterable into the list field, and an object whose
__iter__ hands out a generator once is not an Iterator instance, so the
previous gate let it reach the probe and be consumed. The lift now reads
every iterable except lists, tuples and the shapes the field rejects as a
whole (str, bytes, dict) into a list first, so the probe and the field
always validate the same items.
---------
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
* feat(scheduled-tasks): filter run history by occurrence status
Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com>
* fix(scheduled-tasks): share occurrence status contract
---------
Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(settings): persist account preferences across browsers
* docs(settings): scope preference guidance to user persistence
* fix(settings): preserve SSR and fence custom-agent defaults
* test: include user persistence in scoped guidance inventory
* fix(settings): sync explicit edits and preserve local tab updates
* feat: add opt-in task notes and compacted history recall
* fix: validate task continuity state and preserve user answers
Honor explicit opt-out, preserve clarification replies and capture failure statuses, validate notebook writes, and clear branch archive references. Update the config version and audit optional LLM credentials, with regression and integration evidence.
* fix: align Helm config version with task continuity schema
* fix: preserve mixed task history and declare continuity policies
* fix: recover malformed history and evict archives atomically
Toggling a skill wrote resolved secrets into extensions_config.json. The
Gateway skill toggle and DeerFlowClient.update_skill loaded the file with
ExtensionsConfig.from_file(), which replaces every "$VAR" string with the
environment value (and an unset variable with ""), then serialized that
model back through to_file_dict(). A "$GITHUB_TOKEN" reference was
persisted as the plaintext token, and an unset reference was erased for
good. DeerFlowClient.update_mcp_config had the same flaw for every key
other than mcpServers.
Every writer now does a raw read-modify-write, the way the MCP router
already did: read_raw_extensions_config reads the on-disk JSON,
set_raw_skill_enabled changes only the target entry, and
validate_raw_extensions_config checks the candidate the way the runtime
will load it before the atomic write. When the file does not exist yet,
the Gateway seeds it with the cached skill states only, never the
resolved cached model. The MCP router's raw loader and candidate
validation delegate to the same helpers, so the rule lives in one place,
and to_file_dict() is removed so the unsafe serialization has no entry
point left.
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(gateway): serve XML artifacts as attachments to block same-origin script
GET /api/threads/{id}/artifacts/{path} forced only text/html,
application/xhtml+xml and image/svg+xml to download. Every other XML
document was served inline from the application origin: `.xml` guesses
to text/xml or application/xml depending on the host's mime.types, and
both fell through to the inline text branches. Browsers render any XML
MIME type as a document and run an XHTML-namespaced <script> inside it,
so a report.xml written by a prompt-injected agent and opened from a
chat link executed with the viewer's session: the HttpOnly access_token
rides same-origin fetches, and the double-submit csrf_token cookie is
JS-readable, so state-changing calls are reachable as well.
Treat HTML plus every WHATWG XML MIME type (text/xml, application/xml,
any +xml subtype) and text/xsl, which Blink also renders as XML, as
active content. A single helper owns the rule for both the regular-file
and the .skill-archive-member branches. The artifacts panel already
previews .xml as code through a ranged fetch, so preview and editing
keep working against the attachment response.
* docs(frontend): name XML among the artifacts the Gateway downloads
Review follow-up on #5353: resolveArtifactOpenURL's comment still named
only HTML/SVG as the active content the Gateway serves as a download.
XML documents now join that bucket, so the frontend note matches the
Gateway rule. Comment-only; no behavior change.
* fix(tools): run tool assembly off-loop at async entry points
get_available_tools() may block on MCP cache initialization while it is
called on async agent-assembly paths (task_tool, durable batch execution),
stalling the calling event loop for the full discovery duration.
Dispatch the (unchanged, synchronous) assembly call to a worker thread via
asyncio.to_thread at the two async entry points so the loop keeps processing
requests, SSE frames, cancellations, and timers.
Fixes#5172
* fix(tools): offload lead-agent assembly off-loop and pin with blocking-io anchors
Review follow-up for #5224:
- run_agent now dispatches agent_factory(...) through asyncio.to_thread, so
lead-agent assembly (including both get_available_tools call sites in
_assemble_lead_agent) runs off the event loop — the Gateway headline
scenario from issue #5172.
- _ensure_sync_invocable_tool takes a double-checked threading.Lock, making
the in-place tool.func wrap on the shared tool singletons explicitly
single-shot now that assembly can run concurrently on worker threads.
- Add backend/tests/blocking_io/test_tool_assembly_offloop.py: blocking-probe
anchors for task_tool and SubagentBatchService._execute_item under the
strict Blockbuster gate, plus a meta-check proving the gate trips on the
exact syscall class (ExtensionsConfig.from_file on the loop). Verified the
anchor goes red when the offload is flattened back to a plain call.
* fix(gateway): build checkpoint state accessor off-loop; anchor run_agent offload
Review follow-up for #5224:
- Add abuild_checkpoint_state_accessor (asyncio.to_thread around the
unchanged sync builder) and switch every async call site to it: the
stateless_wait route, thread_runs, both threads call sites, and the
build_thread_checkpoint_state_accessor boundary. The agent-factory
assembly re-enters get_available_tools() and may block on MCP cache
initialization; repeat calls hit _state_accessor_graph_cache and only
pay the thread hop.
- Add a third blocking-io anchor driving the real run_agent with minimal
RunManager/bridge stubs; the factory performs a real production blocking
read (ExtensionsConfig.from_file()) and the test asserts assembly never
runs on the main thread. Verified the anchor goes red when the run_agent
offload is flattened back to a plain call.
- Adapt the test_threads_router checkpoint-builder patch sites to the new
async name.
* refactor(tools): carry assembly offloads on a dedicated bounded pool
Review follow-up for #5224:
- Add utils/assembly_io.py: a dedicated ThreadPoolExecutor (default 8
workers, DEER_FLOW_ASSEMBLY_WORKERS-overridable, mirroring
utils/file_io.py and tools/sync.py) with run_assembly(), which copies
contextvars explicitly. A hung stdio MCP server parks its worker for
the full MCP timeout; carrying assembly hops on the loop's default
executor would let a few parked assemblies queue every other
to_thread/run_in_executor(None, ...) caller behind them.
- Switch all four offloads (run_agent, task_tool, batch _execute_item,
abuild_checkpoint_state_accessor) to run_assembly().
- State the cold-path behavior in the accessor docstring: the graph
cache validates factory identity, so non-identity-stable factories may
duplicate lead-agent assembly across concurrent readers (MCP discovery
stays process-wide single-flight); the pool bounds the duplicates.
- Add a fourth blocking-io anchor driving build_thread_checkpoint_state_
accessor with a per-resolution fresh factory (always a cache miss) and
the real production blocking read; enumerate all four offloads in the
gate's module docstring. Verified the anchor goes red when
abuild_checkpoint_state_accessor is flattened back to a plain call.
* fix(subagents): revalidate batch item before launch; make assembly pool observable
Review follow-up for #5224:
- _execute_item() revalidates the durable state right after assembly and
before executor.execute_async(): renew_item_lease() returns valid=False
when cancel_batch() terminalized the item or the lease was lost while
assembly was parked, and the launch is skipped (the canceller already
finalized the item). Previously the launch was unconditional and the
poll loop's cancellation checks only started after execution began.
- Regression test driving the real SQLite repository: a blocking assembly
probe parks _execute_item, cancel_batch() lands, and the launch is
skipped with the item staying cancelled. Verified the test goes red
when the revalidation is removed.
- run_assembly() tracks pending assemblies and logs a throttled WARNING
once the pending count exceeds the worker count, so assembly starvation
(workers parked on a hung MCP server) is distinguishable from idle.
- The run_agent blocking-io anchor now binds a sentinel extension
snapshot via ctx.extensions and asserts the factory observed it through
get_agent_build_extensions(), pinning run_assembly()'s ContextVar
propagation. Verified red when ctx.run is dropped.
- Document the assembly pool in backend/AGENTS.md.
* fix(utils): decrement the assembly pending count on the pool thread
The pending-assembly counter behind the starvation warning decremented
from the asyncio future's done callback, which never fires once the
submitting loop is closed while its worker is still running: the count
ratcheted up permanently and eventually fired the starvation warning
with no starvation behind it (reproduced at 97dc9bec by review).
Decrement instead from the dispatched work item: run_assembly() wraps
func so a finally drops the count under the pending lock on the pool
thread, and the done callback is gone.
Pin the counter with tests/test_assembly_io.py: a healthy call returns
the count to zero, and an abandoned loop (stopped while the worker is
parked) does not wedge it — the abandoned case goes red against the old
done-callback decrement.
* docs(utils): fix the pending-counter comment after the decrement move
The comment still described the removed done-callback decrement,
contradicting _work()'s own comment; state the actual mechanism
(increment on the loop before dispatch, decrement from the dispatched
work item's finally on a pool thread).
* test(gateway): retarget checkpoint-accessor stubs to the services seam
thread_runs and runs now call abuild_checkpoint_state_accessor, so the
upstream wait-reader, regenerate-prepare, and idempotency tests must stub
the sync builder where abuild resolves it (app.gateway.services); stubbing
the removed router re-exports fails with AttributeError at setup. The
async seam semantics are unchanged: run_assembly invokes the stubbed
sync builder off-loop and propagates its return values and exceptions.
Move the agent/tool assembly off-load note from backend/AGENTS.md to
deerflow/utils/AGENTS.md (next to assembly_io.py) so the effective
instruction chain for agents/middlewares no longer grows past the AG002
hard limit.
* fix(runtime): serialize same-key accessor assembly and release queued-cancel slots
Address the three review follow-ups on the assembly off-load:
- assembly_io: a job cancelled while still queued never runs its work
item, so the dispatched finally never fired and _pending_assemblies
stayed elevated until a false starvation warning. Exactly-once cleanup
now rides the concurrent future's cancelled() state — cancel() only
succeeds before the executor starts the item, so cancelled() is true
precisely when the finally will never run — plus a submit-failure
release; the one-worker queued-cancellation case is pinned red/green.
- services: overlapping cold readers sharing one cache key could both
run full agent assembly. _state_accessor_graph now serializes per key
through a thread-side KeyedLockTable (pool threads, no running loop)
and re-validates factory/app-config identity under the lock, so the
factory runs exactly once while identity changes still rebuild. Cache
dict access is lock-guarded now that construction runs off-loop.
- guidance inventory: register deerflow/utils/AGENTS.md in
EXPECTED_GUIDANCE_PATHS so test_repository_has_the_approved_scoped_
guidance_shape matches the relocated assembly note (CI shard 4).
* test(keyed-lock): pin KeyedLockTable reclamation and waiter bypass directly
Thread-side counterparts of the async table's own tests: overlapping
hold() calls serialize (a late arrival joins the live entry instead of
creating a second lock that bypasses a queued waiter), the last check-in
pops the entry, and many unique keys leave the registry empty. Both
regressions verified red — popping unconditionally trips the late-arrival
test, never reclaiming trips the many-keys test.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(scheduler): add interval schedule type
Allow scheduled tasks to fire every N seconds from last dispatch, not
only wall-clock cron or a single run_at. Cadence is UTC now+N with no
missed-beat catch-up, bounded by min_once_delay_seconds and 30 days.
* fix(scheduler): let interval tasks create, edit, and keep next run
Create/edit now keep every_seconds. Unchanged interval spec no longer
resets next_run_at, including timezone-only PATCH.
* fix(scheduler): keep non-minute intervals on edit
Stop rounding every_seconds to whole minutes in the form. Values that
are not whole minutes or hours now use a seconds unit so edit/duplicate
round-trips the stored cadence instead of rewriting it and resetting
next_run_at. Document that min_once_delay_seconds is also the interval
floor.
* fix(scheduler): clamp interval seconds to the default 60s floor
The new seconds unit allowed 1–59, which the API rejects under the
default min_once_delay_seconds. Clamp the form to >= 60 and show the
floor next to the preview. Also mention interval in the scheduler
field_doc, matching config.example.yaml.
* fix(scheduler): do not clamp interval amount while typing
Keystroke clamp made 90 become 9 -> 60, then 600, and backspace could
not leave 60. Keep the raw field text and apply the 60s floor on blur
and emit only.
* test(scheduler): cover interval input editing
* fix(frontend): preserve saved interval cadence until edited
* style(tests): format scheduled task router tests
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(scheduler): let scheduled tasks pin a custom agent
Create and update accept optional assistant_id, defaulting to lead_agent.
Custom names are normalized and must already exist for the task owner.
The workspace form exposes the same choice, and duplicate copies it.
Fixes#5286
* fix(scheduler): keep assistant-id PR free of interval tests
Drop the six interval tests that belonged to the interval schedule PR
and fail here because this tree still only accepts once/cron.
Treat lead_agent case-insensitively so LEAD_AGENT / lead-agent store
as the default. Omit unchanged assistant_id on edit so a deleted custom
agent does not 422 unrelated PATCH (rename, reschedule).
* fix(scheduler): format task page and browser tests
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(gateway): confine artifact PUT to /mnt/user-data/outputs after path resolution
The outputs-only guard on PUT /api/threads/{id}/artifacts/{path} was a
string-prefix check on the raw path. A percent-encoded `..`
(`outputs/%2e%2e/uploads/x.txt`) survives nginx's variable proxy_pass
untouched, is decoded by Starlette, passes the prefix check, and the
resolver only confines the result to `user-data/` -- so an owner could
overwrite a sibling upload or workspace file in their own thread.
Collapse dot segments before the prefix check, and re-check the resolved
host path against the resolved outputs root so a symlink planted inside
`outputs/` cannot redirect the write either. The normalized virtual path
is what the response echoes and what non-mounted sandboxes receive.
* refactor(gateway): share the outputs-confinement rule with channel attachments
Review follow-up on #5321: the "only under /mnt/user-data/outputs" rule was
implemented independently by the artifact editor and by IM-channel
attachment delivery, and the two copies had already drifted.
Move it into app/gateway/path_utils.py as normalize_outputs_virtual_path
(collapse `..` before the prefix check) and resolve_outputs_confined_path
(re-check the resolved host path against the resolved outputs root, which
also catches a symlink planted inside outputs/). PUT /artifacts and
ChannelManager._resolve_attachments both call the helper; artifact_archive
keeps its stricter ZIP-member rules layered on top.
Tests that previously stubbed resolve_thread_virtual_path for the editor now
stub resolve_outputs_confined_path, and the channel attachment tests patch
path_utils.get_paths, which the helper binds at import like the other
consumers. The confinement itself is pinned by tests/test_gateway_path_utils.py.
* feat(gateway): paginate thread run history (#5282)
GET /api/threads/{thread_id}/runs stays a bare array of the newest 100
runs so LangGraph SDK clients keep working. Add GET /runs/page with a
(created_at, run_id) keyset cursor so callers can walk older history.
* fix(gateway): reject one-sided run history cursors
RunManager.list_by_thread now raises if only one of before_created_at
or before_run_id is set, matching the HTTP 422. Document the per-page
sort cost on the SQL keyset query, and add the missing CHANGELOG
[#5282] link definition.
* fix(gateway): round-trip run page cursors through query strings
Emit next_before_created_at with a Z suffix so '+' is not decoded as a
space. Accept that space, and Z, when parsing. Treat blank cursor
fields as absent and reject a non-ISO before_created_at in RunManager
so a harness caller cannot silently restart at the newest page.
* style(gateway): ruff-format run page cursor files
Collapse the one-sided cursor ValueError and the two before_created_at
asserts so ruff format --check passes at line-length 240.
* feat(projects): project workspaces with scoped chats and thread membership
Backend:
- projects table model and migration; fail-closed ProjectRepository with
ownership checks, CRUD/archive/restore/delete router, and atomic thread
move between projects
- threads_meta.project_id column exposed as reserved deerflow_project_id
metadata; project-aware thread create/search with pagination bounds and
membership echoed in create responses
- first-run admission assigns the project only at genuine first run, seeded
at write time and dropped when invalid; serialized against project
deletion and thread assignment
- branch creation inherits the source thread's project membership (an
archived/deleted project degrades the branch to unassigned instead of
failing the request)
Frontend:
- projects data layer, thread move API, and sidebar projects section with
flat/grouped modes, archived-project threads, and stable virtual-list
offsets
- project detail page with project-scoped new chat
(/workspace/chats/new?project=) and paginated thread list
- move-to-project thread menu, new-project dialog, archived-project gates
- project-scoped new chats pre-create the thread with membership before the
first submit or /goal set, so runs never proceed outside the project
- goal-set preparation is fenced against conversation switches: a stale
continuation is dropped instead of saving the goal or launching the
abandoned submission on the newly opened conversation
- project thread lists join thread lifecycle invalidations (stop, pin) so
an open project page never keeps stale titles, recency, or pagination
* fix(chats): keep archive undo toast when the sidebar row unmounts
The archive success toast was fired from per-mutate callbacks passed to
mutation.mutate. React Query drops those handlers when the observer
component unmounts before the mutation settles; archiving the open chat
removes its sidebar row mid-flight, so the undo toast never appeared and
the e2e archive-undo test timed out waiting for it.
Move the success/error handlers to the mutation level (useArchiveThread
options, same pattern as useMoveThreadToProject) where callbacks are
delivered even after the originating row unmounts.
* fix(projects): pin project thread listing contract and exclude archived chats
GET /api/projects/{id}/threads returned the thread store row verbatim
(list[dict], no response_model): user_id/assistant_id leaked, any future
ThreadMetaRow column would auto-leak, and the OpenAPI schema was empty.
Return a narrow ProjectThreadResponse (the exact fields ProjectThread
declares) with the same metadata secret redaction the surrounding thread
endpoints get from _MetadataRedactingResponse.
The listing also ran search() without the archived filter, so a retired
chat rendered as a normal row on the project page while the sidebar hid
it. Search archived=False to mirror the sidebar's archived:false lists;
restore stays on the global Archived tab.
Both regressions pinned by new router tests: wire-shape allowlist and
archived-member exclusion.
* docs(migrations): record the 0019/0020 chain against the bootstrap reservation
The tree now chains 0018 -> 0019_projects -> 0020_threads_meta_project_id,
so migrations/AGENTS.md was stale twice over: the revision index stopped at
0018 and the rolling-forward section still claimed the tree 'deliberately
remains at 0018'.
Document the new head and record the intentional numeric-prefix reuse of
0019: 0019_projects is in-chain while 0019_thread_incarnations stays the
reserved, allowlisted out-of-tree rollout id. The owning rollout revision
must re-parent onto this tree's head when it merges so alembic never sees
two heads off 0018; bootstrap.py now cross-references that note next to
_FORWARD_COMPATIBLE_REVISION.
* fix(chats): invalidate project thread lists on archive/restore
useArchiveThread refreshed the infinite sidebar cache, threads/search and
the per-thread metadata cache but not the project-scoped list
([...PROJECTS_QUERY_KEY, 'threads', id]) this PR adds — the one thread
mutation not wired to that key, after usePinThread, useRenameThread,
useDeleteThread, useMoveThreadToProject and invalidateStoppedThreadCaches.
An archive from a sidebar row while a project page is open therefore left
the archived chat rendered as a normal row until remount (and undo left it
missing). Invalidate the prefix in the mutation-level success handler.
Regression test asserts the project-list prefix is invalidated on success.
* fix(projects): fetch project discovery only in grouped sidebar mode
RecentChatList mounted two useProjects queries per sidebar render, but
knownProjectIds is consumed only by the grouped-mode exclusion filter; in
the default flat mode every page load paid two GET /api/projects?status=
round trips for data nothing read. Gate both queries on grouped mode —
GroupedProjectList fetches the same keys when the toggle is on and
TanStack dedupes the observers.
Also set retry: false on useProject: a deleted or foreign project 404s
deterministically, and the page renders a dedicated not-found state for
it, so the default 1s/2s/4s retry backoff kept deep links in 'loading'
for ~7s before that state appeared. Matches useThreadMetadata /
useThreadTokenUsage.
* fix(threads): fail closed on project-scoped create in memory mode
MemoryThreadMetaStore.create accepted project_id and silently ignored it,
making memory mode the one membership path that fails open: POST
/api/threads with a project id returned 200 and the run started
unassigned, violating the invariant that a run never proceeds outside the
selected project (the SQL store raises ProjectNotAssignableError inside
the insert transaction for the same request).
Raise ProjectNotAssignableError whenever project_id is present so the
router's existing 404 mapping applies, the frontend keeps the composer
text for a retry, and memory mode behaves exactly like SQL mode.
set_project already reports rejection; create now matches it.
Store-level test (raises, nothing persisted, project filter stays empty,
unscoped creates still work) plus a router-level test asserting the 404
and that no row is left behind.
* fix(projects): window the project page thread list
ProjectThreadsSection rendered every loaded page as a plain Link row, so a
long-lived project accumulated unbounded DOM on the page's scroll surface:
each load-more appended another 100 rows and every formatTimeAgo tick
re-rendered the whole list.
Reuse VirtualThreadList (now generic over any row shape with a
thread_id), pointing its scroll parent at this page's ScrollArea viewport
via the shared [data-slot="scroll-area-viewport"] selector used by
/workspace/chats; under the 60-row threshold it falls back to the plain
render, so small projects are unchanged.
* fix(projects): restore row dividers and pin them with a render test
The row class template literal concatenated transition-colors directly
with the conditional border-b token, so non-final rows rendered the
invalid class 'transition-colorsborder-b' and lost both the divider and
the transition. Compose the row classes with cn() and a boolean guard
instead.
The section moved out of page.tsx into a testable component so the row
markup finally has coverage: a DOM test asserts every row except the
final data row carries border-b (index-based, not last: — correct under
virtualization where the last mounted row is not the last data row), and
the untitled fallback plus load-more button render for a partial page.
* fix(projects): validate forward schemas and fence membership reads
* feat(authz): surface effective route permissions on GET /auth/me (Phase 4, #4063)
GET /api/v1/auth/me now returns the effective route permissions alongside
the user identity, so the frontend can hide actions the caller's role
cannot perform (RFC #4063 Phase 4).
The value reuses the AuthContext that AuthMiddleware already resolves per
request (including PAT-scope intersection and internal-caller semantics),
so /me adds zero extra provider evaluations; a middleware-less composition
falls back to the same resolution _authenticate uses.
Credential-creation responses (register/initialize) leave the field None:
they are public paths where the middleware does not run, and resolving
there would introduce fresh on-loop config loads on those routes.
* test(e2e): expect /auth/me permissions in auth-disabled contract
PR #5228 adds the effective route permissions to GET /auth/me, so the
strict toEqual against the bare AUTH_DISABLED_USER object no longer
holds: the received payload carries six extra keys (the permissions
array). Extend the expected payload with the full registered permission
set in _ALL_PERMISSIONS order — with authorization disabled the gateway
grants exactly that static list, so the pin stays deterministic.
The runtime frontend is unaffected (auth-disabled SSR never calls /me,
and userSchema strips unknown keys); only this contract pin needed the
new field.
* refactor(authz): public resolve_route_permissions_for_request wrapper
Address review nits on the middleware-less fallback: the router reached
into the private authz._is_internal_caller, so expose a thin public
wrapper pairing resolve_route_permissions with the internal-caller
heuristics, and use it from both _authenticate and the /me fallback so
the two cannot drift apart. Also drop an unused tmp_path parameter from
test_auth_disabled_me_includes_default_admin_permissions (_setup_auth
provisions its own tmp directory).
No behavior change: the wrapper delegates to the exact pair of calls the
fallback made before.
* feat(gateway): support idempotent thread runs
Accept Idempotency-Key on thread-scoped create, stream, and wait endpoints, scoped by owner and thread before durable admission.\n\nRefs #5257.
* fix(gateway): handle idempotent run reuse on wait and stream
Reused store-only records have no local task. /wait now waits on the
bridge when it can observe the stream, and otherwise returns durable
status instead of a stale checkpoint. A reused terminal stream that has
been evicted emits gap/reload_durable_state. Replay is bound to the
original input and assistant_id.
* fix(gateway): 409 reused in-flight streams on this worker
A store-only running record on a process-local bridge has no owner
stream. POST /runs/stream used to subscribe anyway, which created an
empty log and waited forever. Match join: 409 unless the run is already
terminal, so missing-stream retries can still emit gap.
* fix(gateway): keep observer joins off the idempotent stream-gap path
sse_consumer keyed missing-stream gap on the sticky
idempotency_reused flag, so a later join of a terminal run inherited
it. Gate that branch on apply_on_disconnect, which already separates
creating streams from joins. Document the retry outcomes clients have
to handle.
* fix(gateway): gate missing-stream gap on creating retry
Reuse apply_on_disconnect to pick gap vs end changed sse_consumer default path, so a missing stream started returning gap for default callers and for out-of-scope POST /api/runs/stream. Keep that branch behind emit_gap_on_missing_stream and pass it only from thread-scoped /runs/stream on this request reuse.
* fix(gateway): keep wait reuse off later checkpoints
Direct handler calls were crashing because FastAPI Header() leaked in as the Python default. Bind Idempotency-Key with Annotated so the default is None, and ignore non-str keys.
A reused completed /wait was still reading the latest thread checkpoint. After a later run on the same thread that is the later run's result. Return durable status instead of claiming the head as this run's output.
* fix(gateway): snapshot wait reuse and refresh store status
idempotency_reused lives on the shared cached record. Capture it before awaiting completion so an overlapping retry cannot suppress the original creating /wait checkpoint.
A store-only peer record still holds admission-time status after the owner publishes END. Refresh durable status/error before returning them.
* fix(memory): cancel buffered extraction when agent is deleted or cleared
* fix(memory): cancel buffered work before agent delete
Address review: cancel before/after delete to close the rmtree race,
scope user_id=None cancels to the legacy root only, and import memory
helpers at module scope.
Signed-off-by: SPEC <zt1y17@soton.ac.uk>
* fix(memory): close remaining cancel races from review
Post-clear cancel, legacy-only all_agents scope, always cancel even when
memory is disabled, and fold cancel+delete into one offloaded thread.
Signed-off-by: SPEC <zt1y17@soton.ac.uk>
* docs(memory): align cancel_by_agent None-scope with legacy root
Document that user_id=None cancels only the legacy no-user bucket, matching
clear/storage semantics, not the whole process-local queue.
Signed-off-by: SPEC <zt1y17@soton.ac.uk>
* test(memory): fix cancel_by_agent docstring regression assertion
Signed-off-by: SPEC <zt1y17@soton.ac.uk>
* fix(memory): address final cancel review nits
Type the delete helper with AgentStore, replace docstring pinning with a
kwargs mapping test, and document scoped cancel + residual window in AGENTS.md.
Signed-off-by: SPEC <zt1y17@soton.ac.uk>
* fix(memory): resolve agent store inside delete worker thread
get_agent_store() does blocking config/FS work; keep it off the event
loop so test_delete_agent_does_not_block_event_loop and backend-blocking-io CI pass.
Signed-off-by: SPEC <zt1y17@soton.ac.uk>
---------
Signed-off-by: SPEC <zt1y17@soton.ac.uk>
