mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-09-25 14:06:18 +00:00
345 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Eilen Shin
|
bd995a6a26
|
fix(agent): align unattended prompt with tool policy (#4919)
* fix(agent): align autonomous interaction guidance * fix(agent): harden interaction policy selection * fix(gateway): protect legacy interaction flags * fix(channels): honor explicit interaction mode * docs(agent): reduce inherited guidance size * fix(agent): honor unattended policy across approval paths --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
YxinMiracle
|
0b7cef2e0b
|
feat(channels): support WeChat QR login from the web UI (#5582)
* feat(channels): add WeChat QR login and binding recovery * fix(channels): enforce single-worker WeChat QR login and preserve bot ID Reject QR login endpoints when multiple Gateway workers are configured, while keeping manual token setup available. Preserve the configured bot ID when the provider omits it or returns an empty value. Add regression coverage for worker guards and credential persistence. * fix(channels): sync WeChat completion state on provider updates Show the connected step when refreshed provider data confirms the binding, so the dialog no longer waits indefinitely after polling is cancelled. Add regression tests for provider updates during pending poll and binding requests, including late responses and expiry. * fix(channels): preserve WeChat pairing codes across waits and redirects --------- Co-authored-by: YxinMiracle <“939157765@qq.com”> |
||
|
Wenchao An
|
1f437f86c6
|
feat(agents): persist default knowledge scopes for custom agents (#5579)
* feat(agents): persist default knowledge scopes for custom agents * style: format agent knowledge guidance * fix(i18n): clarify default knowledge reset hint * fix(knowledge): preserve retries for initially unbound agents |
||
|
Xuehao Xu
|
8ef58eaa90
|
feat(models): manage shared models from Settings (#5596)
* feat(models): add admin UI for shared model management * docs(gateway): keep model guidance within size budget |
||
|
Wenchao An
|
075f4a3607
|
refactor(capabilities): simplify labels and connection validation (#5580) | ||
|
Wenchao An
|
42334f26d7
|
feat(capabilities): unify catalog, plugin configuration and agent selection (#5497)
* feat(capabilities): unify catalog, plugin configuration and agent selection * fix(capabilities): address review isolation, validation and demo issues * fix(capabilities): preserve concurrent selections and guide launcher repair |
||
|
NanPan
|
058b2a49c5
|
fix(extensions): drain service shutdown across cancellation (#5549)
* fix(extensions): drain service shutdown across cancellation * docs(gateway): document extension shutdown drain |
||
|
ZJPex
|
f33b4fb4bf
|
fix(gateway): preserve clarification answers on regenerate (#5544) | ||
|
Hyeonsang Cho
|
f9f3127dc1
|
fix(uploads): delete the requested upload, not a symlink's target (#5547)
* fix(uploads): delete the requested upload, not a symlink's target delete_file_safe resolved the requested path before unlinking it. The uploads directory is writable from local and AIO sandboxes, so a symlink planted under an upload name was followed: deleting alias.pdf removed the victim.pdf it pointed to, and the companion cleanup then removed victim.md, while the link itself survived and the call reported "Deleted alias.pdf". A link resolving outside the directory was already refused by the traversal check, so the damage stayed inside the thread's uploads. The function now checks and unlinks the requested entry itself and treats a symlink as not found, the same way list_files_in_dir already hides it. unlink() never follows the final component, so a file swapped for a link between the check and the unlink removes only the link. Tests cover the helper, the Gateway DELETE route, and DeerFlowClient.delete_upload. * docs(changelog): note upload delete symlink fix (#5547) |
||
|
spud
|
3776f6f5ec
|
fix(threads): clean persisted records safely on thread deletion (#5535)
* fix(events): serialize DB deletion with thread writers * fix(runs): delete thread history without dropping reservations * fix(feedback): support owner-scoped thread cleanup * fix(threads): clean persisted records on deletion * fix(threads): correct the feedback cleanup rationale * test(runs): drop the wall-clock probe from the in-flight delete test * docs: record the thread-delete and event-store fence contracts * fix(threads): preserve legacy event-store delete compatibility |
||
|
NanPan
|
2bdae7518d
|
fix(memory): drain shutdown workers across cancellation (#5531)
* fix(memory): drain shutdown workers across cancellation * fix(memory): contain shutdown config resolution failures --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
zhangwei-way
|
b6503e9a35
|
feat(knowledge): add per-message RAGFlow retrieval scope (#5238)
* feat(knowledge): integrate RAGFlow retrieval and management * test(knowledge): cover merged listing tool * feat(knowledge): add per-message retrieval scope * chore(docs): remove unrelated document * docs(knowledge): add interaction screenshots * feat(knowledge): simplify scope selector trigger * docs(knowledge): refresh selector screenshot * feat(knowledge): defer standalone management * docs(knowledge): show chat-only scope UI * fix(knowledge): honor scope on clarification replies * fix(knowledge): harden scoped replay validation * docs(knowledge): clarify replay scope precedence * fix(knowledge): keep provider settings on tools * fix(config): preserve tools-only knowledge settings * fix(knowledge): submit custom assistant identity * refactor(knowledge): trim PR scope changes * fix(knowledge): sanitize document scope display * feat(knowledge): enable scope selection in main chat * fix(knowledge): emphasize active scope icon without button frame * fix(knowledge): close context scrubbing and refresh e2e checks * fix(knowledge): preserve idempotent canonical retries * fix(knowledge): accept promptless conversation runs * style(knowledge): format backend regression tests * chore(knowledge): trim PR scope and fix frontend format * fix(knowledge): remove shared-scope notice * fix(knowledge): remove scope persistence notice * docs(knowledge): include main chat in catalog scope * fix(knowledge): preserve scope recovery and upgrades * fix(config): preserve LightRAG knowledge upgrades --------- Co-authored-by: foreleven <for-eleven@hotmail.com> |
||
|
zeng-bohan
|
ce3e64242b
|
feat(gateway): checkpoint retention service on the #4189 deletion contract (#5308)
* feat(gateway): thread checkpoint retention service on the #4189 deletion contract Implements exactly the two contract-proven deletion shapes (trailing duration-only leaves, opt-in leaf sibling branches) with head-chain protection, explicit id protection, a strict pending-writes guard, and joint writes-row cleanup. Head resolution uses LangGraph's time-ordered checkpoint ids; storage deletion mirrors the contract's per-backend data model. Ships without a production trigger by design. Validated against the contract suite (12 passed) plus 14 service scenarios across memory and SQLite; Postgres paths are gated on TEST_POSTGRES_URI. Signed-off-by: zengbohan1 <310902929+zengbohan1@users.noreply.github.com> * fix(gateway): survivor-reachability blob GC and memory blob stats in retention service Aligns the deletion service with the review-hardened contract: blob rows are garbage-collected in a whole-thread pass against surviving checkpoints' channel_versions (a real duration-only leaf shares its parent's versions, so per-checkpoint version deletion would corrupt the surviving state), the memory branch of the stats helper counts saver.blobs and returns the full normalized shape, and per-node channel versions are collected during the graph pass that already exists. Signed-off-by: zengbohan1 <310902929+zengbohan1@users.noreply.github.com> * fix(gateway): address review findings on checkpoint retention service Resolves the review at a479cfe (willem-bd): - Untested savers now fail fast: an explicit isinstance allowlist (InMemorySaver / AsyncSqliteSaver / AsyncPostgresSaver) raises NotImplementedError before any row is read or deleted, so a shallow or third-party saver can never issue partial DELETEs. - The chain walk ends (break) instead of raising KeyError when the head's ancestor row is missing, matching the deletable loop's tolerance for missing parents. - enforce_thread_retention takes an optional per-thread lock and documents the concurrency requirement: classification and deletion are two separate passes, so callers must serialize per-thread mutation (runtime _checkpoint_thread_lock) or guarantee quiescence. - Dropped the dead mid-run guard: CheckpointTuple has no `next` field in langgraph-checkpoint 4.1.1, and pending_writes is populated for committed writes too (verified on the list path), so neither is a usable mid-run signal; the caller-held thread lock is the actual protection. - Removed the write-only _node_step/_Node.step and fixed the head-selection docstring (newest by checkpoint id, not (step, checkpoint_id)). - Documented the E1 leaf / history fast-path interaction in the contract doc and module docstring: the wiring PR must sequence retention away from history reads or adopt a policy that spares cache-carrying leaves. - Added regression tests: unsupported saver, missing ancestor row, thread lock parameter. Validation: test_checkpoint_retention_service 18 passed / 8 postgres-gated skipped; contract + lineage suites 18 passed / 6 skipped; ruff check and format clean. * fix(retention): count non-empty writes dicts on memory saver - _checkpoint_ids_with_writes now requires a non-empty writes dict on InMemorySaver: the empty phantom entry for checkpoints whose task wrote nothing no longer counts as "owns writes rows", so the default E1 pruning reaches the memory backend again (it was a silent no-op there). - test_runtime_duration_leaf_pruned_by_default runs the shipping default (strict_pending_write_guard=True) and proves E1 is reachable out of the box on every backend; the stale override and its wrong SQLite premise are dropped. - document that _checkpoint_thread_lock is non-reentrant: a caller already holding it must not pass it in, or retention self-deadlocks. * test(checkpoint-retention): fix stray duplicated def token in test_duration_link_protected_after_next_run The previous push left `async def def test_...` at line 244, which made the module unimportable and failed collection of the whole suite (and ruff format --check). Local copy was already correct; this commit re-pushes the clean file. 18 passed / 8 postgres-skipped verified from a head worktree. * fix(gateway): make retention correct on Postgres and fail closed on a bad cap * validate max_delete_per_run before any store read: a negative cap used to widen the batch (Python slicing) instead of being rejected; * report identical before/after stats for an empty thread instead of returning before stats_after is collected; * protect each namespace's resume head and ancestor chain, so a persistent subgraph's latest checkpoint is no longer treated as a sibling leaf; * read Postgres columns through a row-factory-agnostic helper (the PG savers open cursors with dict_row, where positional access raises KeyError: 0); * classify the duration-only leaf without relying on metadata["writes"], which the Postgres saver strips via get_serializable_checkpoint_metadata. Verified locally on memory, SQLite and a real Postgres 16 instance (62 passed, 0 skipped): the E1 shape now fires on Postgres, which no backend test covered before CI ran the Postgres lig. Signed-off-by: zeng-bohan <zengbh1@gmail.com> * test(gateway): pin the Postgres-shape duration classifier; report per-namespace heads - Deterministic regression for _mark_duration_leaves_without_the_marker: hand-put the Postgres round-trip shape (writes marker popped, source= update + accumulated run_durations + channel_versions identical to the parent) and assert the shipping default prunes it; a control that bumps one channel version (the client update_state shape) with otherwise identical metadata stays protected. Both legs run on memory and SQLite, so the class cannot silently re-widen (a resumable head losing head protection) or re-narrow (E1 never firing on Postgres) without a locally-executing test failing. - RetentionReport.protected_head_id -> protected_head_ids: heads are now selected per namespace, so the report carries every namespace's head (root key = what an unsaved aget_tuple resolves) instead of only the global max - reshape it before the wiring PR starts consuming reports for audit/aggregation. --------- Signed-off-by: zengbohan1 <310902929+zengbohan1@users.noreply.github.com> Signed-off-by: zeng-bohan <zengbh1@gmail.com> Co-authored-by: zengbohan1 <310902929+zengbohan1@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
xiaodu55
|
16f154f32b
|
fix(gateway): preserve owner isolation when thread metadata is missing (#5484)
* fix(gateway): preserve owner isolation when thread metadata is missing Follow-up to the #5448 review P1 (post-merge finding): owner_check=True also authorizes threads whose meta row is missing (legacy compatibility) or NULL-owner (shared/pre-auth data). _run_scope_user_id returned None for every trusted internal caller, which dropped the only remaining per-user filter on those threads and let an internal caller acting for owner A read owner B's persisted runs. _run_scope_user_id now takes the thread_id and consults the thread meta store: when an existing meta row establishes ownership, the authorized thread's runs are still read unfiltered (merged #5448 semantics, including owner-header-less internal callers); when the meta row is missing or NULL-owner, the filter falls back to the acting owner's raw stamp (the exact value start_run writes) — or the synthetic "default" identity without an owner header — so cross-user runs stay hidden. Isolation coverage uses the real MemoryThreadMetaStore with no metadata row (and a NULL-owner row) plus another user's persisted run: /runs and /runs/page must be empty and /runs/{run_id} must 404 for internal callers, while an established-ownership thread keeps the unfiltered read. * fix(gateway): gate run-scoped sub-resource reads for internal callers Review follow-up on #5484: the P1 owner-isolation class remained reachable through run-scoped sibling reads that apply no per-user filter at all — /runs/{run_id}/messages, /events, /join, /stream and /workspace-changes query by (thread_id, run_id) directly, so on missing/NULL-owner threads an internal caller acting for owner A could still read owner B's run content by id (verified 200 at the previous head). - Extract _thread_ownership_established (shared meta-row check) and add _require_run_visible_to_scope: for internal callers on threads without established ownership, the run's own user_id stamp must match the acting owner's raw value (or the legacy "default" stamp) or the read 404s. Established-ownership threads and every non-internal caller keep their existing thread-scoped semantics. - Wire the gate into join, stream, messages, events and workspace-changes; reword the now-stale messages comment to track the new scoping semantics. Regression tests: sub-resource reads 404 for a mismatched internal owner while the matching owner reads them normally, and the owner-less fallback branch (synthetic "default" filter on missing-meta threads) is pinned. Red confirmed against the pre-gate head. * fix(gateway): gate cancel and artifact archive for internal callers Review follow-up on #5484 round 2: POST /cancel resolved runs unscoped (require_existing=True only closes the missing-meta case — NULL-owner meta rows still pass), so an internal caller acting for a different owner could interrupt another owner's active run on a shared thread while /join and /stream were already gated. The archive manifest and download pair likewise leaked the other owner's delivered-file count and a 200-vs-409 delivery oracle on NULL-owner threads (missing-meta threads were already denied by require_existing=True). All three routes now call _require_run_visible_to_scope; its docstring records the extended coverage. NULL-owner-thread regression tests pin: a mismatched internal owner gets 404 from cancel, manifest and archive download, while the acting owner reaches the real conflict path (409 on a terminal run) and reads the manifest (file_count 2). * fix(gateway): tolerate state-less request stand-ins in the scope helpers The new owner-isolation gate and _run_scope_user_id read request.state directly, which crashed the FakeRequest-based unit suites for the run events, workspace-changes and scope endpoints (backend-unit-tests shards 1/2/4 on #5484). Read the state object defensively first: a request without state is simply not an internal caller, so those paths keep their pre-gate semantics. * fix(gateway): scope the thread token-usage aggregate by owner Review follow-up on #5484 round 4: GET /{thread_id}/token-usage called aggregate_tokens_by_thread(thread_id) with no user filter at all, so on missing/NULL-owner threads an internal caller acting for owner A read owner B's spend, model names, run count and (with include_active=true) live activity; the NULL-owner variant reached browser sessions too. build_context_usage's latest-model lookup was unfiltered as well. aggregate_tokens_by_thread gains an optional user_id (mirroring list_by_thread: explicit None = unfiltered, AUTO resolves the contextvar) in the memory store, the SQL repository and the store base; build_context_usage/_resolve_thread_model_name thread the scope through the latest-run lookup; the token-usage endpoint passes _run_scope_user_id's value. Established-ownership threads aggregate unfiltered as before; shared/missing-meta threads narrow to the acting identity. Stale helper-test comment reworded after the #5482 merge adaptation. * test(gateway): pin the unfiltered aggregate on established-ownership threads Review follow-up on #5484 round 5: the established-ownership branch of the token-usage scoping (store receives user_id=None) was the only unpinned half of the contract — the round-4 call-assertions never set app.state.thread_store, so their None came from the user-less stand-in path. test_token_usage_unfiltered_on_established_ownership_for_ internal_callers seeds an established meta row plus runs stamped by two different identities and asserts the totals fold (166 = 111 + 55); together with the isolation tests it now catches both failure modes (always-stamp narrowing and always-None leak). |
||
|
NanPan
|
408b015d5f
|
fix(projects): drain trash reconciliation before cancellation returns (#5511) | ||
|
hataa
|
d811143b52
|
feat(authz): filter per-caller skill visibility on the skill listing surfaces (#4063 Phase 4) (#5489)
* feat(authz): filter per-caller skill visibility on the listing surfaces (#4063 Phase 4) GET /api/skills, GET /api/skills/custom, and GET /api/skills/{name} now filter the user-scoped catalog through filter_resources(principal, "skill", ...) — mirroring list_models. Anonymous callers are unfiltered; provider errors follow authorization.fail_closed (fail-closed -> empty listing / 404, fail-open -> full listing). An invisible skill on the detail surface returns the standard 404 so the endpoint cannot become an existence oracle the filtered list closed. Management endpoints stay require_admin_user-gated; runtime activation is #4541's layer. resolve_skill_authorization joins resolve_model_authorization as a thin sibling over a shared _resolve_route_scoped_authorization core. * docs(authz): reflect per-caller skill visibility in OpenAPI metadata and implementation notes (#5489) Address the two non-blocking review findings on #5489: - The three user-facing GET routes (/skills, /skills/custom, /skills/{name}) now say in their /docs-visible descriptions that authorization filters the response (hidden skills 404 on detail). - Add the dated Phase 4 decision-log entry to the authorization implementation notes, per the convention of every prior merged authz PR: listing-visibility semantics, the 404-vs-403 existence-oracle rationale, anonymous-caller behavior, and the #4541 rebase reconciliation points (config.example.yaml roles comment + this file's decision log). * docs(authz): move route guidance into Gateway module guide --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
FanouZeng-TT
|
22ae3d0e95
|
fix(backend): validate assistant search pagination (#5506)
Co-authored-by: Claude Code <noreply@anthropic.com> |
||
|
NanPan
|
6a94bef908
|
fix(gateway): keep run drain alive across repeated cancellation (#5487)
* test(gateway): cover repeated cancellation during run drain * fix(gateway): keep run drain alive across repeated cancellation * test(gateway): harden run-drain cancellation coverage |
||
|
xiaodu55
|
e493390aea
|
fix(gateway): scope edit/regenerate helper fallbacks by data identity (#5483)
The message edit/regenerate prepare chain resolves source runs through _resolve_run_id_for_message, _require_successful_source_run and _find_interrupted_target_run_id, which filtered by the authorization identity (get_current_user) — the same conflation fixed for the runs and messages read endpoints in #5448, left out of scope there. For trusted internal callers the authorization identity never matches the raw owner-stamped run rows, so regenerate/edit-regenerate prepare fails with 409 on threads the caller is authorized on (#5482). The three helpers now resolve their filter id through _run_scope_user_id as well; both prepare endpoints keep their owner_check=True authorization and browser/API sessions keep the per-user filter. Regression tests extend test_thread_runs_internal_scope.py to the helper fallback paths: internal callers resolve raw-owner-stamped runs (including the interrupted-run and status-409 paths), browser sessions keep the per-user filter and 409 on cross-user runs. |
||
|
alanhuangyoo
|
0f2195e994
|
fix(goal): wait for the user when a turn ends on an unanswered question (#5467)
* fix(goal): wait for the user when a turn ends on an unanswered question ask_clarification and the sandbox network prompt put their question in a ToolMessage and end the graph. The goal evaluator only reads human and AI text, so it never saw the question, judged the goal not met, and the worker queued a hidden continuation telling the agent to keep going while the card was still open. The agent could then act on a guess before the user answered. Stand the goal down with blocker needs_user_input, without calling the evaluator, when the trailing tool results include a human input request. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(goal): cover resuming after answered clarification --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Xuehao Xu
|
f0cb67b223
|
feat(extensions): expose incremental run evidence reader (#5405)
* feat(extensions): expose incremental run evidence reader * fix(extensions): address run evidence review feedback * docs(extensions): clarify run deletion reconciliation * docs(migrations): align current head documentation * fix(extensions): isolate run evidence event reads * test: avoid pinning run change migration to latest head --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Zeren Wang
|
a58ab484a6
|
feat(projects): Projects MVP Phase 2 — instructions, document shelf, promotion, trash (#5443)
* feat(projects): Projects MVP Phase 2 — instructions, document shelf, promotion, trash Implements docs/superpowers/specs/2026-09-12-projects-mvp-phase2-design.md (issue #5160, tracker #5129) in the slice order of the spec's §16. Slices: - A: ProjectsConfig + write-time 422 UTF-8 byte cap; PROJECT_CONTEXT_KEY admission pinning (both server-owned sets + worker hoist); latest-only request-scoped <project> block via DynamicContextMiddleware wrap_model_call/awrap_model_call (idempotent reassembly, reserved ID prefix + marker + provenance, never persisted); journal audit fingerprints; Instructions tab. - B: ProjectDocumentRow + migration 0023; ProjectDocumentRepository with locked check-and-set; hash-qualified immutable shelf storage with Paths helpers; upload/list/content/delete-to-trash routes; project delete trashes the shelf in-transaction; request-scoped bounded <documents> index with honest count/shown + actionable overflow note; list_project_documents/read_project_document tools registered only on pinned runs; PAT allowlist + drift guards; blocking-IO anchors. - C: shared thread-upload ingestion service (uploads router refactored to parity); POST from-thread with provenance; attach-to-thread with lock-staged copy (archived source allowed); read-only thread-files view with per-group truncation reporting. - D: restore (restored/merged/not_found/no_target/content_missing; no file moves), purge (continuous row lock across unlink/delete/commit, retryable on FS errors), retention sweep (lazy + startup, 24h orphan guard, row-side reconciliation never deletes). - E: Documents tab (shelf + conversation-files browser, provenance, archived banner, content-missing rows), /workspace/trash route, sidebar entry, composer attach handoff, i18n (en-US/zh-CN), e2e mocks + specs. Review hardening folded in (10 rounds, all with tests): - force active shelf content (HTML/XML family) to download; nosniff on artifact + content responses; unified unsandboxed-iframe PDF preview (fixes the pre-existing Chromium sandbox blank in the artifact viewer) - scope document trash to the URL project under the document lock - atomic no-overwrite filename reservation for ALL ingestion (seeded claims + os.link commit with suffix retry; same-name re-upload now unique-names instead of replacing); hidden staging only, no visible placeholders; lease cleanup on setup failure - serialize conversion under the document lock with post-lock active revalidation; drain locked filesystem work on cancellation; preserve bytes when an insert's commit state is uncertain (including trashed rows) - original-integrity checks before serving text or cached conversions; content_missing surfaced in list responses (UI reads the flag, no 409-probe); downloads always serve original bytes - bounded streaming document reads with cached char counts; shelf limits declared in middleware release identity - thread-root confinement for from-thread sources; config fallback rejects fractional/infinite values; composer counts staged attachments; pending attachments persist until submission or removal; in-flight instruction/rename edits survive save refetches; shelf and trash pagination; conversation-file and thread-files pages stay subscribed to refetches Docs: README/README_zh, backend API.md/ARCHITECTURE.md, AGENTS.md contracts, config.example.yaml projects block. Review follow-ups (head b4807477 → this revision): - The trash retention sweep is split so repeated lazy triggers stay bounded: the indexed expiry purge still runs on every trigger (GET /api/trash/documents, POST /api/trash/purge) while the O(all rows + all files) reconciliation is throttled to one run per user per 15 minutes (process-local, per-user window). The startup sweep now runs as a background task instead of blocking gateway readiness, and shutdown awaits it (bounded). - The export scrub (stripInternalMarkers) is fence- and indentation-aware like the render path, so a pasted, fenced <project>/<documents> snippet survives markdown export while real injected blocks (never fenced) are still removed. Fence regexes moved to a dependency-free leaf module to avoid the messages↔streamdown import cycle. - The artifact viewer's PDF iframe no longer carries an added title attribute (the upstream e2e contract locates it via :not([title])), and the upstream artifact-preview spec now pins the new contract: PDFs render unsandboxed, images keep sandbox="". * fix(projects): round-2 review — cancel an overrun trash sweep, restore the PDF frame title - Shutdown cancelled only the shield around the background startup sweep, so an all-users reconciliation that outlived the 5s budget kept walking rows and files while the document repo and DB engine were disposed underneath it. The wait now lives in `_shutdown_startup_trash_sweep`, which cancels the task and drains it before worker exit: the shield keeps the wait bounded, the cancel makes it final (CancelledError lands at the sweep's next await, and `_run_startup_trash_sweep` only catches `Exception`, so nothing swallows it). - The browser-preview iframe lost `title={getFileName(filepath)}` in the previous fix round, leaving the PDF frame without an accessible name while its siblings keep theirs. Restore it (WCAG frame titles), assert it in the DOM test, and anchor the e2e on `iframe[title="report.pdf"]` instead of `iframe:not([title])`. * fix(projects): round-3 review — report the sweep's late finish, not a phantom cancel `Task.cancel()` returns False when the sweep already finished inside the window between the deadline firing and the cancel, so the shutdown log claimed a cancellation that never happened. Branch on that outcome: the warning stays for a real cancel, a late finish is logged at info, and both paths still reap the task before worker exit. * fix(projects): round-4 review — make Empty trash delete what it confirms `POST /api/trash/purge` only ran the retention sweep, and the sweep's candidate selection is age-gated, so a freshly trashed document survived "Empty trash" even though the confirmation promises that every listed document is permanently deleted. With one trashed row the route answered `{"purged": 0}` and left it in place; `GET /api/trash/documents` sweeps expired rows before listing, so the visible rows were normally ineligible for the action by construction. Empty trash now drives `purge_all_trashed`: the caller's trashed rows (`list_all_trashed`, no age filter) each go through the same guarded, row-locked `purge` as the single-document delete — bytes first, then the row, in one transaction — so a row restored mid-flight is skipped instead of force-deleted, and an unlink failure rolls that row back and answers 500 with a retryable message. Retention expiry stays where it was: the sweep's `purge_candidates` is now the only age-gated selection, and the lazy retention sweep still runs on the listing and at startup. Tests: the router suite replaces the retention-gated expectation with the reviewer's repro (fresh row purged, bytes unlinked, shelf and other users' trash untouched, a failing unlink stays retryable and 500); a blocking-I/O anchor drives the new entry point through the offload; the mocked e2e covers the action end to end; a new real-backend spec performs it against the real gateway and re-reads `GET /api/trash/documents`. README, API, ARCHITECTURE and the phase-2 design docs (en+zh) state the age-independent contract. |
||
|
xiaodu55
|
0745fb268f
|
fix(gateway): scope runs read endpoints by data identity, not authorization identity (#5448)
* fix(gateway): scope runs read endpoints by data identity, not authorization identity Trusted internal callers are authorized as a synthetic internal user (id "default", or the make_safe_user_id-normalized owner with an owner header), while start_run stamps run rows with the raw trusted-owner value. list_runs / get_run / list_runs_page filtered by the authorization identity, so the store-side user filter never matched and internal callers always saw an empty runs list (or 404) for threads they are authorized to read. The three read endpoints now resolve their filter id through _run_scope_user_id: internal-role callers skip the per-user filter (thread visibility is already authorized by owner_check=True), and browser/API sessions keep the existing per-user filter unchanged. Regression tests cover both identities across the three endpoints (list, keyset page, single get) with a MemoryRunStore seeded with mixed-owner rows; without the fix the four internal-caller cases fail while the browser-session isolation case passes. * fix(gateway): route the message read endpoints through the same data-identity scoping Review follow-up on #5448: list_thread_messages and list_thread_messages_page resolved get_current_user and passed it as the data filter to the event-store scan, hidden-run lookups, turn-duration injection and the feedback queries — the same authorization-vs-data identity conflation fixed for the runs endpoints, leaving the #5437 empty-read symptom in place for lossy owner values. Both endpoints now resolve their filter id through _run_scope_user_id as well. Regression tests extend to the two message endpoints, asserting the resolved filter identity at the runs-store and feedback-repo boundaries (None for internal callers, the session user id for browser sessions). * fix(feedback): deterministic per-run collapse for unfiltered feedback reads Review follow-up on #5448: with _run_scope_user_id returning None for internal callers, the feedback lookups now receive an explicit-None user id, which skips the user_id WHERE in FeedbackRepository. On shared/NULL-owner threads several browser users can hold feedback on the same run, and list_by_thread_grouped / list_by_run_ids collapsed rows per run_id via a dict comprehension over unordered results — the feedback attached to the last AI message would be an arbitrary user's row. Both methods now order by created_at ASC with feedback_id as the tie-break, so the collapse deterministically keeps the most recently created feedback. _run_scope_user_id's docstring now documents that the resolved id also scopes feedback and event-store reads, not just run rows. Regression test seeds multi-user feedback on one run and asserts the collapse outcome is stable across repeated unfiltered reads. * docs(feedback): the collapse keeps the most recently written feedback created_at is refreshed on upsert, so the surviving row per run is the most recently written (created or updated), not the most recently created — align both docstrings with the ordering key's actual semantics. |
||
|
Totoro
|
5b591a9039
|
feat(gateway): accept conversation references in run context and report the capability (#5463)
* feat(gateway): accept conversation references in run context and report the capability LangGraph SDK clients build a fixed run body and drop unknown top-level fields, so they cannot send the conversation_references field from #5399. RunCreateRequest now lifts context.conversation_references into the top-level field before validation, so it keeps the same bounds and error locations, and drops it from context, so it never reaches the merged run context or the checkpointed configurable. Sending both is a 422. GET /api/features reports conversation_references {enabled, max_references} with the same "tool is configured" predicate as run admission, so a client can hide an entry point on deployments without the tool. Related to #5398. * fix(gateway): report the field type error for a malformed top-level reference list A malformed top-level conversation_references sent alongside a context list now fails with the field's own type error instead of the conflict message. The tool-configured predicate reads tool.use directly, and the features test doubles carry that attribute like every real ToolConfig. * fix(gateway): treat every list-like top-level reference value as a conflict Pydantic's lax mode coerces tuples, sets, frozensets and deques into the list[str] field, so a direct Python caller passing one of those together with context.conversation_references now reports the conflict instead of slipping both grants through. Unreachable over HTTP, where JSON has no such types. * fix(gateway): ask pydantic whether a top-level reference value is list-like Enumerating list-like types cannot track pydantic's lax acceptance set (generators, UserList, dict key views also coerce into list[str]). The conflict guard now validates the top-level value with a TypeAdapter for list[Any]: whatever pydantic would coerce reports the conflict when it is non-empty, and whatever it rejects still surfaces the field's own type error. Regression tests cover deque, UserList, dict keys and a generator, plus rejected scalars. * fix(gateway): probe top-level references with the field's own annotation The conflict guard now validates the top-level value with the exact item annotation the field uses, so its acceptance set is the field's rather than a superset: an item the field rejects (an empty string, a non-string, the ints of a range or dict view) surfaces the field's own item error instead of a conflict. The annotation is shared through one alias so the two cannot drift. * fix(gateway): materialise a one-shot iterator before probing top-level references The item-validating probe could consume a generator while collecting an item error, after which the field re-validated the exhausted iterator, coerced it to [] and let the request through with the key still in context. Iterators are now read once into a list that both the probe and the field validate, so a bad item is reported at its index and a valid generator is kept. * fix(gateway): materialise every once-walkable iterable before probing references Pydantic coerces any iterable into the list field, and an object whose __iter__ hands out a generator once is not an Iterator instance, so the previous gate let it reach the probe and be consumed. The lift now reads every iterable except lists, tuples and the shapes the field rejects as a whole (str, bytes, dict) into a list first, so the probe and the field always validate the same items. --------- Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com> |
||
|
PeaceMaker-best
|
14c9d44440
|
feat(runtime): persist tool-progress phase transitions (#5214)
* feat(runtime): persist tool-progress phase transitions Record bounded warn, block, and recover decisions for lead and task subagent runs while preserving event-loop isolation, fail-open behavior, and concurrent transition order. * fix(runtime): trust server-owned tool progress attribution * fix(runtime): centralize trusted audit attribution * fix(runtime): preserve complete tool progress audit state * docs: trim tool progress guidance to pass size check * fix(runtime): fence subagent audit recorder loop * docs(readme): sync tool-progress event coverage Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> --------- Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> Co-authored-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> Co-authored-by: 嗜鵼 <hy2010hy2010@qq.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Totoro
|
4ad55f598f
|
feat(conversation): continue reading a cut message by offset (#5434)
* feat(conversation): continue reading a cut message by offset A referenced message longer than 4,000 characters was cut, and its suffix could not be read back. Cut messages now carry a continuation (message_seq, offset). read_conversation(thread_id, message_seq, offset) returns the next part of that one message, sized to the same tool-output budget as pages. The read scans only the requested row under the existing visibility rules and rechecks ownership. Offsets follow the source's current text; an offset past the end is rejected. Related to #5398. * docs(conversation): say continuations ignore limit A continuation always returns one part of one message, so limit does not apply there. The tool schema now says so instead of discarding it silently. Related to #5398. * fix(conversation): stop instead of looping when no text fits the budget With a read_conversation tool-output budget below the envelope size, the fitted text was empty and the continuation repeated the requested offset, so an agent would repeat the identical call forever. Page and continuation reads now return output_budget_too_small with no continuation. Related to #5398. --------- Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com> |
||
|
Beautyl0ve
|
80f13935c2
|
feat(agents): allow custom agents to disable memory (#5167)
* feat(agents): allow custom agents to disable memory Signed-off-by: Beautyl0ve <74452755+Beautyl0ve@users.noreply.github.com> * fix(agents): honor memory opt-out during compaction Signed-off-by: Beautyl0ve <74452755+Beautyl0ve@users.noreply.github.com> * fix(runtime): preserve agent binding across state rewrites * fix(client): apply named-agent memory policy * fix(agents): address memory policy review feedback Signed-off-by: Beautyl0ve <74452755+Beautyl0ve@users.noreply.github.com> * fix(agents): address remaining memory opt-out reviews Signed-off-by: Beautyl0ve <74452755+Beautyl0ve@users.noreply.github.com> --------- Signed-off-by: Beautyl0ve <74452755+Beautyl0ve@users.noreply.github.com> Co-authored-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
alanhuangyoo
|
0a0d768107
|
fix(goal): stand the goal down once the run has hit its token budget (#5424)
* fix(goal): stand the goal down once the run has hit its token budget Since #5410 goal continuations share the run's token budget, so a continuation queued after the budget's hard stop only spends one more model call before its tool calls are stripped. Pass the run's stop_reason into the goal loop and stand the goal down with "token_capped" in that case. The evaluator still runs first, so a goal the capped run satisfied is cleared as before. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(goal): list the token-budget stop among goal-loop preconditions Also pin that a satisfied goal is cleared, not stood down, when the run hit its token budget. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Totoro
|
6177b07c06
|
fix(conversation): clarify reference semantics and keep reader pages inline (#5421)
Document that read permission expiry and source deletion do not erase text already copied into the destination conversation, and that reads follow the source's current visible history. Truncated results now tell the agent to acknowledge the omission and ask for the missing material before claiming every requirement is covered. Pages were filled to 20,000 text characters by cutting the last message that did not fit, and that suffix could never be paged back. They could also exceed the default 12,000-character tool-output budget, which externalized the page to a file. Pages are now sized by their serialized length against the read_conversation tool-output budget; a message that does not fit starts the next page intact, so only a message over 4,000 characters (or one whose escaped JSON alone exceeds the budget) is cut. Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
tiammomo
|
5d855e9b92
|
feat(scheduled-tasks): filter run history by occurrence status (#5384)
* feat(scheduled-tasks): filter run history by occurrence status Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> * fix(scheduled-tasks): share occurrence status contract --------- Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Totoro
|
533e30e7f2
|
[feat] add opt-in conversation reads for Gateway runs (#5399)
* feat: add scoped conversation reads to gateway runs * refactor: share conversation tool path and reuse parsed text --------- Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com> |
||
|
Hyeonsang Cho
|
bbced51022
|
fix(gateway): close two input-sanitization bypasses (#5375)
* fix(gateway): reject forged framework-injection markers in run input
`is_genuine_user_message` treats `hide_from_ui` and a human `name="summary"`
as proof the framework authored a message, and `InputSanitizationMiddleware`
skips those — escaping a real reminder's blocks would corrupt trusted context.
Neither marker was server-owned, so an external caller could set either one and
place a raw `<system-reminder>` outside the user-input boundary markers, which
the lead-agent system prompt declares trusted internal framework data. The
`hide_from_ui` variant is also filtered out of the thread UI, so the forgery
was invisible where it landed.
Both markers are now stripped from untrusted input, on the run path and on the
thread-state mutation route that writes straight into a checkpoint. Framework
injection happens inside the graph and never crosses this boundary, so nothing
the framework does is affected, and `trusted_internal` callers (IM channels,
the MCP task-notification launcher) keep writing hidden messages.
HumanInputCard replies are the one legitimate external `hide_from_ui`: the
frontend sends it alongside a `human_input_response` payload, so a message
carrying a valid one keeps the marker. That buys no bypass — the predicate
already classifies those as genuine, so they stay sanitized.
The name check reuses the predicate's own `_SUMMARY_MESSAGE_NAME` rather than a
fourth copy of the literal, and matches by `isinstance` exactly as the predicate
does: `HumanMessageChunk` is a `HumanMessage` whose `type` is not `"human"`, so
a type-based check would leave that subclass's marker settable. `name` is only
reserved on human messages — on a ToolMessage it is the tool's own name.
Three tests in test_gateway_services.py and test_message_provenance.py asserted
that a caller-supplied `hide_from_ui` survives. That assumption was the bypass;
they now pin the opposite, with a genuinely caller-owned key kept alongside to
prove the stripper is surgical.
* fix(agents): sanitize every genuine user message, not only the newest
The input guardrail scanned backwards for the first genuine user message and
returned, so only the newest turn was ever sanitized. The transformation is
request-scoped (`wrap_model_call`, never written to state), so thread state
keeps the raw text: once a newer turn arrived, the previous turn's payload was
replayed to the model verbatim, outside the boundary markers the lead-agent
prompt declares trusted framework data. The guardrail therefore held for
exactly one model call.
Reaching it needed no forged metadata and no crafted request body — type the
payload in one turn, then send anything at all in the next. A single request
carrying two user messages did it in one shot, since every message but the last
was skipped.
`_process_request` now walks the whole list and `_sanitize_message` owns the
per-message work; every existing branch (the `original_user_content` split for
upload turns, the multimodal rfind fallback, the metadata repair) is unchanged.
Framework-injected messages stay excluded by `is_genuine_user_message`.
Unexpected errors now fail open per message rather than per request. Iterating
history widened the old blast radius: one unprocessable row would have dropped
sanitization for the whole request, handing an attacker the newest turn by
crafting an older one. `GraphBubbleUp` still propagates.
Side effect worth noting: each turn's rendering is now stable across model
calls. Previously a turn was wrapped on its own call and unwrapped on the next,
changing the prompt prefix behind the newest turn and defeating prompt caching.
test_only_processes_last_user_message pinned the old scope; it now pins that
every turn is processed, and keeps driving the `wrap_model_call` entry point.
* docs: record the message-metadata trust boundary and sanitization scope
`agents/middlewares/AGENTS.md` owns the depth for InputSanitizationMiddleware
and documented only the `original_user_content` half of its trust boundary. Left
alone it would teach an agent that `hide_from_ui` is caller-owned and that the
guardrail covers one turn — and the usual failure mode is an agent "restoring"
the behaviour it believes was lost. The entry now carries both markers, the
HumanInputCard exception, the whole-history scope, and the per-message fail-open
rule.
The note lives only there. The root and `backend/AGENTS.md` layers are
orientation that points at the module guides owning the depth, and
`backend/AGENTS.md` is inherited by every backend chain — prose added there
inflates more than twenty of them, and `scripts/check_agent_guidance.py` shows
the middlewares chain has about a kilobyte of room against its hard limit.
CHANGELOG.md and CHANGELOG_zh.md record it under Security, continuing the
existing prompt-injection lineage.
* fix(gateway): mark caller-hidden messages instead of stripping the marker
Review follow-up. Stripping a caller-supplied `hide_from_ui` closed the bypass
but broke three frontend senders that use the marker purely to keep a context
message out of the transcript: the quoted conversation context
(`buildHiddenConversationQuoteMessage`), the sidecar context prompt
(`buildHiddenSidecarContextMessage`), and the agent save command. None carries a
`human_input_response`, so the HumanInputCard carve-out did not cover them, and
nothing else hides them — `_is_branch_visible_message` and the frontend's
`isHiddenFromUIMessage` both key solely on `hide_from_ui`, and no backend reads
`conversation_quote_context` or `sidecar_context`. All three would have rendered
as user-visible chat bubbles.
The marker plays two roles and only one of them is a vulnerability. The security
requirement is that a caller-supplied marker cannot skip sanitization, not that
it cannot hide a message. So the roles are separated instead of the marker being
removed: the Gateway keeps it and stamps the server-owned `UNTRUSTED_INPUT_KEY`,
and the guardrail now asks `requires_input_sanitization` — the mark, else the
genuine-user test. Hidden stays hidden; untrusted content is sanitized either
way. The reserved `summary` name is handled the same way and no longer rewritten.
Marking rather than removing is also the safer shape in general: `hide_from_ui`
is read for presentation, journal persistence, memory filtering and IM outbound
as well, and this boundary should not silently change any of them.
`is_genuine_user_message` is deliberately left alone. `ToolReceiptMiddleware`
uses it for turn-boundary detection, where a caller's hidden context message must
keep counting as not user-authored; widening it there would move the ledger's
turn window. `requires_input_sanitization` sits beside it in `message_utils` so
the two questions can be compared.
The three tests that asserted a caller-supplied `hide_from_ui` is removed now
assert it survives and carries the mark — which restores the original intent of
the two provenance cases, whose comment already read "caller-owned keys must
survive".
* docs(gateway): rewrite normalize_input's docstring around the mark
Review follow-up. The paragraph still described the pre-4a3344f6 strip model and
contradicted both the implementation and the middlewares AGENTS.md paragraph
updated in that same commit: it called `hide_from_ui` server-owned, said
carrying it skips sanitization entirely, and repeated the premise this branch
disproved — that HumanInputCard replies are the only legitimate external use.
It now describes what the code does: the markers stay caller-owned and are
preserved because three frontend senders rely on `hide_from_ui` for hiding
alone, the message is stamped with `untrusted_input` instead, and
`requires_input_sanitization` sanitizes it anyway. `untrusted_input` joins the
server-owned inventory in the preceding paragraph, which is what makes the stamp
unforgeable and unclearable.
The three surrounding docstrings now also say that these functions mark as well
as strip; `_strip_external_message_metadata` had advertised only the removal,
leaving a reader no way to find the stamp from the name.
* fix(gateway): mark state writes whose message omits additional_kwargs
Review follow-up. The state-route half of the fix missed the most natural
request shape. `_strip_external_metadata_from_message_like` returned early when
`additional_kwargs` was absent or not a dict — there was nothing to strip — and
that early return also skipped the mark. A `POST /threads/{id}/state` body of
`{"values": {"messages": [{"role": "user", "name": "summary", "content":
"<system-reminder>…</system-reminder>"}]}}` therefore reached the checkpoint
unmarked. The messages reducer's `convert_to_messages` then supplies
`additional_kwargs={}`, so at model-call time `requires_input_sanitization` fell
back to `is_genuine_user_message`, which a `summary` name fails, and the forged
tag reached the model raw and outside the boundary markers.
A missing or non-dict `additional_kwargs` is now treated as empty for both the
strip and the mark. The identity return is kept for the case where nothing
changes, so an ordinary key-omitted message does not gain an empty dict just by
passing through. The run path was never affected: `normalize_input` coerces to
BaseMessage first, which always carries the dict.
Every existing state-write test supplied an `additional_kwargs` dict, which is
why this shape slipped through; the regression now covers it at the route and
end to end through the reducer into the guardrail.
While checking the neighbouring shapes, `_skips_input_guardrail` keyed off key
presence where `is_genuine_user_message` keys off truthiness, so
`hide_from_ui: False` — already covered without a mark — would have been
stamped. It now mirrors the predicate exactly, as its docstring claimed.
* docs(middlewares): compress the sanitization note to fit the guidance chain
main's growth left the middlewares AGENTS.md chain 84 bytes under its hard
limit, and the fuller wording did not fit. The load-bearing facts stay — the
markers are marked rather than stripped, and the scan covers every turn — since
those are the two an agent editing this middleware could otherwise get wrong.
The full model lives in the normalize_input, _mark_untrusted_framework_markers
and requires_input_sanitization docstrings.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|
||
|
yang rui
|
7513f16e0e
|
feat(settings): persist account preferences across browsers (#5397)
* feat(settings): persist account preferences across browsers * docs(settings): scope preference guidance to user persistence * fix(settings): preserve SSR and fence custom-agent defaults * test: include user persistence in scoped guidance inventory * fix(settings): sync explicit edits and preserve local tab updates |
||
|
Xuehao Xu
|
56540fab01
|
fix(gateway): make recursion limit configurable (#5390)
* fix(gateway): make recursion limit configurable * docs: keep backend guidance within inherited size budget * fix(gateway): address recursion limit review feedback * docs(gateway): clarify recursion default scope --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Zhengcy05
|
81f2015fe6
|
fix(runtime): keep agent construction off event loop (#5217)
* fix(runtime): keep agent construction off event loop * fix: - offload checkpoint state accessor graph construction to a worker thread - update test * import AsyncKeyedLockTable * update Agents.md * fix: update test * fix: preserve single-flight builds after cancellation --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Wenchao An
|
1b76ab9060
|
feat: add opt-in task notes and compacted history recall (#5382)
* feat: add opt-in task notes and compacted history recall * fix: validate task continuity state and preserve user answers Honor explicit opt-out, preserve clarification replies and capture failure statuses, validate notebook writes, and clear branch archive references. Update the config version and audit optional LLM credentials, with regression and integration evidence. * fix: align Helm config version with task continuity schema * fix: preserve mixed task history and declare continuity policies * fix: recover malformed history and evict archives atomically |
||
|
tiammomo
|
444bfb72ce
|
feat(scheduled-tasks): preview upcoming cron occurrences (#5381)
Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> |
||
|
Hyeonsang Cho
|
bc4a33aba7
|
fix(skills): stop persisting resolved secrets when toggling skills (#5357)
Toggling a skill wrote resolved secrets into extensions_config.json. The Gateway skill toggle and DeerFlowClient.update_skill loaded the file with ExtensionsConfig.from_file(), which replaces every "$VAR" string with the environment value (and an unset variable with ""), then serialized that model back through to_file_dict(). A "$GITHUB_TOKEN" reference was persisted as the plaintext token, and an unset reference was erased for good. DeerFlowClient.update_mcp_config had the same flaw for every key other than mcpServers. Every writer now does a raw read-modify-write, the way the MCP router already did: read_raw_extensions_config reads the on-disk JSON, set_raw_skill_enabled changes only the target entry, and validate_raw_extensions_config checks the candidate the way the runtime will load it before the atomic write. When the file does not exist yet, the Gateway seeds it with the cached skill states only, never the resolved cached model. The MCP router's raw loader and candidate validation delegate to the same helpers, so the rule lives in one place, and to_file_dict() is removed so the unsafe serialization has no entry point left. Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
yang rui
|
ac2b6415ea
|
feat(agents): support Unicode display names for custom agents (#5324)
* feat(agents): support Unicode display names for custom agents * fix(agents): preserve and validate Unicode display names * fix(agents): tolerate invalid stored labels and reject invisible names * fix(agents): identify agent in invalid display name warning * style(frontend): format agent display name fallback --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Hyeonsang Cho
|
806a5bd427
|
fix(gateway): serve XML artifacts as attachments to block same-origin script (#5353)
* fix(gateway): serve XML artifacts as attachments to block same-origin script
GET /api/threads/{id}/artifacts/{path} forced only text/html,
application/xhtml+xml and image/svg+xml to download. Every other XML
document was served inline from the application origin: `.xml` guesses
to text/xml or application/xml depending on the host's mime.types, and
both fell through to the inline text branches. Browsers render any XML
MIME type as a document and run an XHTML-namespaced <script> inside it,
so a report.xml written by a prompt-injected agent and opened from a
chat link executed with the viewer's session: the HttpOnly access_token
rides same-origin fetches, and the double-submit csrf_token cookie is
JS-readable, so state-changing calls are reachable as well.
Treat HTML plus every WHATWG XML MIME type (text/xml, application/xml,
any +xml subtype) and text/xsl, which Blink also renders as XML, as
active content. A single helper owns the rule for both the regular-file
and the .skill-archive-member branches. The artifacts panel already
previews .xml as code through a ranged fetch, so preview and editing
keep working against the attachment response.
* docs(frontend): name XML among the artifacts the Gateway downloads
Review follow-up on #5353: resolveArtifactOpenURL's comment still named
only HTML/SVG as the active content the Gateway serves as a download.
XML documents now join that bucket, so the frontend note matches the
Gateway rule. Comment-only; no behavior change.
|
||
|
Shxiao
|
572744975d
|
fix(tools): run tool assembly off-loop at async entry points (#5224)
* fix(tools): run tool assembly off-loop at async entry points get_available_tools() may block on MCP cache initialization while it is called on async agent-assembly paths (task_tool, durable batch execution), stalling the calling event loop for the full discovery duration. Dispatch the (unchanged, synchronous) assembly call to a worker thread via asyncio.to_thread at the two async entry points so the loop keeps processing requests, SSE frames, cancellations, and timers. Fixes #5172 * fix(tools): offload lead-agent assembly off-loop and pin with blocking-io anchors Review follow-up for #5224: - run_agent now dispatches agent_factory(...) through asyncio.to_thread, so lead-agent assembly (including both get_available_tools call sites in _assemble_lead_agent) runs off the event loop — the Gateway headline scenario from issue #5172. - _ensure_sync_invocable_tool takes a double-checked threading.Lock, making the in-place tool.func wrap on the shared tool singletons explicitly single-shot now that assembly can run concurrently on worker threads. - Add backend/tests/blocking_io/test_tool_assembly_offloop.py: blocking-probe anchors for task_tool and SubagentBatchService._execute_item under the strict Blockbuster gate, plus a meta-check proving the gate trips on the exact syscall class (ExtensionsConfig.from_file on the loop). Verified the anchor goes red when the offload is flattened back to a plain call. * fix(gateway): build checkpoint state accessor off-loop; anchor run_agent offload Review follow-up for #5224: - Add abuild_checkpoint_state_accessor (asyncio.to_thread around the unchanged sync builder) and switch every async call site to it: the stateless_wait route, thread_runs, both threads call sites, and the build_thread_checkpoint_state_accessor boundary. The agent-factory assembly re-enters get_available_tools() and may block on MCP cache initialization; repeat calls hit _state_accessor_graph_cache and only pay the thread hop. - Add a third blocking-io anchor driving the real run_agent with minimal RunManager/bridge stubs; the factory performs a real production blocking read (ExtensionsConfig.from_file()) and the test asserts assembly never runs on the main thread. Verified the anchor goes red when the run_agent offload is flattened back to a plain call. - Adapt the test_threads_router checkpoint-builder patch sites to the new async name. * refactor(tools): carry assembly offloads on a dedicated bounded pool Review follow-up for #5224: - Add utils/assembly_io.py: a dedicated ThreadPoolExecutor (default 8 workers, DEER_FLOW_ASSEMBLY_WORKERS-overridable, mirroring utils/file_io.py and tools/sync.py) with run_assembly(), which copies contextvars explicitly. A hung stdio MCP server parks its worker for the full MCP timeout; carrying assembly hops on the loop's default executor would let a few parked assemblies queue every other to_thread/run_in_executor(None, ...) caller behind them. - Switch all four offloads (run_agent, task_tool, batch _execute_item, abuild_checkpoint_state_accessor) to run_assembly(). - State the cold-path behavior in the accessor docstring: the graph cache validates factory identity, so non-identity-stable factories may duplicate lead-agent assembly across concurrent readers (MCP discovery stays process-wide single-flight); the pool bounds the duplicates. - Add a fourth blocking-io anchor driving build_thread_checkpoint_state_ accessor with a per-resolution fresh factory (always a cache miss) and the real production blocking read; enumerate all four offloads in the gate's module docstring. Verified the anchor goes red when abuild_checkpoint_state_accessor is flattened back to a plain call. * fix(subagents): revalidate batch item before launch; make assembly pool observable Review follow-up for #5224: - _execute_item() revalidates the durable state right after assembly and before executor.execute_async(): renew_item_lease() returns valid=False when cancel_batch() terminalized the item or the lease was lost while assembly was parked, and the launch is skipped (the canceller already finalized the item). Previously the launch was unconditional and the poll loop's cancellation checks only started after execution began. - Regression test driving the real SQLite repository: a blocking assembly probe parks _execute_item, cancel_batch() lands, and the launch is skipped with the item staying cancelled. Verified the test goes red when the revalidation is removed. - run_assembly() tracks pending assemblies and logs a throttled WARNING once the pending count exceeds the worker count, so assembly starvation (workers parked on a hung MCP server) is distinguishable from idle. - The run_agent blocking-io anchor now binds a sentinel extension snapshot via ctx.extensions and asserts the factory observed it through get_agent_build_extensions(), pinning run_assembly()'s ContextVar propagation. Verified red when ctx.run is dropped. - Document the assembly pool in backend/AGENTS.md. * fix(utils): decrement the assembly pending count on the pool thread The pending-assembly counter behind the starvation warning decremented from the asyncio future's done callback, which never fires once the submitting loop is closed while its worker is still running: the count ratcheted up permanently and eventually fired the starvation warning with no starvation behind it (reproduced at 97dc9bec by review). Decrement instead from the dispatched work item: run_assembly() wraps func so a finally drops the count under the pending lock on the pool thread, and the done callback is gone. Pin the counter with tests/test_assembly_io.py: a healthy call returns the count to zero, and an abandoned loop (stopped while the worker is parked) does not wedge it — the abandoned case goes red against the old done-callback decrement. * docs(utils): fix the pending-counter comment after the decrement move The comment still described the removed done-callback decrement, contradicting _work()'s own comment; state the actual mechanism (increment on the loop before dispatch, decrement from the dispatched work item's finally on a pool thread). * test(gateway): retarget checkpoint-accessor stubs to the services seam thread_runs and runs now call abuild_checkpoint_state_accessor, so the upstream wait-reader, regenerate-prepare, and idempotency tests must stub the sync builder where abuild resolves it (app.gateway.services); stubbing the removed router re-exports fails with AttributeError at setup. The async seam semantics are unchanged: run_assembly invokes the stubbed sync builder off-loop and propagates its return values and exceptions. Move the agent/tool assembly off-load note from backend/AGENTS.md to deerflow/utils/AGENTS.md (next to assembly_io.py) so the effective instruction chain for agents/middlewares no longer grows past the AG002 hard limit. * fix(runtime): serialize same-key accessor assembly and release queued-cancel slots Address the three review follow-ups on the assembly off-load: - assembly_io: a job cancelled while still queued never runs its work item, so the dispatched finally never fired and _pending_assemblies stayed elevated until a false starvation warning. Exactly-once cleanup now rides the concurrent future's cancelled() state — cancel() only succeeds before the executor starts the item, so cancelled() is true precisely when the finally will never run — plus a submit-failure release; the one-worker queued-cancellation case is pinned red/green. - services: overlapping cold readers sharing one cache key could both run full agent assembly. _state_accessor_graph now serializes per key through a thread-side KeyedLockTable (pool threads, no running loop) and re-validates factory/app-config identity under the lock, so the factory runs exactly once while identity changes still rebuild. Cache dict access is lock-guarded now that construction runs off-loop. - guidance inventory: register deerflow/utils/AGENTS.md in EXPECTED_GUIDANCE_PATHS so test_repository_has_the_approved_scoped_ guidance_shape matches the relocated assembly note (CI shard 4). * test(keyed-lock): pin KeyedLockTable reclamation and waiter bypass directly Thread-side counterparts of the async table's own tests: overlapping hold() calls serialize (a late arrival joins the live entry instead of creating a second lock that bypasses a queued waiter), the last check-in pops the entry, and many unique keys leave the registry empty. Both regressions verified red — popping unconditionally trips the late-arrival test, never reclaiming trips the many-keys test. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Totoro
|
cd0e74edaf
|
fix(scheduler): reconcile stuck once tasks from committed run outcome (#5035)
* fix(scheduler): reconcile stuck once tasks from committed run outcome Restart recovery (cancel_stuck_once_tasks and the multi-instance reconcile_stuck_once_tasks) blindly flipped every stuck once-task to 'cancelled'. When handle_run_completion crashed between its two transactions, a once-task whose run had already committed 'success' was permanently reported as cancelled. Both reconciliation paths now read the latest scheduled_task_runs row without a status filter and finalize the parent to match: success -> completed (last_error cleared), failed -> failed with the run's error, interrupted -> cancelled with the run's error when present, skipped -> cancelled (no work performed). Active occurrences (queued/launching/running) are left untouched — a concurrent completion or a later recovery pass will finalize them once the run reaches a terminal state. Tasks without a terminal run row keep the previous generic cancellation. Review follow-ups (willem-bd / Huixin615): - Extract _finalise_once_task_from_run() so both recovery paths share one outcome mapping (no more drift between single- and multi-instance paths). Returns bool (True = finalised, False = active/no-op) for explicit counter management at call sites. - Fix a no-op (`run_row.error or None` -> `run_row.error`) in the skipped branch. - Drop the unused `status` parameter from the test task helpers. - Use TERMINAL_RUN_STATUSES / ACTIVE_RUN_STATUSES constants (local copies to avoid circular import; kept in sync with scheduled_task_runs.sql). - [P1] Read the latest run AFTER acquiring the parent task row lock, not from a pre-lock batch snapshot. The latest-run lookup now runs per task under the lock with populate_existing so a concurrently committed status is read back fresh. - [P2] Race tests now use monkeypatch to actually enter the race window: _intercepted_fetch commits success in a separate session at the moment the per-task fetch fires, so a reverted pre-lock batch implementation fails the test, while the current post-lock implementation passes. - [P1] Do not finalize parent for active occurrences. A non-terminal scheduled occurrence means the run is still in progress — the parent must be left untouched until the completion path or a later recovery pass establishes a terminal outcome. - [P2] Add cancel_stuck_once_tasks to the single-instance poll loop so stuck once-tasks are not left permanently "running" when the startup sweep fails (mirrors multi-instance _reconcile_active_state behavior). - Fix stale docstrings in cancel_stuck_once_tasks and _fetch_latest_run. Adds regression tests for multiple historical runs (older success + newer skipped/active) on both paths, monkeypatch-based race tests that prove a concurrent completion committing success is reflected as completed, and active-run tests that verify the parent is left unchanged. Documents the behavior in AGENTS.md. Fixes #5034 * fix(scheduler): address review comments on completion-consistency fix - _fetch_latest_run: drop arbitrary id DESC tie-break; order by scheduled_for DESC (deterministic recency on schedule position) - _finalise_once_task_from_run: annotate bool return type - Centralize TERMINAL/ACTIVE_RUN_STATUSES in scheduled_tasks/model.py; stop duplicating them in scheduled_tasks/sql.py and scheduled_task_runs/sql.py (removes stale circular-import workaround) - cancel_stuck_once_tasks: run unconditionally in single-instance poll loop (remove try/except swallow) - tests: pin created_at/scheduled_for in _create_run so recency ordering is actually exercised; correct docstrings that described the active-occurrence branch as 'generic cancel' instead of 'left unchanged' * fix(scheduler): correct finalizer return annotation * fix: order scheduled task runs by creation time * fix(scheduler): stabilize latest run reconciliation ordering * fix(scheduler): order latest runs by creation time * test: update trace scheduler stub * fix(scheduler): clarify reconciliation diagnostics Signed-off-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com> * fix(scheduler): fail closed on startup recovery Keep single-instance parent reconciliation at startup so it cannot race manual admission. Propagate recovery failures through the Gateway lifespan before channel startup, preventing a half-started scheduler. Tests cover both recovery failure stages and a queued occurrence that survives startup before the ordinary poll drain launches it. Signed-off-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com> * fix(scheduler): order occurrences and fence stale parent writes Allocate per-task occurrence sequences under the parent lock and guard parent projection across launch, recovery, completion, and queue failure paths. Track launch accounting separately so stale occurrences are counted once without replacing newer results. Commit completion and accounting atomically, preserve legacy history, and cover migrations and reordered execution on SQLite and PostgreSQL. * fix(scheduler): tighten completion projection and launch fencing diagnostics Share the once-task outcome mapping between completion and both recovery paths, validate the terminal status before opening the completion transaction, leave cron parent status untouched on completion, log the fenced launch update when an occurrence does not belong to the launched run, and drop the README capability line. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(scheduler): compare caller time only against unsequenced occurrences Among sequenced rows the parent-locked occurrence_seq is the only recency key. An unsequenced row can only be legacy history or an admission by a pre-upgrade Gateway writer, so recovery prefers it over the sequence winner only when its caller timestamp is later, which is the previous ordering for that pair. A rolling upgrade therefore degrades to the pre-sequence behaviour instead of ranking every pre-upgrade admission below every sequenced one. Document that boundary instead of requiring every Gateway writer to stop before the upgrade. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(scheduler): gate once-task recovery on the same projection rule Recovery now finalises a once-task parent only from the occurrence that can_project() accepts: the highest sequenced occurrence whenever one exists, or the timestamp-latest row for a task whose history is entirely unsequenced. An unsequenced row admitted by a pre-upgrade writer can no longer cancel a parent whose sequenced occurrence is still live, nor stall finalisation of a parent whose sequenced occurrence already completed. Document that pre-upgrade instances project their own admissions during a rolling upgrade. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(scheduler): defer once-task recovery while any occurrence is live uq_scheduled_task_run_active allows one non-terminal occurrence per task, so a live row is the task's newest admission whatever its caller clock and whether it carries a sequence. Both once-task recovery paths now probe for any active occurrence after the fresh latest-run read and leave the parent untouched while one exists; cancel_stuck_once_tasks also locks the parent row so admission cannot insert a queued occurrence between that probe and the commit. Once no occurrence is live, the sequence winner decides and a terminalised unsequenced row never overrides it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(persistence): follow the local head past canonical 0019 Main's forward-revision tests assumed 0019_thread_incarnations was the local chain head. With 0022_scheduled_occurrence_seq chained after it, seed the canonical-0019 shape explicitly, assert the real head where a database is upgraded, derive the 0020 rollback binary's revision set from the ancestors of its head, and step the PostgreSQL restart scenario back to canonical 0019 before the rollback binary restarts. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(migrations): describe the chain through 0022_scheduled_occurrence_seq The rolling-forward section still ended the local chain at canonical 0019; it now names 0022_scheduled_occurrence_seq as the head and lists it among the revisions the 0020 rollback-floor binary does not know. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(scheduler): accept CI's sync Postgres URL in occurrence fixtures CI hands over TEST_POSTGRES_URI as postgresql://...?sslmode=disable. The occurrence, ordering and 0022 migration fixtures built async engines from it directly, so SQLAlchemy chose psycopg2, which is not installed. Normalize the scheme to postgresql+asyncpg and drop libpq-only query keys, matching the existing 0019 migration tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(scheduler): keep the backend AGENTS.md chain within its budget The middlewares guidance chain was already above the hard limit on main, so any added byte in backend/AGENTS.md fails the agent guidance check. Leave backend/AGENTS.md identical to main and record the recovery projection rule in the 0022 migration entry, which already describes the occurrence fields. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Signed-off-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com> Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Ryker_Feng
|
f52818fe5e
|
feat(skills): export custom skill packages with revision-bound preview (#5332)
* feat(skills): export custom skill packages with revision preview * docs(gateway): keep export guidance within size budget * ci: retry checks after transient uv setup download failure * docs: focus skill export agent guidance on maintenance invariants * fix(skills): handle export disconnects and bound archive transfers * docs(gateway): remove redundant export guidance to fit merged budget * fix(skills): reset export idle deadline after transfer progress |
||
|
Hyeonsang Cho
|
556975f284
|
fix(gateway): gate github_token and disable_clarification on internal callers (#5338)
* fix(gateway): gate github_token and disable_clarification on internal callers `non_interactive` is honored only for internally-authenticated callers because it strips `ask_clarification` from the lead-agent toolset. The two sibling run-context keys reproduced that effect without the gate. `merge_run_context_overrides` forwarded `_CONTEXT_RUNTIME_ONLY_KEYS` regardless of `internal`, and `strip_internal_context_keys` scrubbed only `_CONTEXT_INTERNAL_CALLER_KEYS` -- so any session or PAT caller could set `disable_clarification` through `body.context`, or through the free-form `body.config` that `build_run_config` copies verbatim. That is not a milder flag than `non_interactive`: ClarificationMiddleware answers every clarification -- `risk_confirmation` included -- with "proceed without asking" instead of interrupting, and SandboxMiddleware reads the two keys as the same non-interactive signal. `github_token` rode the same path into `runtime.context`, where the bash tool exports it as `GH_TOKEN`/`GITHUB_TOKEN`, and a copy smuggled through `body.config['configurable']` reached the checkpoint store the context-only rule exists to avoid. Both keys are produced server-side by the channel run policies, which reach the Gateway over the internally-authenticated request channel, so gate them the same way: forward them only when `internal=True`, and scrub the union `_INTERNAL_ONLY_CONTEXT_KEYS` from both config sections for every other caller. Destination stays an orthogonal axis -- `_CONTEXT_RUNTIME_ONLY_KEYS` still land in `context` alone, never in checkpoint-persisted `configurable`. Regression coverage in tests/test_gateway_services.py pins both smuggling surfaces and replays the real start_run assembly order for a session caller and for an internal one, so the GitHub channel keeps carrying its minted token. * docs(changelog): record the internal-only run-context key gate (#5338) * docs(agents): keep the run-context note inside the AGENTS.md budgets The AG002 inherited-chain check failed at this head. The new backend section and the root scheduled-task sentence added 993 B to the root and backend guidance both the sandbox and middlewares chains inherit, pushing sandbox 6 B over the 98304 B hard limit and growing the middlewares chain, which main already exceeds by 155 B. An already-over chain is only tolerated while it does not grow, so the shared ancestors had to come back to their base size. Condensed the new material and removed prose the root file was duplicating: - The trust-boundary section keeps both gated surfaces, both helpers, the trust-vs-destination split, and the disable_clarification note in half the space. - The root scheduled-task bullet names all three internal-only keys and both smuggling surfaces while staying under its previous size. - Dropped the root `scheduler.recursion_limit` bullet, which restated backend/AGENTS.md:18 almost verbatim; its one unique fact (a YAML edit needs no Gateway restart) moved to that bullet. - Deduplicated the nginx routing sentence, which already deferred to the backend routing table, and tightened the waiver note's sequencing tail. Root and backend guidance now sit 50 B under their combined base size, so the sandbox chain returns to 97310 B and the middlewares chain no longer grows. Every file stays under its AG001 soft budget. |
||
|
wutongyuonce
|
48a8978b7b
|
feat(scheduler): add interval schedule type (#5291)
* feat(scheduler): add interval schedule type Allow scheduled tasks to fire every N seconds from last dispatch, not only wall-clock cron or a single run_at. Cadence is UTC now+N with no missed-beat catch-up, bounded by min_once_delay_seconds and 30 days. * fix(scheduler): let interval tasks create, edit, and keep next run Create/edit now keep every_seconds. Unchanged interval spec no longer resets next_run_at, including timezone-only PATCH. * fix(scheduler): keep non-minute intervals on edit Stop rounding every_seconds to whole minutes in the form. Values that are not whole minutes or hours now use a seconds unit so edit/duplicate round-trips the stored cadence instead of rewriting it and resetting next_run_at. Document that min_once_delay_seconds is also the interval floor. * fix(scheduler): clamp interval seconds to the default 60s floor The new seconds unit allowed 1–59, which the API rejects under the default min_once_delay_seconds. Clamp the form to >= 60 and show the floor next to the preview. Also mention interval in the scheduler field_doc, matching config.example.yaml. * fix(scheduler): do not clamp interval amount while typing Keystroke clamp made 90 become 9 -> 60, then 600, and backspace could not leave 60. Keep the raw field text and apply the 60s floor on blur and emit only. * test(scheduler): cover interval input editing * fix(frontend): preserve saved interval cadence until edited * style(tests): format scheduled task router tests --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
wutongyuonce
|
69f0f483eb
|
feat(scheduler): let scheduled tasks pin a custom agent (#5288)
* feat(scheduler): let scheduled tasks pin a custom agent Create and update accept optional assistant_id, defaulting to lead_agent. Custom names are normalized and must already exist for the task owner. The workspace form exposes the same choice, and duplicate copies it. Fixes #5286 * fix(scheduler): keep assistant-id PR free of interval tests Drop the six interval tests that belonged to the interval schedule PR and fail here because this tree still only accepts once/cron. Treat lead_agent case-insensitively so LEAD_AGENT / lead-agent store as the default. Omit unchanged assistant_id on edit so a deleted custom agent does not 422 unrelated PATCH (rename, reschedule). * fix(scheduler): format task page and browser tests --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Hyeonsang Cho
|
8e86729aa0
|
fix(gateway): confine artifact PUT to /mnt/user-data/outputs after path resolution (#5321)
* fix(gateway): confine artifact PUT to /mnt/user-data/outputs after path resolution
The outputs-only guard on PUT /api/threads/{id}/artifacts/{path} was a
string-prefix check on the raw path. A percent-encoded `..`
(`outputs/%2e%2e/uploads/x.txt`) survives nginx's variable proxy_pass
untouched, is decoded by Starlette, passes the prefix check, and the
resolver only confines the result to `user-data/` -- so an owner could
overwrite a sibling upload or workspace file in their own thread.
Collapse dot segments before the prefix check, and re-check the resolved
host path against the resolved outputs root so a symlink planted inside
`outputs/` cannot redirect the write either. The normalized virtual path
is what the response echoes and what non-mounted sandboxes receive.
* refactor(gateway): share the outputs-confinement rule with channel attachments
Review follow-up on #5321: the "only under /mnt/user-data/outputs" rule was
implemented independently by the artifact editor and by IM-channel
attachment delivery, and the two copies had already drifted.
Move it into app/gateway/path_utils.py as normalize_outputs_virtual_path
(collapse `..` before the prefix check) and resolve_outputs_confined_path
(re-check the resolved host path against the resolved outputs root, which
also catches a symlink planted inside outputs/). PUT /artifacts and
ChannelManager._resolve_attachments both call the helper; artifact_archive
keeps its stricter ZIP-member rules layered on top.
Tests that previously stubbed resolve_thread_virtual_path for the editor now
stub resolve_outputs_confined_path, and the channel attachment tests patch
path_utils.get_paths, which the helper binds at import like the other
consumers. The confinement itself is pinned by tests/test_gateway_path_utils.py.
|
||
|
RongJie G
|
06c827903a
|
feat(persistence): add expand-phase thread incarnation storage (#5216)
* feat(persistence): expand thread incarnation storage Add nullable thread and MCP task incarnation columns while preserving mixed-version writes. New thread records receive stable incarnation IDs, and new task rows copy the matching owned or shared thread incarnation without changing any read, claim, session, or deletion behavior. * test(persistence): pin incarnation rollback compatibility * test(api): pin internal thread response boundary * fix(persistence): rebase incarnation rollout after projects --------- Co-authored-by: CorgiBoyG <CorgiBoyG@users.noreply.github.com> |
||
|
wutongyuonce
|
3c7d3303d3
|
feat(gateway): paginate thread run history (#5283)
* feat(gateway): paginate thread run history (#5282) GET /api/threads/{thread_id}/runs stays a bare array of the newest 100 runs so LangGraph SDK clients keep working. Add GET /runs/page with a (created_at, run_id) keyset cursor so callers can walk older history. * fix(gateway): reject one-sided run history cursors RunManager.list_by_thread now raises if only one of before_created_at or before_run_id is set, matching the HTTP 422. Document the per-page sort cost on the SQL keyset query, and add the missing CHANGELOG [#5282] link definition. * fix(gateway): round-trip run page cursors through query strings Emit next_before_created_at with a Z suffix so '+' is not decoded as a space. Accept that space, and Z, when parsing. Treat blank cursor fields as absent and reject a non-ISO before_created_at in RunManager so a harness caller cannot silently restart at the newest page. * style(gateway): ruff-format run page cursor files Collapse the one-sided cursor ValueError and the two before_created_at asserts so ruff format --check passes at line-length 240. |
||
|
Zeren Wang
|
5951c89b5b
|
feat(projects): project workspaces with scoped chats and thread membership (#5265)
* feat(projects): project workspaces with scoped chats and thread membership
Backend:
- projects table model and migration; fail-closed ProjectRepository with
ownership checks, CRUD/archive/restore/delete router, and atomic thread
move between projects
- threads_meta.project_id column exposed as reserved deerflow_project_id
metadata; project-aware thread create/search with pagination bounds and
membership echoed in create responses
- first-run admission assigns the project only at genuine first run, seeded
at write time and dropped when invalid; serialized against project
deletion and thread assignment
- branch creation inherits the source thread's project membership (an
archived/deleted project degrades the branch to unassigned instead of
failing the request)
Frontend:
- projects data layer, thread move API, and sidebar projects section with
flat/grouped modes, archived-project threads, and stable virtual-list
offsets
- project detail page with project-scoped new chat
(/workspace/chats/new?project=) and paginated thread list
- move-to-project thread menu, new-project dialog, archived-project gates
- project-scoped new chats pre-create the thread with membership before the
first submit or /goal set, so runs never proceed outside the project
- goal-set preparation is fenced against conversation switches: a stale
continuation is dropped instead of saving the goal or launching the
abandoned submission on the newly opened conversation
- project thread lists join thread lifecycle invalidations (stop, pin) so
an open project page never keeps stale titles, recency, or pagination
* fix(chats): keep archive undo toast when the sidebar row unmounts
The archive success toast was fired from per-mutate callbacks passed to
mutation.mutate. React Query drops those handlers when the observer
component unmounts before the mutation settles; archiving the open chat
removes its sidebar row mid-flight, so the undo toast never appeared and
the e2e archive-undo test timed out waiting for it.
Move the success/error handlers to the mutation level (useArchiveThread
options, same pattern as useMoveThreadToProject) where callbacks are
delivered even after the originating row unmounts.
* fix(projects): pin project thread listing contract and exclude archived chats
GET /api/projects/{id}/threads returned the thread store row verbatim
(list[dict], no response_model): user_id/assistant_id leaked, any future
ThreadMetaRow column would auto-leak, and the OpenAPI schema was empty.
Return a narrow ProjectThreadResponse (the exact fields ProjectThread
declares) with the same metadata secret redaction the surrounding thread
endpoints get from _MetadataRedactingResponse.
The listing also ran search() without the archived filter, so a retired
chat rendered as a normal row on the project page while the sidebar hid
it. Search archived=False to mirror the sidebar's archived:false lists;
restore stays on the global Archived tab.
Both regressions pinned by new router tests: wire-shape allowlist and
archived-member exclusion.
* docs(migrations): record the 0019/0020 chain against the bootstrap reservation
The tree now chains 0018 -> 0019_projects -> 0020_threads_meta_project_id,
so migrations/AGENTS.md was stale twice over: the revision index stopped at
0018 and the rolling-forward section still claimed the tree 'deliberately
remains at 0018'.
Document the new head and record the intentional numeric-prefix reuse of
0019: 0019_projects is in-chain while 0019_thread_incarnations stays the
reserved, allowlisted out-of-tree rollout id. The owning rollout revision
must re-parent onto this tree's head when it merges so alembic never sees
two heads off 0018; bootstrap.py now cross-references that note next to
_FORWARD_COMPATIBLE_REVISION.
* fix(chats): invalidate project thread lists on archive/restore
useArchiveThread refreshed the infinite sidebar cache, threads/search and
the per-thread metadata cache but not the project-scoped list
([...PROJECTS_QUERY_KEY, 'threads', id]) this PR adds — the one thread
mutation not wired to that key, after usePinThread, useRenameThread,
useDeleteThread, useMoveThreadToProject and invalidateStoppedThreadCaches.
An archive from a sidebar row while a project page is open therefore left
the archived chat rendered as a normal row until remount (and undo left it
missing). Invalidate the prefix in the mutation-level success handler.
Regression test asserts the project-list prefix is invalidated on success.
* fix(projects): fetch project discovery only in grouped sidebar mode
RecentChatList mounted two useProjects queries per sidebar render, but
knownProjectIds is consumed only by the grouped-mode exclusion filter; in
the default flat mode every page load paid two GET /api/projects?status=
round trips for data nothing read. Gate both queries on grouped mode —
GroupedProjectList fetches the same keys when the toggle is on and
TanStack dedupes the observers.
Also set retry: false on useProject: a deleted or foreign project 404s
deterministically, and the page renders a dedicated not-found state for
it, so the default 1s/2s/4s retry backoff kept deep links in 'loading'
for ~7s before that state appeared. Matches useThreadMetadata /
useThreadTokenUsage.
* fix(threads): fail closed on project-scoped create in memory mode
MemoryThreadMetaStore.create accepted project_id and silently ignored it,
making memory mode the one membership path that fails open: POST
/api/threads with a project id returned 200 and the run started
unassigned, violating the invariant that a run never proceeds outside the
selected project (the SQL store raises ProjectNotAssignableError inside
the insert transaction for the same request).
Raise ProjectNotAssignableError whenever project_id is present so the
router's existing 404 mapping applies, the frontend keeps the composer
text for a retry, and memory mode behaves exactly like SQL mode.
set_project already reports rejection; create now matches it.
Store-level test (raises, nothing persisted, project filter stays empty,
unscoped creates still work) plus a router-level test asserting the 404
and that no row is left behind.
* fix(projects): window the project page thread list
ProjectThreadsSection rendered every loaded page as a plain Link row, so a
long-lived project accumulated unbounded DOM on the page's scroll surface:
each load-more appended another 100 rows and every formatTimeAgo tick
re-rendered the whole list.
Reuse VirtualThreadList (now generic over any row shape with a
thread_id), pointing its scroll parent at this page's ScrollArea viewport
via the shared [data-slot="scroll-area-viewport"] selector used by
/workspace/chats; under the 60-row threshold it falls back to the plain
render, so small projects are unchanged.
* fix(projects): restore row dividers and pin them with a render test
The row class template literal concatenated transition-colors directly
with the conditional border-b token, so non-final rows rendered the
invalid class 'transition-colorsborder-b' and lost both the divider and
the transition. Compose the row classes with cn() and a boolean guard
instead.
The section moved out of page.tsx into a testable component so the row
markup finally has coverage: a DOM test asserts every row except the
final data row carries border-b (index-based, not last: — correct under
virtualization where the last mounted row is not the last data row), and
the untitled fallback plus load-more button render for a partial page.
* fix(projects): validate forward schemas and fence membership reads
|