mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-26 13:16:24 +00:00
92 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Nan Gao
|
a684ae2fe3
|
fix(sandbox): leave SIGTERM/SIGINT teardown to the Gateway lifespan (#7560)
* fix(sandbox): leave SIGTERM/SIGINT teardown to the Gateway lifespan AIO and E2B providers built on the event-loop thread installed SIGTERM/SIGINT handlers that force-destroyed every sandbox synchronously, before uvicorn started the run drain. The Gateway lifespan now holds sandbox shutdown ownership from its first step, so providers skip those handlers and the existing hook closes the provider after run drain and MCP teardown. SIGHUP keeps the provider fallback. AIO acquire now refuses once shutdown began. * docs(sandbox): document Gateway-owned sandbox shutdown and AIO admission fence * test(gateway): inspect the lifespan body now that lifespan wraps it * docs(changelog): reference #7560 for the sandbox SIGTERM drain-order fix * ci: retrigger sandbox image smoke after a registry pull timeout |
||
|
Totoro
|
4e4ccce06b
|
fix(gateway): close initialized sandbox provider on shutdown (#7347)
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com> |
||
|
Hyeonsang Cho
|
66508d7522
|
fix(auth): map malformed OIDC provider bodies to OIDCError (#7258)
* fix(auth): map malformed OIDC provider bodies to OIDCError The SSO login route answers OIDCError with 502 and the callback with an sso_failed redirect, but oidc.py raised OIDCError only for transport and HTTP status errors. A provider answering 200 with an HTML page or any non-object JSON raised JSONDecodeError, KeyError or AttributeError instead, so both routes failed with a 500. A malformed UserInfo body also aborted the login, although UserInfo is best-effort. Decode discovery, token, JWKS and UserInfo bodies through one helper that requires a JSON object, require string endpoints and issuer, reject a non-list JWKS keys field, skip non-object JWK entries, and wrap a malformed ID token header as OIDCValidationError. Build discovery metadata before caching it so a rejected document is not served from the cache. * docs(changelog): reference #7258 for the OIDC malformed body fix * fix(auth): report a falsy non-string OIDC issuer as wrong-typed Check the issuer's type before its presence so 0, false and [] get the non-string message instead of "missing the issuer field"; an absent or empty issuer still reports missing. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Stella-xixi
|
5cef1a11ef
|
fix(checkpoints): run conservative retention after completion (#6051)
* fix(checkpoints): wire conservative retention after runs * fix(checkpoints): fence retention before checkpoint reads * fix(checkpoints): address retention review feedback * fix(checkpoints): retry delta reads after retention race * fix(checkpoints): bound admission retention to the root tail --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Nan Gao
|
085924a3d0
|
fix(channels): run each IM message once across instances, restarts and late redeliveries (#7104)
* fix(channels): run each IM message once across instances, restarts and late redeliveries Inbound IM messages were deduplicated only by the 10-minute inbound dedupe store (#7103). Discord direct messages had no key at all, a Discord thread reply got an instance-dependent key, a Discord thread was created before dedupe ran, a redelivery past the window started a second run, and two instances racing on a conversation's first messages bound it to two threads. - Discord messages are keyed (discord, guild_id or "dm", posting channel, message_id); the posting channel is the same on every instance. - MessageBus.claim_inbound lets an adapter record the manager's dedupe key before a side effect; Discord claims it on the Gateway loop before _create_thread, marks the published message dedupe_claimed, and gives the claim back when the message is never committed. - runs.wait / runs.stream / runs.create send an Idempotency-Key derived from the dedupe key, and keyed human message ids are uuid5 of it (a uuid4 per attempt made every keyed retry a 409). A reused run's status-only /wait or stream gap is resolved to this message's own turn via threads.get_state; 409s about the message's own keyed run keep the key instead of reading as a busy thread; an ended reused GitHub run gets no follow-up watcher. - A conversation's first thread binding is claimed insert-if-absent (claim_thread_id on channel_conversations and channel_thread_bindings); the loser adopts the winner and deletes the empty thread it minted. /new and /agent use still rebind. Docs: IM_CHANNEL_CONNECTIONS.md, CONFIGURATION.md, channels AGENTS.md, CHANGELOG (en/zh). * fix(channels): give back Discord dedupe claims for messages never processed Review round 1 on #7104. A Discord claim records the manager's dedupe key before the message is queued, so every path that drops the message unrun must release it, or a peer refuses the redelivery for the rest of the TTL: - The hand-off task releases the claim on cancellation too (stop() drains it while it resolves the connection identity), via await_drained so the cancellation still propagates. - ChannelManager._discard_pending_inbound releases the key of a claimed message that shutdown drops from the queue, after the grace period and on a cancelled shutdown. - The adapter tracks claims until they are committed. A release refused because stop() already closed cross-loop intake, or a hand-off cancelled before it ran, leaves the claim tracked, and stop() releases it last. * fix(gateway): keep IM run idempotency stable when /new rebinds the conversation Review round 2 on #7104. The Gateway scoped every Idempotency-Key by owner and thread. /new rebinds an IM conversation to a fresh thread, so a late redelivery of a message admitted on the old thread was requested on the new one, got a different scoped key and started a second run. - Keys from trusted internal callers (the IM channel manager) are scoped by owner only (`internal-run:` prefix); browser and API keys keep the owner-and-thread scope (`http-run:`). - A key that resolves to a run of another thread or user raises RunIdempotencyKeyMismatch, a RuntimeError subclass so in-process launchers see the same type, and start_run answers it with 409 "Idempotency-Key already used with a different request" instead of a 500. The channel manager already treats that 409 as the message's admitted run. Docs: API.md, IM_CHANNEL_CONNECTIONS.md, gateway AGENTS.md (reworded to stay under its budget), CHANGELOG (en/zh). --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
CSXizhang
|
3fd03ec2eb
|
fix(compaction): reject manual compaction of threads with pending execution (#6929)
* fix(compaction): reject manual compaction of threads with pending execution
A thread paused at a static breakpoint (interrupt_before=["tools"]) keeps its
pending node send in the checkpoint's __pregel_tasks channel. The compaction
write goes through the state-mutation graph, which schedules no tasks, so
LangGraph drops the pending send ('Ignoring unknown node name tools in pending
sends') and a resumed run finishes success without ever executing the tool.
Fail closed: detect a non-empty __pregel_tasks in the raw checkpoint tuple
inside compact_thread_context and raise ContextCompactionPendingWork, mapped
to HTTP 409 by the compact route. The reservation (reserve_checkpoint_write)
already serializes this check against runs and other checkpoint writers, and
the write still forks from the exact head that was read.
Adds dual-mode (full/delta) regression tests: the rejected compaction leaves
checkpoint id, next and pending task ids untouched, and resuming afterwards
executes the pending tool exactly once; idle threads still compact.
* fix(compaction): reject unavailable checkpoints before summarizing
* docs: keep compaction guidance within merged size budget
|
||
|
王一帆
|
6bc1020d34
|
feat(knowledge): decouple scope and read-only catalogs (phase 2) (#6904)
* feat(knowledge): decouple scope and read-only catalogs * docs(knowledge): keep frontend guidance within repository budget * fix(knowledge): align execution identity and harden provider boundaries * docs: keep knowledge guidance within CI size budgets --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
jswSpace
|
62a5b6cc26
|
fix(auth): preserve OIDC authorization endpoint query parameters (#6831)
* fix(auth): preserve OIDC authorization endpoint query parameters * docs(auth): keep updated guidance within size budget * docs(auth): clarify OIDC endpoint query configuration --------- Co-authored-by: jswSpace <198684035+jswSpace@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Eilen Shin
|
e98179a9d3
|
feat(models): register managed models with provider presets (#6757)
* feat(models): register managed models with provider presets * docs: add PR 6757 UI comparison screenshots * test: fix model management CI checks * docs: remove PR 6757 screenshots --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
ChengFa
|
78540ac0e4
|
fix(scheduler): preserve unchanged once schedules near execution (#6686)
* fix(scheduler): preserve unchanged once schedules near execution * docs: keep schedule guidance within the size limit * style(scheduler): use English for the once-edit comment * docs(scheduler): keep guidance within directory budgets --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Totoro
|
5e67d41c47
|
fix(goal): validate goal text and preserve the current goal on message replay (#6685)
* fix(goal): validate goal text and preserve current goal on message replay * fix(goal): reject malformed continuation counters before admission * fix(goal): reject external reducer overwrite markers --------- Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com> |
||
|
ordinary being
|
a7dd4849eb
|
fix(runtime): mark cancel takeovers with the orphan-recovery stop reason (#6674)
* fix(runtime): mark cancel takeovers with the orphan-recovery stop reason A cross-worker cancel that takes over an expired-lease run writes error but left stop_reason unset. `orphan_recovered` is the only signal that lets a store-only SSE/`wait` consumer synthesize END after a heartbeat, because the producer is known to be gone. Without it, a subscriber that outlives the crashed owner waits forever for an END no surviving worker publishes; periodic reconciliation ignores the row because it is already terminal. Mark the takeover like periodic orphan recovery: claim_for_takeover already accepts the stop reason and the reconcile path already passes it. The new tests pin the takeover row's marker and prove a real takeover wakes a waiting consumer; three existing takeover test doubles now forward the store argument. * docs(gateway): keep guidance within the 48KB budget The takeover stop-reason sentence pushed backend/app/gateway/AGENTS.md over its 48KB hard limit, failing agent-guidance and the backend guidance-budget test. Trim the sentence; the rationale stays in the manager.py comment. * fix(gateway): complete streams after scheduler orphan recovery --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Totoro
|
0fe6e63a68
|
feat(goal): show why a chat goal stopped and when it was met, and redact the goal evaluator and continuation input (#6556)
* feat(goal): record the met goal and keep goal state on the history head - Add a goal_outcome LastValue channel (GoalOutcomeState). Only the guarded clear of a satisfied ordinary goal writes it, in the same checkpoint as the clear; the reply anchor is the latest visible AI message with text, the reply the visible signature keys on. A goal stored without created_at still clears and gets a record. - write_thread_goal removes the record on every other goal write and lists it in channel_versions, new_versions and metadata writes when it changes. Threads that never had a record keep their checkpoint shape. The sync-only SqliteSaver and CachedHistorySaver, which take the #6448 sync fallback, store and drop it the same way. - goal_outcome is server-owned: POST /state and non-internal run input that carry it get 400. A POST /state goal write, or a goal in external run input, clears it. - A branch head written onto its replay base resets the channels the head lacks, so a goal or record that write_thread_goal removed does not come back. Branching from the met turn keeps the record without the met goal. - The POST /history head now carries an active goal and an achieved goal_outcome, also before any message exists, so the post-run history refetch keeps them. Older entries stay title/thread_data. - contracts/thread_goal_contract.json pins the chat stand-down and check-failure codes, the record shape and status, the head keys and the host-written reasons. * feat(frontend): show why a goal stopped and when it was met - The goal bar derives its state from the active goal, the goal_outcome record and the messages: set, in progress, continuing, stopped (limit, auto-continue off, no progress, token budget, missing evidence, failed run, unknown code), waiting for you, waiting, couldn't check the goal, paused and goal met. An 8/8 stand-down now reads "Stopped" with the next step instead of "Continuing 8/8", and a count-0 stand-down no longer looks like a fresh goal. - "Goal met" shows while its anchor reply is the latest visible message (a visible human message, or a visible AI message with text, moves the chat on), so a reload agrees with the live view. An active goal wins over the record, and this tab's /goal set or clear hides it. - useActiveGoal drops the optimistic goal when the record of that same goal arrives, which fixes the bar that stayed after a goal met on the first run (the clear frame has no goal key), also for a goal stored without created_at. Server goal values pass through isGoalState; a /goal status read uses the backend's own active check, so the bar, the toast and the edit lock agree with the server. - Row 2 gives the reason and what to do next; Details holds the full goal, the checker's own note (never host-written text, a check failure or a paused goal's older check), the assumption note and an unknown code, and starts collapsed in each new state. The bar is a labelled section with a live region that announces state changes only (another goal's same state counts) and re-reads its text after a language switch. - The edit pencil on the latest turn is shown disabled with "Editing is off while a goal is set" instead of hidden; the message toolbar shows on focus-within, so keyboard users can reach it. - A /goal set or clear rejected with 409 keeps the draft and says a run is still going; other failures keep the server's message and the draft. A /goal request replaced by a newer one leaves the composer alone, so a double submit no longer empties it before the second one fails. - Copy lives in inputBox.goalBar (en-US, zh-CN); a contract test reads contracts/thread_goal_contract.json, and the raw-identifier guard now catches stand-down codes and goal field names. * test(frontend): cover the goal bar on both chat pages - tests/e2e/goal-status.spec.ts runs every case on the chat page and the agent chat page, since each wires the goal bar and the edit lock itself: a goal stopped at 8/8, one waiting for details and one waiting on an open question card show their reason and next step, also after a reload; the locked edit pencil gets keyboard focus, shows its toolbar and tooltip, does not open the editor, and hides while a file uploads; Goal met after one automatic follow-up (In progress, Continuing 1/8, then "Auto-continued once") survives a reload and hides after the next visible message; a goal met on the first run of a new chat replaces the set bar and frees the pencil, and a new /goal replaces Goal met while its own run goes on; a /goal set or clear refused with 409 keeps the draft for the retry and shows its own busy toast. - The mock POST /history head carries only the contract's history_head_keys, with goal only while active and goal_outcome only when achieved; thread state and run values omit an unset goal instead of sending goal: null. PUT and DELETE /goal drop the met record, setGoalWritesBusy returns the Gateway's 409, and a repeat create of a requested thread id returns the thread as is, like the Gateway. - The raw-identifier guard also catches checker blockers and the met record's field names. - frontend/src/AGENTS.md: the GoalStatus sentence now names goal, goal_outcome and useActiveGoal, reworded in place. * test(frontend): keep the draft through a failed upload and a busy /goal #6412 fires onSent only once a send is dispatched, so a failed upload keeps the text, files and stored draft. A /goal the Gateway refuses with 409 never reaches sendMessage, and the composer rejects it so PromptInput keeps the text. Drive both through the real sendMessage in one composer: a failed upload keeps "summarize it" stored, a busy /goal keeps its text and stored draft with no run, and the retry saves the goal and starts one run with the objective, not the "/goal" text. After that run the composer is empty and no stored draft comes back. * fix(frontend): drop the stop condition's closing period before the pause clause A stored stop condition often ends with a period, so the schedule card read "…remain)., pauses itself" (and "…。，满足后自动暂停" in Chinese). Trim the condition's closing sentence punctuation before the clause that follows it. The event line, where the condition ends the sentence, is unchanged. * fix(frontend): point "evidence missing" at the agent, not a user reply "Evidence missing" asked the user to reply with what's left, which is the push a goal is meant to save. It now asks the agent to show the result or explain what's missing, and offers /goal clear only for a goal that no longer applies: clearing drops the met record, so work that is right would never show "Goal met". zh-CN follows. Copy only: no key, state or mapping changes, and the scheduled page's reason labels are untouched. tests/unit/components/workspace/goal-status-next-steps.test.ts checks the row in en-US and zh-CN, and that no stop row offers /goal clear without that condition. goal-status.dom.test.tsx and goal-stream.dom.test.tsx now expect the new copy. * fix(frontend): say "Couldn't check the goal" once in the goal bar After a goal check that could not run, the bar's second line borrowed the scheduled page's whole reason, so it repeated the chip: "Couldn't check the goal · Couldn't check the goal; the run itself may be fine. It's checked again after your next message." Chinese had the same repeat. The line now gives only why the check failed ("The run itself may be fine", "No final reply was saved" or "The chat changed during the check"), then when it is checked again. The screen-reader announcement follows. The chip and the scheduled page's reason labels are unchanged. tests/unit/components/workspace/goal-status-unchecked.test.ts checks every check-failure code in en-US and zh-CN: the label appears once, and the line still says why the check failed. * feat(goal): redact the evaluator input when pii_redaction is enabled The goal evaluator calls its model directly, outside PiiRedactionMiddleware, so it sent raw user lines, tool arguments, tool results and Human Input Card answers even with pii_redaction enabled. With pii_redaction enabled: - the messages in the evidence window are redacted whole before the evidence caps can cut an identifier in two, as TitleMiddleware does. This reuses redact_queued_messages, which already covers content, call arguments and card answers. Earlier messages are left alone: the evaluator never sees them, and the pass is synchronous work on the event loop (0.84 s for an 800-message, 4 MB thread, 0.06 s for its window); - the assembled input, the goal objective included, is redacted again. A redaction error fails the check with evaluator_failed; raw text is never sent instead. When redaction is off, the input is byte-identical. Thread state keeps the raw text. README.md, config.example.yaml and the middleware docstring list the evaluator with the other redacted inputs. tests/test_goal_pii.py puts an address where the evidence caps cut a tool argument, a tool result and a card answer, and checks the objective, the unchanged input when redaction is off, that only the evidence window reaches the redactor and gives the same evidence as redacting the whole thread, that a redaction error stands down with evaluator_failed before the model is called, and the worker path. * fix(goal): redact the hidden continuation when pii_redaction is enabled The hidden /goal continuation repeats the goal objective and the evaluator's reason and evidence summary. PiiRedactionMiddleware skips it, since it is a framework message and not a user message, so with pii_redaction enabled each continuation turn sent them raw to the agent's model, and the message stayed in the thread for later model calls. The durable-context block and the evaluator input already redact the objective. With pii_redaction enabled, make_goal_continuation_message now redacts the objective, the reason and the evidence summary with redact_text before the message is built. The thread keeps the redacted message, which the UI hides; the goal state keeps the raw objective. The worker builds the message before it counts the continuation. A redaction error fails the check the way an evaluator redaction error does: run_failed with stand_down_reason evaluator_failed, the goal stays active, nothing is sent and the continuation budget is not spent. Only the exception type is stored. With redaction off, the message is byte-identical. README.md, config.example.yaml and the middleware docstring list the continuation with the other redacted inputs. tests/test_goal_continuation_pii.py checks the redacted objective, reason and evidence summary against the tokens redact_text gives, the unchanged message with redaction off or no config, the worker path under the run config, and a redaction error. * docs(changelog): link the goal outcome entries to #6556 Fill the four ([#PR]) placeholders in CHANGELOG.md and CHANGELOG_zh.md with this PR's number and add the matching link references. * fix(frontend): keep an active goal stored without timestamps visible POST /threads/{id}/state and run input can store an active goal without created_at or updated_at. The backend still evaluates it and rejects an edit with 409 (is_active_goal checks only status "active"), but isGoalState required both timestamps, so history and live values dropped it: both chat pages hid the goal bar and unlocked the last-turn pencil, and only a /goal status read, which used a looser predicate, brought them back. isGoalState now accepts any record with a string objective and status "active", and normalizeGoalStatusRead uses it, so thread values, live updates and status reads share one rule. Malformed or inactive values are still dropped. Tests cover a sparse active goal from the history head and from a live update before the post-run refetch (bar shown, hasGoal true), and a malformed goal that stays hidden. * test(goal): check the outcome write metadata the same way on every saver PostgresSaver stores checkpoint metadata through get_serializable_checkpoint_metadata, which drops "writes", so the [postgres] case of test_goal_outcome_round_trips_through_the_sync_fallback failed with KeyError: 'writes' when CI ran it against a live database. The test now records the metadata write_thread_goal passes to put and asserts the clear's and the next goal's "writes" there, for every saver. The stored channel values are checked as before, and on sqlite and cached-sqlite the stored "writes" must still equal what was passed. * docs(agents): keep goal_outcome out of the agents guidance chain The middlewares AGENTS.md chain is already over the 98,304-byte AG002 hard limit on main, and the PR-mode check fails any chain a PR touches, so adding goal_outcome to the ThreadState field list in deerflow/agents/AGENTS.md failed the agent-guidance job. goal_outcome stays documented in runtime/AGENTS.md and backend/app/gateway/AGENTS.md, which are outside that chain. * refactor(goal): read the evaluator's evidence and redaction from one window format_visible_conversation recomputed the window start that the redaction pass takes from _evidence_window. Both now use _evidence_window, so a later change to the cap or to what counts as visible moves both. The window test now fills the window with tool-only replies and tool results. Before, it caught a cap change in one copy but not a change to what counts as visible. --------- Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
RongJie G
|
8b8fdf4387
|
fix(gateway): make resume retries idempotent (#6499)
* fix(gateway): make resume idempotency command-aware Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com> * docs(gateway): document keyed resume rollout restrictions --------- Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Nan Gao
|
4bab280250
|
fix(skills): share skill cache invalidation across Gateway replicas (#6495)
* fix(skills): share skill cache invalidation across Gateway replicas SkillStorage.load_skills() rescans disk and re-reads extensions_config.json on every call, but the prompt layer in deerflow/agents/lead_agent/prompt.py caches the enabled-skills list, the per-(app_config, user_id) entries and the rendered <skill_system> section per process, and every skill mutation in routers/skills.py only refreshed the process that handled it. With several uvicorn workers or several Pods sharing one home volume, a skill installed, edited, deleted, rolled back or toggled through replica A kept being offered as the old skill set by replica B until B restarted, and POST /api/skills/reload had to be sent to every Pod separately. Reuse the MCP cache-reset mechanism. A new generic helper, deerflow.config.shared_reset_marker, owns a hidden JSON marker beside the resolved extensions config (.<config name>.<suffix>.json): publish() replaces it atomically with a random generation under extensions_config_write_lock + extensions_config_file_lock, chaining previous_generation and an optional user_id; SharedResetMarkerTracker polls it at most once per second (monotonic clock, never blocking behind a concurrent poller), adopts the current state silently on its first poll, and classifies a change as user-scoped only when exactly one publication carrying that user happened since the last poll. deerflow/mcp/cache.py now builds its .<config>.mcp-cache-reset.json marker through the same helper with no behavior change. prompt.py publishes .<config>.skills-cache-reset.json via publish_skills_cache_reset(user_id=...) and checks the marker on every cache lookup (global cache, per-config cache, prompt-section LRU, user scoped entries), calling _invalidate_enabled_skills_cache() or invalidate_user_skill_cache(). The publishing process remembers its own generation so it does not retire the caches it just rebuilt. Every mutation tail in routers/skills.py (install/upload, edit, delete, rollback, enable/disable, reload) refreshes locally and then publishes the marker off the event loop; /skills/reload answers scope=shared_config, or scope=process when no extensions config path resolves, and a marker publication failure is a 500 so a 200 never means only one process was refreshed. Docs: README, backend/docs/API.md, skills/config/mcp/gateway AGENTS.md, and frontend skills.mdx (en/zh; also fixes the reference to a non-existent skills/loader.py) describe the shared invalidation and drop the per-Pod guidance. CHANGELOG entry under Unreleased / Fixed. * docs(changelog): link the skills cache reset entry to #6495 * fix(skills): drain the reload tail and keep marker failures out of 404 Two review follow-ups on the shared skills cache reset (#6495). POST /skills/reload awaited the local refresh and the marker publication unshielded. CancelledError is a BaseException, so a caller that disconnected between the two slipped past the `except Exception -> 500` mapping, surfaced no error, and left every peer replica serving the old skill set -- the one outcome the operator hook for external mount writes exists to prevent. The refresh and the publication now run as one `await_drained(_drain_skill_mutation("reload", ...))` tail, the same discipline the install/edit/delete/rollback/toggle tails already use; the exception -> 500 mapping and the scope response are unchanged. Publication errors could surface as 404. resolve_shared_config_path() only swallows the resolution-time FileNotFoundError; a config directory that vanishes between resolution and the lock file / atomic replace still raised FileNotFoundError from the marker write, and the install, edit, delete and rollback handlers mapped that to HTTP 404 with the raw OS error text for a skill that had in fact been applied. publish_skills_cache_reset() now re-raises OSError (and a misconfigured project root's ValueError) as SkillCacheResetPublishError, a RuntimeError that no handler maps to 404/400, so every handler reports it through its generic 500 branch; the message names only the exception type, never a server path. The install handler gains an explicit clause that pins that contract beside its 404 mapping. Tests: cancelling reload after the refresh still publishes the marker; a FileNotFoundError from the marker write yields 500 (not 404) for install, edit, delete and rollback, 500 for the public toggle and the generic reload error; the wrapper type is not an OSError. |
||
|
Nan Gao
|
5e3c24fe94
|
fix(gateway): probe the Redis stream bridge and provisioner in /health/ready (#6447)
* fix(gateway): probe the Redis stream bridge and provisioner in /health/ready GET /health/ready only probed the ORM engine and the LangGraph checkpointer. The Redis stream bridge client connects lazily and nothing ever pinged it, so a Gateway whose Redis was down started, answered 200 ready to the Kubernetes readinessProbe and Compose healthcheck, and kept receiving traffic while every run's first publish failed. Since #6328 the Redis bridge is mandatory for multi-instance deployments, so such a replica is not serviceable at all. - StreamBridge gains `async ping() -> bool`; the base returns True (no external backend), RedisStreamBridge round-trips PING and answers False on RedisError. - /health/ready adds a `stream_bridge` probe (not_configured for the memory bridge; unreachable flips the endpoint to 503; a missing bridge fails closed) and a report-only `provisioner` probe of `{sandbox.provisioner_url}/health` that never changes the status code, since every replica shares one provisioner. The provisioner URL is snapshotted on app.state at startup. - A probe that overruns the endpoint deadline is now the only one marked unreachable; finished verdicts are kept. - The public probe gate is one lock per probe kind, so one request's probes still run concurrently while a thundering herd opens at most one probe connection per kind per process. - Tests: bridge ping cases, stream-bridge/provisioner probe cases, verdict and deadline cases, and the first route-level /health/ready tests via TestClient. - Docs: gateway AGENTS.md, CONFIGURATION.md multi-instance notes, Helm README and probe comment, Compose healthcheck comment. * docs(changelog): note the readiness probe fix (#6447) * fix(gateway): apply the sandbox proxy policy to the provisioner probe Review follow-up for #6447. - The provisioner readiness probe built its httpx client with the default trust_env=True. httpx 0.28 treats CIDR entries in NO_PROXY as host patterns rather than subnet bypasses, so with HTTP_PROXY set and NO_PROXY=10.0.0.0/8 a private provisioner that the requests-based sandbox calls reach directly would have been probed through the proxy and reported unreachable on every check. The probe now reuses `sandbox_http_trust_env`: loopback, private, link-local and cluster-local addresses bypass the proxy, external hosts keep the environment's proxy settings. - Pin the liveness/readiness separation with a route test: `GET /health` stays 200 while the stream bridge and database are unreachable and `/health/ready` is 503, so a Redis blip can never become a restart loop. |
||
|
Totoro
|
35cdcab1c5
|
feat(scheduled-tasks): signals, honest notices and fair queueing for server-started runs (RFC #6340, 2/2) (#6393)
* feat(scheduler): lifecycle rules, goal/limit parity and coded errors for scheduled tasks
- Migration 0031: nullable scheduled_tasks.unmet_streak_after_seq (internal
streak boundary) and stop_condition (the user's stop rule, own column).
- Stop rule is normalized (stop_rule.py) and appended only at launch; the
stored prompt never contains it. Wording adapts when scheduler.tool_enabled
is off.
- Willem 1: page-created tasks get goal evaluation, <standing_notes>, the
previous-run reference and own-schedule stop; request_stop and the
scheduled capability no longer require an origin conversation.
- Repository: single reactivation choke point raising
ScheduledTaskLimitsExhausted; streak boundary moves on goal/prompt/stop
condition/note changes (not on resume); goal-check failures neither count
nor reset the streak; pause of a finished task returns "finished"; trials
on a paused task keep the host pause marker.
- New repository reads: automatic_runs_used_for, active_run_status_for,
list_manageable_from_thread, thread_relation/thread_run on thread tasks,
task_ids_for_thread, run_number, and run_number/total_tokens/summary on
run history rows.
- REST: goal/max_runs/end_at/stop_condition on create and PATCH (null
clears, naive end_at = task timezone), resume recomputes from now with an
optional renewal body, once_time_passed, task_finished, limits_exhausted,
scheduler_not_running on create, trigger returns outcome/existing/thread_id,
task responses add automatic_runs_used and active_run_status.
- Coded errors {code, message, params} for routes, shared validation and the
chat capability (scheduled_task_errors.py), pinned by
contracts/scheduled_task_errors_contract.json; goal notes contract v2 with
check_failure_codes, scheduled_origin_key and host_run_errors (host_notes.py).
- /api/features adds scheduled_tasks {available, running, tool_enabled,
min_interval_seconds}; ScheduledTaskService.is_running.
- Docs: six README scheduler sections, API.md scheduled tasks section
(breaking error shape), CONFIGURATION.md, AGENTS notes, CHANGELOGs.
- Tests: migration 0031, stop rule, error contract, run listing, lifecycle
over HTTP, repository, service and capability cases; updated tests for the
new error shape, finished-task pause and head revision.
* feat(scheduler): conversation update/resume, run-conversation authority and readable launches
- schedule_task: update and resume actions, stop_condition and clear_fields
parameters, plain parameter types (value rules come back as coded results),
rewritten docstrings for schedule_task and stop_scheduled_task.
- Capability: update/resume share the REST validation and error codes;
interactive turns in a task's run conversation may manage that task;
scheduled runs keep only own-stop; grants require a running scheduler.
- Results: compact JSON-safe task views with local times, display card/text,
last run in list, timezone_source/browser_timezone, now_local; no ISO, user,
lease or origin fields; stop_instructions removed.
- Timezone: explicit > browser (context.client_timezone, validated in
start_run and never forwarded to the run config) > not needed for intervals
and offset run_at > timezone_required.
- Trial gate accepts natural direct requests after acknowledgements and
particles; trial results report thread_id and existing.
- Scheduled launches: stable message id scheduled-<task_run_id>, server-owned
deerflow_scheduled_origin metadata with the user-language parts, run thread
title for fresh-thread tasks.
- Worker records goal continuations in the verdict; lead prompt language rule
covers stored task text and scheduled runs.
- Docs: six READMEs, CONFIGURATION, API, runtime/tools AGENTS, CHANGELOGs.
* feat(frontend): scheduled-task core types, formatting, errors and glossary
- throwGatewayApiError now throws GatewayApiError (status, code, params,
rawMessage); message stays the string detail or detail.message
- scheduled-task types gain stop_condition, automatic_runs_used,
active_run_status, origin/notes, run_number/total_tokens/summary, thread
relation, trigger result, renewal and scheduled origin; isTaskBusy and
hasActiveRun helpers
- api: fetchScheduledTask, resume with optional renewal body, typed trigger
result and update payload with null clears
- hooks: polling at 3 s while a run is active (list, thread list, run
history), useScheduledTask with live gating and final 404, useInView,
localized error toasts; run history refreshes tasks when a run finishes
- new pure modules: errors.ts (contract code to localized copy), format.ts
(relative task times, viewer time, display zone), status.ts, actions.ts,
run-error.ts; describeTaskSchedule in cron.ts; goal-outcome adds unchecked
goal checks, describeTaskOutcome and parseStopRunId
- features: scheduled_tasks block with defaults when missing
- buildRunContext sends client_timezone
- titleOfThread takes a localized untitled label; breadcrumb names scheduled
tasks and agents; reference chips use the localized untitled label
- schedule input: shared browserTimeZone, labelled selects, aria-pressed
type toggles, interval minimum from the hint template
- i18n: all PR1 scheduled-task keys in en-US and zh-CN, glossary fixes,
recipe instructions moved into i18n
- e2e mocks follow the new task, trigger, resume, thread-task and features
shapes; expectNoRawIdentifiers helpers for e2e and DOM tests
- tests for errors, format, cron, status, actions, goal outcome, run errors,
hooks, run context, thread titles and features; scheduled e2e copy updated
* feat(frontend): list-first scheduled tasks page with outcomes and editable goals
- page.tsx is now a thin composition of components/workspace/scheduled-tasks:
status tabs (All/Active/Paused/Finished) and search over the task list,
?task_id= deep link with router.replace on selection, ?thread_id= chip
with Show all, empty state, scheduler-off notice and unavailable state
- task list rows show the schedule in words and one status line (next run,
running now, paused by agent, auto-paused, finished by its cap)
- task detail: status badge, actions from availableActions with reasons,
More actions menu (Duplicate, Copy task ID, Delete), outcome notice,
Runs / Stops when / Each run's goal / Does / Notes from chat / History,
origin chat link; success toasts for every action
- run rows: run number or Trial run, summary, goal and assumption badges,
continuations, tokens, Open chat link; raw errors and the evaluator
reason only behind Details
- create/edit/duplicate dialog with goal, stop condition, safety caps and
advanced agent/context options; PATCH sends only changed fields, null
clears a cap
- renew dialog raises or removes caps and resumes in one request; a
limits_exhausted Resume opens it
- removed i18n keys the old page used; added feedback.resumedNoTime
- recorded en-3 and zh-1 page fixtures; scheduled e2e specs rewritten for
the new page; DOM tests for the detail and the form dialog
- CHANGELOG entries and the frontend AGENTS.md scheduled-task note
* feat(frontend): live schedule card, run conversation provenance and header entry
- schedule_task results (create, update, pause, resume, trial, delete) become
an assistant:scheduled-task message group rendered as a live card: schedule,
next run, stop condition with the safety cap, goal, where results go, Run
once now, Pause/Resume and Open task; the card polls the task only while it
is on screen and shows deleted, scheduler-off and limits-exhausted states
- one card per task per assistant turn; the card group is not a turn boundary
for usage, branching, editing, outline or skill attribution
- a human message with deerflow_scheduled_origin stays a human group with
scheduledOrigin and renders as a "Scheduled run / Trial run" block with the
task instructions, stop condition and notes collapsed; the launched text is
never shown and the message is not editable
- readable step labels for schedule_task actions and stop_scheduled_task
- ThreadScheduledTasksButton replaces the header link: hidden without tasks,
shows the count, links to the task (or the chat filter), and a run's chat
links to its task
- recorded en-3 / zh-1 chat and run-thread fixtures; chat-card and run-thread
e2e specs; unit and DOM tests for parsing, grouping, turn boundaries, card,
run block, header button, step labels and reconnect hydration
- i18n: header.countLabelOne added, unused card.next removed; CHANGELOG and
frontend AGENTS.md notes
* fix(scheduler): address verification findings
Move the scheduled-task lifecycle guidance from backend/AGENTS.md into the persistence guide and leave a one-line pointer, so the effective AGENTS.md chains for agents/middlewares, sandbox and subagents stay under the 98304-byte hard limit enforced by scripts/check_agent_guidance.py in lint-check.
* fix(scheduler): address review findings
* fix(gateway): no-transform on SSE responses
All four text/event-stream responses (thread run create stream, join,
existing-run GET/POST stream, stateless create stream) now take their
headers from app/gateway/sse_headers.py and send
Cache-Control: no-cache, no-transform, so compressing proxies such as
Next's rewrite proxy stop buffering streams. Content-Location and
X-Accel-Buffering are unchanged; scheduled and MCP-notification runs keep
stream_mode=None.
Tests pin the headers on every SSE route and fail on a literal no-cache
header left in routers. Docs: API.md SSE headers note, CHANGELOGs.
* feat(scheduler): per-owner run cap and fair queue selection
New scheduler.max_concurrent_runs_per_user (default 2, 0 = off, effective
min(cap, max_concurrent_runs)) caps launching/running scheduled runs per task
owner. claim_queued_run checks it right after the global count under the same
budget lock, so it is atomic across workers and Pods. list_queued_runs ranks
same-thread FIFO heads per owner with ROW_NUMBER() OVER (PARTITION BY owner)
and orders the batch by that rank, so the drain is owner-fair even with the
cap off; owners at their cap drop out before the batch limit. Orphan rows
(task deleted while queued) are never held back by the cap.
Gateway lifespan passes the startup value; the service constructor default
stays 0. Docs: config.example.yaml (config_version 54), CONFIGURATION.md,
the six READMEs, backend AGENTS.md, CHANGELOGs (behavior change flagged).
The live-task quota is untouched.
* feat(scheduler): durable lifecycle events for the originating chat
When a chat-created schedule is paused by the agent, auto-paused or finishes,
finalization now reports task_stopped / task_paused / task_finished, only on a
live -> paused/finished transition. The new finish_task_at_end_condition
replaces the three inline "mark completed" blocks (admission rejected as ended,
claim-time skip, complete_if_ended), so every site emits exactly once.
The scheduler's finalization observer is always installed. It writes
scheduled_task_events rows for the originating chat in the same transaction as
the state change, deduplicated on (task_id, anchor, event), with a title
snapshot, stop condition, run number, last run outcome and the chat's newest
run as placement anchor. Lookups fall back on bad stored data; only database
errors propagate. Manual trials drop only run_* events. The IM step keeps
PR1's selection until B4.
New GET /api/threads/{thread_id}/scheduled-task-events (owner check, caller's
rows, oldest first); rows survive task deletion and go with the chat.
Migration 0032_activity_and_task_events adds the events table plus the
run-origin column/index and read-state tables used later, each object guarded
on its own. Contract scheduled_goal_notes v3 pins the lifecycle vocabulary.
Docs: API.md, migrations chain, backend AGENTS.md, six READMEs, CHANGELOGs.
* feat(scheduler): honest, localized and self-contained IM notices
Every scheduled-task IM notice is now enqueued by the finalization observer
in the transaction that records the outcome, one message per occurrence
(task_stopped > task_paused > task_finished > run_failed > run_unmet >
run_completed; a once task sends only its run event). The merged auto-pause
replaces the separate unmet + pause messages, agent stops and finishes now
notify, and recovery-finalized runs notify exactly once. Idle finishes use a
deterministic idle: key derived from the lifecycle anchor. The old
non-transactional enqueue in handle_run_completion is removed.
New app/channels/capabilities.py holds CHANNEL_CAPABILITIES with a
proactive_notifications flag (adds the missing qq entry); only push-capable
providers (WeCom) get rows, /api/channels/providers returns the flag, and the
worker ends legacy rows for other providers as terminal failed.
New app/scheduler/notification_text.py renders en-US/zh-CN text: title, what
happened (plus the last run's outcome), the tasks page's one-line run summary
(redacted) when the agent replied, and a closing line. No IDs, no links, no
public_base_url. Locale is the owner's new `locale` preference, else the new
channel_connections.notification_locale (default en-US).
Docs: config.example.yaml (config_version 55), CONFIGURATION.md, API.md,
channels AGENTS.md, six READMEs, CHANGELOGs (behavior change flagged).
* feat(gateway): thread origin, activity feed and read markers for server-created threads
Runs started by the server for a user (scheduler, IM channels, GitHub
agents, extension handles, MCP notifications) now carry a server-owned
metadata.deerflow_origin {kind, provider?, namespace?} (new harness module
deerflow.runtime.run_origin, pinned by contracts/thread_origin_contract.json).
It is stamped only from request.state.run_origin or by the internal channel
caller; run admission drops client copies from both metadata and
config.metadata, and thread create/patch strip it. RunRepository derives
runs.origin_kind on put and on create_thread_operation_atomic (agent runs
only), and gains latest_change().
New ThreadReadRepository keeps a per-user, never-decreasing read position
(thread_read_markers) and a per-user read clock (thread_read_versions).
POST /api/threads/{id}/read marks a thread read, thread search items carry
unread, and thread delete removes the markers. GET /api/thread-activity pages
the caller's run changes over the run-change clock, returns only threads
changed by server-originated runs plus read_version, and 503s without SQL;
/api/features reports thread_activity.available.
Docs: API.md, gateway AGENTS.md (condensed to stay under its budget),
CHANGELOGs.
* fix(scheduler): adapt PR2 to PR1 follow-ups
PR1 moved the scheduled-task lifecycle rules out of backend/AGENTS.md into
the persistence guide and grew the sandbox AGENTS.md chain to within 267
bytes of its hard limit. The per-owner queue-fairness detail now lives in
the persistence guide next to the lifecycle rules (backend/AGENTS.md keeps
the cap and a link), so check_agent_guidance passes again.
PR1's goal-outcome unit test pinned scheduled_goal_notes_contract version 2;
PR2 bumps it to 3 (lifecycle vocabulary only, the v2 keys are unchanged).
* fix(scheduled-tasks): polish from live acceptance
- zh schedule wording: an interval of one unit reads 每分钟 / 每小时 / 每秒,
and a saved interval of whole days reads 每天 / 每 2 天 (Every day / Every 2
days) instead of 每 1 分钟 or 每 24 小时; several units keep the number
- zh spacing: 已于{time} 满足 and 在{time} 的运行中 now leave a space between
the interpolated time and the Chinese after it; a copy test guards every
scheduled-task zh string against a placeholder glued to Chinese
- run summary: when the first line of the final reply only leads in to a list
(ends with ":" or "："), the list items follow it as plain text (bullets,
numbering, checkboxes, bold, code and links stripped), joined with "；" for
CJK text and "; " otherwise, within the 160-character cap
- a schedule_task result (create, update, pause, resume, trial, delete) that
arrives in a chat invalidates that chat's task list, the tasks list and the
named task, so the header button and cards update without waiting for the
idle poll; results already in the loaded history do not refetch
- e2e and unit fixtures are now the live acceptance recordings (DeepSeek
Flash, demo data, fictional names): live-weekday.json (create, trial, edit,
pause, resume in one zh chat) and live-minute.json (per-minute chat, run 1
and run 2, paused by the agent), fetched with GET from the demo Gateway and
trimmed; the hand-shaped en-3 / zh-1 fixtures are removed and the chat-card,
run-thread and tasks-page specs run in both locales over the recorded data
* docs(changelog): reference #6378 for the scheduled-tasks entries
* docs(agents): shorten the scheduled-task pointer to stay under the guidance limit
* test(scheduled-tasks): pin the live takes to their recorded time zone
The live fixtures were recorded in Asia/Shanghai, and an interval task
reads in the viewer's zone, so the 16:48 assertions failed on UTC CI
runners. Pin the browser zone in the two live-take e2e specs and the
zh live-take unit test.
* feat(frontend): thread origin, activity polling, read markers and task events core
Frontend core for server-created threads and schedule lifecycle events
(PR2 F1); the sidebar markers (F2) and the chat event line (F3) build on it.
- core/threads/origin.ts: threadOriginOf reads metadata.deerflow_origin,
then channel_source (github -> GitHub origin), then a legacy
scheduled_task_id; key and kinds pinned to
contracts/thread_origin_contract.json. labelOfThreadOrigin and the now
exported labelOfChannelProvider(provider, t) use the new localized
threads.origin.providers names (飞书, 企业微信, ...), with the English map
(gaining GitHub) only as the fallback.
- core/threads/activity.ts: useThreadActivity polls /api/thread-activity
every 15 s (focus refetch, never in the background, off unless
features.thread_activity.available). The first response only seeds the
cursor; server-origin threads, a truncated page or a higher read_version
invalidate the thread lists, a scheduled thread also ["scheduled-tasks"];
an invalid cursor re-seeds. useMarkThreadRead debounces POST /read by 1 s,
skips a thread whose cached unread is false, patches the search and
infinite caches optimistically and stores the returned read_version so a
tab's own read does not refetch its lists. AgentThread gains unread.
- core/scheduled-tasks/events.ts: ScheduledTaskEvent, the events query
(["scheduled-tasks","thread-events",id], one fetch per saved chat, not
gated on the task list so a deleted task's lines stay, refetched when a
task of the chat changes status or finishes a run), describeTaskEvent
(en/zh copy, title and condition as segments, last-run suffix, See that
run / Open task) and placeTaskEvents (end of the after_run_id turn, else
the tail, ordered by created_at); lifecycle names pinned to contract v3.
- features: thread_activity (default unavailable); channels:
proactive_notifications on providers.
- Locale sync: the I18nProvider reports the UI language and a signed-in
preference session saves it as the new `locale` preference when the
account's differs (and on every switch), so IM notices follow the UI
language; nothing is written with auth disabled.
- i18n: threads.* (unread, origin labels, provider names),
scheduledTasks.events.*, history.waitingForSlot (now shown for queued run
rows), channels.scheduledUpdates.*.
- e2e mocks: thread activity feed and controller, POST /read, the events
route, providers with proactive_notifications, search unread, features
thread_activity (off by default). Preference e2e mocks return the UI
locale so they keep asserting only their own edits.
- Docs: frontend AGENTS.md, CHANGELOGs.
* fix(scheduled-tasks): keep untouched end times and drop a chat filter the new task is not in
- The edit, duplicate and renew dialogs keep a stored end time's instant
while its field still shows it. A minute-precision wall value cannot
keep seconds or name the later 01:30 of a DST fall-back, so a
title-only edit moved the cap an hour earlier, and raising the run
limit re-sent the end time without its seconds.
- Creating or duplicating a task from a chat's task view drops the chat
filter when the new task does not belong to that chat, instead of
showing "This task isn't one of this chat's tasks".
* feat(frontend): origin markers and unread state in the thread lists
Server-created threads (PR2 F2) now say who created them and whether they
are new, in the sidebar and on the Chats page, and reach an open sidebar
without a reload.
- ThreadOriginIcon (thread-channel-source.tsx): clock for a scheduled run,
the provider mark for IM threads, the GitHub mark (new in
ChannelProviderIcon) for GitHub threads, a puzzle piece for extensions;
the localized "Scheduled run" / "From Feishu" / "From GitHub" / "From an
extension" label is its accessible name and tooltip. ThreadChannelIcon
stays exported as a wrapper, and the provider chip tooltip is localized
(no more "Feishu channel").
- Unread dot after the title (aria-hidden dot plus sr-only "Unread"); the
row aria-label becomes "{title}, unread" / "{title}，未读". Never shown for
the open thread.
- WorkspaceSidebar mounts useThreadActivity once.
- useMarkOpenThreadRead (components/workspace/chats): the default and
custom-agent chat pages mark the thread read once its metadata has
loaded, when its cached unread flips to true while open, when the tab
becomes visible, and from onFinish (a send or a joined run ended). Each
trigger goes through useMarkThreadRead, so it is debounced and a no-op
while the cached unread is false.
- Tests: thread-origin-icon and recent-chat-list DOM tests (en/zh, active
row, no raw identifiers), use-mark-open-thread-read DOM tests,
e2e thread-activity.spec.ts (new scheduled run appears unread within one
fake-clock poll without a reload, opening it posts /read and clears the
dot, a read on another device clears it, IM and GitHub markers, zh
labels, the Chats page, no polling when unavailable);
thread-history.spec.ts now finds the localized origin icon.
- Docs: frontend AGENTS.md, CHANGELOGs.
* feat(frontend): schedule event line in the originating chat and channel notice support
The chat that created a schedule now says when the schedule paused or
ended, and the channel cards say which apps receive scheduled task updates
(PR2 F3).
- ScheduledTaskEventLine (mockup m2, marker 4): a dashed, muted row with
the describeTaskEvent text (title in <strong>, stop condition clamped to
one line with the full text in its tooltip, last-run suffix), the time
via formatTaskTime and a right-aligned "See that run" (run chat) or
"Open task" link. PauseCircle for paused, CircleCheck for finished and
once-done, CircleX for once-failed. role="note", aria-label "Scheduled
task update" / "定时任务通知", data-task-id and data-event-id only.
- MessageList takes scheduledTaskEvents and renders placeTaskEvents output
through a new VirtualMessageList.renderAfterGroup, so each line sits at
the end of its after_run_id turn inside the group's measured row (also
when the list virtualizes); lines with nowhere to go render at the tail.
- Both chat pages load useThreadScheduledTaskEvents (not gated on the task
list, so the lines stay after a task is deleted). The custom-agent page
passes its agent so "See that run" opens the run on that agent's route.
- ChannelScheduledUpdates: "Scheduled task updates: sent here" / "... not
available for this app yet" (定时任务通知：会发送到这里 / 此应用暂不支持)
on the Settings provider cards and under each sidebar channel row; nothing
when an older backend does not report proactive_notifications.
- e2e mock controller: setScheduledTasks.
- Tests: scheduled-task-event-line and channel-scheduled-updates DOM tests
(every kind in en and zh, hrefs, clamp, no raw identifiers, Chinese
title), MessageList placement DOM tests; e2e
scheduled-task-chat-events.spec.ts (the line appears with the card's
"Paused by agent" without a reload, See that run opens the run chat,
anchored vs tail placement, two events in time order, the line survives
task deletion and Open task shows the task is gone; en-3 and zh-1) and
channels.spec.ts (WeCom sent here, Feishu not yet; en and zh).
- Docs: frontend AGENTS.md, CHANGELOGs.
* fix(scheduled-tasks): tie pause notices to the pause they describe
- Paused by agent links and times only the run named in last_error. When
that run is not loaded, the notice shows no link and the task's own time
instead of an earlier run that also asked to stop.
- The outcome notice reads the latest history page whichever page is
shown, so browsing older runs no longer swaps the auto-pause reason and
"Open latest run" to an earlier miss. It shares that page's request.
* fix(scheduler): address PR2 review findings
- Event lines: hold an event whose anchor run is on an unloaded history
page instead of placing it at the bottom (hasMoreHistory); tail fallback
only for fully loaded (branched/pruned) history or no anchor.
- Idle-finish anchor follows its reason: end:<iso> only for an end_at
finish, seq:<n> otherwise, so a later end-time finish is not deduplicated
away by an earlier max_runs finish.
- Observer takes FOR SHARE on the originating threads_meta row, so a racing
thread delete cannot leave an orphan event line.
- One taskPagePath (core), re-exported by the card.
- Notification locales derived from NotificationLocale; /preferences uses it.
- IM notices drop generic reasons (goal_not_met_yet, unknown) instead of
repeating the sentence.
- Localized provider names on Settings cards, sidebar rows and setup dialog.
- zh event lines: no space after a Chinese title (Latin-aware gap).
- Unread row label keeps the origin marker; chats page keeps the time as a
description.
- Sidebar shows the scheduled-updates line only for connected apps;
Settings says "available after you connect" for unconnected push apps.
- Event line keeps a real space before the time.
* fix(scheduled-tasks): three lifecycle gaps found while re-checking review classes
- The edit form compares a cron in the form its schedule input emits, so an
untouched "0 09 * * 0,6" is not re-sent as "0 9 * * 6,0" and a title-only
edit no longer re-arms or refuses a finished task.
- Chat resume asks for a timezone, as update does, when a naive end_at would
land in the UTC placeholder of a zone-free task.
- Re-arming a task clears a host pause marker, as resume does, so a later
manual pause no longer reads as "Paused by agent".
* test(scheduler): consolidate PR2 backend test setup
- Fold the repeated task, origin-chat and observer setup of the
lifecycle-event repository tests into one lifecycle_task builder.
- Build the per-owner-cap services through one _capped_service builder
that keeps every cap explicit at the call site.
- Table the five SSE route header checks into one parametrized test with
the same routes, bodies and Content-Location expectations.
- Reuse bind_wecom and hoist repeated local imports in the event tests.
Every case and assertion is kept; collected cases are unchanged.
* test(frontend): consolidate PR2 thread activity and origin test setup
Extract shared builders and helpers in the frontend tests PR2 adds, with
every case, input and assertion kept:
- activity.test.ts: seededClient/apply helpers and one LIST_KEYS constant
replace the per-test seed, spy and apply boilerplate.
- mark-read.dom.test.tsx: newClient() and advance() replace the repeated
QueryClient wrapper and fake-timer act blocks.
- origin.test.ts: the English and Chinese label checks become tables; the
no-English-name check now runs on every Chinese row.
- thread-activity.spec.ts: a scheduledRun() builder replaces four inline
copies of the scheduled run thread.
- channels.spec.ts: the PR2 block uses setLocaleCookie and one card helper.
Collected cases are unchanged (unit 211, e2e 60); no test was removed.
* fix(scheduled-tasks): keep the notice's latest-runs query from refreshing older pages
useLatestScheduledTaskRuns kept React Query's default refetch on mount,
focus and reconnect, so going offline and back while an older history page
was shown fetched the latest page again. Refreshing stays with the history,
which refreshes the latest page only while it is shown.
* fix(scheduler): two gaps found in live acceptance
- A fresh scheduled run thread is named when it launches. The worker
copied the title to the thread list only when the run ended, so the
sidebar, which now shows the new run thread at once, listed it as
untitled while it ran.
- A task capped at one run finishes with "its one run is done" /
"唯一一次运行已完成" in the chat event line and the IM notice, instead
of "all 1 runs are done".
* test(scheduled-tasks): let the viewer's time carry its own day label
Between 16:00 and 24:00 UTC, UTC and Asia/Shanghai are on different days,
so the run prompt shows "Tomorrow 01:00 · Today 09:00 your time" and the
fixed "01:00 · 09:00 your time" assertion failed depending on the hour.
* test(scheduler): expect the one-run finish copy in service notices
* fix(gateway): create a scheduled run thread with its title
Pre-setting the name after start_run raced the thread row, which the run
worker creates asynchronously: live, one of two run threads still reached
the sidebar untitled. Run admission now creates a new thread with the title
its run input carries, which the worker would copy only at run end.
* docs(changelog): reference #6393 for the scheduled-task signal entries
* fix(scheduled-tasks): route event-line run links by the agent that ran them
The originating chat's agent was used for "See that run", but a task's
agent can change after the chat created it, so the link could open the run
chat on another agent's route. Lifecycle events now record run_agent_name,
the task's agent when the run finished (edits are refused while a run is
active), and the line routes by it.
* fix(scheduled-tasks): load up to the route's maximum of chat event lines
The chat loads its lifecycle events once, and the route's default of 50
would silently drop the oldest lines of a chat with many schedules. Ask
for the maximum, 200.
* fix(threads): keep activity reads and the first cursor in step with the lists
- Accepting the first activity cursor refreshes the thread lists once. The
lists load on their own, possibly before a run the seed already counts,
and later polls start after it, so such a thread never appeared.
- A local read whose returned read clock jumps past that one read also
covers reads on other devices; the lists are refreshed instead of the
clock silently skipping those threads' stale unread dots.
- An invalid cursor re-seeds through the same path, so it refreshes once.
* docs(agents): bring the middlewares guidance chain back under the hard limit
#6378 and #6363 each fit the budget, but together the middlewares AGENTS.md
chain on main is 98,425 bytes, 121 over the 98,304 hard limit, so Lint
Check fails on main. Move the scheduled-task recovery, launch-update and
queue-timeout rules from backend/AGENTS.md into the persistence guide's
lifecycle section, which backend/AGENTS.md already points to. The chain is
now 98,067 bytes.
* fix(helm): bump the chart's config_version to 55
config.example.yaml moved to 55 in this PR, so the Helm chart's
validate-chart drift check failed. The chart's embedded config needs no
field changes: the new scheduler and channel keys have defaults.
---------
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|
||
|
Totoro
|
2dd2e5415c
|
feat(scheduled-tasks): finish the user-facing side of conversation schedules (RFC #6340, 1/2) (#6378)
* feat(scheduler): lifecycle rules, goal/limit parity and coded errors for scheduled tasks
- Migration 0031: nullable scheduled_tasks.unmet_streak_after_seq (internal
streak boundary) and stop_condition (the user's stop rule, own column).
- Stop rule is normalized (stop_rule.py) and appended only at launch; the
stored prompt never contains it. Wording adapts when scheduler.tool_enabled
is off.
- Willem 1: page-created tasks get goal evaluation, <standing_notes>, the
previous-run reference and own-schedule stop; request_stop and the
scheduled capability no longer require an origin conversation.
- Repository: single reactivation choke point raising
ScheduledTaskLimitsExhausted; streak boundary moves on goal/prompt/stop
condition/note changes (not on resume); goal-check failures neither count
nor reset the streak; pause of a finished task returns "finished"; trials
on a paused task keep the host pause marker.
- New repository reads: automatic_runs_used_for, active_run_status_for,
list_manageable_from_thread, thread_relation/thread_run on thread tasks,
task_ids_for_thread, run_number, and run_number/total_tokens/summary on
run history rows.
- REST: goal/max_runs/end_at/stop_condition on create and PATCH (null
clears, naive end_at = task timezone), resume recomputes from now with an
optional renewal body, once_time_passed, task_finished, limits_exhausted,
scheduler_not_running on create, trigger returns outcome/existing/thread_id,
task responses add automatic_runs_used and active_run_status.
- Coded errors {code, message, params} for routes, shared validation and the
chat capability (scheduled_task_errors.py), pinned by
contracts/scheduled_task_errors_contract.json; goal notes contract v2 with
check_failure_codes, scheduled_origin_key and host_run_errors (host_notes.py).
- /api/features adds scheduled_tasks {available, running, tool_enabled,
min_interval_seconds}; ScheduledTaskService.is_running.
- Docs: six README scheduler sections, API.md scheduled tasks section
(breaking error shape), CONFIGURATION.md, AGENTS notes, CHANGELOGs.
- Tests: migration 0031, stop rule, error contract, run listing, lifecycle
over HTTP, repository, service and capability cases; updated tests for the
new error shape, finished-task pause and head revision.
* feat(scheduler): conversation update/resume, run-conversation authority and readable launches
- schedule_task: update and resume actions, stop_condition and clear_fields
parameters, plain parameter types (value rules come back as coded results),
rewritten docstrings for schedule_task and stop_scheduled_task.
- Capability: update/resume share the REST validation and error codes;
interactive turns in a task's run conversation may manage that task;
scheduled runs keep only own-stop; grants require a running scheduler.
- Results: compact JSON-safe task views with local times, display card/text,
last run in list, timezone_source/browser_timezone, now_local; no ISO, user,
lease or origin fields; stop_instructions removed.
- Timezone: explicit > browser (context.client_timezone, validated in
start_run and never forwarded to the run config) > not needed for intervals
and offset run_at > timezone_required.
- Trial gate accepts natural direct requests after acknowledgements and
particles; trial results report thread_id and existing.
- Scheduled launches: stable message id scheduled-<task_run_id>, server-owned
deerflow_scheduled_origin metadata with the user-language parts, run thread
title for fresh-thread tasks.
- Worker records goal continuations in the verdict; lead prompt language rule
covers stored task text and scheduled runs.
- Docs: six READMEs, CONFIGURATION, API, runtime/tools AGENTS, CHANGELOGs.
* feat(frontend): scheduled-task core types, formatting, errors and glossary
- throwGatewayApiError now throws GatewayApiError (status, code, params,
rawMessage); message stays the string detail or detail.message
- scheduled-task types gain stop_condition, automatic_runs_used,
active_run_status, origin/notes, run_number/total_tokens/summary, thread
relation, trigger result, renewal and scheduled origin; isTaskBusy and
hasActiveRun helpers
- api: fetchScheduledTask, resume with optional renewal body, typed trigger
result and update payload with null clears
- hooks: polling at 3 s while a run is active (list, thread list, run
history), useScheduledTask with live gating and final 404, useInView,
localized error toasts; run history refreshes tasks when a run finishes
- new pure modules: errors.ts (contract code to localized copy), format.ts
(relative task times, viewer time, display zone), status.ts, actions.ts,
run-error.ts; describeTaskSchedule in cron.ts; goal-outcome adds unchecked
goal checks, describeTaskOutcome and parseStopRunId
- features: scheduled_tasks block with defaults when missing
- buildRunContext sends client_timezone
- titleOfThread takes a localized untitled label; breadcrumb names scheduled
tasks and agents; reference chips use the localized untitled label
- schedule input: shared browserTimeZone, labelled selects, aria-pressed
type toggles, interval minimum from the hint template
- i18n: all PR1 scheduled-task keys in en-US and zh-CN, glossary fixes,
recipe instructions moved into i18n
- e2e mocks follow the new task, trigger, resume, thread-task and features
shapes; expectNoRawIdentifiers helpers for e2e and DOM tests
- tests for errors, format, cron, status, actions, goal outcome, run errors,
hooks, run context, thread titles and features; scheduled e2e copy updated
* feat(frontend): list-first scheduled tasks page with outcomes and editable goals
- page.tsx is now a thin composition of components/workspace/scheduled-tasks:
status tabs (All/Active/Paused/Finished) and search over the task list,
?task_id= deep link with router.replace on selection, ?thread_id= chip
with Show all, empty state, scheduler-off notice and unavailable state
- task list rows show the schedule in words and one status line (next run,
running now, paused by agent, auto-paused, finished by its cap)
- task detail: status badge, actions from availableActions with reasons,
More actions menu (Duplicate, Copy task ID, Delete), outcome notice,
Runs / Stops when / Each run's goal / Does / Notes from chat / History,
origin chat link; success toasts for every action
- run rows: run number or Trial run, summary, goal and assumption badges,
continuations, tokens, Open chat link; raw errors and the evaluator
reason only behind Details
- create/edit/duplicate dialog with goal, stop condition, safety caps and
advanced agent/context options; PATCH sends only changed fields, null
clears a cap
- renew dialog raises or removes caps and resumes in one request; a
limits_exhausted Resume opens it
- removed i18n keys the old page used; added feedback.resumedNoTime
- recorded en-3 and zh-1 page fixtures; scheduled e2e specs rewritten for
the new page; DOM tests for the detail and the form dialog
- CHANGELOG entries and the frontend AGENTS.md scheduled-task note
* feat(frontend): live schedule card, run conversation provenance and header entry
- schedule_task results (create, update, pause, resume, trial, delete) become
an assistant:scheduled-task message group rendered as a live card: schedule,
next run, stop condition with the safety cap, goal, where results go, Run
once now, Pause/Resume and Open task; the card polls the task only while it
is on screen and shows deleted, scheduler-off and limits-exhausted states
- one card per task per assistant turn; the card group is not a turn boundary
for usage, branching, editing, outline or skill attribution
- a human message with deerflow_scheduled_origin stays a human group with
scheduledOrigin and renders as a "Scheduled run / Trial run" block with the
task instructions, stop condition and notes collapsed; the launched text is
never shown and the message is not editable
- readable step labels for schedule_task actions and stop_scheduled_task
- ThreadScheduledTasksButton replaces the header link: hidden without tasks,
shows the count, links to the task (or the chat filter), and a run's chat
links to its task
- recorded en-3 / zh-1 chat and run-thread fixtures; chat-card and run-thread
e2e specs; unit and DOM tests for parsing, grouping, turn boundaries, card,
run block, header button, step labels and reconnect hydration
- i18n: header.countLabelOne added, unused card.next removed; CHANGELOG and
frontend AGENTS.md notes
* fix(scheduler): address verification findings
Move the scheduled-task lifecycle guidance from backend/AGENTS.md into the persistence guide and leave a one-line pointer, so the effective AGENTS.md chains for agents/middlewares, sandbox and subagents stay under the 98304-byte hard limit enforced by scripts/check_agent_guidance.py in lint-check.
* fix(scheduler): address review findings
* fix(scheduled-tasks): polish from live acceptance
- zh schedule wording: an interval of one unit reads 每分钟 / 每小时 / 每秒,
and a saved interval of whole days reads 每天 / 每 2 天 (Every day / Every 2
days) instead of 每 1 分钟 or 每 24 小时; several units keep the number
- zh spacing: 已于{time} 满足 and 在{time} 的运行中 now leave a space between
the interpolated time and the Chinese after it; a copy test guards every
scheduled-task zh string against a placeholder glued to Chinese
- run summary: when the first line of the final reply only leads in to a list
(ends with ":" or "："), the list items follow it as plain text (bullets,
numbering, checkboxes, bold, code and links stripped), joined with "；" for
CJK text and "; " otherwise, within the 160-character cap
- a schedule_task result (create, update, pause, resume, trial, delete) that
arrives in a chat invalidates that chat's task list, the tasks list and the
named task, so the header button and cards update without waiting for the
idle poll; results already in the loaded history do not refetch
- e2e and unit fixtures are now the live acceptance recordings (DeepSeek
Flash, demo data, fictional names): live-weekday.json (create, trial, edit,
pause, resume in one zh chat) and live-minute.json (per-minute chat, run 1
and run 2, paused by the agent), fetched with GET from the demo Gateway and
trimmed; the hand-shaped en-3 / zh-1 fixtures are removed and the chat-card,
run-thread and tasks-page specs run in both locales over the recorded data
* docs(changelog): reference #6378 for the scheduled-tasks entries
* docs(agents): shorten the scheduled-task pointer to stay under the guidance limit
* test(scheduled-tasks): pin the live takes to their recorded time zone
The live fixtures were recorded in Asia/Shanghai, and an interval task
reads in the viewer's zone, so the 16:48 assertions failed on UTC CI
runners. Pin the browser zone in the two live-take e2e specs and the
zh live-take unit test.
* fix(scheduled-tasks): keep untouched end times and drop a chat filter the new task is not in
- The edit, duplicate and renew dialogs keep a stored end time's instant
while its field still shows it. A minute-precision wall value cannot
keep seconds or name the later 01:30 of a DST fall-back, so a
title-only edit moved the cap an hour earlier, and raising the run
limit re-sent the end time without its seconds.
- Creating or duplicating a task from a chat's task view drops the chat
filter when the new task does not belong to that chat, instead of
showing "This task isn't one of this chat's tasks".
* fix(scheduled-tasks): tie pause notices to the pause they describe
- Paused by agent links and times only the run named in last_error. When
that run is not loaded, the notice shows no link and the task's own time
instead of an earlier run that also asked to stop.
- The outcome notice reads the latest history page whichever page is
shown, so browsing older runs no longer swaps the auto-pause reason and
"Open latest run" to an earlier miss. It shares that page's request.
* fix(scheduled-tasks): three lifecycle gaps found while re-checking review classes
- The edit form compares a cron in the form its schedule input emits, so an
untouched "0 09 * * 0,6" is not re-sent as "0 9 * * 6,0" and a title-only
edit no longer re-arms or refuses a finished task.
- Chat resume asks for a timezone, as update does, when a naive end_at would
land in the UTC placeholder of a zone-free task.
- Re-arming a task clears a host pause marker, as resume does, so a later
manual pause no longer reads as "Paused by agent".
* fix(scheduled-tasks): keep the notice's latest-runs query from refreshing older pages
useLatestScheduledTaskRuns kept React Query's default refetch on mount,
focus and reconnect, so going offline and back while an older history page
was shown fetched the latest page again. Refreshing stays with the history,
which refreshes the latest page only while it is shown.
* test(scheduled-tasks): let the viewer's time carry its own day label
Between 16:00 and 24:00 UTC, UTC and Asia/Shanghai are on different days,
so the run prompt shows "Tomorrow 01:00 · Today 09:00 your time" and the
fixed "01:00 · 09:00 your time" assertion failed depending on the hour.
---------
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|
||
|
Nan Gao
|
5872d07e9a
|
fix(gateway): gate declared multi-instance deployments and require the redis stream bridge (#6328)
* fix(gateway): gate declared multi-instance deployments and require the redis stream bridge GATEWAY_WORKERS / WEB_CONCURRENCY only count the uvicorn workers of one process tree, so a Kubernetes Deployment with replicas > 1 (one worker per Pod) passed every multi-worker startup check while each Pod's startup orphan reconciliation wrote the other Pods' lease-less runs off as crashed on every rolling update. Add `deployment.multi_instance` as an explicit declaration that runs the same gate, plus a DEER_FLOW_MULTI_INSTANCE env override so deploy tooling can set the declaration from its replica count without editing config.yaml (wiring the Helm chart to export it is a separate change). Both the worker count and the declaration now also require a Redis stream bridge and refuse an explicit memory sandbox ownership store, since the memory bridge and ownership store are process-local. The agent-storage divergence warning, the inbound webhook dedupe warning and the WeChat QR-login guard honor the same declaration. * docs(changelog): link the multi-instance gate entry to #6328 * fix(ci): bump chart config_version to 53 and keep AGENTS.md under the size limits config.example.yaml moved to config_version 53 for the new deployment section, so the Helm chart's embedded config and its README example follow. The multi-instance notes in backend/AGENTS.md and backend/app/gateway/AGENTS.md shrink to one line each: main already sits within 55 bytes of the per-file hard limit for the gateway guide and within 163 bytes of the chain hard limit under backend/, and the full contract lives in backend/docs/CONFIGURATION.md. * fix(gateway): name every active knob in the multi-process rollback When GATEWAY_WORKERS > 1 and a multi-instance declaration are both active, the refusal offered only "Set GATEWAY_WORKERS=1", so an operator who followed it bounced through a second refusal on DEER_FLOW_MULTI_INSTANCE / deployment.multi_instance at the next start. The rollback now withdraws both knobs, and the browser-tools refusal takes the same path instead of the worker-count-only message. multi_instance_declaration() returns a frozen MultiInstanceDeclaration (source, value) with knob/rollback properties instead of a display string, so deps.py no longer branches on message prose. Tests pin the combined rollback text and that following it actually clears the gate. * docs(gateway): keep the gateway AGENTS.md under the size hard limit after merging main main grew backend/app/gateway/AGENTS.md by 30 bytes, which left it 25 bytes under the 49152-byte hard limit; the one-sentence multi-instance note from this branch pushed the merged file to 49164. Shorten that sentence by 18 bytes so scripts/check_agent_guidance.py passes on the merge result. * ci: re-run checks after test_stop_cancels_a_hung_driver_poll flaked on shard 4 tests/test_mcp_task_service.py::test_stop_cancels_a_hung_driver_poll timed out once on backend-unit-tests (4) for 2c16206ac. The MCP task service and its test are identical to main, main's run for cfe2023df passed, and 15 consecutive local runs pass; the test gives service.stop() a 1s budget while cancellation drain allows 5s, so a loaded runner can exceed it. Fork contributors cannot re-run a job, hence this empty commit. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Hyeonsang Cho
|
0223d949bf
|
fix(threads): remove a deleted thread's files off the event loop (#6319)
* fix(threads): remove a deleted thread's files off the event loop
DELETE /api/threads/{id} called shutil.rmtree over the thread directory
directly on the event loop, so deleting a thread with a large workspace
stalled every other Gateway request and live SSE stream until the whole
tree was gone. Branch copying in the same router already offloads its
filesystem work through run_file_io; deletion did not.
The removal now runs on the file-IO pool. It is wrapped in await_drained
rather than bare-awaited: cancelling the await abandons the worker, not
the removal, so the durable delete reservation would otherwise release
while files are still being deleted, and a newly admitted run could
write into a directory that is about to disappear.
* docs(changelog): reference #6319 in the thread delete off-loop entry
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|
||
|
JasonH
|
22623d1b00
|
fix(projects): validate explicit document names before staging (#6287) | ||
|
Jun
|
57d946b730
|
fix(browser): drain live cleanup before releasing session (#6284)
* fix(browser): drain live cleanup before releasing session * test(browser): cover cancellation precedence during cleanup * style(browser): separate cancellation regressions |
||
|
Daoyuan Li
|
a9046508cf
|
fix(artifacts): bound reads before saving edited outputs (#6250) | ||
|
quifox
|
06e9bfd269
|
fix: propagate current run failures from wait endpoints (#6217)
* fix: propagate current run failures from wait endpoints * docs: clarify checkpoint wait errors and SDK behavior |
||
|
Guo-Yixin
|
68014de157
|
fix(artifacts): invalidate digests after external file replacement (#6195)
* fix(artifacts): invalidate digests after external file replacement * docs(gateway): keep artifact guidance within the module budget * fix(artifacts): validate oversized ranges by file identity --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
xiaodu55
|
6176b057a2
|
fix(mcp): drain the config read-modify-write across cancellation (#6210)
* fix(mcp): drain the config read-modify-write across cancellation Route the five MCP config endpoints through the canonical run_drained_write helper and fold the tools-cache reset into the drained unit, so a cancelled caller cannot detach from the worker holding the extensions-config locks or skip the cache invalidation behind a settled write. Caller-facing HTTPException contracts re-raise unlogged. Fixes #6209 * fix(mcp): type the drain helper against the worker signature and unlist drain adopters - _run_drained_mcp_apply now mirrors the canonical helper's [**P, T] Callable[P, T] typing so each call site's argument list is checked; - AGENTS.md stops enumerating the adopted routers, so the line stops needing an edit on every new adopter. --------- Co-authored-by: xiaodu55 <xiaodu55@users.noreply.github.com> |
||
|
lakers-abing
|
fafe904106
|
fix(scheduled-tasks): honor explicitly null assistant_id (#6185)
Co-authored-by: lakers-abing <236384465+lakers-abing@users.noreply.github.com> |
||
|
yetuge
|
7eb41cfc93
|
refactor(gateway): hoist the persistent-write drain into one shared helper (#6151)
* refactor(gateway): hoist the persistent-write drain into one shared helper Each persistent-write surface forked its own per-router drain wrapper: _drained_write in routers/agents.py (action label, expected_errors pass-through, lost-failure type-only logging), _run_store_mutation in routers/subagents.py (bare drain), and the inline await_drained call in routers/managed_models.py. Hoist the superset into app.gateway.persistent_writes.run_drained_write and point all three routers at it, per the review suggestion on #6087 (its family predecessors #6078/#6092/#6093 have since landed). The subagents and managed-models call sites now also emit the type-only lost-failure log line on a drained-then-failed write (previously silent), matching the tradeoff acknowledged in the #6087 review. artifacts.py and personal_mcp.py keep their direct await_drained forms: the former drains a coroutine with its own rollback flow, the latter needs no label or expected-error handling. Router tests move their monkeypatch targets to the shared name; the lost-failure assertion pins the helper's logger. AGENTS.md points the next persistent-write surface at the canonical implementation. Signed-off-by: yetuge <2219677952@qq.com> * fix(gateway): quiet expected domain errors and scope the AGENTS.md drain claim Review follow-ups on #6151: - The subagents create/update call sites pass (ManagedSubagentExistsError,) and (FileNotFoundError,) so the shared helper stays silent on routine 4xx outcomes (duplicate-name 409, stale-store 404 race), matching create_agent's (AgentExistsError,). - The managed-models save passes (HTTPException,), so the helper stops double-logging failures that _save_with_failure_logging already reports and stays silent on its routine 4xx outcomes. - The AGENTS.md sentence now names the three converted routers instead of implying every persistent-write surface uses the helper; the shortened form also keeps the file under the guidance byte budget that agent-guidance enforces. Signed-off-by: yetuge <2219677952@qq.com> --------- Signed-off-by: yetuge <2219677952@qq.com> |
||
|
PeaceMaker-best
|
90d466c917
|
feat(mcp): propagate cache resets across workers (#6126)
* feat(mcp): propagate cache resets across workers Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> * docs: keep MCP reset guidance within instruction budgets --------- Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> Co-authored-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Totoro
|
d73844b997
|
fix(goal): show the goal evaluator the tool calls and results (#6068)
* fix(goal): show the goal evaluator the tool calls and results The evaluator read only the text of user and assistant messages and was told to fail closed without explicit evidence, so it stood down with missing_evidence on most completed file tasks: the tool calls and tool results that the web UI shows never reached it. Include the assistant's tool calls and the tools' results inside the existing message window, shortened (200 characters per argument value, 600 per step, line breaks kept), and tell the evaluator to treat tool results as data, that a successful tool call alone does not meet a goal, and that a guessed input means needs_user_input. The no-progress and final-clear signatures are unchanged. * fix(goal): tighten the evaluator's tool evidence after review - Pair each tool result with the latest call of its id before it. Some providers reuse ids such as call_0 across turns, and the window-wide map labelled an earlier read with a later write's name. - Keep model-supplied tool names on one line in calls and result labels, and escape U+0085, U+2028 and U+2029 in arguments and results, which json.dumps(ensure_ascii=False) leaves raw, so no tool step can start a new evidence line. - Leave out the results of a hidden assistant message's calls. The web UI renders a result only under its call, so neither shows there either. A clarification prompt stays: the UI shows it as its own card. - Over the 12,000-char cap, keep whole lines from the end instead of slicing mid-line. Keep the latest user message that the cut drops at the top, whole when it and the lines after it fit, else shortened to 2,000 chars. If the next assistant message back does not fit whole, keep its end in the room left. A marker counts the lines left out. - Docs: the Gateway AGENTS.md and a worker docstring no longer say the evaluator never sees tool messages. --------- Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com> |
||
|
NanPan
|
48aec0bc84
|
fix(gateway): drain managed model saves on cancellation (#6024)
* fix(gateway): drain managed model saves on cancellation * test(gateway): pin managed model save cancellation drain * fix(gateway): log failed drained managed model saves Signed-off-by: Willem Jiang <willem.jiang@gmail.com> * fix(gateway): keep guidance within hard size limit --------- Signed-off-by: Willem Jiang <willem.jiang@gmail.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
RongJie G
|
d642dfdd2f
|
fix(mcp): reserve notification launches atomically (#5965)
* fix(mcp): reserve notification launches atomically Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com> * fix(mcp): classify busy notification conflicts by type Signed-off-by: Willem Jiang <willem.jiang@gmail.com> --------- Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com> Signed-off-by: Willem Jiang <willem.jiang@gmail.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Storm
|
1b691689da
|
fix(integrations): keep Lark runtime status consistent (#5994)
* fix(integrations): keep Lark runtime status consistent Probe sandbox runtime readiness for every Lark mutation response outside the credential lock. Expose explicit sandbox_runtime_probed state so the frontend can distinguish conservative fallbacks without inferring semantics from runtime detail text, while cancelling stale reads before updating the cache. * style(frontend): format Lark cache test * fix(integrations): snapshot Lark credential status under the lock Why this change is needed: PR #5994 moved the post-mutation status read outside the per-user credential lock so the new sandbox runtime probe would not extend the critical section. As flagged in review, that lets a concurrent app switch or auth flow land between the committed mutation and the status read; in remote-provisioner mode the runtime probe can spend seconds before probe_lark_auth runs, so the raced switch can clear the new tokens and make a completed login report success=false, or mix an old app_id with a newer auth state. What changed: - add _LarkCredentialSnapshot (app config + auth probe) and capture it in the same _lark_credential_lock section that commits the mutation - thread the snapshot through _get_lark_mutation_status into get_lark_integration_status(credential_snapshot=...); only the credential-independent runtime probe still runs after the lock - apply to complete_lark_auth, complete_lark_config, and set_lark_app_credentials; install keeps the one-shot status build Problem solved: Mutation responses stay tied to the just-committed credential generation: a raced app switch can no longer flip a completed device-code login to success=false or blend a stale app_id with newer auth state, while the runtime probe stays off the lock. * fix(integrations): address Lark status review follow-ups - install_lark_integration: snapshot credential-derived status fields under _lark_credential_lock so the install response cannot mix newer credential state read across the unlocked runtime probe - useInstallLarkIntegration: route the cache write through cacheLarkMutationStatus so install cancels in-flight reads and keeps probed runtime fields under frontend/backend version skew - README: soften the UI claim - the status API exposes sandbox_runtime_probed and the mutation cache honors it * test(integrations): tighten Lark status fixtures and guidance docs - cache.test: use a distinct mutation runtime mode in the unprobed-merge case so the assertions prove the cached runtime fields win - e2e mock: include broker in the sandbox_runtime_mode union cast so the fixture cannot drift from LarkSandboxRuntimeMode - AGENTS.md (gateway + skills): mention sandbox_runtime_probed alongside the existing runtime readiness fields, keeping the gateway guidance file under its byte budget * fix(integrations): harden Lark mutation cache against skew and races - cacheLarkMutationStatus: await cancelQueries before writing so an in-flight GET whose fetch already resolved cannot clobber the fresher mutation status, matching the existing threads/archive pattern - self-heal version skew: an unprobed (pre-field backend) mutation response now triggers a status refetch instead of risking the conservative runtime fallback lingering until the next focus/remount - api: normalize a missing sandbox_runtime_probed to false at the boundary so the declared-non-optional type matches skewed runtimes - tests: pin the skew contract (missing-field cases for cache and api) and add a useInstallLarkIntegration regression test proving the install cache write goes through cacheLarkMutationStatus - README: describe the probed flag as cross-version semantics instead of claiming the same-version API surfaces both states - lark_cli: note that install's credential snapshot runs the auth probe subprocess while holding the credential lock * refactor(integrations): derive sandbox_runtime_probed inside the resolver _resolve_sandbox_runtime_readiness now returns a fourth probed element next to the branch that produces the unevaluated remote fallback, so the 'was readiness actually evaluated' invariant can no longer drift from the fallback logic it mirrors. Callers and resolver-level tests updated to pin probed per branch. |
||
|
Wenchao An
|
52a3e2564f
|
fix(mcp): isolate personal connections from deployment configuration (#5966)
* fix(mcp): isolate personal connections from deployment configuration * fix(mcp): preserve personal task ownership and repair plugin E2E fixtures * fix(mcp): handle corrupt personal config and cache owned calls * fix(mcp): pin personal endpoint addresses and handle invalid listings * fix(mcp): recheck current authority for privileged personal connections |
||
|
RongJie G
|
a3d75e843f
|
fix(gateway): distinguish bridge failures from disconnects (#5623)
* fix(gateway): distinguish bridge failures from disconnects * fix(gateway): harden SSE terminal disconnect fencing --------- Co-authored-by: CorgiBoyG <CorgiBoyG@users.noreply.github.com> |
||
|
NanHan Qing
|
2377a49217
|
fix(gateway): close pooled MCP sessions on shutdown (#5961)
* fix(gateway): close pooled MCP sessions on shutdown The Gateway lifespan closes the browser session manager on the way out but never closed MCPSessionPool, so every pooled MCP session was abandoned when the worker exited: the owner task's transport __aexit__ never ran and a stdio MCP server or SSE/HTTP connection had no orderly teardown. MCPSessionPool.close_all() already existed for exactly this and had no production caller -- only the config-retirement path used close_all_sync(). The async form is the right one here: close_all_sync()'s own docstring directs callers running inside a live event loop to await close_all() instead of signalling and returning. Wired into lifespan shutdown with the same bounded, best-effort shape as the neighbouring browser-session hook. * fix(gateway): close MCP sessions after run drain --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Bxl
|
41e8e75f56
|
fix: balance temporary sandbox leases for uploads (#3261)
* fix: balance temporary sandbox leases for uploads * fix: address sandbox upload review feedback --------- Co-authored-by: Bxl <188532127+BXL1015@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
YxinMiracle
|
7d9025c052
|
fix(sandbox): scope viewed image host reads to the current thread (#5824)
* fix(sandbox): scope viewed image host reads to the current thread * docs(gateway): trim agent guidance to fit CI limit * fix(images): honor thread data paths for saved host copies * test(images): align authorization fixture with thread paths --------- Co-authored-by: YxinMiracle <“939157765@qq.com”> |
||
|
AochenShen99
|
5804ee1a7e
|
fix(suggestions): enforce model use authorization (#5816) | ||
|
NanPan
|
fc26204deb
|
fix(gateway): drain artifact updates across cancellation (#5755)
* fix(gateway): drain artifact updates across cancellation * fix(gateway): log cancelled artifact commit failures * refactor(gateway): tighten artifact cancellation follow-up * docs(gateway): trim artifact guidance * docs(gateway): retain XSL download guidance |
||
|
Wenchao An
|
2a9beb34b9
|
feat(plugins): support manifests and static asset directories (#5685)
* feat(plugins): support manifest-backed browser assets * fix(plugins): validate asset roots and clarify module timeouts * fix(plugins): sandbox asset documents and support Turbo imports * docs(gateway): keep guidance within the merged size budget --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
RongJie G
|
e01314442c
|
fix(mcp): scope sessions and task access by thread incarnation (#5556)
* fix(mcp): scope sessions and task access by thread incarnation * fix(mcp): preserve thread incarnation in delegated subagents * fix(mcp): preserve incarnation in durable batches * fix(mcp): bind standalone graph lifecycle context * fix(studio): preserve implicit thread creation metadata --------- Co-authored-by: CorgiBoyG <CorgiBoyG@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Hyeonsang Cho
|
6db7e8091f
|
docs(gateway): bring the gateway guide back under its hard budget (#5707)
backend/app/gateway/AGENTS.md reached 49,155 bytes against the 49,152-byte hard limit, so tests/test_agent_guidance_check.py fails on main for every change, not just ones that touch the file. This trims wording I added to the Uploads row in #5547, #5611 and #5673 rather than anyone else's documented invariants: the delete clause keeps both facts — symlinks 404, a converted .md is kept — without restating that the 404 matches GET /list. That clears the overage with 13 bytes to spare, which is not much. The file is effectively full, so the next addition needs a real slimming pass or a budget decision. |
||
|
YxinMiracle
|
b8ab097ca2
|
fix(models): support official DeepSeek managed profiles (#5718)
Co-authored-by: YxinMiracle <“939157765@qq.com”> |
||
|
Copilot
|
6b0ca6eb8d
|
Fix backend-unit-tests failure by bringing gateway AGENTS.md under hard size budget (#5725)
* Initial plan * docs(gateway): trim AGENTS guidance to stay within hard budget Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com> |
||
|
Keswyk Cheung
|
ef21855b8d
|
fix: discard stale todo reminders during context compaction (#5614)
* fix: rebuild todo reminders after context compaction * chore: remove implementation plan from PR * refactor: share todo reminder message name * docs: keep agent guidance within CI size budgets --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Hyeonsang Cho
|
90f0866fa8
|
fix(uploads): stop deleting a converted companion we cannot prove we wrote (#5673)
* fix(uploads): stop deleting a converted companion we cannot prove we wrote Conversion names a document's Markdown companion after the document's stem and falls back to a _N suffix when that name is already taken, so the .md beside a document may belong to another document sharing the stem, or to the user. Delete removed it anyway: uploading a.docx and a.pdf produces a.md and a_1.md, and deleting a.pdf destroyed a.docx's companion while orphaning a.pdf's own. Delete now removes only the file it was asked to remove. The companion stays listed and can be deleted on its own. Orphans are the cost of not guessing; issue #5672 covers giving companions a provable owner, which is what a safe cleanup needs, along with the two related readers that still guess (the outline injected for a document and the agent's file listing). convertible_extensions loses its last use and is dropped from the signature and both call sites. The gateway router keeps importing CONVERTIBLE_EXTENSIONS for the ingestion bridge and now declares it in __all__, where that module documents its re-exports. * docs(changelog): note that delete keeps the converted markdown (#5673) |
||
|
Wenchao An
|
e2f19d8335
|
feat(plugins): full-stack plugin APIs and bookmarks (#5647)
* feat(plugins): add full-stack contributions and bookmarks example * ci(plugins): provision bookmark gateway for browser tests * fix(plugins): authenticate module downloads through configured backend * fix(plugins): isolate contributions and localize extension UI * fix(plugins): preserve bookmark agent routing and contain async callbacks * fix(plugins): pin durable batch workers to app extension snapshots |
||
|
YxinMiracle
|
ce50a28dfd
|
fix(auth): enforce write permission for Live Browser WebSockets (#5621)
* fix(auth): enforce write permission for Live Browser WebSockets Resolve route permissions before accepting browser streams and require threads:write, matching the existing HTTP navigation endpoint. Preserve shared authorization failure semantics and reject unexpected setup errors before acquiring a browser session. Add authorization, frame delivery, input dispatch, cancellation, and ownership regressions. Document the admission-only permission check. * fix(auth): improve browser authorization diagnostics --------- Co-authored-by: YxinMiracle <“939157765@qq.com”> |
||
|
Hyeonsang Cho
|
29d285731b
|
fix(uploads): convert the bytes we wrote, not the name they landed under (#5611)
* fix(uploads): convert the bytes we wrote, not the name they landed under Document conversion re-opened the upload by name after it was already visible in the thread's uploads directory: the Gateway converted the committed file_path, and DeerFlowClient converted the copy it had just placed there. That directory is writable from local and AIO sandboxes, so a process watching it can replace the name with a symlink in the window between the upload landing and the converter opening it. The converter then reads whatever host file the link points at and writes that content back into the thread as the .md companion, which the sandbox can read. Reproduced end to end on both paths with a real xlsx: the companion came back holding the host file's rows. The Gateway now duplicates the descriptor of the staged file before the link-commit, copies those bytes into a private directory outside the uploads tree, and converts there. A descriptor cannot be redirected by replacing a name, so the conversion input is the content this request wrote. The client converts the caller's own source file instead of the copy in uploads; the source is the file the caller handed in, which the sandbox cannot reach. Both already wrote the companion without following a symlink, so only the read side changes. The uploads directory still receives exactly the same files. * docs(changelog): note upload conversion source fix (#5611) * fix(uploads): close the conversion descriptor when staging its copy fails Review follow-up. The private directory for the conversion copy was created before the try that owns the duplicated descriptor, so a failure there — a full or unwritable temporary filesystem — propagated without closing it. The upload's own cleanup only unlinks the committed name and releases the sandbox lease, so the descriptor stayed open for the life of the process and kept the unlinked staged bytes allocated with it; repeated failures accumulated both. Directory creation now happens inside that try, and the finally removes the directory only once it exists. * fix(uploads): keep the conversion descriptor owned across cancellation Review follow-up. run_file_io cannot interrupt its worker, so cancelling the await around os.dup only abandoned the result: the duplicate was created moments later with nothing left to close it, and it pinned the staged bytes of an upload whose name the cleanup had already unlinked. Cancellation after the duplication was just as leaky, because the commit-path handler caught Exception and CancelledError is not one. The duplication now runs as its own task, shielded from the caller's cancellation, and closes its own result when the caller is gone by the time the worker finishes. The commit path catches BaseException, closing the descriptor it already owns before re-raising. Both windows are pinned: one test stalls the duplication worker after it allocates and cancels ingestion, the other stalls the commit so the cancellation lands while the descriptor is owned. * fix(uploads): drain the conversion copy so its descriptor always closes Review follow-up. The copy worker owns the duplicated descriptor and closes it in its own finally, but a bare await let a cancellation cancel the executor job while it was still queued: the worker never ran, so that finally never ran either, and the enclosing scope had already handed ownership away and saw None. Draining also keeps a late worker from writing into a private directory this scope has since removed. The copy now goes through await_drained, the shield-and-drain helper the Gateway already uses for offloads that must not be abandoned mid-flight. Pinned by a test that holds the copy job queued, cancels ingestion, then releases it and requires the descriptor to come back closed. |