mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-29 22:50:37 +00:00
274 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Tsai Yuan
|
966aa9496a
|
fix(benchmark): validate sandbox timing options (#7651)
* fix(benchmark): validate sandbox timing options * fix(benchmark): guard infinite idle reaping --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
jianyx1
|
4b1e06bbf9
|
feat(runtime): honor the web UI language in replies and DeerMem updates (#7294)
* feat(runtime): honor the web UI language in replies and DeerMem updates Carry Settings > Appearance > Language as request-only run data: the Gateway validates context.locale against a finite allowlist, the lead agent prefers it for replies, and bundled DeerMem writes new facts and summary prose in it. Explicit user language requests still take precedence; checkpoint configurable, static prompts, and memory backend APIs are unchanged. Refs #7287 Co-Authored-By: Claude <noreply@anthropic.com> * fix(runtime): guard language context and pin locale maps * test(runtime): pin UI locales across frontend and Gateway Add the shared locale fixture and document the intentional language fallback for internal runs. Clarify live-locale precedence in the run context contract. --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
ordinary being
|
8bda7c1094
|
fix(scheduler): keep manual-trial launch origins stable across retries (#7492)
* fix(scheduler): keep manual-trial launch origins stable across retries A manual trial's synthetic queued row omitted scheduled_for, and _launch_provenance used the current now for manual triggers. A retry at a later time therefore produced a different origin, which is part of start_run's per-occurrence idempotent input: the retry either conflicted with the first attempt or submitted a different request. Include the admitted scheduled_for in the synthetic row and use the occurrence time for every trigger, falling back to now only when a legacy caller supplies none. Tests: a manual queued row retried at two later clocks produces the same origin and keeps the admitted scheduled_for; existing manual/scheduled provenance and title tests stay green. Full scheduler suite: 1108 passed / 130 skipped; guidance 0 errors / 0 warnings; ruff clean. * test(scheduler): drive manual retry through requeue and drain The unit provenance test pins the stable-origin function directly. Add a driven case that admits a manual occurrence, hits a busy-thread ConflictError so the row requeues, then drains it at a later clock and asserts both launches carry the admitted scheduled_for and the same full origin. * test(scheduler): pin the immediate queue row in the retry test Capture the queued mappings passed to _attempt_queued_run so the driven test asserts both halves explicitly: the synthetic immediate-launch row and the durable row drained later both retain the admitted scheduled_for. --------- Co-authored-by: Tong-bit-art <Tong-bit-art@users.noreply.github.com> |
||
|
spud
|
e9074b7126
|
feat(runtime): publish owner-confirmed run.terminal evidence (#7134)
* feat(runtime): publish authoritative run terminal events * fix(runtime): preserve terminal attribution diagnostics * docs(runtime): consolidate terminal event docs and trim agent guidance --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Kiko0301
|
3138ac42cc
|
fix(gateway): preserve IDs for id-less run inputs (#7349)
* fix(gateway): preserve IDs for id-less run inputs * test(runtime): avoid narrow lease deadline in rollback test --------- Co-authored-by: Kiko0301 <191240073+Kiko0301@users.noreply.github.com> |
||
|
Huang-404-Q
|
c0ba948825
|
fix(auth): treat a signed token with wrong claims as malformed (#7266)
* fix(auth): treat a signed token with wrong claims as malformed
decode_token caught jwt.PyJWTError but not pydantic.ValidationError, so a
correctly signed token without sub/exp raised out of the function. The
OIDC state cookie shares jwt_secret and has exactly that shape, and the
OAuth start endpoint is public, so an unauthenticated caller could turn
it into a 500 (or a broken WebSocket handshake) instead of a 401.
Catch ValidationError and return TokenError.MALFORMED.
The regression test signs an OIDC-state-shaped payload and asserts
MALFORMED; it raises on main.
* fix(auth): treat a signed OIDC state cookie with bad claims as unusable
_verify_state_signed had the same latent shape as decode_token: it caught
only jwt.PyJWTError, so a state cookie signed with jwt_secret but missing
required claims raised pydantic.ValidationError straight into the public
/api/v1/auth/callback/{provider} endpoint instead of returning None and
letting the handler answer 403.
Also pin the decode_token precedence a reviewer raised: an expired token
that is missing sub reports EXPIRED because jwt.decode raises before the
payload model runs.
* docs: reduce inherited backend guidance size
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|
||
|
Xuehao Xu
|
29ba806fc6
|
feat(knowledge): add independent Wiki and entity relation retrieval tools (#7171)
* feat(knowledge): add independent Wiki and entity relation retrieval tools * fix(knowledge): address review diagnostics and CI catalog coverage * fix(knowledge): deduplicate graph evidence and update catalog e2e * fix(knowledge): clarify bounded evidence and pin browser CI runners * Update e2e-tests to run on ubuntu-latest * Update workflow to use ubuntu-latest * test(ci): stop pinning maintainer-controlled browser runner images --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Dragon ze
|
d85e0e8608
|
fix(channels): keep staging cleanup from replacing the save error (#7106)
* fix(channels): keep staging cleanup from replacing the save error The channel stores publish state through a staged temporary file and, if the rename fails, close and unlink that file inside the handler before re-raising. Neither step was guarded, so a failing close skipped the unlink and a failing unlink replaced the error that actually caused the save to fail -- leaving the staged copy, which for the runtime-config and WeChat stores holds an 0600 IM token, behind with nothing reporting it. Make both steps independently best-effort, name the leaked file in a WARNING, and suppress the diagnostic itself so a broken log handler cannot become the reported failure. This is the same contract PR #6871 establishes for browser captures, applied to the three JSON publishers in app/channels. * fix(channels): address staging cleanup review follow-ups --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Willem Jiang
|
a2fcfe0d9b
|
docs: keep inherited backend guidance within budget (#7087) | ||
|
ShiYuan Xiong
|
2254ec6ed5
|
feat(benchmark): add OpenSandbox provider coverage (#7059) | ||
|
Xuehao Xu
|
201b9840cd
|
feat(knowledge): unify read-only retrieval and source contracts (phase 1) (#6716)
* feat(knowledge): unify read-only retrieval and source contracts * fix(knowledge): address citation and contract review feedback * fix(knowledge): resolve citations in delegated task reports * docs(frontend): keep agent guidance within CI size budget * docs(backend): preserve merged guidance chain headroom |
||
|
Totoro
|
cb90f5dca4
|
feat(channels): control IM notifications and add WeChat/QQ delivery (#6850)
* feat(channels): control IM notifications and add WeChat/QQ delivery * fix(scheduler): skip notification preference locks without recipients --------- Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
ZJPex
|
e4c893f2fb
|
feat(acceptance): verify required CSV header columns (#6714)
* feat(acceptance): verify required CSV header columns * fix(acceptance): address CSV header review feedback * fix(acceptance): guard CSV criteria and malformed headers * test(acceptance): isolate CSV deadline clock simulations |
||
|
huyengiang101086-spec
|
6783073a39
|
fix(image-search): share public result URL screening (#6740)
* fix(image-search): screen DuckDuckGo result URLs * fix(image-search): unify provider result URL screening * docs(image-search): scope URL guidance to avoid inherited size limits --------- Co-authored-by: huyengiang101086-spec <270684046+huyengiang101086-spec@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
xiaodu55
|
22ddb77f83
|
docs(agents): trim restated rationale to restore AG002 chain headroom (#6759)
* docs(agents): trim restated rationale to restore AG002 chain headroom The agents/middlewares AGENTS.md chain crossed the 96 KiB AG002 hard limit at 3f7a0769 (98441 > 98304 bytes); that push produced no lint-check run, and later pushes compare equal to their base, so main stayed green while a full-worktree check_agent_guidance run reports an error. Trim restated rationale: - backend/AGENTS.md: the Feishu button-card sentence restated semantics app/channels/AGENTS.md already owns (originator-validated, one-time, fail-closed clicks); keep the flag visibility and point at the module guide. - agents/middlewares/AGENTS.md: drop the membership-probe rationale duplicated by the model_response.py paragraph, the ToolOutputBudget ordering narration, and the lead-only append note restated by the lead-only middlewares heading. Chain: 98441 -> 98198 bytes (106 under the hard limit). * docs(channels): own the Feishu card fail-closed invariant in the module guide Review follow-up on #6759: the backend/AGENTS.md trim dropped two things app/channels/AGENTS.md item 7 does not own - the fail-closed-across-workers guarantee and the owning file's name - so the invariant left every instruction chain and the pointer dead-ended one level above the implementation. State both in item 7, paid by dropping the therefore-clause restated by the single-active-owner invariant above it (the partial unique index and the losing-writer retry already imply deterministic resolution). channels/AGENTS.md: 49134 -> 49129 bytes (AG001 headroom 18 -> 23). --------- Co-authored-by: xiaodu55 <xiaodu55@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Storm
|
3f7a076972
|
feat(channels): Feishu interactive clarification cards (#6670)
* feat(channels): render Feishu option clarifications as button cards When the agent's ask_clarification carries options, the Feishu channel now renders the question as an interactive card with one button per option instead of plain text. This is the outbound half: card construction plus a process-local pending registry that a later card-action callback handler will validate clicks against. - New feishu_card_interaction module owns request extraction, button card rendering, and the pending registry (30-minute TTL, matching the typed-reply clarification window) - Manager attaches the clarification's structured human_input payload to the final outbound's metadata so channels can render richer affordances without touching the Gateway protocol - Feishu channel renders the button card only when card_interaction_enabled is set (default off); free-text, form, and over-four-option clarifications keep the plain-text path - Button cards bypass the markdown wrapper via a raw_card_content flag on the card send/patch helpers; plain-text call signatures unchanged * feat(channels): close the Feishu button-card click loop The inbound half of interactive clarification cards: a click on an option button validates, resolves the clarification, and marks the card in place. - Register the card-action callback only when card_interaction_enabled is set; clicks arrive as WS EVENT frames through the SDK's callback processor map, and the synchronous return value becomes the toast ack - consume_if_originator atomically validates operator identity against the pending entry: non-originators get a refusal toast and the entry survives; unknown or expired requests are toast-only and dropped - A validated click consumes once, updates the card to a terminal resolved state, and injects a hidden inbound carrying the human_input_response so the run resumes on the originating thread with Web-card parity - Manager translates the inbound's human_input_response metadata into the human message's additional_kwargs with hide_from_ui set * docs(channels): document Feishu interactive clarification cards - README / README_zh: optional card_interaction_enabled setup, calling out that card.action.trigger lives under the console's callback configuration tab (separate from event subscriptions) and requires publishing a new app version - config.example.yaml: commented flag with the same console caveat - backend/AGENTS.md: module ownership note for feishu_card_interaction * fix(channels): close Feishu card-click routing and ack gaps from review Address review on #6670: - Carry the routing identity (topic_id, connection_id, owner_user_id) on the pending entry and the synthetic inbound, so clicks resolve to the clarified thread in topic-scoped chats and pass bound-identity admission instead of opening a new thread or being rejected - Validate option_value against the registered options at consume time: an unknown value is treated like a stale card and the entry is kept - Return a warning toast when a click cannot be dispatched, and flip the card to its resolved state only after the inbound is durably in the bus — never display a choice that was not recorded - Drop the dead mark_sent/consume registry APIs and pass the payload's own option ids through to the buttons - Add an end-to-end test through the manager's thread resolution pinning that a card answer reuses the clarified thread * fix(channels): carry workspace_id on the Feishu card-click inbound The connections-enabled admission path re-reads the binding with the message's workspace_id, and Feishu connections are bound with workspace_id = chat_id. The synthetic inbound set connection_id and owner_user_id but never workspace_id, so the re-read missed and every click was rejected as unbound after the entry was consumed. - Set inbound.workspace_id from the entry's chat_id (Feishu's chat-as-workspace binding convention) - Extend the card-click e2e to require_bound_identity=True, driven through the channel's real _resolve_card_click production * fix(channels): stabilize Feishu clarification delivery and routing Option cards could fail through the real reply SDK, resume another conversation, or leave a stale text hint after the answer was accepted. Unify raw-card delivery and persist trusted thread aliases before showing buttons. Keep protocol request IDs separate from bounded, one-time callback tokens; validate originators and option pairs, and retire only the accepted card's text/button state. Treat valid hidden human answers as turn boundaries so old replies and artifacts do not leak into the next answer. Cover consecutive clarifications and /new routing through real SDK request models, stores, and the run journal. Validation: 147 focused tests and 29,421 offline tests passed (405 skipped); strict blocking-I/O passed 245 tests (1 skipped). Ruff and guidance checks passed. * fix(channels): preserve clarification context in Feishu cards Button cards replaced the full clarification text but omitted its background, leaving users without the context needed to choose an option. Carry optional context from request metadata and render its original Markdown before the question. Keep question-only cards for absent, blank, or invalid context. Cover rendering and all real SDK delivery paths, and update both user guides. Validation: 156 focused tests, 245 strict blocking-I/O tests, Ruff, and guidance checks passed. The offline run recorded 29,381 passed, 49 failed, and 405 skipped; 48 import failures reproduced before the fix. All 49 cases passed on both versions in isolated environments, and the Feishu-to-export order check passed 13 tests. |
||
|
Storm
|
588ad010c0
|
fix(mcp): stop persisting URL-sourced ResourceLink blocks and heal bricked threads (#6606)
* fix(agents): sanitize persisted URL content blocks before model calls
Threads whose checkpoints carry URL-sourced file blocks (written before
the MCP conversion-layer fix) fail every model call at serialization: the
Chat Completions translator rejects any file block carrying a url, and the
error handler answers each later turn with a non-retriable fallback. Add a
read-time sanitizer that rewrites those blocks in the model-bound request
view only, so poisoned threads heal on their next turn with no migration
and no user action.
- ModelContentCompatibilityMiddleware wraps the model call (sync and
async) and rewrites ToolMessage list content: URL-sourced file blocks
become "[Resource ({mime or unknown type}) available at {url}]" text
blocks, and image blocks whose scheme is not http/https/data get the
same placeholder. Fetchable image URLs, base64 payloads, text, and
non-ToolMessage messages pass through. The rewrite is order-preserving,
idempotent, and never writes state or mutates checkpoints, so artifact
capture and other state readers keep seeing the original blocks.
- Register on all three chains that replay checkpointed ToolMessages:
build_middlewares (lead; outer of the MODEL_PHYSICAL extension anchor so
its final-request guarantee holds), the SDK factory
_assemble_from_features (new step 13a; docstring updated and its stale
omission of TokenBudgetMiddleware fixed), and
build_subagent_runtime_middlewares (after DurableContext/summarization,
before the date/coalescer tail).
- Subagent coverage verified by tracing every subagent agent-creation
site: executor.py _create_agent is the only caller of
build_subagent_runtime_middlewares and serves both the task tool and the
durable batch worker (SubagentRuntime itself only carries dependencies),
and a wiring test now asserts the middleware is present in that
builder's output; the lead and factory chains gained the same wiring
assertions.
- Update the chain-order contract tests for the new slot: full-chain
order, loop-detection adjacency, custom loop-detection tail positions,
and the subagent chain count (22 -> 23).
* fix(mcp): stop emitting URL-sourced file blocks from ResourceLink results
MCP ResourceLink conversion produced a URL-sourced file content block for
every non-image link, which LangChain's Chat Completions translator rejects
at serialization time. Once such a block was checkpointed in the
ToolMessage, every subsequent turn of the thread failed at model-call time,
permanently bricking the conversation.
- Downgrade every ResourceLink that is not an http(s) image to a short text
placeholder carrying the resource name, MIME type, and the
already-virtualized URL, so host paths never leak and the model still
learns a resource was produced and where it lives.
- Preserve each downgraded link as structured data in the tool-message
artifact under resource_links (merged with structured_content when
present) for artifact capture and future MCP Apps rendering.
- Keep http(s) image links as image blocks; embedded base64 payloads, text
path rewriting, and the error-to-ToolException path are unchanged.
- Re-contract the file-migration tests to the new behavior and document
the conversion contract in the MCP AGENTS.md.
* docs(middlewares): add ModelContentCompatibilityMiddleware to the chain inventory
Insert the read-time sanitizer at its actual lead-chain position (after the
token-budget guard, before the custom-middleware injection point) and renumber
the tail entries 34-39 to 35-40.
* fix(agents): rewrite file blocks on url key presence, not truthiness
langchain-core's OpenAI translator raises for ANY file block carrying a
url key — an empty string or None included — so the read-time sanitizer
must not let such blocks through just because the value is falsy. A
persisted block with url="" or url=None would still brick every later
model call in the thread.
- Gate the file-block rewrite on "url" in block instead of a truthiness
check; a non-empty string URL keeps the
"[Resource (mime) available at url]" placeholder, while an empty or
None URL gets a location-less "[Resource (mime)]" placeholder, so no
unusable location is advertised to the model.
- Pin both shapes with new middleware tests (url="" and url=None file
blocks are both rewritten to text). The image branch is unchanged: an
empty-image case fails provider-side with a 400, not a bricked thread.
* fix(mcp): keep data URIs out of model text and resource_links
A data: scheme ResourceLink fell into the generic placeholder branch,
inlining megabytes of base64 into the model-visible text and into
resource_links, which is checkpointed state — against the repo policy
that data:/blob: URIs never enter state.
- Special-case data: links in _convert_call_tool_result: the
model-visible placeholder becomes "[Resource: name (mime) embedded
inline]" and the raw URI is not added to resource_links at all, so the
payload can neither bloat the context window nor persist in the
checkpoint.
- Pin the behavior with a conversion test asserting both the placeholder
wording and the absence of the payload/URI from every emitted block and
the artifact.
- Add the missing passthrough pin for an HTTPS:// uppercase-scheme image
link: scheme matching is case-insensitive, so the link stays an image
block (the block's URL validation normalizes the scheme to lowercase).
* fix(tools): consume resource_links in extract_artifacts_from_result
The MCP conversion layer preserves downgraded ResourceLinks as
{"name", "uri", "mime_type"} entries under the tool message
artifact's resource_links key, but the artifact extractor only read
structured_content — a result whose only artifact signal was
resource_links produced no entries and the file silently vanished from
the artifacts panel.
- Add a fourth extraction path: each resource_links entry becomes a file
artifact entry, reusing _is_referenceable_url like the
structured-content and content-block paths, so data:/blob: and other
unresolvable URIs stay rejected per existing policy. The entry is named
after the link when a name is present, falling back to the URI's final
segment; a string mime_type is carried through.
- Integration-test a result whose only signal is resource_links, plus the
referenceability gate (data:, ui://, and absolute-path URIs in one
artifact).
* docs(mcp): update the ResourceLink conversion contract bullet
Mention the new artifact-registry consumer and the data: special case so
the bullet stays accurate: uppercase http(s) schemes match
case-insensitively, data: URIs get the embedded-inline placeholder and
are dropped from resource_links, and extract_artifacts_from_result turns
resource_links into registry entries through the shared
referenceability gate.
* refactor(tools): add the canonical resource placeholder formatter
Both the MCP conversion layer and the read-time
ModelContentCompatibilityMiddleware downgrade model-incompatible resource
references to a text placeholder, but each built its own literal. Extract
the single formatter they will share so a thread healed at read time shows
the model the same placeholder shape a fresh conversion produces; the name
segment is optional because persisted content blocks carry no resource name.
* fix(mcp): keep host paths out of ResourceLink placeholder text
Review follow-up on the ResourceLink conversion contract. The placeholder
embedded the resolved URL unconditionally, so a link outside the user-data
tree (or converted without thread context) leaked a raw host path — a
file:// URI or bare filesystem path — into model-visible text, violating
US-16. Compute the resolved URL's scheme and withhold the location segment
when it spells out a host filesystem location (empty scheme, file, or a
single-letter Windows drive scheme); virtual paths and remote schemes keep
their location. The unresolved URI still lands in the artifact channel,
which the model never sees.
- add _model_visible_link_location gate and route the generic placeholder
through it, re-contracting the outside-user-data and no-context tests to
the location-less shape and pinning the Windows-drive case
- treat blob: like data: as non-referenceable inline input: drop its raw
URI from checkpointed resource_links and emit the location-less
placeholder instead of the mismatched 'embedded inline' wording, and fix
the comment that claimed blob: was already excluded
- replace every inline placeholder f-string in the ResourceLink branch
with the shared resource_placeholder_text formatter so the conversion
layer and the read-time middleware emit byte-identical shapes; data:
links now render in the canonical location-less form
- sync the ResourceLink conversion contract bullet in mcp/AGENTS.md and
the _convert_call_tool_result docstring with the host-path suppression
rule and blob: handling
* fix(agents): sanitize URL content blocks on any message role
Address the second-pass review of the request sanitizer with two
changes that share the same code path.
- Adopt the canonical resource_placeholder_text formatter from
deerflow.tools.resource_placeholder instead of the module-local
placeholder helpers, so the read-time rewrite and the MCP conversion
layer emit byte-identical placeholder text from one definition.
- Widen the rewrite from ToolMessage-only to every request message with
list-form content. User uploads are verified to produce base64 blocks,
but imported or cross-client history can carry a URL-sourced file
block on a HumanMessage or AIMessage, and the Chat Completions
serializer rejects such a block regardless of the role carrying it.
The rewrite stays request-view-only: state and checkpoints are never
mutated, and base64/file-id blocks and string content pass through
untouched on any role.
* docs(agents): state the ModelContentCompatibility placement contract
The comment above the registration claimed the sanitizer sits just
before Clarification. In the lead chain it is actually followed by the
custom/extension injection point and the whole terminal tail
(terminal-response, finish-reason, and the always-last
ClarificationMiddleware). Describe the real contract: after every
built-in content transform, before extension injection and the
terminal tail.
* docs(middlewares): clarify provenance stamps for request-view-only rewrites
The message-provenance rule read as if every rewrite needs a
provenance_kwargs stamp. Add one line noting the exception: rewrites
that only modify the model-bound request view and never enter state
need no stamps, citing InputSanitizationMiddleware,
DanglingToolCallMiddleware, and ModelContentCompatibilityMiddleware as
precedents.
* fix(agents): keep host paths and data URIs out of healed placeholders
The read-time ModelContentCompatibilityMiddleware rewrote URL-sourced
file blocks and non-fetchable image blocks into placeholders that
inlined the block's URL verbatim. Pre-fix checkpoints — exactly the
threads this middleware exists to heal — can carry a file:// URI, a bare
or Windows host path (links that never resolved to virtual paths), or a
data: URI embedding megabytes of base64, so healing leaked deployment
filesystem layout into prompts (US-16) and could inject the whole inline
payload into model-visible text.
The conversion layer already suppresses those forms via
_model_visible_link_location. Move that gate into the shared
deerflow.tools.resource_placeholder module as model_visible_location
(now also collapsing data:/blob: URIs) and route both middleware
branches through it, so both layers apply identical suppression rules
with one definition. Conversion behavior is unchanged: its data:/blob:
branch short-circuits before the gate, and the gate covers those schemes
for defense in depth.
- resource_placeholder.py gains model_visible_location alongside the
shared formatter; mcp/tools.py drops its private copy and imports it.
- The middleware's file and image branches pass the URL through the gate
before building the placeholder; module docstring documents the rules.
- Tests: gate unit tests (virtual/remote visible; file://, bare/Windows
host paths, data:/blob: withheld) and middleware regression tests
pinning that a pre-fix data: payload, file:// URI, and Windows drive
path heal to location-less placeholders.
- AGENTS.md bullets in mcp/ and middlewares/ state the shared gate.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(agents): downgrade malformed image URLs instead of raising in the sanitizer
The read-time ModelContentCompatibilityMiddleware gated persisted image
blocks on urlparse(url).scheme, but urlparse raises ValueError on
structurally invalid URLs — a bad IPv6 literal, invalid netloc
characters — instead of returning an unusable scheme. Pre-fix
checkpoints and imported history can carry exactly those URLs, and the
block stays in the checkpoint, so every later model call in the thread
raised again: the sanitizer re-bricked the very threads it exists to
heal, only with a framework-side exception instead of the langchain
serialization error.
Catch ValueError and treat the URL as non-fetchable, so the block
degrades into the same text placeholder as any other unresolvable
scheme (the shared model_visible_location gate already swallows
ValueError, so the location segment is withheld as well).
- wrap the image branch's urlparse call in try/except ValueError and
fall back to an empty scheme, keeping http/https/data pass-through
behavior unchanged for well-formed URLs
- add a regression test pinning that an image block with a bad IPv6
literal heals to the location-less placeholder without raising
Co-Authored-By: Codex <noreply@openai.com>
* fix(tools): dedupe artifact refs across extraction sources
One downgraded ResourceLink produced two registry entries: the typed
resource_links channel added an entry for the URI, and the free-text
scan over the conversion layer's placeholder text — which embeds the
same virtual path or remote URL — matched it again. The two handles
pointed at one resource, spent double of the render budget, and showed
the model two handles for a single artifact.
extract_artifacts_from_result now tracks seen real_refs across all
sources and keeps only the first entry per ref. Extraction order makes
the typed sources (structured content, resource_links, content blocks)
win over the free-text scan, so the surviving entry keeps the link's
display name and mime type.
- route all six entry-producing sites through a local _add_unique
helper keyed on real_ref; per-source behavior is unchanged
- document the cross-source dedup in the function docstring
- add a regression test covering a virtual-path and a remote-URL
placeholder together with their resource_links entries
Co-Authored-By: Codex <noreply@openai.com>
* docs(mcp): state the intentional data: image downgrade asymmetry
The conversion layer downgrades image/* data: ResourceLinks to
location-less text placeholders, while the read-time middleware lets
persisted data: image blocks pass through to the model. The two
contracts looked contradictory but are deliberately layered: the
conversion layer keeps new inline payloads out of checkpointed state,
and the middleware only preserves blocks already occupying one.
- mcp/AGENTS.md's ResourceLink conversion contract and the conversion
comment in mcp/tools.py now state the image/* data: downgrade and
why it differs from the read-time pass-through
- middlewares/AGENTS.md's ModelContentCompatibilityMiddleware entry
states the symmetric read-time rule
- pin the behavior with an image-mime data: ResourceLink conversion
test (location-less placeholder, nothing in the artifact channel)
Co-Authored-By: Codex <noreply@openai.com>
* docs(backend): compact AGENTS.md guidance to fit the chain byte budget
The branch's new contract bullets pushed the middlewares and mcp
effective AGENTS.md chains past the 98304-byte AG002 hard limit (the
sandbox/subagents chains on main were already over it). Compact the
guidance without dropping contracts:
- middlewares item 34 points at the MCP ResourceLink conversion
contract and the module docstring instead of restating both
- the mcp ResourceLink conversion contract bullet is tightened
- backend/AGENTS.md merges the File Upload and Context Summarization
bullet sprawl, folds the stray ATX test note into the upload outline
rule, drops the Nginx routing block the root guide already covers,
merges the duplicate MCP-task pointer bullet, compacts Vision
Support, and scopes the Project Structure tree to the backend
subtree (the root guide maps the full repo)
Co-Authored-By: Codex <noreply@openai.com>
* fix(agents): gate the file-URL downgrade on provider support and neutralize placeholder metadata
Two review findings on the read-time sanitizer:
- The file-block downgrade ran for every provider, but only Chat
Completions rejects URL-sourced file blocks: the OpenAI Responses API
(input_file file_url) and Anthropic (document url source) accept
http(s) document URLs, so a persisted block created for those
providers lost its document from the model request. Keep the block
when the selected model serializes http(s) file URLs natively; every
other URL shape (non-http schemes, empty/None, malformed) still
degrades to the placeholder.
- The placeholder interpolated persisted MIME/location metadata
verbatim, but the middleware runs after the input/tool-result
sanitization passes and historical URL blocks never went through them
(non-text content is left untouched), so values carrying
<system-reminder>-style tags or user-input boundary markers reached
the model raw. Route the interpolated metadata through the shared
neutralize_untrusted_tags guard in both the file and image branches.
Tests: provider gating (Responses/Anthropic preserve, Chat Completions
still downgrades, non-http and malformed URLs downgrade even for
capable providers) and metadata neutralization (injection tags
HTML-escaped, boundary tokens neutralized, file and image branches).
Co-Authored-By: Codex <noreply@openai.com>
* fix(tools): require a directory boundary before exposing virtual locations
model_visible_location admitted any URL starting with the virtual
prefix, so a sibling like /mnt/user-data-backups/ops/private/report.pdf
was treated as a virtual path and its host location reached
model-visible placeholder text — defeating the US-16 host-path
suppression the gate exists for. Require the exact virtual root or a
slash-delimited descendant; siblings now collapse to the location-less
placeholder like any other host path. Both the file and image
downgrade branches share the gate, and the conversion layer applies
the same rules to fresh results.
Regressions: gate unit tests (exact root and descendants stay visible;
/mnt/user-data-backups and /mnt/user-database siblings withheld) and
middleware tests pinning the location-less placeholder for both
branches.
Co-Authored-By: Codex <noreply@openai.com>
* docs(agents): scope the placeholder artifact-visibility claim to the conversion layer
The module docstring claimed the healed placeholder keeps the location
visible to the artifact free-text scan and that the link survives in
the structured artifact channel — true for the conversion layer, whose
placeholders are checkpointed, but not on the healing path: the request
view never reaches artifact capture, and no resource_links entry is
created for a downgraded block. Spell out what actually happens here:
a referenceable original block (http(s) URL or absolute host path) is
still captured, from the capture-time scan of the state block rather
than the placeholder, while a non-referenceable one (file://, data:,
blob:) yields no entry and survives only in the untouched checkpoint.
Co-Authored-By: Codex <noreply@openai.com>
* docs(mcp): clarify healing and artifact preservation contracts
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|
||
|
yc2bgr8
|
127c2c220c
|
fix(community): follow Jina API redirects during web fetch (#5808)
* fix(community): 跟随 Jina API 重定向 * test(community): 覆盖 Jina 跨域重定向凭据边界 * test(community): assert Jina redirects retain request body |
||
|
Nan Gao
|
a83b244a8c
|
fix(uploads): keep in-flight staging files out of the startup sweep (#6445)
* fix(uploads): keep in-flight staging files out of the startup sweep `cleanup_stale_upload_staging_files` removed every `.upload-*.part` it found at Gateway startup. That assumed a single Gateway: nothing can be uploading while it boots, so every staging file is a crash leftover. With several replicas sharing one home volume (Helm RWX), a replica starting during a rolling update deleted the staging file of an upload another replica was still writing; that upload then failed at its `os.link` commit and lost its bytes. The sweep now skips staging files younger than `UPLOAD_STAGING_MIN_AGE` (24 hours, the same guard as the project-document `.staging` orphan sweep in `projects/trash.py`). Chunk writes refresh the mtime, so an upload in flight stays younger than the guard; an unreadable mtime keeps the file. The startup call site is unchanged and uses the default. Tests: the existing sweep tests backdate their orphans; new cases pin young-kept / old-removed, the configurable `min_age`, unreadable and vanished mtimes, and a router test that runs the sweep while an upload is blocked mid-write and asserts the upload still commits. * docs(changelog): link #6445 * fix(uploads): reclaim published staging aliases at any age Review follow-up (#6445): a Gateway that dies after the commit's `os.link` but before the staged-name removal leaves the final file and its `.upload-*.part` alias on one inode. With only the age guard, the next startup kept that alias for 24 hours, and because the sweep runs only at startup, `copy_upload_file_no_symlink` (embedded `DeerFlowClient.upload_files`) kept rejecting the destination for its multiple links. The sweep now stats each staging name with `os.lstat` (a `DirEntry` stat reports `st_nlink == 0` on Windows) and removes a name with `st_nlink > 1` at any age: a staged part only ever gains its second link through its own commit, so such a name is a published alias, never an upload in flight. Lone parts keep the 24 hour mtime guard. The destination hardlink check is unchanged. Tests cover the alias next to a young lone part, the replacement upload succeeding afterwards, and the crash reproduced through the real upload path with the staged-name removal patched out. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
haochen
|
cde9602631
|
docs: run configuration and migration commands in the backend environment (#6425)
* docs: run configuration and migration commands in the backend environment * docs: preserve runtime paths and extras in backend commands --------- Co-authored-by: Zhangchenhao-111 <285911435+Zhangchenhao-111@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Totoro
|
35cdcab1c5
|
feat(scheduled-tasks): signals, honest notices and fair queueing for server-started runs (RFC #6340, 2/2) (#6393)
* feat(scheduler): lifecycle rules, goal/limit parity and coded errors for scheduled tasks
- Migration 0031: nullable scheduled_tasks.unmet_streak_after_seq (internal
streak boundary) and stop_condition (the user's stop rule, own column).
- Stop rule is normalized (stop_rule.py) and appended only at launch; the
stored prompt never contains it. Wording adapts when scheduler.tool_enabled
is off.
- Willem 1: page-created tasks get goal evaluation, <standing_notes>, the
previous-run reference and own-schedule stop; request_stop and the
scheduled capability no longer require an origin conversation.
- Repository: single reactivation choke point raising
ScheduledTaskLimitsExhausted; streak boundary moves on goal/prompt/stop
condition/note changes (not on resume); goal-check failures neither count
nor reset the streak; pause of a finished task returns "finished"; trials
on a paused task keep the host pause marker.
- New repository reads: automatic_runs_used_for, active_run_status_for,
list_manageable_from_thread, thread_relation/thread_run on thread tasks,
task_ids_for_thread, run_number, and run_number/total_tokens/summary on
run history rows.
- REST: goal/max_runs/end_at/stop_condition on create and PATCH (null
clears, naive end_at = task timezone), resume recomputes from now with an
optional renewal body, once_time_passed, task_finished, limits_exhausted,
scheduler_not_running on create, trigger returns outcome/existing/thread_id,
task responses add automatic_runs_used and active_run_status.
- Coded errors {code, message, params} for routes, shared validation and the
chat capability (scheduled_task_errors.py), pinned by
contracts/scheduled_task_errors_contract.json; goal notes contract v2 with
check_failure_codes, scheduled_origin_key and host_run_errors (host_notes.py).
- /api/features adds scheduled_tasks {available, running, tool_enabled,
min_interval_seconds}; ScheduledTaskService.is_running.
- Docs: six README scheduler sections, API.md scheduled tasks section
(breaking error shape), CONFIGURATION.md, AGENTS notes, CHANGELOGs.
- Tests: migration 0031, stop rule, error contract, run listing, lifecycle
over HTTP, repository, service and capability cases; updated tests for the
new error shape, finished-task pause and head revision.
* feat(scheduler): conversation update/resume, run-conversation authority and readable launches
- schedule_task: update and resume actions, stop_condition and clear_fields
parameters, plain parameter types (value rules come back as coded results),
rewritten docstrings for schedule_task and stop_scheduled_task.
- Capability: update/resume share the REST validation and error codes;
interactive turns in a task's run conversation may manage that task;
scheduled runs keep only own-stop; grants require a running scheduler.
- Results: compact JSON-safe task views with local times, display card/text,
last run in list, timezone_source/browser_timezone, now_local; no ISO, user,
lease or origin fields; stop_instructions removed.
- Timezone: explicit > browser (context.client_timezone, validated in
start_run and never forwarded to the run config) > not needed for intervals
and offset run_at > timezone_required.
- Trial gate accepts natural direct requests after acknowledgements and
particles; trial results report thread_id and existing.
- Scheduled launches: stable message id scheduled-<task_run_id>, server-owned
deerflow_scheduled_origin metadata with the user-language parts, run thread
title for fresh-thread tasks.
- Worker records goal continuations in the verdict; lead prompt language rule
covers stored task text and scheduled runs.
- Docs: six READMEs, CONFIGURATION, API, runtime/tools AGENTS, CHANGELOGs.
* feat(frontend): scheduled-task core types, formatting, errors and glossary
- throwGatewayApiError now throws GatewayApiError (status, code, params,
rawMessage); message stays the string detail or detail.message
- scheduled-task types gain stop_condition, automatic_runs_used,
active_run_status, origin/notes, run_number/total_tokens/summary, thread
relation, trigger result, renewal and scheduled origin; isTaskBusy and
hasActiveRun helpers
- api: fetchScheduledTask, resume with optional renewal body, typed trigger
result and update payload with null clears
- hooks: polling at 3 s while a run is active (list, thread list, run
history), useScheduledTask with live gating and final 404, useInView,
localized error toasts; run history refreshes tasks when a run finishes
- new pure modules: errors.ts (contract code to localized copy), format.ts
(relative task times, viewer time, display zone), status.ts, actions.ts,
run-error.ts; describeTaskSchedule in cron.ts; goal-outcome adds unchecked
goal checks, describeTaskOutcome and parseStopRunId
- features: scheduled_tasks block with defaults when missing
- buildRunContext sends client_timezone
- titleOfThread takes a localized untitled label; breadcrumb names scheduled
tasks and agents; reference chips use the localized untitled label
- schedule input: shared browserTimeZone, labelled selects, aria-pressed
type toggles, interval minimum from the hint template
- i18n: all PR1 scheduled-task keys in en-US and zh-CN, glossary fixes,
recipe instructions moved into i18n
- e2e mocks follow the new task, trigger, resume, thread-task and features
shapes; expectNoRawIdentifiers helpers for e2e and DOM tests
- tests for errors, format, cron, status, actions, goal outcome, run errors,
hooks, run context, thread titles and features; scheduled e2e copy updated
* feat(frontend): list-first scheduled tasks page with outcomes and editable goals
- page.tsx is now a thin composition of components/workspace/scheduled-tasks:
status tabs (All/Active/Paused/Finished) and search over the task list,
?task_id= deep link with router.replace on selection, ?thread_id= chip
with Show all, empty state, scheduler-off notice and unavailable state
- task list rows show the schedule in words and one status line (next run,
running now, paused by agent, auto-paused, finished by its cap)
- task detail: status badge, actions from availableActions with reasons,
More actions menu (Duplicate, Copy task ID, Delete), outcome notice,
Runs / Stops when / Each run's goal / Does / Notes from chat / History,
origin chat link; success toasts for every action
- run rows: run number or Trial run, summary, goal and assumption badges,
continuations, tokens, Open chat link; raw errors and the evaluator
reason only behind Details
- create/edit/duplicate dialog with goal, stop condition, safety caps and
advanced agent/context options; PATCH sends only changed fields, null
clears a cap
- renew dialog raises or removes caps and resumes in one request; a
limits_exhausted Resume opens it
- removed i18n keys the old page used; added feedback.resumedNoTime
- recorded en-3 and zh-1 page fixtures; scheduled e2e specs rewritten for
the new page; DOM tests for the detail and the form dialog
- CHANGELOG entries and the frontend AGENTS.md scheduled-task note
* feat(frontend): live schedule card, run conversation provenance and header entry
- schedule_task results (create, update, pause, resume, trial, delete) become
an assistant:scheduled-task message group rendered as a live card: schedule,
next run, stop condition with the safety cap, goal, where results go, Run
once now, Pause/Resume and Open task; the card polls the task only while it
is on screen and shows deleted, scheduler-off and limits-exhausted states
- one card per task per assistant turn; the card group is not a turn boundary
for usage, branching, editing, outline or skill attribution
- a human message with deerflow_scheduled_origin stays a human group with
scheduledOrigin and renders as a "Scheduled run / Trial run" block with the
task instructions, stop condition and notes collapsed; the launched text is
never shown and the message is not editable
- readable step labels for schedule_task actions and stop_scheduled_task
- ThreadScheduledTasksButton replaces the header link: hidden without tasks,
shows the count, links to the task (or the chat filter), and a run's chat
links to its task
- recorded en-3 / zh-1 chat and run-thread fixtures; chat-card and run-thread
e2e specs; unit and DOM tests for parsing, grouping, turn boundaries, card,
run block, header button, step labels and reconnect hydration
- i18n: header.countLabelOne added, unused card.next removed; CHANGELOG and
frontend AGENTS.md notes
* fix(scheduler): address verification findings
Move the scheduled-task lifecycle guidance from backend/AGENTS.md into the persistence guide and leave a one-line pointer, so the effective AGENTS.md chains for agents/middlewares, sandbox and subagents stay under the 98304-byte hard limit enforced by scripts/check_agent_guidance.py in lint-check.
* fix(scheduler): address review findings
* fix(gateway): no-transform on SSE responses
All four text/event-stream responses (thread run create stream, join,
existing-run GET/POST stream, stateless create stream) now take their
headers from app/gateway/sse_headers.py and send
Cache-Control: no-cache, no-transform, so compressing proxies such as
Next's rewrite proxy stop buffering streams. Content-Location and
X-Accel-Buffering are unchanged; scheduled and MCP-notification runs keep
stream_mode=None.
Tests pin the headers on every SSE route and fail on a literal no-cache
header left in routers. Docs: API.md SSE headers note, CHANGELOGs.
* feat(scheduler): per-owner run cap and fair queue selection
New scheduler.max_concurrent_runs_per_user (default 2, 0 = off, effective
min(cap, max_concurrent_runs)) caps launching/running scheduled runs per task
owner. claim_queued_run checks it right after the global count under the same
budget lock, so it is atomic across workers and Pods. list_queued_runs ranks
same-thread FIFO heads per owner with ROW_NUMBER() OVER (PARTITION BY owner)
and orders the batch by that rank, so the drain is owner-fair even with the
cap off; owners at their cap drop out before the batch limit. Orphan rows
(task deleted while queued) are never held back by the cap.
Gateway lifespan passes the startup value; the service constructor default
stays 0. Docs: config.example.yaml (config_version 54), CONFIGURATION.md,
the six READMEs, backend AGENTS.md, CHANGELOGs (behavior change flagged).
The live-task quota is untouched.
* feat(scheduler): durable lifecycle events for the originating chat
When a chat-created schedule is paused by the agent, auto-paused or finishes,
finalization now reports task_stopped / task_paused / task_finished, only on a
live -> paused/finished transition. The new finish_task_at_end_condition
replaces the three inline "mark completed" blocks (admission rejected as ended,
claim-time skip, complete_if_ended), so every site emits exactly once.
The scheduler's finalization observer is always installed. It writes
scheduled_task_events rows for the originating chat in the same transaction as
the state change, deduplicated on (task_id, anchor, event), with a title
snapshot, stop condition, run number, last run outcome and the chat's newest
run as placement anchor. Lookups fall back on bad stored data; only database
errors propagate. Manual trials drop only run_* events. The IM step keeps
PR1's selection until B4.
New GET /api/threads/{thread_id}/scheduled-task-events (owner check, caller's
rows, oldest first); rows survive task deletion and go with the chat.
Migration 0032_activity_and_task_events adds the events table plus the
run-origin column/index and read-state tables used later, each object guarded
on its own. Contract scheduled_goal_notes v3 pins the lifecycle vocabulary.
Docs: API.md, migrations chain, backend AGENTS.md, six READMEs, CHANGELOGs.
* feat(scheduler): honest, localized and self-contained IM notices
Every scheduled-task IM notice is now enqueued by the finalization observer
in the transaction that records the outcome, one message per occurrence
(task_stopped > task_paused > task_finished > run_failed > run_unmet >
run_completed; a once task sends only its run event). The merged auto-pause
replaces the separate unmet + pause messages, agent stops and finishes now
notify, and recovery-finalized runs notify exactly once. Idle finishes use a
deterministic idle: key derived from the lifecycle anchor. The old
non-transactional enqueue in handle_run_completion is removed.
New app/channels/capabilities.py holds CHANNEL_CAPABILITIES with a
proactive_notifications flag (adds the missing qq entry); only push-capable
providers (WeCom) get rows, /api/channels/providers returns the flag, and the
worker ends legacy rows for other providers as terminal failed.
New app/scheduler/notification_text.py renders en-US/zh-CN text: title, what
happened (plus the last run's outcome), the tasks page's one-line run summary
(redacted) when the agent replied, and a closing line. No IDs, no links, no
public_base_url. Locale is the owner's new `locale` preference, else the new
channel_connections.notification_locale (default en-US).
Docs: config.example.yaml (config_version 55), CONFIGURATION.md, API.md,
channels AGENTS.md, six READMEs, CHANGELOGs (behavior change flagged).
* feat(gateway): thread origin, activity feed and read markers for server-created threads
Runs started by the server for a user (scheduler, IM channels, GitHub
agents, extension handles, MCP notifications) now carry a server-owned
metadata.deerflow_origin {kind, provider?, namespace?} (new harness module
deerflow.runtime.run_origin, pinned by contracts/thread_origin_contract.json).
It is stamped only from request.state.run_origin or by the internal channel
caller; run admission drops client copies from both metadata and
config.metadata, and thread create/patch strip it. RunRepository derives
runs.origin_kind on put and on create_thread_operation_atomic (agent runs
only), and gains latest_change().
New ThreadReadRepository keeps a per-user, never-decreasing read position
(thread_read_markers) and a per-user read clock (thread_read_versions).
POST /api/threads/{id}/read marks a thread read, thread search items carry
unread, and thread delete removes the markers. GET /api/thread-activity pages
the caller's run changes over the run-change clock, returns only threads
changed by server-originated runs plus read_version, and 503s without SQL;
/api/features reports thread_activity.available.
Docs: API.md, gateway AGENTS.md (condensed to stay under its budget),
CHANGELOGs.
* fix(scheduler): adapt PR2 to PR1 follow-ups
PR1 moved the scheduled-task lifecycle rules out of backend/AGENTS.md into
the persistence guide and grew the sandbox AGENTS.md chain to within 267
bytes of its hard limit. The per-owner queue-fairness detail now lives in
the persistence guide next to the lifecycle rules (backend/AGENTS.md keeps
the cap and a link), so check_agent_guidance passes again.
PR1's goal-outcome unit test pinned scheduled_goal_notes_contract version 2;
PR2 bumps it to 3 (lifecycle vocabulary only, the v2 keys are unchanged).
* fix(scheduled-tasks): polish from live acceptance
- zh schedule wording: an interval of one unit reads 每分钟 / 每小时 / 每秒,
and a saved interval of whole days reads 每天 / 每 2 天 (Every day / Every 2
days) instead of 每 1 分钟 or 每 24 小时; several units keep the number
- zh spacing: 已于{time} 满足 and 在{time} 的运行中 now leave a space between
the interpolated time and the Chinese after it; a copy test guards every
scheduled-task zh string against a placeholder glued to Chinese
- run summary: when the first line of the final reply only leads in to a list
(ends with ":" or "："), the list items follow it as plain text (bullets,
numbering, checkboxes, bold, code and links stripped), joined with "；" for
CJK text and "; " otherwise, within the 160-character cap
- a schedule_task result (create, update, pause, resume, trial, delete) that
arrives in a chat invalidates that chat's task list, the tasks list and the
named task, so the header button and cards update without waiting for the
idle poll; results already in the loaded history do not refetch
- e2e and unit fixtures are now the live acceptance recordings (DeepSeek
Flash, demo data, fictional names): live-weekday.json (create, trial, edit,
pause, resume in one zh chat) and live-minute.json (per-minute chat, run 1
and run 2, paused by the agent), fetched with GET from the demo Gateway and
trimmed; the hand-shaped en-3 / zh-1 fixtures are removed and the chat-card,
run-thread and tasks-page specs run in both locales over the recorded data
* docs(changelog): reference #6378 for the scheduled-tasks entries
* docs(agents): shorten the scheduled-task pointer to stay under the guidance limit
* test(scheduled-tasks): pin the live takes to their recorded time zone
The live fixtures were recorded in Asia/Shanghai, and an interval task
reads in the viewer's zone, so the 16:48 assertions failed on UTC CI
runners. Pin the browser zone in the two live-take e2e specs and the
zh live-take unit test.
* feat(frontend): thread origin, activity polling, read markers and task events core
Frontend core for server-created threads and schedule lifecycle events
(PR2 F1); the sidebar markers (F2) and the chat event line (F3) build on it.
- core/threads/origin.ts: threadOriginOf reads metadata.deerflow_origin,
then channel_source (github -> GitHub origin), then a legacy
scheduled_task_id; key and kinds pinned to
contracts/thread_origin_contract.json. labelOfThreadOrigin and the now
exported labelOfChannelProvider(provider, t) use the new localized
threads.origin.providers names (飞书, 企业微信, ...), with the English map
(gaining GitHub) only as the fallback.
- core/threads/activity.ts: useThreadActivity polls /api/thread-activity
every 15 s (focus refetch, never in the background, off unless
features.thread_activity.available). The first response only seeds the
cursor; server-origin threads, a truncated page or a higher read_version
invalidate the thread lists, a scheduled thread also ["scheduled-tasks"];
an invalid cursor re-seeds. useMarkThreadRead debounces POST /read by 1 s,
skips a thread whose cached unread is false, patches the search and
infinite caches optimistically and stores the returned read_version so a
tab's own read does not refetch its lists. AgentThread gains unread.
- core/scheduled-tasks/events.ts: ScheduledTaskEvent, the events query
(["scheduled-tasks","thread-events",id], one fetch per saved chat, not
gated on the task list so a deleted task's lines stay, refetched when a
task of the chat changes status or finishes a run), describeTaskEvent
(en/zh copy, title and condition as segments, last-run suffix, See that
run / Open task) and placeTaskEvents (end of the after_run_id turn, else
the tail, ordered by created_at); lifecycle names pinned to contract v3.
- features: thread_activity (default unavailable); channels:
proactive_notifications on providers.
- Locale sync: the I18nProvider reports the UI language and a signed-in
preference session saves it as the new `locale` preference when the
account's differs (and on every switch), so IM notices follow the UI
language; nothing is written with auth disabled.
- i18n: threads.* (unread, origin labels, provider names),
scheduledTasks.events.*, history.waitingForSlot (now shown for queued run
rows), channels.scheduledUpdates.*.
- e2e mocks: thread activity feed and controller, POST /read, the events
route, providers with proactive_notifications, search unread, features
thread_activity (off by default). Preference e2e mocks return the UI
locale so they keep asserting only their own edits.
- Docs: frontend AGENTS.md, CHANGELOGs.
* fix(scheduled-tasks): keep untouched end times and drop a chat filter the new task is not in
- The edit, duplicate and renew dialogs keep a stored end time's instant
while its field still shows it. A minute-precision wall value cannot
keep seconds or name the later 01:30 of a DST fall-back, so a
title-only edit moved the cap an hour earlier, and raising the run
limit re-sent the end time without its seconds.
- Creating or duplicating a task from a chat's task view drops the chat
filter when the new task does not belong to that chat, instead of
showing "This task isn't one of this chat's tasks".
* feat(frontend): origin markers and unread state in the thread lists
Server-created threads (PR2 F2) now say who created them and whether they
are new, in the sidebar and on the Chats page, and reach an open sidebar
without a reload.
- ThreadOriginIcon (thread-channel-source.tsx): clock for a scheduled run,
the provider mark for IM threads, the GitHub mark (new in
ChannelProviderIcon) for GitHub threads, a puzzle piece for extensions;
the localized "Scheduled run" / "From Feishu" / "From GitHub" / "From an
extension" label is its accessible name and tooltip. ThreadChannelIcon
stays exported as a wrapper, and the provider chip tooltip is localized
(no more "Feishu channel").
- Unread dot after the title (aria-hidden dot plus sr-only "Unread"); the
row aria-label becomes "{title}, unread" / "{title}，未读". Never shown for
the open thread.
- WorkspaceSidebar mounts useThreadActivity once.
- useMarkOpenThreadRead (components/workspace/chats): the default and
custom-agent chat pages mark the thread read once its metadata has
loaded, when its cached unread flips to true while open, when the tab
becomes visible, and from onFinish (a send or a joined run ended). Each
trigger goes through useMarkThreadRead, so it is debounced and a no-op
while the cached unread is false.
- Tests: thread-origin-icon and recent-chat-list DOM tests (en/zh, active
row, no raw identifiers), use-mark-open-thread-read DOM tests,
e2e thread-activity.spec.ts (new scheduled run appears unread within one
fake-clock poll without a reload, opening it posts /read and clears the
dot, a read on another device clears it, IM and GitHub markers, zh
labels, the Chats page, no polling when unavailable);
thread-history.spec.ts now finds the localized origin icon.
- Docs: frontend AGENTS.md, CHANGELOGs.
* feat(frontend): schedule event line in the originating chat and channel notice support
The chat that created a schedule now says when the schedule paused or
ended, and the channel cards say which apps receive scheduled task updates
(PR2 F3).
- ScheduledTaskEventLine (mockup m2, marker 4): a dashed, muted row with
the describeTaskEvent text (title in <strong>, stop condition clamped to
one line with the full text in its tooltip, last-run suffix), the time
via formatTaskTime and a right-aligned "See that run" (run chat) or
"Open task" link. PauseCircle for paused, CircleCheck for finished and
once-done, CircleX for once-failed. role="note", aria-label "Scheduled
task update" / "定时任务通知", data-task-id and data-event-id only.
- MessageList takes scheduledTaskEvents and renders placeTaskEvents output
through a new VirtualMessageList.renderAfterGroup, so each line sits at
the end of its after_run_id turn inside the group's measured row (also
when the list virtualizes); lines with nowhere to go render at the tail.
- Both chat pages load useThreadScheduledTaskEvents (not gated on the task
list, so the lines stay after a task is deleted). The custom-agent page
passes its agent so "See that run" opens the run on that agent's route.
- ChannelScheduledUpdates: "Scheduled task updates: sent here" / "... not
available for this app yet" (定时任务通知：会发送到这里 / 此应用暂不支持)
on the Settings provider cards and under each sidebar channel row; nothing
when an older backend does not report proactive_notifications.
- e2e mock controller: setScheduledTasks.
- Tests: scheduled-task-event-line and channel-scheduled-updates DOM tests
(every kind in en and zh, hrefs, clamp, no raw identifiers, Chinese
title), MessageList placement DOM tests; e2e
scheduled-task-chat-events.spec.ts (the line appears with the card's
"Paused by agent" without a reload, See that run opens the run chat,
anchored vs tail placement, two events in time order, the line survives
task deletion and Open task shows the task is gone; en-3 and zh-1) and
channels.spec.ts (WeCom sent here, Feishu not yet; en and zh).
- Docs: frontend AGENTS.md, CHANGELOGs.
* fix(scheduled-tasks): tie pause notices to the pause they describe
- Paused by agent links and times only the run named in last_error. When
that run is not loaded, the notice shows no link and the task's own time
instead of an earlier run that also asked to stop.
- The outcome notice reads the latest history page whichever page is
shown, so browsing older runs no longer swaps the auto-pause reason and
"Open latest run" to an earlier miss. It shares that page's request.
* fix(scheduler): address PR2 review findings
- Event lines: hold an event whose anchor run is on an unloaded history
page instead of placing it at the bottom (hasMoreHistory); tail fallback
only for fully loaded (branched/pruned) history or no anchor.
- Idle-finish anchor follows its reason: end:<iso> only for an end_at
finish, seq:<n> otherwise, so a later end-time finish is not deduplicated
away by an earlier max_runs finish.
- Observer takes FOR SHARE on the originating threads_meta row, so a racing
thread delete cannot leave an orphan event line.
- One taskPagePath (core), re-exported by the card.
- Notification locales derived from NotificationLocale; /preferences uses it.
- IM notices drop generic reasons (goal_not_met_yet, unknown) instead of
repeating the sentence.
- Localized provider names on Settings cards, sidebar rows and setup dialog.
- zh event lines: no space after a Chinese title (Latin-aware gap).
- Unread row label keeps the origin marker; chats page keeps the time as a
description.
- Sidebar shows the scheduled-updates line only for connected apps;
Settings says "available after you connect" for unconnected push apps.
- Event line keeps a real space before the time.
* fix(scheduled-tasks): three lifecycle gaps found while re-checking review classes
- The edit form compares a cron in the form its schedule input emits, so an
untouched "0 09 * * 0,6" is not re-sent as "0 9 * * 6,0" and a title-only
edit no longer re-arms or refuses a finished task.
- Chat resume asks for a timezone, as update does, when a naive end_at would
land in the UTC placeholder of a zone-free task.
- Re-arming a task clears a host pause marker, as resume does, so a later
manual pause no longer reads as "Paused by agent".
* test(scheduler): consolidate PR2 backend test setup
- Fold the repeated task, origin-chat and observer setup of the
lifecycle-event repository tests into one lifecycle_task builder.
- Build the per-owner-cap services through one _capped_service builder
that keeps every cap explicit at the call site.
- Table the five SSE route header checks into one parametrized test with
the same routes, bodies and Content-Location expectations.
- Reuse bind_wecom and hoist repeated local imports in the event tests.
Every case and assertion is kept; collected cases are unchanged.
* test(frontend): consolidate PR2 thread activity and origin test setup
Extract shared builders and helpers in the frontend tests PR2 adds, with
every case, input and assertion kept:
- activity.test.ts: seededClient/apply helpers and one LIST_KEYS constant
replace the per-test seed, spy and apply boilerplate.
- mark-read.dom.test.tsx: newClient() and advance() replace the repeated
QueryClient wrapper and fake-timer act blocks.
- origin.test.ts: the English and Chinese label checks become tables; the
no-English-name check now runs on every Chinese row.
- thread-activity.spec.ts: a scheduledRun() builder replaces four inline
copies of the scheduled run thread.
- channels.spec.ts: the PR2 block uses setLocaleCookie and one card helper.
Collected cases are unchanged (unit 211, e2e 60); no test was removed.
* fix(scheduled-tasks): keep the notice's latest-runs query from refreshing older pages
useLatestScheduledTaskRuns kept React Query's default refetch on mount,
focus and reconnect, so going offline and back while an older history page
was shown fetched the latest page again. Refreshing stays with the history,
which refreshes the latest page only while it is shown.
* fix(scheduler): two gaps found in live acceptance
- A fresh scheduled run thread is named when it launches. The worker
copied the title to the thread list only when the run ended, so the
sidebar, which now shows the new run thread at once, listed it as
untitled while it ran.
- A task capped at one run finishes with "its one run is done" /
"唯一一次运行已完成" in the chat event line and the IM notice, instead
of "all 1 runs are done".
* test(scheduled-tasks): let the viewer's time carry its own day label
Between 16:00 and 24:00 UTC, UTC and Asia/Shanghai are on different days,
so the run prompt shows "Tomorrow 01:00 · Today 09:00 your time" and the
fixed "01:00 · 09:00 your time" assertion failed depending on the hour.
* test(scheduler): expect the one-run finish copy in service notices
* fix(gateway): create a scheduled run thread with its title
Pre-setting the name after start_run raced the thread row, which the run
worker creates asynchronously: live, one of two run threads still reached
the sidebar untitled. Run admission now creates a new thread with the title
its run input carries, which the worker would copy only at run end.
* docs(changelog): reference #6393 for the scheduled-task signal entries
* fix(scheduled-tasks): route event-line run links by the agent that ran them
The originating chat's agent was used for "See that run", but a task's
agent can change after the chat created it, so the link could open the run
chat on another agent's route. Lifecycle events now record run_agent_name,
the task's agent when the run finished (edits are refused while a run is
active), and the line routes by it.
* fix(scheduled-tasks): load up to the route's maximum of chat event lines
The chat loads its lifecycle events once, and the route's default of 50
would silently drop the oldest lines of a chat with many schedules. Ask
for the maximum, 200.
* fix(threads): keep activity reads and the first cursor in step with the lists
- Accepting the first activity cursor refreshes the thread lists once. The
lists load on their own, possibly before a run the seed already counts,
and later polls start after it, so such a thread never appeared.
- A local read whose returned read clock jumps past that one read also
covers reads on other devices; the lists are refreshed instead of the
clock silently skipping those threads' stale unread dots.
- An invalid cursor re-seeds through the same path, so it refreshes once.
* docs(agents): bring the middlewares guidance chain back under the hard limit
#6378 and #6363 each fit the budget, but together the middlewares AGENTS.md
chain on main is 98,425 bytes, 121 over the 98,304 hard limit, so Lint
Check fails on main. Move the scheduled-task recovery, launch-update and
queue-timeout rules from backend/AGENTS.md into the persistence guide's
lifecycle section, which backend/AGENTS.md already points to. The chain is
now 98,067 bytes.
* fix(helm): bump the chart's config_version to 55
config.example.yaml moved to 55 in this PR, so the Helm chart's
validate-chart drift check failed. The chart's embedded config needs no
field changes: the new scheduler and channel keys have defaults.
---------
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|
||
|
Nan Gao
|
4f64ca2ae0
|
fix(config): revalidate the extensions config singleton by file signature (#6386)
* fix(config): revalidate the extensions config singleton by file signature `get_extensions_config()` parsed `extensions_config.json` once per process and only the process that handled a write ever reloaded it. With several uvicorn workers, or several Pods sharing one volume, every other process kept its startup copy: the MCP servers the agent could use and the local-bash absolute path allowlist derived from the filesystem MCP server differed between replicas until each one restarted. The singleton now records the resolved path and the file's (mtime, size, sha256) signature and reloads when either changes, like `get_app_config()` does for `config.yaml` and `mcp/cache.py` does for the tools cache. After a successful load, a revision that cannot be loaded (file gone, truncated, invalid) keeps the last-known-good configuration and is logged once with the exception type only; the first load still raises. `set_extensions_config()` pins the injected instance until a reload or reset. The cached AppConfig's `extensions` snapshot is built from the singleton instance and refreshes when that instance changes, so it inherits the same fail-soft policy instead of parsing the file again. * docs(changelog): link the extensions config freshness entry to #6386 * fix(config): follow an explicitly reloaded extensions config path `reload_extensions_config(config_path=X)` recorded X, but the freshness probe re-resolved the default path, so the next read either flipped the cache back to the default file or pinned a stale copy of X. The cache now remembers the explicit path and both the probe and later reloads follow it until a reset or an argument-less reload. Also shortens the pointer in backend/AGENTS.md so the nested guidance chains stay under the hard size limit enforced by the agent-guidance check. * docs: keep backend guidance within the inherited size budget * fix(config): reload the probed extensions config path --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Totoro
|
2dd2e5415c
|
feat(scheduled-tasks): finish the user-facing side of conversation schedules (RFC #6340, 1/2) (#6378)
* feat(scheduler): lifecycle rules, goal/limit parity and coded errors for scheduled tasks
- Migration 0031: nullable scheduled_tasks.unmet_streak_after_seq (internal
streak boundary) and stop_condition (the user's stop rule, own column).
- Stop rule is normalized (stop_rule.py) and appended only at launch; the
stored prompt never contains it. Wording adapts when scheduler.tool_enabled
is off.
- Willem 1: page-created tasks get goal evaluation, <standing_notes>, the
previous-run reference and own-schedule stop; request_stop and the
scheduled capability no longer require an origin conversation.
- Repository: single reactivation choke point raising
ScheduledTaskLimitsExhausted; streak boundary moves on goal/prompt/stop
condition/note changes (not on resume); goal-check failures neither count
nor reset the streak; pause of a finished task returns "finished"; trials
on a paused task keep the host pause marker.
- New repository reads: automatic_runs_used_for, active_run_status_for,
list_manageable_from_thread, thread_relation/thread_run on thread tasks,
task_ids_for_thread, run_number, and run_number/total_tokens/summary on
run history rows.
- REST: goal/max_runs/end_at/stop_condition on create and PATCH (null
clears, naive end_at = task timezone), resume recomputes from now with an
optional renewal body, once_time_passed, task_finished, limits_exhausted,
scheduler_not_running on create, trigger returns outcome/existing/thread_id,
task responses add automatic_runs_used and active_run_status.
- Coded errors {code, message, params} for routes, shared validation and the
chat capability (scheduled_task_errors.py), pinned by
contracts/scheduled_task_errors_contract.json; goal notes contract v2 with
check_failure_codes, scheduled_origin_key and host_run_errors (host_notes.py).
- /api/features adds scheduled_tasks {available, running, tool_enabled,
min_interval_seconds}; ScheduledTaskService.is_running.
- Docs: six README scheduler sections, API.md scheduled tasks section
(breaking error shape), CONFIGURATION.md, AGENTS notes, CHANGELOGs.
- Tests: migration 0031, stop rule, error contract, run listing, lifecycle
over HTTP, repository, service and capability cases; updated tests for the
new error shape, finished-task pause and head revision.
* feat(scheduler): conversation update/resume, run-conversation authority and readable launches
- schedule_task: update and resume actions, stop_condition and clear_fields
parameters, plain parameter types (value rules come back as coded results),
rewritten docstrings for schedule_task and stop_scheduled_task.
- Capability: update/resume share the REST validation and error codes;
interactive turns in a task's run conversation may manage that task;
scheduled runs keep only own-stop; grants require a running scheduler.
- Results: compact JSON-safe task views with local times, display card/text,
last run in list, timezone_source/browser_timezone, now_local; no ISO, user,
lease or origin fields; stop_instructions removed.
- Timezone: explicit > browser (context.client_timezone, validated in
start_run and never forwarded to the run config) > not needed for intervals
and offset run_at > timezone_required.
- Trial gate accepts natural direct requests after acknowledgements and
particles; trial results report thread_id and existing.
- Scheduled launches: stable message id scheduled-<task_run_id>, server-owned
deerflow_scheduled_origin metadata with the user-language parts, run thread
title for fresh-thread tasks.
- Worker records goal continuations in the verdict; lead prompt language rule
covers stored task text and scheduled runs.
- Docs: six READMEs, CONFIGURATION, API, runtime/tools AGENTS, CHANGELOGs.
* feat(frontend): scheduled-task core types, formatting, errors and glossary
- throwGatewayApiError now throws GatewayApiError (status, code, params,
rawMessage); message stays the string detail or detail.message
- scheduled-task types gain stop_condition, automatic_runs_used,
active_run_status, origin/notes, run_number/total_tokens/summary, thread
relation, trigger result, renewal and scheduled origin; isTaskBusy and
hasActiveRun helpers
- api: fetchScheduledTask, resume with optional renewal body, typed trigger
result and update payload with null clears
- hooks: polling at 3 s while a run is active (list, thread list, run
history), useScheduledTask with live gating and final 404, useInView,
localized error toasts; run history refreshes tasks when a run finishes
- new pure modules: errors.ts (contract code to localized copy), format.ts
(relative task times, viewer time, display zone), status.ts, actions.ts,
run-error.ts; describeTaskSchedule in cron.ts; goal-outcome adds unchecked
goal checks, describeTaskOutcome and parseStopRunId
- features: scheduled_tasks block with defaults when missing
- buildRunContext sends client_timezone
- titleOfThread takes a localized untitled label; breadcrumb names scheduled
tasks and agents; reference chips use the localized untitled label
- schedule input: shared browserTimeZone, labelled selects, aria-pressed
type toggles, interval minimum from the hint template
- i18n: all PR1 scheduled-task keys in en-US and zh-CN, glossary fixes,
recipe instructions moved into i18n
- e2e mocks follow the new task, trigger, resume, thread-task and features
shapes; expectNoRawIdentifiers helpers for e2e and DOM tests
- tests for errors, format, cron, status, actions, goal outcome, run errors,
hooks, run context, thread titles and features; scheduled e2e copy updated
* feat(frontend): list-first scheduled tasks page with outcomes and editable goals
- page.tsx is now a thin composition of components/workspace/scheduled-tasks:
status tabs (All/Active/Paused/Finished) and search over the task list,
?task_id= deep link with router.replace on selection, ?thread_id= chip
with Show all, empty state, scheduler-off notice and unavailable state
- task list rows show the schedule in words and one status line (next run,
running now, paused by agent, auto-paused, finished by its cap)
- task detail: status badge, actions from availableActions with reasons,
More actions menu (Duplicate, Copy task ID, Delete), outcome notice,
Runs / Stops when / Each run's goal / Does / Notes from chat / History,
origin chat link; success toasts for every action
- run rows: run number or Trial run, summary, goal and assumption badges,
continuations, tokens, Open chat link; raw errors and the evaluator
reason only behind Details
- create/edit/duplicate dialog with goal, stop condition, safety caps and
advanced agent/context options; PATCH sends only changed fields, null
clears a cap
- renew dialog raises or removes caps and resumes in one request; a
limits_exhausted Resume opens it
- removed i18n keys the old page used; added feedback.resumedNoTime
- recorded en-3 and zh-1 page fixtures; scheduled e2e specs rewritten for
the new page; DOM tests for the detail and the form dialog
- CHANGELOG entries and the frontend AGENTS.md scheduled-task note
* feat(frontend): live schedule card, run conversation provenance and header entry
- schedule_task results (create, update, pause, resume, trial, delete) become
an assistant:scheduled-task message group rendered as a live card: schedule,
next run, stop condition with the safety cap, goal, where results go, Run
once now, Pause/Resume and Open task; the card polls the task only while it
is on screen and shows deleted, scheduler-off and limits-exhausted states
- one card per task per assistant turn; the card group is not a turn boundary
for usage, branching, editing, outline or skill attribution
- a human message with deerflow_scheduled_origin stays a human group with
scheduledOrigin and renders as a "Scheduled run / Trial run" block with the
task instructions, stop condition and notes collapsed; the launched text is
never shown and the message is not editable
- readable step labels for schedule_task actions and stop_scheduled_task
- ThreadScheduledTasksButton replaces the header link: hidden without tasks,
shows the count, links to the task (or the chat filter), and a run's chat
links to its task
- recorded en-3 / zh-1 chat and run-thread fixtures; chat-card and run-thread
e2e specs; unit and DOM tests for parsing, grouping, turn boundaries, card,
run block, header button, step labels and reconnect hydration
- i18n: header.countLabelOne added, unused card.next removed; CHANGELOG and
frontend AGENTS.md notes
* fix(scheduler): address verification findings
Move the scheduled-task lifecycle guidance from backend/AGENTS.md into the persistence guide and leave a one-line pointer, so the effective AGENTS.md chains for agents/middlewares, sandbox and subagents stay under the 98304-byte hard limit enforced by scripts/check_agent_guidance.py in lint-check.
* fix(scheduler): address review findings
* fix(scheduled-tasks): polish from live acceptance
- zh schedule wording: an interval of one unit reads 每分钟 / 每小时 / 每秒,
and a saved interval of whole days reads 每天 / 每 2 天 (Every day / Every 2
days) instead of 每 1 分钟 or 每 24 小时; several units keep the number
- zh spacing: 已于{time} 满足 and 在{time} 的运行中 now leave a space between
the interpolated time and the Chinese after it; a copy test guards every
scheduled-task zh string against a placeholder glued to Chinese
- run summary: when the first line of the final reply only leads in to a list
(ends with ":" or "："), the list items follow it as plain text (bullets,
numbering, checkboxes, bold, code and links stripped), joined with "；" for
CJK text and "; " otherwise, within the 160-character cap
- a schedule_task result (create, update, pause, resume, trial, delete) that
arrives in a chat invalidates that chat's task list, the tasks list and the
named task, so the header button and cards update without waiting for the
idle poll; results already in the loaded history do not refetch
- e2e and unit fixtures are now the live acceptance recordings (DeepSeek
Flash, demo data, fictional names): live-weekday.json (create, trial, edit,
pause, resume in one zh chat) and live-minute.json (per-minute chat, run 1
and run 2, paused by the agent), fetched with GET from the demo Gateway and
trimmed; the hand-shaped en-3 / zh-1 fixtures are removed and the chat-card,
run-thread and tasks-page specs run in both locales over the recorded data
* docs(changelog): reference #6378 for the scheduled-tasks entries
* docs(agents): shorten the scheduled-task pointer to stay under the guidance limit
* test(scheduled-tasks): pin the live takes to their recorded time zone
The live fixtures were recorded in Asia/Shanghai, and an interval task
reads in the viewer's zone, so the 16:48 assertions failed on UTC CI
runners. Pin the browser zone in the two live-take e2e specs and the
zh live-take unit test.
* fix(scheduled-tasks): keep untouched end times and drop a chat filter the new task is not in
- The edit, duplicate and renew dialogs keep a stored end time's instant
while its field still shows it. A minute-precision wall value cannot
keep seconds or name the later 01:30 of a DST fall-back, so a
title-only edit moved the cap an hour earlier, and raising the run
limit re-sent the end time without its seconds.
- Creating or duplicating a task from a chat's task view drops the chat
filter when the new task does not belong to that chat, instead of
showing "This task isn't one of this chat's tasks".
* fix(scheduled-tasks): tie pause notices to the pause they describe
- Paused by agent links and times only the run named in last_error. When
that run is not loaded, the notice shows no link and the task's own time
instead of an earlier run that also asked to stop.
- The outcome notice reads the latest history page whichever page is
shown, so browsing older runs no longer swaps the auto-pause reason and
"Open latest run" to an earlier miss. It shares that page's request.
* fix(scheduled-tasks): three lifecycle gaps found while re-checking review classes
- The edit form compares a cron in the form its schedule input emits, so an
untouched "0 09 * * 0,6" is not re-sent as "0 9 * * 6,0" and a title-only
edit no longer re-arms or refuses a finished task.
- Chat resume asks for a timezone, as update does, when a naive end_at would
land in the UTC placeholder of a zone-free task.
- Re-arming a task clears a host pause marker, as resume does, so a later
manual pause no longer reads as "Paused by agent".
* fix(scheduled-tasks): keep the notice's latest-runs query from refreshing older pages
useLatestScheduledTaskRuns kept React Query's default refetch on mount,
focus and reconnect, so going offline and back while an older history page
was shown fetched the latest page again. Refreshing stays with the history,
which refreshes the latest page only while it is shown.
* test(scheduled-tasks): let the viewer's time carry its own day label
Between 16:00 and 24:00 UTC, UTC and Asia/Shanghai are on different days,
so the run prompt shows "Tomorrow 01:00 · Today 09:00 your time" and the
fixed "01:00 · 09:00 your time" assertion failed depending on the hour.
---------
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|
||
|
Nan Gao
|
5872d07e9a
|
fix(gateway): gate declared multi-instance deployments and require the redis stream bridge (#6328)
* fix(gateway): gate declared multi-instance deployments and require the redis stream bridge GATEWAY_WORKERS / WEB_CONCURRENCY only count the uvicorn workers of one process tree, so a Kubernetes Deployment with replicas > 1 (one worker per Pod) passed every multi-worker startup check while each Pod's startup orphan reconciliation wrote the other Pods' lease-less runs off as crashed on every rolling update. Add `deployment.multi_instance` as an explicit declaration that runs the same gate, plus a DEER_FLOW_MULTI_INSTANCE env override so deploy tooling can set the declaration from its replica count without editing config.yaml (wiring the Helm chart to export it is a separate change). Both the worker count and the declaration now also require a Redis stream bridge and refuse an explicit memory sandbox ownership store, since the memory bridge and ownership store are process-local. The agent-storage divergence warning, the inbound webhook dedupe warning and the WeChat QR-login guard honor the same declaration. * docs(changelog): link the multi-instance gate entry to #6328 * fix(ci): bump chart config_version to 53 and keep AGENTS.md under the size limits config.example.yaml moved to config_version 53 for the new deployment section, so the Helm chart's embedded config and its README example follow. The multi-instance notes in backend/AGENTS.md and backend/app/gateway/AGENTS.md shrink to one line each: main already sits within 55 bytes of the per-file hard limit for the gateway guide and within 163 bytes of the chain hard limit under backend/, and the full contract lives in backend/docs/CONFIGURATION.md. * fix(gateway): name every active knob in the multi-process rollback When GATEWAY_WORKERS > 1 and a multi-instance declaration are both active, the refusal offered only "Set GATEWAY_WORKERS=1", so an operator who followed it bounced through a second refusal on DEER_FLOW_MULTI_INSTANCE / deployment.multi_instance at the next start. The rollback now withdraws both knobs, and the browser-tools refusal takes the same path instead of the worker-count-only message. multi_instance_declaration() returns a frozen MultiInstanceDeclaration (source, value) with knob/rollback properties instead of a display string, so deps.py no longer branches on message prose. Tests pin the combined rollback text and that following it actually clears the gate. * docs(gateway): keep the gateway AGENTS.md under the size hard limit after merging main main grew backend/app/gateway/AGENTS.md by 30 bytes, which left it 25 bytes under the 49152-byte hard limit; the one-sentence multi-instance note from this branch pushed the merged file to 49164. Shorten that sentence by 18 bytes so scripts/check_agent_guidance.py passes on the merge result. * ci: re-run checks after test_stop_cancels_a_hung_driver_poll flaked on shard 4 tests/test_mcp_task_service.py::test_stop_cancels_a_hung_driver_poll timed out once on backend-unit-tests (4) for 2c16206ac. The MCP task service and its test are identical to main, main's run for cfe2023df passed, and 15 consecutive local runs pass; the test gives service.stop() a 1s budget while cancellation drain allows 5s, so a loaded runner can exceed it. Fork contributors cannot re-run a job, hence this empty commit. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
subhash7476
|
a558884c3a
|
perf(summarization): skip the summarizer LLM call for an input already proven no-op (#6318)
* perf(summarization): skip the summarizer LLM call for an input already proven no-op When the exact rendered summarizer prompt (plus candidate models) previously produced output byte-identical to the current summary_text, reuse that summary instead of calling the LLM. Compaction proceeds through the normal path, so messages, hooks and state transitions are unchanged. - Only proven no-op inputs are cached (bounded LRU, thread-safe); failed attempts are never cached and any cache error falls back to the LLM call. - The middleware:summarize event gains changes.llm_call_skipped. - Applies to both compact_state and acompact_state. Builds on the per-call telemetry in the P0 branch. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(summarization): keep fallback no-ops retryable --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Willem Jiang
|
cfe2023df3
|
test(backend): run make test in four parallel shards (#6324) | ||
|
subhash7476
|
20bd989fc1
|
feat(runtime): add per-LLM-call telemetry and summarization no-op event (#6236)
* feat(runtime): add P0 per-LLM-call telemetry and summarization no-op counter Measurement only; no runtime behavior changes. - RunJournal enriches llm.ai.response / llm.error metadata with langchain_run_id, caller_category, provider, model, stop_reason, status, token counts and rendered request size (chars). Unavailable values are None. - Summarization middleware emits a middleware:summarize event per summarizer call and counts outputs identical to the prior summary_text. The call is never skipped. - All telemetry fails soft. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(runtime): register summarize middleware tag, document telemetry contract - Add MIDDLEWARE_SUMMARIZE_TAG to the middleware event catalog and the contract's known_tags; emit the middleware:summarize event with it. - Document the optional per-call telemetry metadata on llm.ai.response and llm.error in contracts/run_event_stream_contract.json. - Measure request size by walking string leaves instead of json.dumps on the event-loop thread. - Initialise summarization call/no-op counters in __init__. - Move RunJournal telemetry docs to the runtime guide. - Add tests: llm.error without a start, replayed on_llm_end, and a DB-store metadata round trip. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(runtime): refresh token counters on usage replay --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
lilei
|
741948ba9b
|
fix: invalidate converted uploads after same-size source edits (#6165)
* fix: invalidate converted uploads after same-size source edits * fix: log converted upload ownership rejection reasons * docs: clarify source timestamp limits and thread ownership |
||
|
lilei
|
f840e843d3
|
fix: track converted upload ownership for outlines and listings (#6101)
* fix: track ownership of converted upload markdown * fix: invalidate overwritten companions and restore branch ownership * docs: explain legacy upload companion behavior on upgrade * docs: shorten upload companion guidance to fit instruction budget --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
creed
|
71f109275b
|
feat(serper): support relative search time ranges (#6113)
* feat(serper): support relative search time ranges Signed-off-by: 97three <2212371308@qq.com> * docs(serper): address review and trim inherited guidance --------- Signed-off-by: 97three <2212371308@qq.com> |
||
|
PeaceMaker-best
|
90d466c917
|
feat(mcp): propagate cache resets across workers (#6126)
* feat(mcp): propagate cache resets across workers Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> * docs: keep MCP reset guidance within instruction budgets --------- Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> Co-authored-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Daoyuan Li
|
f242164604
|
fix(uploads): preserve link errors during staged cleanup (#5756)
* fix(uploads): preserve link errors during staged cleanup * Keep published uploads successful when staging cleanup fails * docs: keep upload guidance within chain budget --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
YxinMiracle
|
519afe4041
|
fix(security): prevent external system-role message injection (#5651)
* fix(agnet): system prompt bug * fix(security): address system-role review feedback * fix(ci): keep agent guidance within size budget --------- Co-authored-by: YxinMiracle <“939157765@qq.com”> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Eilen Shin
|
bd995a6a26
|
fix(agent): align unattended prompt with tool policy (#4919)
* fix(agent): align autonomous interaction guidance * fix(agent): harden interaction policy selection * fix(gateway): protect legacy interaction flags * fix(channels): honor explicit interaction mode * docs(agent): reduce inherited guidance size * fix(agent): honor unattended policy across approval paths --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Hyeonsang Cho
|
b0cb3a3a8c
|
fix(scheduler): serialize the SQLite launch-budget claim (#5469)
* fix(scheduler): serialize the SQLite launch-budget claim claim_queued_run counts the executing occurrences and then promotes one row to launching. Postgres serializes that pair with a transaction advisory lock; SQLite had no counterpart. pysqlite does not begin a transaction for a SELECT, so the budget count ran in autocommit and the deferred transaction reserved the writer only at the promoting UPDATE. Claimers racing on distinct rows therefore read the same stale count, each passed its own status == 'queued' CAS, and max_concurrent_runs was exceeded. A manual trigger overlapping the poller reaches this concurrently within one process, and scheduler.multi_instance over a shared database file reaches it across processes. Two claims of the same row were already safe, which is why the existing coverage did not catch it. Take the writer before the count with BEGIN IMMEDIATE, the idiom ThreadMetaRepository already uses for its read-modify-write paths and the same reservation _lock_task makes for a parent row. The claim targets one row but the budget is global, so this has to be the database-wide writer rather than a row lock. * docs(changelog): reference #5469 in the SQLite launch-budget entry * test(scheduler): pin the launch-budget test's connection reuse The warm-up gather is what makes the claimers actually overlap, but it silently depended on the SQLite engine keeping pooled connections. If that engine ever moved to a non-pooling class, every claimer would open its own connection, the per-connection PRAGMA setup would stagger them, and this test would pass against an unserialized claim instead of failing -- the cold-pool case it exists to avoid. Assert that the warm-up left connections checked in. A non-pooling class does not implement checkedin() at all, so a missing counter reads as zero reuse and reports the same explanation rather than an AttributeError. Verified against NullPool: the guard fails with "NullPool left 0 connections pooled after the warm-up". Only pool_size connections survive the gather (the overflow is discarded), which is why the pre-fix failure is exactly five claimants over a cap of one rather than eight. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
alanhuangyoo
|
740dbc4b39
|
fix(subagents): keep the subagent system prompt through context compaction (#5454)
* fix(subagents): keep the subagent system prompt through context compaction A subagent carries its whole system prompt (role, report contract, skills, deferred tools) as the leading SystemMessage in state, because its agent is built with system_prompt=None. Summarization only rescued tagged reminders and the latest user message, so the first compaction summarized the prompt away and every later model call in that subagent ran without its instructions. Rescue system messages as well. The lead agent's prompt is not in state, and its in-state SystemMessages are the tagged reminders already rescued, so the lead chain is unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(compaction): clarify preservation contract and trim agent guidance --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
tiammomo
|
5d855e9b92
|
feat(scheduled-tasks): filter run history by occurrence status (#5384)
* feat(scheduled-tasks): filter run history by occurrence status Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> * fix(scheduled-tasks): share occurrence status contract --------- Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Xuehao Xu
|
56540fab01
|
fix(gateway): make recursion limit configurable (#5390)
* fix(gateway): make recursion limit configurable * docs: keep backend guidance within inherited size budget * fix(gateway): address recursion limit review feedback * docs(gateway): clarify recursion default scope --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Wenchao An
|
6d5d7bb1d5
|
feat(subagents): add opt-in parent context snapshots (#5367)
* feat(subagents): add opt-in parent context snapshots * test(subagents): package synthetic snapshot evaluation * fix(subagents): preserve output text and defer snapshot capture * docs(subagents): keep snapshot guidance within chain budget * fix(subagents): omit unpaired tool calls from snapshots * fix(subagents): safely omit unserializable snapshot media |
||
|
wutongyuonce
|
f17ca3777a
|
feat(extensions): add in-place upgrade that keeps private config (#5347)
* feat(extensions): add in-place upgrade that keeps private config Replace a managed local snapshot or re-pin an already-installed requirement without going through remove, which dropped plugins[].config. * fix(extensions): keep snapshot, enabled, and git re-pin on upgrade Rollback keys off staging_root so a failed snapshot rename cannot rmtree the live tree. Upgrade preserves plugins[].enabled. Re-pin identification uses tool.uv.sources so git upgrades adopt the existing plugin record instead of failing closed after uv already switched the revision. * test(extensions): cover requirement re-pin identification on upgrade Re-pinning deerflow-extension-demo==2.0.0 to ==3.0.0 leaves added_names empty, so identification must take the added_specs fallback. Assert private config/required/enabled survive and the lock records 3.0.0. * fix(extensions): reject upgrade of an uninstalled git source Bare git+ URLs are not named Requirements, so the pre-uv-add installed check was skipped and upgrade acted as install. Resolve them against [tool.uv.sources] in the extensions group before uv add. |
||
|
tiammomo
|
444bfb72ce
|
feat(scheduled-tasks): preview upcoming cron occurrences (#5381)
Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> |
||
|
tiammomo
|
f4e740bc49
|
fix(uploads): bound document outline and preview text (#5323)
* fix(uploads): bound document outline and preview text Bound each document outline title to 200 characters and each fallback preview to 2000 characters across its lines, including omission markers. Refs #5322 Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> * test(uploads): cover exact preview budget and restore title guidance Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> --------- Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> |
||
|
tiammomo
|
b9d6b16084
|
fix(uploads): recognize valid ATX headings in document outlines (#5316)
* fix(uploads): recognize valid ATX headings in document outlines Exclude hashtags and indented code from uploaded document outlines; normalize valid ATX heading markers. Fixes #5313 Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> * test(uploads): format long-heading regression command Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> --------- Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> |
||
|
shawn
|
bec0acf6b5
|
fix(subagents): make acceptance checks portable on Windows (#5162)
* fix(subagents): make acceptance checks portable on Windows * fix(subagents): reject drive-root path escapes * fix(subagents): preserve drive-root containment * fix(subagents): use Windows path casing rules * fix(subagents): reject drive-relative cd paths * fix(subagents): reject shell-dependent cd targets * fix(subagents): reject shell-dependent runner paths * fix(subagents): harden cross-shell acceptance checks * fix(subagents): reject ambiguous shell tokenization * fix(subagents): reject tokenizer segment drift * fix(subagents): reject ambiguous PowerShell syntax * fix(subagents): include all PowerShell quote delimiters * fix(subagents): fail closed on cross-family paths * fix(subagents): reject ambiguous Windows aliases * fix(subagents): reject PSDrive alias exclusions * fix(subagents): reject PSDrive-relative aliases --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
qian
|
9f4a7823e2
|
feat(title): use filenames for attachment-only conversations (#5304)
* feat(title): use filenames for attachment-only conversations * docs: trim upload guidance to fit inherited size budget * fix(title): bound attachment-only fallback titles --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
tiammomo
|
b809b7bc7d
|
fix(web-fetch): resolve relative URLs in extracted Markdown (#5310)
* fix(web-fetch): resolve relative URLs in extracted Markdown Pass the request URL through Jina and Browserless extraction and resolve link/image destinations before Readability removes document base tags. Preserve the optional legacy API and fallback text behavior. Fixes #5307 Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> * fix(web-fetch): address provider and base URL review feedback Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> * fix(web-fetch): preserve HTML source when resolving destinations Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> --------- Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> |
||
|
Hyeonsang Cho
|
556975f284
|
fix(gateway): gate github_token and disable_clarification on internal callers (#5338)
* fix(gateway): gate github_token and disable_clarification on internal callers `non_interactive` is honored only for internally-authenticated callers because it strips `ask_clarification` from the lead-agent toolset. The two sibling run-context keys reproduced that effect without the gate. `merge_run_context_overrides` forwarded `_CONTEXT_RUNTIME_ONLY_KEYS` regardless of `internal`, and `strip_internal_context_keys` scrubbed only `_CONTEXT_INTERNAL_CALLER_KEYS` -- so any session or PAT caller could set `disable_clarification` through `body.context`, or through the free-form `body.config` that `build_run_config` copies verbatim. That is not a milder flag than `non_interactive`: ClarificationMiddleware answers every clarification -- `risk_confirmation` included -- with "proceed without asking" instead of interrupting, and SandboxMiddleware reads the two keys as the same non-interactive signal. `github_token` rode the same path into `runtime.context`, where the bash tool exports it as `GH_TOKEN`/`GITHUB_TOKEN`, and a copy smuggled through `body.config['configurable']` reached the checkpoint store the context-only rule exists to avoid. Both keys are produced server-side by the channel run policies, which reach the Gateway over the internally-authenticated request channel, so gate them the same way: forward them only when `internal=True`, and scrub the union `_INTERNAL_ONLY_CONTEXT_KEYS` from both config sections for every other caller. Destination stays an orthogonal axis -- `_CONTEXT_RUNTIME_ONLY_KEYS` still land in `context` alone, never in checkpoint-persisted `configurable`. Regression coverage in tests/test_gateway_services.py pins both smuggling surfaces and replays the real start_run assembly order for a session caller and for an internal one, so the GitHub channel keeps carrying its minted token. * docs(changelog): record the internal-only run-context key gate (#5338) * docs(agents): keep the run-context note inside the AGENTS.md budgets The AG002 inherited-chain check failed at this head. The new backend section and the root scheduled-task sentence added 993 B to the root and backend guidance both the sandbox and middlewares chains inherit, pushing sandbox 6 B over the 98304 B hard limit and growing the middlewares chain, which main already exceeds by 155 B. An already-over chain is only tolerated while it does not grow, so the shared ancestors had to come back to their base size. Condensed the new material and removed prose the root file was duplicating: - The trust-boundary section keeps both gated surfaces, both helpers, the trust-vs-destination split, and the disable_clarification note in half the space. - The root scheduled-task bullet names all three internal-only keys and both smuggling surfaces while staying under its previous size. - Dropped the root `scheduler.recursion_limit` bullet, which restated backend/AGENTS.md:18 almost verbatim; its one unique fact (a YAML edit needs no Gateway restart) moved to that bullet. - Deduplicated the nginx routing sentence, which already deferred to the backend routing table, and tightened the waiver note's sequencing tail. Root and backend guidance now sit 50 B under their combined base size, so the sandbox chain returns to 97310 B and the middlewares chain no longer grows. Every file stays under its AG001 soft budget. |
||
|
tiammomo
|
0d4925305a
|
fix(infoquest): bound HTTP connect and read waits (#5315)
Apply an explicit 30-second connect/read inactivity timeout to InfoQuest reader, web-search and image-search calls. Fixes #5314 Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> |
||
|
tiammomo
|
dde131a808
|
fix(tavily): handle Extract responses without a title (#5280)
* fix(tavily): handle Extract responses without a title Closes #5270 Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> * docs(tavily): keep extraction guidance within instruction budget Keep the approved AGENTS file layout and inherited size limits. Follow-up for #5280; refs #5270. Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> --------- Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> |
||
|
tiammomo
|
05dc8f4123
|
fix(uploads): exclude fenced code from document outlines (#5281)
* fix(uploads): exclude fenced code from document outlines Closes #5271 Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> * docs(uploads): keep outline guidance within instruction budget Keep the AGENTS instruction chain within the upstream hard limit. Follow-up for #5281; refs #5271. Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> --------- Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com> |