0fe6e63a68
* feat(goal): record the met goal and keep goal state on the history head - Add a goal_outcome LastValue channel (GoalOutcomeState). Only the guarded clear of a satisfied ordinary goal writes it, in the same checkpoint as the clear; the reply anchor is the latest visible AI message with text, the reply the visible signature keys on. A goal stored without created_at still clears and gets a record. - write_thread_goal removes the record on every other goal write and lists it in channel_versions, new_versions and metadata writes when it changes. Threads that never had a record keep their checkpoint shape. The sync-only SqliteSaver and CachedHistorySaver, which take the #6448 sync fallback, store and drop it the same way. - goal_outcome is server-owned: POST /state and non-internal run input that carry it get 400. A POST /state goal write, or a goal in external run input, clears it. - A branch head written onto its replay base resets the channels the head lacks, so a goal or record that write_thread_goal removed does not come back. Branching from the met turn keeps the record without the met goal. - The POST /history head now carries an active goal and an achieved goal_outcome, also before any message exists, so the post-run history refetch keeps them. Older entries stay title/thread_data. - contracts/thread_goal_contract.json pins the chat stand-down and check-failure codes, the record shape and status, the head keys and the host-written reasons. * feat(frontend): show why a goal stopped and when it was met - The goal bar derives its state from the active goal, the goal_outcome record and the messages: set, in progress, continuing, stopped (limit, auto-continue off, no progress, token budget, missing evidence, failed run, unknown code), waiting for you, waiting, couldn't check the goal, paused and goal met. An 8/8 stand-down now reads "Stopped" with the next step instead of "Continuing 8/8", and a count-0 stand-down no longer looks like a fresh goal. - "Goal met" shows while its anchor reply is the latest visible message (a visible human message, or a visible AI message with text, moves the chat on), so a reload agrees with the live view. An active goal wins over the record, and this tab's /goal set or clear hides it. - useActiveGoal drops the optimistic goal when the record of that same goal arrives, which fixes the bar that stayed after a goal met on the first run (the clear frame has no goal key), also for a goal stored without created_at. Server goal values pass through isGoalState; a /goal status read uses the backend's own active check, so the bar, the toast and the edit lock agree with the server. - Row 2 gives the reason and what to do next; Details holds the full goal, the checker's own note (never host-written text, a check failure or a paused goal's older check), the assumption note and an unknown code, and starts collapsed in each new state. The bar is a labelled section with a live region that announces state changes only (another goal's same state counts) and re-reads its text after a language switch. - The edit pencil on the latest turn is shown disabled with "Editing is off while a goal is set" instead of hidden; the message toolbar shows on focus-within, so keyboard users can reach it. - A /goal set or clear rejected with 409 keeps the draft and says a run is still going; other failures keep the server's message and the draft. A /goal request replaced by a newer one leaves the composer alone, so a double submit no longer empties it before the second one fails. - Copy lives in inputBox.goalBar (en-US, zh-CN); a contract test reads contracts/thread_goal_contract.json, and the raw-identifier guard now catches stand-down codes and goal field names. * test(frontend): cover the goal bar on both chat pages - tests/e2e/goal-status.spec.ts runs every case on the chat page and the agent chat page, since each wires the goal bar and the edit lock itself: a goal stopped at 8/8, one waiting for details and one waiting on an open question card show their reason and next step, also after a reload; the locked edit pencil gets keyboard focus, shows its toolbar and tooltip, does not open the editor, and hides while a file uploads; Goal met after one automatic follow-up (In progress, Continuing 1/8, then "Auto-continued once") survives a reload and hides after the next visible message; a goal met on the first run of a new chat replaces the set bar and frees the pencil, and a new /goal replaces Goal met while its own run goes on; a /goal set or clear refused with 409 keeps the draft for the retry and shows its own busy toast. - The mock POST /history head carries only the contract's history_head_keys, with goal only while active and goal_outcome only when achieved; thread state and run values omit an unset goal instead of sending goal: null. PUT and DELETE /goal drop the met record, setGoalWritesBusy returns the Gateway's 409, and a repeat create of a requested thread id returns the thread as is, like the Gateway. - The raw-identifier guard also catches checker blockers and the met record's field names. - frontend/src/AGENTS.md: the GoalStatus sentence now names goal, goal_outcome and useActiveGoal, reworded in place. * test(frontend): keep the draft through a failed upload and a busy /goal #6412 fires onSent only once a send is dispatched, so a failed upload keeps the text, files and stored draft. A /goal the Gateway refuses with 409 never reaches sendMessage, and the composer rejects it so PromptInput keeps the text. Drive both through the real sendMessage in one composer: a failed upload keeps "summarize it" stored, a busy /goal keeps its text and stored draft with no run, and the retry saves the goal and starts one run with the objective, not the "/goal" text. After that run the composer is empty and no stored draft comes back. * fix(frontend): drop the stop condition's closing period before the pause clause A stored stop condition often ends with a period, so the schedule card read "…remain)., pauses itself" (and "…。，满足后自动暂停" in Chinese). Trim the condition's closing sentence punctuation before the clause that follows it. The event line, where the condition ends the sentence, is unchanged. * fix(frontend): point "evidence missing" at the agent, not a user reply "Evidence missing" asked the user to reply with what's left, which is the push a goal is meant to save. It now asks the agent to show the result or explain what's missing, and offers /goal clear only for a goal that no longer applies: clearing drops the met record, so work that is right would never show "Goal met". zh-CN follows. Copy only: no key, state or mapping changes, and the scheduled page's reason labels are untouched. tests/unit/components/workspace/goal-status-next-steps.test.ts checks the row in en-US and zh-CN, and that no stop row offers /goal clear without that condition. goal-status.dom.test.tsx and goal-stream.dom.test.tsx now expect the new copy. * fix(frontend): say "Couldn't check the goal" once in the goal bar After a goal check that could not run, the bar's second line borrowed the scheduled page's whole reason, so it repeated the chip: "Couldn't check the goal · Couldn't check the goal; the run itself may be fine. It's checked again after your next message." Chinese had the same repeat. The line now gives only why the check failed ("The run itself may be fine", "No final reply was saved" or "The chat changed during the check"), then when it is checked again. The screen-reader announcement follows. The chip and the scheduled page's reason labels are unchanged. tests/unit/components/workspace/goal-status-unchecked.test.ts checks every check-failure code in en-US and zh-CN: the label appears once, and the line still says why the check failed. * feat(goal): redact the evaluator input when pii_redaction is enabled The goal evaluator calls its model directly, outside PiiRedactionMiddleware, so it sent raw user lines, tool arguments, tool results and Human Input Card answers even with pii_redaction enabled. With pii_redaction enabled: - the messages in the evidence window are redacted whole before the evidence caps can cut an identifier in two, as TitleMiddleware does. This reuses redact_queued_messages, which already covers content, call arguments and card answers. Earlier messages are left alone: the evaluator never sees them, and the pass is synchronous work on the event loop (0.84 s for an 800-message, 4 MB thread, 0.06 s for its window); - the assembled input, the goal objective included, is redacted again. A redaction error fails the check with evaluator_failed; raw text is never sent instead. When redaction is off, the input is byte-identical. Thread state keeps the raw text. README.md, config.example.yaml and the middleware docstring list the evaluator with the other redacted inputs. tests/test_goal_pii.py puts an address where the evidence caps cut a tool argument, a tool result and a card answer, and checks the objective, the unchanged input when redaction is off, that only the evidence window reaches the redactor and gives the same evidence as redacting the whole thread, that a redaction error stands down with evaluator_failed before the model is called, and the worker path. * fix(goal): redact the hidden continuation when pii_redaction is enabled The hidden /goal continuation repeats the goal objective and the evaluator's reason and evidence summary. PiiRedactionMiddleware skips it, since it is a framework message and not a user message, so with pii_redaction enabled each continuation turn sent them raw to the agent's model, and the message stayed in the thread for later model calls. The durable-context block and the evaluator input already redact the objective. With pii_redaction enabled, make_goal_continuation_message now redacts the objective, the reason and the evidence summary with redact_text before the message is built. The thread keeps the redacted message, which the UI hides; the goal state keeps the raw objective. The worker builds the message before it counts the continuation. A redaction error fails the check the way an evaluator redaction error does: run_failed with stand_down_reason evaluator_failed, the goal stays active, nothing is sent and the continuation budget is not spent. Only the exception type is stored. With redaction off, the message is byte-identical. README.md, config.example.yaml and the middleware docstring list the continuation with the other redacted inputs. tests/test_goal_continuation_pii.py checks the redacted objective, reason and evidence summary against the tokens redact_text gives, the unchanged message with redaction off or no config, the worker path under the run config, and a redaction error. * docs(changelog): link the goal outcome entries to #6556 Fill the four ([#PR]) placeholders in CHANGELOG.md and CHANGELOG_zh.md with this PR's number and add the matching link references. * fix(frontend): keep an active goal stored without timestamps visible POST /threads/{id}/state and run input can store an active goal without created_at or updated_at. The backend still evaluates it and rejects an edit with 409 (is_active_goal checks only status "active"), but isGoalState required both timestamps, so history and live values dropped it: both chat pages hid the goal bar and unlocked the last-turn pencil, and only a /goal status read, which used a looser predicate, brought them back. isGoalState now accepts any record with a string objective and status "active", and normalizeGoalStatusRead uses it, so thread values, live updates and status reads share one rule. Malformed or inactive values are still dropped. Tests cover a sparse active goal from the history head and from a live update before the post-run refetch (bar shown, hasGoal true), and a malformed goal that stays hidden. * test(goal): check the outcome write metadata the same way on every saver PostgresSaver stores checkpoint metadata through get_serializable_checkpoint_metadata, which drops "writes", so the [postgres] case of test_goal_outcome_round_trips_through_the_sync_fallback failed with KeyError: 'writes' when CI ran it against a live database. The test now records the metadata write_thread_goal passes to put and asserts the clear's and the next goal's "writes" there, for every saver. The stored channel values are checked as before, and on sqlite and cached-sqlite the stored "writes" must still equal what was passed. * docs(agents): keep goal_outcome out of the agents guidance chain The middlewares AGENTS.md chain is already over the 98,304-byte AG002 hard limit on main, and the PR-mode check fails any chain a PR touches, so adding goal_outcome to the ThreadState field list in deerflow/agents/AGENTS.md failed the agent-guidance job. goal_outcome stays documented in runtime/AGENTS.md and backend/app/gateway/AGENTS.md, which are outside that chain. * refactor(goal): read the evaluator's evidence and redaction from one window format_visible_conversation recomputed the window start that the redaction pass takes from _evidence_window. Both now use _evidence_window, so a later change to the cap or to what counts as visible moves both. The window test now fills the window with tool-only replies and tool results. Before, it caught a cap change in one copy but not a change to what counts as visible. --------- Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Synthetic scheduled-work Gateway pilot
This runs the production Gateway, scheduler, conversation tools, goal worker, sandbox artifact tools and persistence. It does not provide another agent loop. All sources and expected artifacts are explicitly synthetic and pinned by the committed fixture's SHA-256. No external dataset or network source is fetched.
plan is offline. Only an explicit run-live starts paid agent work. The runner
never starts a Gateway, loads credentials, changes models or adds account funds.
Use a separately started, disposable loopback Gateway, a new DEER_FLOW_HOME
and SQLite directory, empty extension/MCP configuration, and disabled channels,
memory and automatic titles. Never point it at existing user data.
Required pilot configuration
The reproduction pilot uses the user's selected official DeepSeek V4.1 Flash
profile: alias and API model ID deepseek-flash, official endpoint
https://api.deepseek.com, non-thinking mode, temperature 0, output cap 1024,
SDK max_retries: 0, normal per-run token budget 60,000, the repository's default
recursion limit 1000 and request
timeout 120 seconds. Credentials belong only in the named DEEPSEEK_API_KEY
environment variable of the separately started Gateway.
The normal budget was raised after one pilot occurrence used about 25,000 tokens to produce and deliver a correct report. Use the same 60,000-token budget for both matched arms so that this early threshold does not confound their comparison. Provenance records the actual configured budget and the normal reference separately; the independent tiny-budget phase remains 1000 tokens.
The configured USD rates are conservative peak estimates: input 0.30 and output
1.20 per million tokens. The operator checked the
official pricing page for
this pilot. Provider billing and off-peak/cache discounts are not established by
the runner. Cost is calculated with production Console _run_cost from durable
RunRow usage, using the configured miss price conservatively.
The pilot configuration must include:
models:
- name: deepseek-flash
use: langchain_openai:ChatOpenAI
model: deepseek-flash
base_url: https://api.deepseek.com
api_key: $DEEPSEEK_API_KEY
temperature: 0
max_tokens: 1024
max_retries: 0
timeout: 120
extra_body:
thinking:
type: disabled
pricing:
currency: USD
input_per_million: 0.30
output_per_million: 1.20
database:
backend: sqlite
sqlite_dir: /ABSOLUTE/DISPOSABLE/DATABASE/DIRECTORY
scheduler:
enabled: true
tool_enabled: true
min_once_delay_seconds: 1
poll_interval_seconds: 1
max_concurrent_runs: 1
token_budget:
enabled: true
max_tokens: 60000
recursion_limit: 1000
max_recursion_limit: 1000
memory:
enabled: false
title:
enabled: false
sandbox:
use: deerflow.sandbox.local:LocalSandboxProvider
tools:
- name: read_scheduled_fixture
group: benchmark
use: scripts.benchmark.scheduled_tasks.fixture_tools:read_scheduled_fixture
- name: write_file
group: file
use: deerflow.sandbox.tools:write_file_tool
- name: read_file
group: file
use: deerflow.sandbox.tools:read_file_tool
- name: read_conversation
group: conversation
use: deerflow.tools.conversation:read_conversation
The fixture reader is operator-controlled Python configuration. It accepts only
four committed case IDs and never opens a caller-selected file or URL. Expected
answers are withheld from its return value. Actual reports use production
write_file and present_files, and are opened through the Gateway artifact
endpoint before an independent JSON/value grade is recorded.
Mark only the disposable database directory after Gateway initialized it:
touch "$SCHEDULED_PILOT_SQLITE_DIR/.scheduled-benchmark-disposable"
Commands
Run from backend/, with the exact configuration used by the separately
started Gateway. Each output directory must be new. Outputs belong in the
outer workspace's outputs/scheduler-agent-20261003/, outside the Git worktree.
uv run --no-sync python -m scripts.benchmark.scheduled_tasks plan \
--config "$SCHEDULED_PILOT_CONFIG" --model deepseek-flash
uv run --no-sync python -m scripts.benchmark.scheduled_tasks run-live \
--config "$SCHEDULED_PILOT_CONFIG" --model deepseek-flash \
--base-url http://127.0.0.1:8023 --phase lifecycle \
--output "$SCHEDULED_PILOT_OUTPUT/lifecycle" \
--max-cost 1 --reserve-per-run 0.08 --timeout 180 --interval 90
uv run --no-sync python -m scripts.benchmark.scheduled_tasks run-live \
--config "$SCHEDULED_PILOT_CONFIG" --model deepseek-flash \
--base-url http://127.0.0.1:8023 --phase paired \
--output "$SCHEDULED_PILOT_OUTPUT/paired" \
--max-cost 1 --reserve-per-run 0.08 --timeout 120
For --phase budget, deliberately restart/reconfigure the disposable Gateway
with token_budget.max_tokens: 1000 and pass that exact new config file. The
runner requires a tiny budget no greater than 2000, schedules one goal fixture,
and checks the durable token_capped stop or its recorded verdict stand-down
reason. The public history API projects title/thread-data/messages and omits the
goal channel. The benchmark reads the same marked disposable SQLite database
through the production CheckpointStateAccessor and native saver with
mode=ro and query_only=ON, using the configured full/delta mode and cadence.
It reads the complete root history, requires zero continuations and counters
within their recorded limit, checks
goal cleanup, and observes no later lead-model turn or second occurrence. Missing
counter evidence fails the phase. It records actual usage, including any one-call
overshoot. Do not change the model or prices for this phase.
An already recorded tiny-budget run can be rechecked without a Gateway or a provider call. Use its exact original configuration and a new output file:
uv run --no-sync python -m scripts.benchmark.scheduled_tasks recheck-budget \
--config "$SCHEDULED_PILOT_BUDGET_CONFIG" --model deepseek-flash \
--run-output "$SCHEDULED_PILOT_OUTPUT/budget-1" \
--output "$SCHEDULED_PILOT_OUTPUT/budget-evidence-recheck-v2.json"
The reader is constrained to the disposable default-owner lead assistant and base schema without custom/plugin middleware. It preserves the original failure/results, matches the actual run's objective, status, calls and usage, and labels the new result as a read-only persisted-state recheck of the existing real Gateway run. Missing snapshots or counters fail; no zero is fabricated.
The USD 1 admission guard sums all runs in the disposable database, including earlier pilot phases, and reserves USD 0.08 before each new run. It polls live usage and cancels active pilot work after a guard failure. This is an estimated spend guard, not a hard provider billing limit: usage becomes visible after calls, token budgets can overshoot once, and billing may differ from configured rates. The reserve and observed usage are preserved in provenance and results. No application-level or provider retries are added by the runner.
What each phase proves
lifecycle creates one schedule through schedule_task in a real conversation.
An automatic fresh-thread occurrence reads the fixture and produces the main
branch report. The runner pauses between occurrences solely to avoid a note
timing race, adds the exact user note through the same original conversation,
then resumes. The second automatic occurrence must read the first through
read_conversation, follow the develop note, write the independently graded
report and call stop_scheduled_task. The runner checks the durable stop-run ID,
parent paused state, different occurrence threads and no next dispatch. Its
temporary between-occurrence pause does not count as self-stop evidence.
An unsuccessful creation or note run, or failed first artifact/source grade,
fails immediately and cleans up instead of paying for the next stage. A task
committed by the creation tool is discovered for cleanup even if the surrounding
interactive run fails. Preserve failed configuration trials separately. A small
graph recursion budget can stop a correct model response because the middleware
chain uses many supersteps per turn; the pilot records configuration failures
without counting them as model loops or implementation defects.
paired uses three matched synthetic inputs, with alternating arm order and
fresh threads. Fixture setup uses production ScheduledTaskRepository.create
and shared Gateway creation validation on one real HTTP-created owner origin
thread. Both arms carry the same origin binding, sources, prompt, output name,
model settings and budget; only the goal field differs. This setup bypasses the
conversation creation step to avoid charging setup-model calls to the study;
the lifecycle phase independently covers that user-facing step. The real
Gateway poller and ordinary run worker execute every occurrence.
An alleged goal arm is rejected unless its actual run has goal metadata and a
durable goal_verdict. A plain arm is rejected if either appears. Artifact and
source-use grades are independent of the evaluator's verdict. Delivery is checked
separately against the host's run.delivery receipt and its successful
present_files path binding; an AI claim or requested tool-call name cannot
establish delivery. Effective independent completion requires correct artifact
values, source use, host delivery and successful run status. A subsequent
token_capped stop does not erase verified completion; a capped run without
delivery remains incomplete. The cap flag, goal verdict and all evidence columns
are preserved separately for analysis of partial results and evaluator false
negatives. Results retain
the raw occurrence status, goal verdict, lead-model turns, total model calls,
input/output/per-model tokens, peak-price cost estimate and artifact hashes.
The summary reports each arm descriptively. Three synthetic cases are not a
statistical or general completion-rate claim, and zero benefit is a valid result.
Recovery and delivery limits
The lifecycle phase records restart_recovery: not_exercised_by_this_phase.
Testing a crash between durable self-stop and completion needs a separately
controlled pause/barrier in the disposable agent and a restart of the exact
Gateway process the operator started. Preserve the stop-request row before the
crash, then verify recovery pauses the parent, leaves no further dispatch and
that a repeated completion after user resume cannot pause it again. Offline
repository/runtime regressions cover this contract; do not relabel them as a
real-Gateway restart observation.
No provider-account IM delivery is attempted. Writing an outbox row is not
evidence of receiving a WeCom notification. The runner reports delivery as
not_tested and never reads or messages existing user chats/platform accounts.
Provenance stores config/fixture/prompt hashes, tracked Git diff hash, explicit untracked Python/contract/fixture source hashes, model parameters, timestamps, timeouts and rates. It never stores credentials, response headers or complete provider request payloads. Failures and cleanup failures remain visible, and terminal task definitions are preserved. If cleanup cannot pause active work, stop the disposable Gateway before leaving it unattended.
Offline validation:
uv run --no-sync pytest tests/test_bench_scheduled_tasks.py -q