diff --git a/CHANGELOG.md b/CHANGELOG.md index af3d20be37..180ed5518a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -208,6 +208,22 @@ This release closes that milestone with **439 merged pull requests**. #### Agents & runtime +- **goal:** A met chat goal is now recorded on the thread, and the history + head keeps goal state. When the evaluator finds a goal met, the checkpoint + that clears it also writes a new `goal_outcome` channel: the objective, when + the goal was set and met, the continuations used and allowed, the + evaluator's reason, `relied_on_assumption` and the id of the reply it was + judged on. Any later goal write removes it, and scheduled-task goals do not + write it. Clients cannot set it: `POST /state` and non-internal run input + that carry `goal_outcome` get 400. The + `POST /api/threads/{thread_id}/history` head now carries an active `goal` + and an achieved `goal_outcome`, so a stopped goal no longer disappears from + the web UI after a run ends or the page reloads; older entries are + unchanged. A chat branched from an earlier turn no longer brings back a goal + or record that turn did not have. `contracts/thread_goal_contract.json` pins + the stand-down and check-failure codes, the record's keys, the history head + keys and the host-written reasons. ([#6556]) + - **gateway:** Threads the server creates for you can be noticed without a reload. Runs started by a schedule, an IM channel, a GitHub agent, an extension or an MCP notification now carry a server-owned @@ -802,6 +818,19 @@ This release closes that milestone with **439 merged pull requests**. chip is removed, older drafts restore their skill selection as an inline reference, and manually typed legacy slash text still submits as a normal message. ([#6154]) +- **frontend:** The goal bar above the composer now says what an active + `/goal` is doing and why it stopped. It shows "In progress" while a run works + on the goal, and after auto-continue stands down "Stopped", "Waiting for + you", "Waiting", "Couldn't check the goal" or "Paused" with the reason and + the next step on a second line, where it used to keep showing + "Continuing 8/8" or look freshly set; "Details" holds the full goal and the + checker's own note. A met goal shows "Goal met" (and how many times it + auto-continued) until the chat moves on, also after a reload, and the bar no + longer stays after a goal met on its first run. While a goal is set, the + latest turn's edit pencil is shown disabled with a tooltip instead of hidden, + and the message toolbar also appears on keyboard focus. A `/goal` refused + because a run is still going keeps the draft and says so. ([#6556]) + ### Fixed - **runtime:** The JSONL event store no longer loses events after a torn final @@ -4201,6 +4230,16 @@ This release closes that milestone with **439 merged pull requests**. messages are rebuilt via `model_copy` so originals are never mutated. The gate stays off by default and behavior is unchanged while unset. ([#5577]) +- **goal:** Enabling `pii_redaction` now also redacts the `/goal` evaluator's + input. The evaluator calls its model directly, outside + `PiiRedactionMiddleware`, so it sent raw user lines, tool arguments, tool + results and Human Input Card answers. The messages it reads are now + redacted whole before the evidence caps can cut an identifier in two, and + the assembled input, goal objective included, is redacted again; thread + state keeps the raw text. A redaction error fails the check + (`evaluator_failed`) instead of sending raw text. With redaction off, the + input is byte-identical. ([#6556]) + - **authz:** Skill authorization is now enforced at agent assembly and activation, so an RBAC policy such as `skills: {allow: ["data-analysis"]}` can actually deny a skill — Phase 2A (#4439) covered Gateway routes, but @@ -4213,6 +4252,16 @@ This release closes that milestone with **439 merged pull requests**. stamped `skill_context_denied`, so durable context, skill allowed-tools, and autonomous secret bindings never activate the denied skill. ([#4541]) +- **goal:** Enabling `pii_redaction` now also redacts the hidden `/goal` + continuation. `PiiRedactionMiddleware` skips this framework message, so + each continuation turn sent the goal objective and the evaluator's reason + and evidence summary raw to the agent's model, and they stayed in the + thread for later model calls. They are now redacted before the message is + built; the thread keeps the redacted message, which the UI hides, and the + goal state keeps the raw objective. A redaction error fails the check + (`evaluator_failed`) before the continuation is counted, instead of + sending raw text. With redaction off, the message is byte-identical. ([#6556]) + - **lark:** The opt-in Lark broker subcommand denylist (`DEERFLOW_LARK_BROKER_DENY_SUBCOMMANDS`) can no longer be bypassed by an option value passed as its own token. Matching dropped only `-`-prefixed @@ -9346,3 +9395,4 @@ with **180 merged pull requests** since the first 2.0 milestone tag. [#6520]: https://github.com/bytedance/deer-flow/pull/6520 [#6537]: https://github.com/bytedance/deer-flow/pull/6537 [#6543]: https://github.com/bytedance/deer-flow/pull/6543 +[#6556]: https://github.com/bytedance/deer-flow/pull/6556 diff --git a/CHANGELOG_zh.md b/CHANGELOG_zh.md index d82d04b2bc..9446e666e3 100644 --- a/CHANGELOG_zh.md +++ b/CHANGELOG_zh.md @@ -161,6 +161,17 @@ #### 智能体与运行时 +- **目标:** 对话中达成的目标现在会记录在线程上,历史接口的最新一条也会保留目标状态。 + 评估器判定目标达成时,清除目标的同一个 checkpoint 会写入新的 `goal_outcome` 通道: + 目标原文、设置和达成的时间、已用和允许的续跑次数、评估理由、`relied_on_assumption` + 以及判定所依据回复的 id。之后任何一次目标写入都会删除它,定时任务的目标不写这条 + 记录。客户端无法写入:`POST /state` 或非内部的运行输入携带 `goal_outcome` 时返回 + 400。`POST /api/threads/{thread_id}/history` 的最新一条现在带上激活的 `goal` 和已 + 达成的 `goal_outcome`,停下的目标不会再在运行结束或刷新页面后从 Web UI 消失;其余 + 条目不变。从较早一轮分支出的对话不会再带回那一轮没有的目标或达成记录。 + `contracts/thread_goal_contract.json` 固定了停止代码、检查失败代码、达成记录的 + 字段、历史头字段和主机自写的原因文本。([#6556]) + - **网关:** 服务端替你创建的对话无需刷新即可被发现。由定时任务、IM 渠道、GitHub 智能体、扩展或 MCP 通知发起的运行,现在带有服务端维护的 `metadata.deerflow_origin`(`{kind, provider?, namespace?}`,由新增的 @@ -692,6 +703,15 @@ 旧的已选技能气泡被移除,旧草稿中的技能选择会恢复为内联引用,手动输入的 旧式斜杠文本仍按普通消息提交。([#6154]) +- **前端:** 输入框上方的目标栏现在说明激活的 `/goal` 在做什么、为什么停下。运行推进 + 目标时显示“进行中”;自动续跑停下后显示“已停止”“等你回复”“等待中”“未能检查目标” + 或“已暂停”,第二行写明原因和下一步,不再一直显示“续跑中 8/8”或看起来像刚设置的 + 目标;“详情”里有完整目标和评估器原话。目标达成后显示“目标已达成”(以及自动续跑 + 了几次),直到对话继续往下走,刷新后也一样;第一次运行就达成的目标不会再残留在 + 目标栏里。设置了目标时,最后一轮的编辑铅笔改为灰显并附提示,而不是直接隐藏;消息 + 工具栏在键盘聚焦时也会显示。因还有运行未结束而被拒绝的 `/goal` 会保留草稿并说明 + 原因。([#6556]) + ### 修复 - **运行时:** JSONL 事件存储在末行写坏后不再丢失事件。写入中途被打断会让文件缺少结尾换行,下一次追加会接在残缺记录后面, @@ -3275,6 +3295,13 @@ 脱敏;缓存负载本身即为干净副本,消息通过 `model_copy` 重建、绝不改动原件。 开关默认关闭,未设置时行为不变。([#5577]) +- **目标:** 开启 `pii_redaction` 后,`/goal` 评估器的输入现在也会被脱敏。评估器直接 + 调用模型,不经过 `PiiRedactionMiddleware`,因此此前会发送原始的用户消息、工具参数、 + 工具结果和 Human Input Card 回答。现在先对评估器读取的完整消息脱敏,避免证据截断把 + 标识符切成两半,再对拼好的输入(包括目标原文)脱敏一次;线程状态保留原文。脱敏出错 + 时这次检查按失败处理(`evaluator_failed`),不会发送原文。关闭脱敏时输入逐字节不变。 + ([#6556]) + - **授权:** 技能授权现在在装配与激活两个环节强制执行,`skills: {allow: ["data-analysis"]}` 这类 RBAC 策略因此能真正拒绝某个技能——Phase 2A (#4439)覆盖的是 Gateway 路由,技能此前仍只受 agent 配置白名单约束。 @@ -3285,6 +3312,13 @@ `read_file` 读取 `SKILL.md` 时会打上 `skill_context_denied` 标记,持久上 下文、技能 allowed-tools 与自主密钥绑定都不会激活被拒绝的技能。([#4541]) +- **目标:** 开启 `pii_redaction` 后,隐藏的 `/goal` 续跑消息现在也会被脱敏。 + `PiiRedactionMiddleware` 不处理这条框架消息,因此每次续跑都会把目标原文以及评估器的 + 理由和证据摘要原样发给 agent 的模型,并留在线程里供之后的模型调用读取。现在先对这些 + 内容脱敏,再拼成消息;线程保存脱敏后的消息(界面不显示),目标状态仍保留原文。 + 脱敏出错时这次检查按失败处理(`evaluator_failed`),不计入续跑次数,也不会发送原文。 + 关闭脱敏时消息逐字节不变。([#6556]) + - **Lark:** 可选的 Lark broker 子命令拒绝列表 (`DEERFLOW_LARK_BROKER_DENY_SUBCOMMANDS`)不再能被以独立 token 传入的选项 值绕过。此前匹配只去掉以 `-` 开头的 token 并从头比较剩余部分,因此 @@ -7561,3 +7595,4 @@ DeerFlow 2.0 是围绕"超级智能体"框架的彻底重写,核心包含子 [#6519]: https://github.com/bytedance/deer-flow/pull/6519 [#6520]: https://github.com/bytedance/deer-flow/pull/6520 [#6537]: https://github.com/bytedance/deer-flow/pull/6537 +[#6556]: https://github.com/bytedance/deer-flow/pull/6556 diff --git a/README.md b/README.md index fdcdff4278..a5b77ff59f 100644 --- a/README.md +++ b/README.md @@ -1828,9 +1828,9 @@ Supported commands: /goal clear # clear it ``` -After each Gateway-backed run, DeerFlow evaluates the visible conversation, including the assistant's tool calls and shortened tool results, against the active goal with a non-thinking evaluator model. A successful tool result alone does not satisfy a goal, and when the assistant had to guess missing or ambiguous information the evaluator reports `needs_user_input`. The evaluator must return a typed blocker (`missing_evidence`, `needs_user_input`, `run_failed`, `external_wait`, or `goal_not_met_yet`) plus visible evidence. DeerFlow only injects a hidden continuation when the latest assistant turn is durably checkpointed, the blocker is `goal_not_met_yet`, the thread did not change during evaluation, and the no-progress breaker has not fired. The safety cap defaults to 8 hidden continuations, and repeated identical non-progress evaluations stop after 2 attempts. `/goal clear` and any user-authored new input win over queued continuations. When the evaluator considers the goal satisfied, DeerFlow clears it after the run successfully finalizes and publishes the updated thread state. If required artifact delivery or its receipt persistence fails, the run reports an error and the goal remains active for a later retry; this does not start another hidden continuation. +After each Gateway-backed run, DeerFlow evaluates the visible conversation, including the assistant's tool calls and shortened tool results, against the active goal with a non-thinking evaluator model. A successful tool result alone does not satisfy a goal, and when the assistant had to guess missing or ambiguous information the evaluator reports `needs_user_input`. The evaluator must return a typed blocker (`missing_evidence`, `needs_user_input`, `run_failed`, `external_wait`, or `goal_not_met_yet`) plus visible evidence. DeerFlow only injects a hidden continuation when the latest assistant turn is durably checkpointed, the blocker is `goal_not_met_yet`, the thread did not change during evaluation, and the no-progress breaker has not fired. The safety cap defaults to 8 hidden continuations, and repeated identical non-progress evaluations stop after 2 attempts. `/goal clear` and any user-authored new input win over queued continuations. When the evaluator considers the goal satisfied, DeerFlow clears it after the run successfully finalizes and publishes the updated thread state. The same checkpoint records the completion as `goal_outcome`, which stays until the next goal change. If required artifact delivery or its receipt persistence fails, the run reports an error and the goal remains active for a later retry; this does not start another hidden continuation. -The Web UI shows the active goal above the composer. The same command is available from the TUI and supported IM channels. In the Web UI and supported IM channels, setting `/goal ` also starts a run with the condition as the task; status and clear commands only manage goal state. Setting or clearing a goal is rejected while that thread has a run in flight, including a run owned by another Gateway worker, so the goal checkpoint cannot branch away from an active run's checkpoint lineage. +The Web UI shows the active goal above the composer, says why auto-continue stopped, and shows a "Goal met" notice after a goal is met. Editing the last message is off while a goal is set. The same command is available from the TUI and supported IM channels. In the Web UI and supported IM channels, setting `/goal ` also starts a run with the condition as the task; status and clear commands only manage goal state. Setting or clearing a goal is rejected while that thread has a run in flight, including a run owned by another Gateway worker, so the goal checkpoint cannot branch away from an active run's checkpoint lineage. When your role lacks `runs:create`, the Web UI rejects a new task or `/goal ` before preparing the thread or saving the goal, and keeps your draft for retry. Goal status, goal clearing, and `/compact` remain governed by their own endpoint permissions. @@ -1850,14 +1850,16 @@ DeerFlow adds a reminder using the latest task statuses before the next model call. Skipped or failed compaction leaves the existing messages unchanged. Optional `pii_redaction.enabled` redacts detected identifiers in user messages, -remote tool results, compaction input, reinjected summaries, and configured -LLM title input. Memory admission, including pre-compaction flushes, also redacts +remote tool results, compaction input, reinjected summaries, configured +LLM title input, and the `/goal` evaluator's input. Memory admission, +including pre-compaction flushes, also redacts detected identifiers in supported text/JSON content, parsed and invalid call arguments/error text, provider-raw/legacy function calls and supported user-content provenance. It is off by default; enabled deployments supply a secret for stable, keyed value-derived placeholders. No token-to-raw mapping is persisted. Raw thread text and local fallback titles remain available for display; memory redaction copies messages without changing the caller's history or tool execution. +The hidden `/goal` continuation is redacted before it is stored. The Web UI preserves persisted message order when merging history with live updates. Streaming steps around a persisted result inside the loaded history stay together, including steps that arrive after the result. Steps captured during compaction also remain visible before their persisted result when history has not refreshed and the UI has not rendered them yet. diff --git a/README_zh.md b/README_zh.md index 27b3f52866..fadd334169 100644 --- a/README_zh.md +++ b/README_zh.md @@ -754,9 +754,9 @@ Web UI 输入框支持浏览器侧语音听写。浏览器提供 Web Speech API /goal clear # 清除它 ``` -每次 Gateway 驱动的 run 结束后,DeerFlow 会用一个 non-thinking 的评估模型,把可见的对话内容(包括助手的工具调用和截短的工具结果)拿去和激活的 goal 比对。工具调用成功本身不代表 goal 已完成;如果助手不得不猜测缺失或含糊的信息,评估模型会报告 `needs_user_input`。评估模型必须返回一个带类型的 blocker(`missing_evidence`、`needs_user_input`、`run_failed`、`external_wait` 或 `goal_not_met_yet`),并附上可见证据。只有在最近一轮 assistant 回复已被持久化 checkpoint、blocker 是 `goal_not_met_yet`、评估期间 thread 没有变化、且无进展熔断器没有触发时,DeerFlow 才会注入一次 hidden continuation。安全上限默认是 8 次 hidden continuation;连续两次相同的无进展评估后就会停止。`/goal clear` 以及任何用户手动输入的新内容,优先级都高于排队中的 continuation。当 goal 被满足时,DeerFlow 会自动清除它,并发布更新后的 thread 状态。 +每次 Gateway 驱动的 run 结束后,DeerFlow 会用一个 non-thinking 的评估模型,把可见的对话内容(包括助手的工具调用和截短的工具结果)拿去和激活的 goal 比对。工具调用成功本身不代表 goal 已完成;如果助手不得不猜测缺失或含糊的信息,评估模型会报告 `needs_user_input`。评估模型必须返回一个带类型的 blocker(`missing_evidence`、`needs_user_input`、`run_failed`、`external_wait` 或 `goal_not_met_yet`),并附上可见证据。只有在最近一轮 assistant 回复已被持久化 checkpoint、blocker 是 `goal_not_met_yet`、评估期间 thread 没有变化、且无进展熔断器没有触发时,DeerFlow 才会注入一次 hidden continuation。安全上限默认是 8 次 hidden continuation;连续两次相同的无进展评估后就会停止。`/goal clear` 以及任何用户手动输入的新内容,优先级都高于排队中的 continuation。当 goal 被满足时,DeerFlow 会自动清除它,并发布更新后的 thread 状态。同一个 checkpoint 会把这次达成记为 `goal_outcome`,保留到下一次修改 goal。 -Web UI 会在输入框上方展示当前激活的 goal。同样的命令在 TUI 和受支持的 IM 渠道里也可用。在 Web UI 和受支持的 IM 渠道里,设置 `/goal <完成条件>` 还会以该条件作为任务启动一次 run;状态查询和清除命令则只管理 goal 状态本身。 +Web UI 会在输入框上方展示当前激活的 goal,说明自动续跑为什么停下,并在 goal 达成后显示「目标已达成」提示;设置了 goal 时不能编辑最后一条消息。同样的命令在 TUI 和受支持的 IM 渠道里也可用。在 Web UI 和受支持的 IM 渠道里,设置 `/goal <完成条件>` 还会以该条件作为任务启动一次 run;状态查询和清除命令则只管理 goal 状态本身。 ### 手动上下文压缩 diff --git a/backend/app/gateway/AGENTS.md b/backend/app/gateway/AGENTS.md index 28a106b2d9..51d84af46e 100644 --- a/backend/app/gateway/AGENTS.md +++ b/backend/app/gateway/AGENTS.md @@ -1,6 +1,6 @@ ### Gateway API (`app/gateway/`) -Reject external run/state writes with `sandbox`, `thread_data`, or `viewed_images`. +Reject external run/state writes with `sandbox`, `thread_data`, `viewed_images`, or `goal_outcome`. Studio retains sanitized creation metadata. diff --git a/backend/app/gateway/routers/thread_runs.py b/backend/app/gateway/routers/thread_runs.py index 665d2d0239..6b5cef1d21 100644 --- a/backend/app/gateway/routers/thread_runs.py +++ b/backend/app/gateway/routers/thread_runs.py @@ -70,6 +70,7 @@ from deerflow.agents.middlewares.dynamic_context_middleware import strip_injecte from deerflow.authz.sandbox_authz import safe_app_config_async from deerflow.config.paths import get_paths, make_safe_user_id from deerflow.runtime import CancelOutcome, ConflictError, RunRecord, RunStatus, ThreadOperationKind, serialize_channel_values_for_api +from deerflow.runtime.goal import is_active_goal from deerflow.runtime.runs.store.base import format_run_cursor_created_at, normalize_run_created_at_iso from deerflow.runtime.secret_context import redact_config_secrets, redact_metadata_secrets from deerflow.runtime.user_context import get_effective_user_id @@ -567,8 +568,7 @@ def _has_title(values: dict[str, Any]) -> bool: def _has_active_goal(snapshot: Any) -> bool: - goal = _checkpoint_values(snapshot).get("goal") - return isinstance(goal, dict) and goal.get("status") == "active" + return is_active_goal(_checkpoint_values(snapshot).get("goal")) def _latest_editable_turn(messages: list[Any], human_message_id: str) -> tuple[int, Any, int, Any, list[str]]: diff --git a/backend/app/gateway/routers/threads.py b/backend/app/gateway/routers/threads.py index 4e64eea1b0..7dbfdfecb7 100644 --- a/backend/app/gateway/routers/threads.py +++ b/backend/app/gateway/routers/threads.py @@ -65,9 +65,11 @@ from deerflow.runtime.context_keys import checkpoint_agent_binding_metadata from deerflow.runtime.events.message_seq import stamp_messages_with_seq from deerflow.runtime.goal import ( DEFAULT_MAX_GOAL_CONTINUATIONS, + GOAL_OUTCOME_CHANNEL, build_goal_state, ensure_thread_checkpoint, goal_thread_lock, + is_active_goal, read_thread_goal, write_thread_goal, ) @@ -135,6 +137,7 @@ _BRANCH_TITLE_SEQUENCE_METADATA_KEY = "branch_title_sequence" # task_history binds source batches to the parent's archive scope. Notes may # carry over, but the branch must not advertise that archive as available. _BRANCH_EXCLUDED_CHANNELS = frozenset({"sandbox", "thread_data", "task_history"}) +_BRANCH_GOAL_CHANNELS = frozenset({"goal", GOAL_OUTCOME_CHANNEL}) _BRANCH_HISTORY_SCAN_LIMIT = 200 _BRANCH_HISTORY_RAW_SCAN_LIMIT = _BRANCH_HISTORY_SCAN_LIMIT * 2 _BRANCH_TITLE_MAX_LENGTH = 256 @@ -1150,15 +1153,22 @@ async def _branch_thread_with_reservation( if branch_reducer_fields is None: branch_reducer_fields = THREAD_STATE_REDUCER_FIELDS - def branch_values(source_snapshot: Any) -> dict[str, Any]: + def branch_values(source_snapshot: Any, *, replaces: Any = None) -> dict[str, Any]: + source_values = dict(source_snapshot.values) values: dict[str, Any] = {} - for key, value in dict(source_snapshot.values).items(): + for key, value in source_values.items(): if key in _BRANCH_EXCLUDED_CHANNELS: continue if key in branch_reducer_fields: values[key] = Overwrite(list(value) if key == "messages" and isinstance(value, list) else value) else: values[key] = value + if replaces is not None: + # The head write lands on the replay base, so goal state the head + # lacks (write_thread_goal pops goal and goal_outcome) would keep + # the base's value. Clear it so the branch matches its source turn. + for key in (_BRANCH_GOAL_CHANNELS & dict(replaces.values).keys()) - source_values.keys(): + values[key] = Overwrite(None) if key in branch_reducer_fields else None if display_name is not None: values["title"] = display_name return values @@ -1184,7 +1194,7 @@ async def _branch_thread_with_reservation( head_config.setdefault("metadata", {}).update(checkpoint_metadata_updates) await branch_accessor.aupdate( head_config, - branch_values(snapshot), + branch_values(snapshot, replaces=replay_base_tuple), as_node="branch", ) except _CHECKPOINT_MODE_ERRORS as exc: @@ -1661,6 +1671,9 @@ async def update_thread_state(thread_id: ThreadId, body: ThreadStateUpdateReques if reducer_fields is None: reducer_fields = THREAD_STATE_REDUCER_FIELDS updates = {key: Overwrite(value) if key in reducer_fields else value for key, value in values.items()} + if "goal" in values and (writable_channels is None or GOAL_OUTCOME_CHANNEL in writable_channels): + # This goal writer bypasses write_thread_goal; drop the met record the same way. + updates[GOAL_OUTCOME_CHANNEL] = None try: async with reserve_checkpoint_write(request, thread_id, user_id=get_effective_user_id()): source_metadata = await accessor.aget_metadata(read_config) @@ -1824,7 +1837,9 @@ async def get_thread_history( """Get materialized graph state history for a thread. Only the latest (first) checkpoint carries the ``messages`` key to - avoid duplicating the complete conversation across every entry. + avoid duplicating the complete conversation across every entry. It also + carries an active ``goal`` and an achieved ``goal_outcome``, because + ``useStream`` rebuilds thread values from this head after every run. """ checkpointer = get_checkpointer(request) try: @@ -1856,6 +1871,11 @@ async def get_thread_history( values["thread_data"] = thread_data if is_latest_checkpoint: + if is_active_goal(goal := materialized_values.get("goal")): + values["goal"] = goal + outcome = materialized_values.get(GOAL_OUTCOME_CHANNEL) + if isinstance(outcome, dict) and outcome.get("status") == "achieved": + values[GOAL_OUTCOME_CHANNEL] = outcome messages = materialized_values.get("messages") if messages: serialized_msgs = serialize_channel_values_for_api({"messages": messages}).get("messages", []) diff --git a/backend/app/gateway/services.py b/backend/app/gateway/services.py index f88845f5a8..4fe39d5dea 100644 --- a/backend/app/gateway/services.py +++ b/backend/app/gateway/services.py @@ -484,11 +484,15 @@ def _normalize_input_messages( return converted +#: State channels only the server writes. ``goal_outcome`` is the "goal met" +#: record that clearing a satisfied goal writes; a caller copy would forge one. +SERVER_OWNED_STATE_CHANNELS = frozenset({"sandbox", "thread_data", "viewed_images", "goal_outcome"}) + + def strip_server_owned_state_metadata(values: Mapping[str, Any]) -> dict[str, Any]: """Validate and sanitize caller-supplied state values before checkpointing. - The server-owned ``sandbox``, ``thread_data``, and ``viewed_images`` channels - are rejected. The ``messages`` channel + The ``SERVER_OWNED_STATE_CHANNELS`` are rejected. The ``messages`` channel is canonicalized to a list of ``BaseMessage`` objects, rejects external system/developer roles with HTTP 400, and strips server-owned metadata. Other channels keep their existing shapes while @@ -500,8 +504,7 @@ def strip_server_owned_state_metadata(values: Mapping[str, Any]) -> dict[str, An transform trails, or privileged message roles. Every channel is walked because middleware-contributed channels can also carry message-like values. """ - server_owned_channels = {"sandbox", "thread_data", "viewed_images"} - rejected = server_owned_channels.intersection(values) + rejected = SERVER_OWNED_STATE_CHANNELS.intersection(values) if rejected: raise HTTPException( status_code=400, @@ -537,10 +540,10 @@ def normalize_input(raw_input: dict[str, Any] | None, *, trusted_internal: bool of bubbling up as a 500. The gateway is a system boundary, so per-entry validation errors are the right shape for clients to retry against. - The ``sandbox``, ``thread_data``, and ``viewed_images`` channels are also - server-owned. External callers cannot select a provider resource by id or - supply host image paths; trusted internal run admission may carry restored - values. + The ``SERVER_OWNED_STATE_CHANNELS`` are also rejected. External callers + cannot select a provider resource by id, supply host image paths, or forge a + met goal; trusted internal run admission may carry restored values. A caller + ``goal`` replaces the goal, so it also clears the previous ``goal_outcome``. ``original_user_content``, dynamic-context reminder markers, the transient view-image context marker, the execution-only knowledge-scope marker, tool @@ -565,8 +568,7 @@ def normalize_input(raw_input: dict[str, Any] | None, *, trusted_internal: bool if raw_input is None: return {} if not trusted_internal: - server_owned_channels = {"sandbox", "thread_data", "viewed_images"} - rejected = server_owned_channels.intersection(raw_input) + rejected = SERVER_OWNED_STATE_CHANNELS.intersection(raw_input) if rejected: raise HTTPException( status_code=400, @@ -583,6 +585,9 @@ def normalize_input(raw_input: dict[str, Any] | None, *, trusted_internal: bool cleaned = [_strip_external_delegation_verdict(entry) for entry in delegations] if cleaned != delegations: result = {**result, "delegations": cleaned} + # merge_goal ignores None, so only a goal value replaces the goal. + if result.get("goal") is not None: + result = {**result, "goal_outcome": None} return result diff --git a/backend/docs/API.md b/backend/docs/API.md index 4a84abbd3e..e2e9f3a2f7 100644 --- a/backend/docs/API.md +++ b/backend/docs/API.md @@ -211,6 +211,17 @@ GET /api/langgraph/threads/{thread_id}/state } ``` +#### Get Thread History + +```http +POST /api/langgraph/threads/{thread_id}/history +``` + +Only the newest entry carries `messages`, the `goal` while it is active, and +`goal_outcome` (the latest met goal, kept until the next goal change). Older +entries carry only `title` and `thread_data`. `goal_outcome` is server-owned: +state updates and run input that include it are rejected with 400. + ### Runs #### Create Run diff --git a/backend/packages/harness/deerflow/agents/goal_state.py b/backend/packages/harness/deerflow/agents/goal_state.py index 38baf6b4cc..776bc409a6 100644 --- a/backend/packages/harness/deerflow/agents/goal_state.py +++ b/backend/packages/harness/deerflow/agents/goal_state.py @@ -30,3 +30,17 @@ class GoalState(TypedDict): no_progress_count: int max_no_progress_continuations: int last_evaluation: NotRequired[dict[str, Any]] + + +class GoalOutcomeState(TypedDict): + """The latest met goal, written only by the checkpoint write that clears it.""" + + status: Literal["achieved"] + objective: str + goal_created_at: str + achieved_at: str + continuation_count: int + max_continuations: int + reason: str + relied_on_assumption: bool + reply_message_id: str | None diff --git a/backend/packages/harness/deerflow/agents/middlewares/pii_redaction_middleware.py b/backend/packages/harness/deerflow/agents/middlewares/pii_redaction_middleware.py index 92d5c6ab07..19da8caba2 100644 --- a/backend/packages/harness/deerflow/agents/middlewares/pii_redaction_middleware.py +++ b/backend/packages/harness/deerflow/agents/middlewares/pii_redaction_middleware.py @@ -34,11 +34,16 @@ Scope model (mirrors the structural guardrails): * The memory-enqueue path is covered by the follow-up slice (#5577): ``redact_queued_messages`` (memory middleware) applies the same configured policy to the extraction payloads queued for the memory backend. +* The hidden ``/goal`` continuation is a framework message this middleware + skips, so ``make_goal_continuation_message`` (``runtime/goal.py``) redacts + its objective, reason and evidence summary before the message is stored. * The compaction and durable-context seams run outside ``wrap_model_call``; :func:`redact_text` is the shared entry point they call, wired from SummarizationMiddleware (compaction input) and DurableContextMiddleware (reinjected ``summary_text``); TitleMiddleware redacts its complete user and - assistant fields before truncation and direct model invocation. + assistant fields before truncation and direct model invocation. The goal + evaluator (``runtime/goal.py``) likewise redacts whole messages before its + evidence caps, and its assembled input. Detector order is fixed and pinned by a regression test; email → api_key → national_id → credit_card → phone. Checksum-gated national IDs run *before* diff --git a/backend/packages/harness/deerflow/agents/thread_state.py b/backend/packages/harness/deerflow/agents/thread_state.py index a16bac627a..edf813d0d1 100644 --- a/backend/packages/harness/deerflow/agents/thread_state.py +++ b/backend/packages/harness/deerflow/agents/thread_state.py @@ -16,7 +16,7 @@ from langgraph.channels import DeltaChannel from langgraph.graph.message import REMOVE_ALL_MESSAGES import deerflow.checkpoint_patches as _checkpoint_patches # noqa: F401 - import-time saver fixes -from deerflow.agents.goal_state import GoalState +from deerflow.agents.goal_state import GoalOutcomeState, GoalState from deerflow.agents.task_continuity.state import TaskNotesChannel, merge_task_notes from deerflow.config.database_config import DEFAULT_CHECKPOINT_SNAPSHOT_FREQUENCY, CheckpointChannelMode from deerflow.subagents.status_contract import SUBAGENT_STATUS_VALUES @@ -366,6 +366,7 @@ class ThreadState(AgentState): artifacts: Annotated[list[str], merge_artifacts] todos: Annotated[list | None, merge_todos] goal: Annotated[GoalState | None, merge_goal] + goal_outcome: NotRequired[GoalOutcomeState | None] uploaded_files: NotRequired[list[dict] | None] viewed_images: Annotated[dict[str, ViewedImageData], merge_viewed_images] # image_path -> metadata (no base64) promoted: Annotated[PromotedTools | None, merge_promoted] diff --git a/backend/packages/harness/deerflow/runtime/AGENTS.md b/backend/packages/harness/deerflow/runtime/AGENTS.md index 91e9a23d7d..5d73e24b7b 100644 --- a/backend/packages/harness/deerflow/runtime/AGENTS.md +++ b/backend/packages/harness/deerflow/runtime/AGENTS.md @@ -70,7 +70,7 @@ from `on_llm_end` before inspecting the response or touching any run state. **Skill history:** `record_skill_usage` saves lead-run snapshots on terminal answers for paginated history. See `docs/skill-usage-ui.md`. -**Run delivery receipts:** Journal artifact evidence and terminal status must finalize before an ordinary satisfied goal is cleared. That cleanup uses a durable checkpoint-write reservation; delivery failure retains the ordinary goal without another continuation. The scheduled-only exception is described below. Details: `backend/docs/runtime-guidance-details.md`. +**Run delivery receipts:** Journal artifact evidence and terminal status must finalize before an ordinary satisfied goal is cleared. That cleanup uses a durable checkpoint-write reservation; delivery failure retains the ordinary goal without another continuation. Only that clear passes `outcome` to `write_thread_goal`, which removes `goal_outcome` on every other goal write. The scheduled-only exception is described below. Details: `backend/docs/runtime-guidance-details.md`. **Deferred terminal commit:** With an event store, the worker stages its terminal status locally and commits it only after finalization's receipt and duration diff --git a/backend/packages/harness/deerflow/runtime/goal.py b/backend/packages/harness/deerflow/runtime/goal.py index 6ecf7421e3..be6d923a88 100644 --- a/backend/packages/harness/deerflow/runtime/goal.py +++ b/backend/packages/harness/deerflow/runtime/goal.py @@ -23,9 +23,10 @@ from langchain_core.messages import HumanMessage, SystemMessage from langgraph.checkpoint.base import empty_checkpoint, uuid6 import deerflow.utils.llm_text as llm_text -from deerflow.agents.goal_state import GoalBlocker, GoalEvaluation, GoalState +from deerflow.agents.goal_state import GoalBlocker, GoalEvaluation, GoalOutcomeState, GoalState from deerflow.agents.human_input import read_human_input_response from deerflow.agents.interaction_policy import RunInteractionPolicy +from deerflow.config.pii_redaction_config import PiiRedactionConfig from deerflow.models import create_chat_model from deerflow.runtime.keyed_lock import AsyncKeyedLockTable from deerflow.tracing import inject_langfuse_metadata @@ -61,6 +62,8 @@ CONTINUABLE_GOAL_BLOCKERS: set[GoalBlocker] = {"goal_not_met_yet"} GOAL_CLEAR_ALIASES = frozenset({"clear", "reset", "off"}) +GOAL_OUTCOME_CHANNEL = "goal_outcome" + _extract_response_text = llm_text.extract_response_text _strip_markdown_code_fence = llm_text.strip_markdown_code_fence _strip_think_blocks = llm_text.strip_think_blocks @@ -123,6 +126,29 @@ def build_goal_state( ) +def is_active_goal(value: object) -> bool: + """Return true when a ``goal`` channel value is an active goal.""" + return isinstance(value, dict) and value.get("status") == "active" + + +def build_goal_outcome(goal: GoalState, evaluation: GoalEvaluation, *, reply_message_id: str | None, now: str | None = None) -> GoalOutcomeState: + """Build the record of a met goal from its satisfied completion verdict.""" + if evaluation.get("satisfied") is not True: + raise ValueError("A goal outcome requires a satisfied evaluation.") + # POST /state and run input can store an active goal without created_at. + return GoalOutcomeState( + status="achieved", + objective=str(goal.get("objective") or ""), + goal_created_at=str(goal.get("created_at") or ""), + achieved_at=now or now_iso(), + continuation_count=int(goal.get("continuation_count", 0)), + max_continuations=int(goal.get("max_continuations", DEFAULT_MAX_GOAL_CONTINUATIONS)), + reason=evaluation.get("reason", ""), + relied_on_assumption=evaluation.get("relied_on_assumption") is True, + reply_message_id=reply_message_id, + ) + + def parse_goal_evaluation_response(text: str, *, require_assumption_attribution: bool = False) -> GoalEvaluation: """Parse the evaluator's JSON object response.""" candidate = _strip_markdown_code_fence(_strip_think_blocks(text)) @@ -321,6 +347,16 @@ def _cap_evidence(lines: list[str]) -> str: return "\n\n".join([*head, *marker, *kept]) +def _evidence_window(messages: list[Any]) -> list[Any]: + """The messages the evaluator reads: from the first of the last ``MAX_GOAL_CONVERSATION_MESSAGES`` visible ones. + + ``format_visible_conversation`` and the redaction pass in ``evaluate_goal_completion`` both + take their messages from here, so a change to the window moves both. + """ + visible_positions = [index for index, message in enumerate(messages) if _is_visible_message(message)] + return messages[visible_positions[-MAX_GOAL_CONVERSATION_MESSAGES:][0] :] if visible_positions else [] + + def format_visible_conversation(messages: list[Any]) -> str: """Return the conversation evidence for goal evaluation. @@ -332,10 +368,9 @@ def format_visible_conversation(messages: list[Any]) -> str: except a clarification prompt, which the UI shows as its own card. The user's answer to such a card is a hidden message too, but the card shows it, so its value is included on its own line. """ - visible_positions = [index for index, message in enumerate(messages) if _is_visible_message(message)] - if not visible_positions: + window = _evidence_window(messages) + if not window: return "" - window = messages[visible_positions[-MAX_GOAL_CONVERSATION_MESSAGES:][0] :] # Tool-call ids can repeat across turns, so each result is paired with the latest call # before it that has its id, not with the last call of that id anywhere in the window. calls: dict[str, tuple[str | None, bool]] = {} @@ -419,7 +454,16 @@ async def evaluate_goal_completion( callbacks to lift it — same fix as PR #2944 (main graph) and PR #3902 (memory_agent/suggest_agent). """ - conversation = format_visible_conversation(messages) + # This model call bypasses PiiRedactionMiddleware. As TitleMiddleware does, whole messages are + # redacted before the evidence caps can split an identifier, and the input once more. Only the + # evidence window is redacted: the pass is synchronous work on the event loop, and the evaluator + # never sees earlier messages. Imported here because the redaction middleware reaches this module + # through deerflow.tools and the runtime. + from deerflow.agents.middlewares.memory_middleware import redact_queued_messages + from deerflow.agents.middlewares.pii_redaction_middleware import redact_text + + pii_redaction = getattr(app_config, "pii_redaction", None) + conversation = format_visible_conversation(redact_queued_messages(_evidence_window(messages), pii_redaction) if pii_redaction is not None and pii_redaction.enabled else messages) if not conversation or not has_visible_assistant_evidence(messages): return GoalEvaluation( satisfied=False, @@ -469,7 +513,7 @@ async def evaluate_goal_completion( ) prompt_messages = [ SystemMessage(content=system_instruction), - HumanMessage(content=user_content), + HumanMessage(content=redact_text(user_content, pii_redaction)), ] source_id = "goal-evaluator:" + uuid4().hex @@ -560,13 +604,26 @@ def latest_visible_assistant_signature(messages: list[Any]) -> str: continuation adds no new visible assistant output, the signature is unchanged and the breaker can recognise the stalled turn. """ + reply = _latest_visible_assistant_reply(messages) + return hashlib.sha256(reply[1].encode("utf-8")).hexdigest() if reply else "" + + +def latest_visible_assistant_message_id(messages: list[Any]) -> str | None: + """Return the id of the reply ``latest_visible_assistant_signature`` keys on.""" + reply = _latest_visible_assistant_reply(messages) + message_id = _message_field(reply[0], "id") if reply else None + return message_id if isinstance(message_id, str) and message_id else None + + +def _latest_visible_assistant_reply(messages: list[Any]) -> tuple[Any, str] | None: + """Return the latest visible AI message with non-empty text, and that text.""" for message in reversed(messages): if not _is_visible_message(message) or _message_type(message) != "ai": continue text = message_to_text(message).strip() if text: - return hashlib.sha256(text.encode("utf-8")).hexdigest() - return "" + return message, text + return None def compute_goal_progress_key(evaluation: GoalEvaluation, *, evidence_signature: str = "") -> str: @@ -598,13 +655,23 @@ def compute_no_progress_count(goal: GoalState, evaluation: GoalEvaluation, *, ev return 0 -def make_goal_continuation_message(goal: GoalState, evaluation: GoalEvaluation) -> HumanMessage: - """Build the hidden user message that asks the agent to keep working.""" +def make_goal_continuation_message(goal: GoalState, evaluation: GoalEvaluation, *, pii_redaction: PiiRedactionConfig | None = None) -> HumanMessage: + """Build the hidden user message that asks the agent to keep working. + + PiiRedactionMiddleware skips this framework message, so with + ``pii_redaction`` enabled the objective, the reason and the evidence + summary are redacted here. The thread keeps the redacted message, which + the UI hides. A redaction error propagates; the caller must not send the + raw text instead. + """ + from deerflow.agents.middlewares.pii_redaction_middleware import redact_text + + objective, reason, evidence_summary = (redact_text(text, pii_redaction) for text in (goal["objective"], evaluation["reason"], evaluation.get("evidence_summary") or "")) content = ( "\n" - f"Active goal: {goal['objective']}\n" - f"Evaluator result: not satisfied. Blocker: {evaluation['blocker']}. Reason: {evaluation['reason'] or 'No reason provided.'}\n" - f"Visible evidence: {evaluation.get('evidence_summary') or 'No evidence summary provided.'}\n" + f"Active goal: {objective}\n" + f"Evaluator result: not satisfied. Blocker: {evaluation['blocker']}. Reason: {reason or 'No reason provided.'}\n" + f"Visible evidence: {evidence_summary or 'No evidence summary provided.'}\n" "Continue working toward the active goal. Use the available tools and conversation context. " "Do not ask the user to continue unless you are genuinely blocked.\n" "" @@ -703,11 +770,16 @@ async def write_thread_goal( as_node: str = "goal", create_if_missing: bool = False, expected_checkpoint_id: str | None = None, + outcome: GoalOutcomeState | None = None, ) -> dict[str, Any]: """Write a new checkpoint with the thread goal set or cleared. + Every goal write removes the ``goal_outcome`` record in the same + checkpoint; only the write that clears a met goal passes ``outcome``. Returns the updated channel values. """ + if goal is not None and outcome is not None: + raise ValueError("A goal outcome is written only when clearing the goal.") if create_if_missing: await ensure_thread_checkpoint(checkpointer, thread_id) @@ -727,15 +799,20 @@ async def write_thread_goal( metadata: dict[str, Any] = dict(getattr(checkpoint_tuple, "metadata", {}) or {}) channel_values: dict[str, Any] = dict(checkpoint.get("channel_values", {}) or {}) - if goal is None: - channel_values.pop("goal", None) - else: - channel_values["goal"] = copy.deepcopy(goal) + writes: dict[str, Any] = {"goal": goal} + if GOAL_OUTCOME_CHANNEL in channel_values or outcome is not None: + # Threads that never had a record keep their checkpoint shape. + writes[GOAL_OUTCOME_CHANNEL] = outcome + for channel, value in writes.items(): + if value is None: + channel_values.pop(channel, None) + else: + channel_values[channel] = copy.deepcopy(value) channel_versions = dict(checkpoint.get("channel_versions", {}) or {}) - current_version = channel_versions.get("goal") - next_version = _next_channel_version(checkpointer, current_version) - channel_versions["goal"] = next_version + # The saver stores a channel's value only when its version is in new_versions. + new_versions = {channel: _next_channel_version(checkpointer, channel_versions.get(channel)) for channel in writes} + channel_versions.update(new_versions) checkpoint["channel_values"] = channel_values checkpoint["channel_versions"] = channel_versions @@ -743,7 +820,7 @@ async def write_thread_goal( metadata["updated_at"] = now_iso() metadata["source"] = "update" metadata["step"] = metadata.get("step", 0) + 1 - metadata["writes"] = {as_node: {"goal": goal}} + metadata["writes"] = {as_node: writes} write_config = { "configurable": { @@ -756,7 +833,7 @@ async def write_thread_goal( "checkpoint_id": _checkpoint_id_from_tuple(checkpoint_tuple), } } - await _call_checkpointer_method(checkpointer, "aput", "put", write_config, checkpoint, metadata, {"goal": next_version}) + await _call_checkpointer_method(checkpointer, "aput", "put", write_config, checkpoint, metadata, new_versions) return channel_values diff --git a/backend/packages/harness/deerflow/runtime/runs/worker.py b/backend/packages/harness/deerflow/runtime/runs/worker.py index 25ea918b85..abd2214a17 100644 --- a/backend/packages/harness/deerflow/runtime/runs/worker.py +++ b/backend/packages/harness/deerflow/runtime/runs/worker.py @@ -75,11 +75,13 @@ from deerflow.runtime.goal import ( _is_visible_message, _message_type, attach_goal_evaluation, + build_goal_outcome, build_goal_state, compute_no_progress_count, create_goal_evaluator_model, evaluate_goal_completion, goal_thread_lock, + latest_visible_assistant_message_id, latest_visible_assistant_signature, make_goal_continuation_message, read_thread_goal, @@ -1987,7 +1989,7 @@ async def run_agent( class _GoalCompletionCandidate: goal: GoalState conversation_signature: str - evaluation: GoalEvaluation | None = None + evaluation: GoalEvaluation def _scheduled_goal_objective(record: RunRecord, scheduled_runtime: Mapping[str, Any] | None) -> str | None: @@ -2130,12 +2132,14 @@ async def _clear_completed_goal( return # Duration bookkeeping may advance the checkpoint after evaluation. # Compare goal/conversation above, then guard against stale writes. + # The record lands in the same checkpoint as the clear, or not at all. values = await write_thread_goal( checkpointer, record.thread_id, None, as_node="goal_evaluator", expected_checkpoint_id=_checkpoint_id(checkpoint_tuple), + outcome=build_goal_outcome(candidate.goal, candidate.evaluation, reply_message_id=latest_visible_assistant_message_id(messages)), ) await bridge.publish(record.run_id, "values", serialize(values, mode="values")) except GoalWriteConflict: @@ -2489,6 +2493,23 @@ async def _prepare_goal_continuation_input( await _persist(goal, evaluation, no_progress_count, stand_down_reason=stand_down_reason) return None + # Built before the continuation is counted: the message is redacted under the run's + # pii_redaction, and a redaction error fails the check as an evaluator redaction error + # does, so the raw objective and reason are never sent and the budget is not spent. + try: + continuation_message = make_goal_continuation_message(goal, evaluation, pii_redaction=getattr(app_config, "pii_redaction", None)) + except Exception as exc: + logger.warning("Could not redact the goal continuation for thread %s after run %s", thread_id, run_id, exc_info=True) + # Only the exception type is stored: the error message could quote the text. + evaluation = GoalEvaluation( + satisfied=False, + blocker="run_failed", + reason=f"The goal continuation could not be redacted ({type(exc).__name__}).", + evidence_summary="", + ) + await _persist(goal, evaluation, compute_no_progress_count(goal, evaluation, evidence_signature=evidence_signature), stand_down_reason="evaluator_failed") + return None + next_count = int(goal.get("continuation_count", 0)) + 1 updated_goal = await _persist(goal, evaluation, no_progress_count, continuation_count=next_count) if updated_goal is None: @@ -2527,7 +2548,7 @@ async def _prepare_goal_continuation_input( updated_goal.get("continuation_count", next_count), updated_goal.get("max_continuations", 0), ) - return {"messages": [make_goal_continuation_message(updated_goal, evaluation)]} + return {"messages": [continuation_message]} def _is_edit_replay_run(record: RunRecord) -> bool: diff --git a/backend/scripts/benchmark/scheduled_tasks/budget_evidence.py b/backend/scripts/benchmark/scheduled_tasks/budget_evidence.py index db4548ac81..a38542d146 100644 --- a/backend/scripts/benchmark/scheduled_tasks/budget_evidence.py +++ b/backend/scripts/benchmark/scheduled_tasks/budget_evidence.py @@ -126,7 +126,7 @@ async def recheck_budget(args: Any) -> dict[str, Any]: "original_results_sha256": hashlib.sha256(original_path.read_bytes()).hexdigest(), "original_failure_preserved": (args.run_output / "failure.json").is_file(), "original_api_goal_history": previous["goal_history"], - "reader_correction": "Public history projects title/thread_data/messages and omits goal; native accessor reads the persisted goal channel", + "reader_correction": "Older public history entries omit goal (only the head carries an active goal); native accessor reads the persisted goal channel", } finally: await engine.dispose() diff --git a/backend/tests/test_bench_scheduled_budget_evidence.py b/backend/tests/test_bench_scheduled_budget_evidence.py index dd2e34447f..da9761070f 100644 --- a/backend/tests/test_bench_scheduled_budget_evidence.py +++ b/backend/tests/test_bench_scheduled_budget_evidence.py @@ -16,7 +16,9 @@ from scripts.benchmark.scheduled_tasks.protocol import goal_history_evidence @pytest.mark.asyncio @pytest.mark.parametrize("mode", ["full", "delta"]) -async def test_public_history_projection_omits_goal_but_native_accessor_reads_real_counters(tmp_path, monkeypatch, mode): +async def test_public_history_projection_omits_older_goals_but_native_accessor_reads_real_counters(tmp_path, monkeypatch, mode): + """Only the history head carries the goal, and only while it is active; the + benchmark therefore reads earlier counters through the native accessor.""" from scripts.benchmark.scheduled_tasks.budget_evidence import read_native_goal_evidence objective = "Observe a synthetic budget stop" @@ -29,13 +31,20 @@ async def test_public_history_projection_omits_goal_but_native_accessor_reads_re goal = build_goal_state(objective) goal["continuation_count"] = 2 await accessor.aupdate(config, {"goal": goal}, as_node="evidence") + monkeypatch.setattr(threads, "get_checkpointer", lambda _request: saver) + monkeypatch.setattr(threads, "build_thread_checkpoint_state_accessor", AsyncMock(return_value=(accessor, config))) + + async def public_history(): + projected = await call_unwrapped(threads.get_thread_history, thread_id="budget-thread", body=threads.ThreadHistoryRequest(limit=100), request=SimpleNamespace(), background_tasks=BackgroundTasks()) + return [entry.model_dump(mode="json") for entry in projected] + + active_shape = await public_history() + assert active_shape[0]["values"]["goal"]["continuation_count"] == 2 + assert all("goal" not in entry["values"] for entry in active_shape[1:]) # The production control writer removes the channel; the reducer's # ordinary None update deliberately preserves an active goal. await write_thread_goal(saver, "budget-thread", None) - monkeypatch.setattr(threads, "get_checkpointer", lambda _request: saver) - monkeypatch.setattr(threads, "build_thread_checkpoint_state_accessor", AsyncMock(return_value=(accessor, config))) - projected = await call_unwrapped(threads.get_thread_history, thread_id="budget-thread", body=threads.ThreadHistoryRequest(limit=100), request=SimpleNamespace(), background_tasks=BackgroundTasks()) - api_shape = [entry.model_dump(mode="json") for entry in projected] + api_shape = await public_history() assert all("goal" not in entry["values"] for entry in api_shape) assert goal_history_evidence(api_shape, objective)["goal_snapshots_observed"] == 0 diff --git a/backend/tests/test_gateway_services.py b/backend/tests/test_gateway_services.py index d3ac35976f..cf7d5c7217 100644 --- a/backend/tests/test_gateway_services.py +++ b/backend/tests/test_gateway_services.py @@ -225,6 +225,59 @@ def test_external_image_runtime_state_is_rejected(boundary, channel): assert error.value.status_code == 400 +@pytest.mark.parametrize("boundary", ["run", "state"]) +def test_external_goal_outcome_is_rejected(boundary): + from fastapi import HTTPException + + from app.gateway.services import normalize_input, strip_server_owned_state_metadata + + transform = normalize_input if boundary == "run" else strip_server_owned_state_metadata + with pytest.raises(HTTPException) as error: + transform({"goal_outcome": {"status": "achieved", "objective": "forged"}}) + + assert error.value.status_code == 400 + assert error.value.detail == "External goal_outcome state is not allowed" + + +@pytest.mark.parametrize("mode", ["full", "delta"]) +def test_external_run_goal_clears_the_previous_goal_outcome(mode): + """A caller goal replaces the goal, so the record of the earlier met goal must go.""" + from langgraph.checkpoint.memory import InMemorySaver + from langgraph.graph import END, START, StateGraph + + from app.gateway.services import normalize_input + from deerflow.agents.thread_state import get_thread_state_schema + from deerflow.runtime.checkpoint_state import CheckpointStateAccessor + from deerflow.runtime.goal import build_goal_outcome, build_goal_state, write_thread_goal + + checkpointer = InMemorySaver() + builder = StateGraph(get_thread_state_schema(mode)) + builder.add_node("agent", lambda state: {}) + builder.add_edge(START, "agent") + builder.add_edge("agent", END) + graph = builder.compile(checkpointer=checkpointer) + accessor = CheckpointStateAccessor.bind(graph, checkpointer, mode=mode) + config = {"configurable": {"thread_id": "run-goal-thread"}} + record = build_goal_outcome(build_goal_state("Ship it"), {"satisfied": True, "blocker": "none", "reason": "Shipped."}, reply_message_id=None) + next_goal = build_goal_state("Ship the follow-up") + + async def scenario(): + await graph.ainvoke({"messages": [{"role": "user", "content": "Ship it"}]}, config) + await write_thread_goal(checkpointer, "run-goal-thread", None, outcome=record) + before = (await accessor.aget(config)).values + await graph.ainvoke(normalize_input({"messages": [{"role": "user", "content": "Now the follow-up"}], "goal": next_goal}), config) + return before, (await accessor.aget(config)).values + + before, after = asyncio.run(scenario()) + + assert before["goal_outcome"] == record + assert after["goal"] == next_goal + assert after["goal_outcome"] is None + # merge_goal keeps the goal on None; trusted internal input is not rewritten. + assert "goal_outcome" not in normalize_input({"goal": None}) + assert "goal_outcome" not in normalize_input({"goal": next_goal}, trusted_internal=True) + + def test_trusted_internal_run_preserves_image_runtime_state(): from app.gateway.services import normalize_input diff --git a/backend/tests/test_goal_continuation_pii.py b/backend/tests/test_goal_continuation_pii.py new file mode 100644 index 0000000000..97c22105b8 --- /dev/null +++ b/backend/tests/test_goal_continuation_pii.py @@ -0,0 +1,139 @@ +"""With pii_redaction enabled, the hidden goal continuation reaches the agent redacted.""" + +import json +from types import SimpleNamespace +from unittest.mock import AsyncMock + +import pytest +from langchain_core.messages import AIMessage, HumanMessage +from langgraph.checkpoint.base import empty_checkpoint +from langgraph.checkpoint.memory import InMemorySaver + +from deerflow.agents.middlewares import pii_redaction_middleware +from deerflow.agents.middlewares.pii_redaction_middleware import redact_text +from deerflow.config.app_config import AppConfig +from deerflow.config.pii_redaction_config import PiiRedactionConfig +from deerflow.config.sandbox_config import SandboxConfig +from deerflow.runtime import goal +from deerflow.runtime.checkpoint_state import CheckpointStateAccessor, build_state_mutation_graph +from deerflow.runtime.runs import worker + +_PII = PiiRedactionConfig(enabled=True, token_secret="unit-test-deployment-secret-0123456789") +_ENABLED = AppConfig(sandbox=SandboxConfig(use="test"), pii_redaction=_PII) +_DISABLED = AppConfig(sandbox=SandboxConfig(use="test"), pii_redaction=PiiRedactionConfig(enabled=False)) +_OBJECTIVE = "Draft a reply to carol@example.com and text 13800138000 when it is sent" +_REASON = "The reply to carol@example.com is drafted, not sent; 13800138000 has not been texted." +_SUMMARY = "outputs/reply.md is addressed to carol@example.com." +_VERDICT = {"satisfied": False, "blocker": "goal_not_met_yet", "reason": _REASON, "evidence_summary": _SUMMARY, "relied_on_assumption": False} + + +def _evaluation(reason=_REASON, evidence_summary=_SUMMARY): + return goal.GoalEvaluation(satisfied=False, blocker="goal_not_met_yet", reason=reason, evidence_summary=evidence_summary) + + +def _plain_content(objective, reason, evidence_summary): + return ( + "\n" + f"Active goal: {objective}\n" + f"Evaluator result: not satisfied. Blocker: goal_not_met_yet. Reason: {reason or 'No reason provided.'}\n" + f"Visible evidence: {evidence_summary or 'No evidence summary provided.'}\n" + "Continue working toward the active goal. Use the available tools and conversation context. Do not ask the user to continue unless you are genuinely blocked.\n" + "" + ) + + +def test_continuation_redacts_the_objective_reason_and_evidence(): + message = goal.make_goal_continuation_message(goal.build_goal_state(_OBJECTIVE), _evaluation(), pii_redaction=_PII) + + # Only the identifiers change, each to the token the user's own message gets for it. + expected = _plain_content(_OBJECTIVE, _REASON, _SUMMARY) + for value in ("carol@example.com", "13800138000"): + expected = expected.replace(value, redact_text(value, _PII)) + assert message.content == expected + assert message.additional_kwargs == {"hide_from_ui": True, "deerflow_goal_continuation": True} + + +@pytest.mark.parametrize("pii_redaction", [None, PiiRedactionConfig(enabled=False)], ids=["no config", "disabled"]) +def test_continuation_is_unchanged_when_redaction_is_off(pii_redaction): + message = goal.make_goal_continuation_message(goal.build_goal_state(_OBJECTIVE), _evaluation(), pii_redaction=pii_redaction) + + assert message.content == _plain_content(_OBJECTIVE, _REASON, _SUMMARY) + # Enabled, a continuation without identifiers is unchanged too. + clean = goal.make_goal_continuation_message(goal.build_goal_state("Write outputs/notes.md"), _evaluation(reason="Not written yet.", evidence_summary=""), pii_redaction=_PII) + assert clean.content == _plain_content("Write outputs/notes.md", "Not written yet.", "") + + +class _Bridge: + async def publish(self, _run_id, _event, _payload): + pass + + +async def _continue_in_worker(app_config): + checkpointer = InMemorySaver() + thread_id = "continuation-pii-thread" + checkpoint = empty_checkpoint() + checkpoint["channel_values"] = {"messages": [HumanMessage(content="Draft a reply to carol@example.com."), AIMessage(content="I drafted outputs/reply.md.")]} + checkpoint["channel_versions"] = {"messages": 1} + checkpointer.put({"configurable": {"thread_id": thread_id, "checkpoint_ns": ""}}, checkpoint, {"step": 1}, {"messages": 1}) + await goal.write_thread_goal(checkpointer, thread_id, goal.build_goal_state(_OBJECTIVE)) + model = SimpleNamespace(ainvoke=AsyncMock(return_value=AIMessage(content=json.dumps(_VERDICT)))) + + continuation = await worker._prepare_goal_continuation_input( + accessor=CheckpointStateAccessor.bind(build_state_mutation_graph("goal_evaluator", "full"), checkpointer, mode="full"), + bridge=_Bridge(), + checkpointer=checkpointer, + thread_id=thread_id, + run_id="run-1", + model_name="test-model", + app_config=app_config, + evaluator_model_factory=lambda: model, + ) + return continuation, await goal.read_thread_goal(checkpointer, thread_id), model + + +@pytest.mark.asyncio +async def test_worker_continuation_is_redacted_under_the_run_config(): + continuation, latest_goal, _ = await _continue_in_worker(_ENABLED) + + [message] = continuation["messages"] + assert message.content == goal.make_goal_continuation_message(goal.build_goal_state(_OBJECTIVE), _evaluation(), pii_redaction=_PII).content + assert "carol@example.com" not in message.content + assert "13800138000" not in message.content + # Goal state keeps the raw objective, and the continuation is counted. + assert latest_goal["objective"] == _OBJECTIVE + assert latest_goal["continuation_count"] == 1 + + +@pytest.mark.asyncio +@pytest.mark.parametrize("app_config", [None, _DISABLED], ids=["no config", "disabled"]) +async def test_worker_continuation_is_unchanged_when_redaction_is_off(app_config): + continuation, _, _ = await _continue_in_worker(app_config) + + [message] = continuation["messages"] + assert message.content == _plain_content(_OBJECTIVE, _REASON, _SUMMARY) + + +@pytest.mark.asyncio +async def test_redaction_error_fails_the_check_instead_of_sending_raw_text(monkeypatch): + redact = pii_redaction_middleware.redact_text + + def fail_on_the_reason(text, config): + # Only the continuation redacts the evaluator's reason alone; the evaluator's own input redacts as usual. + if isinstance(text, str) and _REASON in text: + raise RuntimeError(f"cannot redact {text}") + return redact(text, config) + + monkeypatch.setattr(pii_redaction_middleware, "redact_text", fail_on_the_reason) + + continuation, latest_goal, model = await _continue_in_worker(_ENABLED) + + model.ainvoke.assert_awaited_once() + assert continuation is None + # As when the evaluator's input cannot be redacted: the goal stays active, and the check failed. + assert latest_goal["status"] == "active" + assert latest_goal["last_evaluation"]["stand_down_reason"] == "evaluator_failed" + assert latest_goal["last_evaluation"]["blocker"] == "run_failed" + assert latest_goal["last_evaluation"]["reason"] == "The goal continuation could not be redacted (RuntimeError)." + # No continuation was sent, so none is counted, and the error's text is not stored. + assert latest_goal["continuation_count"] == 0 + assert "carol" not in json.dumps(latest_goal["last_evaluation"]) diff --git a/backend/tests/test_goal_delivery_consistency.py b/backend/tests/test_goal_delivery_consistency.py index 5b6cc87997..3815eefccf 100644 --- a/backend/tests/test_goal_delivery_consistency.py +++ b/backend/tests/test_goal_delivery_consistency.py @@ -44,6 +44,7 @@ async def _run_goal_delivery( checkpoint_mode="full", after_terminal_completion=None, scheduled=False, + goal_without_created_at=False, ): """Keep graphs, checkpoints, output scanning and delivery verification real. @@ -144,7 +145,7 @@ async def _run_goal_delivery( updates["artifacts"] = command.update.get("artifacts", []) if stop_reason is not None: context["stop_reason"] = stop_reason - updates["messages"] = [*messages, AIMessage(content="The report is complete and ready for delivery.")] + updates["messages"] = [*messages, AIMessage(id=f"reply-{turn}", content="The report is complete and ready for delivery.")] return updates graph = StateGraph(get_thread_state_schema(checkpoint_mode)) @@ -154,6 +155,9 @@ async def _run_goal_delivery( compiled = graph.compile(checkpointer=checkpointer) accessor = CheckpointStateAccessor.bind(compiled, checkpointer, mode=checkpoint_mode) goal = build_goal_state("Create and present report.md.", max_continuations=2) + if goal_without_created_at: + # POST /state and run input accept an active goal build_goal_state did not write. + del goal["created_at"] if not scheduled: await accessor.aupdate( {"configurable": {"thread_id": thread_id}}, @@ -184,6 +188,7 @@ async def _run_goal_delivery( durable_run=await run_store.get(record.run_id), original_goal=goal, goal=await read_thread_goal(checkpointer, thread_id), + goal_outcome=(await accessor.aget({"configurable": {"thread_id": thread_id}})).values.get("goal_outcome"), model_calls=model_calls, turns=turns, delivery=[event["content"] for event in events if event["event_type"] == "run.delivery"], @@ -209,6 +214,7 @@ async def test_delivery_failure_preserves_satisfied_goal_without_continuing(tmp_ assert result.goal["objective"] == result.original_goal["objective"] assert result.goal["continuation_count"] == 0 assert all(call.args[2].get("goal") is not None for call in result.bridge.publish.call_args_list if call.args[1] == "values" and "goal" in call.args[2]) + assert result.goal_outcome is None @pytest.mark.asyncio @@ -221,6 +227,22 @@ async def test_successful_delivery_clears_satisfied_goal(tmp_path, monkeypatch, assert result.record.status == RunStatus.success assert result.durable_run["status"] == "success" assert result.goal is None + outcome = dict(result.goal_outcome) + assert outcome.pop("achieved_at") > result.original_goal["created_at"] + assert outcome == { + "status": "achieved", + "objective": result.original_goal["objective"], + "goal_created_at": result.original_goal["created_at"], + "continuation_count": 0, + "max_continuations": 2, + "reason": "Fixed evaluator response for the worker regression.", + "relied_on_assumption": False, + "reply_message_id": "reply-1", + } + # The clear's own values frame carries the record before the stream ends. + last_values = [call.args[2] for call in result.bridge.publish.call_args_list if call.args[1] == "values"][-1] + assert last_values["goal_outcome"] == result.goal_outcome + assert "goal" not in last_values assert len(result.goals_at_receipt) == 1 assert result.goals_at_receipt[0] is not None assert result.goals_at_receipt[0]["objective"] == result.original_goal["objective"] @@ -244,6 +266,7 @@ async def test_goal_completion_waits_for_durable_delivery_receipt(tmp_path, monk assert result.delivery == [] assert result.goal is not None assert result.goal["objective"] == result.original_goal["objective"] + assert result.goal_outcome is None @pytest.mark.asyncio @@ -257,6 +280,22 @@ async def test_unmet_goal_can_still_continue_and_deliver(tmp_path, monkeypatch): assert result.delivery[0]["satisfied"] is True assert result.goal is None assert result.goals_at_receipt[0]["continuation_count"] == 1 + assert result.goal_outcome["continuation_count"] == 1 + assert result.goal_outcome["reply_message_id"] == "reply-2" + # The continuation moved updated_at; the record still names the goal instance. + assert result.goals_at_receipt[0]["updated_at"] != result.original_goal["created_at"] + assert result.goal_outcome["goal_created_at"] == result.original_goal["created_at"] + + +@pytest.mark.asyncio +async def test_satisfied_goal_without_created_at_is_still_cleared_and_recorded(tmp_path, monkeypatch): + result = await _run_goal_delivery(tmp_path, monkeypatch, present_on_turn=1, goal_without_created_at=True) + + assert result.record.status == RunStatus.success + assert result.goal is None + assert result.goal_outcome["objective"] == result.original_goal["objective"] + assert result.goal_outcome["goal_created_at"] == "" + assert result.goal_outcome["reply_message_id"] == "reply-1" @pytest.mark.asyncio @@ -271,6 +310,7 @@ async def test_cancellation_during_evaluation_preserves_goal(tmp_path, monkeypat assert result.record.status == RunStatus.interrupted assert result.durable_run["status"] == "interrupted" assert result.goal == result.original_goal + assert result.goal_outcome is None @pytest.mark.asyncio @@ -283,6 +323,7 @@ async def test_scheduled_goal_cleanup_and_atomic_verdict_follow_delivery_finaliz assert result.durable_run["goal_verdict"]["satisfied"] is True assert result.durable_run["goal_verdict"]["relied_on_assumption"] is False assert result.goals_at_receipt[0]["objective"] == "Create and present report.md." + assert result.goal_outcome is None @pytest.mark.asyncio @@ -298,6 +339,7 @@ async def test_new_goal_during_delivery_receipt_is_not_cleared(tmp_path, monkeyp assert result.turns == [1] assert result.record.status == RunStatus.success assert result.goal == replacement + assert result.goal_outcome is None @pytest.mark.asyncio @@ -311,6 +353,7 @@ async def test_ownership_loss_during_delivery_receipt_preserves_goal(tmp_path, m assert result.turns == [1] assert result.record.ownership_lost is True assert result.goal == result.original_goal + assert result.goal_outcome is None @pytest.mark.asyncio @@ -325,6 +368,7 @@ async def test_durable_cancellation_during_delivery_receipt_preserves_goal(tmp_p assert result.record.status == RunStatus.interrupted assert result.durable_run["status"] == "interrupted" assert result.goal == result.original_goal + assert result.goal_outcome is None @pytest.mark.asyncio @@ -349,6 +393,7 @@ async def test_changed_goal_evaluation_during_delivery_receipt_is_preserved(tmp_ assert result.record.status == RunStatus.success assert len(changed_goals) == 1 assert result.goal == changed_goals[0] + assert result.goal_outcome is None @pytest.mark.asyncio @@ -368,6 +413,7 @@ async def test_new_user_message_during_delivery_receipt_preserves_goal(tmp_path, assert result.turns == [1] assert result.record.status == RunStatus.success assert result.goal == result.original_goal + assert result.goal_outcome is None @pytest.mark.asyncio @@ -381,6 +427,7 @@ async def test_user_clear_during_delivery_receipt_is_not_restored(tmp_path, monk assert result.turns == [1] assert result.record.status == RunStatus.error assert result.goal is None + assert result.goal_outcome is None @pytest.mark.asyncio @@ -394,6 +441,7 @@ async def test_unconfirmed_terminal_status_preserves_goal(tmp_path, monkeypatch, assert result.failed_terminal_writes assert result.durable_run["status"] == "running" assert result.goal == result.original_goal + assert result.goal_outcome is None @pytest.mark.asyncio @@ -406,6 +454,7 @@ async def test_recovered_terminal_status_can_clear_satisfied_goal(tmp_path, monk assert result.delivery[0]["satisfied"] is True assert result.durable_run["status"] == "success" assert result.goal is None + assert result.goal_outcome["status"] == "achieved" @pytest.mark.asyncio @@ -418,9 +467,11 @@ async def test_delta_checkpoint_goal_follows_delivery_outcome(tmp_path, monkeypa if present_on_turn is None: assert result.record.status == RunStatus.error assert result.goal == result.original_goal + assert result.goal_outcome is None else: assert result.record.status == RunStatus.success assert result.goal is None + assert result.goal_outcome["reply_message_id"] == "reply-1" @pytest.mark.asyncio @@ -437,6 +488,7 @@ async def test_peer_admitted_before_goal_clear_preserves_goal(tmp_path, monkeypa assert result.record.status == RunStatus.success assert result.durable_run["status"] == "success" assert result.goal == result.original_goal + assert result.goal_outcome is None @pytest.mark.asyncio diff --git a/backend/tests/test_goal_pii.py b/backend/tests/test_goal_pii.py new file mode 100644 index 0000000000..57f1b6b1bb --- /dev/null +++ b/backend/tests/test_goal_pii.py @@ -0,0 +1,198 @@ +"""With pii_redaction enabled, the goal evaluator's direct model call gets redacted input.""" + +import json +import re +from types import SimpleNamespace +from unittest.mock import AsyncMock + +import pytest +from langchain_core.messages import AIMessage, HumanMessage, ToolMessage +from langgraph.checkpoint.base import empty_checkpoint +from langgraph.checkpoint.memory import InMemorySaver + +from deerflow.agents.middlewares import memory_middleware, pii_redaction_middleware +from deerflow.agents.middlewares.pii_redaction_middleware import redact_text +from deerflow.config.app_config import AppConfig +from deerflow.config.pii_redaction_config import PiiRedactionConfig +from deerflow.config.sandbox_config import SandboxConfig +from deerflow.runtime import goal +from deerflow.runtime.checkpoint_state import CheckpointStateAccessor, build_state_mutation_graph +from deerflow.runtime.runs import worker + +_PII = PiiRedactionConfig(enabled=True, token_secret="unit-test-deployment-secret-0123456789") +_ENABLED = AppConfig(sandbox=SandboxConfig(use="test"), pii_redaction=_PII) +_DISABLED = AppConfig(sandbox=SandboxConfig(use="test"), pii_redaction=PiiRedactionConfig(enabled=False)) +_OBJECTIVE = "Collect the contacts into outputs/contacts.md and send it to bob@example.com" + + +def _evaluator(): + verdict = {"satisfied": False, "blocker": "goal_not_met_yet", "reason": "Not sent yet.", "evidence_summary": "", "relied_on_assumption": False} + return SimpleNamespace(ainvoke=AsyncMock(return_value=AIMessage(content=json.dumps(verdict)))) + + +def _cut_identifiers(cuts, length=5000): + # An email at each cut, so a value shortened there ends mid-address ("alice600@exa"). + text = "" + for cut in cuts: + start = cut - len(f"alice{cut}@exa") + text += "x" * (start - len(text) - 1) + f" alice{cut}@example.com " + return text + "x" * (length - len(text)) + + +def _contacts_run(): + # A tool argument, a tool result and a card answer, each with an address where its evidence cap cuts it. + answer = {"version": 1, "kind": "human_input_response", "source": "ask_clarification", "request_id": "req-1", "response_kind": "text", "value": _cut_identifiers([goal.MAX_GOAL_REQUEST_CHARS], length=2500)} + return [ + HumanMessage(content="Collect the contacts into outputs/contacts.md."), + AIMessage(content="", tool_calls=[{"name": "write_file", "args": {"path": "/mnt/user-data/outputs/contacts.md", "content": _cut_identifiers([goal.MAX_GOAL_TOOL_VALUE_CHARS])}, "id": "call-write"}]), + ToolMessage(content=_cut_identifiers([goal.MAX_GOAL_TOOL_STEP_CHARS]), tool_call_id="call-write", name="write_file"), + HumanMessage(content="My answer is in the card.", additional_kwargs={"hide_from_ui": True, "human_input_response": answer}), + AIMessage(content="outputs/contacts.md lists the contacts."), + ] + + +async def _evaluator_input(messages, app_config, objective=_OBJECTIVE): + model = _evaluator() + await goal.evaluate_goal_completion(goal.build_goal_state(objective), messages, model=model, app_config=app_config) + system_message, human_message = model.ainvoke.await_args.args[0] + return system_message.content, human_message.content + + +@pytest.mark.asyncio +async def test_identifiers_are_redacted_before_the_evidence_cuts_them(): + messages = _contacts_run() + before = [message.model_dump() for message in messages] + + _, raw = await _evaluator_input(messages, _DISABLED) + _, redacted = await _evaluator_input(messages, _ENABLED) + + # Off, the argument, the result and the card answer are each cut inside an address, + # where a pass over the assembled input no longer finds it. + assert len(re.findall(r"alice\d+@exa(?!mple)", raw)) == 3 + assert "alice" not in redacted + assert "@exa" not in redacted + # Each value shows a placeholder where its first address was. + for prefix in ('Assistant tool call: write_file {"path": "/mnt/user-data/outputs/contacts.md", "content": "', 'Tool result (write_file): "', goal.GOAL_CARD_ANSWER_PREFIX): + assert re.search(re.escape(prefix) + r"x+ \[EMAIL_[a-z]", redacted) + # Thread state keeps the raw text for display. + assert [message.model_dump() for message in messages] == before + + +@pytest.mark.asyncio +async def test_goal_objective_is_redacted(): + _, raw = await _evaluator_input(_contacts_run(), _DISABLED) + _, redacted = await _evaluator_input(_contacts_run(), _ENABLED) + + assert "send it to bob@example.com" in raw + assert "bob@example.com" not in redacted + assert f"Active goal:\nCollect the contacts into outputs/contacts.md and send it to {redact_text('bob@example.com', _PII)}\n\n" in redacted + + +@pytest.mark.asyncio +@pytest.mark.parametrize("app_config", [None, _DISABLED], ids=["no config", "disabled"]) +async def test_evaluator_input_is_unchanged_when_redaction_is_off(app_config): + messages = _contacts_run() + system, human = await _evaluator_input(messages, app_config) + enabled_system, _ = await _evaluator_input(messages, _ENABLED) + + assert human == f"Active goal:\n{_OBJECTIVE}\n\nVisible conversation evidence:\n{goal.format_visible_conversation(messages)}\n\nIs the active goal fully satisfied?" + assert system == enabled_system + # Enabled, input without identifiers is unchanged too. + clean = [ + HumanMessage(content="Summarize notes.md."), + AIMessage(content="", tool_calls=[{"name": "read_file", "args": {"path": "notes.md"}, "id": "call-read"}]), + ToolMessage(content="y" * 5000, tool_call_id="call-read", name="read_file"), + AIMessage(content="Done."), + ] + assert await _evaluator_input(clean, _ENABLED, "Summarize notes.md") == await _evaluator_input(clean, app_config, "Summarize notes.md") + + +@pytest.mark.asyncio +async def test_only_the_evidence_window_is_redacted(monkeypatch): + # The first exchange is outside the evaluator's window, so redacting it would only hold the event loop. + early = [HumanMessage(content="Write to dave@example.com first."), AIMessage(content="Noted, dave@example.com.")] + # Tool-only replies and tool results fill the window, so a windowing that counted them + # differently from the evidence would start elsewhere. + steps = goal.MAX_GOAL_CONVERSATION_MESSAGES // 2 - 1 + later = [ + *( + message + for index in range(steps) + for message in ( + HumanMessage(content=f"Step {index} for erin@example.com."), + AIMessage(content="", tool_calls=[{"name": "write_file", "args": {"path": f"step{index}.md", "content": "erin@example.com"}, "id": f"call-{index}"}]), + ToolMessage(content=f"Wrote step {index} for erin@example.com.", tool_call_id=f"call-{index}", name="write_file"), + ) + ), + HumanMessage(content="Wrap up."), + AIMessage(content=f"All {steps} steps done."), + ] + messages = [*early, *later] + redact_queued_messages = memory_middleware.redact_queued_messages + redacted_batches = [] + + def spy(batch, config): + redacted_batches.append(list(batch)) + return redact_queued_messages(batch, config) + + monkeypatch.setattr(memory_middleware, "redact_queued_messages", spy) + _, redacted = await _evaluator_input(messages, _ENABLED) + + assert redacted_batches == [later] + # The evidence is the same as when the whole thread is redacted. + assert f"Visible conversation evidence:\n{goal.format_visible_conversation(redact_queued_messages(messages, _PII))}\n\n" in redacted + assert "erin@example.com" not in redacted + + +class _Bridge: + async def publish(self, _run_id, _event, _payload): + pass + + +async def _evaluate_in_worker(model): + checkpointer = InMemorySaver() + thread_id = "pii-thread" + checkpoint = empty_checkpoint() + checkpoint["channel_values"] = {"messages": [HumanMessage(content="Draft a reply to carol@example.com."), AIMessage(content="Drafted the reply to carol@example.com.")]} + checkpoint["channel_versions"] = {"messages": 1} + checkpointer.put({"configurable": {"thread_id": thread_id, "checkpoint_ns": ""}}, checkpoint, {"step": 1}, {"messages": 1}) + await goal.write_thread_goal(checkpointer, thread_id, goal.build_goal_state("Draft a reply to carol@example.com")) + + await worker._prepare_goal_continuation_input( + accessor=CheckpointStateAccessor.bind(build_state_mutation_graph("goal_evaluator", "full"), checkpointer, mode="full"), + bridge=_Bridge(), + checkpointer=checkpointer, + thread_id=thread_id, + run_id="run-1", + model_name="test-model", + app_config=_ENABLED, + evaluator_model_factory=lambda: model, + ) + return await goal.read_thread_goal(checkpointer, thread_id) + + +@pytest.mark.asyncio +async def test_worker_evaluation_redacts_under_the_run_config(): + model = _evaluator() + + await _evaluate_in_worker(model) + + _, human_message = model.ainvoke.await_args.args[0] + assert "carol@example.com" not in human_message.content + assert human_message.content.count(redact_text("carol@example.com", _PII)) == 3 + + +@pytest.mark.asyncio +@pytest.mark.parametrize("redactor", [(memory_middleware, "redact_queued_messages"), (pii_redaction_middleware, "redact_text")], ids=["messages", "input"]) +async def test_redaction_error_fails_the_check_instead_of_sending_raw_text(monkeypatch, redactor): + def fail(*_args, **_kwargs): + raise RuntimeError("redactor unavailable") + + monkeypatch.setattr(*redactor, fail) + model = _evaluator() + + latest_goal = await _evaluate_in_worker(model) + + model.ainvoke.assert_not_awaited() + assert latest_goal["status"] == "active" + assert latest_goal["last_evaluation"]["stand_down_reason"] == "evaluator_failed" diff --git a/backend/tests/test_goal_runtime.py b/backend/tests/test_goal_runtime.py index 4ea99a536d..6ea5bfd4d4 100644 --- a/backend/tests/test_goal_runtime.py +++ b/backend/tests/test_goal_runtime.py @@ -4,8 +4,10 @@ from unittest.mock import AsyncMock, MagicMock import pytest from langchain_core.messages import AIMessage, HumanMessage, SystemMessage, ToolMessage +from langgraph.checkpoint.memory import InMemorySaver from deerflow.runtime import goal +from deerflow.runtime.checkpoint_state import CheckpointStateAccessor, build_state_mutation_graph def test_build_goal_state_defaults_to_claude_stop_hook_cap(): @@ -647,6 +649,103 @@ def test_latest_visible_assistant_signature_tracks_last_ai_evidence(): assert goal.latest_visible_assistant_signature([HumanMessage(content="only human"), hidden]) == "" +def test_latest_visible_assistant_message_id_skips_replies_without_text(): + """The goal_outcome anchor is the reply the visible signature keys on.""" + messages = [ + HumanMessage(id="h1", content="Write the report"), + AIMessage(id="a1", content="The report is ready.", tool_calls=[{"id": "call-1", "name": "present_files", "args": {}}]), + ToolMessage(id="t1", content="presented", tool_call_id="call-1"), + AIMessage(id="a2", content="", tool_calls=[{"id": "call-2", "name": "ls", "args": {}}]), + ToolMessage(id="t2", content="report.md", tool_call_id="call-2"), + AIMessage(id="a3", content=""), + AIMessage(id="a4", content="hidden", additional_kwargs={"hide_from_ui": True}), + ] + + assert goal.latest_visible_assistant_message_id(messages) == "a1" + assert goal.latest_visible_assistant_signature(messages) == goal.latest_visible_assistant_signature(messages[:2]) + assert goal.latest_visible_assistant_message_id(messages[:1]) is None + assert goal.latest_visible_assistant_message_id([AIMessage(id="", content="Done.")]) is None + + +def test_build_goal_outcome_records_the_met_goal_instance(): + met_goal = {**goal.build_goal_state("Ship it", max_continuations=3, now="2026-10-07T00:00:00Z"), "continuation_count": 2} + evaluation = goal.GoalEvaluation(satisfied=True, blocker="none", reason="Shipped.", evidence_summary="Release notes posted.", relied_on_assumption=True) + + assert goal.build_goal_outcome(met_goal, evaluation, reply_message_id="a9", now="2026-10-07T01:00:00Z") == { + "status": "achieved", + "objective": "Ship it", + "goal_created_at": "2026-10-07T00:00:00Z", + "achieved_at": "2026-10-07T01:00:00Z", + "continuation_count": 2, + "max_continuations": 3, + "reason": "Shipped.", + "relied_on_assumption": True, + "reply_message_id": "a9", + } + # Only a real True counts as relying on an assumption. + assert goal.build_goal_outcome(met_goal, {**evaluation, "relied_on_assumption": "false"}, reply_message_id=None)["relied_on_assumption"] is False + with pytest.raises(ValueError): + goal.build_goal_outcome(met_goal, {**evaluation, "satisfied": False}, reply_message_id=None) + + +@pytest.mark.parametrize("mode", ["full", "delta"]) +def test_write_thread_goal_keeps_goal_outcome_only_on_the_clearing_write(mode): + """Only the clear of a met goal writes the record; every other goal write drops it.""" + checkpointer = InMemorySaver() + accessor = CheckpointStateAccessor.bind(build_state_mutation_graph("seed", mode), checkpointer, mode=mode) + config = {"configurable": {"thread_id": "outcome-thread", "checkpoint_ns": ""}} + met_goal = goal.build_goal_state("Ship it") + record = goal.build_goal_outcome(met_goal, goal.GoalEvaluation(satisfied=True, blocker="none", reason="Shipped."), reply_message_id="a1") + + async def materialized(): + return (await accessor.aget(config)).values + + async def scenario(): + await accessor.aupdate(config, {"messages": [HumanMessage(id="h1", content="Ship it"), AIMessage(id="a1", content="Shipped.")]}, as_node="seed") + await goal.write_thread_goal(checkpointer, "outcome-thread", met_goal) + published = await goal.write_thread_goal(checkpointer, "outcome-thread", None, as_node="goal_evaluator", outcome=record) + cleared = await checkpointer.aget_tuple(config) + after_clear = await materialized() + await goal.write_thread_goal(checkpointer, "outcome-thread", goal.build_goal_state("Next goal")) + after_set = await materialized() + await goal.write_thread_goal(checkpointer, "outcome-thread", None, outcome=record) + await goal.write_thread_goal(checkpointer, "outcome-thread", None) + return published, cleared, after_clear, after_set, await materialized() + + published, cleared, after_clear, after_set, after_plain_clear = asyncio.run(scenario()) + + assert published["goal_outcome"] == record + assert "goal" not in published + assert cleared.metadata["writes"] == {"goal_evaluator": {"goal": None, "goal_outcome": record}} + assert after_clear["goal_outcome"] == record + assert "goal" not in after_clear + assert [message.id for message in after_clear["messages"]] == ["h1", "a1"] + assert after_set["goal"]["objective"] == "Next goal" + assert "goal_outcome" not in after_set + assert "goal_outcome" not in after_plain_clear + assert "goal" not in after_plain_clear + # The returned values are a snapshot, not the caller's record. + record["reason"] = "Edited later." + assert published["goal_outcome"]["reason"] == "Shipped." + + +def test_write_thread_goal_leaves_threads_without_a_record_unchanged(): + checkpointer = InMemorySaver() + active = goal.build_goal_state("Ship it") + record = goal.build_goal_outcome(active, goal.GoalEvaluation(satisfied=True, blocker="none", reason="Shipped."), reply_message_id=None) + + async def scenario(): + await goal.write_thread_goal(checkpointer, "plain-thread", active, create_if_missing=True) + with pytest.raises(ValueError): + await goal.write_thread_goal(checkpointer, "plain-thread", active, outcome=record) + return await checkpointer.aget_tuple({"configurable": {"thread_id": "plain-thread", "checkpoint_ns": ""}}) + + written = asyncio.run(scenario()) + + assert "goal_outcome" not in written.checkpoint["channel_versions"] + assert written.metadata["writes"] == {"goal": {"goal": active}} + + def test_no_progress_count_keys_on_evidence_not_volatile_free_text(): """The breaker must survive the evaluator rewording its reason. diff --git a/backend/tests/test_goal_sync_checkpointer_cancellation.py b/backend/tests/test_goal_sync_checkpointer_cancellation.py index e30bb95405..dd54c21902 100644 --- a/backend/tests/test_goal_sync_checkpointer_cancellation.py +++ b/backend/tests/test_goal_sync_checkpointer_cancellation.py @@ -6,8 +6,10 @@ import threading from types import SimpleNamespace import pytest +from _thread_checkpoint_helpers import make_saver -from deerflow.runtime.goal import build_goal_state, read_thread_goal, write_thread_goal +from deerflow.agents.goal_state import GoalEvaluation +from deerflow.runtime.goal import build_goal_outcome, build_goal_state, read_thread_goal, write_thread_goal class _BlockingSyncCheckpointer: @@ -118,3 +120,43 @@ async def test_goal_read_keeps_not_implemented_without_sync_method() -> None: with pytest.raises(NotImplementedError): await read_thread_goal(_AsyncOnlyUnsupportedCheckpointer(), "thread-1") + + +@pytest.mark.asyncio +@pytest.mark.parametrize("saver_kind", ["sqlite", "cached-sqlite", "postgres"]) +async def test_goal_outcome_round_trips_through_the_sync_fallback(saver_kind) -> None: + """On a sync-only saver the clear writes ``goal_outcome`` and the next goal write drops it.""" + config = {"configurable": {"thread_id": "outcome-thread", "checkpoint_ns": ""}} + met_goal = build_goal_state("Ship it") + record = build_goal_outcome(met_goal, GoalEvaluation(satisfied=True, blocker="none", reason="Shipped."), reply_message_id="a1") + + with make_saver(saver_kind) as saver: + with pytest.raises(NotImplementedError): + await saver.aget_tuple(config) + sync_put = saver.put + put_versions: list[set[str]] = [] + put_writes: list[object] = [] + + def recording_put(write_config, checkpoint, metadata, new_versions): + put_versions.append(set(new_versions)) + put_writes.append(metadata.get("writes")) + return sync_put(write_config, checkpoint, metadata, new_versions) + + saver.put = recording_put + await write_thread_goal(saver, "outcome-thread", met_goal, create_if_missing=True) + await write_thread_goal(saver, "outcome-thread", None, as_node="goal_evaluator", outcome=record) + cleared = await asyncio.to_thread(saver.get_tuple, config) + await write_thread_goal(saver, "outcome-thread", build_goal_state("Next goal")) + replaced = await asyncio.to_thread(saver.get_tuple, config) + + assert put_versions == [set(), {"goal"}, {"goal", "goal_outcome"}, {"goal", "goal_outcome"}] + assert put_writes[2] == {"goal_evaluator": {"goal": None, "goal_outcome": record}} + assert put_writes[3]["goal"]["goal_outcome"] is None + assert cleared.checkpoint["channel_values"]["goal_outcome"] == record + assert "goal" not in cleared.checkpoint["channel_values"] + assert replaced.checkpoint["channel_values"]["goal"]["objective"] == "Next goal" + assert "goal_outcome" not in replaced.checkpoint["channel_values"] + # PostgresSaver drops metadata["writes"] on put; the others store it. + if saver_kind != "postgres": + assert cleared.metadata["writes"] == put_writes[2] + assert replaced.metadata["writes"] == put_writes[3] diff --git a/backend/tests/test_goal_worker.py b/backend/tests/test_goal_worker.py index 4139b70a71..0a7daab9cd 100644 --- a/backend/tests/test_goal_worker.py +++ b/backend/tests/test_goal_worker.py @@ -1389,3 +1389,37 @@ async def test_run_agent_strips_branch_checkpoint_for_goal_continuation(monkeypa assert "checkpoint_id" not in second_config assert "checkpoint_map" not in second_config assert second_config["thread_id"] == "thread-branch-continuation" + + +@pytest.mark.asyncio +@pytest.mark.parametrize("mode", ["full", "delta"]) +async def test_rollback_restores_the_met_goal_record_a_cancelled_run_dropped(mode): + """A rolled-back run whose input replaced the goal must not lose the earlier record.""" + from langgraph.graph import END, START, StateGraph + + from deerflow.agents.thread_state import get_thread_state_schema + from deerflow.runtime.goal import build_goal_outcome + + checkpointer = InMemorySaver() + builder = StateGraph(get_thread_state_schema(mode)) + builder.add_node("agent", lambda state: {"messages": [AIMessage(content="Working on it.")]}) + builder.add_edge(START, "agent") + builder.add_edge("agent", END) + graph = builder.compile(checkpointer=checkpointer) + accessor = CheckpointStateAccessor.bind(graph, checkpointer, mode=mode) + thread_id = "rollback-outcome-thread" + config = {"configurable": {"thread_id": thread_id, "checkpoint_ns": ""}} + record = build_goal_outcome(build_goal_state("Ship it"), GoalEvaluation(satisfied=True, blocker="none", reason="Shipped."), reply_message_id=None) + + await graph.ainvoke({"messages": [HumanMessage(content="Ship it")]}, config) + await write_thread_goal(checkpointer, thread_id, None, outcome=record) + rollback_point = await worker._capture_rollback_point(accessor, checkpointer, config) + await graph.ainvoke({"messages": [HumanMessage(content="Next")], "goal": build_goal_state("Next goal"), "goal_outcome": None}, config) + assert (await accessor.aget(config)).values["goal_outcome"] is None + + restored = await worker._rollback_to_pre_run_checkpoint(accessor=accessor, checkpointer=checkpointer, thread_id=thread_id, run_id="run-1", rollback_point=rollback_point, snapshot_capture_failed=False) + + values = (await accessor.aget(config)).values + assert restored is True + assert values["goal_outcome"] == record + assert values.get("goal") is None diff --git a/backend/tests/test_thread_goal_contract.py b/backend/tests/test_thread_goal_contract.py new file mode 100644 index 0000000000..fa4ea94417 --- /dev/null +++ b/backend/tests/test_thread_goal_contract.py @@ -0,0 +1,96 @@ +"""Contract tests for the chat goal state the web UI reads. + +The chat goal bar translates stand-down codes, recognizes host-written reasons +and reads ``goal_outcome`` from the history head, so the backend must keep +producing exactly what ``contracts/thread_goal_contract.json`` lists. +""" + +from __future__ import annotations + +import json +import re +from pathlib import Path +from types import SimpleNamespace +from typing import get_args, get_type_hints +from unittest.mock import AsyncMock + +import pytest +from _router_auth_helpers import call_unwrapped +from fastapi import BackgroundTasks +from langchain_core.messages import HumanMessage + +from app.gateway.routers import threads +from app.gateway.services import SERVER_OWNED_STATE_CHANNELS +from deerflow.agents.goal_state import GoalOutcomeState +from deerflow.agents.thread_state import ThreadState +from deerflow.persistence.scheduled_task_runs import finalization +from deerflow.runtime import goal +from deerflow.runtime.runs import worker + +_REPO_ROOT = Path(__file__).resolve().parents[2] +_CONTRACT = json.loads((_REPO_ROOT / "contracts" / "thread_goal_contract.json").read_text(encoding="utf-8")) +_MET = {"satisfied": True, "blocker": "none", "reason": "The report is complete."} + + +def test_stand_down_reason_codes_match_the_worker(): + unmet = {"satisfied": False, "blocker": "goal_not_met_yet", "reason": ""} + # goal_not_met_yet never stands down by itself; the caps below decide it. + blocked = {worker._stand_down_reason({}, {**unmet, "blocker": blocker}, 0) for blocker in goal.GOAL_BLOCKERS - {"none"}} - {None} + caps = { + worker._stand_down_reason({"continuation_count": goal.DEFAULT_MAX_GOAL_CONTINUATIONS}, unmet, 0), + worker._stand_down_reason({"continuation_count": 0}, unmet, goal.DEFAULT_MAX_NO_PROGRESS_CONTINUATIONS), + } + literals = set(re.findall(r'stand_down_reason\s*=\s*"([a-z_:]+)"', Path(worker.__file__).read_text(encoding="utf-8"))) + assert literals, "pattern no longer matches worker.py; update this test" + assert set(_CONTRACT["stand_down_reason_codes"]) == blocked | caps | literals + + +def test_check_failure_codes_match_the_scheduled_tasks_page(): + assert tuple(_CONTRACT["check_failure_codes"]) == finalization.CHECK_FAILURE_CODES + assert set(_CONTRACT["check_failure_codes"]) <= set(_CONTRACT["stand_down_reason_codes"]) + + +def test_goal_outcome_shape_matches_the_state_type_and_its_builder(): + contract = _CONTRACT["goal_outcome"] + hints = get_type_hints(GoalOutcomeState) + record = goal.build_goal_outcome(goal.build_goal_state("Ship it"), _MET, reply_message_id="ai-1") + + assert contract["channel"] == goal.GOAL_OUTCOME_CHANNEL + assert contract["channel"] in get_type_hints(ThreadState) + assert contract["channel"] in SERVER_OWNED_STATE_CHANNELS + assert list(hints) == list(record) == contract["keys"] + assert list(get_args(hints["status"])) == contract["statuses"] + assert record["status"] in contract["statuses"] + + +def test_host_reason_texts_are_the_reasons_the_host_writes(): + pattern = r'GoalEvaluation\(\s*satisfied=False,\s*blocker="[a-z_]+",\s*reason="([^"]+)"' + texts = {text for module in (goal, worker) for text in re.findall(pattern, Path(module.__file__).read_text(encoding="utf-8"))} + assert texts, "pattern no longer matches goal.py/worker.py; update this test" + assert set(_CONTRACT["host_reason_texts"]) == texts + + +@pytest.mark.asyncio +async def test_history_head_carries_exactly_the_contract_keys(monkeypatch): + active = goal.build_goal_state("Ship it") + values = { + "title": "Report", + "thread_data": {"workspace_path": "/synthetic/workspace"}, + "messages": [HumanMessage(id="h1", content="Ship it")], + # Server writers never leave both set; the projection must still carry each. + "goal": active, + "goal_outcome": goal.build_goal_outcome(active, _MET, reply_message_id=None), + "todos": [{"content": "write tests", "status": "pending"}], + "artifacts": ["/mnt/user-data/outputs/report.md"], + } + snapshots = [SimpleNamespace(values=values, config={"configurable": {"checkpoint_id": checkpoint_id}}, parent_config=None, metadata={}, next=(), created_at=None) for checkpoint_id in ("head", "older")] + accessor = SimpleNamespace(ahistory=AsyncMock(return_value=snapshots)) + monkeypatch.setattr(threads, "get_checkpointer", lambda _request: None) + monkeypatch.setattr(threads, "build_thread_checkpoint_state_accessor", AsyncMock(return_value=(accessor, {}))) + + request = SimpleNamespace(app=SimpleNamespace(state=SimpleNamespace())) + entries = await call_unwrapped(threads.get_thread_history, thread_id="contract-thread", body=threads.ThreadHistoryRequest(limit=2), request=request, background_tasks=BackgroundTasks()) + + head, older = (entry.model_dump(mode="json")["values"] for entry in entries) + assert set(head) == set(_CONTRACT["history_head_keys"]) + assert set(older) == {"title", "thread_data"} diff --git a/backend/tests/test_thread_regenerate_prepare.py b/backend/tests/test_thread_regenerate_prepare.py index a92fe652d9..76cf24eee8 100644 --- a/backend/tests/test_thread_regenerate_prepare.py +++ b/backend/tests/test_thread_regenerate_prepare.py @@ -1307,6 +1307,37 @@ def test_prepare_edit_regenerate_payload_rejects_active_goal(): assert exc.value.detail == "Cannot edit while a goal is active" +@pytest.mark.parametrize( + "latest_values", + [ + # Clearing a met goal leaves only its record. + {"goal_outcome": {"status": "achieved", "objective": "finish", "reply_message_id": "ai-1"}}, + # POST /state stores an unbuilt goal the history head does not show as active. + {"goal": {"objective": "finish"}}, + ], + ids=["met-goal-record", "goal-without-status"], +) +def test_prepare_edit_regenerate_payload_allows_editing_without_an_active_goal(latest_values): + from app.gateway.routers.thread_runs import _prepare_edit_regenerate_payload + + human = HumanMessage(id="human-1", content="question") + ai = AIMessage(id="ai-1", content="answer v1") + latest = _checkpoint("ckpt-ai", [human, ai]) + latest.checkpoint["channel_values"].update(latest_values) + event_store, run_manager = _answer_run_fixtures() + + response = asyncio.run( + _prepare_edit_regenerate_payload( + "thread-1", + "human-1", + "updated question", + _request(FakeCheckpointer([latest, _checkpoint("ckpt-human", [human]), _checkpoint("ckpt-base", [])]), event_store, run_manager=run_manager), + ) + ) + + assert response.checkpoint["checkpoint_id"] == "ckpt-base" + + def test_prepare_edit_regenerate_payload_requires_successful_source_run(): from app.gateway.routers.thread_runs import _prepare_edit_regenerate_payload diff --git a/backend/tests/test_threads_router.py b/backend/tests/test_threads_router.py index 184831bbb3..7d98cf9713 100644 --- a/backend/tests/test_threads_router.py +++ b/backend/tests/test_threads_router.py @@ -34,6 +34,7 @@ from deerflow.persistence.thread_meta.memory import THREADS_NS, MemoryThreadMeta from deerflow.runtime import ConflictError, ThreadOperationKind from deerflow.runtime.checkpoint_state import CheckpointStateAccessor from deerflow.runtime.context_keys import CHECKPOINT_AGENT_NAME_METADATA_KEY +from deerflow.runtime.goal import build_goal_outcome, build_goal_state, write_thread_goal from deerflow.runtime.user_context import reset_current_user, set_current_user from deerflow.uploads.companions import companion_names, register_companion, resolve_companion from deerflow.utils.file_outline import extract_outline_for_file @@ -3739,6 +3740,162 @@ def test_update_thread_state_overwrite_into_never_written_channel(monkeypatch, m assert read_response.json()["values"]["goal"] == {"objective": "finish"} +def _met_goal_record(reply_message_id: str | None = "a2") -> dict: + return build_goal_outcome(build_goal_state("ship the fix"), {"satisfied": True, "blocker": "none", "reason": "Shipped."}, reply_message_id=reply_message_id) + + +def _record_met_goal(checkpointer, thread_id: str, record: dict) -> None: + asyncio.run(write_thread_goal(checkpointer, thread_id, None, as_node="goal_evaluator", outcome=record)) + + +def _create_extension_thread(client, checkpointer, custom_factory, mode, thread_id) -> None: + created = client.post("/api/threads", json={"thread_id": thread_id, "metadata": {}, "assistant_id": "extension-agent"}) + assert created.status_code == 200, created.text + asyncio.run(_seed_extension_source(checkpointer, custom_factory, mode, thread_id)) + + +@pytest.mark.parametrize("mode", ["full", "delta"]) +def test_history_head_carries_goal_state_without_messages(monkeypatch, mode) -> None: + """useStream rebuilds thread values from the head, so a goal set on /chats/new + and a met-goal record must be there even before any message exists.""" + app, _store, checkpointer = _build_thread_app() + _wire_extension_agent(monkeypatch, app, checkpointer, mode) + thread_id = f"goal-head-{mode}" + record = _met_goal_record(None) + + with TestClient(app) as client: + goal_response = client.put(f"/api/threads/{thread_id}/goal", json={"objective": "ship the fix"}) + assert goal_response.status_code == 200, goal_response.text + set_head = client.post(f"/api/threads/{thread_id}/history", json={"limit": 1}).json()[0]["values"] + _record_met_goal(checkpointer, thread_id, record) + met_history = client.post(f"/api/threads/{thread_id}/history", json={"limit": 10}).json() + + assert set_head == {"goal": goal_response.json()["goal"]} + assert met_history[0]["values"] == {"goal_outcome": record} + # Older entries stay title/thread_data only. + assert len(met_history) > 1 + assert all(not {"goal", "goal_outcome"} & set(entry["values"]) for entry in met_history[1:]) + + +@pytest.mark.parametrize("mode", ["full", "delta"]) +def test_history_head_skips_goal_state_that_is_not_active_or_achieved(monkeypatch, mode) -> None: + """POST /state stores an unbuilt goal; the head must not show it as a goal the backend ignores.""" + app, _store, checkpointer = _build_thread_app() + custom_factory = _wire_extension_agent(monkeypatch, app, checkpointer, mode) + thread_id = f"malformed-goal-head-{mode}" + + with TestClient(app) as client: + _create_extension_thread(client, checkpointer, custom_factory, mode, thread_id) + update_response = client.post(f"/api/threads/{thread_id}/state", json={"values": {"goal": {"objective": "finish"}}}) + assert update_response.status_code == 200, update_response.text + malformed_goal_head = client.post(f"/api/threads/{thread_id}/history", json={"limit": 1}).json()[0]["values"] + _record_met_goal(checkpointer, thread_id, {**_met_goal_record(), "status": "cleared"}) + unknown_outcome_head = client.post(f"/api/threads/{thread_id}/history", json={"limit": 1}).json()[0]["values"] + + assert "goal" not in malformed_goal_head + assert [message["id"] for message in malformed_goal_head["messages"]] == ["h1", "a1", "h2", "a2"] + assert "goal_outcome" not in unknown_outcome_head + + +@pytest.mark.parametrize("mode", ["full", "delta"]) +def test_goal_writes_drop_the_met_goal_record(monkeypatch, mode) -> None: + """PUT/DELETE /goal and a POST /state goal write must not leave an older 'goal met' behind.""" + app, _store, checkpointer = _build_thread_app() + custom_factory = _wire_extension_agent(monkeypatch, app, checkpointer, mode) + thread_id = f"goal-outcome-writers-{mode}" + record = _met_goal_record() + writers = { + "put": lambda client: client.put(f"/api/threads/{thread_id}/goal", json={"objective": "next goal"}), + "delete": lambda client: client.delete(f"/api/threads/{thread_id}/goal"), + "state": lambda client: client.post(f"/api/threads/{thread_id}/state", json={"values": {"goal": None}}), + } + after = {} + + with TestClient(app) as client: + _create_extension_thread(client, checkpointer, custom_factory, mode, thread_id) + for name, write in writers.items(): + _record_met_goal(checkpointer, thread_id, record) + assert client.get(f"/api/threads/{thread_id}/state").json()["values"]["goal_outcome"] == record + response = write(client) + assert response.status_code == 200, response.text + after[name] = client.get(f"/api/threads/{thread_id}/state").json()["values"] + head = client.post(f"/api/threads/{thread_id}/history", json={"limit": 1}).json()[0]["values"] + + assert after["put"]["goal"]["objective"] == "next goal" + assert all(values.get("goal_outcome") is None for values in after.values()) + assert "goal_outcome" not in head + + +def _seed_turn(checkpointer, custom_factory, mode, thread_id, human_id: str, ai_id: str, **values) -> None: + accessor = CheckpointStateAccessor.bind(custom_factory(), checkpointer, mode=mode) + config = {"configurable": {"thread_id": thread_id, "checkpoint_ns": ""}} + + async def seed() -> None: + await accessor.aupdate(config, {"messages": [HumanMessage(id=human_id, content=f"question {human_id}")], **values}, as_node="model") + await accessor.aupdate(config, {"messages": [AIMessage(id=ai_id, content=f"answer {ai_id}")]}, as_node="model") + + asyncio.run(seed()) + + +def _branch_head(client, thread_id: str, message_id: str) -> tuple[str, dict]: + branch_response = client.post(f"/api/threads/{thread_id}/branches", json={"message_id": message_id, "message_ids": [message_id]}) + assert branch_response.status_code == 200, branch_response.text + branch_thread_id = branch_response.json()["thread_id"] + return branch_thread_id, client.post(f"/api/threads/{branch_thread_id}/history", json={"limit": 1}).json()[0]["values"] + + +@pytest.mark.parametrize("mode", ["full", "delta"]) +def test_branch_from_the_met_turn_keeps_the_record_without_the_met_goal(monkeypatch, mode) -> None: + """The branch head is written onto the replay base from before the turn, where the + goal was still active; the branch keeps the record and its message ids, not the goal.""" + app, _store, checkpointer = _build_thread_app() + custom_factory = _wire_extension_agent(monkeypatch, app, checkpointer, mode) + thread_id = f"met-goal-branch-{mode}" + + with TestClient(app) as client: + created = client.post("/api/threads", json={"thread_id": thread_id, "metadata": {}, "assistant_id": "extension-agent"}) + assert created.status_code == 200, created.text + _seed_turn(checkpointer, custom_factory, mode, thread_id, "h1", "a1") + goal_response = client.put(f"/api/threads/{thread_id}/goal", json={"objective": "ship the fix"}) + assert goal_response.status_code == 200, goal_response.text + _seed_turn(checkpointer, custom_factory, mode, thread_id, "h2", "a2") + record = build_goal_outcome(goal_response.json()["goal"], {"satisfied": True, "blocker": "none", "reason": "Shipped."}, reply_message_id="a2") + _record_met_goal(checkpointer, thread_id, record) + branch_thread_id, branch_head = _branch_head(client, thread_id, "a2") + branch_goal = client.get(f"/api/threads/{branch_thread_id}/goal").json()["goal"] + + assert branch_head["goal_outcome"] == record + assert "goal" not in branch_head + assert branch_goal is None + assert [message["id"] for message in branch_head["messages"]] == ["h1", "a1", "h2", "a2"] + + +@pytest.mark.parametrize("mode", ["full", "delta"]) +def test_branch_does_not_take_an_older_met_goal_record_from_the_replay_base(monkeypatch, mode) -> None: + """A run-input goal clears the older record and a stand-down then drops the cleared + key, so only the replay base still holds that record; the branch must not revive it.""" + app, _store, checkpointer = _build_thread_app() + custom_factory = _wire_extension_agent(monkeypatch, app, checkpointer, mode) + thread_id = f"older-record-branch-{mode}" + next_goal = build_goal_state("next goal") + + with TestClient(app) as client: + created = client.post("/api/threads", json={"thread_id": thread_id, "metadata": {}, "assistant_id": "extension-agent"}) + assert created.status_code == 200, created.text + _seed_turn(checkpointer, custom_factory, mode, thread_id, "h1", "a1") + _record_met_goal(checkpointer, thread_id, _met_goal_record("a1")) + # normalize_input's shape for an external run-input goal. + _seed_turn(checkpointer, custom_factory, mode, thread_id, "h2", "a2", goal=next_goal, goal_outcome=None) + asyncio.run(write_thread_goal(checkpointer, thread_id, next_goal, as_node="goal_evaluator")) + source_head = client.post(f"/api/threads/{thread_id}/history", json={"limit": 1}).json()[0]["values"] + _branch_thread_id, branch_head = _branch_head(client, thread_id, "a2") + + assert source_head["goal"] == next_goal + assert "goal_outcome" not in source_head + assert branch_head["goal"] == next_goal + assert "goal_outcome" not in branch_head + + @pytest.mark.parametrize("mode", ["full", "delta"]) def test_update_thread_state_preserves_agent_binding_for_manual_compaction(monkeypatch, mode) -> None: """A manual state rewrite must retain the state-producing agent policy.""" diff --git a/config.example.yaml b/config.example.yaml index d255d31593..a27b5dde77 100644 --- a/config.example.yaml +++ b/config.example.yaml @@ -1474,10 +1474,11 @@ loop_detection: # rewritten to keyed, value-derived placeholders like # [EMAIL_bdqkxwcfmqvrqzblcchjkqbcdqw] before they reach the model. Tokens are # HMAC digests keyed by token_secret, so they cannot be re-derived offline. +# The hidden /goal continuation is redacted before it is stored. # Raw text stays in thread state for the UI; queued memory-extraction -# payloads are redacted too. Compaction and configured LLM title inputs are -# redacted as well; placeholders are value-derived, so an identity redacted -# before compaction renders the same token afterwards. +# payloads are redacted too. Compaction, configured LLM title and /goal +# evaluator inputs are redacted as well; placeholders are value-derived, so +# an identity redacted before compaction renders the same token afterwards. pii_redaction: enabled: false # redact_email: true diff --git a/contracts/thread_goal_contract.json b/contracts/thread_goal_contract.json new file mode 100644 index 0000000000..dbe0245e59 --- /dev/null +++ b/contracts/thread_goal_contract.json @@ -0,0 +1,44 @@ +{ + "version": 1, + "description": "Cross-language contract fixture for the chat goal state. When auto-continue stands down, the worker stores one of stand_down_reason_codes in goal.last_evaluation.stand_down_reason (backend/packages/harness/deerflow/runtime/runs/worker.py); check_failure_codes are the ones where the goal could not be checked. Clearing a met goal writes the goal_outcome channel in the same checkpoint, and every other goal write removes it (runtime/goal.py write_thread_goal); clients cannot write it. The POST /api/threads/{thread_id}/history head carries at most history_head_keys, projecting goal only while it is active and goal_outcome only when achieved. host_reason_texts are the English last_evaluation.reason values the host writes itself instead of the evaluator; the evaluator_failed reason embeds an exception type name and is covered by check_failure_codes. The frontend reads the same file so it never shows a raw code and never labels host text as the evaluator's.", + "stand_down_reason_codes": [ + "blocked:missing_evidence", + "blocked:needs_user_input", + "blocked:run_failed", + "blocked:external_wait", + "max_continuations_reached", + "no_progress_detected", + "token_capped", + "evaluator_failed", + "no_durable_end_of_turn", + "thread_changed_after_evaluation", + "thread_changed_before_continuation" + ], + "check_failure_codes": [ + "evaluator_failed", + "no_durable_end_of_turn", + "thread_changed_after_evaluation", + "thread_changed_before_continuation" + ], + "goal_outcome": { + "channel": "goal_outcome", + "statuses": ["achieved"], + "keys": [ + "status", + "objective", + "goal_created_at", + "achieved_at", + "continuation_count", + "max_continuations", + "reason", + "relied_on_assumption", + "reply_message_id" + ] + }, + "history_head_keys": ["title", "thread_data", "messages", "goal", "goal_outcome"], + "host_reason_texts": [ + "No visible assistant evidence is available yet.", + "The turn ended on a question to the user that has not been answered.", + "No durable assistant end-of-turn receipt was available." + ] +} diff --git a/frontend/src/AGENTS.md b/frontend/src/AGENTS.md index 060d05d2da..af94013729 100644 --- a/frontend/src/AGENTS.md +++ b/frontend/src/AGENTS.md @@ -147,7 +147,7 @@ Conversation references (`read_conversation`, opt-in on the backend) are attache Auth UI note: the login page's "keep me signed in" option submits only `remember_me` to the Gateway and may persist only the email address through `core/auth/remember-login.ts`. Passwords and tokens must never be stored in frontend storage; the `HttpOnly access_token` and readable `csrf_token` cookies remain Gateway-owned. -The web `/` picker lists only `/goal` and `/compact`; skills are selected through the inline `@` picker. `/goal` and `/compact` are built-in composer commands, not skill activations. `src/components/workspace/input-box.tsx` intercepts `/goal`, `/goal clear`, and `/goal ` before normal chat submission, calling Gateway `GET/PUT/DELETE /api/threads/{thread_id}/goal`. Setting `/goal ` also submits the condition text as the next user task so the agent starts running immediately; status and clear do not start a run. On a project-scoped new chat (`/workspace/chats/new?project=…`), the chat page's project pre-create runs before the goal PUT via the composer's `onPrepareThread` callback: the goal endpoint materializes a missing thread row itself, and an unassigned row would make the later idempotent thread create return it without assigning the project. Goal and compact requests are tied to the current `threadId` with an `AbortController`, so switching threads or unmounting the composer aborts in-flight requests and stale responses cannot update the new thread's composer state. The chat pages render `GoalStatus` above the composer from `AgentThreadState.goal`, with local optimistic state until an incremental goal update or final state reload arrives. `/compact` calls `POST /api/threads/{thread_id}/compact` to summarize older active context while leaving the full visible chat history intact; it is skipped on new/empty threads and blocked server-side while a run is in flight. Thread rename uses the same serialized state-write route; the rename dialog stays open and surfaces the server error when an active run returns 409. +The web `/` picker lists only `/goal` and `/compact`; skills are selected through the inline `@` picker. `/goal` and `/compact` are built-in composer commands, not skill activations. `src/components/workspace/input-box.tsx` intercepts `/goal`, `/goal clear`, and `/goal ` before normal chat submission, calling Gateway `GET/PUT/DELETE /api/threads/{thread_id}/goal`. Setting `/goal ` also submits the condition text as the next user task so the agent starts running immediately; status and clear do not start a run. On a project-scoped new chat (`/workspace/chats/new?project=…`), the chat page's project pre-create runs before the goal PUT via the composer's `onPrepareThread` callback: the goal endpoint materializes a missing thread row itself, and an unassigned row would make the later idempotent thread create return it without assigning the project. Goal and compact requests are tied to the current `threadId` with an `AbortController`, so switching threads or unmounting the composer aborts in-flight requests and stale responses cannot update the new thread's composer state. Both chat pages render `GoalStatus` above the composer from `goal`, `goal_outcome` and messages via `useActiveGoal` (e2e: `goal-status.spec.ts`); a `/goal` result stays local until a `goal` value or that goal's met record arrives. `/compact` calls `POST /api/threads/{thread_id}/compact` to summarize older active context while leaving the full visible chat history intact; it is skipped on new/empty threads and blocked server-side while a run is in flight. Thread rename uses the same serialized state-write route; the rename dialog stays open and surfaces the server error when an active run returns 409. Composer `runs:create` gating belongs in `InputBox.handleSubmit` after command classification and before dispatch. Reject message and goal-set actions before `onPrepareThread`, goal persistence, success feedback, or draft clearing. Goal status/clear and compact do not start runs; preserve their endpoint-specific gates. Regression coverage lives in `input-box-send-gating.dom.test.tsx`. diff --git a/frontend/src/app/workspace/agents/[agent_name]/chats/[thread_id]/page.tsx b/frontend/src/app/workspace/agents/[agent_name]/chats/[thread_id]/page.tsx index 5b9fe344eb..b6aecf17ba 100644 --- a/frontend/src/app/workspace/agents/[agent_name]/chats/[thread_id]/page.tsx +++ b/frontend/src/app/workspace/agents/[agent_name]/chats/[thread_id]/page.tsx @@ -338,9 +338,11 @@ export default function AgentChatPage() { (agent.tool_groups == null || agent.tool_groups.includes("browser")); const browserEnabled = !isNewThread && !isMock && browserControlEnabled && agentBrowserEnabled; - const { activeGoal, hasGoal, setLocalGoal } = useActiveGoal( + const { activeGoal, hasGoal, goalOutcome, setLocalGoal } = useActiveGoal( threadId, thread.values.goal, + thread.values.goal_outcome, + thread.messages, ); const hasOpenHumanInputCard = useMemo( () => @@ -350,6 +352,14 @@ export default function AgentChatPage() { ), [thread.messages], ); + // A goal blocks edit-and-rerun server-side; show the pencil locked. + const editBase = + !isNewThread && + !isMock && + env.NEXT_PUBLIC_STATIC_WEBSITE_ONLY !== "true" && + !isUploading && + !thread.isLoading && + !hasOpenHumanInputCard; return ( @@ -467,15 +477,8 @@ export default function AgentChatPage() { !thread.isLoading } onRegenerateMessage={handleRegenerate} - canEdit={ - !isNewThread && - !isMock && - env.NEXT_PUBLIC_STATIC_WEBSITE_ONLY !== "true" && - !isUploading && - !thread.isLoading && - !hasGoal && - !hasOpenHumanInputCard - } + canEdit={editBase && !hasGoal} + editLockedByGoal={editBase && hasGoal} onEditAndRegenerateMessage={handleEditAndRegenerate} onSubmitHumanInput={ isMock || env.NEXT_PUBLIC_STATIC_WEBSITE_ONLY === "true" @@ -501,7 +504,7 @@ export default function AgentChatPage() { : "max-w-(--container-width-md)", )} > - {(hasGoal || hasTodos) && ( + {(hasGoal || goalOutcome !== null || hasTodos) && (
- {activeGoal && } + {hasTodos && ( 0; const browserEnabled = !isNewThread && !isMock && browserControlEnabled; - const { activeGoal, hasGoal, setLocalGoal } = useActiveGoal( + const { activeGoal, hasGoal, goalOutcome, setLocalGoal } = useActiveGoal( threadId, thread.values.goal, + thread.values.goal_outcome, + thread.messages, ); const hasOpenHumanInputCard = useMemo( () => @@ -450,6 +452,15 @@ export default function ChatPage() { !isNewThread && !isMock && threadMetadata.data ? projectIdOfThread(threadMetadata.data) : null; + // A goal blocks edit-and-rerun server-side; show the pencil locked. + const editBase = + !isNewThread && + !isMock && + env.NEXT_PUBLIC_STATIC_WEBSITE_ONLY !== "true" && + !isUploading && + !thread.isLoading && + !branchThread.isPending && + !hasOpenHumanInputCard; return ( @@ -549,16 +560,8 @@ export default function ChatPage() { !thread.isLoading } onRegenerateMessage={handleRegenerate} - canEdit={ - !isNewThread && - !isMock && - env.NEXT_PUBLIC_STATIC_WEBSITE_ONLY !== "true" && - !isUploading && - !thread.isLoading && - !branchThread.isPending && - !hasGoal && - !hasOpenHumanInputCard - } + canEdit={editBase && !hasGoal} + editLockedByGoal={editBase && hasGoal} onEditAndRegenerateMessage={handleEditAndRegenerate} onSubmitHumanInput={ isMock || env.NEXT_PUBLIC_STATIC_WEBSITE_ONLY === "true" @@ -592,7 +595,7 @@ export default function ChatPage() { : "max-w-(--container-width-md)", )} > - {(hasGoal || hasTodos) && ( + {(hasGoal || goalOutcome !== null || hasTodos) && (
- {activeGoal && } + {hasTodos && ( 0`). Before that, the raw "0/8" reads as a mysterious - * score, so the counter is hidden; once continuation starts it surfaces as - * "{count}/{max}" with an explanatory tooltip. + * English `last_evaluation.reason` values the host writes itself instead of + * the evaluator (contract `host_reason_texts`). They are never shown as the + * evaluator's note. */ -export function getGoalContinuationDisplay( - goal: Pick, -): GoalContinuationDisplay | null { - const count = goal.continuation_count ?? 0; - const max = goal.max_continuations ?? 0; - if (!Number.isFinite(count) || count <= 0) { +export const GOAL_HOST_REASON_TEXTS: readonly string[] = [ + "No visible assistant evidence is available yet.", + "The turn ended on a question to the user that has not been answered.", + "No durable assistant end-of-turn receipt was available.", +]; + +export function isThreadGoalOutcome( + value: unknown, +): value is ThreadGoalOutcome { + // A primitive or an array has no `status`, so this rejects them too. + const record = value as Record | null; + return ( + record?.status === "achieved" && + typeof record.objective === "string" && + typeof record.goal_created_at === "string" + ); +} + +/** True when `outcome` records this very goal, not an earlier one with the same text. */ +export function sameGoalInstance( + goal: Pick, + outcome: Pick, +): boolean { + return ( + goal.objective === outcome.objective && + (goal.created_at ?? "") === outcome.goal_created_at + ); +} + +/** Identity of one met record; a later record of the same goal text differs. */ +export function goalOutcomeKey(outcome: ThreadGoalOutcome | null): string { + if (!outcome) { + return "none"; + } + const { objective, goal_created_at, achieved_at } = outcome; + return [objective, goal_created_at, achieved_at].join("|"); +} + +/** + * Whether a message shows the conversation moved on: a visible human + * message, or a visible AI message with text. The backend anchors the met + * record on the latest visible AI message with text, so a tool-call-only or + * empty AI message after that anchor is not a later turn either. + */ +function isVisibleConversationMessage(message: Message): boolean { + return ( + !isHiddenFromUIMessage(message) && + (message.type === "human" || + (message.type === "ai" && + extractTextFromMessage(message).trim().length > 0)) + ); +} + +/** + * "Goal met" stays current while its anchor reply is in the conversation and + * nothing visible came after it. Derived from server state plus messages, so + * a reload agrees with the live view. + */ +export function isGoalOutcomeCurrent( + outcome: Pick, + messages: readonly Message[], +): boolean { + const anchorId = outcome.reply_message_id; + const anchorIndex = anchorId + ? messages.findIndex((message) => message.id === anchorId) + : -1; + return ( + anchorIndex >= 0 && + !messages.slice(anchorIndex + 1).some(isVisibleConversationMessage) + ); +} + +/** A row of the stand-down copy in `inputBox.goalBar.next`. */ +export type GoalNextKey = keyof Translations["inputBox"]["goalBar"]["next"]; +export type GoalStopKey = Exclude; +/** Who acts next: amber "Stopped", primary "Waiting for you", muted otherwise. */ +export type GoalBarChip = "stopped" | "waitingForYou" | "waiting" | "unchecked"; + +export type GoalBarView = + | { kind: "none" } + | { kind: "set"; goal: GoalState } + | { kind: "inProgress"; goal: GoalState } + | { kind: "continuing"; goal: GoalState; count: number; max: number } + | { + kind: "stopped"; + goal: GoalState; + stop: GoalStopKey; + chip: GoalBarChip; + reasonKey: GoalReasonKey | null; + count: number; + max: number; + /** The evaluator's own words; null when the host wrote the reason. */ + note: string | null; + /** The stand-down code, only when this build has no copy for it. */ + rawCode: string | null; + } + | { kind: "paused"; goal: GoalState; count: number; max: number } + | { + kind: "met"; + outcome: ThreadGoalOutcome; + count: number; + max: number; + reliedOnAssumption: boolean; + note: string | null; + }; + +function countOf(value: unknown): number { + return typeof value === "number" && Number.isFinite(value) && value > 0 + ? Math.floor(value) + : 0; +} + +function evaluatorNote(reason: unknown): string | null { + const note = typeof reason === "string" ? reason.trim() : ""; + return note && !GOAL_HOST_REASON_TEXTS.includes(note) ? note : null; +} + +type StopRow = { stop: GoalStopKey; chip: GoalBarChip }; + +// Stand-down codes that map one-to-one through the shared reason keys. +const STOP_BY_REASON: Partial> = { + noProgress: { stop: "noProgress", chip: "stopped" }, + tokenCapped: { stop: "tokenCapped", chip: "stopped" }, + missingEvidence: { stop: "missingEvidence", chip: "stopped" }, + runFailed: { stop: "runFailed", chip: "stopped" }, + externalWait: { stop: "external", chip: "waiting" }, +}; + +function stopRowOf( + code: string, + max: number, + hasOpenHumanInputCard: boolean, +): StopRow { + if (CHECK_FAILURE_CODES.includes(code)) { + return { stop: "unchecked", chip: "unchecked" }; + } + const reasonKey = goalReasonKeyOf(code); + if (reasonKey === "maxContinuations") { + return { stop: max > 0 ? "limit" : "autoOff", chip: "stopped" }; + } + if (reasonKey === "needsUserInput") { + // Either the turn ended on an open question card, or the evaluator saw + // the agent guess; only the first has a question to answer. + const stop = hasOpenHumanInputCard ? "needsInputCard" : "needsInputReply"; + return { stop, chip: "waitingForYou" }; + } + const row = reasonKey ? STOP_BY_REASON[reasonKey] : undefined; + return row ?? { stop: "unknown", chip: "stopped" }; +} + +/** + * The goal bar state. An active goal always wins over a met record; while a + * run streams, a stand-down reason left by an earlier run is not shown. + * Idle: a stand-down, then a paused auto-continue, then a plain set goal. + * `outcome` is a record `useActiveGoal` already judged current. + */ +export function describeGoalBar({ + goal, + outcome, + isRunning, + hasOpenHumanInputCard = false, +}: { + goal: GoalState | null; + outcome: ThreadGoalOutcome | null; + isRunning: boolean; + hasOpenHumanInputCard?: boolean; +}): GoalBarView { + if (goal) { + const evaluation = goal.last_evaluation; + const code = evaluation?.stand_down_reason?.trim() ?? ""; + const count = countOf(goal.continuation_count); + const max = countOf(goal.max_continuations); + if (isRunning) { + return !code && evaluation?.satisfied !== true && count > 0 + ? { kind: "continuing", goal, count, max } + : { kind: "inProgress", goal }; + } + if (code) { + const row = stopRowOf(code, max, hasOpenHumanInputCard); + return { + kind: "stopped", + goal, + ...row, + reasonKey: goalReasonKeyOf(code), + count, + max, + // A failed check has no verdict to show, even when the stored + // evaluation says satisfied. + note: + row.stop === "unchecked" ? null : evaluatorNote(evaluation?.reason), + rawCode: row.stop === "unknown" ? code : null, + }; + } + if (evaluation && count > 0) { + // No checker note: when a satisfied verdict's clear was skipped, the + // stored evaluation is the earlier unmet one. + return { kind: "paused", goal, count, max }; + } + return { kind: "set", goal }; + } + if (outcome) { + return { + kind: "met", + outcome, + count: countOf(outcome.continuation_count), + max: countOf(outcome.max_continuations), + reliedOnAssumption: outcome.relied_on_assumption === true, + note: evaluatorNote(outcome.reason), + }; + } + return { kind: "none" }; +} + +function fill(template: string, values: Record) { + // Function replacer, so an objective with `$&`/`$1` stays literal. + return template.replace(/\{(\w+)\}/g, (match, key: string) => + key in values ? String(values[key]) : match, + ); +} + +export type GoalBarCopy = { + /** "Goal" or "Goal met". */ + label: string; + objective: string; + /** The state chip; null for a plain set goal and a met goal. */ + chip: string | null; + /** A met goal's auto-continue count; null when it never auto-continued. */ + meta: string | null; + /** The second line: why it stopped and what to do next. */ + detail: string | null; + /** Live-region text; empty for states that are not announced. */ + announcement: string; +}; + +/** Localized copy of a goal bar state; null when no bar is shown. */ +export function describeGoalBarCopy( + view: GoalBarView, + t: Translations, +): GoalBarCopy | null { + const bar = t.inputBox.goalBar; + const announce = (status: string, detail: string) => + fill(bar.announce, { status, detail }); + if (view.kind === "none") { return null; } - return { count, max }; + if (view.kind === "met") { + const { count } = view; + return { + label: t.scheduledTasks.goal.met, + objective: view.outcome.objective, + chip: null, + meta: + count === 1 + ? bar.autoContinuedOnce + : count > 1 + ? fill(bar.autoContinuedMany, { count }) + : null, + detail: null, + announcement: announce(t.scheduledTasks.goal.met, view.outcome.objective), + }; + } + const copy: GoalBarCopy = { + label: t.inputBox.goalLabel, + objective: view.goal.objective, + chip: null, + meta: null, + detail: null, + announcement: "", + }; + // Stopped and paused rows announce their chip with the next step. + const withDetail = (chip: string, detail: string) => ({ + ...copy, + chip, + detail, + announcement: announce(chip, detail), + }); + switch (view.kind) { + case "set": + return copy; + case "inProgress": + return { ...copy, chip: bar.inProgress }; + case "continuing": { + const chip = fill(t.inputBox.goalContinuing, { + count: view.count, + max: view.max, + }); + return { ...copy, chip, announcement: announce(copy.label, chip) }; + } + case "paused": + return withDetail( + bar.paused, + fill(bar.next.paused, { count: view.count, max: view.max }), + ); + case "stopped": { + const goalCopy = t.scheduledTasks.goal; + // Only the "unchecked" row names its reason, in the scheduled page's + // words after the "Couldn't check the goal" its chip already shows. + const uncheckedReasons: Partial> = + bar.uncheckedReasons; + const reason = view.reasonKey + ? (uncheckedReasons[view.reasonKey] ?? goalCopy.reasons[view.reasonKey]) + : goalCopy.unchecked; + return withDetail( + view.chip === "unchecked" ? goalCopy.unchecked : bar[view.chip], + fill(bar.next[view.stop], { count: view.count, max: view.max, reason }), + ); + } + } +} + +/** + * What the live region and Details key on: a new state, or the same state of + * another goal (a thread switch), changes it. Streaming re-renders and a + * rising counter do not, and a set or in-progress goal is silent. + */ +export function goalBarAnnounceKey(view: GoalBarView): string { + if ( + view.kind === "none" || + view.kind === "set" || + view.kind === "inProgress" + ) { + return ""; + } + if (view.kind === "met") { + return `met:${goalOutcomeKey(view.outcome)}`; + } + const state = view.kind === "stopped" ? `stopped:${view.stop}` : view.kind; + return [state, view.goal.objective, view.goal.created_at].join("|"); } /** diff --git a/frontend/src/components/workspace/goal-status.tsx b/frontend/src/components/workspace/goal-status.tsx index 01dd170755..9559076cfc 100644 --- a/frontend/src/components/workspace/goal-status.tsx +++ b/frontend/src/components/workspace/goal-status.tsx @@ -1,51 +1,276 @@ "use client"; -import { RefreshCwIcon, TargetIcon } from "lucide-react"; +import { + ChevronDownIcon, + CircleCheckIcon, + CircleHelpIcon, + CircleStopIcon, + ClockIcon, + GaugeIcon, + LoaderCircleIcon, + MessageCircleQuestionIcon, + PauseIcon, + RefreshCwIcon, + TargetIcon, + TriangleAlertIcon, + type LucideIcon, +} from "lucide-react"; +import { useId, useState } from "react"; import { useI18n } from "@/core/i18n/hooks"; -import type { GoalState } from "@/core/threads"; +import type { GoalState, ThreadGoalOutcome } from "@/core/threads"; import { cn } from "@/lib/utils"; -import { getGoalContinuationDisplay } from "./goal-status-helpers"; +import { + describeGoalBar, + describeGoalBarCopy, + goalBarAnnounceKey, + type GoalBarChip, + type GoalStopKey, +} from "./goal-status-helpers"; import { Tooltip } from "./tooltip"; +const STOP_ICONS: Record = { + limit: TriangleAlertIcon, + autoOff: CircleStopIcon, + noProgress: CircleStopIcon, + tokenCapped: GaugeIcon, + missingEvidence: CircleStopIcon, + runFailed: TriangleAlertIcon, + needsInputCard: MessageCircleQuestionIcon, + needsInputReply: MessageCircleQuestionIcon, + external: ClockIcon, + unchecked: CircleHelpIcon, + unknown: TriangleAlertIcon, +}; + +const CHIP_TONES: Record = { + stopped: "border-amber-500/40 text-amber-700 dark:text-amber-400", + waitingForYou: "border-primary/40 text-primary", + waiting: "text-muted-foreground", + unchecked: "text-muted-foreground", + paused: "text-muted-foreground", +}; + +/** + * The goal bar above the composer: the active goal and why it stopped, or a + * current "Goal met" record. Renders nothing without either. + */ export function GoalStatus({ className, goal, + outcome = null, + isRunning = false, + hasOpenHumanInputCard = false, }: { className?: string; - goal: GoalState; + goal: GoalState | null; + outcome?: ThreadGoalOutcome | null; + isRunning?: boolean; + hasOpenHumanInputCard?: boolean; }) { - const { t } = useI18n(); - const continuation = getGoalContinuationDisplay(goal); + const { locale, t } = useI18n(); + const bar = t.inputBox.goalBar; + const view = describeGoalBar({ + goal, + outcome, + isRunning, + hasOpenHumanInputCard, + }); + const copy = describeGoalBarCopy(view, t); + const announceKey = goalBarAnnounceKey(view); + const announcementText = copy?.announcement ?? ""; + const [announced, setAnnounced] = useState({ + key: announceKey, + locale, + text: announcementText, + }); + const [detailsOpen, setDetailsOpen] = useState(false); + if (announced.key !== announceKey || announced.locale !== locale) { + // Re-announce on a state change only, never on a streaming re-render; a + // language switch refreshes the text once. A new state starts collapsed. + if (announced.key !== announceKey) { + setDetailsOpen(false); + } + setAnnounced({ key: announceKey, locale, text: announcementText }); + } + const detailsId = useId(); + const assumptionId = useId(); + + if (!copy) { + return null; + } + const met = view.kind === "met"; + const hasDetails = view.kind === "stopped" || view.kind === "paused" || met; + const note = "note" in view ? view.note : null; + const rawCode = "rawCode" in view ? view.rawCode : null; + const reliedOnAssumption = met && view.reliedOnAssumption; + const chipStyle = + view.kind === "stopped" + ? { Icon: STOP_ICONS[view.stop], tone: CHIP_TONES[view.chip] } + : view.kind === "paused" + ? { Icon: PauseIcon, tone: CHIP_TONES.paused } + : null; + return ( -
- -
- - {t.inputBox.goalLabel} - - {goal.objective} -
- {continuation && ( - + - - - {t.inputBox.goalContinuing - .replace("{count}", String(continuation.count)) - .replace("{max}", String(continuation.max))} + {met ? ( + + ) : ( + + )} + +
+ + {copy.label} - + {copy.objective} +
+ {view.kind === "inProgress" && ( + + + {copy.chip} + + )} + {view.kind === "continuing" && ( + + + + {copy.chip} + + + )} + {chipStyle && ( + + + {copy.chip} + + )} + {copy.meta && ( + + {copy.meta} + + )} + {reliedOnAssumption && ( + + + {t.scheduledTasks.goal.assumptionBadge} + + + )} + {hasDetails && ( + + )} +
+ {copy.detail && ( +

+ {copy.detail} +

)} -
+ {reliedOnAssumption && ( + + {bar.assumptionTooltip} + + )} + {hasDetails && ( + + )} +

+ {announced.text} +

+ ); } diff --git a/frontend/src/components/workspace/input-box.tsx b/frontend/src/components/workspace/input-box.tsx index 81ebd2bb55..1759af7330 100644 --- a/frontend/src/components/workspace/input-box.tsx +++ b/frontend/src/components/workspace/input-box.tsx @@ -205,9 +205,16 @@ import { } from "./model-picker-content"; import { ReferenceAttachmentSummary, useMaybeSidecar } from "./sidecar"; import { Tooltip } from "./tooltip"; +import { + normalizeGoalStatusRead, + type GoalChangeKind, +} from "./use-active-goal"; const COMPOSER_DRAFT_SAVE_DELAY_MS = 300; +/** A `/goal` request either saved, failed (toast shown), or went stale. */ +type GoalCommandResult = "saved" | "failed" | "stale"; + function focusContentEditableEnd(element: HTMLElement | null) { if (!element) { return; @@ -392,7 +399,7 @@ export function InputBox({ options?: { automatic: boolean }, ) => void; onFollowupsVisibilityChange?: (visible: boolean) => void; - onGoalChange?: (goal: GoalState | null) => void; + onGoalChange?: (goal: GoalState | null, kind: GoalChangeKind) => void; /** * Prepare a not-yet-materialized thread before a builtin command creates * it server-side. The `/goal ` PUT endpoint materializes a @@ -1215,9 +1222,16 @@ export function InputBox({ ); const handleGoalCommand = useCallback( - async (command: GoalCommand): Promise => { + async (command: GoalCommand): Promise => { const request = beginGoalRequest(goalRequestStateRef.current, threadId); const signal = request.controller.signal; + // A 409 means a run this tab is not streaming is still in flight. + const requestError = async (response: Response) => + new Error( + response.status === 409 + ? t.inputBox.goalBar.busy + : await readGoalResponseError(response), + ); try { let goal: GoalState | null = null; if (command.kind === "status") { @@ -1228,11 +1242,11 @@ export function InputBox({ { method: "GET", signal }, ); if (!response.ok) { - throw new Error(await readGoalResponseError(response)); + throw await requestError(response); } - goal = - ((await response.json()) as { goal?: GoalState | null }).goal ?? - null; + goal = normalizeGoalStatusRead( + ((await response.json()) as { goal?: unknown }).goal, + ); if ( !isCurrentGoalRequest( goalRequestStateRef.current, @@ -1240,7 +1254,7 @@ export function InputBox({ threadId, ) ) { - return false; + return "stale"; } const objective = goal?.objective; toast.info( @@ -1250,7 +1264,7 @@ export function InputBox({ t.inputBox.goalActive.replace("{goal}", () => objective) : t.inputBox.goalNone, ); - onGoalChange?.(goal); + onGoalChange?.(goal, "status"); } else if (command.kind === "clear") { const response = await fetch( `${getBackendBaseURL()}/api/threads/${encodeURIComponent( @@ -1259,7 +1273,7 @@ export function InputBox({ { method: "DELETE", signal }, ); if (!response.ok) { - throw new Error(await readGoalResponseError(response)); + throw await requestError(response); } if ( !isCurrentGoalRequest( @@ -1268,10 +1282,10 @@ export function InputBox({ threadId, ) ) { - return false; + return "stale"; } toast.success(t.inputBox.goalCleared); - onGoalChange?.(null); + onGoalChange?.(null, "clear"); } else { const response = await fetch( `${getBackendBaseURL()}/api/threads/${encodeURIComponent( @@ -1285,7 +1299,7 @@ export function InputBox({ }, ); if (!response.ok) { - throw new Error(await readGoalResponseError(response)); + throw await requestError(response); } goal = ((await response.json()) as { goal?: GoalState | null }).goal ?? @@ -1297,24 +1311,24 @@ export function InputBox({ threadId, ) ) { - return false; + return "stale"; } toast.success(t.inputBox.goalSet); - onGoalChange?.(goal); + onGoalChange?.(goal, "set"); } textInput.setInput(""); - return true; + return "saved"; } catch (error) { if ( isAbortError(error) || !isCurrentGoalRequest(goalRequestStateRef.current, request, threadId) ) { - return false; + return "stale"; } toast.error( error instanceof Error ? error.message : t.inputBox.goalFailed, ); - return false; + return "failed"; } finally { finishGoalRequest(goalRequestStateRef.current, request); } @@ -1322,6 +1336,7 @@ export function InputBox({ [ onGoalChange, t.inputBox.goalActive, + t.inputBox.goalBar.busy, t.inputBox.goalCleared, t.inputBox.goalFailed, t.inputBox.goalNone, @@ -1709,9 +1724,15 @@ export function InputBox({ setFollowups([]); setFollowupsHidden(false); setFollowupsLoading(false); - const saved = await handleGoalCommand(submitAction.command); + const result = await handleGoalCommand(submitAction.command); + if (result !== "saved") { + // Reject so PromptInput leaves the composer alone: a failed command + // keeps the /goal text for a retry, and a stale one leaves it to the + // request or conversation that replaced it. + return Promise.reject(new Error(`goal-command-${result}`)); + } // Only start a run when a goal was actually saved; status/clear never run. - if (saved && submitAction.command.kind === "set") { + if (submitAction.command.kind === "set") { return submitThreadMessage({ ...message, text: submitAction.command.objective, diff --git a/frontend/src/components/workspace/messages/message-list-item.tsx b/frontend/src/components/workspace/messages/message-list-item.tsx index 0c97d518a6..4a02c90a62 100644 --- a/frontend/src/components/workspace/messages/message-list-item.tsx +++ b/frontend/src/components/workspace/messages/message-list-item.tsx @@ -11,6 +11,7 @@ import { import { memo, useCallback, + useId, useMemo, useState, type ImgHTMLAttributes, @@ -148,6 +149,7 @@ export function MessageListItem({ showWorkspaceChanges = false, durationSeconds, canEdit = false, + editLockedByGoal = false, isEditPending = false, onEditAndRegenerate, }: { @@ -162,10 +164,13 @@ export function MessageListItem({ showWorkspaceChanges?: boolean; durationSeconds?: number; canEdit?: boolean; + /** Edit would be allowed but for an active goal: show the pencil locked. */ + editLockedByGoal?: boolean; isEditPending?: boolean; onEditAndRegenerate?: (replacementText: string) => void | Promise; }) { const { t } = useI18n(); + const editLockedId = useId(); const isHuman = message.type === "human"; // One derivation serves both editing and the toolbar, and only runs when // either consumer can use it: assistant rows never render this toolbar @@ -247,8 +252,10 @@ export function MessageListItem({ isHuman ? "absolute right-0 -bottom-9 left-0 justify-end" : "absolute right-0 bottom-0 left-0", - "z-20 opacity-0 transition-opacity delay-200 duration-300 group-hover/conversation-message:opacity-100", + // focus-within: a keyboard user tabbing to a button sees it. + "z-20 opacity-0 transition-opacity delay-200 duration-300 group-hover/conversation-message:opacity-100 focus-within:opacity-100", )} + data-testid="message-toolbar" >
@@ -266,6 +273,32 @@ export function MessageListItem({ )} + {!canEdit && + editLockedByGoal && + isHuman && + onEditAndRegenerate && + !isEditing && ( + <> + + + + + {t.inputBox.goalBar.editLocked} + + + )} {feedback !== undefined && runId && threadId && ( void | Promise; canRegenerate?: boolean; canEdit?: boolean; + /** Edit would be allowed but for an active goal: show the pencil locked. */ + editLockedByGoal?: boolean; canBranch?: boolean; enableSidecarActions?: boolean; enableConversationOutline?: boolean; @@ -1259,6 +1262,13 @@ export function MessageList({ )} > {group.messages.map((msg) => { + const isLatestEditableHumanMessage = + group.type === "human" && + !group.scheduledOrigin && + Boolean(msg.id) && + msg.id === latestEditableHumanMessageId && + !replayActionBusy && + Boolean(onEditAndRegenerateMessage); const item = ( void; + /** The current "Goal met" record; null while a goal is active. */ + goalOutcome: ThreadGoalOutcome | null; + /** Apply the result of a `/goal` command (`null` hides the goal). */ + setLocalGoal: (goal: GoalState | null, kind: GoalChangeKind) => void; }; export function resolveActiveGoal( @@ -19,6 +32,35 @@ export function resolveActiveGoal( return localGoal !== undefined ? localGoal : (serverGoal ?? null); } +/** + * The goal and met record to render. An active goal wins over any record; a + * record shows only while it is current (`isGoalOutcomeCurrent`) and this + * tab's own `/goal` set or clear has not removed it on the server. + */ +export function resolveGoalView({ + localGoal, + serverGoal, + serverOutcome, + hiddenOutcomeKey, + messages, +}: { + localGoal: GoalState | null | undefined; + serverGoal: GoalState | null | undefined; + serverOutcome: unknown; + hiddenOutcomeKey: string | null; + messages: readonly Message[]; +}): { activeGoal: GoalState | null; goalOutcome: ThreadGoalOutcome | null } { + const activeGoal = resolveActiveGoal(localGoal, serverGoal); + const goalOutcome = + !activeGoal && + isThreadGoalOutcome(serverOutcome) && + goalOutcomeKey(serverOutcome) !== hiddenOutcomeKey && + isGoalOutcomeCurrent(serverOutcome, messages) + ? serverOutcome + : null; + return { activeGoal, goalOutcome }; +} + export function shouldResetLocalGoalOverride({ serverGoalProvided, threadChanged, @@ -32,37 +74,99 @@ export function shouldResetLocalGoalOverride({ return serverGoalProvided; } +/** A present `goal` value that is not an active goal counts as no goal. */ +export function normalizeServerGoal( + value: unknown, +): GoalState | null | undefined { + return value === undefined ? undefined : isGoalState(value) ? value : null; +} + /** - * Reconciles the optimistic `/goal`-command result with the server's goal state. + * A `/goal` status read, judged by the same rule as thread values. A goal + * stored without `created_at` stays; its met record has `goal_created_at` "". + */ +export function normalizeGoalStatusRead(value: unknown): GoalState | null { + return isGoalState(value) ? value : null; +} + +/** + * Reconciles the optimistic `/goal`-command result with the server's goal + * state and its met record. * - * A `/goal` command updates the UI immediately via `setLocalGoal`, but that - * override is dropped as soon as the server explicitly reports goal state — - * switching threads, a new `continuation_count`, or a cleared goal. A stream - * chunk that omits the `goal` field is not treated as a clear, because - * clarification interrupts can publish partial values while the active goal is - * still present in the checkpoint. + * A `/goal` command updates the UI immediately via `setLocalGoal`. The + * override yields when the server reports goal state: a thread switch, a + * present `goal` value (a new `continuation_count`, a stand-down), or a met + * record of that same goal. Chat streams carry no `values` snapshots, so a + * goal met on the first run arrives only as a frame without the `goal` key + * plus `goal_outcome`. A missing key alone is not a clear, since partial + * values can omit it; the record is. Matching the record to the override's + * own goal keeps a goal set in the same tick. */ export function useActiveGoal( threadId: string, - serverGoal: GoalState | null | undefined, + serverGoalValue: unknown, + serverOutcome: unknown, + messages: readonly Message[], ): UseActiveGoalResult { - const [localGoal, setLocalGoal] = useState( + const [localGoal, setLocalGoalState] = useState( undefined, ); + const [hiddenOutcomeKey, setHiddenOutcomeKey] = useState(null); const previousThreadIdRef = useRef(threadId); + const serverGoal = normalizeServerGoal(serverGoalValue); const serverGoalProvided = serverGoal !== undefined; const serverGoalKey = serverGoalProvided ? goalReconciliationKey(serverGoal) : "missing"; + const outcome = isThreadGoalOutcome(serverOutcome) ? serverOutcome : null; + const outcomeKey = goalOutcomeKey(outcome); + const outcomeRef = useRef(outcome); + outcomeRef.current = outcome; useEffect(() => { const threadChanged = previousThreadIdRef.current !== threadId; previousThreadIdRef.current = threadId; + if (threadChanged) { + setHiddenOutcomeKey(null); + } if (shouldResetLocalGoalOverride({ serverGoalProvided, threadChanged })) { - setLocalGoal(undefined); + setLocalGoalState(undefined); } }, [serverGoalKey, serverGoalProvided, threadId]); - const activeGoal = resolveActiveGoal(localGoal, serverGoal); - return { activeGoal, hasGoal: Boolean(activeGoal), setLocalGoal }; + useEffect(() => { + const met = outcomeRef.current; + if (!met) { + return; + } + setLocalGoalState((previous) => + previous && sameGoalInstance(previous, met) ? undefined : previous, + ); + }, [outcomeKey]); + + const setLocalGoal = useCallback( + (goal: GoalState | null, kind: GoalChangeKind) => { + setLocalGoalState(goal); + if (kind !== "status") { + // The write removed the record on the server, but this tab's values + // keep it until the next run. + setHiddenOutcomeKey(goalOutcomeKey(outcomeRef.current)); + } + }, + [], + ); + + const { activeGoal, goalOutcome } = resolveGoalView({ + localGoal, + serverGoal, + serverOutcome: outcome, + hiddenOutcomeKey, + messages, + }); + return { + activeGoal, + hasGoal: Boolean(activeGoal), + goalOutcome, + setLocalGoal, + }; } diff --git a/frontend/src/core/i18n/locales/en-US.ts b/frontend/src/core/i18n/locales/en-US.ts index a734ff73a4..bd136919cc 100644 --- a/frontend/src/core/i18n/locales/en-US.ts +++ b/frontend/src/core/i18n/locales/en-US.ts @@ -494,6 +494,53 @@ export const enUS: Translations = { goalFailed: "Goal command failed.", goalTooLong: "Goal is too long. Keep it under {max} characters.", goalLengthCounter: "Goal length: {length}/{max} characters", + goalBar: { + regionLabel: "Goal status", + inProgress: "In progress", + stopped: "Stopped", + waitingForYou: "Waiting for you", + waiting: "Waiting", + paused: "Paused", + autoContinuedOnce: "Auto-continued once", + autoContinuedMany: "Auto-continued {count} times", + details: "Details", + hideDetails: "Hide details", + noteLabel: "Goal check note (from the checker)", + codeLabel: "Stop code", + assumptionTooltip: + "The agent filled in something the goal didn't specify and said so in its reply. Check the reply above.", + announce: "{status}: {detail}", + editLocked: + "Editing is off while a goal is set. Run /goal clear to edit.", + busy: "A run is still going in this chat. Try again when it finishes.", + next: { + limit: + "Continuation limit reached · {count}/{max}. It won't auto-continue again. Reply to keep going, or set the goal again with /goal to start a fresh count (this also starts a new run).", + autoOff: + "Goal check: not met yet. Auto-continue is off for this goal. Reply to keep going.", + noProgress: + "No progress between turns. Reply with what's missing, or rephrase the goal with /goal.", + tokenCapped: + "Token budget reached. Send a message to continue; a new run starts with a fresh budget.", + missingEvidence: + "Goal check: evidence missing. Ask it to show the result or explain what's missing; if the goal no longer applies, run /goal clear.", + runFailed: + "Goal check: the run did not finish the work. Check the reply for errors, then send a message to retry.", + needsInputCard: "Answer the question above to continue.", + needsInputReply: "Reply with the missing details to continue.", + external: + "Goal check: waiting on something external. Send a message when it's ready.", + unchecked: "{reason}. It's checked again after your next message.", + unknown: "Send a message to keep going.", + paused: + "Auto-continued {count}/{max}. The run ended before the goal was confirmed. Send a message to check again.", + }, + uncheckedReasons: { + evaluatorFailed: "The run itself may be fine", + noDurableEndOfTurn: "No final reply was saved", + threadChanged: "The chat changed during the check", + }, + }, compactSuccess: "Earlier context compacted. The full chat remains visible; future model calls will use the summary and recent messages.", compactSkipped: "The current context does not need compaction yet.", diff --git a/frontend/src/core/i18n/locales/types.ts b/frontend/src/core/i18n/locales/types.ts index 02833ef227..d4359907fc 100644 --- a/frontend/src/core/i18n/locales/types.ts +++ b/frontend/src/core/i18n/locales/types.ts @@ -404,6 +404,44 @@ export interface Translations { goalFailed: string; goalTooLong: string; goalLengthCounter: string; + goalBar: { + regionLabel: string; + inProgress: string; + stopped: string; + waitingForYou: string; + waiting: string; + paused: string; + autoContinuedOnce: string; + autoContinuedMany: string; + details: string; + hideDetails: string; + noteLabel: string; + codeLabel: string; + assumptionTooltip: string; + announce: string; + editLocked: string; + busy: string; + next: { + limit: string; + autoOff: string; + noProgress: string; + tokenCapped: string; + missingEvidence: string; + runFailed: string; + needsInputCard: string; + needsInputReply: string; + external: string; + unchecked: string; + unknown: string; + paused: string; + }; + /** Why a check failed, without the "Couldn't check the goal" of its chip. */ + uncheckedReasons: { + evaluatorFailed: string; + noDurableEndOfTurn: string; + threadChanged: string; + }; + }; compactSuccess: string; compactSkipped: string; compactFailed: string; diff --git a/frontend/src/core/i18n/locales/zh-CN.ts b/frontend/src/core/i18n/locales/zh-CN.ts index eac7ab6fa4..b66a3e3d9d 100644 --- a/frontend/src/core/i18n/locales/zh-CN.ts +++ b/frontend/src/core/i18n/locales/zh-CN.ts @@ -460,6 +460,50 @@ export const zhCN: Translations = { goalFailed: "目标命令执行失败。", goalTooLong: "目标过长,请控制在 {max} 个字符以内。", goalLengthCounter: "目标长度:{length}/{max} 字符", + goalBar: { + regionLabel: "目标状态", + inProgress: "进行中", + stopped: "已停止", + waitingForYou: "等你回复", + waiting: "等待中", + paused: "已暂停", + autoContinuedOnce: "自动续跑了 1 次", + autoContinuedMany: "自动续跑了 {count} 次", + details: "详情", + hideDetails: "收起详情", + noteLabel: "目标检查说明(评估器原文)", + codeLabel: "停止代码", + assumptionTooltip: + "智能体补全了目标里没写明的内容,并在回复里做了说明,请核对上面的回复。", + announce: "{status}:{detail}", + editLocked: "设置了目标时不能编辑,先运行 /goal clear 清除目标。", + busy: "这个对话还有运行未结束,结束后再试。", + next: { + limit: + "已达到自动续跑上限 {count}/{max},不会再自动续跑。回复即可继续;或用 /goal <完成条件> 重新设置目标,次数从零开始(同时会开始一次新的运行)。", + autoOff: "目标检查:尚未达成。这个目标没有开启自动续跑,回复即可继续。", + noProgress: + "连续几轮没有进展。回复说明还缺什么,或用 /goal 换个说法重新设置目标。", + tokenCapped: + "已达到 token 预算。发送消息即可继续,新的运行会重新计算预算。", + missingEvidence: + "目标检查:缺少依据。可以让它展示结果或说明卡在哪里;如果目标已不适用,用 /goal clear 清除目标。", + runFailed: + "目标检查:这次运行没有完成任务。先看看回复里的错误,再发送消息重试。", + needsInputCard: "回答上面的问题即可继续。", + needsInputReply: "回复补充缺少的信息即可继续。", + external: "目标检查:正在等待外部条件,就绪后发送消息即可继续。", + unchecked: "{reason}。下次发消息后会重新检查。", + unknown: "发送消息即可继续。", + paused: + "已自动续跑 {count}/{max} 次,运行在确认达成前结束了。发送消息即可重新检查。", + }, + uncheckedReasons: { + evaluatorFailed: "这次运行本身可能没问题", + noDurableEndOfTurn: "没有保存最终回复", + threadChanged: "检查期间对话发生了变化", + }, + }, compactSuccess: "已压缩早期上下文。完整聊天记录仍保留,后续模型将基于摘要和最近消息继续。", compactSkipped: "当前上下文还不需要压缩。", diff --git a/frontend/src/core/scheduled-tasks/goal-outcome.ts b/frontend/src/core/scheduled-tasks/goal-outcome.ts index ed4c418100..e148d4a50e 100644 --- a/frontend/src/core/scheduled-tasks/goal-outcome.ts +++ b/frontend/src/core/scheduled-tasks/goal-outcome.ts @@ -50,7 +50,11 @@ export const CHECK_FAILURE_CODES: readonly string[] = [ "thread_changed_before_continuation", ]; -function reasonKeyOf(code: string): GoalReasonKey | null { +/** + * The label key of a host reason code, with or without the `blocked:` + * prefix; null for a code this build does not know. + */ +export function goalReasonKeyOf(code: string): GoalReasonKey | null { return REASON_KEYS[code.replace(/^blocked:/, "")] ?? null; } @@ -66,9 +70,13 @@ export function describeGoalOutcome( if (run.status === "unmet") { const code = run.error ?? null; if (code && CHECK_FAILURE_CODES.includes(code)) { - return { kind: "unchecked", code, reasonKey: reasonKeyOf(code) }; + return { kind: "unchecked", code, reasonKey: goalReasonKeyOf(code) }; } - return { kind: "unmet", code, reasonKey: code ? reasonKeyOf(code) : null }; + return { + kind: "unmet", + code, + reasonKey: code ? goalReasonKeyOf(code) : null, + }; } if (run.status === "success" && run.goal_verdict?.satisfied === true) { return { @@ -111,7 +119,7 @@ export function describeTaskLastError( if (lastError === AUTO_PAUSE_LAST_ERROR) { return { kind: "autoPause" }; } - const reasonKey = reasonKeyOf(lastError); + const reasonKey = goalReasonKeyOf(lastError); return reasonKey ? { kind: "goalUnmet", reasonKey } : null; } @@ -207,7 +215,7 @@ export function describeTaskOutcome( ); return { kind: "autoPaused", - latestReasonKey: latest?.error ? reasonKeyOf(latest.error) : null, + latestReasonKey: latest?.error ? goalReasonKeyOf(latest.error) : null, latestThreadId: latest?.thread_id ?? null, latestSummary: latest?.summary ?? null, }; diff --git a/frontend/src/core/threads/stream-state.ts b/frontend/src/core/threads/stream-state.ts index 6062292d09..48a3f19e84 100644 --- a/frontend/src/core/threads/stream-state.ts +++ b/frontend/src/core/threads/stream-state.ts @@ -6,13 +6,19 @@ function isRecord(value: unknown): value is Record { return typeof value === "object" && value !== null && !Array.isArray(value); } -function isGoalState(value: unknown): value is GoalState { +/** + * An active goal as the backend judges one (`status` "active"), so a + * malformed or inactive `goal` in history, worker frames or a status read + * never renders, and the bar and the edit lock agree with the server. POST + * /state and run input can store one without `created_at`/`updated_at`. + * `goal_outcome` is not folded here: only the worker's clear writes it, + * never through `updates`. + */ +export function isGoalState(value: unknown): value is GoalState { return ( isRecord(value) && typeof value.objective === "string" && - value.status === "active" && - typeof value.created_at === "string" && - typeof value.updated_at === "string" + value.status === "active" ); } diff --git a/frontend/src/core/threads/types.ts b/frontend/src/core/threads/types.ts index 507f4bb680..85c07caffd 100644 --- a/frontend/src/core/threads/types.ts +++ b/frontend/src/core/threads/types.ts @@ -31,6 +31,26 @@ export interface GoalState { }; } +/** + * The record of the latest met goal (the `goal_outcome` channel). Only the + * checkpoint write that clears a satisfied goal writes it, and every other + * goal write removes it; `contracts/thread_goal_contract.json` pins the keys. + * Named apart from the scheduled-run `GoalOutcome` in `core/scheduled-tasks`. + */ +export interface ThreadGoalOutcome { + status: "achieved"; + objective: string; + /** The met goal's `created_at`; empty for a goal stored without one. */ + goal_created_at: string; + achieved_at: string; + continuation_count: number; + max_continuations: number; + reason: string; + relied_on_assumption: boolean; + /** The reply the verdict judged: the latest visible AI message with text. */ + reply_message_id: string | null; +} + export interface ArtifactEntry { handle: string; tool_name: string; @@ -50,6 +70,7 @@ export interface AgentThreadState extends Record { artifacts?: string[]; todos?: Todo[]; goal?: GoalState | null; + goal_outcome?: ThreadGoalOutcome | null; tool_artifacts?: ArtifactEntry[]; } diff --git a/frontend/tests/e2e/goal-status.spec.ts b/frontend/tests/e2e/goal-status.spec.ts new file mode 100644 index 0000000000..718769bf3a --- /dev/null +++ b/frontend/tests/e2e/goal-status.spec.ts @@ -0,0 +1,666 @@ +import { createServer } from "node:http"; +import type { AddressInfo } from "node:net"; + +import { expect, test, type Page, type Route } from "@playwright/test"; + +import type { GoalState, ThreadGoalOutcome } from "@/core/threads/types"; + +import { + MOCK_RUN_ID, + mockGoal, + mockGoalOutcome, + mockLangGraphAPI, + runStreamThreadId, + visibleRunInputMessages, + type MockAPIOptions, + type MockThread, +} from "./utils/mock-api"; +import { expectNoRawIdentifiers } from "./utils/readable"; + +/** + * A Gateway-like run stream. The response names the created run + * (`Content-Location`, which moves a new chat to its own URL), and each + * `values` frame is written `gapMs` after the previous one, so the goal bar + * renders every state a live goal run passes through. As elsewhere in these + * mocks, messages ride the `values` frames. The Gateway streams a chat run's + * messages as message events and sends `values` only for goal writes, so a + * frame that adds messages and leaves the goal as it was stands in for those + * events. `saved` runs before the last frame: by then the backend has stored + * the run's final state for the post-run history refetch. + */ +async function startRunStream(gapMs: number) { + let run: { + threadId: string; + steps: Record[]; + saved: () => void; + } = { threadId: "", steps: [], saved: () => undefined }; + const server = createServer((_request, response) => { + const { threadId, steps, saved } = run; + response.writeHead(200, { + "Access-Control-Allow-Origin": "*", + "Access-Control-Expose-Headers": "Content-Location", + "Cache-Control": "no-cache", + "Content-Location": `/threads/${threadId}/runs/${MOCK_RUN_ID}`, + "Content-Type": "text/event-stream", + }); + const frame = (event: string, data: unknown) => + `event: ${event}\ndata: ${JSON.stringify(data)}\n\n`; + response.write( + frame("metadata", { run_id: MOCK_RUN_ID, thread_id: threadId }), + ); + const timers = steps.map((values, index) => + setTimeout(() => { + if (index < steps.length - 1) { + response.write(frame("values", values)); + return; + } + saved(); + response.end(frame("values", values) + frame("end", {})); + }, index * gapMs), + ); + response.once("close", () => timers.forEach(clearTimeout)); + }); + await new Promise((resolve, reject) => { + server.once("error", reject); + server.listen(0, "127.0.0.1", () => { + server.off("error", reject); + resolve(); + }); + }); + const { port } = server.address() as AddressInfo; + return { + /** Answer the run `route` starts with these `values` frames. */ + serve(route: Route, steps: Record[], saved: () => void) { + run = { threadId: runStreamThreadId(route), steps, saved }; + return route.continue({ url: `http://127.0.0.1:${port}/runs/stream` }); + }, + async close() { + server.closeAllConnections(); + await new Promise((resolve, reject) => { + server.close((error) => (error ? reject(error) : resolve())); + }); + }, + }; +} + +// Both chat pages wire the goal bar and the edit lock themselves, so every +// case runs on each of them. +const CHAT_PAGES: { + name: string; + path: string; + options: MockAPIOptions; + agentName?: string; +}[] = [ + { name: "chat", path: "/workspace/chats", options: {} }, + { + name: "agent chat", + path: "/workspace/agents/test-agent/chats", + options: { + agents: [ + { + name: "test-agent", + description: "A test agent for E2E tests", + system_prompt: "You are a test agent.", + }, + ], + }, + agentName: "test-agent", + }, +]; + +const THREAD_ID = "00000000-0000-0000-0000-000000000601"; +const BUSY = "A run is still going in this chat. Try again when it finishes."; + +function goalBar(page: Page) { + return page.getByRole("region", { name: "Goal status" }); +} + +function composer(page: Page) { + return page.getByPlaceholder(/how can i assist you/i); +} + +/** A question card the agent left open (`ask_clarification`). */ +const OPEN_QUESTION = [ + { + type: "ai", + id: "goal-ask", + content: "", + tool_calls: [ + { + id: "call-room", + name: "ask_clarification", + args: { + question: "Which time works for you?", + clarification_type: "missing_info", + options: ["10:00", "14:00"], + }, + }, + ], + }, + { + type: "tool", + id: "goal-question", + name: "ask_clarification", + tool_call_id: "call-room", + content: "Which time works for you?", + artifact: { + human_input: { + version: 1, + kind: "human_input_request", + source: "ask_clarification", + request_id: "clarification:call-room", + tool_call_id: "call-room", + clarification_type: "missing_info", + question: "Which time works for you?", + input_mode: "choice_with_other", + options: [ + { id: "option-1", label: "10:00", value: "10:00" }, + { id: "option-2", label: "14:00", value: "14:00" }, + ], + }, + }, + }, +]; + +const STOOD_DOWN = [ + { + name: "a goal that used every automatic follow-up", + objective: "Make every test in the suite pass", + goal: { + continuation_count: 8, + last_evaluation: { + satisfied: false, + blocker: "goal_not_met_yet", + reason: "Two tests still fail.", + stand_down_reason: "max_continuations_reached", + }, + }, + reply: [ + { + type: "ai", + id: "goal-ai", + content: "Six of the eight tests pass now.", + }, + ], + chip: "Stopped", + detail: + "Continuation limit reached · 8/8. It won't auto-continue again. Reply to keep going, or set the goal again with /goal to start a fresh count (this also starts a new run).", + note: "Two tests still fail.", + }, + { + name: "a goal that needs details before any follow-up", + objective: "Book a meeting room for Friday", + goal: { + last_evaluation: { + satisfied: false, + blocker: "needs_user_input", + reason: "The user has to pick a time first.", + stand_down_reason: "blocked:needs_user_input", + }, + }, + reply: [ + { type: "ai", id: "goal-ai", content: "Which time works for you?" }, + ], + chip: "Waiting for you", + detail: "Reply with the missing details to continue.", + note: "The user has to pick a time first.", + }, + { + name: "a goal waiting on an open question", + objective: "Book a meeting room for Friday", + goal: { + // The host writes this reason itself; it is not the checker's note. + last_evaluation: { + satisfied: false, + blocker: "needs_user_input", + reason: + "The turn ended on a question to the user that has not been answered.", + stand_down_reason: "blocked:needs_user_input", + }, + }, + reply: OPEN_QUESTION, + chip: "Waiting for you", + detail: "Answer the question above to continue.", + note: null, + }, +] satisfies { + name: string; + objective: string; + goal: Partial; + reply: unknown[]; + chip: string; + detail: string; + note: string | null; +}[]; + +for (const chatPage of CHAT_PAGES) { + const threadUrl = `${chatPage.path}/${THREAD_ID}`; + + /** A chat with one finished turn, optionally under a stood-down goal. */ + function chatThread({ + objective = "Summarize the README", + goal, + reply = [ + { type: "ai", id: "goal-ai", content: "The README covers setup." }, + ], + }: { + objective?: string; + goal?: Partial; + reply?: unknown[]; + } = {}): MockThread { + return { + thread_id: THREAD_ID, + title: "Goal chat", + agent_name: chatPage.agentName, + messages: [ + { + type: "human", + id: "goal-human", + content: [{ type: "text", text: objective }], + }, + ...reply, + ], + ...(goal ? { goal: mockGoal(objective, goal) } : {}), + }; + } + + test.describe(`Goal bar on the ${chatPage.name} page`, () => { + for (const scenario of STOOD_DOWN) { + test(`explains ${scenario.name}, also after a reload`, async ({ + page, + }) => { + mockLangGraphAPI(page, { + ...chatPage.options, + threads: [chatThread(scenario)], + }); + await page.goto(threadUrl); + + const bar = goalBar(page); + for (const reload of [false, true]) { + if (reload) { + await page.reload(); + } + await expect(bar.getByTestId("goal-status-chip")).toHaveText( + scenario.chip, + { timeout: 15_000 }, + ); + await expect(bar.getByTestId("goal-status-detail")).toHaveText( + scenario.detail, + ); + await expect(bar).not.toContainText("Continuing"); + await expectNoRawIdentifiers(bar); + } + + await bar.getByRole("button", { name: "Details" }).click(); + const details = bar.getByTestId("goal-status-details"); + await expect(details).toContainText(scenario.objective); + if (scenario.note) { + await expect(bar.getByTestId("goal-status-note")).toHaveText( + scenario.note, + ); + } else { + await expect(details).not.toContainText("Goal check note"); + } + }); + } + + test("keeps the locked edit pencil visible to the keyboard, and hides it during an upload", async ({ + page, + }) => { + mockLangGraphAPI(page, { + ...chatPage.options, + threads: [chatThread(STOOD_DOWN[0])], + }); + await page.goto(threadUrl); + await expect(goalBar(page).getByTestId("goal-status-chip")).toHaveText( + "Stopped", + { timeout: 15_000 }, + ); + + // No pointer over the message: only keyboard focus can show its toolbar. + await page.mouse.move(0, 0); + const toolbar = page + .getByTestId("message-toolbar") + .filter({ has: page.getByTestId("message-edit-locked") }); + const pencil = toolbar.getByRole("button", { name: "Edit and rerun" }); + await toolbar.getByRole("button", { name: "Copy to clipboard" }).focus(); + await page.keyboard.press("Tab"); + + await expect(pencil).toBeFocused(); + await expect(pencil).toBeDisabled(); + await expect(toolbar).toHaveCSS("opacity", "1"); + await expect( + page.getByRole("tooltip").filter({ + hasText: + "Editing is off while a goal is set. Run /goal clear to edit.", + }), + ).toBeVisible(); + + await page.keyboard.press("Enter"); + await expect( + page.getByRole("button", { name: "Update and rerun" }), + ).toBeHidden(); + + // No turn can be edited while a file uploads, so no pencil shows, not + // even the goal's locked one. + let releaseUpload!: () => void; + const uploadHeld = new Promise((resolve) => { + releaseUpload = resolve; + }); + await page.route("**/api/threads/*/uploads", async (route) => { + await uploadHeld; + return route.fulfill({ + status: 500, + contentType: "application/json", + body: JSON.stringify({ detail: "Upload failed." }), + }); + }); + await page.getByLabel("Upload files").setInputFiles({ + name: "notes.txt", + mimeType: "text/plain", + buffer: Buffer.from("notes"), + }); + await composer(page).fill("Summarize these notes"); + await composer(page).press("Enter"); + await expect(page.getByTestId("message-edit-locked")).toHaveCount(0); + releaseUpload(); + await expect(page.getByTestId("message-edit-locked")).toHaveCount(1); + }); + + test("shows Goal met after one automatic follow-up until the chat moves on", async ({ + page, + }) => { + const objective = "Write a three-line summary of the README"; + const stream = await startRunStream(600); + let runs = 0; + const api = mockLangGraphAPI(page, { + ...chatPage.options, + threads: [chatThread()], + runStreamHandler: async (route) => { + runs += 1; + const thread = api.getThread(runStreamThreadId(route))!; + const turn = [ + ...(thread.messages ?? []), + ...visibleRunInputMessages(route), + ]; + + if (runs > 1) { + // A later turn: the backend keeps the record and adds the turn. + const messages = [ + ...turn, + { type: "ai", id: "goal-ai-3", content: "Glad it helps." }, + ]; + return stream.serve( + route, + [{ messages, goal_outcome: thread.goal_outcome }], + () => api.upsertThread({ ...thread, messages }), + ); + } + + const goal = thread.goal as GoalState; + const draft = { + type: "ai", + id: "goal-ai-1", + content: "Draft: the summary has two lines so far.", + }; + const continuation = { + type: "human", + id: "goal-continuation-1", + content: `\nActive goal: ${objective}\n`, + additional_kwargs: { + hide_from_ui: true, + deerflow_goal_continuation: true, + }, + }; + const reply = { + type: "ai", + id: "goal-ai-2", + content: "Done: the summary now has three lines.", + }; + const continuing: GoalState = { + ...goal, + continuation_count: 1, + last_evaluation: { + satisfied: false, + blocker: "goal_not_met_yet", + reason: "The summary has two lines, not three.", + run_id: MOCK_RUN_ID, + }, + }; + const record = mockGoalOutcome(continuing, reply.id, { + reason: "The summary has three lines.", + }); + const messages = [...turn, draft, continuation, reply]; + // The checkpoint has the submitted turn while the run goes on. + api.upsertThread({ ...thread, messages: turn }); + return stream.serve( + route, + [ + { messages: [...turn, draft], goal }, + { messages: [...turn, draft, continuation], goal: continuing }, + { messages, goal: continuing }, + // The clear of the met goal: the record, and no `goal` key. + { messages, goal_outcome: record }, + ], + () => + api.upsertThread({ + ...thread, + goal: null, + goal_outcome: record, + messages, + }), + ); + }, + }); + + try { + await page.goto(threadUrl); + const textarea = composer(page); + await expect(page.getByText("The README covers setup.")).toBeVisible({ + timeout: 15_000, + }); + await textarea.fill(`/goal ${objective}`); + await textarea.press("Enter"); + + const bar = goalBar(page); + await expect(bar).toContainText("In progress"); + await expect(bar).toContainText("Continuing 1/8"); + await expect(bar).toHaveAttribute("data-goal-state", "met"); + await expect(bar).toContainText(`Goal met${objective}`); + await expect(bar).toContainText("Auto-continued once"); + await expect(bar).not.toContainText("Continuing"); + await expect( + page.getByText("Done: the summary now has three lines."), + ).toBeVisible(); + // The goal is no longer active, so edit-and-rerun is back. + await expect( + page.getByRole("button", { name: "Edit and rerun" }), + ).toBeEnabled(); + await expect(page.getByTestId("message-edit-locked")).toHaveCount(0); + await expectNoRawIdentifiers(bar); + + await page.reload(); + await expect(bar).toContainText("Goal met", { timeout: 15_000 }); + await expect(bar).toContainText("Auto-continued once"); + + await textarea.fill("Thanks, that works."); + await textarea.press("Enter"); + await expect(page.getByText("Glad it helps.")).toBeVisible(); + await expect(bar).toBeHidden(); + await page.reload(); + await expect(page.getByText("Glad it helps.")).toBeVisible({ + timeout: 15_000, + }); + await expect(bar).toBeHidden(); + expect(runs).toBe(2); + } finally { + await stream.close(); + } + }); + + test("replaces the set goal with Goal met when the first run meets it, and Goal met with a new goal", async ({ + page, + }) => { + const objective = "Summarize the README in three lines"; + const next = "List the README's sections"; + const nextReply = "Sections: Setup, Usage, License."; + const stream = await startRunStream(600); + let runs = 0; + let lastRecord: ThreadGoalOutcome | undefined; + const api = mockLangGraphAPI(page, { + ...chatPage.options, + // The goal PUT and the run share the chat's own id, as on the Gateway. + honorRequestedThreadId: true, + createdThreadMessages: [], + // Each goal is met on its own first run. + runStreamHandler: async (route) => { + runs += 1; + const thread = api.getThread(runStreamThreadId(route))!; + const turn = [ + ...(thread.messages ?? []), + ...visibleRunInputMessages(route), + ]; + const reply = { + type: "ai", + id: `goal-ai-${runs}`, + content: + runs === 1 ? "README: setup, usage and license." : nextReply, + }; + const messages = [...turn, reply]; + const record = mockGoalOutcome(thread.goal as GoalState, reply.id); + const earlier = lastRecord; + lastRecord = record; + return stream.serve( + route, + [ + // The turn's message events: the tab keeps the values it had, + // an earlier goal's record included. + { messages: turn, ...(earlier ? { goal_outcome: earlier } : {}) }, + // The clear of the met goal: the record, and no `goal` key at all. + { messages, goal_outcome: record }, + ], + () => + api.upsertThread({ + ...thread, + goal: null, + goal_outcome: record, + messages, + }), + ); + }, + }); + + try { + await page.goto(`${chatPage.path}/new`); + const textarea = composer(page); + await expect(textarea).toBeVisible({ timeout: 15_000 }); + await textarea.fill(`/goal ${objective}`); + await textarea.press("Enter"); + + const bar = goalBar(page); + await expect(bar).toHaveAttribute("data-goal-state", "met"); + await expect(bar).toContainText(`Goal met${objective}`); + await expect(bar).not.toContainText("Auto-continued"); + await expect( + page.getByText("README: setup, usage and license."), + ).toBeVisible(); + await expect(page).not.toHaveURL(/\/new$/); + await expect( + page.getByRole("button", { name: "Edit and rerun" }), + ).toBeEnabled(); + + await page.reload(); + await expect(bar).toHaveAttribute("data-goal-state", "met", { + timeout: 15_000, + }); + await expect(bar).toContainText(`Goal met${objective}`); + + // A new goal replaces Goal met at once, and its own run meets it. + await textarea.fill(`/goal ${next}`); + await textarea.press("Enter"); + await expect(bar).toHaveAttribute("data-goal-state", "inProgress"); + await expect(bar).toContainText(next); + await expect(bar).toHaveAttribute("data-goal-state", "met"); + await expect(bar).toContainText(`Goal met${next}`); + await expect(bar).not.toContainText(objective); + + await textarea.fill("/goal clear"); + await textarea.press("Enter"); + await expect(page.getByText("Goal cleared.")).toBeVisible(); + await expect(bar).toBeHidden(); + await page.reload(); + await expect(page.getByText(nextReply)).toBeVisible({ + timeout: 15_000, + }); + await expect(bar).toBeHidden(); + expect(runs).toBe(2); + } finally { + await stream.close(); + } + }); + + test("keeps the /goal draft when a run is still going", async ({ + page, + }) => { + const objective = "Finish the quarterly report"; + const api = mockLangGraphAPI(page, { + ...chatPage.options, + threads: [chatThread()], + }); + let runStreams = 0; + page.on("request", (request) => { + if ( + request.method() === "POST" && + request.url().includes("/runs/stream") + ) { + runStreams += 1; + } + }); + await page.goto(threadUrl); + const textarea = composer(page); + const bar = goalBar(page); + await expect(page.getByText("The README covers setup.")).toBeVisible({ + timeout: 15_000, + }); + + // Setting the goal: the Gateway's 409 keeps the draft for a retry. + api.setGoalWritesBusy(true); + await textarea.fill(`/goal ${objective}`); + await textarea.press("Enter"); + await expect(page.getByText(BUSY)).toBeVisible(); + await expect(textarea).toHaveValue(`/goal ${objective}`); + await expect(bar).toBeHidden(); + expect(runStreams).toBe(0); + + api.setGoalWritesBusy(false); + await textarea.press("Enter"); + await expect(bar).toContainText(objective); + await expect(textarea).toHaveValue(""); + await expect(page.getByText("Hello from DeerFlow!")).toBeVisible(); + expect(runStreams).toBe(1); + + // Clearing it: the same. The set half's toast closes first, so the busy + // toast below can only be the clear's own. + await expect(page.getByText(BUSY)).toHaveCount(0, { timeout: 10_000 }); + api.setGoalWritesBusy(true); + await textarea.fill("/goal clear"); + const refused = page.waitForResponse( + (response) => + response.request().method() === "DELETE" && + response.url().endsWith("/goal"), + ); + await textarea.press("Enter"); + expect((await refused).status()).toBe(409); + await expect(page.getByText(BUSY)).toBeVisible(); + await expect(textarea).toHaveValue("/goal clear"); + await expect(bar).toContainText(objective); + + api.setGoalWritesBusy(false); + await textarea.press("Enter"); + await expect(page.getByText("Goal cleared.")).toBeVisible(); + await expect(bar).toBeHidden(); + await expect(textarea).toHaveValue(""); + expect(runStreams).toBe(1); + }); + }); +} diff --git a/frontend/tests/e2e/utils/mock-api.ts b/frontend/tests/e2e/utils/mock-api.ts index 2e5cd7e607..34332b0579 100644 --- a/frontend/tests/e2e/utils/mock-api.ts +++ b/frontend/tests/e2e/utils/mock-api.ts @@ -11,6 +11,8 @@ import path from "node:path"; import type { Page, Route } from "@playwright/test"; +import type { GoalState, ThreadGoalOutcome } from "@/core/threads/types"; + // --------------------------------------------------------------------------- // Constants — deterministic IDs used across tests // --------------------------------------------------------------------------- @@ -28,6 +30,19 @@ export const THREAD_PINNED_METADATA_KEY = "deerflow_pinned"; // membership. export const THREAD_PROJECT_METADATA_KEY = "deerflow_project_id"; +/** + * The keys the Gateway's POST /history head may carry, from the goal contract + * the backend router test also reads (`contracts/thread_goal_contract.json`). + */ +const HISTORY_HEAD_KEYS: readonly string[] = ( + JSON.parse( + readFileSync( + path.resolve(process.cwd(), "../contracts/thread_goal_contract.json"), + "utf8", + ), + ) as { history_head_keys: string[] } +).history_head_keys; + const MOCK_AUTH_USER = { id: "default", email: "default@test.local", @@ -47,7 +62,12 @@ export type MockThread = { metadata?: Record; messages?: unknown[]; artifacts?: string[]; - goal?: Record | null; + goal?: GoalState | Record | null; + /** + * The record of a met goal (`mockGoalOutcome`). Runs keep it, like the + * backend; PUT and DELETE /goal drop it. + */ + goal_outcome?: ThreadGoalOutcome | null; /** Thread search `unread` (a server-originated run changed since it was opened); default false. */ unread?: boolean; }; @@ -199,6 +219,13 @@ export type MockThreadActivityEntry = { export type MockAPIController = { /** Add or replace a thread in the thread search results. */ upsertThread: (thread: MockThread) => void; + /** A thread as the mock now stores it, e.g. the goal a PUT /goal saved. */ + getThread: (threadId: string) => MockThread | undefined; + /** + * While on, PUT and DELETE /goal return the Gateway's 409 for a thread + * with a run in flight. + */ + setGoalWritesBusy: (busy: boolean) => void; /** Record run changes; the next activity poll after the seed returns them. */ pushThreadActivity: (...entries: MockThreadActivityEntry[]) => void; /** Simulate a read on another device (raises `read_version`). */ @@ -399,7 +426,8 @@ function mockMessageRunId(message: unknown, fallback: string) { return fallback; } -function visibleRunInputMessages(route: Route) { +/** The visible messages a run stream request submits. */ +export function visibleRunInputMessages(route: Route) { try { const body = route.request().postDataJSON() as { input?: { messages?: unknown[] }; @@ -457,7 +485,8 @@ function mockStreamMessages( ]; } -function runStreamThreadId(route: Route) { +/** The thread a run stream request targets (path first, then the body). */ +export function runStreamThreadId(route: Route) { const pathThreadId = /\/threads\/([^/]+)\/runs\/stream/.exec( new URL(route.request().url()).pathname, )?.[1]; @@ -484,6 +513,56 @@ function runStreamThreadId(route: Route) { } } +/** The backend's `is_active_goal`: only an active goal is projected or met. */ +function isActiveGoal(goal: MockThread["goal"]): goal is GoalState { + return goal?.status === "active"; +} + +/** An active goal as PUT /goal stores it; `overrides` sets counts or a check. */ +export function mockGoal( + objective: string, + overrides: Partial = {}, +): GoalState { + const now = new Date().toISOString(); + return { + objective, + status: "active", + created_at: now, + updated_at: now, + continuation_count: 0, + max_continuations: 8, + no_progress_count: 0, + max_no_progress_continuations: 2, + ...overrides, + }; +} + +/** + * The record the backend writes when it clears a met goal + * (runtime/goal.py `build_goal_outcome`), anchored to `replyMessageId`. + */ +export function mockGoalOutcome( + goal: Pick< + GoalState, + "objective" | "created_at" | "continuation_count" | "max_continuations" + >, + replyMessageId: string | null, + overrides: Partial = {}, +): ThreadGoalOutcome { + return { + status: "achieved", + objective: goal.objective, + goal_created_at: goal.created_at, + achieved_at: new Date().toISOString(), + continuation_count: goal.continuation_count, + max_continuations: goal.max_continuations, + reason: "The reply meets the goal.", + relied_on_assumption: false, + reply_message_id: replyMessageId, + ...overrides, + }; +} + // --------------------------------------------------------------------------- // mockLangGraphAPI // --------------------------------------------------------------------------- @@ -588,6 +667,7 @@ export function mockLangGraphAPI( let activitySeq = 100; let readVersion = 0; const readRequests: string[] = []; + let goalWritesBusy = false; const scheduledTaskEvents: Record = { ...(options?.scheduledTaskEvents ?? {}), }; @@ -599,6 +679,45 @@ export function mockLangGraphAPI( ]; }; + // The materialized thread state. Like a checkpoint, it has no `goal` key + // before a goal is set or after a clear, and `goal_outcome` only once met. + const threadStateValues = (thread: MockThread): Record => ({ + title: thread.title ?? "Untitled", + messages: thread.messages ?? [ + { + type: "human", + id: `msg-human-${thread.thread_id}`, + content: [{ type: "text", text: "Previous question" }], + }, + { + type: "ai", + id: `msg-ai-${thread.thread_id}`, + content: `Response in thread ${thread.title ?? thread.thread_id}`, + }, + ], + artifacts: thread.artifacts ?? [], + ...(thread.goal ? { goal: thread.goal } : {}), + ...(thread.goal_outcome ? { goal_outcome: thread.goal_outcome } : {}), + }); + + // The POST /history head: only the contract's `history_head_keys`, with + // `goal` only while active and `goal_outcome` only when achieved (threads.py + // get_thread_history). `artifacts` is mock-only: the real head lacks it, + // but the artifact specs preload their artifacts through it. + const historyHeadValues = (thread: MockThread) => { + const state = threadStateValues(thread); + const projected = HISTORY_HEAD_KEYS.filter( + (key) => + key in state && + (key !== "goal" || isActiveGoal(thread.goal)) && + (key !== "goal_outcome" || thread.goal_outcome?.status === "achieved"), + ); + return { + ...Object.fromEntries(projected.map((key) => [key, state[key]])), + artifacts: state.artifacts, + }; + }; + const threadSearchResult = (thread: MockThread) => ({ thread_id: thread.thread_id, created_at: "2025-01-01T00:00:00Z", @@ -1211,6 +1330,25 @@ export function mockLangGraphAPI( ? (route.request().postDataJSON() as { thread_id?: string } | null) ?.thread_id : undefined) ?? MOCK_THREAD_ID; + // Like the Gateway, a repeat create of a requested id returns the + // thread as it is, e.g. with the goal a /goal command saved first. + const existing = options?.honorRequestedThreadId + ? threads.find((thread) => thread.thread_id === threadId) + : undefined; + if (existing) { + return route.fulfill({ + status: 200, + contentType: "application/json", + body: JSON.stringify({ + thread_id: threadId, + created_at: existing.updated_at ?? new Date().toISOString(), + updated_at: existing.updated_at ?? new Date().toISOString(), + metadata: existing.metadata ?? {}, + status: "idle", + values: {}, + }), + }); + } upsertThread({ thread_id: threadId, title: "New Chat", @@ -1899,9 +2037,27 @@ export function mockLangGraphAPI( }); } + if ( + goalWritesBusy && + (route.request().method() === "PUT" || + route.request().method() === "DELETE") + ) { + const action = route.request().method() === "PUT" ? "Set" : "Clear"; + return route.fulfill({ + status: 409, + contentType: "application/json", + body: JSON.stringify({ + detail: `Thread has a run in flight. ${action} the goal after the run finishes.`, + }), + }); + } + + // Every goal write also drops the met record (runtime/goal.py + // write_thread_goal), so an older "Goal met" cannot come back. if (route.request().method() === "DELETE") { if (matchingThread) { matchingThread.goal = null; + matchingThread.goal_outcome = null; } return route.fulfill({ status: 200, @@ -1914,22 +2070,13 @@ export function mockLangGraphAPI( const payload = route.request().postDataJSON() as { objective?: string; }; - const goal = { - objective: payload.objective ?? "", - status: "active", - created_at: new Date().toISOString(), - updated_at: new Date().toISOString(), - continuation_count: 0, - max_continuations: 8, - no_progress_count: 0, - max_no_progress_continuations: 2, - }; + const goal = mockGoal(payload.objective ?? ""); matchingThread ??= { thread_id: threadId, title: "New Chat", updated_at: new Date().toISOString(), }; - upsertThread({ ...matchingThread, goal }); + upsertThread({ ...matchingThread, goal, goal_outcome: null }); return route.fulfill({ status: 200, contentType: "application/json", @@ -2028,23 +2175,7 @@ export function mockLangGraphAPI( contentType: "application/json", body: JSON.stringify([ { - values: { - title: matchingThread.title ?? "Untitled", - goal: matchingThread.goal ?? null, - messages: matchingThread.messages ?? [ - { - type: "human", - id: `msg-human-${matchingThread.thread_id}`, - content: [{ type: "text", text: "Previous question" }], - }, - { - type: "ai", - id: `msg-ai-${matchingThread.thread_id}`, - content: `Response in thread ${matchingThread.title ?? matchingThread.thread_id}`, - }, - ], - artifacts: matchingThread.artifacts ?? [], - }, + values: historyHeadValues(matchingThread), next: [], metadata: {}, created_at: "2025-01-01T00:00:00Z", @@ -2075,25 +2206,9 @@ export function mockLangGraphAPI( status: 200, contentType: "application/json", body: JSON.stringify({ - values: { - title: matchingThread?.title ?? "Untitled", - goal: matchingThread?.goal ?? null, - messages: matchingThread - ? (matchingThread.messages ?? [ - { - type: "human", - id: `msg-human-${matchingThread.thread_id}`, - content: [{ type: "text", text: "Previous question" }], - }, - { - type: "ai", - id: `msg-ai-${matchingThread.thread_id}`, - content: `Response in thread ${matchingThread.title ?? matchingThread.thread_id}`, - }, - ]) - : [], - artifacts: matchingThread?.artifacts ?? [], - }, + values: matchingThread + ? threadStateValues(matchingThread) + : { title: "Untitled", messages: [], artifacts: [] }, next: [], metadata: {}, created_at: "2025-01-01T00:00:00Z", @@ -2200,15 +2315,21 @@ export function mockLangGraphAPI( ); const fallbackGoal = threads.find((thread) => thread.goal)?.goal ?? null; const goal = existingThread?.goal ?? fallbackGoal; + const goalOutcome = existingThread?.goal_outcome ?? null; upsertThread({ thread_id: threadId, title: threadId === MOCK_SIDECAR_THREAD_ID ? "Side chat" : "New Chat", updated_at: new Date().toISOString(), goal, + goal_outcome: goalOutcome, metadata: existingThread?.metadata, messages: mockStreamMessages(route), }); - return handleRunStream(route, { goal }); + // Values frames carry every set channel and no key for an unset one. + return handleRunStream(route, { + ...(goal ? { goal } : {}), + ...(goalOutcome ? { goal_outcome: goalOutcome } : {}), + }); }); void page.route("**/api/langgraph/runs/stream", handleMockRunStream); @@ -2562,6 +2683,11 @@ export function mockLangGraphAPI( return { upsertThread, + getThread: (threadId) => + threads.find((thread) => thread.thread_id === threadId), + setGoalWritesBusy: (busy) => { + goalWritesBusy = busy; + }, pushThreadActivity: (...entries) => { for (const entry of entries) { activitySeq += 1; diff --git a/frontend/tests/e2e/utils/raw-identifiers.ts b/frontend/tests/e2e/utils/raw-identifiers.ts index acacbade50..92eb602cb6 100644 --- a/frontend/tests/e2e/utils/raw-identifiers.ts +++ b/frontend/tests/e2e/utils/raw-identifiers.ts @@ -1,8 +1,10 @@ /** - * Text that must never appear in a default view of scheduled tasks: UUIDs, - * task/run ids, ISO timestamps, five-field cron strings and internal enum or - * field names. Pure (no test-runner imports) so the Playwright helper - * (`readable.ts`) and the unit helper (`tests/unit/helpers/readable.ts`) + * Text that must never appear in a default view of scheduled tasks or the + * chat goal bar: UUIDs, task/run ids, ISO timestamps, five-field cron + * strings, internal enum or field names, goal stand-down codes + * (`contracts/thread_goal_contract.json`), checker blockers and the met + * record's field names. Pure (no test-runner imports) so the Playwright + * helper (`readable.ts`) and the unit helper (`tests/unit/helpers/readable.ts`) * share one list. */ export const RAW_IDENTIFIER_PATTERNS: readonly RegExp[] = [ @@ -11,6 +13,8 @@ export const RAW_IDENTIFIER_PATTERNS: readonly RegExp[] = [ /\d{4}-\d{2}-\d{2}T\d{2}:\d{2}/, /(^|\s)[\d*/,-]+(\s[\d*/,-]+){4}(\s|$)/, /\b(fresh_thread_per_run|reuse_thread|lead_agent|goal_objective|max_runs|end_at|unmet|stop_scheduled_task|schedule_task)\b/, + /\b(blocked:[a-z_]+|max_continuations_reached|no_progress_detected|token_capped|evaluator_failed|no_durable_end_of_turn|thread_changed_(after_evaluation|before_continuation)|goal_outcome|stand_down_reason|last_evaluation)\b/, + /\b(goal_not_met_yet|needs_user_input|missing_evidence|external_wait|run_failed|goal_created_at|reply_message_id|relied_on_assumption|continuation_count|max_continuations)\b/, ]; /** The patterns `text` matches, with the matched text, for a readable failure message. */ diff --git a/frontend/tests/unit/components/workspace/goal-status-helpers.test.ts b/frontend/tests/unit/components/workspace/goal-status-helpers.test.ts index 4484c06454..c08d7393d0 100644 --- a/frontend/tests/unit/components/workspace/goal-status-helpers.test.ts +++ b/frontend/tests/unit/components/workspace/goal-status-helpers.test.ts @@ -1,10 +1,17 @@ +import type { Message } from "@langchain/langgraph-sdk"; import { describe, expect, it } from "@rstest/core"; import { - getGoalContinuationDisplay, + describeGoalBar, + GOAL_HOST_REASON_TEXTS, + goalBarAnnounceKey, + goalOutcomeKey, goalReconciliationKey, + isGoalOutcomeCurrent, + isThreadGoalOutcome, + sameGoalInstance, } from "@/components/workspace/goal-status-helpers"; -import type { GoalState } from "@/core/threads/types"; +import type { GoalState, ThreadGoalOutcome } from "@/core/threads/types"; function makeGoal(overrides: Partial = {}): GoalState { return { @@ -20,44 +27,409 @@ function makeGoal(overrides: Partial = {}): GoalState { }; } -describe("getGoalContinuationDisplay", () => { - it("hides the counter before the agent has auto-continued", () => { - expect( - getGoalContinuationDisplay({ - continuation_count: 0, - max_continuations: 8, - }), - ).toBeNull(); +function stoodDown( + code: string, + overrides: Partial = {}, + reason = "The report has no totals yet.", +): GoalState { + return makeGoal({ + updated_at: "2026-01-01T00:05:00Z", + ...overrides, + last_evaluation: { + satisfied: false, + blocker: "goal_not_met_yet", + reason, + stand_down_reason: code, + }, + }); +} + +function makeOutcome( + overrides: Partial = {}, +): ThreadGoalOutcome { + return { + status: "achieved", + objective: "ship it", + goal_created_at: "2026-01-01T00:00:00Z", + achieved_at: "2026-01-01T00:09:00Z", + continuation_count: 1, + max_continuations: 8, + reason: "The report lists every total.", + relied_on_assumption: false, + reply_message_id: "ai-2", + ...overrides, + }; +} + +const idle = (goal: GoalState | null, hasOpenHumanInputCard = false) => + describeGoalBar({ + goal, + outcome: null, + isRunning: false, + hasOpenHumanInputCard, }); - it("shows count and max once continuation has started", () => { - expect( - getGoalContinuationDisplay({ - continuation_count: 1, - max_continuations: 8, - }), - ).toEqual({ count: 1, max: 8 }); - expect( - getGoalContinuationDisplay({ - continuation_count: 8, - max_continuations: 8, - }), - ).toEqual({ count: 8, max: 8 }); +describe("describeGoalBar", () => { + it("shows a fresh goal as set while idle and in progress while running", () => { + const goal = makeGoal(); + expect(idle(goal)).toEqual({ kind: "set", goal }); + expect(describeGoalBar({ goal, outcome: null, isRunning: true })).toEqual({ + kind: "inProgress", + goal, + }); }); - it("treats missing or negative counts as hidden", () => { + it("counts automatic follow-ups while one is running", () => { + const goal = makeGoal({ + continuation_count: 1, + last_evaluation: { + satisfied: false, + blocker: "goal_not_met_yet", + reason: "Totals missing.", + }, + }); + expect(describeGoalBar({ goal, outcome: null, isRunning: true })).toEqual({ + kind: "continuing", + goal, + count: 1, + max: 8, + }); + // A stored satisfied verdict is no reason to continue. + const satisfied = makeGoal({ + continuation_count: 1, + last_evaluation: { satisfied: true, blocker: "none", reason: "Done." }, + }); expect( - getGoalContinuationDisplay({ - continuation_count: -1, - max_continuations: 8, - }), - ).toBeNull(); + describeGoalBar({ goal: satisfied, outcome: null, isRunning: true }), + ).toEqual({ kind: "inProgress", goal: satisfied }); + }); + + it("hides a reason left by an earlier run while a new run streams", () => { + const goal = stoodDown("max_continuations_reached", { + continuation_count: 8, + }); + expect(describeGoalBar({ goal, outcome: null, isRunning: true })).toEqual({ + kind: "inProgress", + goal, + }); + }); + + it("reads 8/8 with a stand-down as stopped at the limit, never Continuing", () => { expect( - getGoalContinuationDisplay({ - continuation_count: undefined as unknown as number, - max_continuations: undefined as unknown as number, + idle(stoodDown("max_continuations_reached", { continuation_count: 8 })), + ).toMatchObject({ + kind: "stopped", + stop: "limit", + chip: "stopped", + reasonKey: "maxContinuations", + count: 8, + max: 8, + note: "The report has no totals yet.", + rawCode: null, + }); + }); + + it.each([ + [ + "max_continuations_reached", + { max_continuations: 0 }, + "autoOff", + "stopped", + ], + ["no_progress_detected", {}, "noProgress", "stopped"], + ["token_capped", {}, "tokenCapped", "stopped"], + ["blocked:missing_evidence", {}, "missingEvidence", "stopped"], + ["blocked:run_failed", {}, "runFailed", "stopped"], + ["blocked:external_wait", {}, "external", "waiting"], + ] as const)("maps %s to its stop row", (code, overrides, stop, chip) => { + expect(idle(stoodDown(code, overrides))).toMatchObject({ + kind: "stopped", + stop, + chip, + rawCode: null, + }); + }); + + it("tells a count-0 stand-down apart from a fresh goal", () => { + const fresh = idle(makeGoal()); + const stopped = idle(stoodDown("blocked:missing_evidence")); + expect(fresh.kind).toBe("set"); + expect(stopped).toMatchObject({ kind: "stopped", count: 0 }); + }); + + it("asks for the open question only when a card is open", () => { + const goal = stoodDown( + "blocked:needs_user_input", + {}, + "The turn ended on a question to the user that has not been answered.", + ); + expect(idle(goal, true)).toMatchObject({ + stop: "needsInputCard", + chip: "waitingForYou", + // Host-written, so never shown as the checker's note. + note: null, + }); + expect( + idle(stoodDown("blocked:needs_user_input", {}, "It guessed the year.")), + ).toMatchObject({ + stop: "needsInputReply", + chip: "waitingForYou", + note: "It guessed the year.", + }); + }); + + it.each([ + "evaluator_failed", + "no_durable_end_of_turn", + "thread_changed_after_evaluation", + "thread_changed_before_continuation", + ])("shows %s as unchecked with no evaluator note", (code) => { + expect(idle(stoodDown(code))).toMatchObject({ + kind: "stopped", + stop: "unchecked", + chip: "unchecked", + note: null, + rawCode: null, + }); + }); + + it("keeps a stored satisfied verdict unchecked when the chat changed", () => { + const goal = makeGoal({ + last_evaluation: { + satisfied: true, + blocker: "none", + reason: "Done.", + stand_down_reason: "thread_changed_after_evaluation", + }, + }); + expect(idle(goal)).toMatchObject({ stop: "unchecked", note: null }); + }); + + it("keeps an unknown code only for Details", () => { + expect(idle(stoodDown("blocked:new_reason"))).toMatchObject({ + stop: "unknown", + chip: "stopped", + reasonKey: null, + rawCode: "blocked:new_reason", + }); + // A known label key that is not a chat stand-down is still unknown. + expect(idle(stoodDown("blocked:goal_not_met_yet"))).toMatchObject({ + stop: "unknown", + rawCode: "blocked:goal_not_met_yet", + }); + }); + + it("shows a run that ended after an automatic follow-up as paused", () => { + const goal = makeGoal({ + continuation_count: 2, + last_evaluation: { + satisfied: false, + blocker: "goal_not_met_yet", + reason: "Totals missing.", + }, + }); + // No checker note: a skipped clear leaves the earlier unmet verdict. + expect(idle(goal)).toEqual({ kind: "paused", goal, count: 2, max: 8 }); + expect(idle({ ...goal, continuation_count: 0 }).kind).toBe("set"); + }); + + it("shows a current record as met when no goal is active", () => { + const outcome = makeOutcome({ continuation_count: 3 }); + expect(describeGoalBar({ goal: null, outcome, isRunning: false })).toEqual({ + kind: "met", + outcome, + count: 3, + max: 8, + reliedOnAssumption: false, + note: "The report lists every total.", + }); + expect( + describeGoalBar({ + goal: null, + outcome: makeOutcome({ relied_on_assumption: true }), + isRunning: true, }), - ).toBeNull(); + ).toMatchObject({ kind: "met", reliedOnAssumption: true }); + }); + + it("lets an active goal win over a record", () => { + const goal = makeGoal({ objective: "next" }); + expect( + describeGoalBar({ goal, outcome: makeOutcome(), isRunning: false }), + ).toEqual({ kind: "set", goal }); + }); + + it("renders nothing without a goal or a record", () => { + expect(idle(null)).toEqual({ kind: "none" }); + }); + + it("never treats host-written reasons as the checker's note", () => { + for (const reason of GOAL_HOST_REASON_TEXTS) { + expect( + idle(stoodDown("blocked:missing_evidence", {}, reason)), + ).toMatchObject({ note: null }); + } + }); +}); + +describe("goalBarAnnounceKey", () => { + it("stays put while the counter rises and changes with the state", () => { + const running = (count: number) => + describeGoalBar({ + goal: makeGoal({ continuation_count: count }), + outcome: null, + isRunning: true, + }); + expect(goalBarAnnounceKey(running(1))).toBe(goalBarAnnounceKey(running(2))); + expect(goalBarAnnounceKey(running(0))).toBe(""); + expect(goalBarAnnounceKey(idle(makeGoal()))).toBe(""); + expect(goalBarAnnounceKey(idle(stoodDown("token_capped")))).not.toBe( + goalBarAnnounceKey(idle(stoodDown("no_progress_detected"))), + ); + }); + + it("changes for the same state of another goal", () => { + const key = (overrides: Partial) => + goalBarAnnounceKey(idle(stoodDown("token_capped", overrides))); + expect(key({})).toBe(key({ updated_at: "2026-01-01T00:07:00Z" })); + expect(key({})).not.toBe(key({ created_at: "2026-01-02T00:00:00Z" })); + expect(key({})).not.toBe(key({ objective: "other" })); + + const paused = (overrides: Partial) => + goalBarAnnounceKey( + idle( + makeGoal({ + continuation_count: 2, + last_evaluation: { + satisfied: false, + blocker: "goal_not_met_yet", + reason: "Totals missing.", + }, + ...overrides, + }), + ), + ); + expect(paused({})).toBe(paused({ continuation_count: 3 })); + expect(paused({})).not.toBe(paused({ objective: "other" })); + expect(paused({})).not.toBe(paused({ created_at: "2026-01-02T00:00:00Z" })); + + const met = (outcome: ThreadGoalOutcome) => + goalBarAnnounceKey( + describeGoalBar({ goal: null, outcome, isRunning: false }), + ); + expect(met(makeOutcome())).toBe(met(makeOutcome())); + expect(met(makeOutcome())).not.toBe( + met(makeOutcome({ achieved_at: "2026-01-01T00:10:00Z" })), + ); + }); +}); + +function human(id: string, extra: Partial = {}): Message { + return { id, type: "human", content: "next please", ...extra } as Message; +} + +function ai(id: string, content: Message["content"], extra = {}): Message { + return { id, type: "ai", content, ...extra } as Message; +} + +describe("isGoalOutcomeCurrent", () => { + const anchor = ai("ai-2", "Here is the report."); + const outcome = makeOutcome({ reply_message_id: "ai-2" }); + + it.each([ + ["the anchor is the last message", [human("h-1"), anchor], true], + ["a visible human message follows", [anchor, human("h-2")], false], + ["a visible AI reply follows", [anchor, ai("ai-3", "More.")], false], + [ + "only a hidden human message follows", + [anchor, human("h-2", { additional_kwargs: { hide_from_ui: true } })], + true, + ], + [ + "an AI message with tool calls and no text follows", + [ + anchor, + ai("ai-3", "", { + tool_calls: [{ id: "call-1", name: "present_files", args: {} }], + }), + { + id: "tool-1", + type: "tool", + content: "ok", + tool_call_id: "call-1", + } as Message, + ], + true, + ], + [ + "an empty final AI message follows", + [anchor, ai("ai-3", [{ type: "text", text: " " }])], + true, + ], + ["the anchor is gone (regenerated)", [human("h-1")], false], + [ + "the anchor is not among the loaded messages", + [human("h-0", { additional_kwargs: { hide_from_ui: true } })], + false, + ], + ] as const)("%s", (_case, messages, current) => { + expect(isGoalOutcomeCurrent(outcome, messages)).toBe(current); + }); + + it("is never current without an anchor", () => { + expect( + isGoalOutcomeCurrent(makeOutcome({ reply_message_id: null }), [anchor]), + ).toBe(false); + }); +}); + +describe("goal outcome identity", () => { + it("accepts only achieved records with their identity fields", () => { + expect(isThreadGoalOutcome(makeOutcome())).toBe(true); + expect(isThreadGoalOutcome({ ...makeOutcome(), goal_created_at: "" })).toBe( + true, + ); + expect(isThreadGoalOutcome({ ...makeOutcome(), status: "cleared" })).toBe( + false, + ); + expect( + isThreadGoalOutcome({ ...makeOutcome(), objective: undefined }), + ).toBe(false); + expect( + isThreadGoalOutcome({ ...makeOutcome(), goal_created_at: undefined }), + ).toBe(false); + expect(isThreadGoalOutcome(null)).toBe(false); + expect(isThreadGoalOutcome([makeOutcome()])).toBe(false); + }); + + it("matches a record to its own goal, not a later one with the same text", () => { + const goal = makeGoal(); + expect(sameGoalInstance(goal, makeOutcome())).toBe(true); + expect( + sameGoalInstance( + makeGoal({ created_at: "2026-01-02T00:00:00Z" }), + makeOutcome(), + ), + ).toBe(false); + expect( + sameGoalInstance(makeGoal({ objective: "other" }), makeOutcome()), + ).toBe(false); + }); + + it("matches a goal stored without created_at to its '' record", () => { + // A status read keeps such a goal; the backend records it with "". + const goal = { objective: "ship it" } as GoalState; + expect(sameGoalInstance(goal, makeOutcome({ goal_created_at: "" }))).toBe( + true, + ); + expect(sameGoalInstance(goal, makeOutcome())).toBe(false); + }); + + it("keys a later record of the same goal differently", () => { + expect(goalOutcomeKey(null)).toBe("none"); + expect(goalOutcomeKey(makeOutcome())).not.toBe( + goalOutcomeKey(makeOutcome({ achieved_at: "2026-01-01T00:10:00Z" })), + ); }); }); diff --git a/frontend/tests/unit/components/workspace/goal-status-next-steps.test.ts b/frontend/tests/unit/components/workspace/goal-status-next-steps.test.ts new file mode 100644 index 0000000000..56e73e9825 --- /dev/null +++ b/frontend/tests/unit/components/workspace/goal-status-next-steps.test.ts @@ -0,0 +1,74 @@ +import { describe, expect, it } from "@rstest/core"; + +import { + describeGoalBar, + describeGoalBarCopy, +} from "@/components/workspace/goal-status-helpers"; +import { enUS } from "@/core/i18n/locales/en-US"; +import type { Translations } from "@/core/i18n/locales/types"; +import { zhCN } from "@/core/i18n/locales/zh-CN"; + +const LOCALES = [ + { + locale: "en-US", + t: enUS, + askForResult: "Ask it to show the result", + replyWithRest: "what's left", + noLongerApplies: "if the goal no longer applies", + }, + { + locale: "zh-CN", + t: zhCN, + askForResult: "让它展示结果", + replyWithRest: "还差什么", + noLongerApplies: "如果目标已不适用", + }, +] as const; + +/** The goal bar's second line for a goal the host stood down with `code`. */ +function detailOf(code: string, t: Translations) { + const view = describeGoalBar({ + goal: { + objective: "ship it", + status: "active", + created_at: "2026-01-01T00:00:00Z", + updated_at: "2026-01-01T00:05:00Z", + continuation_count: 0, + max_continuations: 8, + no_progress_count: 0, + max_no_progress_continuations: 2, + last_evaluation: { + satisfied: false, + blocker: "goal_not_met_yet", + reason: "The report has no totals yet.", + stand_down_reason: code, + }, + }, + outcome: null, + isRunning: false, + hasOpenHumanInputCard: false, + }); + return describeGoalBarCopy(view, t)?.detail ?? ""; +} + +describe.each(LOCALES)( + "goal bar next steps in $locale", + ({ t, askForResult, replyWithRest, noLongerApplies }) => { + it("asks the agent for missing evidence, not the user for the rest", () => { + const detail = detailOf("blocked:missing_evidence", t); + expect(detail).toContain(askForResult); + expect(detail).not.toContain(replyWithRest); + expect(detail).toContain("/goal clear"); + }); + + it("offers /goal clear only for a goal that no longer applies", () => { + // Clearing drops the met record, so work that is right would never + // show "Goal met". + for (const text of Object.values(t.inputBox.goalBar.next)) { + if (text.includes("/goal clear")) { + expect(text).toContain(noLongerApplies); + } + } + }); + }, +); diff --git a/frontend/tests/unit/components/workspace/goal-status-unchecked.test.ts b/frontend/tests/unit/components/workspace/goal-status-unchecked.test.ts new file mode 100644 index 0000000000..60da10971a --- /dev/null +++ b/frontend/tests/unit/components/workspace/goal-status-unchecked.test.ts @@ -0,0 +1,63 @@ +import { describe, expect, it } from "@rstest/core"; + +import { + describeGoalBar, + describeGoalBarCopy, +} from "@/components/workspace/goal-status-helpers"; +import { enUS } from "@/core/i18n/locales/en-US"; +import type { Translations } from "@/core/i18n/locales/types"; +import { zhCN } from "@/core/i18n/locales/zh-CN"; +import { + CHECK_FAILURE_CODES, + goalReasonKeyOf, +} from "@/core/scheduled-tasks/goal-outcome"; + +/** The goal bar copy after a goal check that could not run (`code`). */ +function uncheckedCopy(code: string, t: Translations) { + const view = describeGoalBar({ + goal: { + objective: "ship it", + status: "active", + created_at: "2026-01-01T00:00:00Z", + updated_at: "2026-01-01T00:05:00Z", + continuation_count: 0, + max_continuations: 8, + no_progress_count: 0, + max_no_progress_continuations: 2, + last_evaluation: { + satisfied: false, + blocker: "goal_not_met_yet", + reason: "The report has no totals yet.", + stand_down_reason: code, + }, + }, + outcome: null, + isRunning: false, + }); + return describeGoalBarCopy(view, t)!; +} + +describe.each([ + ["en-US", enUS], + ["zh-CN", zhCN], +] as const)("the goal bar after a failed check in %s", (_locale, t) => { + const label = t.scheduledTasks.goal.unchecked; + + it.each(CHECK_FAILURE_CODES)("says the label once for %s", (code) => { + const copy = uncheckedCopy(code, t); + expect(copy.chip).toBe(label); + expect(copy.detail).not.toContain(label); + expect(copy.announcement.split(label)).toHaveLength(2); + }); + + it.each(CHECK_FAILURE_CODES)("keeps why the check failed for %s", (code) => { + // The scheduled page's reason is the label, then why; the bar keeps why. + const scheduled = t.scheduledTasks.goal.reasons[goalReasonKeyOf(code)!]; + expect(scheduled.startsWith(label)).toBe(true); + const why = scheduled.slice(label.length).replace(/^[;:,:]\s*/, ""); + expect(why).not.toBe(""); + expect(uncheckedCopy(code, t).detail?.toLowerCase()).toContain( + why.toLowerCase(), + ); + }); +}); diff --git a/frontend/tests/unit/components/workspace/goal-status.dom.test.tsx b/frontend/tests/unit/components/workspace/goal-status.dom.test.tsx new file mode 100644 index 0000000000..c506188c04 --- /dev/null +++ b/frontend/tests/unit/components/workspace/goal-status.dom.test.tsx @@ -0,0 +1,620 @@ +import { afterEach, describe, expect, it } from "@rstest/core"; +import { cleanup, fireEvent, render, screen } from "@testing-library/react"; +import type { ComponentProps } from "react"; + +import { GoalStatus } from "@/components/workspace/goal-status"; +import { I18nContext } from "@/core/i18n/context"; +import type { Locale } from "@/core/i18n/locale"; +import { enUS } from "@/core/i18n/locales/en-US"; +import { zhCN } from "@/core/i18n/locales/zh-CN"; +import type { GoalState, ThreadGoalOutcome } from "@/core/threads/types"; + +import { expectNoRawIdentifiers } from "../../helpers/readable"; + +afterEach(cleanup); + +const OBJECTIVE = "Write a three-line summary of README.md"; + +function makeGoal(overrides: Partial = {}): GoalState { + return { + objective: OBJECTIVE, + status: "active", + created_at: "2026-10-07T00:00:00Z", + updated_at: "2026-10-07T00:00:00Z", + continuation_count: 0, + max_continuations: 8, + no_progress_count: 0, + max_no_progress_continuations: 2, + ...overrides, + }; +} + +function stoodDown( + code: string, + overrides: Partial = {}, + reason = "The summary has only two lines.", +): GoalState { + return makeGoal({ + updated_at: "2026-10-07T00:05:00Z", + ...overrides, + last_evaluation: { + satisfied: false, + blocker: "goal_not_met_yet", + reason, + run_id: "run-1", + evaluated_at: "2026-10-07T00:05:00Z", + stand_down_reason: code, + }, + }); +} + +function makeOutcome( + overrides: Partial = {}, +): ThreadGoalOutcome { + return { + status: "achieved", + objective: OBJECTIVE, + goal_created_at: "2026-10-07T00:00:00Z", + achieved_at: "2026-10-07T00:09:00Z", + continuation_count: 1, + max_continuations: 8, + reason: "The summary has three lines.", + relied_on_assumption: false, + reply_message_id: "ai-2", + ...overrides, + }; +} + +type Props = ComponentProps; + +const TRANSLATIONS = { "en-US": enUS, "zh-CN": zhCN } as const; + +function bar(props: Props, locale: Locale = "en-US") { + return ( + undefined, t: TRANSLATIONS[locale] }} + > + + + ); +} + +function section() { + return screen.getByTestId("goal-status"); +} + +function liveRegion() { + return section().querySelector('[role="status"]')!; +} + +/** The row text a sighted user reads, without the screen-reader-only parts. */ +function visibleRowText() { + const row = section().firstElementChild!; + return [ + row.textContent, + screen.queryByTestId("goal-status-detail")?.textContent, + ] + .filter(Boolean) + .join(" | "); +} + +type Case = { + name: string; + props: Props; + en: string; + zh: string; +}; + +const CASES: Case[] = [ + { + name: "set", + props: { goal: makeGoal() }, + en: `Goal${OBJECTIVE}`, + zh: `目标${OBJECTIVE}`, + }, + { + name: "in progress", + props: { goal: makeGoal(), isRunning: true }, + en: `Goal${OBJECTIVE}In progress`, + zh: `目标${OBJECTIVE}进行中`, + }, + { + name: "continuing", + props: { + goal: makeGoal({ + continuation_count: 1, + last_evaluation: { + satisfied: false, + blocker: "goal_not_met_yet", + reason: "Two lines so far.", + }, + }), + isRunning: true, + }, + en: `Goal${OBJECTIVE}Continuing 1/8`, + zh: `目标${OBJECTIVE}续跑中 1/8`, + }, + { + name: "stopped at the limit", + props: { + goal: stoodDown("max_continuations_reached", { continuation_count: 8 }), + }, + en: `Goal${OBJECTIVE}StoppedDetails | Continuation limit reached · 8/8. It won't auto-continue again. Reply to keep going, or set the goal again with /goal to start a fresh count (this also starts a new run).`, + zh: `目标${OBJECTIVE}已停止详情 | 已达到自动续跑上限 8/8,不会再自动续跑。回复即可继续;或用 /goal <完成条件> 重新设置目标,次数从零开始(同时会开始一次新的运行)。`, + }, + { + name: "stopped with auto-continue off", + props: { + goal: stoodDown("max_continuations_reached", { max_continuations: 0 }), + }, + en: `Goal${OBJECTIVE}StoppedDetails | Goal check: not met yet. Auto-continue is off for this goal. Reply to keep going.`, + zh: `目标${OBJECTIVE}已停止详情 | 目标检查:尚未达成。这个目标没有开启自动续跑,回复即可继续。`, + }, + { + name: "stopped without progress", + props: { goal: stoodDown("no_progress_detected") }, + en: `Goal${OBJECTIVE}StoppedDetails | No progress between turns. Reply with what's missing, or rephrase the goal with /goal.`, + zh: `目标${OBJECTIVE}已停止详情 | 连续几轮没有进展。回复说明还缺什么,或用 /goal 换个说法重新设置目标。`, + }, + { + name: "stopped at the token budget", + props: { goal: stoodDown("token_capped") }, + en: `Goal${OBJECTIVE}StoppedDetails | Token budget reached. Send a message to continue; a new run starts with a fresh budget.`, + zh: `目标${OBJECTIVE}已停止详情 | 已达到 token 预算。发送消息即可继续,新的运行会重新计算预算。`, + }, + { + name: "stopped without evidence (count 0)", + props: { goal: stoodDown("blocked:missing_evidence") }, + en: `Goal${OBJECTIVE}StoppedDetails | Goal check: evidence missing. Ask it to show the result or explain what's missing; if the goal no longer applies, run /goal clear.`, + zh: `目标${OBJECTIVE}已停止详情 | 目标检查:缺少依据。可以让它展示结果或说明卡在哪里;如果目标已不适用,用 /goal clear 清除目标。`, + }, + { + name: "stopped after a failed run", + props: { goal: stoodDown("blocked:run_failed") }, + en: `Goal${OBJECTIVE}StoppedDetails | Goal check: the run did not finish the work. Check the reply for errors, then send a message to retry.`, + zh: `目标${OBJECTIVE}已停止详情 | 目标检查:这次运行没有完成任务。先看看回复里的错误,再发送消息重试。`, + }, + { + name: "waiting for an answer to the open card", + props: { + goal: stoodDown( + "blocked:needs_user_input", + {}, + "The turn ended on a question to the user that has not been answered.", + ), + hasOpenHumanInputCard: true, + }, + en: `Goal${OBJECTIVE}Waiting for youDetails | Answer the question above to continue.`, + zh: `目标${OBJECTIVE}等你回复详情 | 回答上面的问题即可继续。`, + }, + { + name: "waiting for missing details", + props: { goal: stoodDown("blocked:needs_user_input") }, + en: `Goal${OBJECTIVE}Waiting for youDetails | Reply with the missing details to continue.`, + zh: `目标${OBJECTIVE}等你回复详情 | 回复补充缺少的信息即可继续。`, + }, + { + name: "waiting on something external", + props: { goal: stoodDown("blocked:external_wait") }, + en: `Goal${OBJECTIVE}WaitingDetails | Goal check: waiting on something external. Send a message when it's ready.`, + zh: `目标${OBJECTIVE}等待中详情 | 目标检查:正在等待外部条件,就绪后发送消息即可继续。`, + }, + { + name: "unchecked", + props: { goal: stoodDown("evaluator_failed") }, + en: `Goal${OBJECTIVE}Couldn't check the goalDetails | The run itself may be fine. It's checked again after your next message.`, + zh: `目标${OBJECTIVE}未能检查目标详情 | 这次运行本身可能没问题。下次发消息后会重新检查。`, + }, + { + name: "stopped for an unknown reason", + props: { goal: stoodDown("blocked:new_reason") }, + en: `Goal${OBJECTIVE}StoppedDetails | Send a message to keep going.`, + zh: `目标${OBJECTIVE}已停止详情 | 发送消息即可继续。`, + }, + { + name: "paused", + props: { + goal: makeGoal({ + continuation_count: 2, + last_evaluation: { + satisfied: false, + blocker: "goal_not_met_yet", + reason: "Two lines so far.", + }, + }), + }, + en: `Goal${OBJECTIVE}PausedDetails | Auto-continued 2/8. The run ended before the goal was confirmed. Send a message to check again.`, + zh: `目标${OBJECTIVE}已暂停详情 | 已自动续跑 2/8 次,运行在确认达成前结束了。发送消息即可重新检查。`, + }, + { + name: "met after one follow-up", + props: { goal: null, outcome: makeOutcome() }, + en: `Goal met${OBJECTIVE}Auto-continued onceDetails`, + zh: `目标已达成${OBJECTIVE}自动续跑了 1 次详情`, + }, + { + name: "met after several follow-ups", + props: { goal: null, outcome: makeOutcome({ continuation_count: 3 }) }, + en: `Goal met${OBJECTIVE}Auto-continued 3 timesDetails`, + zh: `目标已达成${OBJECTIVE}自动续跑了 3 次详情`, + }, + { + name: "met on the first run", + props: { goal: null, outcome: makeOutcome({ continuation_count: 0 }) }, + en: `Goal met${OBJECTIVE}Details`, + zh: `目标已达成${OBJECTIVE}详情`, + }, + { + name: "met with an assumption", + props: { + goal: null, + outcome: makeOutcome({ + continuation_count: 0, + relied_on_assumption: true, + }), + }, + en: `Goal met${OBJECTIVE}Assumption madeDetails`, + zh: `目标已达成${OBJECTIVE}含假设详情`, + }, +]; + +describe.each(["en-US", "zh-CN"] as const)("GoalStatus in %s", (locale) => { + it.each(CASES)("$name", ({ props, en, zh }) => { + render(bar(props, locale)); + expect(visibleRowText()).toBe(locale === "en-US" ? en : zh); + expectNoRawIdentifiers(section()); + }); + + it("renders nothing without a goal or a record", () => { + const { container } = render(bar({ goal: null }, locale)); + expect(container.textContent).toBe(""); + }); +}); + +describe("GoalStatus details", () => { + it("toggles the panel with its full objective and the checker's note", () => { + render( + bar({ + goal: stoodDown("max_continuations_reached", { continuation_count: 8 }), + }), + ); + const toggle = screen.getByRole("button", { name: /Details/ }); + const panel = document.getElementById( + toggle.getAttribute("aria-controls")!, + )!; + expect(toggle.getAttribute("aria-expanded")).toBe("false"); + expect(panel.hidden).toBe(true); + + fireEvent.click(toggle); + expect(toggle.getAttribute("aria-expanded")).toBe("true"); + expect(panel.hidden).toBe(false); + expect(panel.textContent).toContain(`Goal${OBJECTIVE}`); + expect(panel.textContent).toContain("Goal check note (from the checker)"); + const note = screen.getByTestId("goal-status-note"); + expect(note.textContent).toBe("The summary has only two lines."); + expect(note.getAttribute("dir")).toBe("auto"); + expect(toggle.textContent).toBe("Hide details"); + }); + + it("labels the note as the checker's own words in zh-CN", () => { + render(bar({ goal: null, outcome: makeOutcome() }, "zh-CN")); + const toggle = screen.getByRole("button", { name: "详情" }); + fireEvent.click(toggle); + expect(screen.getByTestId("goal-status-details").textContent).toContain( + "目标检查说明(评估器原文)The summary has three lines.", + ); + expect(toggle.textContent).toBe("收起详情"); + }); + + it("shows no checker note for a paused goal", () => { + // A skipped clear leaves the earlier unmet verdict in last_evaluation. + render( + bar({ + goal: makeGoal({ + continuation_count: 2, + last_evaluation: { + satisfied: false, + blocker: "goal_not_met_yet", + reason: "Totals missing.", + }, + }), + }), + ); + fireEvent.click(screen.getByRole("button", { name: /Details/ })); + const panel = screen.getByTestId("goal-status-details"); + expect(panel.textContent).toContain(OBJECTIVE); + expect(panel.textContent).not.toContain("Totals missing."); + expect(screen.queryByTestId("goal-status-note")).toBeNull(); + }); + + it("starts each new state, or another goal, with Details collapsed", () => { + const limit = stoodDown("max_continuations_reached", { + continuation_count: 8, + }); + const { rerender } = render(bar({ goal: limit })); + const toggle = () => screen.getByRole("button", { name: /details/i }); + const expanded = () => toggle().getAttribute("aria-expanded"); + fireEvent.click(toggle()); + expect(expanded()).toBe("true"); + + // A language switch keeps it open; a counter-free re-render too. + rerender(bar({ goal: { ...limit } }, "zh-CN")); + expect(screen.getByRole("button", { name: "收起详情" })).toBeTruthy(); + rerender(bar({ goal: limit })); + + rerender( + bar({ goal: makeGoal({ continuation_count: 8 }), isRunning: true }), + ); + rerender(bar({ goal: null, outcome: makeOutcome() })); + expect(expanded()).toBe("false"); + expect(screen.getByTestId("goal-status-details").hidden).toBe(true); + expect(screen.queryByTestId("goal-status-note")).toBeNull(); + + fireEvent.click(toggle()); + rerender(bar({ goal: limit })); + fireEvent.click(toggle()); + expect(expanded()).toBe("true"); + // The same state of another goal, as after a thread switch. + rerender( + bar({ + goal: stoodDown("max_continuations_reached", { + created_at: "2026-10-08T00:00:00Z", + continuation_count: 8, + }), + }), + ); + expect(expanded()).toBe("false"); + }); + + it.each([ + ["a check failure", stoodDown("thread_changed_after_evaluation"), false], + [ + "a host-written reason", + stoodDown( + "blocked:needs_user_input", + {}, + "The turn ended on a question to the user that has not been answered.", + ), + true, + ], + ])("shows no checker note for %s", (_case, goal, hasOpenHumanInputCard) => { + render(bar({ goal, hasOpenHumanInputCard }, "zh-CN")); + fireEvent.click(screen.getByRole("button", { name: /详情/ })); + const panel = screen.getByTestId("goal-status-details"); + expect(panel.textContent).toContain(OBJECTIVE); + expect(panel.textContent).not.toContain("目标检查说明"); + expect(panel.textContent).not.toContain(goal.last_evaluation!.reason); + expect(screen.queryByTestId("goal-status-note")).toBeNull(); + }); + + it("keeps an unknown code out of the default view and shows it in Details", () => { + render(bar({ goal: stoodDown("blocked:new_reason") })); + expect(section().textContent).not.toContain("new_reason"); + fireEvent.click(screen.getByRole("button", { name: /Details/ })); + expect(screen.getByTestId("goal-status-details").textContent).toContain( + "blocked:new_reason", + ); + }); + + it("describes the assumption badge and repeats it in Details", () => { + render( + bar({ + goal: null, + outcome: makeOutcome({ relied_on_assumption: true }), + }), + ); + const badge = screen.getByTestId("goal-status-assumption"); + expect(badge.tabIndex).toBe(0); + const description = document.getElementById( + badge.getAttribute("aria-describedby")!, + ); + expect(description?.textContent).toBe( + enUS.inputBox.goalBar.assumptionTooltip, + ); + fireEvent.click(screen.getByRole("button", { name: /Details/ })); + expect(screen.getByTestId("goal-status-details").textContent).toContain( + "The agent filled in something the goal didn't specify", + ); + }); + + it("has no Details for a goal that is only set or running", () => { + render(bar({ goal: makeGoal(), isRunning: true })); + expect(screen.queryByRole("button")).toBeNull(); + }); +}); + +describe("GoalStatus a11y", () => { + it("is a labelled region with the met icon state in text", () => { + render(bar({ goal: null, outcome: makeOutcome() }, "zh-CN")); + expect(screen.getByRole("region", { name: "目标状态" })).toBe(section()); + expect(section().getAttribute("data-goal-state")).toBe("met"); + cleanup(); + render(bar({ goal: makeGoal() })); + expect(screen.getByRole("region", { name: "Goal status" })).toBe(section()); + }); + + it("announces a state change once, not each streamed counter", () => { + const running = (count: number) => + bar({ + goal: makeGoal({ continuation_count: count }), + isRunning: true, + }); + const { rerender } = render(running(0)); + expect(liveRegion().textContent).toBe(""); + + rerender(running(1)); + expect(liveRegion().textContent).toBe("Goal: Continuing 1/8"); + rerender(running(2)); + expect(screen.getByText("Continuing 2/8")).toBeTruthy(); + expect(liveRegion().textContent).toBe("Goal: Continuing 1/8"); + + rerender( + bar({ goal: null, outcome: makeOutcome({ continuation_count: 2 }) }), + ); + expect(liveRegion().textContent).toBe(`Goal met: ${OBJECTIVE}`); + }); + + it("announces the stop reason with its next step", () => { + const { rerender } = render(bar({ goal: makeGoal(), isRunning: true })); + rerender(bar({ goal: stoodDown("token_capped") })); + expect(liveRegion().textContent).toBe( + "Stopped: Token budget reached. Send a message to continue; a new run starts with a fresh budget.", + ); + }); + + it("re-reads the live text in the new language", () => { + const limit = stoodDown("max_continuations_reached", { + continuation_count: 8, + }); + const { rerender } = render(bar({ goal: limit })); + expect(liveRegion().textContent).toContain( + "Stopped: Continuation limit reached · 8/8.", + ); + rerender(bar({ goal: limit }, "zh-CN")); + expect(liveRegion().textContent).toBe( + "已停止:已达到自动续跑上限 8/8,不会再自动续跑。回复即可继续;或用 /goal <完成条件> 重新设置目标,次数从零开始(同时会开始一次新的运行)。", + ); + }); + + it("announces the same state of another goal or record", () => { + const { rerender } = render( + bar({ + goal: stoodDown("max_continuations_reached", { continuation_count: 8 }), + }), + ); + rerender( + bar({ + goal: stoodDown("max_continuations_reached", { + objective: "Translate it", + created_at: "2026-10-08T00:00:00Z", + continuation_count: 3, + max_continuations: 3, + }), + }), + ); + expect(liveRegion().textContent).toContain( + "Continuation limit reached · 3/3.", + ); + + rerender(bar({ goal: null, outcome: makeOutcome() })); + expect(liveRegion().textContent).toBe(`Goal met: ${OBJECTIVE}`); + rerender( + bar({ + goal: null, + outcome: makeOutcome({ + objective: "Translate it", + goal_created_at: "2026-10-08T00:00:00Z", + achieved_at: "2026-10-08T00:09:00Z", + }), + }), + ); + expect(liveRegion().textContent).toBe("Goal met: Translate it"); + }); +}); + +describe("GoalStatus visuals", () => { + function chipClass() { + return screen.getByTestId("goal-status-chip").className; + } + + it.each([ + [ + "stopped", + stoodDown("max_continuations_reached", { continuation_count: 8 }), + "text-amber-700", + ], + [ + "stopped (unknown code)", + stoodDown("blocked:new_reason"), + "text-amber-700", + ], + ["waiting for you", stoodDown("blocked:needs_user_input"), "text-primary"], + ["waiting", stoodDown("blocked:external_wait"), "text-muted-foreground"], + ["unchecked", stoodDown("evaluator_failed"), "text-muted-foreground"], + ])("tones the %s chip", (_name, goal, tone) => { + render(bar({ goal })); + const classes = chipClass().split(" "); + expect(classes).toContain(tone); + for (const other of [ + "text-amber-700", + "text-primary", + "text-muted-foreground", + ]) { + if (other !== tone) { + expect(classes).not.toContain(other); + } + } + }); + + it.each([ + [ + "auto-continue off", + stoodDown("max_continuations_reached", { max_continuations: 0 }), + ], + ["no progress", stoodDown("no_progress_detected")], + ["missing evidence", stoodDown("blocked:missing_evidence")], + ])( + "marks the %s stop with a stop icon, not the pause icon", + (_name, goal) => { + render(bar({ goal })); + const chip = screen.getByTestId("goal-status-chip"); + expect(chip.querySelector(".lucide-circle-stop")).not.toBeNull(); + expect(chip.querySelector(".lucide-circle-pause")).toBeNull(); + }, + ); + + it("tones the paused chip muted", () => { + render( + bar({ + goal: makeGoal({ + continuation_count: 2, + last_evaluation: { + satisfied: false, + blocker: "goal_not_met_yet", + reason: "Two lines so far.", + }, + }), + }), + ); + expect(chipClass().split(" ")).toContain("text-muted-foreground"); + expect(chipClass()).not.toContain("amber"); + }); + + it("replaces the target icon with a green check when the goal is met", () => { + const { rerender } = render(bar({ goal: makeGoal() })); + const leading = () => section().firstElementChild!.firstElementChild!; + expect(leading().getAttribute("aria-hidden")).toBe("true"); + expect(leading().querySelector(".lucide-target")).not.toBeNull(); + expect(leading().className).toContain("text-primary"); + + rerender(bar({ goal: null, outcome: makeOutcome() })); + expect(leading().getAttribute("aria-hidden")).toBe("true"); + expect(leading().querySelector(".lucide-target")).toBeNull(); + expect(leading().querySelector(".lucide-circle-check")).not.toBeNull(); + expect(leading().className).toContain("text-emerald-600"); + // The color change follows the reduced-motion setting. + expect(leading().className.split(" ")).toContain( + "motion-safe:transition-colors", + ); + }); + + it("hides every icon from screen readers and spins only with motion allowed", () => { + const { rerender } = render(bar({ goal: makeGoal(), isRunning: true })); + const spinner = section().querySelector(".lucide-loader-circle")!; + const spinnerClasses = spinner.getAttribute("class")!.split(" "); + expect(spinnerClasses).toContain("motion-safe:animate-spin"); + expect(spinnerClasses).not.toContain("animate-spin"); + + for (const goal of [ + makeGoal({ continuation_count: 2 }), + stoodDown("max_continuations_reached", { continuation_count: 8 }), + ]) { + rerender(bar({ goal, isRunning: goal.continuation_count === 2 })); + const icons = section().querySelectorAll("svg"); + expect(icons.length).toBeGreaterThan(0); + for (const icon of icons) { + expect(icon.closest('[aria-hidden="true"]')).not.toBeNull(); + } + } + }); +}); diff --git a/frontend/tests/unit/components/workspace/goal-stream.dom.test.tsx b/frontend/tests/unit/components/workspace/goal-stream.dom.test.tsx new file mode 100644 index 0000000000..8326ce5c7e --- /dev/null +++ b/frontend/tests/unit/components/workspace/goal-stream.dom.test.tsx @@ -0,0 +1,410 @@ +// Drives the real LangGraph SDK useStream through the real useThreadStream, +// useActiveGoal and GoalStatus, with a fake client that sends the frames and +// history head the backend sends around a goal run. +import { readFileSync } from "node:fs"; +import { resolve } from "node:path"; + +import type { Message } from "@langchain/langgraph-sdk"; +import { afterEach, beforeEach, describe, expect, it, rs } from "@rstest/core"; +import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; +import { act, cleanup, render, screen, waitFor } from "@testing-library/react"; +import { useState, type ReactNode } from "react"; + +import { GoalStatus } from "@/components/workspace/goal-status"; +import { useActiveGoal } from "@/components/workspace/use-active-goal"; +import { I18nContext } from "@/core/i18n/context"; +import { enUS } from "@/core/i18n/locales/en-US"; +import { DEFAULT_LOCAL_SETTINGS } from "@/core/settings/local"; +import { useThreadStream } from "@/core/threads/hooks"; +import type { GoalState, ThreadGoalOutcome } from "@/core/threads/types"; + +type Frame = { event: string; data: unknown }; + +const HISTORY_HEAD_KEYS = ( + JSON.parse( + readFileSync( + resolve(__dirname, "../../../../../contracts/thread_goal_contract.json"), + "utf-8", + ), + ) as { history_head_keys: string[] } +).history_head_keys; + +/** The POST /history head: only the keys the backend projects onto it. */ +function historyHead(values: Record) { + return Object.fromEntries( + Object.entries(values).filter(([key]) => HISTORY_HEAD_KEYS.includes(key)), + ); +} + +const server = rs.hoisted(() => ({ + historyValues: {} as Record, + afterRunHistoryValues: null as Record | null, + frames: [] as { event: string; data: unknown }[], + streamCalls: 0, +})); + +rs.mock("@/core/api", () => ({ + getAPIClient: () => ({ + runs: { + cancel: async () => undefined, + joinStream: async function* () { + // No reconnect in these tests. + }, + list: async () => [], + stream: async function* ( + threadId: string, + _assistantId: string, + payload?: { + onRunCreated?: (meta: { run_id: string; thread_id: string }) => void; + }, + ) { + server.streamCalls += 1; + payload?.onRunCreated?.({ run_id: "run-1", thread_id: threadId }); + for (const frame of server.frames) { + // Separate macrotasks, so SDK throttling cannot merge the frames. + await new Promise((resolve) => setTimeout(resolve, 15)); + yield frame; + } + if (server.afterRunHistoryValues) { + server.historyValues = server.afterRunHistoryValues; + } + }, + }, + threads: { + create: async ({ threadId }: { threadId?: string }) => ({ + thread_id: threadId ?? "thread-1", + }), + getHistory: async () => [ + { + values: server.historyValues, + next: [], + tasks: [], + metadata: {}, + created_at: "2026-10-07T00:00:00Z", + checkpoint: { + thread_id: "thread-1", + checkpoint_ns: "", + checkpoint_id: `ckpt-${server.streamCalls}`, + }, + parent_checkpoint: null, + }, + ], + getState: async () => ({ values: server.historyValues }), + }, + }), +})); + +const GOAL: GoalState = { + objective: "Write a three-line summary of README.md", + status: "active", + created_at: "2026-10-07T00:00:00Z", + updated_at: "2026-10-07T00:00:00Z", + continuation_count: 0, + max_continuations: 8, + no_progress_count: 0, + max_no_progress_continuations: 2, +}; + +const HUMAN = { type: "human", id: "h-1", content: GOAL.objective } as Message; +const AI = { type: "ai", id: "ai-1", content: "Done: summary." } as Message; + +const RECORD: ThreadGoalOutcome = { + status: "achieved", + objective: GOAL.objective, + goal_created_at: GOAL.created_at, + achieved_at: "2026-10-07T00:01:00Z", + continuation_count: 0, + max_continuations: 8, + reason: "The summary has three lines.", + relied_on_assumption: false, + reply_message_id: "ai-1", +}; + +function wrapper({ children }: { children: ReactNode }) { + const queryClient = new QueryClient({ + defaultOptions: { queries: { retry: false } }, + }); + return ( + + undefined, t: enUS }} + > + {children} + + + ); +} + +type Exposed = { + sendMessage: ReturnType["sendMessage"]; + setLocalGoal: ReturnType["setLocalGoal"]; + isLoading: boolean; + hasGoal: boolean; + hasGoalKey: boolean; +}; + +// Mirrors the chat pages: useThreadStream, then useActiveGoal on the thread's +// goal, record and messages, then the goal bar. +function Harness({ + startNew = false, + expose, +}: { + startNew?: boolean; + expose: (value: Exposed) => void; +}) { + const threadId = "thread-1"; + const [isNewThread, setIsNewThread] = useState(startNew); + const { thread, sendMessage } = useThreadStream({ + threadId: isNewThread ? undefined : threadId, + displayThreadId: threadId, + context: DEFAULT_LOCAL_SETTINGS.context, + onStart: () => setIsNewThread(false), + }); + const { activeGoal, hasGoal, goalOutcome, setLocalGoal } = useActiveGoal( + threadId, + thread.values.goal, + thread.values.goal_outcome, + thread.messages, + ); + expose({ + sendMessage, + setLocalGoal, + isLoading: thread.isLoading, + hasGoal, + hasGoalKey: Object.hasOwn(thread.values, "goal"), + }); + return ( + + ); +} + +beforeEach(() => { + window.sessionStorage.clear(); + server.streamCalls = 0; + server.frames = []; + server.afterRunHistoryValues = null; + server.historyValues = historyHead({ title: "Chat", messages: [] }); + rs.stubGlobal( + "fetch", + rs.fn( + async () => + new Response( + JSON.stringify({ data: [], has_more: false, next_before_seq: null }), + { status: 200, headers: { "Content-Type": "application/json" } }, + ), + ), + ); + rs.spyOn(console, "error").mockImplementation(() => undefined); + rs.spyOn(console, "warn").mockImplementation(() => undefined); +}); + +afterEach(() => { + cleanup(); + rs.unstubAllGlobals(); + rs.restoreAllMocks(); +}); + +const settle = () => + act(async () => { + await new Promise((resolve) => setTimeout(resolve, 30)); + }); + +function barState() { + return screen.queryByTestId("goal-status")?.getAttribute("data-goal-state"); +} + +async function mountHarness(startNew = false) { + let current!: Exposed; + render( + { + current = value; + }} + />, + { wrapper }, + ); + await settle(); + return () => current; +} + +/** `/goal `: the PUT result becomes the override, then the run starts. */ +async function setGoalAndRun(current: () => Exposed) { + act(() => current().setLocalGoal(GOAL, "set")); + expect(barState()).toBe("set"); + await act(async () => { + await current().sendMessage("thread-1", { + text: GOAL.objective, + files: [], + }); + }); + await waitFor(() => expect(server.streamCalls).toBe(1)); + await waitFor(() => expect(current().isLoading).toBe(false)); + await settle(); +} + +describe.each([false, true])("a goal run (new chat: %s)", (startNew) => { + it("met on the first run replaces the set bar with Goal met", async () => { + // The satisfied clear: the frame drops the goal key and adds the record. + server.frames = [ + { event: "metadata", data: { run_id: "run-1", thread_id: "thread-1" } }, + { event: "updates", data: { agent: { messages: [AI] } } }, + { + event: "values", + data: { title: "Chat", messages: [HUMAN, AI], goal_outcome: RECORD }, + }, + ] satisfies Frame[]; + server.afterRunHistoryValues = historyHead({ + title: "Chat", + messages: [HUMAN, AI], + goal_outcome: RECORD, + todos: [], + }); + + const current = await mountHarness(startNew); + await setGoalAndRun(current); + + expect(current().hasGoalKey).toBe(false); + expect(current().hasGoal).toBe(false); + expect(barState()).toBe("met"); + expect(screen.getByTestId("goal-status").textContent).toContain( + `Goal met${GOAL.objective}`, + ); + }); + + it("keeps a stand-down and the edit lock through the post-run refetch", async () => { + const stoodDown: GoalState = { + ...GOAL, + updated_at: "2026-10-07T00:01:00Z", + last_evaluation: { + satisfied: false, + blocker: "missing_evidence", + reason: "No summary was shown.", + run_id: "run-1", + stand_down_reason: "blocked:missing_evidence", + }, + }; + server.frames = [ + { event: "metadata", data: { run_id: "run-1", thread_id: "thread-1" } }, + { + event: "values", + data: { title: "Chat", messages: [HUMAN, AI], goal: stoodDown }, + }, + ] satisfies Frame[]; + server.afterRunHistoryValues = historyHead({ + title: "Chat", + messages: [HUMAN, AI], + goal: stoodDown, + }); + + const current = await mountHarness(startNew); + await setGoalAndRun(current); + + expect(current().hasGoal).toBe(true); + expect(barState()).toBe("stopped"); + expect(screen.getByTestId("goal-status-detail").textContent).toBe( + "Goal check: evidence missing. Ask it to show the result or explain what's missing; if the goal no longer applies, run /goal clear.", + ); + }); +}); + +describe("a met goal after a reload", () => { + it("shows Goal met from the history head, and a new /goal set hides it", async () => { + server.historyValues = historyHead({ + title: "Chat", + messages: [HUMAN, AI], + goal_outcome: { ...RECORD, continuation_count: 1 }, + }); + + const current = await mountHarness(); + await waitFor(() => expect(barState()).toBe("met")); + expect(screen.getByTestId("goal-status").textContent).toContain( + "Auto-continued once", + ); + expect(current().hasGoal).toBe(false); + + const next = { + ...GOAL, + objective: "Translate the summary", + created_at: "2026-10-07T00:05:00Z", + updated_at: "2026-10-07T00:05:00Z", + }; + act(() => current().setLocalGoal(next, "set")); + expect(barState()).toBe("set"); + expect(screen.getByTestId("goal-status").textContent).toBe( + `Goal${next.objective}`, + ); + expect(current().hasGoal).toBe(true); + }); +}); + +// POST /state and run input can store an active goal without timestamps. The +// backend still evaluates it and locks edit, so the bar and `hasGoal` (the +// chat pages' pencil lock) must show it too. +describe("an active goal stored without timestamps", () => { + const SPARSE = { objective: GOAL.objective, status: "active" }; + + it("shows from the history head", async () => { + server.historyValues = historyHead({ + title: "Chat", + messages: [HUMAN], + goal: SPARSE, + }); + + const current = await mountHarness(); + await waitFor(() => expect(current().hasGoal).toBe(true)); + expect(barState()).toBe("set"); + }); + + it("shows from a live update before the post-run refetch", async () => { + server.frames = [ + { event: "metadata", data: { run_id: "run-1", thread_id: "thread-1" } }, + { event: "updates", data: { goal_evaluator: { goal: SPARSE } } }, + { event: "updates", data: { agent: { messages: [AI] } } }, + ] satisfies Frame[]; + server.afterRunHistoryValues = historyHead({ + title: "Chat", + messages: [HUMAN, AI], + goal: SPARSE, + }); + const seen: { isLoading: boolean; hasGoal: boolean }[] = []; + let current!: Exposed; + render( + { + seen.push({ isLoading: value.isLoading, hasGoal: value.hasGoal }); + current = value; + }} + />, + { wrapper }, + ); + await settle(); + expect(current.hasGoal).toBe(false); + + await act(async () => { + await current.sendMessage("thread-1", { text: "go", files: [] }); + }); + await waitFor(() => expect(current.isLoading).toBe(false)); + await settle(); + + // The history head has no goal while the run streams. + expect(seen.some((s) => s.isLoading && s.hasGoal)).toBe(true); + expect(current.hasGoal).toBe(true); + }); + + it("still drops a malformed goal from the history head", async () => { + server.historyValues = historyHead({ + title: "Chat", + messages: [HUMAN], + goal: { objective: GOAL.objective }, + }); + + const current = await mountHarness(); + expect(current().hasGoal).toBe(false); + expect(barState()).toBeUndefined(); + }); +}); diff --git a/frontend/tests/unit/components/workspace/input-box-failed-upload.dom.test.tsx b/frontend/tests/unit/components/workspace/input-box-failed-upload.dom.test.tsx index aeb5a7e7d0..e1a2d1df92 100644 --- a/frontend/tests/unit/components/workspace/input-box-failed-upload.dom.test.tsx +++ b/frontend/tests/unit/components/workspace/input-box-failed-upload.dom.test.tsx @@ -7,6 +7,7 @@ import { screen, waitFor, } from "@testing-library/react"; +import { toast } from "sonner"; import { PromptInputProvider } from "@/components/ai-elements/prompt-input"; import { InputBox } from "@/components/workspace/input-box"; @@ -14,9 +15,14 @@ import { ThreadContext } from "@/components/workspace/messages/context"; import { AuthProvider } from "@/core/auth/AuthProvider"; import { DEFAULT_LOCALE } from "@/core/i18n"; import { I18nProvider } from "@/core/i18n/context"; +import { enUS } from "@/core/i18n/locales/en-US"; import { stageProjectAttachment } from "@/core/projects/composer-attach"; import type { AttachProjectDocumentResult } from "@/core/projects/types"; import { DEFAULT_LOCAL_SETTINGS } from "@/core/settings/local"; +import { + buildComposerDraftKey, + readComposerDraft, +} from "@/core/threads/composer-draft"; import { useThreadStream } from "@/core/threads/hooks"; import type { UploadResponse } from "@/core/uploads/api"; @@ -203,3 +209,80 @@ it("retries a failed upload with the staged project attachment still attached", "/mnt/user-data/uploads/notes.txt", ]); }); + +it("keeps the draft through a failed upload and a busy /goal, then runs the goal's objective once", async () => { + // A /goal the Gateway refuses with 409 never reaches sendMessage, so like + // a failed upload it must leave the composer and its stored draft alone. + const goalStatuses: number[] = []; + const baseFetch = globalThis.fetch; + rs.stubGlobal("fetch", async (input: unknown, init?: RequestInit) => { + if (String(input).endsWith("/api/threads/thread-1/goal")) { + const status = goalStatuses.length === 0 ? 409 : 200; + goalStatuses.push(status); + return status === 409 + ? json({ detail: "Thread has a run in flight." }, 409) + : json({ goal: { objective: "finish all tests", status: "active" } }); + } + return baseFetch(input as RequestInfo, init); + }); + const error = rs.spyOn(toast, "error"); + const draftKey = buildComposerDraftKey({ + userId: "user-1", + threadId: "thread-1", + }); + const storedDraft = () => + readComposerDraft(window.sessionStorage, draftKey)?.text; + const textbox = () => screen.getByRole("textbox"); + const { container } = renderChat(); + const submit = () => fireEvent.submit(container.querySelector("form")!); + + fireEvent.change(textbox(), { target: { value: "summarize it" } }); + fireEvent.change(document.querySelector('input[type="file"]')!, { + target: { + files: [new File(["notes"], "notes.txt", { type: "text/plain" })], + }, + }); + await screen.findByText("notes.txt"); + await waitFor(() => expect(storedDraft()).toBe("summarize it")); + submit(); + await waitFor(() => expect(uploadAttempts).toBe(1)); + await waitFor(() => expect(textbox().disabled).toBe(false)); + expect(textbox().value).toBe("summarize it"); + expect(storedDraft()).toBe("summarize it"); + + fireEvent.click(screen.getByRole("button", { name: "Remove attachment" })); + fireEvent.change(textbox(), { + target: { value: "/goal finish all tests" }, + }); + await waitFor(() => expect(storedDraft()).toBe("/goal finish all tests")); + submit(); + await waitFor(() => + expect(error).toHaveBeenCalledWith(enUS.inputBox.goalBar.busy), + ); + // Let PromptInput settle the rejected submit before reading the composer. + await new Promise((resolve) => setTimeout(resolve, 50)); + expect(textbox().value).toBe("/goal finish all tests"); + expect(storedDraft()).toBe("/goal finish all tests"); + expect(mocks.submit).not.toHaveBeenCalled(); + + submit(); + await waitFor(() => expect(mocks.submit).toHaveBeenCalledTimes(1)); + expect(goalStatuses).toEqual([409, 200]); + expect(uploadAttempts).toBe(1); + const [input] = mocks.submit.mock.calls[0] as unknown as [ + { + messages: Array<{ + content: unknown; + additional_kwargs?: { files?: unknown }; + }>; + }, + ]; + const sent = input.messages.at(-1); + expect(JSON.stringify(sent?.content)).toContain("finish all tests"); + expect(JSON.stringify(sent?.content)).not.toContain("/goal"); + expect(sent?.additional_kwargs?.files).toBeUndefined(); + await waitFor(() => expect(textbox().value).toBe("")); + // Past the draft-save delay: nothing writes the sent text back. + await new Promise((resolve) => setTimeout(resolve, 400)); + expect(storedDraft()).toBeUndefined(); +}); diff --git a/frontend/tests/unit/components/workspace/input-box-send-gating.dom.test.tsx b/frontend/tests/unit/components/workspace/input-box-send-gating.dom.test.tsx index 4361f038cf..c47d5fed9c 100644 --- a/frontend/tests/unit/components/workspace/input-box-send-gating.dom.test.tsx +++ b/frontend/tests/unit/components/workspace/input-box-send-gating.dom.test.tsx @@ -14,7 +14,7 @@ import { PromptInputProvider } from "@/components/ai-elements/prompt-input"; import { InputBox } from "@/components/workspace/input-box"; import { ThreadContext } from "@/components/workspace/messages/context"; import { AuthProvider } from "@/core/auth/AuthProvider"; -import { DEFAULT_LOCALE } from "@/core/i18n"; +import { DEFAULT_LOCALE, type Locale } from "@/core/i18n"; import { I18nProvider } from "@/core/i18n/context"; rs.mock("next/navigation", () => ({ @@ -58,18 +58,20 @@ function renderComposer({ onPrepareThread, onGoalChange, isWelcomeMode = false, + locale = DEFAULT_LOCALE, }: { canCreateRuns?: boolean; onSubmit: () => void; onPrepareThread?: () => void | Promise; onGoalChange?: () => void; isWelcomeMode?: boolean; + locale?: Locale; }) { const queryClient = new QueryClient({ defaultOptions: { queries: { retry: false }, mutations: { retry: false } }, }); const tree = (onSubmitProp: () => void): ReactNode => ( - + { expect(submit.tagName).toBe("BUTTON"); }); }); + +describe("InputBox /goal rejected by the server", () => { + const BUSY = "A run is still going in this chat. Try again when it finishes."; + const GOAL = { + objective: "finish all tests", + status: "active", + created_at: "2026-10-07T00:00:00Z", + updated_at: "2026-10-07T00:00:00Z", + continuation_count: 0, + max_continuations: 8, + no_progress_count: 0, + max_no_progress_continuations: 2, + }; + + function mockGoalResponses( + respond: (method: string) => { status: number; body: unknown }, + ) { + const methods: string[] = []; + rs.stubGlobal( + "fetch", + rs.fn(async (input: RequestInfo | URL, init?: RequestInit) => { + const url = + typeof input === "string" + ? input + : input instanceof URL + ? input.href + : input.url; + if (!url.endsWith("/goal")) { + return new Response("{}", { status: 200 }); + } + const method = init?.method ?? "GET"; + methods.push(method); + const { status, body } = respond(method); + return new Response(JSON.stringify(body), { + status, + headers: { "Content-Type": "application/json" }, + }); + }), + ); + return methods; + } + + it.each([ + [ + "/goal finish all tests", + "PUT", + "Thread has a run in flight. Set the goal after the run finishes.", + ], + [ + "/goal clear", + "DELETE", + "Thread has a run in flight. Clear the goal after the run finishes.", + ], + ])( + "keeps %s in the composer when %s returns 409", + async (command, method, detail) => { + const methods = mockGoalResponses(() => ({ + status: 409, + body: { detail }, + })); + const error = rs.spyOn(toast, "error"); + const success = rs.spyOn(toast, "success"); + const onSubmit = rs.fn(); + const onGoalChange = rs.fn(); + const { container } = renderComposer({ onSubmit, onGoalChange }); + + submitForm(container, command); + await waitFor(() => expect(error).toHaveBeenCalledWith(BUSY)); + // Let PromptInput settle the submit promise before reading the draft. + await new Promise((resolve) => setTimeout(resolve, 50)); + + expect(methods).toEqual([method]); + expect(onSubmit).not.toHaveBeenCalled(); + expect(onGoalChange).not.toHaveBeenCalled(); + expect(success).not.toHaveBeenCalled(); + expect(container.querySelector("textarea")?.value).toBe(command); + }, + ); + + it("says a run is still going in zh-CN and keeps the draft", async () => { + // useI18n re-reads the locale cookie on mount. + document.cookie = "locale=zh-CN; path=/"; + try { + mockGoalResponses(() => ({ + status: 409, + body: { detail: "Thread has a run in flight." }, + })); + const error = rs.spyOn(toast, "error"); + const { container } = renderComposer({ + onSubmit: rs.fn(), + locale: "zh-CN", + }); + + submitForm(container, "/goal clear"); + await waitFor(() => + expect(error).toHaveBeenCalledWith( + "这个对话还有运行未结束,结束后再试。", + ), + ); + await new Promise((resolve) => setTimeout(resolve, 50)); + expect(container.querySelector("textarea")?.value).toBe("/goal clear"); + } finally { + document.cookie = "locale=; max-age=0; path=/"; + } + }); + + it("keeps the draft when a second submit replaces the first and then fails", async () => { + let calls = 0; + rs.stubGlobal( + "fetch", + rs.fn(async (input: RequestInfo | URL, init?: RequestInit) => { + const url = + typeof input === "string" + ? input + : input instanceof URL + ? input.href + : input.url; + if (!url.endsWith("/goal")) { + return new Response("{}", { status: 200 }); + } + calls += 1; + if (calls === 1) { + // Pending until the second submit aborts it. + return new Promise((_resolve, reject) => { + init?.signal?.addEventListener("abort", () => + reject(new DOMException("aborted", "AbortError")), + ); + }); + } + await new Promise((resolve) => setTimeout(resolve, 50)); + return new Response(JSON.stringify({ detail: "busy" }), { + status: 409, + headers: { "Content-Type": "application/json" }, + }); + }), + ); + const error = rs.spyOn(toast, "error"); + const { container } = renderComposer({ onSubmit: rs.fn() }); + const draft = () => container.querySelector("textarea")?.value; + + submitForm(container, "/goal clear"); + await waitFor(() => expect(calls).toBe(1)); + fireEvent.submit(container.querySelector("form")!); + await waitFor(() => expect(calls).toBe(2)); + // The stale first request leaves the composer to the second one. + await new Promise((resolve) => setTimeout(resolve, 20)); + expect(draft()).toBe("/goal clear"); + + await waitFor(() => expect(error).toHaveBeenCalledWith(BUSY)); + await new Promise((resolve) => setTimeout(resolve, 50)); + expect(error).toHaveBeenCalledTimes(1); + expect(draft()).toBe("/goal clear"); + }); + + it("keeps the server's detail and the draft for other failures", async () => { + mockGoalResponses(() => ({ + status: 422, + body: { detail: "Goal objective is too long." }, + })); + const error = rs.spyOn(toast, "error"); + const { container } = renderComposer({ onSubmit: rs.fn() }); + + submitForm(container, "/goal finish all tests"); + await waitFor(() => + expect(error).toHaveBeenCalledWith("Goal objective is too long."), + ); + await new Promise((resolve) => setTimeout(resolve, 50)); + expect(container.querySelector("textarea")?.value).toBe( + "/goal finish all tests", + ); + }); + + it.each([ + [ + "a goal without a status", + { objective: "finish" }, + null, + "No active goal.", + ], + ["a paused goal", { ...GOAL, status: "paused" }, null, "No active goal."], + [ + "an active goal without created_at", + { objective: "finish", status: "active" }, + { objective: "finish", status: "active" }, + "Active goal: finish", + ], + ])( + "reads %s like the backend's active-goal check", + async (_name, stored, expected, message) => { + mockGoalResponses(() => ({ status: 200, body: { goal: stored } })); + const info = rs.spyOn(toast, "info"); + const onGoalChange = rs.fn(); + const { container } = renderComposer({ onSubmit: rs.fn(), onGoalChange }); + + submitForm(container, "/goal"); + await waitFor(() => expect(onGoalChange).toHaveBeenCalledTimes(1)); + expect(onGoalChange).toHaveBeenCalledWith(expected, "status"); + expect(info).toHaveBeenCalledWith(message); + }, + ); + + it.each([ + ["/goal finish all tests", GOAL, "set"], + ["/goal clear", null, "clear"], + ["/goal", GOAL, "status"], + ] as const)( + "reports %s to the page as a %s change", + async (command, goal, kind) => { + mockGoalResponses(() => ({ status: 200, body: { goal } })); + const onGoalChange = rs.fn(); + const { container } = renderComposer({ onSubmit: rs.fn(), onGoalChange }); + + submitForm(container, command); + await waitFor(() => expect(onGoalChange).toHaveBeenCalledTimes(1)); + expect(onGoalChange).toHaveBeenCalledWith(goal, kind); + await waitFor(() => + expect(container.querySelector("textarea")?.value).toBe(""), + ); + }, + ); +}); diff --git a/frontend/tests/unit/components/workspace/messages/message-list-item.dom.test.tsx b/frontend/tests/unit/components/workspace/messages/message-list-item.dom.test.tsx index e4c28a7115..e2f8bd9223 100644 --- a/frontend/tests/unit/components/workspace/messages/message-list-item.dom.test.tsx +++ b/frontend/tests/unit/components/workspace/messages/message-list-item.dom.test.tsx @@ -1,6 +1,12 @@ import type { Message } from "@langchain/langgraph-sdk"; import { afterEach, describe, expect, it, rs } from "@rstest/core"; -import { cleanup, render } from "@testing-library/react"; +import { + act, + cleanup, + fireEvent, + render, + screen, +} from "@testing-library/react"; import { MessageListItem } from "@/components/workspace/messages/message-list-item"; import { I18nContext } from "@/core/i18n/context"; @@ -148,3 +154,84 @@ describe("MessageListItem conversation reference chip", () => { }, ); }); + +describe("MessageListItem edit lock while a goal is set", () => { + function renderHumanRow( + flags: { canEdit?: boolean; editLockedByGoal?: boolean }, + locale: "en-US" | "zh-CN" = "en-US", + ) { + return render( + undefined, + t: locale === "en-US" ? enUS : zhCN, + }} + > + true} + {...flags} + /> + , + ); + } + + it("shows a locked pencil that explains itself and opens no editor", async () => { + const view = renderHumanRow({ editLockedByGoal: true }); + const pencil = screen.getByRole("button", { name: "Edit and rerun" }); + expect(pencil.getAttribute("aria-disabled")).toBe("true"); + // Only the icon is dimmed, so the keyboard focus ring keeps full contrast. + expect(pencil.className.split(" ")).not.toContain( + "aria-disabled:opacity-50", + ); + expect(pencil.querySelector("svg")?.getAttribute("class")).toContain( + "opacity-50", + ); + const description = document.getElementById( + pencil.getAttribute("aria-describedby")!, + ); + expect(description?.textContent).toBe( + "Editing is off while a goal is set. Run /goal clear to edit.", + ); + + fireEvent.click(pencil); + expect(view.container.querySelector("textarea")).toBeNull(); + + // The tooltip opens on keyboard focus, and the toolbar shows while focus + // is inside it, so the locked pencil is visible to keyboard users. + await act(async () => { + pencil.focus(); + }); + expect( + (await screen.findAllByText(enUS.inputBox.goalBar.editLocked)).length, + ).toBeGreaterThan(1); + expect(screen.getByTestId("message-toolbar").className).toContain( + "focus-within:opacity-100", + ); + }); + + it("explains the lock in zh-CN", () => { + renderHumanRow({ editLockedByGoal: true }, "zh-CN"); + const pencil = screen.getByRole("button", { name: "编辑并重新运行" }); + expect( + document.getElementById(pencil.getAttribute("aria-describedby")!) + ?.textContent, + ).toBe("设置了目标时不能编辑,先运行 /goal clear 清除目标。"); + }); + + it("keeps the normal pencil when edit is allowed", () => { + const view = renderHumanRow({ canEdit: true }); + const pencil = screen.getByRole("button", { name: "Edit and rerun" }); + expect(pencil.hasAttribute("aria-disabled")).toBe(false); + fireEvent.click(pencil); + expect(view.container.querySelector("textarea")).not.toBeNull(); + }); + + it("renders no pencil when neither flag is set", () => { + renderHumanRow({}); + expect(screen.queryByRole("button", { name: "Edit and rerun" })).toBeNull(); + }); +}); diff --git a/frontend/tests/unit/components/workspace/messages/message-list-scheduled.dom.test.tsx b/frontend/tests/unit/components/workspace/messages/message-list-scheduled.dom.test.tsx index 3d12206fce..93c05ccecc 100644 --- a/frontend/tests/unit/components/workspace/messages/message-list-scheduled.dom.test.tsx +++ b/frontend/tests/unit/components/workspace/messages/message-list-scheduled.dom.test.tsx @@ -51,13 +51,16 @@ rs.mock("@/components/workspace/messages/message-list-item", () => ({ MessageListItem: ({ message, canEdit, + editLockedByGoal, }: { message: Message; canEdit?: boolean; + editLockedByGoal?: boolean; }) => (
), })); @@ -84,6 +87,7 @@ function view( messages: Message[], isLoading: boolean, scheduledTaskEvents?: ScheduledTaskEvent[], + { goalSet = false }: { goalSet?: boolean } = {}, ) { return ( true} thread={ { @@ -139,6 +144,18 @@ describe("MessageList with scheduled runs", () => { ); }); + it("locks only the latest editable human turn while a goal is set", () => { + render(view(chatThreadWithRuns(), false, undefined, { goalSet: true })); + const humans = screen.getAllByTestId("item-human"); + expect(humans.length).toBeGreaterThan(1); + expect(humans.map((item) => item.getAttribute("data-edit-locked"))).toEqual( + [...humans.slice(0, -1).map(() => "false"), "true"], + ); + expect( + humans.every((item) => item.getAttribute("data-can-edit") === "false"), + ).toBe(true); + }); + it("turn duration and the active-answer indicator match an ordinary turn", () => { const scheduled = runThread(); const ordinary = withOrdinaryHumanTurn(scheduled); diff --git a/frontend/tests/unit/components/workspace/messages/scheduled-task-card.dom.test.tsx b/frontend/tests/unit/components/workspace/messages/scheduled-task-card.dom.test.tsx index 72be73bd0d..5bfc56469c 100644 --- a/frontend/tests/unit/components/workspace/messages/scheduled-task-card.dom.test.tsx +++ b/frontend/tests/unit/components/workspace/messages/scheduled-task-card.dom.test.tsx @@ -247,6 +247,27 @@ describe("ScheduledTaskCard", () => { expectNoRawIdentifiers(card()); }); + test.each([ + ["en-US", "every item is checked.", "every item is checked, pauses itself"], + ["zh-CN", "清单全部勾完。", "清单全部勾完,满足后自动暂停"], + ] as const)( + "%s drops the stop condition's closing period before the pause clause", + async (locale, stopCondition, expected) => { + api.fetchScheduledTask.mockResolvedValue( + liveTask({ stop_condition: stopCondition }), + ); + renderCard( + toolResult({ + task: { ...toolResult().task, stop_condition: stopCondition }, + }), + locale, + ); + const stops = await screen.findByTestId("scheduled-task-card-stops"); + await waitFor(() => expect(stops.textContent).toContain(expected)); + expect(stops.textContent).not.toMatch(/[.。][,,]/); + }, + ); + test("a recurring task mid-run disables Pause and Run once now", async () => { api.fetchScheduledTask.mockResolvedValue( liveTask({ status: "enabled", active_run_status: "running" }), diff --git a/frontend/tests/unit/components/workspace/thread-goal-contract.test.ts b/frontend/tests/unit/components/workspace/thread-goal-contract.test.ts new file mode 100644 index 0000000000..e7be4067ed --- /dev/null +++ b/frontend/tests/unit/components/workspace/thread-goal-contract.test.ts @@ -0,0 +1,112 @@ +import { readFileSync } from "node:fs"; +import { resolve } from "node:path"; + +import { describe, expect, it } from "@rstest/core"; + +import { + describeGoalBar, + describeGoalBarCopy, + GOAL_HOST_REASON_TEXTS, + GOAL_OUTCOME_KEYS, + isThreadGoalOutcome, +} from "@/components/workspace/goal-status-helpers"; +import { enUS } from "@/core/i18n/locales/en-US"; +import { zhCN } from "@/core/i18n/locales/zh-CN"; +import { CHECK_FAILURE_CODES } from "@/core/scheduled-tasks/goal-outcome"; +import type { GoalState } from "@/core/threads/types"; + +import { findRawIdentifiers } from "../../../e2e/utils/raw-identifiers"; + +type ContractFile = { + version: number; + stand_down_reason_codes: string[]; + check_failure_codes: string[]; + goal_outcome: { channel: string; statuses: string[]; keys: string[] }; + history_head_keys: string[]; + host_reason_texts: string[]; +}; + +const CONTRACT = JSON.parse( + readFileSync( + resolve(__dirname, "../../../../../contracts/thread_goal_contract.json"), + "utf-8", + ), +) as ContractFile; + +function stoodDown(code: string): GoalState { + return { + objective: "ship it", + status: "active", + created_at: "2026-01-01T00:00:00Z", + updated_at: "2026-01-01T00:05:00Z", + continuation_count: 8, + max_continuations: 8, + no_progress_count: 0, + max_no_progress_continuations: 2, + last_evaluation: { + satisfied: false, + blocker: "goal_not_met_yet", + reason: "Totals missing.", + stand_down_reason: code, + }, + }; +} + +describe("thread goal contract", () => { + it.each(CONTRACT.stand_down_reason_codes)( + "%s has its own readable stop copy in en-US and zh-CN", + (code) => { + for (const hasOpenHumanInputCard of [false, true]) { + const view = describeGoalBar({ + goal: stoodDown(code), + outcome: null, + isRunning: false, + hasOpenHumanInputCard, + }); + expect(view).toMatchObject({ kind: "stopped", rawCode: null }); + if (view.kind !== "stopped") continue; + expect(view.stop).not.toBe("unknown"); + expect(view.reasonKey).not.toBeNull(); + for (const t of [enUS, zhCN]) { + const copy = describeGoalBarCopy(view, t); + expect(copy?.chip).toBeTruthy(); + expect(copy?.detail).toBeTruthy(); + expect(copy?.detail).not.toContain("{"); + expect(findRawIdentifiers(copy?.detail ?? "")).toEqual([]); + } + } + }, + ); + + it("check-failure codes are the scheduled page's and read as unchecked", () => { + expect(CONTRACT.check_failure_codes).toEqual([...CHECK_FAILURE_CODES]); + for (const code of CONTRACT.check_failure_codes) { + expect( + describeGoalBar({ + goal: stoodDown(code), + outcome: null, + isRunning: false, + }), + ).toMatchObject({ stop: "unchecked", chip: "unchecked", note: null }); + } + }); + + it("the record keys and statuses match the frontend shape", () => { + expect(CONTRACT.goal_outcome.channel).toBe("goal_outcome"); + expect(CONTRACT.goal_outcome.keys).toEqual([...GOAL_OUTCOME_KEYS]); + expect(CONTRACT.history_head_keys).toContain(CONTRACT.goal_outcome.channel); + expect(CONTRACT.history_head_keys).toContain("goal"); + const record = { + objective: "ship it", + goal_created_at: "", + }; + for (const status of CONTRACT.goal_outcome.statuses) { + expect(isThreadGoalOutcome({ ...record, status })).toBe(true); + } + expect(isThreadGoalOutcome({ ...record, status: "cleared" })).toBe(false); + }); + + it("host-written reasons are the ones the bar never shows as a note", () => { + expect([...GOAL_HOST_REASON_TEXTS]).toEqual(CONTRACT.host_reason_texts); + }); +}); diff --git a/frontend/tests/unit/components/workspace/use-active-goal.dom.test.tsx b/frontend/tests/unit/components/workspace/use-active-goal.dom.test.tsx new file mode 100644 index 0000000000..e1a363fd07 --- /dev/null +++ b/frontend/tests/unit/components/workspace/use-active-goal.dom.test.tsx @@ -0,0 +1,200 @@ +import type { Message } from "@langchain/langgraph-sdk"; +import { describe, expect, it } from "@rstest/core"; +import { act, renderHook } from "@testing-library/react"; + +import { useActiveGoal } from "@/components/workspace/use-active-goal"; +import type { GoalState, ThreadGoalOutcome } from "@/core/threads/types"; + +const GOAL: GoalState = { + objective: "Write a three-line summary of README.md", + status: "active", + created_at: "2026-10-07T00:00:00Z", + updated_at: "2026-10-07T00:00:00Z", + continuation_count: 0, + max_continuations: 8, + no_progress_count: 0, + max_no_progress_continuations: 2, +}; + +const HUMAN = { id: "h-1", type: "human", content: GOAL.objective } as Message; +const AI = { id: "ai-1", type: "ai", content: "Done: summary." } as Message; + +function recordOf( + goal: GoalState, + overrides: Partial = {}, +): ThreadGoalOutcome { + return { + status: "achieved", + objective: goal.objective, + goal_created_at: goal.created_at, + achieved_at: "2026-10-07T00:01:00Z", + continuation_count: 0, + max_continuations: 8, + reason: "The summary has three lines.", + relied_on_assumption: false, + reply_message_id: "ai-1", + ...overrides, + }; +} + +type Props = { + threadId: string; + goal?: unknown; + outcome?: unknown; + messages: Message[]; +}; + +function renderGoal(initial: Partial = {}) { + return renderHook( + ({ threadId, goal, outcome, messages }: Props) => + useActiveGoal(threadId, goal, outcome, messages), + { + initialProps: { + threadId: "thread-1", + messages: [HUMAN], + ...initial, + } as Props, + }, + ); +} + +describe("useActiveGoal", () => { + it("drops the set override once the server records that goal as met", () => { + const { result, rerender } = renderGoal(); + act(() => result.current.setLocalGoal(GOAL, "set")); + expect(result.current.hasGoal).toBe(true); + + // The satisfied clear: values without the goal key, plus the record. + rerender({ + threadId: "thread-1", + outcome: recordOf(GOAL), + messages: [HUMAN, AI], + }); + + expect(result.current.hasGoal).toBe(false); + expect(result.current.activeGoal).toBeNull(); + expect(result.current.goalOutcome).toEqual(recordOf(GOAL)); + }); + + it("keeps an override whose goal is not the recorded one", () => { + const { result, rerender } = renderGoal(); + const again = { ...GOAL, created_at: "2026-10-07T00:02:00Z" }; + act(() => result.current.setLocalGoal(again, "set")); + rerender({ + threadId: "thread-1", + outcome: recordOf(GOAL), + messages: [HUMAN, AI], + }); + expect(result.current.activeGoal).toBe(again); + expect(result.current.goalOutcome).toBeNull(); + }); + + it("keeps a goal set in the same tick the earlier goal's record arrives", () => { + const { result, rerender } = renderGoal(); + act(() => result.current.setLocalGoal(GOAL, "set")); + const next = { + ...GOAL, + objective: "Translate it", + created_at: "2026-10-07T00:03:00Z", + }; + act(() => { + result.current.setLocalGoal(next, "set"); + rerender({ + threadId: "thread-1", + outcome: recordOf(GOAL), + messages: [HUMAN, AI], + }); + }); + expect(result.current.activeGoal).toBe(next); + }); + + it("hides the record after this tab's clear, but not after a status read", () => { + const shown = { outcome: recordOf(GOAL), messages: [HUMAN, AI] }; + const status = renderGoal(shown); + expect(status.result.current.goalOutcome).not.toBeNull(); + // GET /goal answers goal: null while only the record exists. + act(() => status.result.current.setLocalGoal(null, "status")); + expect(status.result.current.goalOutcome).toEqual(recordOf(GOAL)); + + const clear = renderGoal(shown); + act(() => clear.result.current.setLocalGoal(null, "clear")); + expect(clear.result.current.goalOutcome).toBeNull(); + expect(clear.result.current.hasGoal).toBe(false); + }); + + it("shows a newer record again after hiding an older one", () => { + const { result, rerender } = renderGoal({ + outcome: recordOf(GOAL), + messages: [HUMAN, AI], + }); + act(() => result.current.setLocalGoal(GOAL, "set")); + expect(result.current.goalOutcome).toBeNull(); + + const newer = recordOf(GOAL, { achieved_at: "2026-10-07T00:09:00Z" }); + rerender({ threadId: "thread-1", outcome: newer, messages: [HUMAN, AI] }); + expect(result.current.hasGoal).toBe(false); + expect(result.current.goalOutcome).toEqual(newer); + }); + + it("resets the override and the hidden record on a thread switch", () => { + const shown = { outcome: recordOf(GOAL), messages: [HUMAN, AI] }; + const { result, rerender } = renderGoal(shown); + act(() => result.current.setLocalGoal(null, "clear")); + expect(result.current.goalOutcome).toBeNull(); + + rerender({ threadId: "thread-2", ...shown }); + expect(result.current.goalOutcome).toEqual(recordOf(GOAL)); + + act(() => result.current.setLocalGoal(GOAL, "set")); + rerender({ threadId: "thread-3", messages: [] }); + expect(result.current.activeGoal).toBeNull(); + }); + + it("lets an active server goal hide a coexisting record", () => { + const { result } = renderGoal({ + goal: { ...GOAL, objective: "Next" }, + outcome: recordOf(GOAL), + messages: [HUMAN, AI], + }); + expect(result.current.activeGoal?.objective).toBe("Next"); + expect(result.current.goalOutcome).toBeNull(); + }); + + it("keeps a thread-value goal stored without timestamps", () => { + // POST /state stored it without created_at; the backend still locks edit. + const bare = { objective: GOAL.objective, status: "active" } as GoalState; + const { result } = renderGoal({ goal: bare }); + expect(result.current.hasGoal).toBe(true); + expect(result.current.activeGoal).toBe(bare); + }); + + it("drops a status-read goal without created_at once its '' record arrives", () => { + const bare = { objective: GOAL.objective, status: "active" } as GoalState; + const { result, rerender } = renderGoal(); + act(() => result.current.setLocalGoal(bare, "status")); + expect(result.current.activeGoal).toBe(bare); + + // The satisfied clear: no goal key, plus the record. + rerender({ + threadId: "thread-1", + outcome: recordOf(GOAL, { goal_created_at: "" }), + messages: [HUMAN, AI], + }); + expect(result.current.hasGoal).toBe(false); + expect(result.current.goalOutcome).toEqual( + recordOf(GOAL, { goal_created_at: "" }), + ); + }); + + it("does not treat a malformed goal value as a set goal", () => { + for (const goal of [ + { objective: "finish" }, + { status: "active" }, + { ...GOAL, status: "paused" }, + ]) { + const { result } = renderGoal({ goal }); + expect(result.current.hasGoal).toBe(false); + expect(result.current.activeGoal).toBeNull(); + } + }); +}); diff --git a/frontend/tests/unit/components/workspace/use-active-goal.test.ts b/frontend/tests/unit/components/workspace/use-active-goal.test.ts index 8a1987afba..8d44136b42 100644 --- a/frontend/tests/unit/components/workspace/use-active-goal.test.ts +++ b/frontend/tests/unit/components/workspace/use-active-goal.test.ts @@ -1,10 +1,15 @@ +import type { Message } from "@langchain/langgraph-sdk"; import { describe, expect, it } from "@rstest/core"; +import { goalOutcomeKey } from "@/components/workspace/goal-status-helpers"; import { + normalizeGoalStatusRead, + normalizeServerGoal, resolveActiveGoal, + resolveGoalView, shouldResetLocalGoalOverride, } from "@/components/workspace/use-active-goal"; -import type { GoalState } from "@/core/threads/types"; +import type { GoalState, ThreadGoalOutcome } from "@/core/threads/types"; function makeGoal(overrides: Partial = {}): GoalState { return { @@ -42,7 +47,7 @@ describe("shouldResetLocalGoalOverride", () => { ).toBe(false); }); - it("resets an optimistic goal when the server explicitly clears the goal", () => { + it("resets an optimistic goal when the server reports a goal value", () => { expect( shouldResetLocalGoalOverride({ serverGoalProvided: true, @@ -60,3 +65,95 @@ describe("shouldResetLocalGoalOverride", () => { ).toBe(true); }); }); + +describe("normalizeServerGoal", () => { + it("keeps a missing key missing and drops a malformed or inactive goal", () => { + const goal = makeGoal(); + expect(normalizeServerGoal(undefined)).toBeUndefined(); + expect(normalizeServerGoal(goal)).toBe(goal); + expect(normalizeServerGoal(null)).toBeNull(); + // POST /state can store an active goal without timestamps: still a goal. + const bare = { objective: "finish", status: "active" }; + expect(normalizeServerGoal(bare)).toBe(bare); + // It can also store a dict that is not an active goal. + expect(normalizeServerGoal({ objective: "finish" })).toBeNull(); + expect(normalizeServerGoal({ ...goal, status: "paused" })).toBeNull(); + expect(normalizeServerGoal({ status: "active" })).toBeNull(); + expect(normalizeServerGoal([goal])).toBeNull(); + }); +}); + +describe("normalizeGoalStatusRead", () => { + it("keeps what the backend treats as an active goal, and nothing else", () => { + const goal = makeGoal(); + expect(normalizeGoalStatusRead(goal)).toBe(goal); + // Stored without created_at: still active on the server, so edit is 409. + const bare = { objective: "finish", status: "active" }; + expect(normalizeGoalStatusRead(bare)).toBe(bare); + // No status: the server allows the edit, so this is no goal. + expect(normalizeGoalStatusRead({ objective: "finish" })).toBeNull(); + expect(normalizeGoalStatusRead({ ...goal, status: "paused" })).toBeNull(); + expect(normalizeGoalStatusRead({ status: "active" })).toBeNull(); + expect(normalizeGoalStatusRead(undefined)).toBeNull(); + expect(normalizeGoalStatusRead(null)).toBeNull(); + expect(normalizeGoalStatusRead([goal])).toBeNull(); + expect(normalizeGoalStatusRead("finish")).toBeNull(); + }); +}); + +describe("resolveGoalView", () => { + const anchor = { id: "ai-1", type: "ai", content: "Done." } as Message; + const outcome: ThreadGoalOutcome = { + status: "achieved", + objective: "ship the landing page", + goal_created_at: "2026-01-01T00:00:00Z", + achieved_at: "2026-01-01T00:05:00Z", + continuation_count: 0, + max_continuations: 8, + reason: "Shipped.", + relied_on_assumption: false, + reply_message_id: "ai-1", + }; + const view = (overrides: Partial[0]>) => + resolveGoalView({ + localGoal: undefined, + serverGoal: undefined, + serverOutcome: outcome, + hiddenOutcomeKey: null, + messages: [anchor], + ...overrides, + }); + + it("shows a current record when no goal is active", () => { + expect(view({})).toEqual({ activeGoal: null, goalOutcome: outcome }); + }); + + it("lets an active goal, local or server, hide the record", () => { + const goal = makeGoal({ objective: "next" }); + expect(view({ localGoal: goal })).toEqual({ + activeGoal: goal, + goalOutcome: null, + }); + expect(view({ serverGoal: goal }).goalOutcome).toBeNull(); + }); + + it("hides the record this tab removed and one the chat moved past", () => { + expect( + view({ hiddenOutcomeKey: goalOutcomeKey(outcome) }).goalOutcome, + ).toBeNull(); + expect( + view({ + messages: [ + anchor, + { id: "h-2", type: "human", content: "more" } as Message, + ], + }).goalOutcome, + ).toBeNull(); + }); + + it("ignores a record of an unknown status", () => { + expect( + view({ serverOutcome: { ...outcome, status: "cleared" } }).goalOutcome, + ).toBeNull(); + }); +}); diff --git a/frontend/tests/unit/core/scheduled-tasks/goal-outcome.test.ts b/frontend/tests/unit/core/scheduled-tasks/goal-outcome.test.ts index c46ee257d2..1608b12f30 100644 --- a/frontend/tests/unit/core/scheduled-tasks/goal-outcome.test.ts +++ b/frontend/tests/unit/core/scheduled-tasks/goal-outcome.test.ts @@ -9,6 +9,7 @@ import { describeGoalOutcome, describeTaskLastError, describeTaskOutcome, + goalReasonKeyOf, parseStopRunId, requestedScheduleStop, } from "@/core/scheduled-tasks/goal-outcome"; @@ -99,6 +100,16 @@ test("an unknown unmet code stays visible without a label", () => { }); }); +test("goalReasonKeyOf reads a code with or without the blocked: prefix", () => { + expect(goalReasonKeyOf("blocked:needs_user_input")).toBe("needsUserInput"); + expect(goalReasonKeyOf("needs_user_input")).toBe("needsUserInput"); + expect(goalReasonKeyOf("thread_changed_before_continuation")).toBe( + "threadChanged", + ); + expect(goalReasonKeyOf("blocked:none")).toBeNull(); + expect(goalReasonKeyOf("future_reason")).toBeNull(); +}); + test.each([ ["run-2", "run-2", true], ["run-2", "run-1", false], diff --git a/frontend/tests/unit/helpers/readable.test.ts b/frontend/tests/unit/helpers/readable.test.ts index 479f1e31d8..993b08b299 100644 --- a/frontend/tests/unit/helpers/readable.test.ts +++ b/frontend/tests/unit/helpers/readable.test.ts @@ -27,6 +27,18 @@ describe("raw identifier guard", () => { "Agent: lead_agent", "max_runs reached", "Status: unmet", + "Stopped: blocked:needs_user_input", + "max_continuations_reached", + "Reason token_capped", + "thread_changed_after_evaluation", + "goal_outcome missing", + "last_evaluation.stand_down_reason", + "Blocker: needs_user_input", + "goal_not_met_yet", + "Waiting: external_wait", + "reply_message_id: null", + "continuation_count 1 of max_continuations 8", + "relied_on_assumption", ])("flags %s", (text) => { expect(findRawIdentifiers(text).length).toBeGreaterThan(0); }); @@ -36,6 +48,8 @@ describe("raw identifier guard", () => { "工作日 09:00 (Asia/Shanghai) · 下次:明天 09:00", "Safety cap: 2 of 5 runs used", "Every 30 minutes", + "Continuation limit reached · 8/8. Reply to keep going.", + "Token budget reached", ])("accepts readable copy: %s", (text) => { expectNoRawIdentifiers(text); }); @@ -55,4 +69,17 @@ describe("raw identifier guard", () => { } }, ); + + it.each([ + ["en-US", enUS], + ["zh-CN", zhCN], + ] as const)( + "no goal bar copy in %s contains a raw identifier", + (_locale, t) => { + for (const [path, text] of strings(t.inputBox.goalBar)) { + const shown = text.replace(/\{[a-z_]+\}/g, "1"); + expect([path, findRawIdentifiers(shown)]).toEqual([path, []]); + } + }, + ); });