AnoobFeng 47b0f604f4
feat(frontend):enhance the ask_clarification interaction with visualized card (#3956)
* feat(frontend): add structured human input cards for ask_clarification

Implement a reusable Human Input Card flow for ask_clarification while keeping
the existing text fallback for older clients and IM channels.

Backend:
- Add structured ToolMessage.artifact.human_input payloads for clarification requests.
- Preserve ToolMessage.content as the readable Markdown/text fallback.
- Normalize clarification options from native lists, JSON strings, plain strings,
  mixed scalar values, None, and missing options.
- Derive input_mode as choice_with_other when options exist, otherwise free_text.
- Keep disable_clarification non-interactive behavior as a plain ToolMessage with
  no human_input artifact.
- Cover artifact persistence and Gateway message metadata preservation in tests.

Frontend:
- Add human input protocol types, runtime guards, extractors, response builders,
  and thread-state helpers.
- Add reusable HumanInputCard with option buttons, free-text input, pending,
  read-only, disabled, and answered states.
- Render structured clarification cards from artifact.human_input, with Markdown
  fallback for malformed or legacy tool messages.
- Preserve line breaks in structured question/context/option text.
- Hide submitted clarification bridge messages from the chat UI via
  additional_kwargs.hide_from_ui.
- Send structured human_input_response metadata through the fourth sendMessage
  options argument, preserving run context in the third argument.
- Wire submissions for normal chats, custom agent chats, agent bootstrap chats,
  and sidecar chats.
- Derive answered state from raw thread.messages so hidden replies still update
  the original card.
- Clear pending state when the hidden reply arrives, dispatch is dropped, or a
  later async stream failure appears on thread.error.

* perf(frontend): optimize HumanInputCard UI interactions

- Support Enter key to submit text input (Shift+Enter for newline)
- Render question and context fields as Markdown instead of plain text
- Replace deprecated FormEventHandler type with structural typing

* test(frontend): add unit test cover optimize HumanInputCard UI interactions

* feat(frontend): disabled chatbox when has new human-input-card

* fix(style): lint error fix

* fix: sanitize hidden human input replies

- Preserve IME composition safety for human input card Enter submits
- Treat hidden human input responses as genuine user messages for sanitization
- Keep hidden card replies in memory filtering while excluding malformed/internal hidden messages
- Add regression coverage for card IME handling and hidden reply sanitization

* fix: tighten human input response validation

- Reject empty hidden human input response values
- Remove invalid list ARIA role from human input card options
- Add backend coverage for empty response payloads

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-06 22:34:41 +08:00

77 lines
2.2 KiB
Python

"""Structured human-input message metadata helpers."""
from __future__ import annotations
from collections.abc import Mapping
from typing import Literal, TypedDict
HUMAN_INPUT_RESPONSE_KEY = "human_input_response"
class HumanInputTextResponse(TypedDict):
version: Literal[1]
kind: Literal["human_input_response"]
source: str
request_id: str
response_kind: Literal["text"]
value: str
class HumanInputOptionResponse(TypedDict):
version: Literal[1]
kind: Literal["human_input_response"]
source: str
request_id: str
response_kind: Literal["option"]
option_id: str
value: str
HumanInputResponse = HumanInputTextResponse | HumanInputOptionResponse
def _non_empty_string(value: object) -> str | None:
return value if isinstance(value, str) and value.strip() else None
def read_human_input_response(additional_kwargs: Mapping[str, object] | None) -> HumanInputResponse | None:
"""Read a valid human-input response payload from message metadata."""
if not additional_kwargs:
return None
raw = additional_kwargs.get(HUMAN_INPUT_RESPONSE_KEY)
if not isinstance(raw, Mapping):
return None
if raw.get("version") != 1 or raw.get("kind") != "human_input_response":
return None
source = _non_empty_string(raw.get("source"))
request_id = _non_empty_string(raw.get("request_id"))
value = _non_empty_string(raw.get("value"))
if source is None or request_id is None or value is None:
return None
response_kind = raw.get("response_kind")
if response_kind == "text":
return {
"version": 1,
"kind": "human_input_response",
"source": source,
"request_id": request_id,
"response_kind": "text",
"value": value,
}
if response_kind == "option":
option_id = _non_empty_string(raw.get("option_id"))
if option_id is None:
return None
return {
"version": 1,
"kind": "human_input_response",
"source": source,
"request_id": request_id,
"response_kind": "option",
"option_id": option_id,
"value": value,
}
return None