deer-flow/backend/tests/test_clarification_middleware.py
阿泽 1baa8ad696
feat(clarification): structured form fields for human-input cards (#4400 Phase 1) (#4406)
* feat(clarification): structured form fields for human-input cards

Add a request-side v2 `form` mode to the ask_clarification protocol so
business flows (e.g. expense reimbursement) can collect several values
in one card instead of sequential free-text questions:

- `ask_clarification` gains a restricted `fields` parameter (text /
  textarea / number / select / multi_select / checkbox / date)
- ClarificationMiddleware validates and normalizes fields explicitly
  (whitelisted types, unknown -> text, select-likes without options ->
  text, duplicate/invalid entries dropped, all-invalid falls back to
  the legacy modes) since the middleware short-circuits before tool
  execution; the plain-text fallback lists fields for IM channels
- Form payloads carry `version: 2` so older frontends degrade to the
  text fallback; replies stay on the v1 response protocol — the card
  submits a readable summary as `response_kind: "text"`, so journal
  persistence and answered-card recovery are unchanged
- Frontend renders typed field controls with required-field validation
  and compact multi-select chips

Part of #4400 (scope narrowed per maintainer feedback: request-side
only, no new response kinds, no top-level multi_choice).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(clarification): harden form protocol per review feedback

Address the five review points on #4406:

- Reject field names colliding with JS Object.prototype members on both
  sides; frontend reads form values via own-property access only, so
  `constructor`/`toString`-style names can no longer leak inherited
  members into required validation or the submitted summary
- Close open requests answered through the legacy text fallback: a
  visible plain human reply (no response metadata) now marks every
  previously-opened request as answered, so upgrading to a v2-aware
  frontend cannot leave the composer locked on an already-answered card
- Give checkbox fields deterministic boolean semantics: values are
  seeded to an explicit false ("no" in the summary) and `required` means
  must-agree/consent; documented in the tool schema
- Make middleware field validation atomic: structurally broken entries
  (bad/duplicate/reserved names, over-cap field/option counts or text
  lengths) degrade the whole form instead of silently dropping fields;
  options are trimmed/deduped with blanks removed so the backend never
  emits payloads the frontend parser rejects
- Associate form labels/controls (htmlFor/id), aria-required,
  aria-invalid, and error descriptions for accessibility

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(clarification): type the fields item schema via TypedDict

Replace `fields: list[dict[str, Any]]` with `list[ClarificationFormField]`
(a TypedDict with `name` required and the type whitelist as a Literal) so
the provider-facing tool schema documents the item shape instead of an
opaque object relying on the docstring. Runtime validation is unchanged
and stays in ClarificationMiddleware, which intercepts the call before
tool execution. Addresses the non-blocking review suggestion on #4406.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(frontend): drop unsupported aria-invalid from multi-select group

jsx-a11y: role=group does not support aria-invalid; the error linkage
stays via aria-describedby.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(clarification): coerce numeric required flags and normalize fields once

- `_normalize_bool` now coerces 1/0 (some providers serialize booleans
  as integers), so `required: 1` no longer silently flips to optional
- `_handle_clarification` normalizes `fields` once and passes the result
  to both the text fallback and the payload builder

Addresses the non-blocking review nits on #4406.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(clarification): harden form protocol per contract review round 2

Backend:
- Guard unhashable JSON in the intercept path: `type: []`/`{}` degrades
  the field to text and `clarification_type: []` coerces to str instead
  of raising TypeError (which, with return_direct, ended the turn with
  an error and no card or fallback)
- Add a total budget over the serialized normalized fields (16KB UTF-8
  bytes): per-item caps alone admitted forms whose IM text fallback
  exceeded channel delivery limits (Slack 40k chars, Feishu ~30KB card),
  silently truncating trailing fields; a boundary test proves any
  accepted form's fallback stays deliverable

Frontend:
- Submission value now appends a JSON block keyed by stable field names
  (readable summary alone is delimiter-ambiguous), with a collision
  regression test
- Parser boundary tightened to match backend constraints: empty option
  values (Radix SelectItem crash), duplicate option ids/values,
  duplicate field names, and the form<->version-2 binding are rejected
- Keep the error node mounted while any field is still invalid so
  aria-describedby never points at a removed element (happy-dom
  interaction test)
- Required semantics are now accessible: native checkbox control (no
  HTML required attribute — it would intercept the custom submit path),
  visually-hidden localized "required" markers next to the aria-hidden
  asterisks
- Legacy-fallback closure narrowed to the latest unanswered request:
  nothing guarantees a single outstanding clarification across runs, and
  closing all would silently swallow older decisions; an older request
  left open becomes the active card again

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(frontend): keep clarification selects controlled

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 14:05:31 +08:00

771 lines
31 KiB
Python

"""Tests for ClarificationMiddleware, focusing on options type coercion."""
import json
from types import SimpleNamespace
import pytest
from langgraph.graph.message import add_messages
from deerflow.agents.middlewares.clarification_middleware import ClarificationMiddleware
@pytest.fixture
def middleware():
return ClarificationMiddleware()
class TestFormatClarificationMessage:
"""Tests for _format_clarification_message options handling."""
def test_options_as_native_list(self, middleware):
"""Normal case: options is already a list."""
args = {
"question": "Which env?",
"clarification_type": "approach_choice",
"options": ["dev", "staging", "prod"],
}
result = middleware._format_clarification_message(args)
assert "1. dev" in result
assert "2. staging" in result
assert "3. prod" in result
def test_options_as_json_string(self, middleware):
"""Bug case (#1995): model serializes options as a JSON string."""
args = {
"question": "Which env?",
"clarification_type": "approach_choice",
"options": json.dumps(["dev", "staging", "prod"]),
}
result = middleware._format_clarification_message(args)
assert "1. dev" in result
assert "2. staging" in result
assert "3. prod" in result
# Must NOT contain per-character output
assert "1. [" not in result
assert '2. "' not in result
def test_options_as_json_string_scalar(self, middleware):
"""JSON string decoding to a non-list scalar is treated as one option."""
args = {
"question": "Which env?",
"clarification_type": "approach_choice",
"options": json.dumps("development"),
}
result = middleware._format_clarification_message(args)
assert "1. development" in result
# Must be a single option, not per-character iteration.
assert "2." not in result
def test_options_as_plain_string(self, middleware):
"""Edge case: options is a non-JSON string, treated as single option."""
args = {
"question": "Which env?",
"clarification_type": "approach_choice",
"options": "just one option",
}
result = middleware._format_clarification_message(args)
assert "1. just one option" in result
def test_options_none(self, middleware):
"""Options is None — no options section rendered."""
args = {
"question": "Tell me more",
"clarification_type": "missing_info",
"options": None,
}
result = middleware._format_clarification_message(args)
assert "1." not in result
def test_options_empty_list(self, middleware):
"""Options is an empty list — no options section rendered."""
args = {
"question": "Tell me more",
"clarification_type": "missing_info",
"options": [],
}
result = middleware._format_clarification_message(args)
assert "1." not in result
def test_options_missing(self, middleware):
"""Options key is absent — defaults to empty list."""
args = {
"question": "Tell me more",
"clarification_type": "missing_info",
}
result = middleware._format_clarification_message(args)
assert "1." not in result
def test_context_included(self, middleware):
"""Context is rendered before the question."""
args = {
"question": "Which env?",
"clarification_type": "approach_choice",
"context": "Need target env for config",
"options": ["dev", "prod"],
}
result = middleware._format_clarification_message(args)
assert "Need target env for config" in result
assert "Which env?" in result
assert "1. dev" in result
def test_json_string_with_mixed_types(self, middleware):
"""JSON string containing non-string elements still works."""
args = {
"question": "Pick one",
"clarification_type": "approach_choice",
"options": json.dumps(["Option A", 2, True, None]),
}
result = middleware._format_clarification_message(args)
assert "1. Option A" in result
assert "2. 2" in result
assert "3. True" in result
assert "4. None" in result
class TestHumanInputPayload:
"""Tests for structured human input request payloads."""
def test_payload_with_native_options(self, middleware):
payload = middleware._build_human_input_payload(
{
"question": "Which environment should I deploy to?",
"clarification_type": "approach_choice",
"context": "Need the target environment for config.",
"options": ["development", "staging", "production"],
},
tool_call_id="call-abc",
request_id="clarification:call-abc",
)
assert payload == {
"version": 1,
"kind": "human_input_request",
"source": "ask_clarification",
"request_id": "clarification:call-abc",
"tool_call_id": "call-abc",
"clarification_type": "approach_choice",
"question": "Which environment should I deploy to?",
"context": "Need the target environment for config.",
"input_mode": "choice_with_other",
"options": [
{"id": "option-1", "label": "development", "value": "development"},
{"id": "option-2", "label": "staging", "value": "staging"},
{"id": "option-3", "label": "production", "value": "production"},
],
}
def test_payload_with_json_string_options(self, middleware):
payload = middleware._build_human_input_payload(
{
"question": "Pick one",
"clarification_type": "approach_choice",
"options": json.dumps(["Option A", 2, True, None]),
},
tool_call_id="call-abc",
request_id="clarification:call-abc",
)
assert payload["input_mode"] == "choice_with_other"
assert payload["options"] == [
{"id": "option-1", "label": "Option A", "value": "Option A"},
{"id": "option-2", "label": "2", "value": "2"},
{"id": "option-3", "label": "True", "value": "True"},
{"id": "option-4", "label": "None", "value": "None"},
]
def test_payload_with_plain_string_option(self, middleware):
payload = middleware._build_human_input_payload(
{
"question": "Pick one",
"clarification_type": "approach_choice",
"options": "just one option",
},
tool_call_id="call-abc",
request_id="clarification:call-abc",
)
assert payload["input_mode"] == "choice_with_other"
assert payload["options"] == [{"id": "option-1", "label": "just one option", "value": "just one option"}]
def test_payload_without_options_is_free_text(self, middleware):
payload = middleware._build_human_input_payload(
{
"question": "Tell me more",
"clarification_type": "missing_info",
"options": None,
},
tool_call_id="call-abc",
request_id="clarification:call-abc",
)
assert payload["input_mode"] == "free_text"
assert "options" not in payload
def test_payload_missing_options_is_free_text(self, middleware):
payload = middleware._build_human_input_payload(
{
"question": "Tell me more",
"clarification_type": "missing_info",
},
tool_call_id="call-abc",
request_id="clarification:call-abc",
)
assert payload["input_mode"] == "free_text"
assert "options" not in payload
class TestFormPayload:
"""v2 protocol: fields render a structured form card."""
def _fields(self):
return [
{"name": "amount", "label": "Amount", "type": "number", "required": True},
{"name": "category", "label": "Category", "type": "select", "options": ["travel", "meals"], "required": True},
{"name": "receipts", "label": "Receipts", "type": "multi_select", "options": ["A-1", "A-2"]},
{"name": "note", "type": "textarea"},
]
def test_form_payload_with_native_fields(self, middleware):
payload = middleware._build_human_input_payload(
{
"question": "Please provide the expense details.",
"clarification_type": "missing_info",
"fields": self._fields(),
},
tool_call_id="call-abc",
request_id="clarification:call-abc",
)
assert payload["version"] == 2
assert payload["input_mode"] == "form"
assert payload["fields"] == [
{"name": "amount", "label": "Amount", "type": "number", "required": True},
{
"name": "category",
"label": "Category",
"type": "select",
"required": True,
"options": [
{"id": "category-option-1", "label": "travel", "value": "travel"},
{"id": "category-option-2", "label": "meals", "value": "meals"},
],
},
{
"name": "receipts",
"label": "Receipts",
"type": "multi_select",
"required": False,
"options": [
{"id": "receipts-option-1", "label": "A-1", "value": "A-1"},
{"id": "receipts-option-2", "label": "A-2", "value": "A-2"},
],
},
{"name": "note", "label": "note", "type": "textarea", "required": False},
]
def test_form_payload_with_json_string_fields(self, middleware):
payload = middleware._build_human_input_payload(
{
"question": "Details please",
"clarification_type": "missing_info",
"fields": json.dumps([{"name": "amount", "type": "number"}]),
},
tool_call_id="call-abc",
request_id="clarification:call-abc",
)
assert payload["input_mode"] == "form"
assert payload["fields"] == [{"name": "amount", "label": "amount", "type": "number", "required": False}]
def test_form_takes_precedence_over_options(self, middleware):
payload = middleware._build_human_input_payload(
{
"question": "Details please",
"clarification_type": "missing_info",
"fields": [{"name": "amount"}],
"options": ["dev", "prod"],
},
tool_call_id="call-abc",
request_id="clarification:call-abc",
)
assert payload["input_mode"] == "form"
assert "options" not in payload
def test_unknown_field_type_degrades_to_text(self, middleware):
payload = middleware._build_human_input_payload(
{
"question": "Details please",
"clarification_type": "missing_info",
"fields": [{"name": "amount", "type": "slider"}],
},
tool_call_id="call-abc",
request_id="clarification:call-abc",
)
assert payload["fields"][0]["type"] == "text"
def test_select_without_options_degrades_to_text(self, middleware):
payload = middleware._build_human_input_payload(
{
"question": "Details please",
"clarification_type": "missing_info",
"fields": [{"name": "category", "type": "select"}],
},
tool_call_id="call-abc",
request_id="clarification:call-abc",
)
assert payload["fields"][0]["type"] == "text"
assert "options" not in payload["fields"][0]
def test_any_invalid_field_entry_degrades_whole_form(self, middleware):
"""Atomic validation: a structurally broken entry must not silently
vanish from an otherwise-rendered form — the whole form degrades."""
for bad_entry in ["not-a-dict", {"label": "no name"}, {"name": " "}]:
payload = middleware._build_human_input_payload(
{
"question": "Details please",
"clarification_type": "missing_info",
"fields": [bad_entry, {"name": "amount", "type": "number"}],
"options": ["dev", "prod"],
},
tool_call_id="call-abc",
request_id="clarification:call-abc",
)
assert payload["version"] == 1
assert payload["input_mode"] == "choice_with_other"
assert "fields" not in payload
def test_duplicate_field_names_degrade_whole_form(self, middleware):
payload = middleware._build_human_input_payload(
{
"question": "Details please",
"clarification_type": "missing_info",
"fields": [
{"name": "amount", "type": "number"},
{"name": "amount", "type": "text"},
],
},
tool_call_id="call-abc",
request_id="clarification:call-abc",
)
assert payload["version"] == 1
assert payload["input_mode"] == "free_text"
assert "fields" not in payload
def test_reserved_prototype_field_names_degrade_whole_form(self, middleware):
"""`__proto__`/`constructor`/... collide with JS Object.prototype in the
frontend form-value store and must be rejected server-side."""
for reserved in ["__proto__", "constructor", "toString", "hasOwnProperty"]:
payload = middleware._build_human_input_payload(
{
"question": "Details please",
"clarification_type": "missing_info",
"fields": [{"name": reserved, "type": "text"}],
},
tool_call_id="call-abc",
request_id="clarification:call-abc",
)
assert payload["input_mode"] == "free_text", reserved
assert "fields" not in payload
def test_field_options_are_trimmed_deduped_and_blank_dropped(self, middleware):
"""Backend must never emit blank option labels — the frontend parser
rejects the whole payload on them."""
payload = middleware._build_human_input_payload(
{
"question": "Details please",
"clarification_type": "missing_info",
"fields": [{"name": "category", "type": "select", "options": [" travel ", "", " ", "travel", "meals"]}],
},
tool_call_id="call-abc",
request_id="clarification:call-abc",
)
assert [option["label"] for option in payload["fields"][0]["options"]] == ["travel", "meals"]
def test_field_count_over_cap_degrades_whole_form(self, middleware):
fields = [{"name": f"field_{i}", "type": "text"} for i in range(17)]
payload = middleware._build_human_input_payload(
{"question": "Details please", "clarification_type": "missing_info", "fields": fields},
tool_call_id="call-abc",
request_id="clarification:call-abc",
)
assert payload["input_mode"] == "free_text"
assert "fields" not in payload
def test_option_count_over_cap_degrades_whole_form(self, middleware):
options = [f"option-{i}" for i in range(25)]
payload = middleware._build_human_input_payload(
{
"question": "Details please",
"clarification_type": "missing_info",
"fields": [{"name": "category", "type": "select", "options": options}],
},
tool_call_id="call-abc",
request_id="clarification:call-abc",
)
assert payload["input_mode"] == "free_text"
assert "fields" not in payload
def test_overlong_field_text_degrades_whole_form(self, middleware):
payload = middleware._build_human_input_payload(
{
"question": "Details please",
"clarification_type": "missing_info",
"fields": [{"name": "amount", "label": "x" * 201, "type": "number"}],
},
tool_call_id="call-abc",
request_id="clarification:call-abc",
)
assert payload["input_mode"] == "free_text"
assert "fields" not in payload
def test_top_level_options_are_trimmed_and_blank_dropped(self, middleware):
payload = middleware._build_human_input_payload(
{
"question": "Which env?",
"clarification_type": "approach_choice",
"options": [" dev ", "", "prod", "dev"],
},
tool_call_id="call-abc",
request_id="clarification:call-abc",
)
assert payload["options"] == [
{"id": "option-1", "label": "dev", "value": "dev"},
{"id": "option-2", "label": "prod", "value": "prod"},
]
def test_unhashable_field_type_degrades_to_text_not_crash(self, middleware):
"""`type: []` / `type: {}` are legal JSON from a model; an unhashable
membership check would raise TypeError and (with return_direct=True)
end the turn with an error instead of any card or fallback."""
for bad_type in [[], {}, ["select"], {"t": "select"}]:
payload = middleware._build_human_input_payload(
{
"question": "Details please",
"clarification_type": "missing_info",
"fields": [{"name": "amount", "type": bad_type}],
},
tool_call_id="call-abc",
request_id="clarification:call-abc",
)
assert payload["fields"][0]["type"] == "text"
def test_unhashable_clarification_type_does_not_crash(self, middleware):
"""Same crash class as unhashable field types: `clarification_type: []`
must not raise in the icon lookup or payload builder."""
from langgraph.types import Command
request = SimpleNamespace(
tool_call={
"name": "ask_clarification",
"id": "call-clarify-1",
"args": {"question": "q?", "clarification_type": [], "fields": [{"name": "amount", "type": []}]},
},
runtime=None,
)
result = middleware.wrap_tool_call(request, lambda _req: pytest.fail("handler should not be called"))
assert isinstance(result, Command)
message = result.update["messages"][0]
assert message.artifact["human_input"]["input_mode"] == "form"
def test_serialized_fields_over_byte_budget_degrade_whole_form(self, middleware):
"""Per-item caps alone allow a form whose IM text fallback exceeds
channel limits (Slack 40k chars, Feishu ~30KB card); a total serialized
byte budget must bound the whole definition."""
fields = [
{
"name": f"field_{i}",
"label": "" * 190,
"type": "select",
"options": ["" * 190 + str(j) for j in range(20)],
}
for i in range(16)
]
payload = middleware._build_human_input_payload(
{"question": "Details please", "clarification_type": "missing_info", "fields": fields},
tool_call_id="call-abc",
request_id="clarification:call-abc",
)
assert payload["input_mode"] == "free_text"
assert "fields" not in payload
def test_accepted_forms_keep_text_fallback_under_channel_limits(self, middleware):
"""Boundary: any form the byte budget accepts must produce an IM text
fallback deliverable to the strictest supported channel (~30KB)."""
# Grow a form until the budget rejects it; the largest accepted form's
# fallback must stay under the channel bound.
largest_accepted_fallback = None
for count in range(1, 17):
fields = [
{
"name": f"field_{i}",
"label": "" * 150,
"type": "select",
"options": ["" * 100 + str(j) for j in range(10)],
}
for i in range(count)
]
args = {"question": "Q", "clarification_type": "missing_info", "fields": fields}
payload = middleware._build_human_input_payload(args, tool_call_id="c", request_id="clarification:c")
if payload["input_mode"] != "form":
break
largest_accepted_fallback = middleware._format_clarification_message(args)
assert largest_accepted_fallback is not None
assert len(largest_accepted_fallback.encode("utf-8")) < 30_000
def test_required_accepts_integer_serialization(self, middleware):
"""Some providers emit 1/0 for booleans; `required: 1` must not
silently flip a model-intended required field to optional."""
payload = middleware._build_human_input_payload(
{
"question": "Details please",
"clarification_type": "missing_info",
"fields": [
{"name": "amount", "type": "number", "required": 1},
{"name": "note", "type": "text", "required": 0},
],
},
tool_call_id="call-abc",
request_id="clarification:call-abc",
)
assert payload["fields"][0]["required"] is True
assert payload["fields"][1]["required"] is False
def test_field_placeholder_is_preserved(self, middleware):
payload = middleware._build_human_input_payload(
{
"question": "Details please",
"clarification_type": "missing_info",
"fields": [{"name": "note", "placeholder": "Optional remarks"}],
},
tool_call_id="call-abc",
request_id="clarification:call-abc",
)
assert payload["fields"][0]["placeholder"] == "Optional remarks"
def test_form_fallback_text_lists_fields(self, middleware):
result = middleware._format_clarification_message(
{
"question": "Please provide the expense details.",
"clarification_type": "missing_info",
"fields": self._fields(),
}
)
assert "1. Amount (required)" in result
assert "2. Category (required) — options: travel / meals" in result
assert "3. Receipts — options: A-1 / A-2 (multiple allowed)" in result
assert "4. note" in result
assert "Please reply with a value for each field." in result
class TestClarificationToolSchema:
"""The tool schema must expose the v2 form parameter (request-side only)."""
def test_tool_exposes_fields_argument(self):
from deerflow.tools.builtins.clarification_tool import ask_clarification_tool
schema = ask_clarification_tool.args
assert "fields" in schema
# The reply protocol stays v1 (text/option): no top-level multi_select
# mode — a standalone multi-select question is a one-field form.
assert "multi_select" not in schema
def test_fields_item_schema_is_typed(self):
"""The provider-facing schema must expose the field item shape (typed
via ClarificationFormField), not an opaque object relying on the
docstring alone."""
from langchain_core.utils.function_calling import convert_to_openai_tool
from deerflow.tools.builtins.clarification_tool import ask_clarification_tool
parameters = convert_to_openai_tool(ask_clarification_tool)["function"]["parameters"]
items = parameters["properties"]["fields"]["anyOf"][0]["items"]
assert items["required"] == ["name"]
assert sorted(items["properties"].keys()) == ["label", "name", "options", "placeholder", "required", "type"]
assert items["properties"]["type"]["enum"] == ["text", "textarea", "number", "select", "multi_select", "checkbox", "date"]
class TestClarificationCommandIdempotency:
"""Clarification tool-call retries should not duplicate messages in state."""
def test_repeated_tool_call_uses_stable_message_id(self, middleware):
request = SimpleNamespace(
tool_call={
"name": "ask_clarification",
"id": "call-clarify-1",
"args": {
"question": "Which environment should I use?",
"clarification_type": "approach_choice",
"options": ["dev", "prod"],
},
}
)
first = middleware.wrap_tool_call(request, lambda _req: pytest.fail("handler should not be called"))
second = middleware.wrap_tool_call(request, lambda _req: pytest.fail("handler should not be called"))
first_message = first.update["messages"][0]
second_message = second.update["messages"][0]
assert first_message.id == "clarification:call-clarify-1"
assert second_message.id == first_message.id
assert second_message.tool_call_id == first_message.tool_call_id
assert first_message.artifact["human_input"]["request_id"] == "clarification:call-clarify-1"
assert first_message.artifact["human_input"]["tool_call_id"] == "call-clarify-1"
assert first_message.artifact["human_input"]["clarification_type"] == "approach_choice"
assert first_message.artifact["human_input"]["input_mode"] == "choice_with_other"
merged = add_messages(add_messages([], [first_message]), [second_message])
assert len(merged) == 1
assert merged[0].id == "clarification:call-clarify-1"
assert merged[0].content == first_message.content
assert merged[0].artifact == first_message.artifact
def test_tool_message_model_dump_preserves_human_input_artifact(self, middleware):
request = SimpleNamespace(
tool_call={
"name": "ask_clarification",
"id": "call-clarify-1",
"args": {
"question": "Which environment should I use?",
"clarification_type": "approach_choice",
"options": ["dev", "prod"],
},
}
)
result = middleware.wrap_tool_call(request, lambda _req: pytest.fail("handler should not be called"))
message = result.update["messages"][0]
dumped = message.model_dump()
assert dumped["artifact"]["human_input"]["request_id"] == "clarification:call-clarify-1"
assert dumped["artifact"]["human_input"]["options"] == [
{"id": "option-1", "label": "dev", "value": "dev"},
{"id": "option-2", "label": "prod", "value": "prod"},
]
assert "Which environment should I use?" in dumped["content"]
class TestClarificationDisabled:
"""When ``disable_clarification`` is set in runtime context, a clarification
must NOT interrupt the run — it returns a ToolMessage nudging the agent to
proceed, so non-interactive channels (GitHub) don't dead-end."""
def _request(self, *, runtime_context):
return SimpleNamespace(
tool_call={
"name": "ask_clarification",
"id": "call-clarify-1",
"args": {"question": "Should I create the issue?", "clarification_type": "suggestion"},
},
runtime=SimpleNamespace(context=runtime_context),
)
def test_disabled_returns_toolmessage_not_command(self, middleware):
request = self._request(runtime_context={"disable_clarification": True})
result = middleware.wrap_tool_call(request, lambda _req: pytest.fail("handler should not be called"))
# Not a Command(goto=END) — a plain ToolMessage so the loop continues.
from langchain_core.messages import ToolMessage
assert isinstance(result, ToolMessage)
assert result.tool_call_id == "call-clarify-1"
assert result.artifact is None
def test_disabled_message_tells_agent_to_proceed(self, middleware):
request = self._request(runtime_context={"disable_clarification": True})
result = middleware.wrap_tool_call(request, lambda _req: pytest.fail("handler should not be called"))
assert "disabled" in result.content.lower()
assert "proceed" in result.content.lower()
def test_disabled_async_path(self, middleware):
request = self._request(runtime_context={"disable_clarification": True})
async def handler(_req):
return pytest.fail("handler should not be called")
import asyncio
result = asyncio.run(middleware.awrap_tool_call(request, handler))
from langchain_core.messages import ToolMessage
assert isinstance(result, ToolMessage)
def test_not_disabled_still_interrupts(self, middleware):
"""Without the flag, the original goto=END behavior is preserved."""
from langgraph.types import Command
request = self._request(runtime_context={}) # no disable_clarification
result = middleware.wrap_tool_call(request, lambda _req: pytest.fail("handler should not be called"))
assert isinstance(result, Command)
assert result.goto == "__end__"
def test_no_runtime_context_still_interrupts(self, middleware):
"""Defensive: missing runtime/context falls back to interrupting."""
from langgraph.types import Command
request = SimpleNamespace(
tool_call={
"name": "ask_clarification",
"id": "c1",
"args": {"question": "q?", "clarification_type": "missing_info"},
},
runtime=None,
)
result = middleware.wrap_tool_call(request, lambda _req: pytest.fail("handler should not be called"))
assert isinstance(result, Command)
def test_non_clarification_tool_call_unaffected_by_flag(self, middleware):
"""The flag only affects ask_clarification; other tools run normally."""
other = SimpleNamespace(
tool_call={"name": "bash", "id": "b1", "args": {"command": "echo hi"}},
runtime=SimpleNamespace(context={"disable_clarification": True}),
)
sentinel = "ran"
result = middleware.wrap_tool_call(other, lambda _req: sentinel)
assert result == sentinel
def test_missing_tool_call_id_still_gets_stable_message_id(self, middleware):
request = SimpleNamespace(
tool_call={
"name": "ask_clarification",
"args": {
"question": "Which environment should I use?",
"clarification_type": "missing_info",
},
}
)
first = middleware.wrap_tool_call(request, lambda _req: pytest.fail("handler should not be called"))
second = middleware.wrap_tool_call(request, lambda _req: pytest.fail("handler should not be called"))
first_message = first.update["messages"][0]
second_message = second.update["messages"][0]
assert first_message.id.startswith("clarification:")
assert second_message.id == first_message.id
merged = add_messages(add_messages([], [first_message]), [second_message])
assert len(merged) == 1