mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-09-11 06:28:58 +00:00
* fix(mcp): reject credentials that cannot travel as HTTP header values
A request-scoped secret or user_auth credential with a trailing newline
(the usual result of reading a token from a file, or a CRLF env-file),
CR/LF, surrounding whitespace, or characters outside Latin-1 sailed
through the credential interceptors into the HTTP client, where httpx/h11
reject it with an exception that echoes the full value:
LocalProtocolError: Illegal header value b'Bearer sk-...\n'
ToolErrorHandlingMiddleware copies that message into a model-visible
ToolMessage, so the secret landed in the prompt, the checkpoint, and
traces - everywhere headers_from_context promises it never goes.
Add illegal_header_value_reason to mcp/headers.py, mirroring the
transport's own rules (Latin-1 encodable; h11's field_vchar is [^\x00\s]
with SP/HTAB legal only between visible characters), and fail closed in
both interceptors before the value can reach the client. The denial names
only the secret key (plus the reason) and never repeats the value.
Illegal values are denied regardless of on_missing: the key is present,
so a passthrough fallback would silently run the call under the shared
discovery credential - the exact authority confusion the deny default
exists to prevent.
Values the transport accepts are not rejected: embedded SP/HTAB
('Bearer <token>'), Latin-1 high bytes, and DEL all still pass, pinned
by tests against h11's observed behaviour.
* fix(mcp): tighten header value validation to httpx's ASCII boundary
The validator mirrored h11's Latin-1 boundary, but the transport rejects
more than h11 does: build_server_params hands dict[str, str] headers
through the MCP SDK's create_mcp_http_client into httpx.AsyncClient, and
httpx (pinned 0.28.1) encodes str header values as ASCII - so a Latin-1
high byte like 'Bearer caf\xe9' passed validation here only to raise
UnicodeEncodeError inside httpx before h11 ever ran, with the exception
message repeating the offending value.
Validate str values against ASCII instead, flip the tests that pinned
Latin-1 high bytes as transportable, and pin the boundary against the
real client: create_mcp_http_client must reject what the validator
flags and construct cleanly for what it accepts (embedded SP/HTAB and
DEL still pass).
Addresses review feedback on the ASCII vs Latin-1 boundary.
* fix(mcp): validate OAuth and static header values at the same boundary
The validator added for headers_from_context and user_auth left two paths
uncovered. A token endpoint returning an access_token or token_type with a
newline reached httpx/h11, which raise with the full token in the message, and
ToolErrorHandlingMiddleware copies that message into a model-visible
ToolMessage -- the leak this PR set out to close. The operator's static headers
had the same hole.
OAuthTokenManager.get_authorization_header now renders the Authorization value
through one checked helper, so the tool interceptor, the initial discovery
headers and the durable task path are all covered by a single guard. The
rendered value is what gets checked rather than the two fields separately,
because that is what the transport sees: an access_token with leading
whitespace is legal once it follows "Bearer ".
build_server_params applies the same check to statically configured headers.
build_servers_config already isolates a per-server failure, so a bad value
drops that one server and logs the reason instead of the value.
* docs(mcp): correct which transport echoes the full header value
The rationale claimed httpx and h11 both render the full value into their
exception message. Only h11 does, on the line break and surrounding whitespace
cases. httpx's ASCII failure is a UnicodeEncodeError naming the offending
character and its position, not the credential, so at most one character
escapes there; refusing the value up front buys an actionable error rather than
an encode failure raised from inside the client.
Corrected in headers.py and in every copy of the claim: context_headers.py,
user_scoped_auth.py, oauth.py, client.py, mcp/AGENTS.md, docs/MCP_SERVER.md,
the frontend mcp.mdx, and the test comments carrying the same wording. No
behavior change.
---------
Co-authored-by: Terminator666666 <Terminator666666@users.noreply.github.com>
207 lines
10 KiB
Python
207 lines
10 KiB
Python
"""Per-request credential injection for shared MCP servers.
|
|
|
|
``user_auth`` binds a credential to a *configured* DeerFlow user, which forces
|
|
one MCP server entry per credential when the credential is chosen by the caller
|
|
at request time (multi-tenant gateways, per-run API keys). This module closes
|
|
that gap: a server opts in by declaring a ``headers_from_context`` block
|
|
(:class:`McpContextHeadersConfig`) mapping HTTP header names to keys of the run
|
|
request's ``config.context.secrets`` carrier. On every tool call the interceptor
|
|
resolves the mapping from the live run context and rewrites those headers via
|
|
``request.override(headers=...)`` — the same per-call mechanism the OAuth and
|
|
user-scoped auth interceptors use.
|
|
|
|
The secret values arrive out-of-band with the run request and stay there: they
|
|
are never rendered into the prompt, the tool arguments, or trace payloads (see
|
|
``runtime/secret_context.py``). Only the *names* live in the config file, so no
|
|
credential is written to disk or returned by the config API.
|
|
|
|
Registered last in ``mcp/interceptors.py``, so for a server declaring several
|
|
credential sources the per-request value wins the final header — interceptors
|
|
wrap outermost-first, and the later-registered one runs closer to the transport.
|
|
Header names are written case-insensitively through ``mcp/headers.py``, so a
|
|
mapped ``Authorization`` replaces a static ``authorization`` rather than putting
|
|
a second copy of the field on the wire ahead of it.
|
|
|
|
Fail-closed by default: a mapped key that is absent from the request secrets
|
|
(or resolved empty) gets an actionable ``ToolException`` rather than silently
|
|
falling back to the server's static discovery credential, which in a
|
|
multi-tenant deployment would send one tenant's request under another
|
|
tenant's authority. ``on_missing: "passthrough"`` is the explicit opt-out.
|
|
|
|
A resolved value the transport would refuse (line break, surrounding
|
|
whitespace, non-ASCII — see ``mcp/headers.py``) is always denied, regardless
|
|
of ``on_missing``. h11 renders the full value into its exception message when
|
|
it refuses a line break or surrounding whitespace, and that message would
|
|
otherwise travel into a model-visible tool error and the trace; the non-ASCII
|
|
case fails inside httpx instead, which names only the offending character, so
|
|
denying it here buys an actionable error rather than secrecy. Either way a
|
|
passthrough fallback would run the call under the shared discovery credential
|
|
even though the caller did supply a key.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import logging
|
|
from typing import Any
|
|
|
|
from langchain_core.tools import ToolException
|
|
|
|
from deerflow.config.extensions_config import ExtensionsConfig, McpContextHeadersConfig
|
|
from deerflow.mcp.headers import (
|
|
apply_header_overrides,
|
|
header_spellings,
|
|
illegal_header_value_reason,
|
|
)
|
|
from deerflow.runtime.secret_context import extract_request_secrets
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
|
|
def _current_runtime() -> Any | None:
|
|
"""Best-effort access to the LangGraph runtime for the current tool call.
|
|
|
|
``get_runtime()`` raises outside a runtime context (embedded clients, unit
|
|
tests, discovery paths). Mirrors ``mcp/user_scoped_auth.py``: a failure here
|
|
only means the request carries no resolvable secrets, which the caller then
|
|
handles through ``on_missing``.
|
|
"""
|
|
try:
|
|
from langgraph.runtime import get_runtime
|
|
|
|
return get_runtime()
|
|
except Exception:
|
|
return None
|
|
|
|
|
|
def _request_secrets(request: Any) -> dict[str, str]:
|
|
"""Return the run request's ``config.context.secrets``, or ``{}``.
|
|
|
|
Prefer the runtime attached to the request: LangGraph's tool node injects it
|
|
into any tool parameter named ``runtime``, which covers both the pooled
|
|
stdio wrapper and ``langchain_mcp_adapters``' own HTTP/SSE tool. Fall back to
|
|
the ambient runtime for call paths outside a tool node.
|
|
|
|
Deliberately not read from ``langgraph.config.get_config()``: the run context
|
|
is carried on the runtime, not on the ``RunnableConfig`` propagated to child
|
|
runnables, so ``get_config().get("context")`` is ``None`` inside a tool call.
|
|
"""
|
|
runtime = getattr(request, "runtime", None)
|
|
if runtime is None:
|
|
runtime = _current_runtime()
|
|
return extract_request_secrets(getattr(runtime, "context", None))
|
|
|
|
|
|
def build_context_headers_interceptor(extensions_config: ExtensionsConfig) -> Any | None:
|
|
"""Build a tool interceptor injecting per-request headers, or ``None``.
|
|
|
|
Returns ``None`` when no enabled server declares a usable
|
|
``headers_from_context`` block, so callers can skip registration entirely
|
|
(mirrors ``build_oauth_tool_interceptor`` / ``build_user_scoped_auth_interceptor``).
|
|
"""
|
|
mapping_by_server: dict[str, McpContextHeadersConfig] = {}
|
|
# The server's static header spellings, so a mapped name that differs from
|
|
# the configured one only in case still *replaces* it at the adapter's
|
|
# case-sensitive connection merge instead of riding alongside it.
|
|
spellings_by_server: dict[str, dict[str, str]] = {}
|
|
for server_name, server_config in extensions_config.get_enabled_mcp_servers().items():
|
|
context_headers = server_config.headers_from_context
|
|
if context_headers is None or not context_headers.enabled or not context_headers.headers:
|
|
continue
|
|
if server_config.type not in ("sse", "http"):
|
|
# A stdio server has no HTTP headers: the pooled stdio path forwards
|
|
# rewritten headers as call meta, never a transport header, so the
|
|
# credential would go nowhere while deny errors still fired for
|
|
# runs that carry no secrets. Warn-and-skip matches user_auth.
|
|
logger.warning(
|
|
"MCP server '%s' declares headers_from_context but uses the '%s' transport; request-scoped headers only apply to 'sse'/'http' servers — ignoring headers_from_context for this server",
|
|
server_name,
|
|
server_config.type,
|
|
)
|
|
continue
|
|
if server_config.task_toolsets:
|
|
# Submitting a durable task happens inside the Agent run and carries
|
|
# the request secrets; the later status/cancel polls do not, because
|
|
# the task runtime drives them long after that run ended. Those calls
|
|
# deliberately skip this interceptor (see McpTaskToolCaller), so the
|
|
# background half authenticates with the server's own credentials.
|
|
logger.warning(
|
|
"MCP server '%s' declares both headers_from_context and task_toolsets; background task status/cancel polls run outside an Agent run and will use this server's static/OAuth credentials instead of the per-request headers",
|
|
server_name,
|
|
)
|
|
mapping_by_server[server_name] = context_headers
|
|
spellings_by_server[server_name] = header_spellings(server_config.headers)
|
|
|
|
if not mapping_by_server:
|
|
return None
|
|
|
|
async def context_headers_interceptor(request: Any, handler: Any) -> Any:
|
|
context_headers = mapping_by_server.get(request.server_name)
|
|
if context_headers is None:
|
|
return await handler(request)
|
|
|
|
secrets = _request_secrets(request)
|
|
resolved: dict[str, str] = {}
|
|
missing: list[str] = []
|
|
illegal: dict[str, str] = {}
|
|
for header_name, secret_key in context_headers.headers.items():
|
|
# Empty string covers a caller-side `$ENV_VAR` that was unset: an
|
|
# empty credential must fail closed rather than send an empty header.
|
|
value = secrets.get(secret_key, "")
|
|
if not value:
|
|
missing.append(secret_key)
|
|
continue
|
|
# A value the transport would refuse (trailing newline from reading
|
|
# a token file, CR/LF, non-ASCII) must be rejected *here*. h11
|
|
# renders the full value into its exception message on the line
|
|
# break and whitespace cases, and ToolErrorHandlingMiddleware
|
|
# copies that message into a model-visible ToolMessage — putting
|
|
# the secret in the prompt, the checkpoint, and traces, everywhere
|
|
# this module promises it never goes. Always denied, regardless of
|
|
# on_missing: the key is present, so falling back to the discovery
|
|
# credential would silently run this tenant's call under the shared
|
|
# authority.
|
|
reason = illegal_header_value_reason(value)
|
|
if reason is not None:
|
|
illegal[secret_key] = reason
|
|
continue
|
|
resolved[header_name] = value
|
|
|
|
if illegal:
|
|
illegal_keys = ", ".join(sorted(illegal))
|
|
logger.warning(
|
|
"Denied MCP tool call to server '%s': request-scoped secret(s) %s cannot be sent as an HTTP header value",
|
|
request.server_name,
|
|
illegal_keys,
|
|
)
|
|
details = "; ".join(f"'{key}' {reason}" for key, reason in sorted(illegal.items()))
|
|
# Like the missing-key denial below, only the configured key names
|
|
# (plus the reason) are surfaced — never the value.
|
|
raise ToolException(
|
|
f"MCP server '{request.server_name}' cannot send request-scoped credential(s) as HTTP header values: {details}. "
|
|
"Fix the value passed in config.context.secrets; a stray newline picked up when reading a token from a file is the usual cause."
|
|
)
|
|
|
|
if missing and context_headers.on_missing == "deny":
|
|
missing_keys = ", ".join(sorted(missing))
|
|
logger.warning(
|
|
"Denied MCP tool call to server '%s': request context is missing secret(s) %s",
|
|
request.server_name,
|
|
missing_keys,
|
|
)
|
|
# Only the configured *key names* are surfaced — they already live in
|
|
# the config file, so this leaks nothing the operator has not written
|
|
# down, while telling the caller exactly what to send.
|
|
raise ToolException(f"MCP server '{request.server_name}' needs request-scoped credential(s) {missing_keys}. Send them in config.context.secrets, or set this server's headers_from_context.on_missing to 'passthrough'.")
|
|
|
|
if not resolved:
|
|
return await handler(request)
|
|
|
|
updated_headers = apply_header_overrides(
|
|
request.headers,
|
|
resolved,
|
|
spellings=spellings_by_server.get(request.server_name),
|
|
)
|
|
return await handler(request.override(headers=updated_headers))
|
|
|
|
return context_headers_interceptor
|