fix(models): reuse the Claude Code OAuth token read from a file descriptor (#5411)

* fix(models): reuse the Claude Code OAuth token read from a file descriptor

ClaudeChatModel accepts a Claude Code OAuth token through
CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR, but that handoff can be drained
only once: a pipe returns EOF after the first read and a file descriptor
keeps its advanced offset. _read_secret_from_file_descriptor read it again
on every call and kept nothing. Every ClaudeChatModel instance loads
credentials in model_post_init, and create_chat_model builds fresh
instances per run, so with a descriptor-only handoff the first model
authenticated and every model after it -- including the title model of
the very first run -- had no credential. The Anthropic SDK then raised
"Could not resolve authentication method" before sending a request.

A secret read from a descriptor is now kept for the life of the process,
keyed by (env_var, fd), so a different descriptor is still read fresh.
The read happens under a lock so two threads building their first model
concurrently cannot race one of them to EOF. Empty reads and OSError are
not cached and behave as before; lookup order, config keys, and log
messages are unchanged.

* docs(changelog): reference #5411 in the Claude Code OAuth descriptor fix entry

* test(models): pin that a closed descriptor handoff keeps its token

Review follow-up on #5411: the descriptor secret cache is keyed on the
fd number, which the OS recycles. Folding os.fstat identity into the key
would break the property the cache exists for -- once the handoff fd is
closed after the first read, fstat raises EBADF and every later model
would lose the token again -- and it would still miss a regular file
rewritten in place, which keeps its st_dev/st_ino.

The handoff is fixed at process start, so keep the number as the key and
state the invariant instead: a closed handoff keeps serving its token, a
secret placed on a recycled number is not re-read, and anything handing
over a new secret in-process must clear the cache. A new test pins the
closed-handoff behavior; an fstat-fingerprinted key fails exactly that
test.
This commit is contained in:
Hyeonsang Cho 2026-09-14 10:23:12 +09:00 committed by GitHub
parent d5ae3882b6
commit 1dd48d14d2
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
6 changed files with 171 additions and 6 deletions

View File

@ -582,6 +582,13 @@ This section accumulates work toward the **2.1.0** milestone
### Fixed
- **models:** Stop every Claude model after the first from losing its
credential when the Claude Code OAuth token is handed off through
`CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR`. Every `ClaudeChatModel` instance
loaded credentials again, but a descriptor can be drained only once, so the
title, summarization, and subagent models — and every later run — had no
credential and failed with `TypeError: Could not resolve authentication
method`. The token is now read once per process and reused. ([#5411])
- **models:** Stop the lead agent from failing to build whenever a model with
`supports_reasoning_effort: true` also gets a `reasoning_effort` from its
profile — at the top level, in `when_thinking_enabled` or
@ -2796,3 +2803,4 @@ with **180 merged pull requests** since the first 2.0 milestone tag.
[#5393]: https://github.com/bytedance/deer-flow/pull/5393
[#5401]: https://github.com/bytedance/deer-flow/pull/5401
[#5403]: https://github.com/bytedance/deer-flow/pull/5403
[#5411]: https://github.com/bytedance/deer-flow/pull/5411

View File

@ -397,6 +397,10 @@
### 修复
- **模型:** 通过 `CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR` 传递 Claude Code OAuth
token 时,第一个之后的 Claude 模型不再丢失凭据。每个 `ClaudeChatModel` 实例都会重新加载凭据,
但文件描述符只能读取一次导致标题、摘要、subagent 模型以及之后的每次运行都没有凭据,并以
`TypeError: Could not resolve authentication method` 失败。现在 token 在每个进程中只读取一次并复用。([#5411])
- **模型:**`supports_reasoning_effort: true` 的模型同时从 profile 获得
`reasoning_effort`(顶层、`when_thinking_enabled``when_thinking_disabled`
中,或由 `extra_body.thinking` 的关闭路径注入lead agent 不再构建失败。lead
@ -2143,3 +2147,4 @@ DeerFlow 2.0 是围绕"超级智能体"框架的彻底重写,核心包含子
[#5393]: https://github.com/bytedance/deer-flow/pull/5393
[#5401]: https://github.com/bytedance/deer-flow/pull/5401
[#5403]: https://github.com/bytedance/deer-flow/pull/5403
[#5411]: https://github.com/bytedance/deer-flow/pull/5411

View File

@ -86,6 +86,7 @@ models:
- `CodexChatModel` loads Codex CLI auth from `~/.codex/auth.json`
- The Codex Responses endpoint currently rejects `max_tokens` and `max_output_tokens`, so `CodexChatModel` does not expose a request-level token cap
- `ClaudeChatModel` accepts `CLAUDE_CODE_OAUTH_TOKEN`, `ANTHROPIC_AUTH_TOKEN`, `CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR`, `CLAUDE_CODE_CREDENTIALS_PATH`, or plaintext `~/.claude/.credentials.json`
- A `CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR` handoff is drained on first use and the token is kept for the life of the process, so every `ClaudeChatModel` instance reuses it
- On macOS, DeerFlow does not probe Keychain automatically. Use `scripts/export_claude_code_oauth.py` to export Claude Code auth explicitly when needed
To use OpenAI's `/v1/responses` endpoint with LangChain, keep using `langchain_openai:ChatOpenAI` and set:

View File

@ -8,6 +8,11 @@
- Config values starting with `$` resolved as environment variables
- Missing provider modules surface actionable install hints from reflection resolvers (for example `uv add langchain-google-genai`)
### Claude Code Credentials (`packages/harness/deerflow/models/credential_loader.py`)
- `ClaudeChatModel.model_post_init` calls `load_claude_code_credential()` for every instance, and `create_chat_model` builds fresh instances per run (lead agent, title, summarization, subagents)
- `$CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR` is a one-shot handoff: a pipe returns EOF and a file keeps its advanced offset. `_read_secret_from_file_descriptor` therefore caches a non-empty secret per `(env_var, fd)` under a lock held across the read. Do not drop the cache or the lock — later instances would get no credential, and the Anthropic SDK raises `TypeError: Could not resolve authentication method` before sending. Empty reads and `OSError` are not cached. The key is the descriptor number on purpose — a closed handoff keeps serving its token, and a secret placed on a recycled number in-process is not re-read unless the cache is cleared. The cache is per process, so a new process (e.g. a uvicorn `--reload` worker) cannot recover a drained descriptor. Pinned by `tests/test_credential_loader.py`, including a two-instance `ClaudeChatModel` test
### vLLM Provider (`packages/harness/deerflow/models/vllm_provider.py`)
- `VllmChatModel` subclasses `langchain_openai:ChatOpenAI` for vLLM 0.19.0 OpenAI-compatible endpoints

View File

@ -15,6 +15,7 @@ Implements two credential strategies:
import json
import logging
import os
import threading
import time
from dataclasses import dataclass
from pathlib import Path
@ -25,6 +26,18 @@ logger = logging.getLogger(__name__)
# Required beta headers for Claude Code OAuth tokens
OAUTH_ANTHROPIC_BETAS = "oauth-2025-04-20,claude-code-20250219,interleaved-thinking-2025-05-14"
# A descriptor handoff can be drained only once: a pipe returns EOF and a file
# keeps its advanced offset. Every ClaudeChatModel instance loads credentials, so
# secrets read from a descriptor are kept for the life of the process.
#
# The key is the descriptor number, not its identity: the handoff is fixed when
# the process starts, so a number means one secret for the process lifetime. That
# keeps the token available after the descriptor is closed, but a secret later
# placed on a recycled number is not read. Anything that hands over a new secret
# in-process must clear this cache.
_fd_secret_cache: dict[tuple[str, int], str] = {}
_fd_secret_lock = threading.Lock()
def is_oauth_token(token: str) -> bool:
"""Check if a token is a Claude Code OAuth token (not a standard API key)."""
@ -96,13 +109,22 @@ def _read_secret_from_file_descriptor(env_var: str) -> str | None:
logger.warning(f"{env_var} must be an integer file descriptor, got: {fd_value}")
return None
try:
secret = os.read(fd, 1024 * 1024).decode().strip()
except OSError as e:
logger.warning(f"Failed to read {env_var}: {e}")
return None
# Hold the lock across the read so concurrent first loads cannot race to EOF.
with _fd_secret_lock:
cached = _fd_secret_cache.get((env_var, fd))
if cached is not None:
return cached
return secret or None
try:
secret = os.read(fd, 1024 * 1024).decode().strip()
except OSError as e:
logger.warning(f"Failed to read {env_var}: {e}")
return None
if not secret:
return None
_fd_secret_cache[(env_var, fd)] = secret
return secret
def _credential_from_direct_token(access_token: str, source: str) -> ClaudeCodeCredential | None:

View File

@ -1,12 +1,25 @@
import json
import os
import threading
import time
from concurrent.futures import ThreadPoolExecutor
import pytest
from deerflow.models import credential_loader
from deerflow.models.claude_provider import ClaudeChatModel
from deerflow.models.credential_loader import (
load_claude_code_credential,
load_codex_cli_credential,
)
@pytest.fixture(autouse=True)
def _isolate_file_descriptor_secret_cache(monkeypatch):
# Descriptor numbers are recycled across tests, so a shared cache would leak tokens.
monkeypatch.setattr(credential_loader, "_fd_secret_cache", {})
def _clear_claude_code_env(monkeypatch) -> None:
for env_var in (
"CLAUDE_CODE_OAUTH_TOKEN",
@ -59,6 +72,117 @@ def test_load_claude_code_credential_from_file_descriptor(monkeypatch):
assert cred.source == "claude-cli-fd"
def _pipe_with_secret(secret: bytes) -> int:
read_fd, write_fd = os.pipe()
os.write(write_fd, secret)
os.close(write_fd)
return read_fd
def test_load_claude_code_credential_reuses_drained_file_descriptor(tmp_path, monkeypatch):
_clear_claude_code_env(monkeypatch)
monkeypatch.setenv("HOME", str(tmp_path))
read_fd = _pipe_with_secret(b"sk-ant-oat01-fd")
try:
monkeypatch.setenv("CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR", str(read_fd))
first = load_claude_code_credential()
second = load_claude_code_credential()
finally:
os.close(read_fd)
assert first is not None
assert second is not None
assert second.access_token == first.access_token == "sk-ant-oat01-fd"
assert second.source == "claude-cli-fd"
def test_load_claude_code_credential_rereads_when_file_descriptor_changes(monkeypatch):
_clear_claude_code_env(monkeypatch)
first_fd = _pipe_with_secret(b"sk-ant-oat01-first")
second_fd = _pipe_with_secret(b"sk-ant-oat01-second")
try:
monkeypatch.setenv("CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR", str(first_fd))
first = load_claude_code_credential()
monkeypatch.setenv("CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR", str(second_fd))
second = load_claude_code_credential()
finally:
os.close(first_fd)
os.close(second_fd)
assert first is not None and first.access_token == "sk-ant-oat01-first"
assert second is not None and second.access_token == "sk-ant-oat01-second"
def test_load_claude_code_credential_survives_closed_file_descriptor(tmp_path, monkeypatch):
_clear_claude_code_env(monkeypatch)
monkeypatch.setenv("HOME", str(tmp_path))
read_fd = _pipe_with_secret(b"sk-ant-oat01-fd")
monkeypatch.setenv("CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR", str(read_fd))
first = load_claude_code_credential()
# Closing a drained handoff must not strand the models built after it.
os.close(read_fd)
second = load_claude_code_credential()
assert first is not None and first.access_token == "sk-ant-oat01-fd"
assert second is not None and second.access_token == "sk-ant-oat01-fd"
def test_concurrent_loads_drain_file_descriptor_once(tmp_path, monkeypatch):
_clear_claude_code_env(monkeypatch)
monkeypatch.setenv("HOME", str(tmp_path))
read_fd = _pipe_with_secret(b"sk-ant-oat01-fd")
real_read = os.read
handoff_reads = []
def slow_read(fd, length):
if fd == read_fd:
handoff_reads.append(fd)
# Hold the first reader inside os.read so an unguarded second reader drains EOF.
time.sleep(0.1)
return real_read(fd, length)
monkeypatch.setattr(os, "read", slow_read)
barrier = threading.Barrier(2)
def load():
barrier.wait()
return load_claude_code_credential()
try:
monkeypatch.setenv("CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR", str(read_fd))
with ThreadPoolExecutor(max_workers=2) as pool:
creds = list(pool.map(lambda _: load(), range(2)))
finally:
os.close(read_fd)
assert [cred.access_token if cred else None for cred in creds] == ["sk-ant-oat01-fd"] * 2
assert handoff_reads == [read_fd]
def test_claude_chat_model_instances_share_file_descriptor_token(tmp_path, monkeypatch):
_clear_claude_code_env(monkeypatch)
monkeypatch.delenv("ANTHROPIC_API_KEY", raising=False)
monkeypatch.setenv("HOME", str(tmp_path))
read_fd = _pipe_with_secret(b"sk-ant-oat01-fd")
try:
monkeypatch.setenv("CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR", str(read_fd))
# Each run builds fresh models (lead agent, title, subagents) from the same handoff.
models = [ClaudeChatModel(model="claude-sonnet-4-6") for _ in range(2)]
finally:
os.close(read_fd)
for model in models:
assert model._is_oauth is True
assert model._client.api_key is None
assert model._client.auth_token == "sk-ant-oat01-fd"
def test_load_claude_code_credential_from_override_path(tmp_path, monkeypatch):
_clear_claude_code_env(monkeypatch)
cred_path = tmp_path / "claude-credentials.json"