fix(console): disable cost reporting when model pricing mixes currencies (#4564)

The console cost estimator summed per-run spend across every priced model
without checking that they shared a currency, so a deployment that priced
one model in CNY and another in USD produced a meaningless aggregate under
a single currency label. Track the first priced currency and, on a
mismatch, log a warning and drop all pricing so cost/currency fields report
null instead of an invalid sum. Currency codes are now trimmed and
case-normalized so equivalent spellings don't false-trip the guard.
This commit is contained in:
Ryker_Feng 2026-07-29 22:24:04 +08:00 committed by GitHub
parent 6b5f5e789a
commit 594dbe1205
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
5 changed files with 92 additions and 4 deletions

View File

@ -138,6 +138,10 @@ That prompt is intended for coding agents. It tells the agent to clone the repo
> **Advanced / manual configuration**: If you prefer to edit `config.yaml` directly, run `make config` instead to copy the full template. See `config.example.yaml` for the complete reference including CLI-backed providers (Codex CLI, Claude Code OAuth), OpenRouter, Responses API, subagent runtime caps such as `subagents.max_total_per_run`, and more. > **Advanced / manual configuration**: If you prefer to edit `config.yaml` directly, run `make config` instead to copy the full template. See `config.example.yaml` for the complete reference including CLI-backed providers (Codex CLI, Claude Code OAuth), OpenRouter, Responses API, subagent runtime caps such as `subagents.max_total_per_run`, and more.
Optional per-model pricing must use one currency across all priced models.
DeerFlow disables Console cost estimates when currencies are mixed rather
than presenting an invalid aggregate.
<details> <details>
<summary>Manual model configuration examples</summary> <summary>Manual model configuration examples</summary>

View File

@ -452,7 +452,7 @@ Localhost persistence deliberately reads the direct request `Host` and ignores `
|--------|-----------| |--------|-----------|
| **Models** (`/api/models`) | `GET /` - list models; `GET /{name}` - model details | | **Models** (`/api/models`) | `GET /` - list models; `GET /{name}` - model details |
| **Features** (`/api/features`) | `GET /` - report config-gated feature availability (`agents_api.enabled`, `browser_control.enabled`) for frontend UI gating | | **Features** (`/api/features`) | `GET /` - report config-gated feature availability (`agents_api.enabled`, `browser_control.enabled`) for frontend UI gating |
| **Console** (`/api/console`) | Read-only cross-thread observability for the current user (the data layer for an operations dashboard or external monitoring): `GET /stats` - headline counters (runs/threads/agents/tokens/cost); `GET /runs` - paginated run history joined with thread titles (per-run cost); `GET /usage` - zero-filled daily token series + per-model breakdown with spend. Queries `runs`/`threads_meta` directly as a reporting layer (no new `RunStore` methods); requires a SQL database backend — returns 503 on `database.backend: memory`. Real-cost estimation reads optional `models[*].pricing` (`currency`, `input_per_million`, `output_per_million`, `input_cache_hit_per_million`; `ModelConfig` is `extra="allow"`, so no schema change) and prices each run from its `token_usage_by_model` input/output split. Pricing is **cache-aware**: `RunJournal` accumulates prompt-cache hits from `usage_metadata.input_token_details.cache_read` into a sparse `cache_read_tokens` bucket key (also threaded through `SubagentTokenCollector``record_external_llm_usage_records`), and cache-hit input tokens are billed at `input_cache_hit_per_million` (omitted → billed at the miss price, a conservative upper bound). Legacy rows fall back to run-level totals at `model_name`; unpriced models yield `cost: null` and cost fields are null when no pricing is configured | | **Console** (`/api/console`) | Read-only cross-thread observability for the current user (the data layer for an operations dashboard or external monitoring): `GET /stats` - headline counters (runs/threads/agents/tokens/cost); `GET /runs` - paginated run history joined with thread titles (per-run cost); `GET /usage` - zero-filled daily token series + per-model breakdown with spend. Queries `runs`/`threads_meta` directly as a reporting layer (no new `RunStore` methods); requires a SQL database backend — returns 503 on `database.backend: memory`. Real-cost estimation reads optional `models[*].pricing` (`currency`, `input_per_million`, `output_per_million`, `input_cache_hit_per_million`; `ModelConfig` is `extra="allow"`, so no schema change) and prices each run from its `token_usage_by_model` input/output split. Pricing is **cache-aware**: `RunJournal` accumulates prompt-cache hits from `usage_metadata.input_token_details.cache_read` into a sparse `cache_read_tokens` bucket key (also threaded through `SubagentTokenCollector``record_external_llm_usage_records`), and cache-hit input tokens are billed at `input_cache_hit_per_million` (omitted → billed at the miss price, a conservative upper bound). All priced models must use one currency; mixed currencies disable cost reporting and leave cost/currency fields null instead of producing invalid aggregates. Legacy rows fall back to run-level totals at `model_name`; unpriced models yield `cost: null` and cost fields are null when no pricing is configured |
| **MCP** (`/api/mcp`) | `GET /config` - get config; `PUT /config` - update config (saves to extensions_config.json) | | **MCP** (`/api/mcp`) | `GET /config` - get config; `PUT /config` - update config (saves to extensions_config.json) |
| **Skills** (`/api/skills`) | `GET /` - list skills; `GET /{name}` - details; `PUT /{name}` - update enabled; `POST /install` - install from .skill archive (accepts standard optional frontmatter like `version`, `author`, `compatibility`); `POST /reload` - admin-only process-local prompt-cache invalidation after trusted external filesystem changes | | **Skills** (`/api/skills`) | `GET /` - list skills; `GET /{name}` - details; `PUT /{name}` - update enabled; `POST /install` - install from .skill archive (accepts standard optional frontmatter like `version`, `author`, `compatibility`); `POST /reload` - admin-only process-local prompt-cache invalidation after trusted external filesystem changes |
| **Integrations** (`/api/integrations`) | `GET /lark/status` - inspect managed Lark/Feishu CLI integration state, including `sandbox_runtime_mode` / `sandbox_runtime_ready` (whether `lark-cli` will actually be present in the sandbox at chat time); `POST /lark/install` - admin-only install of the official `lark-*` managed skill pack; `POST /lark/config/start` and `/lark/config/complete` - internal first-time Lark connection setup; `POST /lark/auth/start` and `/lark/auth/complete` - browser device-flow user authorization without terminal access, with optional `domains` / exact `scope` for incremental permission grants | | **Integrations** (`/api/integrations`) | `GET /lark/status` - inspect managed Lark/Feishu CLI integration state, including `sandbox_runtime_mode` / `sandbox_runtime_ready` (whether `lark-cli` will actually be present in the sandbox at chat time); `POST /lark/install` - admin-only install of the official `lark-*` managed skill pack; `POST /lark/config/start` and `/lark/config/complete` - internal first-time Lark connection setup; `POST /lark/auth/start` and `/lark/auth/complete` - browser device-flow user authorization without terminal access, with optional `domains` / exact `scope` for incremental permission grants |

View File

@ -167,6 +167,8 @@ def _build_pricing_map() -> dict[str, _ModelPricing]:
return {} return {}
pricing: dict[str, _ModelPricing] = {} pricing: dict[str, _ModelPricing] = {}
pricing_currency: str | None = None
pricing_currency_model: str | None = None
for model_cfg in models or []: for model_cfg in models or []:
raw = getattr(model_cfg, "pricing", None) raw = getattr(model_cfg, "pricing", None)
if not isinstance(raw, dict): if not isinstance(raw, dict):
@ -181,8 +183,20 @@ def _build_pricing_map() -> dict[str, _ModelPricing]:
continue continue
if input_price <= 0 and output_price <= 0: if input_price <= 0 and output_price <= 0:
continue continue
currency = str(raw.get("currency") or "USD").upper() model_currency = str(raw.get("currency") or "USD").strip().upper() or "USD"
entry = _ModelPricing(input_price, output_price, currency, cache_hit_price) if pricing_currency is None:
pricing_currency = model_currency
pricing_currency_model = model_cfg.name
elif model_currency != pricing_currency:
logger.warning(
"console: disabling cost reporting because model pricing mixes currencies (%s on %s, %s on %s)",
pricing_currency,
pricing_currency_model,
model_currency,
model_cfg.name,
)
return {}
entry = _ModelPricing(input_price, output_price, model_currency, cache_hit_price)
for key in (model_cfg.name, getattr(model_cfg, "model", None)): for key in (model_cfg.name, getattr(model_cfg, "model", None)):
if key: if key:
pricing.setdefault(key, entry) pricing.setdefault(key, entry)

View File

@ -12,6 +12,7 @@ and serving TestClient requests in another never share a connection).
""" """
import asyncio import asyncio
import logging
from datetime import UTC, datetime, timedelta from datetime import UTC, datetime, timedelta
from types import SimpleNamespace from types import SimpleNamespace
from unittest.mock import AsyncMock from unittest.mock import AsyncMock
@ -262,6 +263,74 @@ _R4_COST = 600 * 8e-6 + 399 * 32e-6 # 0.017568
class TestPricing: class TestPricing:
def test_mixed_currencies_disable_cost_reporting(self, client, monkeypatch, caplog):
monkeypatch.setattr(
console,
"get_app_config",
lambda: SimpleNamespace(
models=[
SimpleNamespace(
name="minimax-m2",
model="MiniMax-M2",
pricing={"currency": "CNY", "input_per_million": 8, "output_per_million": 32},
),
SimpleNamespace(
name="gpt-x",
model="gpt-x-1",
pricing={"currency": "USD", "input_per_million": 1, "output_per_million": 4},
),
]
),
)
with caplog.at_level(logging.WARNING, logger=console.logger.name):
stats = client.get("/api/console/stats").json()
assert stats["currency"] is None
assert stats["total_cost"] is None
# The warning names both offending models so operators can locate the misconfiguration.
assert any("minimax-m2" in rec.getMessage() and "gpt-x" in rec.getMessage() for rec in caplog.records)
usage = client.get("/api/console/usage").json()
assert usage["currency"] is None
assert usage["total_cost"] is None
assert all(day["cost"] == 0 for day in usage["days"])
assert all(model["cost"] is None for model in usage["by_model"].values())
runs = client.get("/api/console/runs", params={"limit": 50}).json()
assert all(run["cost"] is None for run in runs["runs"])
def test_shared_currency_across_models_prices_normally(self, client, monkeypatch):
"""Multiple priced models on one currency (incl. case variants) must not trip the guard."""
monkeypatch.setattr(
console,
"get_app_config",
lambda: SimpleNamespace(
models=[
SimpleNamespace(
name="minimax-m2",
model="MiniMax-M2",
pricing={"currency": "CNY", "input_per_million": 8, "output_per_million": 32},
),
SimpleNamespace(
name="qwen",
model="qwen",
pricing={"currency": "cny", "input_per_million": 8, "output_per_million": 32},
),
]
),
)
# qwen's only run (r5) is now priced alongside the minimax runs.
qwen_cost = 40 * 8e-6 + 30 * 32e-6
stats = client.get("/api/console/stats").json()
assert stats["currency"] == "CNY"
# No cache-hit price configured → r1 billed at the miss price.
assert stats["total_cost"] == pytest.approx(_R1_COST_UNCACHED + _R2_COST + _R4_COST + qwen_cost)
usage = client.get("/api/console/usage").json()
assert usage["currency"] == "CNY"
assert usage["by_model"]["qwen"]["cost"] == pytest.approx(qwen_cost)
def test_costs_use_cache_hit_price(self, client, monkeypatch): def test_costs_use_cache_hit_price(self, client, monkeypatch):
monkeypatch.setattr(console, "get_app_config", lambda: _priced_config()) monkeypatch.setattr(console, "get_app_config", lambda: _priced_config())
stats = client.get("/api/console/stats").json() stats = client.get("/api/console/stats").json()

View File

@ -98,7 +98,8 @@ max_recursion_limit: 1000
# #
# Optional per-model pricing (powers the real-cost display on the workspace # Optional per-model pricing (powers the real-cost display on the workspace
# console). Add a `pricing` block to any model entry; use ONE currency across # console). Add a `pricing` block to any model entry; use ONE currency across
# all models. Prices are per one million tokens. # all models. Mixed currencies disable cost reporting to prevent invalid sums.
# Prices are per one million tokens.
# #
# pricing: # pricing:
# currency: CNY # ISO code shown in the console (CNY, USD, ...) # currency: CNY # ISO code shown in the console (CNY, USD, ...)