fix(console): disable cost reporting when model pricing mixes currencies (#4564)

The console cost estimator summed per-run spend across every priced model
without checking that they shared a currency, so a deployment that priced
one model in CNY and another in USD produced a meaningless aggregate under
a single currency label. Track the first priced currency and, on a
mismatch, log a warning and drop all pricing so cost/currency fields report
null instead of an invalid sum. Currency codes are now trimmed and
case-normalized so equivalent spellings don't false-trip the guard.
This commit is contained in:
Ryker_Feng 2026-07-29 22:24:04 +08:00 committed by GitHub
parent 6b5f5e789a
commit 594dbe1205
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
5 changed files with 92 additions and 4 deletions

View File

@ -138,6 +138,10 @@ That prompt is intended for coding agents. It tells the agent to clone the repo
> **Advanced / manual configuration**: If you prefer to edit `config.yaml` directly, run `make config` instead to copy the full template. See `config.example.yaml` for the complete reference including CLI-backed providers (Codex CLI, Claude Code OAuth), OpenRouter, Responses API, subagent runtime caps such as `subagents.max_total_per_run`, and more.
Optional per-model pricing must use one currency across all priced models.
DeerFlow disables Console cost estimates when currencies are mixed rather
than presenting an invalid aggregate.
<details>
<summary>Manual model configuration examples</summary>

View File

@ -452,7 +452,7 @@ Localhost persistence deliberately reads the direct request `Host` and ignores `
|--------|-----------|
| **Models** (`/api/models`) | `GET /` - list models; `GET /{name}` - model details |
| **Features** (`/api/features`) | `GET /` - report config-gated feature availability (`agents_api.enabled`, `browser_control.enabled`) for frontend UI gating |
| **Console** (`/api/console`) | Read-only cross-thread observability for the current user (the data layer for an operations dashboard or external monitoring): `GET /stats` - headline counters (runs/threads/agents/tokens/cost); `GET /runs` - paginated run history joined with thread titles (per-run cost); `GET /usage` - zero-filled daily token series + per-model breakdown with spend. Queries `runs`/`threads_meta` directly as a reporting layer (no new `RunStore` methods); requires a SQL database backend — returns 503 on `database.backend: memory`. Real-cost estimation reads optional `models[*].pricing` (`currency`, `input_per_million`, `output_per_million`, `input_cache_hit_per_million`; `ModelConfig` is `extra="allow"`, so no schema change) and prices each run from its `token_usage_by_model` input/output split. Pricing is **cache-aware**: `RunJournal` accumulates prompt-cache hits from `usage_metadata.input_token_details.cache_read` into a sparse `cache_read_tokens` bucket key (also threaded through `SubagentTokenCollector``record_external_llm_usage_records`), and cache-hit input tokens are billed at `input_cache_hit_per_million` (omitted → billed at the miss price, a conservative upper bound). Legacy rows fall back to run-level totals at `model_name`; unpriced models yield `cost: null` and cost fields are null when no pricing is configured |
| **Console** (`/api/console`) | Read-only cross-thread observability for the current user (the data layer for an operations dashboard or external monitoring): `GET /stats` - headline counters (runs/threads/agents/tokens/cost); `GET /runs` - paginated run history joined with thread titles (per-run cost); `GET /usage` - zero-filled daily token series + per-model breakdown with spend. Queries `runs`/`threads_meta` directly as a reporting layer (no new `RunStore` methods); requires a SQL database backend — returns 503 on `database.backend: memory`. Real-cost estimation reads optional `models[*].pricing` (`currency`, `input_per_million`, `output_per_million`, `input_cache_hit_per_million`; `ModelConfig` is `extra="allow"`, so no schema change) and prices each run from its `token_usage_by_model` input/output split. Pricing is **cache-aware**: `RunJournal` accumulates prompt-cache hits from `usage_metadata.input_token_details.cache_read` into a sparse `cache_read_tokens` bucket key (also threaded through `SubagentTokenCollector``record_external_llm_usage_records`), and cache-hit input tokens are billed at `input_cache_hit_per_million` (omitted → billed at the miss price, a conservative upper bound). All priced models must use one currency; mixed currencies disable cost reporting and leave cost/currency fields null instead of producing invalid aggregates. Legacy rows fall back to run-level totals at `model_name`; unpriced models yield `cost: null` and cost fields are null when no pricing is configured |
| **MCP** (`/api/mcp`) | `GET /config` - get config; `PUT /config` - update config (saves to extensions_config.json) |
| **Skills** (`/api/skills`) | `GET /` - list skills; `GET /{name}` - details; `PUT /{name}` - update enabled; `POST /install` - install from .skill archive (accepts standard optional frontmatter like `version`, `author`, `compatibility`); `POST /reload` - admin-only process-local prompt-cache invalidation after trusted external filesystem changes |
| **Integrations** (`/api/integrations`) | `GET /lark/status` - inspect managed Lark/Feishu CLI integration state, including `sandbox_runtime_mode` / `sandbox_runtime_ready` (whether `lark-cli` will actually be present in the sandbox at chat time); `POST /lark/install` - admin-only install of the official `lark-*` managed skill pack; `POST /lark/config/start` and `/lark/config/complete` - internal first-time Lark connection setup; `POST /lark/auth/start` and `/lark/auth/complete` - browser device-flow user authorization without terminal access, with optional `domains` / exact `scope` for incremental permission grants |

View File

@ -167,6 +167,8 @@ def _build_pricing_map() -> dict[str, _ModelPricing]:
return {}
pricing: dict[str, _ModelPricing] = {}
pricing_currency: str | None = None
pricing_currency_model: str | None = None
for model_cfg in models or []:
raw = getattr(model_cfg, "pricing", None)
if not isinstance(raw, dict):
@ -181,8 +183,20 @@ def _build_pricing_map() -> dict[str, _ModelPricing]:
continue
if input_price <= 0 and output_price <= 0:
continue
currency = str(raw.get("currency") or "USD").upper()
entry = _ModelPricing(input_price, output_price, currency, cache_hit_price)
model_currency = str(raw.get("currency") or "USD").strip().upper() or "USD"
if pricing_currency is None:
pricing_currency = model_currency
pricing_currency_model = model_cfg.name
elif model_currency != pricing_currency:
logger.warning(
"console: disabling cost reporting because model pricing mixes currencies (%s on %s, %s on %s)",
pricing_currency,
pricing_currency_model,
model_currency,
model_cfg.name,
)
return {}
entry = _ModelPricing(input_price, output_price, model_currency, cache_hit_price)
for key in (model_cfg.name, getattr(model_cfg, "model", None)):
if key:
pricing.setdefault(key, entry)

View File

@ -12,6 +12,7 @@ and serving TestClient requests in another never share a connection).
"""
import asyncio
import logging
from datetime import UTC, datetime, timedelta
from types import SimpleNamespace
from unittest.mock import AsyncMock
@ -262,6 +263,74 @@ _R4_COST = 600 * 8e-6 + 399 * 32e-6 # 0.017568
class TestPricing:
def test_mixed_currencies_disable_cost_reporting(self, client, monkeypatch, caplog):
monkeypatch.setattr(
console,
"get_app_config",
lambda: SimpleNamespace(
models=[
SimpleNamespace(
name="minimax-m2",
model="MiniMax-M2",
pricing={"currency": "CNY", "input_per_million": 8, "output_per_million": 32},
),
SimpleNamespace(
name="gpt-x",
model="gpt-x-1",
pricing={"currency": "USD", "input_per_million": 1, "output_per_million": 4},
),
]
),
)
with caplog.at_level(logging.WARNING, logger=console.logger.name):
stats = client.get("/api/console/stats").json()
assert stats["currency"] is None
assert stats["total_cost"] is None
# The warning names both offending models so operators can locate the misconfiguration.
assert any("minimax-m2" in rec.getMessage() and "gpt-x" in rec.getMessage() for rec in caplog.records)
usage = client.get("/api/console/usage").json()
assert usage["currency"] is None
assert usage["total_cost"] is None
assert all(day["cost"] == 0 for day in usage["days"])
assert all(model["cost"] is None for model in usage["by_model"].values())
runs = client.get("/api/console/runs", params={"limit": 50}).json()
assert all(run["cost"] is None for run in runs["runs"])
def test_shared_currency_across_models_prices_normally(self, client, monkeypatch):
"""Multiple priced models on one currency (incl. case variants) must not trip the guard."""
monkeypatch.setattr(
console,
"get_app_config",
lambda: SimpleNamespace(
models=[
SimpleNamespace(
name="minimax-m2",
model="MiniMax-M2",
pricing={"currency": "CNY", "input_per_million": 8, "output_per_million": 32},
),
SimpleNamespace(
name="qwen",
model="qwen",
pricing={"currency": "cny", "input_per_million": 8, "output_per_million": 32},
),
]
),
)
# qwen's only run (r5) is now priced alongside the minimax runs.
qwen_cost = 40 * 8e-6 + 30 * 32e-6
stats = client.get("/api/console/stats").json()
assert stats["currency"] == "CNY"
# No cache-hit price configured → r1 billed at the miss price.
assert stats["total_cost"] == pytest.approx(_R1_COST_UNCACHED + _R2_COST + _R4_COST + qwen_cost)
usage = client.get("/api/console/usage").json()
assert usage["currency"] == "CNY"
assert usage["by_model"]["qwen"]["cost"] == pytest.approx(qwen_cost)
def test_costs_use_cache_hit_price(self, client, monkeypatch):
monkeypatch.setattr(console, "get_app_config", lambda: _priced_config())
stats = client.get("/api/console/stats").json()

View File

@ -98,7 +98,8 @@ max_recursion_limit: 1000
#
# Optional per-model pricing (powers the real-cost display on the workspace
# console). Add a `pricing` block to any model entry; use ONE currency across
# all models. Prices are per one million tokens.
# all models. Mixed currencies disable cost reporting to prevent invalid sums.
# Prices are per one million tokens.
#
# pricing:
# currency: CNY # ISO code shown in the console (CNY, USD, ...)