Skip to content

fix: price models missing from TOKEN_COSTS instead of adding $0 (max_budget never trips on GPT-5/Claude 4) - #2213

Open
Mattbusel wants to merge 1 commit into
FoundationAgents:mainfrom
tensorust-ai:fix/price-models-missing-from-token-costs
Open

Mattbusel wants to merge 1 commit into
FoundationAgents:mainfrom
tensorust-ai:fix/price-models-missing-from-token-costs

Conversation

@Mattbusel

Copy link
Copy Markdown

CostManager.update_cost returns early when the model is not in TOKEN_COSTS (cost_manager.py#L48-L50):

if model not in self.token_costs:
    logger.warning(f"Model {model} not found in TOKEN_COSTS.")
    return

TOKEN_COSTS has no gpt-4.1, GPT-5, GPT-6, o3, Claude 4 or Gemini 2.5+ rows, so every call to those models adds $0 to total_cost. Team._check_balance compares total_cost with max_budget (team.py#L98-L100), so team.invest(3.0) never raises NoMoneyException on them. A gpt-5.5 call with a 100K-token prompt and 5K output costs $0.65 and is booked at $0.

What this changes

  • A model missing from TOKEN_COSTS is priced from llm-prices (dated list prices with the provider page each one came from, no dependencies, Python 3.9+). TOKEN_COSTS and any token_costs passed in still win when they have the model.
  • A model neither source knows is recorded in CostManager.unpriced_models, and the warning now says its usage is not covered by max_budget.
  • Five rows that no longer match OpenAI's list prices are corrected:
Model Was (per 1M) Now (per 1M)
gpt-4o 5 / 15 2.50 / 10
gpt-4-0613 60 / 120 30 / 60
gpt-3.5-turbo 1.50 / 2 0.50 / 1.50
gpt-3.5-turbo-0125 1 / 2 0.50 / 1.50
text-embedding-ada-002 0.40 0.10

Tests

tests/metagpt/utils/test_cost_manager.py gets five tests: gpt-5.5 priced at $0.65 for the call above, two Claude Opus 4.5 calls reaching a $1 max_budget, token_costs winning over llm-prices, an unknown model landing in unpriced_models, and gpt-4o at the current price. All 6 tests in the file pass with the change; 4 of the new ones fail on main. black, isort and flake8 are clean on the touched files.

Disclosure: I maintain llm-prices.

CostManager.update_cost logged a warning and returned when the model was
not in TOKEN_COSTS, so every call to gpt-4.1, GPT-5, o3, Claude 4 or
Gemini 2.5 added $0 to total_cost and Team's max_budget check
(NoMoneyException) never fired for them.

Models missing from the table are now priced from llm-prices (dated,
source-cited list prices, zero dependencies). TOKEN_COSTS still wins
when it has the model. Models neither source knows are recorded in
CostManager.unpriced_models and the warning says their usage is not
covered by max_budget.

Also corrects rows that no longer match OpenAI's list prices: gpt-4o
(2.50/10), gpt-4-0613 (30/60), gpt-3.5-turbo and -0125 (0.50/1.50) and
text-embedding-ada-002 (0.10).

This branch is waiting to be deployed

1 waiting deployment
unittest — 2a217917 Waiting Nov 19, 2026 by Mattbusel via build (3.9) #1778
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant