Repository navigation
fix: price models missing from TOKEN_COSTS instead of adding $0 (max_budget never trips on GPT-5/Claude 4) - #2213
Open
Mattbusel wants to merge 1 commit into
Conversation
CostManager.update_cost logged a warning and returned when the model was not in TOKEN_COSTS, so every call to gpt-4.1, GPT-5, o3, Claude 4 or Gemini 2.5 added $0 to total_cost and Team's max_budget check (NoMoneyException) never fired for them. Models missing from the table are now priced from llm-prices (dated, source-cited list prices, zero dependencies). TOKEN_COSTS still wins when it has the model. Models neither source knows are recorded in CostManager.unpriced_models and the warning says their usage is not covered by max_budget. Also corrects rows that no longer match OpenAI's list prices: gpt-4o (2.50/10), gpt-4-0613 (30/60), gpt-3.5-turbo and -0125 (0.50/1.50) and text-embedding-ada-002 (0.10).
This branch is waiting to be deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CostManager.update_costreturns early when the model is not inTOKEN_COSTS(cost_manager.py#L48-L50):TOKEN_COSTShas no gpt-4.1, GPT-5, GPT-6, o3, Claude 4 or Gemini 2.5+ rows, so every call to those models adds $0 tototal_cost.Team._check_balancecomparestotal_costwithmax_budget(team.py#L98-L100), soteam.invest(3.0)never raisesNoMoneyExceptionon them. A gpt-5.5 call with a 100K-token prompt and 5K output costs $0.65 and is booked at $0.What this changes
TOKEN_COSTSis priced from llm-prices (dated list prices with the provider page each one came from, no dependencies, Python 3.9+).TOKEN_COSTSand anytoken_costspassed in still win when they have the model.CostManager.unpriced_models, and the warning now says its usage is not covered bymax_budget.Tests
tests/metagpt/utils/test_cost_manager.pygets five tests: gpt-5.5 priced at $0.65 for the call above, two Claude Opus 4.5 calls reaching a $1max_budget,token_costswinning over llm-prices, an unknown model landing inunpriced_models, and gpt-4o at the current price. All 6 tests in the file pass with the change; 4 of the new ones fail onmain. black, isort and flake8 are clean on the touched files.Disclosure: I maintain llm-prices.