Repository navigation
Conversation
WriteCode.write_code, WriteCodeReview.write_code_review_and_rewrite and ActionNode._aask_v1 are decorated with @Retry(stop=stop_after_attempt(6)) and no retry= filter, so any exception, a 400 context_length_exceeded or a 401 bad key included, re-sent the same prompt up to 6 times. Under them, acompletion_text retries connection errors 6 times and the OpenAI SDK twice more, so one bad request could become dozens of paid calls. Adds metagpt.utils.common.is_retryable_llm_error: an error carrying a 4xx status_code other than 408/409/429 is not retried; parse failures of the model's answer, timeouts, 429 and 5xx are retried as before.
This branch is waiting to be deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WriteCode.write_code(metagpt/actions/write_code.py:94),WriteCodeReview.write_code_review_and_rewrite(metagpt/actions/write_code_review.py:147) andActionNode._aask_v1(metagpt/actions/action_node.py:423) are wrapped in@retry(stop=stop_after_attempt(6))with noretry=filter. Any exception, a 400context_length_exceededor a 401 bad key included, re-sends the same prompt up to 6 times.Underneath,
acompletion_text(metagpt/provider/openai_api.py:165-171) retriesAPIConnectionError(whichAPITimeoutErrorsubclasses) 6 times, and theAsyncOpenAIclient is built withoutmax_retries(L72), so the SDK adds 2 more. During a timeout outage one code-writing step can become 6 x 6 x 3 = 108 requests; a request the API rejects is sent 6 times.This PR adds
metagpt.utils.common.is_retryable_llm_error: an error carrying a 4xxstatus_code(orresponse.status_code) other than 408/409/429 is not retried. Parse failures of the model's answer, timeouts, 429 and 5xx keep retrying as before, so the existing "re-ask until the code block parses" behaviour is unchanged. It is applied to the three decorators above. It readsstatus_codeinstead of importing SDK exception classes, so it covers the openai and anthropic SDKs and httpx-based providers alike.tests/metagpt/utils/test_retryable_llm_error.py: 14 tests (each 4xx class, 409/429/500, parse and timeout errors, anhttpx.HTTPStatusError, and the action retry policy itself: a 400 is sent once, a parse failure is retried until it succeeds).The nested connection-error retries are left as they are; that is a bigger change, and #2170 touches the provider layer.
Found with
burn auditfrom tokio-prompt-orchestrator, which I maintain. Nothing here depends on it.