Skip to content

fix: don't retry LLM calls the provider rejected (400/401/403/404/422) - #2214

Open
Mattbusel wants to merge 1 commit into
FoundationAgents:mainfrom
tensorust-ai:fix/no-retry-on-client-errors
Open

Mattbusel wants to merge 1 commit into
FoundationAgents:mainfrom
tensorust-ai:fix/no-retry-on-client-errors

Conversation

@Mattbusel

Copy link
Copy Markdown

WriteCode.write_code (metagpt/actions/write_code.py:94), WriteCodeReview.write_code_review_and_rewrite (metagpt/actions/write_code_review.py:147) and ActionNode._aask_v1 (metagpt/actions/action_node.py:423) are wrapped in @retry(stop=stop_after_attempt(6)) with no retry= filter. Any exception, a 400 context_length_exceeded or a 401 bad key included, re-sends the same prompt up to 6 times.

Underneath, acompletion_text (metagpt/provider/openai_api.py:165-171) retries APIConnectionError (which APITimeoutError subclasses) 6 times, and the AsyncOpenAI client is built without max_retries (L72), so the SDK adds 2 more. During a timeout outage one code-writing step can become 6 x 6 x 3 = 108 requests; a request the API rejects is sent 6 times.

This PR adds metagpt.utils.common.is_retryable_llm_error: an error carrying a 4xx status_code (or response.status_code) other than 408/409/429 is not retried. Parse failures of the model's answer, timeouts, 429 and 5xx keep retrying as before, so the existing "re-ask until the code block parses" behaviour is unchanged. It is applied to the three decorators above. It reads status_code instead of importing SDK exception classes, so it covers the openai and anthropic SDKs and httpx-based providers alike.

tests/metagpt/utils/test_retryable_llm_error.py: 14 tests (each 4xx class, 409/429/500, parse and timeout errors, an httpx.HTTPStatusError, and the action retry policy itself: a 400 is sent once, a parse failure is retried until it succeeds).

The nested connection-error retries are left as they are; that is a bigger change, and #2170 touches the provider layer.

Found with burn audit from tokio-prompt-orchestrator, which I maintain. Nothing here depends on it.

WriteCode.write_code, WriteCodeReview.write_code_review_and_rewrite and
ActionNode._aask_v1 are decorated with @Retry(stop=stop_after_attempt(6))
and no retry= filter, so any exception, a 400 context_length_exceeded or a
401 bad key included, re-sent the same prompt up to 6 times. Under them,
acompletion_text retries connection errors 6 times and the OpenAI SDK
twice more, so one bad request could become dozens of paid calls.

Adds metagpt.utils.common.is_retryable_llm_error: an error carrying a 4xx
status_code other than 408/409/429 is not retried; parse failures of the
model's answer, timeouts, 429 and 5xx are retried as before.

This branch is waiting to be deployed

1 waiting deployment
unittest — 5c2e7ed3 Waiting Nov 19, 2026 by Mattbusel via build (3.9) #1779
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant