Description
Currently, the OpenaiChatModel class in OpenJudge does not explicitly handle network timeouts or transient API errors. If the OpenAI API hangs or returns a temporary 5xx error, the OpenaiChatModel might wait indefinitely or crash immediately.
Suggested Implementation
The official OpenAI Python SDK (v1.x) natively supports max_retries and timeout parameters, but OpenJudge is not currently exposing or utilizing them in the model wrapper. This limits the robustness of the evaluation pipeline in unstable network environments.
I would like to expose the native max_retries and timeout parameters in the OpenaiChatModel initialization and pass them to the underlying OpenAI client.
Implementation Steps:
Modify OpenaiChatModel:
Update the init method in openjudge/models/openai_chat_model.py (or the relevant file path) to accept max_retries (int, default e.g., 2) and timeout (float, default e.g., 60.0).
Pass these parameters when initializing the AsyncOpenAI or OpenAI client within the class.
Add Unit Tests:
Create a test case in tests/models/test_openai_model.py.
Use unittest.mock to mock the OpenAI client.
Verify that when OpenaiChatModel is initialized with specific retry/timeout values, those values are correctly passed to the OpenAI client constructor.
Note: The test should not make real API calls; it should only verify parameter passing.
Alternatives
We could implement a custom retry decorator (using libraries like tenacity) around the API call method. However, since the OpenAI SDK v1.x provides built-in support for retries and timeouts, using the native implementation is cleaner and requires fewer dependencies.
Example usage after implementation:
model = OpenaiChatModel(
model="gpt-4",
api_key="...",
timeout=30.0, # Fail if request takes longer than 30s
max_retries=3 # Retry 3 times on connection errors
)
Description
Currently, the OpenaiChatModel class in OpenJudge does not explicitly handle network timeouts or transient API errors. If the OpenAI API hangs or returns a temporary 5xx error, the OpenaiChatModel might wait indefinitely or crash immediately.
Suggested Implementation
The official OpenAI Python SDK (v1.x) natively supports max_retries and timeout parameters, but OpenJudge is not currently exposing or utilizing them in the model wrapper. This limits the robustness of the evaluation pipeline in unstable network environments.
I would like to expose the native max_retries and timeout parameters in the OpenaiChatModel initialization and pass them to the underlying OpenAI client.
Implementation Steps:
Modify OpenaiChatModel:
Update the init method in openjudge/models/openai_chat_model.py (or the relevant file path) to accept max_retries (int, default e.g., 2) and timeout (float, default e.g., 60.0).
Pass these parameters when initializing the AsyncOpenAI or OpenAI client within the class.
Add Unit Tests:
Create a test case in tests/models/test_openai_model.py.
Use unittest.mock to mock the OpenAI client.
Verify that when OpenaiChatModel is initialized with specific retry/timeout values, those values are correctly passed to the OpenAI client constructor.
Note: The test should not make real API calls; it should only verify parameter passing.
Alternatives
We could implement a custom retry decorator (using libraries like tenacity) around the API call method. However, since the OpenAI SDK v1.x provides built-in support for retries and timeouts, using the native implementation is cleaner and requires fewer dependencies.
Example usage after implementation: