馃殌 Feature
- Ability to expose assistants through an API that follows the Responses API standard and specs.
import litserve as ls
class SimpleLitAPI(ls.LitAPI):
def setup(self, device):
self.model = None
def predict(self, prompt):
# `prompt` is a list of dictionary containing role and content
# example: [{'role': 'user', 'content': 'How can I help you today?'}]
yield "This is a sample generated output"
if __name__ == "__main__":
# Enable the OpenAISpec in LitServer
api = SimpleLitAPI()
server = ls.LitServer(api, spec=ls.ResponsesOpenAISpec())
server.run(port=8000)Motivation
OpenAI announced last week a new Responses API that contains the feature set of the Chat Completions API and replaces the Assistants API (the latter is now deprecated and will be sunseted mid-2026).
Azure already announced that they provide this API day one and considering OpenAI's position on the market, we can expect that it will become an industry standard as is the Chat Completions API today.
Additional context
馃殌 Feature
Motivation
OpenAI announced last week a new Responses API that contains the feature set of the Chat Completions API and replaces the Assistants API (the latter is now deprecated and will be sunseted mid-2026).
Azure already announced that they provide this API day one and considering OpenAI's position on the market, we can expect that it will become an industry standard as is the Chat Completions API today.
Additional context