feat(DATAPF-2790): Upgrade vLLM to 0.20.0 with RunAI streamer s… - #9
Open
Yannick Lambruschi (ylambruschi) wants to merge 6 commits into
Open
feat(DATAPF-2790): Upgrade vLLM to 0.20.0 with RunAI streamer s…#9Yannick Lambruschi (ylambruschi) wants to merge 6 commits into
Yannick Lambruschi (ylambruschi) wants to merge 6 commits into
Conversation
…upport - Bump vLLM to 0.18.0, lmcache to 0.4.2, flashinfer to 0.6.6 - Rename OpenAIServingEmbedding → ServingEmbedding (vLLM 0.18.0 breaking change) - Add RunAI streamer support: skip Storage.download and architecture validation for remote URIs (s3://, gs://, etc.) and when --backend=vllm is explicit - Add LD_LIBRARY_PATH and AWS_REGION env vars to prod Dockerfile Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Jeanine Harb (jharb-mirakl)
approved these changes
Mar 30, 2026
vLLM 0.18.0 introduced OpenAIServingRender to handle prompt rendering/preprocessing. log_error_stack and rendering args moved out of OpenAIServingChat/OpenAIServingCompletion into this new required dependency. Create OpenAIServingRender via renderer_from_config, get_io_processor, and OpenAIModelRegistry, then pass it to both serving objects. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Bump vLLM to 0.20.0, lmcache to 0.4.4 - Remove flashinfer (dropped in vLLM 0.20), switch to uv pip install - Fix ServingScores import: pooling/score → pooling/scoring - Remove OpenAIModelRegistry/renderer_from_config/get_io_processor imports - Simplify OpenAIServingRender construction: renderer from engine_client, registry from serving_models, add reasoning_parser, drop io_processor - Remove disable_frontend_multiprocessing arg (removed from vLLM 0.20 API) - Call ServingEmbedding/ServingScores as callables (dropped .create_embedding/.do_rerank) - Reranking task check: "score" → "classify" in supported_tasks - Add --return_offsets_mapping arg for token classification (upstream) - pyproject.toml: bump version 0.17.0→0.18.0, pillow CVE fix, openai≥2.0.0 All import paths and constructor signatures verified against vLLM v0.20.0 source. RunAI streamer hacks (s3:// URI bypass, force_vllm, LD_LIBRARY_PATH, AWS_REGION) preserved. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
vLLM 0.20.0 pulls in nvidia-nvjitlink (no -cu12 suffix) alongside the existing nvidia-nvjitlink-cu12. Both carry Other/Proprietary License and must be excluded from the allowlist check. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ackages vLLM 0.20.0 ships nvidia packages without the -cu12 suffix (e.g. nvidia-nvtx, nvidia-cublas, etc.) alongside the old -cu12 variants. Add the full set of bare names to the pip-licenses ignore list to avoid repeated one-at-a-time build failures. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
vLLM 0.20.0 renamed vllm.entrypoints.pooling.score → pooling.scoring. Update the RerankRequest/RerankResponse import in the kserve openai types module accordingly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Yannick Lambruschi (ylambruschi)
changed the title
Alexandre Blanchard (ablanchard)
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DATAPF-2790 (added by bigbrother)
…upport