Skip to content

feat: Refactor vLLM build to use upstream container image - #8697

Merged
mc-nv merged 9 commits into
mainfrom
mchornyi/TRI-775/rebuild-vllm
Mar 13, 2026
Merged

feat: Refactor vLLM build to use upstream container image#8697
mc-nv merged 9 commits into
mainfrom
mchornyi/TRI-775/rebuild-vllm

Conversation

@mc-nv

@mc-nv mc-nv commented Mar 12, 2026

Copy link
Copy Markdown
Contributor

Description

This PR refactors the vLLM backend build process to use the official NVIDIA vLLM container image as a base instead of building vLLM from scratch. This change simplifies the build process by removing complex installation logic and leveraging pre-built, optimized vLLM images.

The key motivation is to:

  • Reduce build complexity and maintenance overhead
  • Leverage tested and optimized upstream vLLM containers
  • Improve build reliability by removing custom vLLM installation code
  • Allow flexibility to use custom inference images when needed

Changes

  • Removed vLLM installation logic: Deleted ~45 lines of complex vLLM installation code including architecture-specific dependencies (MKL for x86_64, NVPL for aarch64), pytorch-triton custom builds, and private index URL handling
  • Added inference image support: Introduced new INFERENCE_IMAGE parameter to allow specifying custom base images for the final container
  • Default vLLM image fallback: When vLLM backend is enabled and no custom inference image is provided, automatically uses nvcr.io/nvidia/vllm:{version}-py3 as the base image
  • Simplified BASE_IMAGE logic: BASE_IMAGE now intelligently switches to INFERENCE_IMAGE when building vLLM backends
  • Extended --image flag: Added support for --image inference, to specify custom inference base images

Affected Files

  • build.py - Main build orchestration script
    • Lines 1204-1224: New inference image selection logic
    • Lines 1472-1509: Removed vLLM installation code (42 lines deleted)
    • Lines 1682-1688: Added INFERENCE_IMAGE to dockerfile argument map
    • Line 2926: Extended --image flag validation to support "inference"

@mc-nv mc-nv self-assigned this Mar 12, 2026
whoisj
whoisj previously approved these changes Mar 13, 2026

@whoisj whoisj left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved w/ 1 suggestion.

Comment thread build.py Outdated
Co-authored-by: J Wyman <jwyman@nvidia.com>
@mc-nv
mc-nv merged commit 0177f59 into main Mar 13, 2026
3 checks passed
@mc-nv
mc-nv deleted the mchornyi/TRI-775/rebuild-vllm branch March 13, 2026 15:43
mc-nv added a commit that referenced this pull request Mar 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants