Skip to content

FEAT: Slim Docker runtime image#5108

Draft
OliverBryant wants to merge 1 commit into
xorbitsai:mainfrom
OliverBryant:fix/slim-docker-image
Draft

FEAT: Slim Docker runtime image#5108
OliverBryant wants to merge 1 commit into
xorbitsai:mainfrom
OliverBryant:fix/slim-docker-image

Conversation

@OliverBryant

Copy link
Copy Markdown
Collaborator

Summary

  • Rework GPU, CPU, and aarch64 Dockerfiles to use python:3.12-slim-bookworm multi-stage builds instead of vLLM/Miniconda bases.
  • Remove globally installed model/engine runtime dependencies from Docker base requirements and delete the Docker-only heavy requirements files.
  • Keep runtime virtualenv tooling (uv, pip, setuptools, wheel) and core service dependencies in the base image.
  • Add Docker CI builds with image size reporting, plus Docker docs describing the slim image boundary.
  • Fix default virtualenv subpool environment injection when enable_virtual_env is omitted.

Validation

  • python -m py_compile xinference/core/utils.py xinference/core/tests/test_utils.py
  • Static scan confirmed slim Dockerfiles/base requirements no longer reference vLLM, Transformers, Torch, SGLang, xllamacpp, Diffusers, image/audio/video runtime packages, or deleted heavy requirements files.
  • Workflow smoke check verified the new Docker CI workflow has no tabs and includes Docker build steps.

Docker build was attempted locally, but the local Docker daemon is not running: Cannot connect to the Docker daemon at unix:///Users/oliver/.docker/run/docker.sock. The new Docker CI workflow will build all three images on PRs that touch Docker inputs.

@XprobeBot XprobeBot added the gpu label Jul 1, 2026
@XprobeBot XprobeBot added this to the v2.x milestone Jul 1, 2026
@OliverBryant OliverBryant changed the title [codex] Slim Docker runtime image FEAT: Slim Docker runtime image Jul 1, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request transitions the Xinference Docker images to slim runtime images by utilizing multi-stage builds that exclude Node.js from the final stage and removing pre-installed model and engine dependencies, which are now installed on-demand in virtual environments. It also updates the environment-building logic to fallback to global defaults when enable_virtual_env is not specified, alongside adding corresponding tests and documentation. Feedback on the changes points out that the CPU base requirements file is missing several OpenTelemetry packages that are present in the other platform-specific requirements files, which should be added to ensure consistency.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

bcrypt>=4.0.0
aioprometheus[starlette]>=23.12.0
nvidia-ml-py
pynvml>=12

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The CPU base requirements file (requirements_cpu-base.txt) is missing several OpenTelemetry packages that are present in requirements-base.txt and requirements_aarch64.txt:\n- opentelemetry-exporter-otlp-proto-grpc>=1.20.0\n- opentelemetry-instrumentation-fastapi>=0.41b0\n- opentelemetry-instrumentation-httpx>=0.41b0\n\nPlease add these packages to the end of this file to ensure consistent OpenTelemetry support across all Docker image variants.

@OliverBryant OliverBryant force-pushed the fix/slim-docker-image branch from 141f757 to cbd1dc2 Compare July 1, 2026 04:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants