FEAT: Slim Docker runtime image#5108
Conversation
There was a problem hiding this comment.
Code Review
This pull request transitions the Xinference Docker images to slim runtime images by utilizing multi-stage builds that exclude Node.js from the final stage and removing pre-installed model and engine dependencies, which are now installed on-demand in virtual environments. It also updates the environment-building logic to fallback to global defaults when enable_virtual_env is not specified, alongside adding corresponding tests and documentation. Feedback on the changes points out that the CPU base requirements file is missing several OpenTelemetry packages that are present in the other platform-specific requirements files, which should be added to ensure consistency.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| bcrypt>=4.0.0 | ||
| aioprometheus[starlette]>=23.12.0 | ||
| nvidia-ml-py | ||
| pynvml>=12 |
There was a problem hiding this comment.
The CPU base requirements file (requirements_cpu-base.txt) is missing several OpenTelemetry packages that are present in requirements-base.txt and requirements_aarch64.txt:\n- opentelemetry-exporter-otlp-proto-grpc>=1.20.0\n- opentelemetry-instrumentation-fastapi>=0.41b0\n- opentelemetry-instrumentation-httpx>=0.41b0\n\nPlease add these packages to the end of this file to ensure consistent OpenTelemetry support across all Docker image variants.
141f757 to
cbd1dc2
Compare
Summary
python:3.12-slim-bookwormmulti-stage builds instead of vLLM/Miniconda bases.uv, pip, setuptools, wheel) and core service dependencies in the base image.enable_virtual_envis omitted.Validation
python -m py_compile xinference/core/utils.py xinference/core/tests/test_utils.pyDocker build was attempted locally, but the local Docker daemon is not running:
Cannot connect to the Docker daemon at unix:///Users/oliver/.docker/run/docker.sock. The new Docker CI workflow will build all three images on PRs that touch Docker inputs.