feat(TRI-1118): add --release-version + PEP 440/817 wheel scheme - #8845
Merged
mc-nv merged 11 commits intoJun 22, 2026
Merged
Conversation
build_wheel.py (src/python/build_wheel.py):
- --release-version CLI flag + TRITON_RELEASE_VERSION env-var fallback.
Precedence: CLI flag > env var > in-tree TRITON_VERSION file. CI can
pin a release tag without editing source-tree TRITON_VERSION.
- Classifies the base version: X.Y.Z (release-semantic) emits
PEP 440 normalized output (no local segment, no build tag, no
variant label) -- PyPI-clean. Anything else emits a PEP 817 variant
label (nv<container>.cu<major><minor>) appended to the wheel
filename, keeping the PEP 427 build tag for CI rerun disambiguation.
- ANSI-colored info lines surface the chosen path in CI logs.
build.py:
- Propagates FLAGS.release_version into the wheel build via a single
-e TRITON_RELEASE_VERSION=<value> on the docker-run command that
drives ./cmake_build inside the tritonserver_buildbase container.
For host (--no-container-build) builds, an os.environ.setdefault
in main() ensures the cmake_build subprocess inherits the same env.
Refs TRI-1118.
Merged
11 tasks
…shape
Previously `os.environ.setdefault("TRITON_RELEASE_VERSION", FLAGS.release_version)`
fired unconditionally, which forced PEP 440 release-clean wheel naming
even when the build was for a dev / pre-release `--version` (e.g.
`2.70.0.dev0`) -- the wheel would mismatch the rest of the build.
New rule (TRI-1118):
--release-version <X.Y.Z> -> always set TRITON_RELEASE_VERSION
(explicit override).
FLAGS.version is X.Y.Z -> propagate FLAGS.version as
TRITON_RELEASE_VERSION (release path).
FLAGS.version is anything else -> leave TRITON_RELEASE_VERSION unset
so build_wheel.py reads the in-tree
TRITON_VERSION file (PEP 817 variant).
Also:
- `--release-version` default changed from DEFAULT_TRITON_VERSION_MAP
["release_version"] to None so we can detect explicit pass-through.
- `docker run -e` now mirrors `os.environ`: only forwarded when set.
Refs TRI-1118.
…uildbase build_wheel.py reads CI_PIPELINE_ID (for the PEP 427 build tag) and NVIDIA_UPSTREAM_VERSION (for the PEP 817 variant label's nv-part) from the env at wheel-build time, but those vars are set on the GitLab CI runner host -- they were never reaching the tritonserver_buildbase container. Result: dev wheels came out without the build tag and with a fallback nv-part (via NVIDIA_TRITON_SERVER_VERSION ENV in the image). Forward both via `docker run -e`, plus NVIDIA_BUILD_ID as a backup build-tag source for non-CI runs that pass --build-id. CUDA_VERSION is intentionally not forwarded -- the container's own CUDA base image must define it; host CUDA may differ (see comment in core/python/build_wheel.py:_detect_cuda_version). Refs TRI-1118.
…EP 427 build tag Mirror of the core change. When the resolved base version matches X.Y.Z.devN and CI_PIPELINE_ID is a positive integer, replace the dev counter with the pipeline id (e.g. 2.70.0.dev0 + CI_PIPELINE_ID=12345 -> 2.70.0.dev12345). The corresponding `--build-number=<N>` injection on `setup.py bdist_wheel` is removed; the pipeline id now lives in the version string itself, where PyPI accepts it and pip can sort by it. Refs TRI-1118.
Mirror of the core fix. _repair_wheel_with_auditwheel() now writes the manylinux wheel to dist_dir (so CMake's `install(DIRECTORY WHEEL_OUT_DIR)` copies it into /opt/tritonserver/python/) and removes the original linux_<arch> wheel on success. Dev-counter regex relaxed to ^(\d+\.\d+\.\d+)\.?dev\d*$ to accept the legacy `X.Y.Zdev` shape used in the in-tree TRITON_VERSION file. Refs TRI-1118.
…o .so Mirror of the core change. Adds `_wheel_has_so()` and routes: - Has `.so` -> `auditwheel repair` -> PEP 513/599/600 manylinux - No `.so` -> `py3-none-any` (PEP 425 pure-Python); manylinux omitted Replaces the previous hardcoded `manylinux_2_28_<arch>` fallback that was emitted whenever auditwheel reported "no ELF" -- which incorrectly attached a manylinux compatibility promise to wheels with no native extensions. Refs TRI-1118.
build_wheel.py needs auditwheel to apply PEP 513/599/600 manylinux tags
to wheels containing native .so files. Previously the tool's presence
relied entirely on the BASE_IMAGE:
- py3-base-rhel (cuda:*-manylinux-* base) -- shipped with auditwheel
by manylinux convention. Worked.
- py3-base (cuda:*-ubuntu* base) -- did NOT ship auditwheel.
Wheels with .so silently stayed at the raw linux_<arch> tag and were
rejected by PyPI / unusable from Artifactory PyPI mirrors.
Add `auditwheel` to the pip install list in both create_dockerfile_buildbase
and create_dockerfile_buildbase_rhel so every buildbase image has it
regardless of base. Unpinned -- recent auditwheel releases all understand
PEP 600 manylinux_X_Y semantics; pin if a regression appears.
Refs TRI-1118.
Mirror of the core fix. Symptom seen in CI:
tritonserver-2.70.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Cause: CMake invokes the wheel custom command twice (build + install
phases); the second pass finds the manylinux wheel left over from the
first pass alongside the newly-built linux_<arch> wheel, and runs
auditwheel on both. auditwheel applied to an already-tagged wheel
emits a compressed PEP 425 tag set.
Fixes:
1. Clean wheel/dist/ before `setup.py bdist_wheel` so leftovers
from a previous CMake pass can't accumulate.
2. In _repair_wheel_with_auditwheel, skip wheels whose filename
already carries a manylinux or musllinux tag.
Refs TRI-1118.
…sion Mirror of the core change. If a wheel ends up with a compressed PEP 425 platform-tag set containing multiple manylinux entries, collapse to the highest manylinux version (strictest glibc baseline). Refs TRI-1118.
Symptom (pipeline 55073480 / py3-wheel-publish): the publish job's
`jf rt upload 'tritonfrontend-*.whl'` finds no matches in
/opt/tritonserver/python/ inside the base image and aborts with
`[Error] No errors, but also no files affected (fail-no-op flag)`.
Cause: in core_build(), the existing `cp .../install/python/triton*.whl
.../install/python/` line is supposed to pick up BOTH the core's
tritonserver wheel AND the server's tritonfrontend wheel. The glob is
expanded at shell-execution time against repo_install_dir/python/.
The trace shows only `tritonserver-*-manylinux_2_39_x86_64.whl` ever
lands there -- CMake's `install(DIRECTORY ${WHEEL_OUT_DIR})` rule for
tritonfrontend (server/src/python/CMakeLists.txt) lives in the
triton-server sub-build (build/triton-server/python/) and is not
traversed by the outer install pass that handles the core wheel from
_deps/repo-core-build. Result: tritonfrontend wheel is built but never
copied to the base image's /opt/tritonserver/python/, and the publish
job has nothing to upload.
Workaround in build.py rather than CMake: after `makeinstall()`, run
`find <repo_build_dir> -path '*/wheel/dist/tritonfrontend-*.whl'
-exec cp {} <repo_install_dir>/python/ \;` so the wheel lands in the
same directory the subsequent `triton*.whl` glob already targets.
Safe no-op when the wheel is absent (downstream builds with frontend
disabled). Defensive against both build locations the wheel could end
up in across CMake layouts.
Refs TRI-1118.
The legacy `find /opt/tritonserver/python ... | xargs -I {} pip install`
silently no-ops when find produces zero matches: xargs runs the command
zero times and the layer succeeds. That masked the TRI-1118 install-dir
gap (pipeline 55073480) -- tritonfrontend wheel never reached
/opt/tritonserver/python, the layer succeeded, and the publish job was
the first to notice ("no files affected").
Replace with an explicit existence check per package, then xargs the
list through pip. A missing wheel now breaks the Docker build at this
layer with a clear error, rather than as a silent downstream regression.
Defensive layer on top of the cp-time fix in core_build() (commit
13675ec).
Refs TRI-1118.
mc-nv
marked this pull request as ready for review
June 22, 2026 18:32
whoisj
approved these changes
Jun 22, 2026
mc-nv
deleted the
mchornyi/tri-1118-ci-artifactory-publish-wheels-to-the-artifactory
branch
June 22, 2026 20:21
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does the PR do?
Two coordinated changes for TRI-1118 release-wheel publishing.
src/python/build_wheel.py— same scheme as the core PR:--release-versionCLI flag +TRITON_RELEASE_VERSIONenv-var fallback. Precedence: CLI flag > env var > in-treeTRITON_VERSIONfile.X.Y.Z(release-semantic) → PEP 440 normalized (PyPI-clean, no local segment, no build tag, no variant label). Anything else → PEP 817 variant label (nv<container>.cu<major><minor>) appended to the wheel filename, with the PEP 427 build tag retained for CI rerun disambiguation.build.py— wires the flag through to the wheel build step:-e TRITON_RELEASE_VERSION=${FLAGS.release_version}on thedocker runcommand increate_docker_build_script.os.environ.setdefaultinmain()so the same env reaches the cmake_build subprocess on the--no-container-build(host) path.Verified via dryrun:
./build.py --release-version 9.9.9 --enable-all --dryrunwrites the value intobuild/docker_build.Checklist
<commit_type>: <Title>Commit Type:
Related PRs:
triton-inference-server/core#503
Where should the reviewer start?
src/python/build_wheel.py—_compose_variant_labeland the inline release/variant decision inmain().build.py— the two-line wiring (-eon docker run +os.environ.setdefault).Test plan:
./build.py --release-version 9.9.9 --enable-all --dryrun→ grepTRITON_RELEASE_VERSION=9.9.9inbuild/docker_build.Helper unit-test for
_compose_variant_label+ classifier (no Docker — matrix in TRI-1118).Container build with
--release-version 2.70.0→ PyPI-cleantritonfrontend-2.70.0-...whl.Container build with
--release-version 2.70.0.dev0→ variant-labeled wheel.L0_* tests pass with no new failures.
CI Pipeline ID:
Caveats:
variant.jsonmetadata. Real PEP 817 tooling will want a follow-up.Background
TRI-1118: CI artifactory publishing. PyPI rejects PEP 440 local-version segments and PEP 427 build tags, so release wheels need to be clean. Dev wheels still need disambiguation across container trains — PEP 817 variant labels do that without polluting the version.
Related Issues: