Fix: Add CMake package files to Windows artifact #28468#28897
Fix: Add CMake package files to Windows artifact #28468#28897Jayashree-mcw wants to merge 4 commits into
Conversation
|
@microsoft-github-policy-service agree |
decc75b to
d62ce45
Compare
d62ce45 to
92e9d98
Compare
There was a problem hiding this comment.
Pull request overview
This PR fixes the Windows C API artifact packaging so downstream CMake consumers can use find_package(onnxruntime) by shipping the generated CMake package metadata, while also making the packaging step more resilient when optional execution provider binaries aren’t present.
Changes:
- Adds
lib/cmake/onnxruntime/to the Windows artifact and copies generatedonnxruntimeConfig*.cmake+ exportedonnxruntimeTargets*.cmakefiles into it. - Adjusts artifact layout to place runtime DLLs under
bin/and import libraries/PDBs underlib/, and preserves the expectedinclude/onnxruntime/...hierarchy. - Adds existence checks for several optional provider/dependency binaries (CUDA, WebGPU, TensorRT) to avoid packaging failures when they aren’t built.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
135236d to
7116f4a
Compare
2e9e7a7 to
9e0b87e
Compare
9e0b87e to
6018ec1
Compare
|
@tianleiwu Kindly review my updated PR. |
tianleiwu
left a comment
There was a problem hiding this comment.
Review Summary
Thanks for tackling #28468. The overall direction is sound: moving DLLs to bin\ and import libs/PDBs to lib\ matches the CMake install destinations (RUNTIME -> bin, ARCHIVE -> lib, PUBLIC_HEADER -> include/onnxruntime), so the exported onnxruntimeTargets-*.cmake _IMPORT_PREFIX paths can resolve and find_package(onnxruntime) can work against the unzipped artifact. However, there are a few cross-cutting concerns that should be addressed before merge.
High priority
-
The DLL relocation (
lib\->bin\) is a breaking change to the published artifact layout and breaks a downstream consumer.tools/ci_build/github/windows/bundle_dlls_gpu.batrepackages the GPU zip and moves binaries out of...\lib\:move onnxruntime-win-x64-tensorrt\lib\onnxruntime.dll ...\lib\onnxruntime.dll move onnxruntime-win-x64-tensorrt\lib\onnxruntime_providers_tensorrt.dll ...\lib\... move onnxruntime-win-x64-tensorrt\lib\onnxruntime_providers_shared.dll ...\lib\...
After this PR those
.dllfiles live inbin\, so every.dllmove above fails and the bundled GPU package is silently shipped without its DLLs (the.lib/.pdbmoves still succeed, masking the failure). This script must be updated in lockstep, and any other consumer/doc that assumeslib\onnxruntime.dll(release notes, samples, existing user scripts) needs to be checked. Please call the layout change out explicitly in the PR description / release notes. -
CI has not been run on this branch (the PR checklist item is unchecked). This template only runs in the Windows C-API packaging pipeline and cannot be validated locally. The new logic that is most likely to fail there — the
onnxruntimeConfig.cmake/ConfigVersion.cmakecopies, thefor /d %%D in (...\CMakeFiles\Export\*)export-targets discovery, and the fullfind_packageround-trip — needs to be proven by an actual packaging run before merge. -
Unguarded CMake config copies can produce a silently incomplete artifact.
onnxruntimeConfig.cmakeandonnxruntimeConfigVersion.cmakeare copied withoutif existguards, unlike everything else in this PR. They are only generated for certain build configurations (e.g.cmake/CMakeLists.txtskips target-file generation for static builds with WebGPU/Emscripten+XNNPACK). When absent, thecopyprints an error but the step still "succeeds" because the task's exit code comes from the trailingfor /dloop. Either guard these copies and fail loudly when the config is expected but missing, or verify the config is always generated for the configs this template runs against.
Should fix
-
onnx_test_runner.exeshould not ship in the C-API redistributable. It is a test tool, inflates the package, and was not part of the original artifact. Please confirm intent or drop it. -
Header placement consistency. The session/C-API headers are copied flat into
include\onnxruntime\, while the CUDA provider headers nest underinclude\onnxruntime\core\providers\.... The CMake install instead places the session headers underinclude\onnxruntime\core\session\. Please confirm the layout you ship matches what the generatedonnxruntimeConfigadvertises as the include interface, sofind_packageconsumers resolve#include "onnxruntime_c_api.h"correctly.
Nits
The inline comments already on the PR cover the remaining lower-level batch-syntax items — non-ASCII characters / # vs REM comment lines, inconsistent indentation, trailing whitespace, the unquoted for /d source glob, and mkdir errorlevel behavior. Please address those as well; the non-ASCII arrows in comments in particular can corrupt the script under a non-UTF-8 cmd code page.
…API package Fixes microsoft#28468 - Move DLLs to bin\ and import libs/PDBs to lib\ to match CMake install destinations (RUNTIME -> bin, ARCHIVE -> lib) - Add CMake package files (onnxruntimeConfig.cmake, onnxruntimeConfigVersion.cmake, onnxruntimeTargets*.cmake) to lib\cmake\onnxruntime\ in the artifact - Add if exist guards for all optional provider binaries (CUDA, TensorRT, WebGPU, QNN) to prevent packaging failures when not built - Add loud exit /b 1 failure when CMake config files are missing - Update bundle_dlls_gpu.bat in lockstep: .dll source paths changed from \lib\ to \bin\, mkdir \bin added, 7z repack updated to include both \bin and \lib - Drop onnx_test_runner.exe from redistributable package - Replace non-ASCII arrows and # comments with REM and -> - Guard all mkdir calls with if not exist - Quote for /d glob path - Normalize indentation throughout
@tianleiwu thanks for the detailed review. All points have been addressed:
|
|
@tianleiwu Kindly review my updated PR. |
|
Hi @hariharans29, I am currently working on this issue. I made an initial commit, which was reviewed by @tianleiwu, who provided some insights. I have resolved those issues. I requested a review but haven’t received any response. Could you please review my PR instead? Thank you! |
|
CC: @sanaa-hamel-microsoft @eserscor to triage and review as this PR will directly influence release artifact structure |
|
Hi @tianleiwu, @hariharans29, @sanaa-hamel-microsoft, and @eserscor, Just following up on this PR. All review feedback from the initial review has been addressed, and the branch has been updated accordingly. Since this change fixes the packaging layout issue and is not yet included in a released version, I'd appreciate a review when time permits. If there are any remaining concerns or additional validation needed from my side, please let me know. Thank you. |
186fd8e to
6ecde97
Compare
tianleiwu
left a comment
There was a problem hiding this comment.
Follow-up on my earlier review. Note that the two changed files are unchanged since that round (the intervening commit is a rebase on top of main), so the points I raised before still apply. Re-reviewing the latest head surfaced one decisive issue that changes the conclusion, plus two concrete consumer breakages worth pinning down.
Blocking
-
find_package(onnxruntime)will still fail with this artifact —onnxruntimeTargets.cmakeis not produced by a plain build.onnxruntimeConfig.cmake/onnxruntimeConfigVersion.cmakeare emitted byconfigure_package_config_file()/write_basic_package_version_file()at CMake configure time, so the twoexit /b 1guards pass. ButonnxruntimeTargets.cmake(andonnxruntimeTargets-<config>.cmake) come frominstall(EXPORT onnxruntimeTargets ...)— these land inCMakeFiles/Export/<hash>/only when the install rule runs (cmake --install/ buildingINSTALL). The Windows packaging job (templates/win-ci.yml) builds viatools/ci_build/build.py ... --build_shared_lib --testwith no install step, so thefor /d %%D in (...\CMakeFiles\Export\*)loop finds nothing andonnxruntimeTargets.cmakeis never packaged.onnxruntimeConfig.cmaketheninclude()s a file that isn't there, andfind_packageerrors out. The "verified exported target files are included" check almost certainly reflects a local build whereINSTALLwas built, not what this pipeline produces. -
Recommended approach — package the CMake install tree, as Linux/macOS already do. The issue itself notes the cmake files are present on Linux/macOS. That works because
tools/ci_build/github/linux/copy_strip_binary.shpackages the install tree (mv installed/usr/local $ARTIFACT_NAME), which already contains a self-consistentlib/cmake/onnxruntime/{Config,ConfigVersion,Targets,Targets-<config>}.cmake, correctbin/+lib/placement, and headers. Adding acmake --install <build_dir> --config RelWithDebInfo --prefix <stage>step on Windows and zipping that staged tree would yield a workingfind_packageby construction and avoid hand-replicating install layout in batch (it also makes the two inline points below moot).
Inline (concrete consumer/layout breakages)
See the three inline comments on the DLL-relocation, header-relocation, and export-discovery lines.
Everything else from my prior review (CI not yet run on this branch; onnx_test_runner.exe shipping in a redistributable; the batch-syntax nits already flagged inline by the bot) still stands.
0ae9823 to
dccdd1c
Compare
|
@tianleiwu Thanks for the detailed review. All three points have been addressed in the latest commit: 1. DLL relocation (NuGet pipeline breakage)
2. Header flattening (cross-platform consistency)
3. CMake export harvesting (robustness)
Validation: To confirm the fix works locally, I used a PowerShell simulation script ( |
Description
Fixes #28468
The Windows C API artifact package was missing the CMake package metadata required for downstream CMake consumers. As a result, projects using the packaged artifact could not integrate ONNX Runtime through
find_package().In addition, the packaging script assumed the presence of optional provider binaries (CUDA, TensorRT, WebGPU), which could cause packaging failures when those components were not built.
Root Cause
The Windows artifact packaging template (
c-api-artifacts-package-and-publish-steps-windows.yml) did not:find_package().Changes Made
Modified:
tools/ci_build/github/azure-pipelines/templates/c-api-artifacts-package-and-publish-steps-windows.ymlCMake Package Support
Added packaging of:
onnxruntimeConfig.cmakeonnxruntimeConfigVersion.cmakeonnxruntimeTargets.cmakeonnxruntimeTargets-release.cmakeonnxruntimeTargets-relwithdebinfo.cmake(when generated)Created:
and copied the generated CMake package files into the final artifact.
Optional Component Handling
Added conditional copy operations for:
dxcompiler.dll,dxil.dll)This prevents packaging failures when optional execution providers are not built.
Expected Artifact Structure
How Consumers Benefit
Consumers can now use the packaged Windows artifact directly with standard CMake package discovery:
This improves compatibility with CMake-based projects and aligns the packaged artifact with expected CMake package conventions.
Testing
lib/cmake/onnxruntime.find_package(onnxruntime)resolves successfully using the packaged artifact.Type of Change