Skip to content

Add Linux-to-Windows-MSVC CMake cross-build toolchain #1292

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
MarijnS95 wants to merge 1 commit into
base: main
Choose a base branch
from
+310 −0
Draft

Add Linux-to-Windows-MSVC CMake cross-build toolchain #1292

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
192 changes: 192 additions & 0 deletions cmake/toolchains/windows-msvc.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,192 @@
# CMake toolchain for cross-compiling OffloadTest from a non-Windows host to
# the x86_64 Windows MSVC ABI. Targets clang as the compiler and lld-link
# as the linker; produces `offloader.exe` (and any other targets in the
# build) that can then be invoked under Wine + vkd3d-proton on Linux, on
# WSL via the WSL D3D12 path, or copied to a real Windows machine.
#
# The toolchain is **agnostic about how the Windows SDK / CRT are
# acquired**. Common paths:
#
# - xwin https://github.com/Jake-Shadle/xwin
# - msvc-wine https://github.com/mstorsjo/msvc-wine
# - EWDK (Enterprise WDK) Microsoft's portable SDK + build tools ZIP
# - Hand-copied from a Windows install of Visual Studio
#
# Any of these produce a tree containing UM / shared / UCRT / WinRT headers
# and matching libs. Point this toolchain at them via the four cache /
# environment variables below and the build is independent of which tool
# produced the tree.
#
# Required (set via -D… on the configure line, or as environment variables):
#
# WINDOWS_SDK_INCLUDE_ROOT Directory containing `um/`, `shared/`,
# `ucrt/`, and `winrt/` header subdirs.
# xwin layout: ~/.xwin-cache/splat/sdk/include
# msvc-wine: <root>/kits/<ver>/Include/<ver>
# EWDK: <ewdk>/Program Files/Windows Kits/10/Include/<ver>
# WINDOWS_SDK_LIB_ROOT Directory containing `ucrt/x86_64/` and
# `um/x86_64/` lib subdirs.
# xwin layout: ~/.xwin-cache/splat/sdk/lib
# msvc-wine: <root>/kits/<ver>/Lib/<ver>
# EWDK: <ewdk>/Program Files/Windows Kits/10/Lib/<ver>
# WINDOWS_CRT_INCLUDE_DIR Directory containing MSVC CRT headers
# (`stdint.h`, `stdio.h`, `xmmintrin.h` proxies,
# `cwchar`, …).
# xwin layout: ~/.xwin-cache/splat/crt/include
# msvc-wine: <root>/vc/Tools/MSVC/<ver>/include
# EWDK: <ewdk>/Program Files/Microsoft Visual Studio/<ed>/<ver>/VC/Tools/MSVC/<ver>/include
# WINDOWS_CRT_LIB_DIR Directory containing MSVC CRT libs
# (`msvcrt.lib`, `vcruntime.lib`, …) for x86_64.
# xwin layout: ~/.xwin-cache/splat/crt/lib/x86_64
# msvc-wine: <root>/vc/Tools/MSVC/<ver>/lib/x64
# EWDK: <ewdk>/…/MSVC/<ver>/lib/x64
#
# Optional (sensible defaults below):
#
# WINDOWS_TARGET_ARCH Target architecture triple stem; default
# `x86_64`. Set to `aarch64` for ARM64 Windows.
# CMAKE_C_COMPILER / Default: the `clang` / `clang++` on PATH.
# CMAKE_CXX_COMPILER Set to a specific compiler when the host
# has multiple clang versions.
#
# Example configure (xwin layout, adjust paths for your SDK source):
#
# cmake -S llvm-project/llvm -B build-win -G Ninja \
# -DCMAKE_TOOLCHAIN_FILE=$PWD/OffloadTest/cmake/toolchains/windows-msvc.cmake \
# -DCMAKE_BUILD_TYPE=Release \
# -DLLVM_TABLEGEN=$PWD/build/bin/llvm-tblgen \
# -DCLANG_TABLEGEN=$PWD/build/bin/clang-tblgen \
# -DLLVM_ENABLE_PROJECTS=clang \
# -DLLVM_EXTERNAL_PROJECTS=OffloadTest \
# -DLLVM_EXTERNAL_OFFLOADTEST_SOURCE_DIR=$PWD/OffloadTest \
# -DOFFLOADTEST_ENABLE_VULKAN=OFF \
# -DOFFLOADTEST_ENABLE_METAL=OFF \
# -DOFFLOADTEST_ENABLE_D3D12=ON \
# -DD3D12_INCLUDE_DIRS=$HOME/.xwin-cache/splat/sdk/include/um \
# -DD3D12_LIBRARIES='d3d12.lib;dxcore.lib;dxguid.lib;d3dcompiler.lib' \
# -DWINDOWS_SDK_INCLUDE_ROOT=$HOME/.xwin-cache/splat/sdk/include \
# -DWINDOWS_SDK_LIB_ROOT=$HOME/.xwin-cache/splat/sdk/lib \
# -DWINDOWS_CRT_INCLUDE_DIR=$HOME/.xwin-cache/splat/crt/include \
# -DWINDOWS_CRT_LIB_DIR=$HOME/.xwin-cache/splat/crt/lib/x86_64 \
# -DPNG_INTEL_SSE=OFF
#
# cmake --build build-win --target offloader
#
# See docs/cross-build.md for the rationale behind each flag and tips for
# acquiring the SDK from the various sources above.

set(CMAKE_SYSTEM_NAME Windows)
set(CMAKE_SYSTEM_PROCESSOR AMD64)

# CMake re-loads the toolchain file inside `try_compile`, in a fresh scope
# that doesn't see the outer cache. Mark these variables on
# `CMAKE_TRY_COMPILE_PLATFORM_VARIABLES` so they propagate into the inner
# project, and read them from the environment as a fallback so callers
# can use either `-DWINDOWS_…=…` or `export WINDOWS_…=…`.
set(_windows_msvc_required_vars
WINDOWS_SDK_INCLUDE_ROOT
WINDOWS_SDK_LIB_ROOT
WINDOWS_CRT_INCLUDE_DIR
WINDOWS_CRT_LIB_DIR
)
list(APPEND CMAKE_TRY_COMPILE_PLATFORM_VARIABLES
${_windows_msvc_required_vars}
WINDOWS_TARGET_ARCH
)

foreach(var ${_windows_msvc_required_vars})
if(NOT DEFINED ${var} AND DEFINED ENV{${var}})
set(${var} "$ENV{${var}}")
endif()
if(NOT DEFINED ${var})
message(FATAL_ERROR
"${var} is required by the windows-msvc cross toolchain. "
"See cmake/toolchains/windows-msvc.cmake for the expected layout "
"and docs/cross-build.md for SDK-acquisition options.")
endif()
endforeach()

if(NOT DEFINED WINDOWS_TARGET_ARCH)
set(WINDOWS_TARGET_ARCH x86_64)
endif()

# Default to clang / clang++ on PATH. Callers can override before the
# project() call by setting CMAKE_C_COMPILER / CMAKE_CXX_COMPILER on the
# configure line.
if(NOT CMAKE_C_COMPILER)
set(CMAKE_C_COMPILER clang)
endif()
if(NOT CMAKE_CXX_COMPILER)
set(CMAKE_CXX_COMPILER clang++)
endif()

# Drive Windows tooling from system LLVM utilities. lld-link / llvm-rc /
# llvm-lib are required (host LLVM build doesn't always include lld).
# CMake's MSVC archive rule uses `<CMAKE_LINKER> /lib /OUT:foo.lib obj…`
# (when CMAKE_AR is unset) — setting CMAKE_AR forces a GNU `ar qc` style
# that llvm-lib rejects, so leave CMAKE_AR unset.
if(NOT CMAKE_LINKER)
find_program(_lld_link NAMES lld-link REQUIRED)
set(CMAKE_LINKER ${_lld_link})
endif()
if(NOT CMAKE_RC_COMPILER)
find_program(_llvm_rc NAMES llvm-rc REQUIRED)
set(CMAKE_RC_COMPILER ${_llvm_rc})
endif()

set(CMAKE_C_COMPILER_TARGET ${WINDOWS_TARGET_ARCH}-pc-windows-msvc)
set(CMAKE_CXX_COMPILER_TARGET ${WINDOWS_TARGET_ARCH}-pc-windows-msvc)

# Force lld-link via clang's `-fuse-ld=lld` so the host's default linker
# isn't picked up. CMake's LINKER_TYPE mechanism does the same on newer
# CMake versions; both are wired here for portability.
set(CMAKE_C_USING_LINKER_LLD "-fuse-ld=lld")
set(CMAKE_CXX_USING_LINKER_LLD "-fuse-ld=lld")
set(CMAKE_LINKER_TYPE LLD)

# `-Xclang -nostdsysteminc` suppresses clang's default Linux system header
# search (e.g. `/usr/include`) so xwin's CRT / Windows SDK headers aren't
# shadowed by glibc when host paths happen to leak into `-I`. Unlike
# `-nostdinc`, it keeps clang's *builtin* include path
# (`<resource-dir>/include`) reachable — that's where the SSE / AVX
# intrinsic wrappers (`xmmintrin.h`, `emmintrin.h`, …) live, and without
# them libpng's SIMD optimizations end up with undefined `_mm_*` symbols
# at link time.
set(_includes
"-Xclang -nostdsysteminc"
"-isystem ${WINDOWS_CRT_INCLUDE_DIR}"
"-isystem ${WINDOWS_SDK_INCLUDE_ROOT}/ucrt"
"-isystem ${WINDOWS_SDK_INCLUDE_ROOT}/um"
"-isystem ${WINDOWS_SDK_INCLUDE_ROOT}/shared"
"-isystem ${WINDOWS_SDK_INCLUDE_ROOT}/winrt"
)
string(JOIN " " _includes_str ${_includes})

set(_linker_libpaths
"-Xlinker" "/libpath:${WINDOWS_CRT_LIB_DIR}"
"-Xlinker" "/libpath:${WINDOWS_SDK_LIB_ROOT}/ucrt/${WINDOWS_TARGET_ARCH}"
"-Xlinker" "/libpath:${WINDOWS_SDK_LIB_ROOT}/um/${WINDOWS_TARGET_ARCH}"
)
string(JOIN " " _linker_libpaths_str ${_linker_libpaths})

# Force the release-flavour CRT. Several SDK-acquisition tools (xwin
# included) only ship `msvcrt.lib`, not `msvcrtd.lib`, so the default
# debug-CRT pull breaks the configure-time try_compile.
set(CMAKE_MSVC_RUNTIME_LIBRARY "MultiThreadedDLL")
set(CMAKE_TRY_COMPILE_CONFIGURATION Release)

set(CMAKE_C_FLAGS_INIT "${_includes_str} -fms-compatibility -fms-extensions")
set(CMAKE_CXX_FLAGS_INIT "${_includes_str} -nostdinc++ -fms-compatibility -fms-extensions")
set(CMAKE_EXE_LINKER_FLAGS_INIT "${_linker_libpaths_str}")
set(CMAKE_SHARED_LINKER_FLAGS_INIT "${_linker_libpaths_str}")
set(CMAKE_MODULE_LINKER_FLAGS_INIT "${_linker_libpaths_str}")
set(CMAKE_STATIC_LINKER_FLAGS_INIT "")

# Cross-build: don't search the host filesystem for libraries or includes,
# only the toolchain-provided ones. Programs on the host are still
# reachable so tablegen / `find_program(DXC_EXECUTABLE …)` etc. can find
# their host-arch counterparts.
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
118 changes: 118 additions & 0 deletions docs/cross-build.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
# Cross-building OffloadTest on Linux for Windows / Wine

`cmake/toolchains/windows-msvc.cmake` is a CMake toolchain file that lets a non-Windows host (typically Linux) build `offloader.exe` and any other targets in the project against the x86_64 Windows MSVC ABI. The cross-built binary can then be run under Wine + vkd3d-proton on Linux for an end-to-end D3D12 / Vulkan smoke loop without dual-booting or a Windows VM.

This is currently a contributor-iteration aid — no in-tree CI runs this configuration today. Treat it as a "really experimental, YMMV" workflow until the wg-hlsl support-tier discussion lands a place for it in `CI.md`.

## What you need

- A recent **clang** with both Linux and Windows-MSVC targets. Distro clang ≥ 17 works; bleeding-edge from an LLVM build is also fine. `clang++` and `clang` must be on `PATH` (or pointed at via `CMAKE_C_COMPILER` / `CMAKE_CXX_COMPILER`).
- **lld-link**, **llvm-rc**, **llvm-lib** on `PATH`. `lld-link` is the cross linker; the others handle Windows resource compilation and import-library generation. Distro `llvm` package usually ships all three.
- A **Windows SDK + CRT tree** (see acquisition options below).
- For runtime testing: **Wine** and **vkd3d-proton** (D3D12) or **wine-vulkan** (Vulkan). `binfmt_misc` registration for `.exe` (the `wine-binfmt` package on Arch, or the equivalent on your distro) lets the `.exe` run as if it were a native binary from any shell — no explicit `wine` prefix needed.

## Acquiring the Windows SDK + CRT

The toolchain is agnostic about *how* you get the SDK headers / libs / CRT. Four common paths, none preferred:

- **[xwin](https://github.com/Jake-Shadle/xwin)** — Rust tool that downloads the MSVC build tools via the official Visual Studio installer manifests and extracts a flat tree of headers and libs. Lightweight, single-command setup. Output is normally at `~/.xwin-cache/splat/`.
- **[msvc-wine](https://github.com/mstorsjo/msvc-wine)** — installs the actual Visual Studio Build Tools through Wine and scripts up wrapper compilers. Heavier than xwin but uses official Microsoft bits end-to-end. Widely used by FFmpeg / mpv / Mesa projects.
- **EWDK (Enterprise WDK)** — Microsoft's portable SDK + Build Tools ZIP. Mount it on Linux, set env vars, point CMake at the headers and libs. Licensed but free, no Wine involved.
- **Manually copied from a Windows install of Visual Studio.** `rsync C:\Program Files (x86)\Windows Kits\10\Include\10.0.x.0\{um,shared,ucrt}` and the matching `Lib\…` to a Linux directory. Works if you've got a Windows machine around once for setup.

Whichever you use, point the toolchain at the resulting tree with these four variables:

| Variable | What it contains |
|----------------------------|------------------------------------------------------------------------|
| `WINDOWS_SDK_INCLUDE_ROOT` | Parent of `um/`, `shared/`, `ucrt/`, `winrt/` SDK headers |
| `WINDOWS_SDK_LIB_ROOT` | Parent of `ucrt/x86_64/` and `um/x86_64/` SDK libs |
| `WINDOWS_CRT_INCLUDE_DIR` | MSVC CRT headers (`stdint.h`, `cwchar`, …) |
| `WINDOWS_CRT_LIB_DIR` | MSVC CRT libs for the target arch (`msvcrt.lib`, `vcruntime.lib`, …) |

The toolchain file's header comment maps each variable to the corresponding directory in xwin / msvc-wine / EWDK trees.

## Configure + build

Two-stage build: a native host build that produces tablegen helpers + the host LLVM, then a cross build that targets Windows MSVC and consumes the host tablegens.

### Stage 1 — host (Linux) build

A normal LLVM + OffloadTest build. Produces `llvm-tblgen`, `clang-tblgen`, and the host `offloader` / `clang` / `dxc` you'd normally use for the Linux Vulkan lit suite.

```bash
cmake -S llvm-project/llvm -B build -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_PROJECTS='clang;clang-tools-extra' \
-DLLVM_EXTERNAL_PROJECTS=OffloadTest \
-DLLVM_EXTERNAL_OFFLOADTEST_SOURCE_DIR=$PWD/OffloadTest
cmake --build build --target offloader
```

### Stage 2 — Windows-MSVC cross build

Uses the toolchain file from this PR, points `LLVM_TABLEGEN` / `CLANG_TABLEGEN` at the stage-1 binaries, and disables Vulkan / Metal (Linux Vulkan headers would shadow the xwin SDK include path via `find_package(Vulkan)`; Metal is macOS-only). `D3D12_INCLUDE_DIRS` / `D3D12_LIBRARIES` are passed directly because `FindD3D12.cmake` keys off a Win10 SDK registry path that doesn't match any of the cross-SDK layouts.

```bash
cmake -S llvm-project/llvm -B build-win -G Ninja \
-DCMAKE_TOOLCHAIN_FILE=$PWD/OffloadTest/cmake/toolchains/windows-msvc.cmake \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_TABLEGEN=$PWD/build/bin/llvm-tblgen \
-DCLANG_TABLEGEN=$PWD/build/bin/clang-tblgen \
-DLLVM_ENABLE_PROJECTS=clang \
-DLLVM_EXTERNAL_PROJECTS=OffloadTest \
-DLLVM_EXTERNAL_OFFLOADTEST_SOURCE_DIR=$PWD/OffloadTest \
-DLLVM_TARGETS_TO_BUILD='X86;SPIRV' \
-DOFFLOADTEST_TEST_CLANG=OFF \
-DCMAKE_DISABLE_FIND_PACKAGE_Vulkan=TRUE \
-DOFFLOADTEST_ENABLE_VULKAN=OFF \
-DOFFLOADTEST_ENABLE_METAL=OFF \
-DOFFLOADTEST_ENABLE_D3D12=ON \
-DD3D12_INCLUDE_DIRS=$HOME/.xwin-cache/splat/sdk/include/um \
-DD3D12_LIBRARIES='d3d12.lib;dxcore.lib;dxguid.lib;d3dcompiler.lib' \
-DWINDOWS_SDK_INCLUDE_ROOT=$HOME/.xwin-cache/splat/sdk/include \
-DWINDOWS_SDK_LIB_ROOT=$HOME/.xwin-cache/splat/sdk/lib \
-DWINDOWS_CRT_INCLUDE_DIR=$HOME/.xwin-cache/splat/crt/include \
-DWINDOWS_CRT_LIB_DIR=$HOME/.xwin-cache/splat/crt/lib/x86_64 \
-DPNG_INTEL_SSE=OFF
cmake --build build-win --target offloader
```

Adjust the four `WINDOWS_…` paths if you're using msvc-wine / EWDK / a hand-copy instead of xwin.

`PNG_INTEL_SSE=OFF` is needed because libpng's hand-rolled SSE2 wrapper functions emit external `_mm_*` symbols that `lld-link` doesn't resolve under cross-MSVC (the intrinsic *headers* are reachable, but the inline-resolution path differs). Disabling its SIMD codegen leaves the C fallback, which is fine for a test runner.

## Running the cross-built binary

With `binfmt_misc` configured for `.exe`:

```bash
./build-win/bin/offloader.exe --api dx pipeline.yaml shader.o
```

Without `binfmt_misc`:

```bash
wine ./build-win/bin/offloader.exe --api dx pipeline.yaml shader.o
```

Both routes pick up the host's Wine prefix, so `WINEPREFIX` / `WINEDEBUG` / `VKD3D_*` / `DXVK_*` environment variables work the same as they would for any other Wine binary.

## Running the lit suite

The lit configuration the native build uses works from the cross-build directory unchanged — `add_offloadtest_lit_suite` resolves `%offloader` to `$<TARGET_FILE:offloader>`, which for the cross build is the cross-compiled `.exe`. With `binfmt_misc` in place lit invokes the binary like any other test program and Wine takes over transparently:

```bash
ninja -C build-win check-hlsl-d3d12
```

`WINEDEBUG=-all` and `VKD3D_CONFIG=…` flow through the environment exactly as for a regular Wine invocation.

If `binfmt_misc` for `.exe` isn't registered on the host, lit can't directly invoke the cross-built binary — there is no in-tree launcher hook today. Registering the binfmt entry once (most distros' `wine-binfmt` package does this for you) avoids needing one.

## Gotchas

- **Don't set `CMAKE_AR`.** The toolchain deliberately leaves it unset so CMake's MSVC-mode archive rule picks the right `<CMAKE_LINKER> /lib /OUT:foo.lib obj…` invocation. Setting `CMAKE_AR=llvm-lib` forces CMake to a GNU `ar qc …` style that `llvm-lib` rejects.
- **`find_package(Vulkan)` on the host finds Linux Vulkan.** The Linux SDK's `/usr/include` then shadows the xwin SDK headers when the discovered Vulkan include path makes it into any target's `-I`. Pass `-DCMAKE_DISABLE_FIND_PACKAGE_Vulkan=TRUE` to suppress it on the cross build (the offloader builds fine without Vulkan support on the cross side; Wine routes the cross-built `vulkan-1.dll` to the host's Vulkan loader at runtime regardless).
- **`-Xclang -nostdsysteminc`, not `-nostdinc`.** `-nostdinc` also strips clang's *builtin* include directory, which is where the SSE / AVX intrinsic wrappers live; libpng then fails to link with undefined `_mm_*` symbols. `-Xclang -nostdsysteminc` keeps the builtin path while suppressing the default Linux system search.
- **xwin / msvc-wine only ship the release-flavour CRT.** The default debug-CRT (`msvcrtd.lib`) is missing from those trees, so `CMAKE_MSVC_RUNTIME_LIBRARY=MultiThreadedDLL` and `CMAKE_TRY_COMPILE_CONFIGURATION=Release` are set by the toolchain. If your SDK source ships both, you can override these in a follow-up `-D…` after the toolchain file loads.

Loading