ci(windows): ship Ninja build as ninja-windows classifier alongside permanent MSVC

Builds on the prior Ninja evaluation jobs. Per owner decision, the MSVC /
Visual Studio Windows build is the default JAR and is kept permanently (the
sccache cache loss on it is accepted); the Ninja Multi-Config build is shipped
ALONGSIDE it as a separate classifier JAR, never as a replacement. The two
generators produce different jllama.dll files, so they cannot share a resource
path in one JAR — hence the classifier (mirrors the cuda / opencl-android
pattern). Result: 4 permanent Windows build jobs, both generators distributed
and tested end-to-end.

- pom.xml: add `windows-ninja` profile producing a <classifier>ninja-windows</classifier>
  JAR from ${outputDirectory}_windows_ninja (separate compile pass + resource
  copy + classified jar; mirrors cuda / opencl-android).
- publish.yml: the package, publish-snapshot, and publish-release jobs download
  Windows-{x86_64,x86}-ninja into src/main/resources_windows_ninja/ and activate
  the `windows-ninja` profile (-P ...,windows-ninja). Add a
  test-java-windows-x86_64-ninja job that loads the Ninja DLL via JNI and runs
  the full model-backed suite (parity with test-java-windows-x86_64). Wire the
  Ninja build + Java-test jobs into the package `needs:` graph.
- .gitignore: ignore src/main/resources_windows_ninja/ (CI-staged, never committed).
- README.md: add the `ninja-windows` classifier row + dependency snippet.
- CLAUDE.md: add "Windows Ninja artifact" section; refresh the sccache "Windows"
  note (no CMakeLists change — routing is CI-download + pom-profile, not a GGML flag).
- TODO.md: rewrite the Windows section to the final dual-build design (MSVC kept
  forever; remaining work is cache-hit verification, not a redesign).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SfvSZ76NW4e1qX1PjL4RKq

Author: claude

.github/workflows/publish.yml

@@ -1191,6 +1191,99 @@ jobs:
             ${{ github.workspace }}/src/main/resources/net/ladenthin/llama/**/*
           if-no-files-found: warn
 
+  # Java/inference validation of the Ninja-built x86_64 DLL (the analogue of
+  # test-java-windows-x86_64 for the MSVC build). Loads the Ninja jllama.dll via
+  # JNI and runs the full model-backed suite, so both Windows generators are
+  # validated end-to-end before the `ninja-windows` classifier JAR ships.
+  test-java-windows-x86_64-ninja:
+    name: Java Tests Windows 2025 x86_64 (Ninja, eval)
+    needs: build-windows-x86_64-ninja
+    runs-on: windows-2025-vs2026
+    steps:
+      - uses: actions/checkout@v7
+      - name: Display CPU Info
+        shell: pwsh
+        run: |
+          Write-Host "=== CPU Information (Get-CimInstance - All Properties) ==="
+          Get-CimInstance Win32_Processor | Select-Object * | Format-List
+          Write-Host ""
+          Write-Host "=== CPU Information (systeminfo) ==="
+          systeminfo | Select-String "Processor"
+          Write-Host ""
+          Write-Host "=== CPU Information (Get-ComputerInfo) ==="
+          Get-ComputerInfo -Property "CsProcessors*" 2>$null || Write-Host "Get-ComputerInfo not available"
+      - uses: actions/download-artifact@v8
+        with:
+          name: Windows-x86_64-ninja
+          path: ${{ github.workspace }}/src/main/resources/net/ladenthin/llama/
+      - name: Download text generation model
+        run: curl -L --proto =https --proto-redir =https --fail --retry 5 --retry-all-errors $env:MODEL_URL --create-dirs -o models/$env:MODEL_NAME
+      - name: Download reranking model
+        run: curl -L --proto =https --proto-redir =https --fail --retry 5 --retry-all-errors $env:RERANKING_MODEL_URL --create-dirs -o models/$env:RERANKING_MODEL_NAME
+      - name: Download draft model
+        run: curl -L --proto =https --proto-redir =https --fail --retry 5 --retry-all-errors $env:DRAFT_MODEL_URL --create-dirs -o models/$env:DRAFT_MODEL_NAME
+      - name: Download reasoning model
+        run: curl -L --proto =https --proto-redir =https --fail --retry 5 --retry-all-errors $env:REASONING_MODEL_URL --create-dirs -o models/$env:REASONING_MODEL_NAME
+      - name: Download tool-calling model
+        run: curl -L --proto =https --proto-redir =https --fail --retry 5 --retry-all-errors $env:TOOL_MODEL_URL --create-dirs -o models/$env:TOOL_MODEL_NAME
+      - name: Download vision model (issues #103 / #34)
+        run: curl -L --proto =https --proto-redir =https --fail --retry 5 --retry-all-errors $env:VISION_MODEL_URL --create-dirs -o models/$env:VISION_MODEL_NAME
+      - name: Download vision mmproj
+        run: curl -L --proto =https --proto-redir =https --fail --retry 5 --retry-all-errors $env:VISION_MMPROJ_URL --create-dirs -o models/$env:VISION_MMPROJ_NAME
+      - name: List files in models directory
+        run: ls -l models/
+      - name: Validate model files
+        run: .github\validate-models.bat
+      - uses: actions/setup-java@v5
+        with:
+          distribution: 'temurin'
+          java-version: ${{ env.JAVA_VERSION }}
+      - name: Memory before tests
+        run: Get-CimInstance Win32_OperatingSystem | Select-Object FreePhysicalMemory,TotalVisibleMemorySize | Format-List
+        shell: pwsh
+      - name: Enable WER LocalDumps for java.exe
+        # Windows Error Reporting writes minidumps when java.exe (or any other
+        # registered process) crashes via __fastfail / abort / unhandled SEH.
+        # We use it as the Windows analogue of Linux core dumps so that a JVM
+        # crash inside the JNI layer leaves us a real native callstack instead
+        # of just surefire's "VM terminated without saying goodbye" line.
+        # DumpType=2 == MiniDumpWithFullMemory; the workspace dumps/ folder is
+        # globbed by the failure-upload step below.
+        shell: pwsh
+        run: |
+          $key = 'HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\java.exe'
+          New-Item -Path $key -Force | Out-Null
+          New-Item -Path "${{ github.workspace }}\dumps" -ItemType Directory -Force | Out-Null
+          New-ItemProperty -Path $key -Name 'DumpFolder' -Value "${{ github.workspace }}\dumps" -PropertyType ExpandString -Force | Out-Null
+          New-ItemProperty -Path $key -Name 'DumpType'   -Value 2 -PropertyType DWord -Force | Out-Null
+          New-ItemProperty -Path $key -Name 'DumpCount'  -Value 5 -PropertyType DWord -Force | Out-Null
+          Get-ItemProperty -Path $key | Format-List
+      - name: Run tests
+        run: |
+          mvn -e --no-transfer-progress test `
+            "-Dnet.ladenthin.llama.tool.model=models/$env:TOOL_MODEL_NAME" `
+            "-Dnet.ladenthin.llama.vision.model=models/$env:VISION_MODEL_NAME" `
+            "-Dnet.ladenthin.llama.vision.mmproj=models/$env:VISION_MMPROJ_NAME" `
+            "-Dnet.ladenthin.llama.vision.image=$env:VISION_IMAGE_PATH"
+      - name: Memory after tests
+        if: always()
+        run: Get-CimInstance Win32_OperatingSystem | Select-Object FreePhysicalMemory,TotalVisibleMemorySize | Format-List
+        shell: pwsh
+      - if: failure()
+        uses: actions/upload-artifact@v7
+        with:
+          name: windows-output-ninja
+          path: |
+            ${{ github.workspace }}\hs_err_pid*.log
+            ${{ github.workspace }}\*.hprof
+            ${{ github.workspace }}\dumps\*.dmp
+            ${{ github.workspace }}\target\surefire-reports\*.dump
+            ${{ github.workspace }}\target\surefire-reports\*.dumpstream
+            ${{ github.workspace }}\target\surefire-reports\*.txt
+            ${{ github.workspace }}\target\surefire-reports\TEST-*.xml
+            ${{ github.workspace }}/src/main/resources/net/ladenthin/llama/**/*
+          if-no-files-found: warn
+
   # ---------------------------------------------------------------------------
   # Package and publish
   # ---------------------------------------------------------------------------
@@ -1203,13 +1296,16 @@ jobs:
       - crosscompile-android-aarch64
       - crosscompile-android-aarch64-opencl
       - build-windows-x86
+      - build-windows-x86_64-ninja
+      - build-windows-x86-ninja
       - test-cpp-linux-x86_64
       - build-macos-arm64-metal-15
       - test-java-linux-x86_64
       - test-java-macos-arm64-metal
       - test-java-macos-arm64-no-metal
       - test-java-macos-arm64-metal-15
       - test-java-windows-x86_64
+      - test-java-windows-x86_64-ninja
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v7
@@ -1226,6 +1322,17 @@ jobs:
         with:
           name: android-libraries-opencl
           path: ${{ github.workspace }}/src/main/resources_android_opencl/net/ladenthin/llama/
+      # Ninja-built Windows natives -> separate tree consumed by the `windows-ninja`
+      # Maven profile (the `ninja-windows` classifier JAR). The default JAR keeps the
+      # MSVC `*-libraries` natives downloaded above.
+      - uses: actions/download-artifact@v8
+        with:
+          name: Windows-x86_64-ninja
+          path: ${{ github.workspace }}/src/main/resources_windows_ninja/net/ladenthin/llama/
+      - uses: actions/download-artifact@v8
+        with:
+          name: Windows-x86-ninja
+          path: ${{ github.workspace }}/src/main/resources_windows_ninja/net/ladenthin/llama/
       - uses: actions/setup-java@v5
         with:
           distribution: 'temurin'
@@ -1236,7 +1343,8 @@ jobs:
         # default-platform native libs in one drop-on-classpath JAR, runnable via its
         # OpenAiCompatServer Main-Class). It lands in target/ and is uploaded in the `llama-jars`
         # artifact below - a CI run artifact only, not a Maven Central / GitHub-Release asset.
-        run: mvn --batch-mode --no-transfer-progress -P release,cuda,opencl-android,assembly -Dmaven.test.skip=true -Dgpg.skip=true package
+        # `windows-ninja` attaches the `ninja-windows` classifier JAR (Ninja-built Windows natives).
+        run: mvn --batch-mode --no-transfer-progress -P release,cuda,opencl-android,windows-ninja,assembly -Dmaven.test.skip=true -Dgpg.skip=true package
       - name: Upload JARs
         uses: actions/upload-artifact@v7
         with:
@@ -1314,6 +1422,14 @@ jobs:
         with:
           name: android-libraries-opencl
           path: ${{ github.workspace }}/src/main/resources_android_opencl/net/ladenthin/llama/
+      - uses: actions/download-artifact@v8
+        with:
+          name: Windows-x86_64-ninja
+          path: ${{ github.workspace }}/src/main/resources_windows_ninja/net/ladenthin/llama/
+      - uses: actions/download-artifact@v8
+        with:
+          name: Windows-x86-ninja
+          path: ${{ github.workspace }}/src/main/resources_windows_ninja/net/ladenthin/llama/
       - name: Set up Maven Central Repository
         uses: actions/setup-java@v5
         with:
@@ -1334,7 +1450,7 @@ jobs:
             *) echo "::error::Refusing to publish non-SNAPSHOT version '$VERSION' from the snapshot job. Snapshot publishing requires a -SNAPSHOT version; releases go through the v* tag path."; exit 1 ;;
           esac
       - name: Publish snapshot
-        run: mvn --batch-mode --no-transfer-progress -P release,cuda,opencl-android -Dmaven.test.skip=true deploy
+        run: mvn --batch-mode --no-transfer-progress -P release,cuda,opencl-android,windows-ninja -Dmaven.test.skip=true deploy
         env:
           MAVEN_USERNAME: ${{ secrets.CENTRAL_USERNAME }}
           MAVEN_PASSWORD: ${{ secrets.CENTRAL_TOKEN }}
@@ -1398,6 +1514,14 @@ jobs:
         with:
           name: android-libraries-opencl
           path: ${{ github.workspace }}/src/main/resources_android_opencl/net/ladenthin/llama/
+      - uses: actions/download-artifact@v8
+        with:
+          name: Windows-x86_64-ninja
+          path: ${{ github.workspace }}/src/main/resources_windows_ninja/net/ladenthin/llama/
+      - uses: actions/download-artifact@v8
+        with:
+          name: Windows-x86-ninja
+          path: ${{ github.workspace }}/src/main/resources_windows_ninja/net/ladenthin/llama/
       - name: Set up Maven Central Repository
         uses: actions/setup-java@v5
         with:
@@ -1409,7 +1533,7 @@ jobs:
           gpg-private-key: ${{ secrets.GPG_PRIVATE_KEY }}
           gpg-passphrase: MAVEN_GPG_PASSPHRASE
       - name: Publish release
-        run: mvn --batch-mode --no-transfer-progress -P release,cuda,opencl-android -Dmaven.test.skip=true deploy
+        run: mvn --batch-mode --no-transfer-progress -P release,cuda,opencl-android,windows-ninja -Dmaven.test.skip=true deploy
         env:
           MAVEN_USERNAME: ${{ secrets.CENTRAL_USERNAME }}
           MAVEN_PASSWORD: ${{ secrets.CENTRAL_TOKEN }}

.gitignore

@@ -40,6 +40,7 @@ replay_pid*
 models/*.gguf
 src/main/cpp/net_ladenthin_llama_*.h
 src/main/resources_cuda_linux/
+src/main/resources_windows_ninja/
 src/main/resources/**/*.so
 src/main/resources/**/*.dylib
 src/main/resources/**/*.dll

CLAUDE.md

@@ -160,6 +160,51 @@ At runtime the device must provide its own OpenCL ICD (`libOpenCL.so`);
 Qualcomm Adreno drivers do. Devices without an ICD should use the default
 CPU-only Android JAR.
 
+## Windows Ninja artifact (sccache-cached, parallel to the MSVC build)
+
+The Visual Studio generator ignores `CMAKE_{C,CXX}_COMPILER_LAUNCHER`, so the two MSVC Windows
+jobs (`build-windows-x86_64`, `build-windows-x86`) **cannot** use the sccache/Depot cache. Rather
+than switch the trusted MSVC build, the repo builds the **same CPU natives a second time** with the
+**`Ninja Multi-Config`** generator (which *does* honor the launcher) and ships them as a separate
+**`ninja-windows`** Maven classifier JAR. **The MSVC build is the default JAR and is kept
+permanently** — the Ninja artifact is an additional, cache-accelerated, independently
+end-to-end-tested option, not a replacement. (Upstream llama.cpp ships its `windows-cuda` artifact
+with Ninja Multi-Config + MSVC, proving the combination works on the same tree.)
+
+Unlike the CUDA / OpenCL classifiers — which differ by a **GGML backend flag** and route their
+output in `CMakeLists.txt` — the Ninja Windows build differs only by **generator/toolchain**, so
+there is **no `CMakeLists.txt` change**: both generators emit to the canonical
+`src/main/resources/.../Windows/{x86_64,x86}/`. Routing to the classifier tree happens purely at the
+CI-download + pom-profile level. Four places wire it together:
+
+1. **`.github/build.bat`** — sccache probe guard mirroring `build.sh`'s `sccache_can_wrap_compiler()`:
+   when `USE_CACHE=true` and `sccache` is on PATH, it compiles a trivial TU through `sccache cl.exe`;
+   only on success does it pass `-DCMAKE_{C,CXX}_COMPILER_LAUNCHER=sccache` and print
+   `sccache --show-stats`. A missing/crashing sccache falls back to a green uncached build. The MSVC
+   jobs do not set `USE_CACHE`, so the guard is inert for them.
+2. **`.github/workflows/publish.yml`** — build jobs `build-windows-x86_64-ninja` /
+   `build-windows-x86-ninja` (`windows-2025-vs2026`, `ilammy/msvc-dev-cmd@v1` for the arch env,
+   sccache v0.16.0 from the GitHub release **zip** + Depot WebDAV, `build.bat -G "Ninja Multi-Config"`),
+   uploading artifacts `Windows-{x86_64,x86}-ninja` (**not** `*-libraries`, so the `package` job's
+   `pattern: "*-libraries"` ignores them). `test-java-windows-x86_64-ninja` loads the Ninja DLL via
+   JNI and runs the full model-backed suite. The `package`, `publish-snapshot`, and `publish-release`
+   jobs download `Windows-*-ninja` into `src/main/resources_windows_ninja/` and activate the
+   `windows-ninja` Maven profile.
+3. **`pom.xml`** — the `windows-ninja` profile produces a second JAR with `<classifier>ninja-windows</classifier>`
+   from the `${project.build.outputDirectory}_windows_ninja` tree (separate compile pass + resource
+   copy + classified jar; mirrors the `cuda` / `opencl-android` profiles). Activated only in CI.
+4. **`README.md`** — the `ninja-windows` row + dependency snippet in "Choosing the right classifier".
+
+`src/main/resources_windows_ninja/` is git-ignored (staged by CI, never committed — same policy as
+the native libs and the CUDA/OpenCL trees).
+
+**Local sanity build** (needs MSVC + a Ninja on PATH; sccache optional):
+```bat
+mvn -q compile
+.github\build.bat -G "Ninja Multi-Config" -DOS_NAME=Windows -DOS_ARCH=x86_64 -DBUILD_TESTING=ON
+ctest --test-dir build --output-on-failure
+```
+
 ## WebUI (llama.cpp Svelte UI) embedding
 
 The llama.cpp WebUI is **built once in CI and shared to every native build**, then
@@ -271,9 +316,11 @@ Per-job recipe: add `env:` { `USE_CACHE`, `SCCACHE_WEBDAV_ENDPOINT`, `SCCACHE_WE
 `DOCKCROSS_ARGS: "-e SCCACHE_WEBDAV_ENDPOINT -e SCCACHE_WEBDAV_TOKEN -e USE_CACHE"` — the
 dockcross wrapper only forwards host env it is explicitly told to via `-e`. The fetched sccache
 version is the `SCCACHE_DL_VERSION` knob in `build.sh` (default **0.16.0**; overridable per-job
-to try a different build against a container that crashed another). **Windows** (`build.bat` +
-MSVC) is separate and last: use `mozilla-actions/sccache-action` / sccache's MSVC support, not
-the `build.sh` musl fetch.
+to try a different build against a container that crashed another). **Windows** is handled
+separately (the Visual Studio generator ignores `CMAKE_*_COMPILER_LAUNCHER`): see
+"Windows Ninja artifact" below — the cached path uses the **Ninja Multi-Config** generator with a
+`build.bat` sccache probe and a direct sccache zip download (not `mozilla-actions/sccache-action`),
+shipped as a parallel `ninja-windows` classifier JAR while the MSVC default stays the trusted build.
 
 **Cross-repo scope.** This Depot/sccache compiler cache makes sense only for java-llama.cpp —
 it is the only sibling repo with a native (C++/JNI) build. It does not apply to the pure-Maven

README.md

@@ -157,14 +157,16 @@ If any of these match your platform, you can include the Maven dependency and ge
 ### Choosing the right classifier
 
 The Maven coordinate `net.ladenthin:llama` publishes one default JAR (CPU-only)
-plus two optional GPU/accelerator JARs selected via a Maven `<classifier>`.
-Pick at most one — they are mutually exclusive.
+plus optional JARs selected via a Maven `<classifier>`: two GPU/accelerator
+builds and one alternate-toolchain Windows build. Pick at most one GPU/accelerator
+classifier — those are mutually exclusive — and optionally the Windows build.
 
 | Classifier | Backend | Target platform | Runtime requirement |
 |---|---|---|---|
-| _(none)_ | CPU | Linux x86-64 / aarch64, macOS x86-64 / aarch64, Windows x86-64, Android aarch64 (CPU) | None beyond a JDK 8+ JVM |
+| _(none)_ | CPU | Linux x86-64 / aarch64, macOS x86-64 / aarch64, Windows x86-64 (MSVC / Visual Studio generator), Android aarch64 (CPU) | None beyond a JDK 8+ JVM |
 | `cuda13-linux-x86-64` | CUDA 13 | Linux x86-64 with NVIDIA GPU | NVIDIA driver + CUDA 13 runtime libraries (`libcudart.so.13`, `libcublas.so.13`) installed on the host. The shared library is dynamically linked against them and will fail to `dlopen` if they are absent — there is no automatic fallback to CPU. |
 | `opencl-android-aarch64` | OpenCL (Adreno) | Android aarch64 with Qualcomm Adreno GPU | A device-supplied OpenCL ICD (`libOpenCL.so`). Devices without an ICD (e.g. most non-Snapdragon Android hardware) must use the default CPU JAR. |
+| `ninja-windows` | CPU (Ninja Multi-Config + MSVC) | Windows x86-64 and x86 | None beyond a JDK 8+ JVM. Same CPU backend as the default JAR's Windows natives, but compiled with the `Ninja Multi-Config` generator (sccache-cached in CI) instead of the Visual Studio generator. Provided so both Windows builds are available; functionally equivalent for normal use. |
 
 ```xml
 <!-- CPU (default) -->
@@ -189,6 +191,14 @@ Pick at most one — they are mutually exclusive.
     <version>5.0.2</version>
     <classifier>opencl-android-aarch64</classifier>
 </dependency>
+
+<!-- Windows natives built with the Ninja Multi-Config generator (CPU) -->
+<dependency>
+    <groupId>net.ladenthin</groupId>
+    <artifactId>llama</artifactId>
+    <version>5.0.2</version>
+    <classifier>ninja-windows</classifier>
+</dependency>
 ```
 
 > [!IMPORTANT]