bug: version >=v3.15.0 not support CUDA v13.1 & v13.2

[xinxi053]

Issue description

windows11, node-llama-cpp: >= 3.15.0 with error: CUDA is detected, but using it failed

Expected Behavior

expect works right with node-llama-cpp: >= 3.15.0

Actual Behavior

PS C:\Users\Administrator> npx --yes node-llama-cpp@3.15.0 inspect gpu
OS: Windows 10.0.26200 (x64)
Node: 24.13.0 (x64)

node-llama-cpp: 3.15.0
Prebuilt binaries: b7698

CUDA: CUDA is detected, but using it failed
To resolve errors related to CUDA, see the CUDA guide: https://node-llama-cpp.withcat.ai/guide/CUDA
Vulkan: Vulkan is detected, but using it failed
To resolve errors related to Vulkan, see the Vulkan guide: https://node-llama-cpp.withcat.ai/guide/vulkan

CPU model: 13th Gen Intel(R) Core(TM) i9-13900K
Math cores: 1.16985510423e-311
Used RAM: 39.94% (12.68GB/31.75GB)
Free RAM: 60.05% (19.07GB/31.75GB)
Used swap: 42.5% (17.1GB/40.25GB)
Max swap size: 40.25GB
mmap: supported

Steps to reproduce

npx --yes node-llama-cpp@3.15.0 inspect gpu not work
npx --yes node-llama-cpp@3.14.5 inspect gpu <= 3.14.5 all works right

My Environment

PS C:\Users\Administrator> npx --yes node-llama-cpp@3.14.4 inspect gpu
npm warn deprecated npmlog@6.0.2: This package is no longer supported.
npm warn deprecated are-we-there-yet@3.0.1: This package is no longer supported.
npm warn deprecated gauge@4.0.4: This package is no longer supported.
OS: Windows 10.0.26200 (x64)
Node: 24.13.0 (x64)

node-llama-cpp: 3.14.4
Prebuilt binaries: b7324

CUDA: available
Vulkan: available

CUDA device: NVIDIA GeForce RTX 4090
CUDA used VRAM: 6.21% (1.49GB/23.99GB)
CUDA free VRAM: 93.78% (22.5GB/23.99GB)

Vulkan device: NVIDIA GeForce RTX 4090
Vulkan used VRAM: 4.56% (1.08GB/23.57GB)
Vulkan free VRAM: 95.43% (22.5GB/23.57GB)

CPU model: 13th Gen Intel(R) Core(TM) i9-13900K
Math cores: 8.728467834306e-312
Used RAM: 42.02% (13.34GB/31.75GB)
Free RAM: 57.97% (18.41GB/31.75GB)
Used swap: 44.5% (17.91GB/40.25GB)
Max swap size: 40.25GB
mmap: supported

PS C:\Users\Administrator> npx --yes node-llama-cpp@3.15.0 inspect gpu
OS: Windows 10.0.26200 (x64)
Node: 24.13.0 (x64)

node-llama-cpp: 3.15.0
Prebuilt binaries: b7698

CUDA: CUDA is detected, but using it failed
To resolve errors related to CUDA, see the CUDA guide: https://node-llama-cpp.withcat.ai/guide/CUDA
Vulkan: Vulkan is detected, but using it failed
To resolve errors related to Vulkan, see the Vulkan guide: https://node-llama-cpp.withcat.ai/guide/vulkan

CPU model: 13th Gen Intel(R) Core(TM) i9-13900K
Math cores: 1.16985510423e-311
Used RAM: 39.94% (12.68GB/31.75GB)
Free RAM: 60.05% (19.07GB/31.75GB)
Used swap: 42.5% (17.1GB/40.25GB)
Max swap size: 40.25GB
mmap: supported

PS C:\Users\Administrator> nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2026 NVIDIA Corporation
Built on Mon_Mar__2_21:54:11_Pacific_Standard_Time_2026
Cuda compilation tools, release 13.2, V13.2.51
Build cuda_13.2.r13.2/compiler.37434383_0

PS C:\Users\Administrator> nvidia-smi
Tue Mar 10 15:24:36 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 595.71 Driver Version: 595.71 CUDA Version: 13.2 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Driver-Model | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 4090 WDDM | 00000000:01:00.0 On | Off |
| 0% 41C P8 28W / 450W | 1890MiB / 24564MiB | 1% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+

Additional Context

I saw the release list:
v3.15.0
support new CUDA 13.1 archs (#538) (734693d)
build the prebuilt binaries with CUDA 13.1 instead of 13.0 (#538) (734693d)
but actually, not work with me.

Relevant Features Used

• Metal support
• CUDA support
• Vulkan support
• Grammar
• Function calling

Are you willing to resolve this issue by submitting a Pull Request?

Yes, I have the time, and I know how to start.

[giladgd]

@xinxi053 Do you happen to have multiple CUDA versions installed?
Also, have you updated the Nvidia driver to the latest version?
Can you please try running npx --yes node-llama-cpp@latest and see whether it's able to detect the GPU?
There have been a few changes in llama.cpp since 3.15.0 was shipped so the latest version might help with these.

Also, if you build from source using the command below, does it compile successfully and the GPU is detected afterwards?

npx --no node-llama-cpp source download

I'm trying to figure out whether this is an issue that stems from the prebuilt binaries build of node-llama-cpp or something in llama.cpp itself.
Since it seems that Vulkan also isn't detected, I suspect the issue might have something to do in llama.cpp, since I haven't changed anything related to Vulkan in 3.15.0.
In 3.15.0 I had to update the CUDA version used in the CI due to changes in llama.cpp that use new features present only in CUDA 13.1, but that shouldn't have affected the Vulkan backend.

[xinxi053]

@giladgd
1.Only CUDA 13.2 is installed in the current system.
2.Has the Nvidia driver been updated to the latest version?
Yes, it has been updated to the latest version.
3.Executing the command: npx --yes node-llama-cpp@latest inspect gpu
OS: Windows 10.0.26200 (x64) Node: 24.13.0 (x64) node-llama-cpp: 3.17.1 Prebuilt binaries: b8179 CUDA: CUDA is detected, but using it failed To resolve errors related to CUDA, see the CUDA guide: https://node-llama-cpp.withcat.ai/guide/CUDA Vulkan: Vulkan is detected, but using it failed To resolve errors related to Vulkan, see the Vulkan guide: https://node-llama-cpp.withcat.ai/guide/vulkan CPU model: 13th Gen Intel(R) Core(TM) i9-13900K Math cores: 0 Used RAM: 56.33% (17.88GB/31.75GB) Free RAM: 43.66% (13.86GB/31.75GB) Used swap: 60.38% (24.3GB/40.25GB) Max swap size: 40.25GB mmap: supported
This command results in "CUDA&Vulkan failed" for node-llama-cpp@3.15.0 and all subsequent versions. For node-llama-cpp@3.14.5 and earlier versions, everything works normally, and both CUDA and Vulkan can be detected.
4.Executing the command: npx --no node-llama-cpp source download --gpu vulkan
The source code compiles successfully. Now I can support version 3.16.2.
PS C:\Users\Administrator> npx --yes node-llama-cpp inspect gpu OS: Windows 10.0.26200 (x64) Node: 24.13.0 (x64) node-llama-cpp: 3.16.2 Prebuilt binaries: b8121 Cloned source: b8121 CUDA: available Vulkan: available CUDA device: NVIDIA GeForce RTX 4090 CUDA used VRAM: 6.21% (1.49GB/23.99GB) CUDA free VRAM: 93.78% (22.5GB/23.99GB) Vulkan device: NVIDIA GeForce RTX 4090 Vulkan used VRAM: 4.56% (1.08GB/23.57GB) Vulkan free VRAM: 95.43% (22.5GB/23.57GB) CPU model: 13th Gen Intel(R) Core(TM) i9-13900K Math cores: 24 Used RAM: 50.9% (16.16GB/31.75GB) Free RAM: 49.09% (15.59GB/31.75GB) Used swap: 71.63% (28.83GB/40.25GB) Max swap size: 40.25GB mmap: supported
Final Issue:
The problem lies with the prebuilt binary packages of node-llama-cpp. Verification is as follows:
When executing the command: npm install -g node-llama-cpp@3.14.5 (installing the prebuilt binary package of 3.14.5), the directory
C:\Users\Administrator\AppData\Roaming\npm\node_modules\node-llama-cpp\node_modules@node-llama-cpp
contains the following folders: win-arm64, win-x64, win-x64-cuda, win-x64-cuda-ext, win-x64-vulkan.
When executing the command: npm install -g node-llama-cpp@3.15.0, the directory only contains two folders: win-arm64 and win-x64.
In other words, starting from version 3.15.0, the prebuilt binary packages no longer include support for CUDA and Vulkan.

[giladgd]

@xinxi053 Do you have a custom npm registry configured in your .npmrc or are you using the official one?
Perhaps the prebuilt binary modules are missing from the npm registry that you use.

You can run this command to see what npm registry you have configured:

npm config get registry

[xinxi053]

PS C:\Users\Administrator&gt; npm config get registry https://registry.npmmirror.com Jon&nbsp;Snow ***@***.***

[giladgd]

@xinxi053 It seems that there's a size limit to the packages that this mirror can pull.
I've opened a PR to fix it on their end.
I'll also add an indication in the inspect gpu command for when prebuilt binary modules are missing, and when that happens also check for whether a custom npm registry is configured and log it as well. This should help detecting these kinds of issues faster.

[giladgd]

@xinxi053 This npm mirror should now have the latest prebuilt binary modules.
Please try removing the node_modules directory and package-lock.json file, reinstall node-llama-cpp and the run the npx --yes node-llama-cpp inspect gpu command again and let me know whether it's now able to detect the GPU without building from source.

[xinxi053]

@giladgd
Everything is working normally now.

PS C:\Users\Administrator> npm install -g node-llama-cpp@3.16.2
added 155 packages in 15s
43 packages are looking for funding
run npm fund for details

C:\Users\Administrator\AppData\Roaming\npm\node_modules\node-llama-cpp\node_modules@node-llama-cpp
Mode LastWriteTime Length Name

---

d----- 2026/3/15/星期日 8:27 win-arm64
d----- 2026/3/15/星期日 8:27 win-x64
d----- 2026/3/15/星期日 8:27 win-x64-cuda
d----- 2026/3/15/星期日 8:27 win-x64-cuda-ext
d----- 2026/3/15/星期日 8:27 win-x64-vulkan

PS C:\Users\Administrator> npx --yes node-llama-cpp inspect gpu
OS: Windows 10.0.26200 (x64)
Node: 24.13.0 (x64)
node-llama-cpp: 3.16.2
Prebuilt binaries: b8121
CUDA: available
Vulkan: available
CUDA device: NVIDIA GeForce RTX 4090
CUDA used VRAM: 6.21% (1.49GB/23.99GB)
CUDA free VRAM: 93.78% (22.5GB/23.99GB)
Vulkan device: NVIDIA GeForce RTX 4090
Vulkan used VRAM: 4.56% (1.08GB/23.57GB)
Vulkan free VRAM: 95.43% (22.5GB/23.57GB)
CPU model: 13th Gen Intel(R) Core(TM) i9-13900K
Math cores: 0
Used RAM: 50.47% (16.02GB/31.75GB)
Free RAM: 49.52% (15.72GB/31.75GB)
Used swap: 53.68% (21.61GB/40.25GB)
Max swap size: 40.25GB
mmap: supported