You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It is also possible to install a pre-built wheel with CUDA support. As long as your system meets some requirements:
130
130
131
131
- CUDA Version is 12.4, 12.6, 12.8 or 13.0
132
-
- Python Version is 3.10, 3.11, 3.12or 3.13
133
-
- Basic version: A version compiled without using AVX instructions (for compatibility with CPU platforms lacking AVX instructions or with AVX instruction compatibility issues).
132
+
- Python Version is 3.10, 3.11, 3.12, 3.13 or 3.14
133
+
- Basic version(Default): A version compiled without using AVX instructions (for compatibility with CPU platforms lacking AVX instructions or with AVX instruction compatibility issues).
134
134
- AVX2 version: A version compiled using AVX2 instructions.
To install with hipBLAS / ROCm support for AMD cards, set the `GGML_HIPBLAS=on` environment variable before installing:
177
+
This provides GPU acceleration on HIP-supported AMD GPUs. Make sure to have ROCm installed.
178
+
179
+
You can download it from your Linux distro's package manager or from here: [ROCm Quick Start (Linux)](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/tutorial/quick-start.html#rocm-install-quick).
180
+
181
+
To install with HIP / ROCm support for AMD cards, set the `GGML_HIP=ON` environment variable before installing:
0 commit comments