Skip to content

Commit 9642299

Browse files
committed
README updated with llamaTornado usage example
1 parent 10caa33 commit 9642299

1 file changed

Lines changed: 10 additions & 4 deletions

File tree

README.md

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -119,11 +119,9 @@ We are at the early stages of Java entering the AI world with features added to
119119
| | M4 Pro | 16.77 tokens/s | 8.56 tokens/s | (WIP) |
120120
| **AMD / OpenCL** | Radeon RX | (WIP) | (WIP) | (WIP) |
121121

122-
##### ⚠️ Note on Apple Silicon Performance
122+
##### Apple Silicon Support
123123

124-
TornadoVM currently runs on Apple Silicon via [OpenCL](https://developer.apple.com/opencl/), which has been officially deprecated since macOS 10.14.
125-
126-
Despite being deprecated, OpenCL can still run on Apple Silicon; albeit, with older drivers which do not support all optimizations of TornadoVM. Therefore, the performance is not optimal since TornadoVM does not have a Metal backend yet (it currently has OpenCL, PTX, and SPIR-V backends). We recommend using Apple silicon for development and for performance testing to use OpenCL/PTX compatible Nvidia GPUs for the time being (until we add a Metal backend to TornadoVM and start optimizing it).
124+
TornadoVM 4.0 includes a native [Metal](https://developer.apple.com/metal/) backend, enabling GPU-accelerated inference on Apple Silicon.
127125

128126
-----------
129127
## 📦 Maven Dependency
@@ -313,6 +311,14 @@ Enable GPU acceleration with Q8_0 quantization:
313311
./llama-tornado --gpu --verbose-init --model beehive-llama-3.2-1b-instruct-fp16.gguf --prompt "tell me a joke"
314312
```
315313

314+
#### Running with `llamaTornado` (Java 25 single-file script)
315+
316+
`llamaTornado` is a zero-dependency Java 25 single-file script that replaces the Python launcher. It requires `java 25+` on your PATH:
317+
318+
```bash
319+
./llamaTornado --gpu --verbose-init --metal --model /Users/abien/work/workspaces/llms/Mistral-7B-Instruct-v0.3.Q8_0.gguf --prompt "what is java"
320+
```
321+
316322
-----------
317323

318324
## 🐳 Docker

0 commit comments

Comments
 (0)