Skip to content

Commit 734b938

Browse files
committed
Move Quick Start (Getting Started) setup instructions to top of README
1 parent 8eec8e1 commit 734b938

1 file changed

Lines changed: 40 additions & 38 deletions

File tree

README.md

Lines changed: 40 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,46 @@ No Python runtime, no Global Interpreter Lock (GIL), no unnecessary memory copie
1010

1111
---
1212

13+
## 🏁 Getting Started
14+
15+
### Fastest: Download Pre-built Binary
16+
17+
Download the latest release tarball from the [Releases page](https://github.com/SharpAI/SwiftLM/releases).
18+
The archive is **self-contained**`mlx.metallib` is bundled alongside the binary.
19+
20+
```bash
21+
tar -xzf SwiftLM-<version>-macos-arm64.tar.gz
22+
./SwiftLM --model mlx-community/Qwen2.5-3B-Instruct-4bit --port 5413
23+
```
24+
25+
> **⚠️ Metal GPU Error?** If you see `Failed to load the default metallib`, make sure `mlx.metallib` is co-located with the `SwiftLM` binary.
26+
27+
### Build from Source
28+
29+
The build script handles everything: submodules, cmake, Metal kernel compilation, and the Swift build.
30+
31+
```bash
32+
git clone --recursive https://github.com/SharpAI/SwiftLM
33+
cd SwiftLM
34+
./build.sh
35+
```
36+
37+
This will:
38+
1. Initialize git submodules
39+
2. Install `cmake` via Homebrew (if not already installed)
40+
3. Compile `mlx.metallib` from the Metal kernel sources
41+
4. Build the `SwiftLM` binary in release mode
42+
43+
Then start the server (models download automatically if not cached):
44+
```bash
45+
.build/release/SwiftLM \
46+
--model mlx-community/Qwen3.5-122B-A10B-4bit \
47+
--stream-experts \
48+
--port 5413
49+
```
50+
51+
*(Add `--stream-experts` when running oversized MoE models like Qwen3.5 122B to bypass macOS virtual memory swapping and stream expert layers directly from NVMe.)*
52+
1353
## 📊 Performance: Gemma 4-26B on Apple Silicon
1454

1555
Benchmark results for `gemma-4-26b-a4b-it-4bit` (26B MoE, 4-bit) on M5 Pro 64 GB.
@@ -148,45 +188,7 @@ Then in Xcode:
148188
149189
---
150190

151-
## 🛠️ Quick Start (macOS Server)
152-
153-
### Fastest: Download Pre-built Binary
154-
155-
Download the latest release tarball from the [Releases page](https://github.com/SharpAI/SwiftLM/releases).
156-
The archive is **self-contained**`mlx.metallib` is bundled alongside the binary.
157191

158-
```bash
159-
tar -xzf SwiftLM-<version>-macos-arm64.tar.gz
160-
./SwiftLM --model mlx-community/Qwen2.5-3B-Instruct-4bit --port 5413
161-
```
162-
163-
> **⚠️ Metal GPU Error?** If you see `Failed to load the default metallib`, make sure `mlx.metallib` is co-located with the `SwiftLM` binary.
164-
165-
### Build from Source
166-
167-
The build script handles everything: submodules, cmake, Metal kernel compilation, and the Swift build.
168-
169-
```bash
170-
git clone --recursive https://github.com/SharpAI/SwiftLM
171-
cd SwiftLM
172-
./build.sh
173-
```
174-
175-
This will:
176-
1. Initialize git submodules
177-
2. Install `cmake` via Homebrew (if not already installed)
178-
3. Compile `mlx.metallib` from the Metal kernel sources
179-
4. Build the `SwiftLM` binary in release mode
180-
181-
Then run:
182-
```bash
183-
.build/release/SwiftLM \
184-
--model mlx-community/Qwen3.5-122B-A10B-4bit \
185-
--stream-experts \
186-
--port 5413
187-
```
188-
189-
*(Add `--stream-experts` when running oversized MoE models like Qwen3.5 122B to bypass macOS virtual memory swapping and stream expert layers directly from NVMe.)*
190192

191193
---
192194

0 commit comments

Comments
 (0)