Skip to content

Commit 37296f6

Browse files
skill-based updates
1 parent f9e855f commit 37296f6

2 files changed

Lines changed: 25 additions & 22 deletions

File tree

.github/skills/seo-geo-aeo-review/SKILL.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ On any given page:
5050
## Review rules
5151

5252
- Optimize for selection and usefulness, not ranking alone.
53-
- For Learning Paths, prefer verb-led titles such as `Install`, `Deploy`, `Configure`, `Analyze`, `Optimize`, or `Verify`. Install guides are named after the tool being installed and don't feature a verb because install is implied.
53+
- For Learning Paths, prefer verb-led titles such as `Install`, `Deploy`, `Configure`, `Analyze`, `Optimize`, or `Verify`. Install guides are named after the tool being installed and don't feature a verb because install is implied. Install guides don't have an h2 title.
5454
- Preserve content-type boundaries: install guides cover installation and verification; Learning Paths cover applied end-to-end tasks.
5555
- Use Arm-specific terminology naturally when it is supported by the content.
5656
- Don't add speculative keywords, unsupported performance claims, or marketing language.

content/install-guides/litespark-inference.md

Lines changed: 24 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
title: Litespark-Inference
33

4-
description: Install Litespark-Inference on Arm or x86 Linux and Apple silicon macOS to run BitNet ternary LLMs on the CPU.
4+
description: Install Litespark-Inference on Arm Linux and Apple silicon macOS to run BitNet ternary LLMs on the CPU.
55

66
minutes_to_complete: 10
77

@@ -38,10 +38,10 @@ layout: installtoolsall
3838

3939
[Litespark-Inference](https://github.com/Mindbeam-AI/Litespark-Inference)
4040
is an open-source CPU inference runtime for
41-
[BitNet b1.58](https://arxiv.org/abs/2402.17764) ternary-weight LLMs. A
42-
single `pip install` reads your CPU's feature flags and compiles the
43-
right C++ kernel for it using NEON and SDOT on Arm or AVX-512/VNNI and AVX2+FMA
44-
on x86. This saves you from needing to pick the right C++ kernel for best performance.
41+
[BitNet b1.58](https://arxiv.org/abs/2402.17764) ternary-weight LLMs.
42+
43+
A single `pip install` reads your CPU's feature flags and compiles the
44+
right C++ kernel for it using Neon and SDOT on Arm. This saves you from needing to manually pick the right C++ kernel for best performance.
4545

4646
You can install Litespark-Inference on Linux and macOS.
4747

@@ -55,10 +55,10 @@ Before installing Litespark-Inference, make sure your local machine has the foll
5555

5656
To check the Python version on your machine, run `python3 --version`.
5757

58-
The BitNet-2B model is downloaded from Hugging Face on your first run.
58+
The first run downloads BitNet-2B model from Hugging Face.
5959

60-
It's a good practice to install into a clean virtual environment so the
61-
install doesn't conflict with anything else on your machine:
60+
Install Litespark-Inference into a clean virtual environment so the
61+
install doesn't conflict with anything else on your machine.
6262

6363
If you don't have Python virtual environment support installed, run:
6464

@@ -78,13 +78,12 @@ python -m pip install --upgrade pip wheel setuptools
7878
## Install Litespark-Inference on Arm Linux
7979

8080
The released package builds the correct kernel for your CPU
81-
automatically. It uses Neon and
82-
SDOT on Arm.
81+
automatically. It uses Neon and SDOT on Arm.
8382

8483
Supported CPUs include:
8584

8685
- AWS Graviton 2, 3, or 4
87-
- Ampere
86+
- Ampere family processors
8887
- Neoverse N1, N2, V1, or V2
8988
- Raspberry Pi 5
9089

@@ -103,7 +102,7 @@ sudo dnf install -y gcc-c++ clang ninja-build git python3-pip
103102
pip install litespark-inference
104103
```
105104

106-
Confirm the package imports and reports the kernel selected for your CPU:
105+
Confirm the package imports and list the installed version of Litespark-Inference:
107106

108107
```bash
109108
python3 -c "import litespark_inference; print(litespark_inference.__version__)"
@@ -115,7 +114,7 @@ The output is similar to:
115114
1.0.3
116115
```
117116

118-
To inspect which kernel was built, run:
117+
Inspect which kernel was built for your CPU:
119118

120119
```bash
121120
python -m litespark_inference.torchless info
@@ -139,15 +138,15 @@ Apple's CPUs have Neon SDOT, and Litespark-Inference uses it directly.
139138
Litespark uses OpenMP for multi-threading inside the kernel. However, Apple's
140139
toolchain doesn't ship a built-in OpenMP runtime. To address this, you'll need to install `libomp`.
141140

142-
Install Xcode command-line tools, `libomp`, and the Litespark-Inference Python package. The full set of Xcode tools isn't required:
141+
Install Xcode command-line tools, `libomp`, and the Litespark-Inference Python package:
143142

144143
```console
145144
xcode-select --install
146145
brew install libomp
147146
pip install litespark-inference
148147
```
149148

150-
Verify the install:
149+
Verify that the installation was successful:
151150

152151
```console
153152
python -m litespark_inference.torchless info
@@ -164,15 +163,19 @@ litespark_inference.torchless
164163
Accelerate: True
165164
```
166165

166+
### Troubleshoot missing OpenMP support
167+
167168
If you see `OpenMP : False`, the build didn't find Homebrew's `libomp`. The
168-
most common cause is that Homebrew is installed under `/opt/homebrew`
169-
(the Apple silicon default), but `pip install` ran in an environment that
170-
hides it. Re-run `pip install litespark-inference` from a normal
171-
shell to fix it.
169+
most common cause is that Homebrew is installed under `/opt/homebrew`, which is the
170+
Apple silicon default, but `pip install` ran in an environment that
171+
hides it.
172+
173+
To fix this, re-run `pip install litespark-inference` from a normal
174+
shell.
172175

173-
## Install Litespark-Inference from source
176+
## (Optional) Install Litespark-Inference from source
174177

175-
To modify the runtime or kernels, install from source instead of from
178+
To modify the runtime or kernels, install from source rather than from
176179
PyPI:
177180

178181
```console

0 commit comments

Comments
 (0)