You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Refactor documentation for clarity and consistency: update phrasing, section titles, and formatting in setup and build guides for CPU and GPU versions of llama.cpp
@@ -99,7 +99,7 @@ The following table gives you more information about the key specifications of t
99
99
| NUMA Topology | Single NUMA node (node0: 0–19) | Simplifies memory access pattern for unified memory workloads |
100
100
| Security and Reliability | Not affected by Meltdown, Spectre, Retbleed, or similar vulnerabilities | Ensures stable and secure operation for long-running inference tasks |
101
101
102
-
Its SVE2, BF16, and INT8 matrix multiplication (I8MM) capabilities are what make it ideal for quantized LLM workloads, as these provide a power-efficient foundation for both CPU-only inference and CPU–GPU hybrid processing.
102
+
Its SVE2, BF16, and INT8 matrix multiplication (I8MM) capabilities are what make it ideal for quantized LLM workloads, as these provide a power-efficient foundation for both CPU-only inference and CPU-GPU hybrid processing.
103
103
104
104
You can also verify the operating system running on your DGX Spark by using the following command:
105
105
@@ -120,7 +120,7 @@ This shows you that your DGX Spark runs on Ubuntu 24.04 LTS, a developer-friendl
120
120
121
121
Nice work! You've confirmed your operating system is Ubuntu 24.04 LTS, so you can move on to the next step.
122
122
123
-
## Step 2: Verify the Blackwell GPU and driver
123
+
## Step 2: verify the Blackwell GPU and driver
124
124
125
125
After confirming your CPU configuration, verify that the Blackwell GPU inside the GB10 Grace Blackwell Superchip is available and ready for CUDA workloads by using the following:
126
126
@@ -161,7 +161,7 @@ The `nvidia-smi` tool reports GPU hardware specifications and provides valuable
161
161
The table below provides more explanation of the `nvidia-smi` output:
162
162
|**Category**|**Specification (from nvidia-smi)**|**Description / impact for LLM inference**|
@@ -174,7 +174,7 @@ The table below provides more explanation of the `nvidia-smi` output:
174
174
Excellent! Your Blackwell GPU is recognized and ready for CUDA workloads. This means your system is set up for GPU-accelerated LLM inference.
175
175
176
176
177
-
## Step 3: Check the CUDA toolkit
177
+
## Step 3: check the CUDA toolkit
178
178
179
179
To build the CUDA version of llama.cpp, the system must have a CUDA toolkit installed.
180
180
@@ -210,7 +210,7 @@ At this point, you have verified that:
210
210
211
211
Your DGX Spark environment is now fully prepared for the next section, where you will build and configure both CPU and GPU versions of llama.cpp, laying the foundation for running quantized LLMs efficiently on the Grace Blackwell platform.
212
212
213
-
## What have I achieved?
213
+
## What you have accomplished
214
214
215
215
In this entire setup section, you have achieved the following:
Copy file name to clipboardExpand all lines: content/learning-paths/laptops-and-desktops/dgx_spark_llamacpp/3_gb10_llamacpp_cpu.md
+5-6Lines changed: 5 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,7 +9,7 @@ In this section, you'll build and test the CPU-only version of llama.cpp, optimi
9
9
10
10
The Grace CPU features Arm Cortex-X925 and Cortex-A725 cores with advanced vector extensions including SVE2, BFloat16, and I8MM. These extensions make the CPU highly efficient for quantized inference workloads, even without GPU acceleration.
11
11
12
-
###Configure and build the CPU-only version
12
+
## Configure and build the CPU-only version
13
13
14
14
15
15
In this session, you will configure and build the CPU-only version of llama.cpp, optimized for the Armv9-based Grace CPU.
@@ -83,8 +83,7 @@ The build output is shown below:
83
83
```
84
84
85
85
After the build finishes, you'll find the CPU-optimized binaries at `~/llama.cpp/build-cpu/bin/`
86
-
87
-
### Step 2: Validate the CPU-Enabled Build (CPU Mode)
86
+
## Validate the CPU-enabled build (CPU mode)
88
87
89
88
First, validate that the binary was compiled in CPU-only mode and runs correctly on the Grace CPU:
90
89
@@ -116,7 +115,7 @@ Here is an explanation of the key flags:
If the build is successful, you will observe smooth model initialization and token generation, with CPU utilization increasing across all cores.
118
+
If the build is successful, you will see smooth model initialization and token generation, with CPU utilization increasing across all cores.
120
119
121
120
To monitor live CPU utilization and power metrics during inference, use `htop`:
122
121
@@ -125,7 +124,7 @@ htop
125
124
```
126
125
127
126
The following screenshot shows CPU utilization and thread activity during TinyLlama inference on DGX Spark, confirming full multi-core engagement.
128
-

127
+
](htop.png"TinyLlama CPU Utilization")
129
128
130
129
The `htop` interface shows:
131
130
@@ -138,7 +137,7 @@ The `htop` interface shows:
138
137
In htop, press F6 to sort by CPU% and verify load distribution, or press `t` to toggle the tree view, which shows the `llama-cli` main process and its worker threads.
139
138
{{% /notice %}}
140
139
141
-
## What have I accomplished?
140
+
## What you have accomplished
142
141
143
142
In this section you have:
144
143
- Built and validated the CPU-only version of llama.cpp.
0 commit comments