Update performance_llama_cpp_sme2 content

Author: madeline-underwood

content/learning-paths/mobile-graphics-and-gaming/performance_llama_cpp_sme2/build_llama_cpp.md

@@ -6,9 +6,11 @@ weight: 4
 layout: learningpathall
 ---
 
+## Overview
+
 In this section, you set up a GCC cross-compile toolchain and build a statically linked `llama-cli` binary with KleidiAI and SME2 enabled.
 
-For convenience, llama.cpp is statically linked. You use the aarch64 GCC cross compile toolchain, *aarch64-none-linux-gnu-*, to build the project. To support SME2, GCC compiler version 14.2 and onwards is required.
+For convenience, llama.cpp is statically linked. You use the aarch64 GCC cross compile toolchain, `aarch64-none-linux-gnu-`, to build the project. To support SME2, GCC compiler version 14.2 or later is required.
 
 The build uses the Linux-hosted Arm GNU Toolchain. If you are working on macOS or Windows, run these commands in a Linux environment (for example, a Linux VM, container, or a Linux development machine).
 
@@ -36,33 +38,33 @@ tar -xf "${TOOLCHAIN_TAR}"
 export PATH="$PWD/${TOOLCHAIN_TAR%.tar.xz}/bin:$PATH"
 ```
 
-Verify the installation was successful by printing the toolchain version:
+Confirm the installation succeeded by printing the compiler version:
 
 ```bash
 aarch64-none-linux-gnu-gcc --version
 ```
 
 ## Clone the llama.cpp repository
 
-The llama.cpp with tag b7610 is used in this tutorial. Newer versions should also work, but they are not tested.
+This Learning Path uses llama.cpp tag b7610. Newer versions should also work but are not tested.
 
-Next, download the llama.cpp source code and check out the tag used in this tutorial:
+Download the llama.cpp source code and check out that tag:
 
 ```bash
 cd $HOME
 git clone --depth 1 --branch b7610 https://github.com/ggml-org/llama.cpp.git
 ```
 
-Create a new directory *build* under the llama.cpp root directory and change to the new directory:
+Create a new `build` directory under the llama.cpp root directory and change to it:
 
 ```bash
 cd $HOME/llama.cpp
 mkdir build && cd build
 ```
 
-## Compile the binary 
+## Compile llama-cli for Android
 
-Next, configure the project using the following command:
+Configure the project. The key flags enable KleidiAI support (`-DGGML_CPU_KLEIDIAI=ON`) and SME2 instructions (`-march=...+sme2`), produce a statically linked binary (`-static`) that runs across Android and Linux environments, and include debug symbols (`-g`) for profiling:
 
 ```bash
 cmake .. \
@@ -91,16 +93,22 @@ Set `CMAKE_C_COMPILER` and `CMAKE_CXX_COMPILER` to your cross compiler path if i
 
 The `-static` and `-g` options are specified to produce a statically linked executable, in order to run on different Arm64 Linux/Android environments and include debug information.
 
-Next, build the project,
+Build the project:
 
 ```bash
 cd $HOME/llama.cpp/build
 cmake --build ./ --config Release -j $(nproc)
 ```
-After the building process completes, verify that *llama-cli* exists in the binary directory:
+After the build completes, confirm that `llama-cli` exists in the binary directory:
 
 ```bash 
 ls -la $HOME/llama.cpp/build/bin | grep llama-cli
 ```
 
-Now that you have a `llama-cli` build with KleidiAI enabled, move on to the next section to run the model on your SME2 device and observe the performance impact when you enable the microkernels.
+## What you've accomplished and what's next
+
+In this section:
+- You installed the Arm GNU Toolchain (GCC 14.2) and configured it for aarch64 cross-compilation
+- You built a statically linked `llama-cli` binary with KleidiAI and SME2 enabled, ready to run on your Android target
+
+In the next section, you'll transfer the binary and model to the device and compare inference performance with SME2 on and off.

content/learning-paths/mobile-graphics-and-gaming/performance_llama_cpp_sme2/introduction.md

@@ -1,22 +1,28 @@
 ---
-title: Overview
+title: Understand how SME2 and KleidiAI accelerate LLM inference in llama.cpp
 weight: 2
 
 ### FIXED, DO NOT MODIFY
 layout: learningpathall
 ---
+## How SME2 and KleidiAI accelerate LLM inference in llama.cpp
 
-## Introduction 
-In this section, you get a quick mental model of SME2, KleidiAI, and what llama.cpp is doing when it runs LLM inference on an Arm CPU.
-Arm’s latest Client CPU processors such as Arm C1 include Scalable Matrix Extension 2 (SME2). SME2 accelerates the matrix-heavy AI operations behind large language models (LLMs), media processing, speech recognition, computer vision, real-time apps and multimodal apps.
+In this Learning Path, you'll optimize llama.cpp inference on an Arm CPU by enabling SME2 acceleration through Arm KleidiAI microkernels.
 
-llama.cpp provides extensive support for many LLMs, including Phi, Llama, DeepSeek, Gemma and Qwen. It's designed for efficient CPU-based inference, enabling on-device LLM execution that reduces latency and enhances privacy.
+You will measure the performance difference between the default CPU path and the SME2-optimized path using a 3 billion parameter LLM. By the end, you will understand:
 
-By default, llama.cpp integrates with Arm KleidiAI, a suite of optimized microkernels for Arm CPUs. KleidiAI includes SME2-optimized microkernels to deliver additional performance benefits.
+- What SME2 changes in the matrix execution path
+- How KleidiAI integrates into llama.cpp’s CPU backend
+- How to verify that SME2 microkernels are active
+- What measurable improvement you should expect
 
-This Learning Path uses the `Llama-3.2-3B-Instruct-Q4_0.gguf` model with 3 billion parameters.
+Scalable Matrix Extension 2 (SME2) is an Arm architectural feature designed to accelerate matrix-heavy workloads. Large language model (LLM) inference relies heavily on matrix multiplication, especially in transformer layers. When SME2 is available on the CPU, KleidiAI provides optimized microkernels that replace generic implementations inside llama.cpp.
 
-Move on to the next section to see how KleidiAI microkernels integrate into llama.cpp's CPU backend.
+llama.cpp is a CPU-focused LLM inference engine. On Arm systems, it integrates with KleidiAI by default. If SME2 is supported and enabled at runtime, llama.cpp dispatches SME2-optimized matrix kernels for supported operations.
+
+This Learning Path uses the `Llama-3.2-3B-Instruct-Q4_0.gguf` model (3 billion parameters) as a reproducible test case.
+
+In the next section, you'll examine how KleidiAI microkernels integrate into the llama.cpp backend and where SME2 is activated in the execution path.
 
 
 

content/learning-paths/mobile-graphics-and-gaming/performance_llama_cpp_sme2/kleidiai_integration.md

@@ -1,13 +1,24 @@
 ---
-title: Integration of SME2 optimized KleidiAI microkernels in llama.cpp
+title: Trace how KleidiAI and SME2 accelerate llama.cpp from model load to token decode
 weight: 3
 
 ### FIXED, DO NOT MODIFY
 layout: learningpathall
 ---
 
-## Integration of SME2 optimized KleidiAI microkernels in llama.cpp
-In this section, you look at how SME2 connects into the Arm software ecosystem by following the path from llama.cpp, through its ggml CPU backend, down to Arm KleidiAI microkernels that take advantage of SME2 when the hardware supports it.
+## Overview 
+
+In this section, you trace how SME2 acceleration flows through llama.cpp across the full inference lifecycle — from model load to prefill and token decode.
+
+Rather than treating SME2 as a feature flag, you examine where acceleration is selected, how weights and activations are packed, and which microkernels execute at each stage of inference. You follow the path from high-level llama.cpp operations down into the ggml-cpu backend and finally into the KleidiAI microkernels that use SME2, I8MM, or DotProd instructions depending on hardware support.
+
+By the end of this section, you will understand:
+- Where KleidiAI integrates into the llama.cpp CPU backend
+- Which operators are eligible for SME2 acceleration
+- How microkernel selection priority works (SME2 → I8MM → DotProd)
+- What changes between model load, prefill, and token decode
+
+This architectural view prepares you to build, profile, and validate SME2 acceleration in the next sections.
 
 ### Where does KleidiAI fit in the llama.cpp CPU backend? 
 
@@ -21,7 +32,7 @@ The integration source code is located in the following directory of llama.cpp:
  ./ggml/src/ggml-cpu/kleidiai
 ```
 
-### Which matmul operators can use KleidiAI microkernels? 
+### Which matmul operators can use KleidiAI microkernels?
 
 KleidiAI matmul microkernels can be used for some types of `GGML_OP_MUL_MAT` operators. The table below lists some matmul operators with specific input and output data type that can be accelerated by KleidiAI microkernels. 
 
@@ -39,7 +50,7 @@ In this table, LHS is short for left-hand source (left-hand input matrix) and RH
 
 More operators and data types are being supported by KleidiAI microkernels.
 
-### How does the KleidiAI path work?
+### How the KleidiAI microkernel selection and packing path works
 
 The figure below shows how KleidiAI microkernels are used for `matmul` with `GGML_TYPE_Q4_0` or `GGML_TYPE_Q8_0` RHS(weight).
 
@@ -54,11 +65,11 @@ SME2 -> I8MM -> DotProd
 ```
 Once the matmul microkernel is decided, its corresponding RHS packing and LHS quantizing & packing micro-kernel will be used.
 
-## Call stacks
+## Execution call stacks for KleidiAI microkernels
 
 The call stacks below are included as a way to make this integration concrete. Read each stack from top to bottom: the top frames show the high-level llama.cpp operation you triggered (loading the model or decoding tokens), the middle frames show where execution enters the `ggml-cpu` backend, and the bottom frame is the specific KleidiAI microkernel that runs. The first stack shows a one-time weight repack during model load, which happens because the model weights are constant during inference. The later stacks show what happens during inference: activations change from token to token, so the LHS is quantized and packed each time, and then the selected SME2 GEMM or GEMV microkernel executes the matmul.
 
-### During model load
+### RHS weight packing during model load
 
 In case of using the Llama-3.2-3B-Instruct-Q4_0.gguf model and SME2 microkernels, RHS packing is done by the *kai_run_rhs_pack_nxk_qsi4c32ps1s0scalef16_qsu4c32s16s0_neon* microkernel when loading the model. It is shown in following function call stack,
 
@@ -72,7 +83,7 @@ llama_model_load
                         kai_run_rhs_pack_nxk_qsi4c32ps1s0scalef16_qsu4c32s16s0_neon
 ```
 
-### During inference
+### LHS activation quantization during inference
 
 The F32 activation input matrix (LHS) is dynamically quantized and packed by the *kai_run_lhs_quant_pack_qsi8d32p_f32_neon* microkernel every time, since the activation input keeps changing during the model run. It is done by following function call stack,
 
@@ -92,7 +103,7 @@ llama_context::decode
   ```
 Once the LHS and RHS is ready, KleidiAI matmul microkernel can be executed. 
 
-### During prefill
+### SME2 GEMM microkernel execution during prefill
 
 `Llama-3.2-3B-Instruct-Q4_0.gguf` model and 512-bit SME2 streaming vector length are used as an example. At the Prefill stage, the KleidiAI GEMM microkernel optimized with SME2, *kai_matmul_clamp_f32_qsi8d32p1vlx4_qsi4c32p4vlx4_1vlx4vl_sme2_mopa*, is selected by the KleidiAI trait, it produces a dequantized F32 output matrix. It is done right after LHS quantizing and packing by function call stack shown below.
 ```text
@@ -111,7 +122,7 @@ llama_context::decode
                                             kai_matmul_clamp_f32_qsi8d32p1vlx4_qsi4c32p4vlx4_1vlx4vl_sme2_mopa								 
 ```
 
-### During token decode
+### SME2 GEMV microkernel execution during token decode
 
 At the LLM decode stage, KleidiAI GEMV micro-kernel optimized with SME2, *kai_run_matmul_clamp_f32_qsi8d32p1x4_qsi4c32p4vlx4_1x4vl_sme2_sdot*， is selected by the KleidiAI trait in llama.cpp, it produces a dequantized F32 output vector. It is done right after LHS quantizing & packing by function call stack shown below,
 

content/learning-paths/mobile-graphics-and-gaming/performance_llama_cpp_sme2/run_llm.md

@@ -1,12 +1,12 @@
 ---
-title: Run the Llama-3.2-3B-Instruct-Q4_0.gguf model with llama-cli
+title: Measure SME2 acceleration in llama.cpp on Android
 weight: 5
 
 ### FIXED, DO NOT MODIFY
 layout: learningpathall
 ---
 
-## Run the Llama-3.2-3B-Instruct-Q4_0.gguf model with llama-cli 
+## Compare performance with SME2 enabled and disabled
 
 In this section, you run the model on an SME2-capable Android device and compare performance with SME2 enabled and disabled.
 
@@ -19,8 +19,8 @@ curl -L -o Llama-3.2-3B-Instruct-Q4_0.gguf \
     https://huggingface.co/unsloth/Llama-3.2-3B-Instruct-GGUF/resolve/main/Llama-3.2-3B-Instruct-Q4_0.gguf
 ```
 
-## Transfer the files using ADB
-This subsection shows a repeatable way to copy your `llama-cli` binary and GGUF model from your host machine to an Android target device, and then run the same commands from an interactive shell on the device.
+### Transfer the files using ADB
+This section shows a repeatable way to copy your `llama-cli` binary and GGUF model from your host machine to an Android target device, and then run the same commands from an interactive shell on the device.
 
 Enable **Developer options** and **USB debugging** on the Android device, connect it over USB (or over the network if your setup supports it), and verify that your host can see the device:
 
@@ -47,7 +47,7 @@ cd /data/local/tmp/llama_sme2
 The figure below shows the architecture of the Llama-3.2-3B model:
 ![Architecture diagram of the Llama-3.2-3B model showing the transformer block structure, including attention heads, feed-forward layers, and the overall token embedding and output projection flow alt-txt#center](images/llama-3.2-3b_architecture.jpg "Architecture of Llama-3.2-3B")
 
-## Inference with SME2 enabled
+### Run inference with SME2 enabled
 
 To enable SME2 microkernels, set the `GGML_KLEIDIAI_SME` environment variable before running the application. The flags used here are: `taskset 2` pins the process to CPU core 2 (the Arm C1-Pro core), `-st` enables single-token generation mode to report per-token performance stats, `-C 0x2 -Cb 0x2` sets CPU affinity for operator execution, and `-t 1` limits inference to one thread. If `taskset` is not available on your device, run the command without it.
 
@@ -67,7 +67,7 @@ With the SME2 kernels enabled, you'll notice the following performance output:
 
 Your results will vary depending on the device and the SME2 streaming vector length available.
 
-## Baseline performance without SME2
+### Baseline performance without SME2
 
 For a performance comparison, run the model with SME2 microkernels disabled. In this scenario, I8MM and DotProd microkernels are used instead.
 
@@ -81,7 +81,7 @@ The output is similar to:
 [ Prompt: 7.9 t/s | Generation: 5.9 t/s ]
 ```
 
-## Optional: Use Streamline to profile 
+## Profile with Arm Streamline (optional)
 
 You can profile the model execution with the approach introduced in [Profile llama.cpp performance with Arm Streamline and KleidiAI LLM kernels](https://learn.arm.com/learning-paths/servers-and-cloud-computing/llama_cpp_streamline/). 
 
@@ -93,7 +93,7 @@ The Streamline Call Paths view below indicates similar speedup, it also shows th
 
 ![Arm Streamline Call Paths view comparing inference with SME2 enabled against DotProd and I8MM fallback paths, showing the KleidiAI microkernel selected in each case alt-txt#center](images/streamline_call_paths_combined.jpg "Combined Streamline Call Paths view with and without SME2")
 
-## Optional: Print the kernel names at runtime
+## Print KleidiAI kernel names at runtime (optional)
 
 To investigate which operators in the model graph are delegated to KleidiAI microkernels, you can add some code as below to `./ggml/src/ggml-cpu/kleidiai/kleidiai.cpp` and recompile the binary. When you run the inference, the names of operators that make use of KleidiAI microkernels will be printed. This is only for debugging purposes.
 
@@ -127,7 +127,7 @@ kai matmul Q4_0 ffn_up-27
 kai matmul Q4_0 ffn_out-27
 ```
 
-## Summary
+## What you've accomplished and what's next
 
 You've run a quantized LLM on an SME2-capable Android device and measured the throughput difference with KleidiAI SME2 microkernels enabled and disabled. The results demonstrate meaningful gains at both the Prefill and Decode stages, making on-device LLM inference faster and more power-efficient on Arm hardware.