new learning path: Unleashing leading On-Device AI performance with llama.cpp, SME2 and KleidiAI

Author: Zenon Xiu

content/learning-paths/mobile-graphics-and-gaming/performance_llama_cpp_sme2/_index.md

@@ -0,0 +1,59 @@
+---
+title: Unleashing leading On-Device AI performance with llama.cpp, SME2 and KleidiAI
+
+minutes_to_complete: 40
+
+who_is_this_for: This is an advanced topic for software developers, performance engineers, and AI practitioners 
+
+learning_objectives: 
+    - Build llama.cpp library with KleidiAI and SME2 support
+    - Profile performance of LLMs running on llama-cli
+    - Learn how KleidiAI and SME2 accelerates LLM operators
+
+prerequisites:
+    - Knowledge of KleidiAI and SME2
+    - An Linux or Android device with Arm SME2 support
+
+author: Zenon Zhilong Xiu
+
+### Tags
+skilllevels: Advanced
+subjects: ML
+armips:
+    - Arm C1 CPU
+    - Arm SME2 unit
+tools_software_languages:
+    - C++
+    - llama.cpp
+operatingsystems:
+    - Android
+    - Linux
+
+
+
+further_reading:
+    - resource:
+        title: part 1 Arm Scalable Matrix Extension Introduction 
+        link: https://developer.arm.com/community/arm-community-blogs/b/architectures-and-processors-blog/posts/arm-scalable-matrix-extension-introduction
+        type: blog
+    - resource:
+        title: part 2 Arm Scalable Matrix Extension Instructions 
+        link: https://developer.arm.com/community/arm-community-blogs/b/architectures-and-processors-blog/posts/arm-scalable-matrix-extension-introduction-p2
+        type: blog
+    - resource:
+        title: part4 Arm SME2 Introduction 
+        link: https://developer.arm.com/community/arm-community-blogs/b/architectures-and-processors-blog/posts/part4-arm-sme2-introduction
+        type: blog
+    - resource:
+        title: Profile llama.cpp performance with Arm Streamline and KleidiAI LLM kernels
+        link: https://learn.arm.com/learning-paths/servers-and-cloud-computing/llama_cpp_streamline/
+        type: blog
+        
+
+
+### FIXED, DO NOT MODIFY
+# ================================================================================
+weight: 1                       # _index.md always has weight of 1 to order correctly
+layout: "learningpathall"       # All files under learning paths have this same wrapper
+learning_path_main_page: "yes"  # This should be surfaced when looking for related content. Only set for _index.md of learning path content.
+---

content/learning-paths/mobile-graphics-and-gaming/performance_llama_cpp_sme2/_next-steps.md

@@ -0,0 +1,8 @@
+---
+# ================================================================================
+#       FIXED, DO NOT MODIFY THIS FILE
+# ================================================================================
+weight: 21                  # The weight controls the order of the pages. _index.md always has weight 1.
+title: "Next Steps"         # Always the same, html page title.
+layout: "learningpathall"   # All files under learning paths have this same wrapper for Hugo processing.
+---

content/learning-paths/mobile-graphics-and-gaming/performance_llama_cpp_sme2/build_llama_cpp.md

@@ -0,0 +1,57 @@
+---
+title: Build llama.cpp with KleidiAI and SME2 enabled
+weight: 4
+
+### FIXED, DO NOT MODIFY
+layout: learningpathall
+---
+
+## Build llama.cpp with KleidiAI and SME2 enabled
+For convenience, llama.cpp is statically linked. We use aarch64 GCC cross compile toolchain, *aarch64-none-linux-gnu-* to build the project. To support SME2, GCC compiler version 14.2 and onwards is required. The toolchain can be downloaded [here](https://developer.arm.com/downloads/-/arm-gnu-toolchain-downloads) . 
+
+The llama.cpp with Tag b7611 is used in this tutorial, newer versions should also work but they are not tested. 
+
+After downloading the llama.cpp source code [here](https://github.com/ggml-org/llama.cpp/releases/tag/b7610), create a new directory *build* under the llama.cpp root directory and change to the new directory,
+
+```bash
+cd ~/llama.cpp
+mkdir build && cd build
+```
+Next, configure the project,
+
+```bash
+cmake .. \
+  -DCMAKE_SYSTEM_NAME=Linux \
+  -DCMAKE_SYSTEM_PROCESSOR=arm \
+  -DCMAKE_C_COMPILER=aarch64-linux-gnu-gcc \
+  -DCMAKE_CXX_COMPILER=aarch64-linux-gnu-g++ \
+  -DLLAMA_NATIVE=OFF \
+  -DLLAMA_F16C=OFF \
+  -DLLAMA_GEMM_ARM=ON \
+  -DBUILD_SHARED_LIBS=OFF \
+  -DCMAKE_EXE_LINKER_FLAGS="-static -g" \
+  -DGGML_OPENMP=OFF \
+  -DCMAKE_C_FLAGS=" -march=armv8.7-a+sve+i8mm+dotprod+sme2 -g" \
+  -DCMAKE_CXX_FLAGS=" -march=armv8.7-a+sve+i8mm+dotprod+sme2 -g" \
+  -DLLAMA_BUILD_TESTS=OFF  \
+  -DLLAMA_BUILD_EXAMPLES=ON \
+  -DLLAMA_CURL=OFF \
+  -DGGML_CPU_KLEIDIAI=ON 
+```
+Set CMAKE_C_COMPILER and CMAKE_CXX_COMPILER to your cross compiler path. Make sure that *-march* in CMAKE_C_FLAGS and CMAKE_CXX_FLAGS includes "+sme2".
+
+The *-static* and *-g* options are specified to produce a statically linked executable, in oder to run on different Arm64 Linux/Android environments and include debug information.
+
+Next, build the project,
+
+```bash
+cd ~/llama.cpp/build
+cmake --build ./ --config Release -j $(nproc)
+```
+After the building process completes, you can find the application,*llama-cli*,in the ~/llama.cpp/build/bin/ directory.
+
+To enable SME2 microkernels, you must set following environment variable before running the application.
+
+```bash 
+GGML_KLEIDIAI_SME="1"
+```

content/learning-paths/mobile-graphics-and-gaming/performance_llama_cpp_sme2/images/kai_matmul_kernel.jpg

@@ -0,0 +1,19 @@
+---
+title: Overview
+weight: 2
+
+### FIXED, DO NOT MODIFY
+layout: Overview
+---
+
+## Introduction 
+Arm’s latest Client CPU processors such as Arm C1 include Scalable Matrix Extension 2 (SME2). SME2 accelerates the matrix-heavy AI operations behind large language models (LLMs), media processing, speech recognition, computer vision, real-time apps and multimodal apps.
+
+llama.cpp provides extensive support for many LLMs, including Phi, Llama, DeepSeek, Gemma and Qwen. Llama.cpp is designed for efficient CPU-based inference. It enables on-device LLM execution, reducing latency and enhancing privacy.
+
+By default llama.cpp integrates with Arm KleidiAI, a suite of optimized microkernels for Arm CPUs. KleidiAI includes SME2 optimized microkernels to get more performance benefits. 
+
+In this learning path, llama.cpp and Llama-3.2-3B-Instruct-Q4_0.gguf model with 3 Billion parameters is used for the tutorial. 
+
+
+