Skip to content

Commit 65dadc4

Browse files
Refactor LiteRT documentation for clarity and structure; remove obsolete model creation section
1 parent ddb25be commit 65dadc4

3 files changed

Lines changed: 9 additions & 14 deletions

File tree

content/learning-paths/mobile-graphics-and-gaming/litert-sme/1-litert-kleidiai-sme2.md

Lines changed: 9 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ weight: 2
66
layout: learningpathall
77
---
88

9-
## LiteRT, XNNPACK, KleidiAI, and SME2
9+
## Understand the LiteRT software stack and SME2 integration
1010

1111
LiteRT (Lightweight Runtime, formerly TensorFlow Lite) is a runtime for on-device AI on Arm platforms. The default CPU acceleration library used by LiteRT is XNNPACK (an open-source library providing highly optimized neural-network operators).
1212

@@ -16,27 +16,26 @@ KleidiAI is a library developed by Arm that offers performance-critical micro-ke
1616

1717
The software stack for LiteRT is shown below.
1818

19-
![Diagram showing the software stack for on-device AI on Arm platforms. The stack is organized in layers from top to bottom: LiteRT at the top, followed by XNNPACK, then KleidiAI, and SME2 at the bottom. Arrows indicate the flow of execution from LiteRT through XNNPACK to KleidiAI and SME2. The diagram includes the following text labels: LiteRT, XNNPACK, KleidiAI, SME2. The environment is technical and structured, emphasizing the integration of Arm-optimized libraries for efficient AI inference. #center](./litert-sw-stack.png "LiteRT, XNNPACK, KleidiAI and SME2")
19+
![Diagram showing the software stack for on-device AI on Arm platforms. The stack is organized in layers from top to bottom: LiteRT at the top, followed by XNNPACK, then KleidiAI, and SME2 at the bottom. Arrows indicate the flow of execution from LiteRT through XNNPACK to KleidiAI and SME2. The diagram includes the following text labels: LiteRT, XNNPACK, KleidiAI, SME2. The environment is technical and structured, emphasizing the integration of Arm-optimized libraries for efficient AI inference. alt-text#center](./litert-sw-stack.png "LiteRT, XNNPACK, KleidiAI and SME2")
2020

2121
## How KleidiAI works in LiteRT
2222

2323
To understand how KleidiAI SME2 micro-kernels work in LiteRT, consider a LiteRT model with one Fully Connected operator using the FP32 data type.
2424

2525
The following diagrams illustrate the execution workflow of XNNPACK’s implementation compared with the workflow when KleidiAI SME2 is enabled in XNNPACK.
2626

27-
### LiteRT → XNNPACK workflow
27+
## LiteRT → XNNPACK workflow
2828

29-
![Diagram showing the workflow for a Fully Connected operator in LiteRT using XNNPACK. The diagram depicts the flow from LiteRT to XNNPACK, highlighting the use of NEON instructions for matrix multiplication and weight packing on Arm platforms. The technical environment emphasizes operator traversal, hardware detection, and parallel computation. #center](./litert-xnnpack-workflow.png "LiteRT, XNNPACK workflow")
29+
![Diagram showing the workflow for a Fully Connected operator in LiteRT using XNNPACK. The diagram depicts the flow from LiteRT to XNNPACK, highlighting the use of NEON instructions for matrix multiplication and weight packing on Arm platforms. The technical environment emphasizes operator traversal, hardware detection, and parallel computation. alt-text #center](./litert-xnnpack-workflow.png "LiteRT, XNNPACK workflow")
30+
A Fully Connected operator works by multiplying two matrices together.
3031

31-
A Fully Connected operator is essentially implemented as a matrix multiplication.
32+
When LiteRT loads a model, it reads the operators and builds a computation graph. If you choose the CPU as the accelerator, LiteRT uses XNNPACK by default.
3233

33-
When LiteRT loads a model, it parses the operators and creates a computation graph. If the CPU is selected as the accelerator, LiteRT uses XNNPACK by default.
34+
XNNPACK scans the computation graph and tries to replace operators with its own optimized versions. At this stage, XNNPACK packs the weight matrix. On Arm platforms, it uses NEON instructions to make this process faster. XNNPACK includes different implementations for different hardware. At runtime, it checks the hardware and picks the best micro-kernel.
3435

35-
XNNPACK traverses the operators in the graph and tries to replace them with its own implementations. During this stage, XNNPACK packs the weight matrix, using NEON instructions for Arm platforms to speed up the process. XNNPACK provides different implementations for different hardware platforms. At runtime, it detects hardware capabilities and selects the appropriate micro-kernel.
36+
During inference, XNNPACK multiplies the activation matrix (left-hand side, LHS) by the packed weight matrix (right-hand side, RHS). It splits the matrices into smaller tiles and runs the multiplications in parallel using multiple threads. NEON instructions help speed up these calculations.
3637

37-
During model inference, XNNPACK performs matrix multiplication on the activation matrix (left-hand side, LHS) and the repacked weight matrix (right-hand side, RHS). XNNPACK applies tiling strategies to the matrices and performs parallel multiplication across the resulting tiles using multiple threads. To accelerate computation, XNNPACK uses NEON instructions.
38-
39-
### LiteRT → XNNPACK → KleidiAI workflow
38+
## LiteRT → XNNPACK → KleidiAI workflow
4039

4140
![Diagram showing the workflow for a Fully Connected operator in LiteRT using XNNPACK and KleidiAI with SME2. The diagram illustrates the flow from LiteRT to XNNPACK, then to KleidiAI, highlighting SME2 micro-kernel integration for matrix multiplication and packing. The technical context emphasizes runtime hardware detection and optimized operator execution. alt-text #center](./litert-xnnpack-kleidiai-workflow.png "LiteRT, XNNPACK, KleidiAI workflow")
4241

content/learning-paths/mobile-graphics-and-gaming/litert-sme/2-build-tool.md

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -104,8 +104,6 @@ This build enables the KleidiAI and SME2 micro-kernels integrated into XNNPACK a
104104
bazel-bin/litert/tools/benchmark_model
105105
```
106106

107-
---
108-
109107
## Build the LiteRT benchmark tool without KleidiAI (baseline comparison)
110108

111109
To compare the performance of the KleidiAI SME2 implementation against XNNPACK’s original implementation, build another version of the LiteRT benchmark tool without KleidiAI and SME2 enabled.
@@ -140,6 +138,4 @@ adb push bazel-bin/litert/tools/benchmark_model /data/local/tmp/
140138
adb shell chmod +x /data/local/tmp/benchmark_model
141139
```
142140

143-
---
144-
145141
You have now built both versions of the LiteRT benchmark tool. You are ready to benchmark and compare SME2-accelerated and baseline performance on your Arm-based Android device.

content/learning-paths/mobile-graphics-and-gaming/litert-sme/3-buid-model.md renamed to content/learning-paths/mobile-graphics-and-gaming/litert-sme/3-build-model.md

File renamed without changes.

0 commit comments

Comments
 (0)