Skip to content

Commit d9c92d1

Browse files
authored
Update 4-benchmark.md
1 parent 28becd5 commit d9c92d1

1 file changed

Lines changed: 3 additions & 1 deletion

File tree

  • content/learning-paths/mobile-graphics-and-gaming/litert-sme

content/learning-paths/mobile-graphics-and-gaming/litert-sme/4-benchmark.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -182,7 +182,7 @@ Memory (bytes): count=0
182182
5 nodes observed
183183
```
184184

185-
From these benchmarking results you should notice significant latency reduction and throughput increase when KleidiAI SME2 micro-kernels are enabled.
185+
From these benchmarking results you should notice significant throughput uplift and speedup in inference time when KleidiAI SME2 micro-kernels are enabled.
186186

187187
### Interpreting Node Type Names for KleidiAI
188188
As you can see from the results, for the same model, the XNNPACK node type name is different. For the non-KleidiAI implementation, the node type is `Fully Connected (NC, F32) GEMM`, whereas for the KleidiAI implementation, it is `Fully Connected (NC, PF32) GEMM`.
@@ -211,3 +211,5 @@ For example, in `Convolution (NHWC, PQS8, QS8, QC8W)`, this represents a Conv2D
211211
* The input is packed INT8 quantized,
212212
* The weights are per-channel INT8 quantized,
213213
* The output is INT8 quantized.
214+
215+
By comparing benchmark_model runs with and without KleidiAI + SME2, and inspecting the profiled node types (PF32, PF16, QP8, PQS8), you can reliably confirm that LiteRT is dispatching to SME2-optimized KleidiAI micro-kernels and quantify their performance impact on your Android device.

0 commit comments

Comments
 (0)