Merge pull request #2652 from madeline-underwood/tflite

Tflite_JA to sign off

Author: jasonrandrews

content/learning-paths/mobile-graphics-and-gaming/litert-sme/1-litert-kleidiai-sme2.md

@@ -1,53 +1,52 @@
 ---
-title: Understand LiteRT, XNNPACK, KleidiAI and SME2
+title: Explore LiteRT, XNNPACK, KleidiAI, and SME2
 weight: 2
 
 ### FIXED, DO NOT MODIFY
 layout: learningpathall
 ---
 
-## LiteRT, XNNPACK, KleidiAI and SME2
+## Inside the LiteRT software stack
 
-LiteRT (Lite Runtime), formerly known as TensorFlow Lite, is a runtime for on-device AI.
-The default CPU acceleration library used by LiteRT is XNNPACK.
+LiteRT (Lightweight Runtime, formerly TensorFlow Lite) is a runtime for on-device AI on Arm platforms. The default CPU acceleration library used by LiteRT is XNNPACK.
 
-XNNPACK is an open-source library that provides highly optimized implementations of neural-network operators. It continuously integrates KleidiAI library to leverage new CPU features such as SME2.
+XNNPACK is an open-source library that provides highly optimized implementations of neural-network operators. It continuously integrates the KleidiAI library to use new CPU features such as Scalable Matrix Extension 2 (SME2).
 
-KleidiAI is a library developed by Arm that offers performance-critical micro-kernels leveraging Arm architecture features, such as SME2.
+KleidiAI is a library developed by Arm that offers performance-critical micro-kernels using Arm architecture features, such as SME2.
 
-Both XNNPACK and KleidiAI are external dependencies of LiteRT. LiteRT specifies the versions of these libraries to use.
-When LiteRT is built with both XNNPACK and KleidiAI enabled, XNNPACK invokes KleidiAI’s micro-kernels at runtime to accelerate operators with supported data types; otherwise, it falls back to its own implementation.
+The software stack for LiteRT is shown below.
 
-The software stack for LiteRT is as follows.
+![Diagram showing the software stack for on-device AI on Arm platforms. The stack is organized in layers from top to bottom: LiteRT at the top, followed by XNNPACK, then KleidiAI, and SME2 at the bottom. Arrows indicate the flow of execution from LiteRT through XNNPACK to KleidiAI and SME2. The diagram includes the following text labels: LiteRT, XNNPACK, KleidiAI, SME2. The environment is technical and structured, emphasizing the integration of Arm-optimized libraries for efficient AI inference. alt-text#center](./litert-sw-stack.png "LiteRT, XNNPACK, KleidiAI, and SME2 software stack")
 
-![LiteRT, XNNPACK, KleidiAI and SME2#center](./litert-sw-stack.png "LiteRT, XNNPACK, KleidiAI and SME2")
+## How KleidiAI works in LiteRT
 
+To understand how KleidiAI SME2 micro-kernels work in LiteRT, think about a LiteRT model with one fully connected operator using the FP32 data type. The following diagrams illustrate the execution workflow of XNNPACK’s implementation compared with the workflow when KleidiAI SME2 is enabled in XNNPACK.
 
-## Understand how KleidiAI works in LiteRT
+### LiteRT → XNNPACK workflow
 
-To understand how KleidiAI SME2 micro-kernel works in LiteRT, a LiteRT model with one Fully Connected operator with FP32 datatype is used as an example. 
+![Diagram showing the workflow for a fully connected operator in LiteRT using XNNPACK. The diagram depicts the flow from LiteRT to XNNPACK, highlighting the use of NEON instructions for matrix multiplication and weight packing on Arm platforms. The technical environment emphasizes operator traversal, hardware detection, and parallel computation. alt-text #center](./litert-xnnpack-workflow.png "LiteRT, XNNPACK workflow")
+A fully connected operator multiplies two matrices: the input activations (LHS) and the weights (RHS).
 
-The following illustrates the execution workflow of XNNPACK’s implementation compared with the workflow when KleidiAI SME2 is enabled in XNNPACK.
+When LiteRT loads a model, it reads the operators and builds a computation graph. If you select the CPU as the accelerator, LiteRT uses XNNPACK by default.
 
-### LiteRT → XNNPACK workflow
+XNNPACK scans the computation graph and looks for operators it can optimize. It packs the weight matrix to prepare for efficient computation. On Arm platforms, XNNPACK uses NEON instructions to speed up this packing and the matrix multiplication.
 
-![LiteRT, XNNPACK workflow#center](./litert-xnnpack-workflow.png "LiteRT, XNNPACK workflow")
+At runtime, XNNPACK checks the hardware and chooses the best available micro-kernel. During inference, it splits the matrices into smaller tiles and runs the multiplications in parallel across multiple threads, using NEON instructions for faster processing.
 
-A Fully Connected operator can be essentially implemented as a matrix multiplication.
+### LiteRT → XNNPACK → KleidiAI workflow
 
-When LiteRT loads a model, it parses the operators and create a computation graph. If the CPU is selected as the accelerator, LiteRT uses XNNPACK by default.
+![Diagram showing the workflow for a fully connected operator in LiteRT using XNNPACK and KleidiAI with SME2. The diagram illustrates the flow from LiteRT to XNNPACK, then to KleidiAI, highlighting SME2 micro-kernel integration for matrix multiplication and packing. The technical context emphasizes runtime hardware detection and optimized operator execution. alt-text #center](./litert-xnnpack-kleidiai-workflow.png "LiteRT, XNNPACK, KleidiAI workflow")
 
-XNNPACK traverses the operators in the graph and tries to replace them with its own implementations. During this stage, XNNPACK performs the necessary packing of the weight matrix. To speed up the packing process, XNNPACK uses NEON instructions for Arm platform. XNNPACK provides different implementations for different hardware platforms. At runtime, it detects the hardware capabilities and selects the appropriate micro-kernel.
+When KleidiAI and SME2 are enabled at build time, the KleidiAI SME2 micro-kernels are compiled into XNNPACK.
 
-During model inference, XNNPACK performs matrix multiplication on the activation matrix (the left-hand side matrix, LHS) and the repacked weight matrix (the right-hand side matrix, RHS). In this stage, XNNPACK applies tiling strategies to the matrices and performs parallel multiplication across the resulting tiles using multiple threads. To accelerate the computation, XNNPACK uses NEON instructions.
+During the model loading stage, when XNNPACK optimizes the subgraph, it checks the operator’s data type to determine whether a KleidiAI implementation is available. If KleidiAI supports it, XNNPACK bypasses its own default implementation. As a result, RHS packing is performed using the KleidiAI SME packing micro-kernel. Because KleidiAI typically requires packing of the LHS, a flag is also set during this stage.
 
+During model inference, the LHS packing micro-kernel is invoked. After the LHS is packed, XNNPACK performs the matrix multiplication. At this point, the KleidiAI SME micro-kernel is used to compute the matrix product.
 
-### LiteRT → XNNPACK → KleidiAI workflow
+## What you've accomplished and what's next
 
-![LiteRT, XNNPACK, KleidiAI workflow#center](./litert-xnnpack-kleidiai-workflow.png "LiteRT, XNNPACK, KleidiAI workflow")
+In this section, you explored how LiteRT leverages XNNPACK and KleidiAI to accelerate fully connected operators on Arm platforms. You learned how XNNPACK uses NEON instructions for efficient matrix operations and how enabling KleidiAI with SME2 further optimizes performance by introducing specialized micro-kernels for packing and matrix multiplication.
 
-When KleidiAI and SME2 are enabled at building stage, the KleidiAI SME2 micro-kernels are compiled into the XNNPACK. 
+You have completed the overview of LiteRT, XNNPACK, KleidiAI, and SME2 integration. Next, you’ll dive deeper into building and benchmarking models with these technologies.
 
-During the model loading stage, when XNNPACK optimizes the subgraph, it checks the operator’s data type to determine whether a KleidiAI implementation is available. If KleidiAI supports it, XNNPACK bypasses its own default implementation. As a result, RHS packing is performed using the KleidiAI SME packing micro-kernel. In addition, because KleidiAI typically requires packing of the LHS, a flag is also set during this stage.
 
-During model inference, the LHS packing micro-kernel is invoked. After the LHS is packed, XNNPACK performs the matrix multiplication. At this point, the KleidiAI SME micro-kernel is used to compute the matrix product.

content/learning-paths/mobile-graphics-and-gaming/litert-sme/2-build-tool.md

@@ -1,73 +1,85 @@
 ---
 title: Build the LiteRT benchmark tool
-weight: 3
+weight: 4
 
 ### FIXED, DO NOT MODIFY
 layout: learningpathall
 ---
 
-### Build the LiteRT benchmark tool with KleidiAI and SME2 enabled
+## Build the LiteRT benchmark tool with KleidiAI and SME2 enabled
 
-LiteRT provides provides a standalone performance measurement utility called `benchmark_model` for evaluating the performance of LiteRT models. 
+LiteRT provides a standalone performance measurement utility called `benchmark_model` for evaluating the performance of LiteRT models.
 
 In this section, you will build two versions of the benchmark tool:
-  * With KleidiAI + SME/SME2 enabled → uses Arm-optimized micro-kernels
-  * Without KleidiAI + SME/SME2 → baseline performance (NEON/SVE2 fallback)
-This comparison clearly demonstrates the gains provided by SME2 acceleration.
+- With KleidiAI and Scalable Matrix Extension version 2 (SME2) enabled, which uses Arm-optimized micro-kernels
+- Without KleidiAI and SME2, which provides baseline performance using NEON or SVE2 fallback
 
-First, clone the LiteRT repository.
+This comparison demonstrates the performance gains provided by SME2 acceleration.
 
-``` bash
+First, clone the LiteRT repository:
+
+```bash
 cd $WORKSPACE
 git clone https://github.com/google-ai-edge/LiteRT.git
 ```
-Because LiteRT integrates KleidiAI through XNNPACK, you must build LiteRT from source to enable SME2 micro-kernels.
 
-Next, set up your Android build environment using Docker on your Linux developement machine.
-Google provides a Dockerfile that installs the toolchain needed for TFLite/LiteRT Android builds.
+Because LiteRT integrates KleidiAI through XNNPACK (an open-source library providing highly optimized neural-network operators), you must build LiteRT from source to enable SME2 micro-kernels.
+
+Next, set up your Android build environment using Docker on your Linux development machine. Google provides a Dockerfile that installs the toolchain needed for TensorFlow Lite (TFLite)/LiteRT Android builds.
 
 Download the Dockerfile:
-``` bash
+
+```bash
 wget https://raw.githubusercontent.com/tensorflow/tensorflow/master/tensorflow/lite/tools/tflite-android.Dockerfile
 ```
+
 Build the Docker image:
+
 ```bash
 docker build . -t tflite-builder -f tflite-android.Dockerfile
 ```
-The Docker image includes Bazel, NDK, CMake, toolchains, and Python required for cross-compiling Android binaries.
 
-You will now install Android SDK/NDK Components inside the Container.
-Launch the docker container:
+The Docker image includes Bazel, Android Native Development Kit (NDK), CMake, toolchains, and Python required for cross-compiling Android binaries.
+
+Now, install Android Software Development Kit (SDK) and NDK components inside the container.
+
+Launch the Docker container:
+
 ```bash
 docker run -it -v $PWD:/host_dir tflite-builder bash
 ```
+
 Install Android platform tools:
-``` bash
+
+```bash
 sdkmanager \
   "build-tools;${ANDROID_BUILD_TOOLS_VERSION}" \
   "platform-tools" \
   "platforms;android-${ANDROID_API_LEVEL}"
 ```
 
-Configure LiteRT Build Options inside your running container:
+Configure LiteRT build options inside your running container:
 
-``` bash
+```bash
 cd /host_dir/LiteRT
 ./configure
 ```
 
-Use default values for all the prompts except when prompted:
+Use default values for all prompts except when asked:
+
 ```output
 Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]
 ```
-Type in `y`.
 
-LiteRT's configuration script will detect SDK + NDK paths, set toolchain versions, configure Android ABI (arm64-v8a) and initialize Bazel workspace rules.
+Type `y` and press Enter.
 
-Now, you can build the benchmark tool with KleidiAI + SME2 Enabled.
+LiteRT's configuration script will detect SDK and NDK paths, set toolchain versions, configure the Android Application Binary Interface (ABI) to arm64-v8a, and initialize Bazel workspace rules.
+
+Now, you can build the benchmark tool with KleidiAI and SME2 enabled.
 
 Enable XNNPACK, quantization paths, and SME2 acceleration:
-``` bash
+
+```bash
 export BENCHMARK_TOOL_PATH="litert/tools:benchmark_model"
 export XNNPACK_OPTIONS="--define tflite_with_xnnpack=true \
 --define=tflite_with_xnnpack_qs8=true \
@@ -77,24 +89,28 @@ export XNNPACK_OPTIONS="--define tflite_with_xnnpack=true \
 --define=xnn_enable_arm_sme2=true \
 --define=xnn_enable_kleidiai=true"
 ```
+
 Build for Android:
+
 ```bash
 bazel build -c opt --config=android_arm64 \
 ${XNNPACK_OPTIONS} "${BENCHMARK_TOOL_PATH}" \
 --repo_env=HERMETIC_PYTHON_VERSION=3.12
 ```
 
-This build enables the KleidiAI and SME2 micro-kernels integrated into XNNPACK and produces an Android binary under:
+This build enables the KleidiAI and SME2 micro-kernels integrated into XNNPACK and produces an Android binary at:
 
 ```output
 bazel-bin/litert/tools/benchmark_model
 ```
 
-### Build the LiteRT benchmark tool without KleidiAI (Baseline Comparison)
+## Build the LiteRT benchmark tool without KleidiAI (baseline comparison)
 
-To compare the performance of KleidiAI SME2 implementation against XNNPACK’s original implementation, you can build another version of LiteRT benchmark tool without KleidiAI and SME2 enabled.
+To compare the performance of the KleidiAI SME2 implementation against XNNPACK’s original implementation, build another version of the LiteRT benchmark tool without KleidiAI and SME2 enabled.
 
-``` bash
+Set the build options to disable SME2 and KleidiAI:
+
+```bash
 export BENCHMARK_TOOL_PATH="litert/tools:benchmark_model"
 export XNNPACK_OPTIONS="--define tflite_with_xnnpack=true \
 --define=tflite_with_xnnpack_qs8=true \
@@ -106,15 +122,20 @@ export XNNPACK_OPTIONS="--define tflite_with_xnnpack=true \
 ```
 
 Then rebuild:
+
 ```bash
 bazel build -c opt --config=android_arm64 \
 ${XNNPACK_OPTIONS} "${BENCHMARK_TOOL_PATH}" \
 --repo_env=HERMETIC_PYTHON_VERSION=3.12
 ```
-This build of the `benchmark_model` disables all SME2 micro-kernels and forces fallback to XNNPACK’s NEON/SVE2 kernels.
-You can then use ADB to push the benchmark tool to your Android device.
+
+This build of the `benchmark_model` disables all SME2 micro-kernels and forces fallback to XNNPACK’s NEON or SVE2 kernels.
+
+You can then use Android Debug Bridge (ADB) to push the benchmark tool to your Android device:
 
 ```bash
 adb push bazel-bin/litert/tools/benchmark_model /data/local/tmp/
 adb shell chmod +x /data/local/tmp/benchmark_model
 ```
+
+You have now built both versions of the LiteRT benchmark tool. You are ready to benchmark and compare SME2-accelerated and baseline performance on your Arm-based Android device.