diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/1-overview.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/1-overview.md new file mode 100644 index 0000000000..729cabb763 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/1-overview.md @@ -0,0 +1,57 @@ +--- +title: Overview +weight: 2 +layout: learningpathall +--- + +## Overview + +[MNIST](https://en.wikipedia.org/wiki/MNIST_database), widely classified as the "Hello World" of machine learning, is a dataset containing 70,000 28x28 pixel grayscale images of handwritten digits 0-9. It is commonly used for training image processing systems. + +In this Learning Path, you will run an MNIST digit-classification model on the Arm Ethos-U85 NPU. You can either use the provided `.pte` model or follow an optional walkthrough to train and export your own MNIST model. + +## Hardware Overview - Alif Ensemble E8 Series + +The Alif Ensemble E8 DevKit features two dual-core Arm processors (Cortex-A32 and Cortex-M55) and three neural processing units (NPUs): two Ethos-U55 and one Ethos-U85. + +
+ Alif Ensemble E8 Board SoC highlighted +
Arm Ethos-U NPU location
+
+ +### Connecting to the DevKit + +- Unplug all USB cables from the DevKit before changing any jumpers. + +- Verify that the jumpers are in their factory default positions, as shown in the Alif Ensemble E8 DevKit (DK-E8) User Guide on [alifsemi.com](https://alifsemi.com/support/kits/ensemble-e8devkit/). + +- Connect a USB-C cable from your computer to the PRG USB port on the bottom edge of the DevKit. + +![PRG USB Port Location alt-text#center](prg-usb-port.png "Connect USB-C cable to the PRG USB port") +- Confirm that a green LED illuminates near the E1 device and the UART switch (SW4). + +Leave SW4 in its default position. This routes the on-board USB UART to SEUART, which the Alif Security Toolkit (SETOOLS) uses for programming. + +#### Verify USB connection + +Check that your computer recognizes the DevKit: + +{{< tabpane code=true >}} + {{< tab header="macOS" language="bash">}} +ls /dev/cu.* + {{< /tab >}} + + {{< tab header="Linux" language="bash">}} +ls /dev/ttyACM* /dev/ttyUSB* 2>/dev/null + {{< /tab >}} + + {{< tab header="Windows" language="powershell">}} +Get-CimInstance Win32_SerialPort | Select-Object DeviceID,Name,Description + {{< /tab >}} +{{< /tabpane >}} + +{{% notice Important %}} +Close any terminal application that’s connected to SEUART, such as PuTTY, minicom, or screen, before you use the Security Toolkit (SETOOLS). The DevKit exposes only one SEUART interface, so SETOOLS can’t access the port if another application is already using it. +{{% /notice %}} + +You should see a SEGGER J-Link device. If you are unsure which entry belongs to the DevKit, run the command before and after connecting the board and compare the output. In the case where no device appears, check that the USB cable is connected to the **PRG USB** port and that the cable supports data, not only charging. \ No newline at end of file diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/2-tool-installation.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/2-tool-installation.md new file mode 100644 index 0000000000..ff6bf88a82 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/2-tool-installation.md @@ -0,0 +1,161 @@ +--- +title: Install development tools +weight: 3 +layout: learningpathall +--- + +## Overview + +This section covers installing all required tools for ExecuTorch development on the Alif Ensemble E8 DevKit. + + +Start by creating a project directory: +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash" >}} +mkdir -p ~/mnist_alif + {{< /tab >}} + + {{< tab header="Windows (PowerShell)" language="powershell" >}} +New-Item -ItemType Directory -Force -Path ~\mnist_alif + {{< /tab >}} +{{< /tabpane >}} + +## Install SETOOLS + +SETOOLS (Secure Enclave Tools) is Alif's toolset for flashing firmware to MRAM via the Secure Enclave. + +- Download the SETOOLS package from the [Alif Ensemble E8 DevKit support page](https://alifsemi.com/support/kits/ensemble-e8devkit/) and extract it to `~/mnist_alif`. +Make sure to edit the command below with the name of your `.tar` or `.zip` file. +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash" >}} +cd ~/Downloads +tar xvf -C ~/mnist_alif + {{< /tab >}} + + {{< tab header="Windows (PowerShell)" language="powershell" >}} +cd ~/Downloads +Expand-Archive <.\replace_with_your_alif_security_toolkit_download.zip> -DestinationPath ~\mnist_alif + {{< /tab >}} +{{< /tabpane >}} + +- Verify the installation. The extracted folder name can vary by SETOOLS release. The commands below assume the package extracts to `app-release-exec-*`. +Each command should print a `usage:` message. If either command fails, check that you are in the extracted SETOOLS directory for your operating system. +{{< tabpane code=true >}} + {{< tab header="macOS" language="bash" >}} +cd ~/mnist_alif/app-release-exec-macos +./app-write-mram -h +./app-gen-toc -h + {{< /tab >}} + {{< tab header="Linux" language="bash" >}} +cd ~/mnist_alif/app-release-exec-linux +./app-write-mram -h +./app-gen-toc -h + {{< /tab >}} + {{< tab header="Windows" language="powershell" >}} +cd ~\mnist_alif\app-release-exec-windows +.\app-write-mram.exe -h +.\app-gen-toc.exe -h + {{< /tab >}} +{{< /tabpane >}} + +{{% notice Important %}} +On macOS, the system may block the unsigned binary the first time you run it. If this happens, open **System Settings** or **System Preferences**, go to **Privacy & Security**, and select **Allow Anyway**. Then run the command again. (You may need to reapprove for both ./app-* commands) +{{% /notice %}} + + +## Install J-Link + +SEGGER J-Link provides the debug connection for RTT (Real-Time Transfer) output, which you use later to view inference results. +Version 7.94 or later is required for Alif Ensemble E8 support. + +{{< tabpane code=true >}} + {{< tab header="macOS" language="bash">}} +brew install --cask segger-jlink +JLinkExe --version + {{< /tab >}} + {{< tab header="Linux" language="bash">}} +wget https://www.segger.com/downloads/jlink/JLink_Linux_x86_64.deb +sudo dpkg -i JLink_Linux_x86_64.deb +JLinkExe --version + {{< /tab >}} + {{< tab header="Windows" language="text">}} +1. Download installer from https://www.segger.com/downloads/jlink/ +2. Run the installer and follow prompts +3. Verify in Command Prompt: JLink.exe --version + {{< /tab >}} +{{< /tabpane >}} + + +## Set up the Alif VS Code template +- Clone the Alif VS Code template repository and checkout to a known-working commit. This is to avoid breakage if the template has been updated. +```bash +cd ~/mnist_alif +git clone https://github.com/alifsemi/alif_vscode-template.git +cd alif_vscode-template +git checkout 8b1aa0b09eacf68a28850af00c11f0b5af03c100 +git submodule update --init +``` + +- Open the project in VS Code: +```bash +code . +``` +VS Code might prompt you to install the recommended extensions for this workspace. If it does, install the following: + +- Arm CMSIS Solution +- Arm Tools Environment Manager +- Cortex-Debug +- Microsoft C/C++ Extension Pack + +When prompted, select **Always Allow** or **Allow for Selected Workspace**. + +The recommended VS Code extensions are listed in .vscode/extensions.json. If you don’t get an automatic trigger to enable them, you can open the Extensions view and look for a “Workspace Recommendations” section to install or enable them manually. + +Restart VS Code if prompted. + +## Install CMSIS packs +CMSIS (Common Microcontroller Software Interface Standard) is a set of APIs, software components, and metadata that simplifies development on Arm Cortex-M processors. +Installing the CMSIS pack will provide the device definitions, startup files, drivers, and middleware components we need for our Alif E8 target. To install, follow these steps: + +In VS Code, open the Command Palette using **Ctrl+Shift+P** on Windows/Linux or **Command+Shift+P** on macOS. + +Click **Tasks: Run Task**, then select **First time pack installation**. When prompted, press **A** to accept all licenses. + +If the task does not appear, run **Developer: Reload Window** from the Command Palette and try again. + +## Configure VS Code settings + +VS Code need to know where the external Alif SETOOLS and SEGGER J-Link tools are installed. + +Open the Command Palette and run **Preferences: Open User Settings (JSON)**. + +Add the following settings, updating the paths for your operating system: + +```json +{ + "alif.setools.root": "path/to/your/setools-folder", + "cortex-debug.JLinkGDBServerPath": "/Applications/SEGGER/JLink/JLinkGDBServerCLExe" +} +``` +If your settings file already contains entries, add only these two settings inside the existing braces. + +### Verify your toolchain with Blinky +Before moving on to the ML application, build and flash the built-in Blinky example. +- In VS Code, select the **CMSIS** icon in the left sidebar. +- Select the **gear icon**. +- Set **Active Target** to **E8-HE**. +- Set **Active Project** to **blinky**. +- Select the **Build** (hammer) icon. +- Open the **Command Palette** and run **Tasks: Run Task**. +- Select **Program with Security Toolkit (select COM port)**. +- Choose the DevKit port when prompted. + +If the RGB LED blinks, your VS Code setup, CMSIS packs, SETOOLS configuration, and board connection are working. + +## Summary + +You have installed: +- ✅ SETOOLS for flashing firmware to MRAM +- ✅ J-Link (7.94+) for programming and debugging +- ✅ Required VS Code extensions & CMSIS packs + diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/3-docker-executorch-setup.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/3-docker-executorch-setup.md new file mode 100644 index 0000000000..a41bec4b32 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/3-docker-executorch-setup.md @@ -0,0 +1,394 @@ +--- +title: (Optional) Set up Docker development environment +weight: 4 +layout: learningpathall +--- + +## Overview + +This section creates the Docker environment used to train and export the MNIST model to ExecuTorch `.pte` format. + +{{% notice Note %}} +If you are using the provided `.pte` file, you can skip this section. Complete this section only if you want to train and export the model yourself. +{{% /notice %}} + +## Install Docker Desktop + +Docker provides an isolated environment with all the build dependencies needed. Download Docker Desktop from [docker.com](https://www.docker.com/products/docker-desktop), run the installer for your operating system, and follow the setup prompts. + +After installation, start Docker Desktop and verify that Docker is available from your terminal: +```bash +docker --version +``` +Expected output (or similar): +```output +Docker version 24.0.7, build afdd53b +``` + +### Verify Docker Installation + +Test Docker is working: + +```bash +docker run hello-world +``` + +The output is similar to: +```output +Hello from Docker! +This message shows that your installation appears to be working correctly. +``` + +## Create the Docker workspace + +Create a folder for the Docker files, model scripts, and generated output: + +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash" >}} +cd ~/mnist_alif +mkdir -p executorch-alif/models executorch-alif/output +cd executorch-alif + {{< /tab >}} + + {{< tab header="Windows (PowerShell)" language="powershell" >}} +cd ~\mnist_alif +New-Item -ItemType Directory -Force -Path .\executorch-alif\models, .\executorch-alif\output +cd .\executorch-alif + {{< /tab >}} +{{< /tabpane >}} + +The directory will be used like this: +```text +executorch-alif/ +├── Dockerfile +├── start-dev.sh # macOS/Linux +├── start-dev.ps1 # Windows +├── models/ # mounted to /home/developer/models +└── output/ # mounted to /home/developer/output +``` + +## Create the Dockerfile + +Create a file named `Dockerfile` +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash" >}} +touch Dockerfile +code Dockerfile + {{< /tab >}} + + {{< tab header="Windows (PowerShell)" language="powershell" >}} +New-Item -ItemType File -Path .\Dockerfile +notepad .\Dockerfile + {{< /tab >}} +{{< /tabpane >}} + +and paste this in: + +```dockerfile +FROM ubuntu:22.04 +ENV DEBIAN_FRONTEND=noninteractive +RUN apt-get update && apt-get install -y build-essential ca-certificates cmake curl git ninja-build python3 python3-pip python3-venv unzip vim wget xxd xz-utils && rm -rf /var/lib/apt/lists/* +RUN useradd -m -s /bin/bash developer +USER developer +WORKDIR /home/developer +RUN python3 -m venv /home/developer/executorch-venv +RUN /bin/bash -c "source /home/developer/executorch-venv/bin/activate && pip install --upgrade pip setuptools wheel && pip install torch==2.9.0 torchvision==0.24.0 torchaudio==2.9.0 --index-url https://download.pytorch.org/whl/cpu && pip install ethos-u-vela==4.4.1" +ENV VIRTUAL_ENV=/home/developer/executorch-venv +ENV PATH="$VIRTUAL_ENV/bin:$PATH" +CMD ["/bin/bash"] +``` + +## Build the Docker Image + +Build the image from the `executorch-alif` directory (this takes 5-10 minutes): + +```bash +docker build -t executorch-alif:latest . +``` + +The output shows: +```output +[+] Building 320.5s (12/12) FINISHED + => [internal] load build definition from Dockerfile + => => transferring dockerfile: 1.2kB + => [internal] load .dockerignore + ... + => => naming to docker.io/library/executorch-alif:latest +``` + +Verify the image: + +```bash +docker images +``` + +The output should include: +```output +executorch-alif latest +``` + +## Create the container startup script + +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash" >}} +cat > start-dev.sh << 'EOF' +#!/bin/bash +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +docker run -it --rm --name executorch-alif-dev -v "${SCRIPT_DIR}/models:/home/developer/models" -v "${SCRIPT_DIR}/output:/home/developer/output" -w /home/developer executorch-alif:latest /bin/bash +EOF +chmod +x start-dev.sh + {{< /tab >}} + + {{< tab header="Windows (PowerShell)" language="powershell" >}} +@' +$ScriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path +docker run -it --rm --name executorch-alif-dev -v "${ScriptDir}/models:/home/developer/models" -v "${ScriptDir}/output:/home/developer/output" -w /home/developer executorch-alif:latest /bin/bash +'@ | Set-Content -Encoding ascii .\start-dev.ps1 + {{< /tab >}} +{{< /tabpane >}} + +## Start the Development Container + +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash">}} +./start-dev.sh + {{< /tab >}} + {{< tab header="Windows" language="powershell">}} +.\start-dev.ps1 + {{< /tab >}} +{{< /tabpane >}} + +You should see the following prompt: + +```output +developer@container_ID:~$ +``` + +You are now inside the Docker container. + +## Clone ExecuTorch + +Inside the Docker container, clone and install ExecuTorch v1.0.0: + +```bash +cd /home/developer +# Clone ExecuTorch +git clone https://github.com/pytorch/executorch.git +cd executorch +# Checkout stable release +git checkout v1.0.0 +# Initialize submodules +git submodule sync +git submodule update --init --recursive +``` + +{{% notice Note %}} +Submodule initialization may take 5-10 minutes depending on your connection. +{{% /notice %}} + +Set Environment Variable (`ET_HOME`): +```bash +export ET_HOME=/home/developer/executorch +echo 'export ET_HOME=/home/developer/executorch' >> ~/.bashrc +``` + +### Install Python Dependencies +Activate the Python environment and run the installer script. + +```bash +# Ensure virtual environment is active +source ~/executorch-venv/bin/activate +cd $ET_HOME +# Upgrade pip +pip install --upgrade pip +# Install ExecuTorch base dependencies +./install_requirements.sh +# IMPORTANT: Install lxml with version compatible with Vela +# Vela 4.4.1 requires lxml>=4.7.1,<6.0.1 +pip install 'lxml>=4.7.1,<6.0.1' +``` + +### Install ExecuTorch + +```bash +cd $ET_HOME +# Install ExecuTorch in editable mode +CMAKE_BUILD_PARALLEL_LEVEL=2 pip install --no-build-isolation -e . +``` + +{{% notice Note %}} +The `--no-build-isolation` flag is required so ExecuTorch finds the PyTorch installation from `install_requirements.sh`. + +`CMAKE_BUILD_PARALLEL_LEVEL=2` limits the number of parallel CMake build jobs during installation. This reduces peak memory usage and helps avoid out-of-memory failures. +{{% /notice %}} + +Verify the installation: + +```bash +python3 -c "from executorch.exir import to_edge; print('ExecuTorch installed successfully')" +``` + +Expected output: +```output +ExecuTorch installed successfully +``` + +## Set Up Arm Ethos-U Dependencies +Run the ExecuTorch Arm setup script: +```bash +cd $ET_HOME +# Run the Arm setup script +./examples/arm/setup.sh --i-agree-to-the-contained-eula +``` + +This script sets up: +- TOSA Libraries +- Ethos-U SDK structure +- CMake toolchain files + + + +### Create Environment Setup Script + +Create a reusable environment script for future sessions: + +```bash +cat > $ET_HOME/setup_arm_env.sh << 'EOF' +#!/usr/bin/env bash + +export ET_HOME=/home/developer/executorch +source ~/executorch-venv/bin/activate +if [ -f "$ET_HOME/examples/arm/arm-scratch/setup_path.sh" ]; then + source "$ET_HOME/examples/arm/arm-scratch/setup_path.sh" +fi +if [ -f "$ET_HOME/examples/arm/ethos-u-scratch/setup_path.sh" ]; then + source "$ET_HOME/examples/arm/ethos-u-scratch/setup_path.sh" +fi +export TARGET_CPU=cortex-m55 +export ETHOSU_TARGET_NPU_CONFIG=ethos-u85-256 +export SYSTEM_CONFIG=Ethos_U85_SYS_DRAM_Mid +export MEMORY_MODE=Shared_Sram +echo "ExecuTorch Arm environment loaded" +echo "ET_HOME: $ET_HOME" +echo "Vela: $(which vela 2>/dev/null || echo not found)" +EOF + +chmod +x $ET_HOME/setup_arm_env.sh +echo 'source $ET_HOME/setup_arm_env.sh' >> ~/.bashrc +source $ET_HOME/setup_arm_env.sh +``` + +## Verify Complete Installation + +Run all verification checks: + + + +Check Vela Compiler + +```bash +vela --version +``` + +Expected output: +```output +4.4.1 +``` +Check ExecuTorch + +```bash +python3 -c "from executorch.exir import to_edge; print('ExecuTorch OK')" +``` + +Expected output: +```output +ExecuTorch OK +``` + +## Quick Verification Test + +Run a minimal export test to verify the complete setup: + +```bash +cd $ET_HOME +python3 -m examples.arm.aot_arm_compiler --model_name=add --delegate --quantize --target=ethos-u85-256 --output=/home/developer/output/add_ethos_u85.pte +``` + +Expected output: +```output +Exporting model add... +Lowering to TOSA... +Compiling with Vela... +PTE file saved as add_arm_delegate_ethos-u55-128.pte +``` + +Verify the `.pte` file was created: + +```bash +ls -lh /home/developer/output/add_ethos_u85.pte +``` + +The `output` directory is mounted from your host machine, so the file is also available at: +```bash +~/mnist_alif/executorch-alif/output/ +``` + +## Save Container State (Optional) + +To preserve your work, you can commit the container to a new image: + +```bash +# On your host machine (outside Docker) +docker commit executorch-alif-dev executorch-alif:configured +``` + +## Summary + +You have created a Docker environment for model export. The container has: +- Python and PyTorch +- ExecuTorch +- Ethos-U Vela +- Mounted models and output directories for sharing files with the host + +In the next section, you'll export a PyTorch model to ExecuTorch `.pte` format. diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/4-model-export.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/4-model-export.md new file mode 100644 index 0000000000..4dc5b30db4 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/4-model-export.md @@ -0,0 +1,195 @@ +--- +title: (Optional) Export PyTorch model to ExecuTorch format +weight: 5 +layout: learningpathall +--- + +## Overview + +This section exports an MNIST PyTorch model to ExecuTorch `.pte` format for the Arm Ethos-U85 NPU on the Alif Ensemble E8 DevKit. + + +{{% notice Note %}} +If you are using the provided `.pte` file, you can skip this section. +{{% /notice %}} + +## Start your container + +Ensure you are inside your container and load up the Executorch environment: + +```bash +# Inside Docker container +source ~/executorch-venv/bin/activate +source $ET_HOME/setup_arm_env.sh +``` + +Verify the tools: +```bash +vela --version +python3 -c "from executorch.exir import to_edge; print('ExecuTorch OK')" +``` + +## Download the model files + +From inside your container, create the model and output directories: + +```bash +# Create directories (use full paths, not ~) +mkdir -p /home/developer/models +mkdir -p /home/developer/output +``` + +Download the model script and the training script: + +```bash +curl -L -o /home/developer/models/mnist_model.py https://raw.githubusercontent.com/arm-education/alif-ethos-u85-npu-mnist/main/mnist_model.py +curl -L -o /home/developer/models/train_mnist.py https://raw.githubusercontent.com/arm-education/alif-ethos-u85-npu-mnist/main/train_mnist.py +``` + +The model script (`mnist_model.py`) defines a small convolutional network for 28 x 28 grayscale MNIST images. It has two convolution blocks followed by two fully connected layers: +```python +def __init__(self): + super().__init__() + self.conv1 = nn.Conv2d(1, 16, kernel_size=3, stride=1, padding=1) + self.relu1 = nn.ReLU() + self.pool1 = nn.MaxPool2d(kernel_size=2, stride=2) + self.conv2 = nn.Conv2d(16, 32, kernel_size=3, stride=1, padding=1) + self.relu2 = nn.ReLU() + self.pool2 = nn.MaxPool2d(kernel_size=2, stride=2) + # Fully Connected Layers + self.fc1 = nn.Linear(32 * 7 * 7, 64) + self.relu3 = nn.ReLU() + self.fc2 = nn.Linear(64, 10) +``` +The convolution layers extract image features such as strokes, edges, and digit shapes. Each output channel from a convolution is called a **feature map**. +Layer 1 (`fc1`) takes in 32 feature maps of size 7 x 7 each, and reduces those values to 64 features. +The final layer (`fc2`) then maps those 64 features to 10 digit scores (0 to 9). + +The same file also exposes the objects expected by the ExecuTorch ahead-of-time compiler: +```python +ModelUnderTest = MNISTModel() # PyTorch model to export +ModelInputs = (load_calibration_input(),) +``` + +The training script, however, downloads the [MNIST dataset](https://datasetsearch.research.google.com/search?query=mnist&docid=L2cvMTF0ZGh0cHRmMA%3D%3D), normalizes the images, trains the model, and saves the trained weights to: +```text +/home/developer/models/mnist_model.pth +``` + + +## Add calibration input + +The model is exported with int8 quantization, so the compiler needs a representative MNIST input during export. +This input is not the image that runs on the board. It is only used to set quantization ranges when generating the `.pte` file. + +Download the calibration sample: +```bash +curl -L -o /home/developer/output/sample_one.pt https://raw.githubusercontent.com/arm-education/alif-ethos-u85-npu-mnist/main/sample_one.pt +``` + +The file downloaded contains one normalized MNIST-style grayscale image, saved as a PyTorch tensor (`.pt`) with shape `[1, 1, 28, 28]` and type `float32`. +This allows `mnist_model.py` to load the exact format it needs without any image preprocessing step. + +For production-quality quantization, you would normally calibrate with a larger and more diverse set of representative inputs. This Learning Path uses one sample to keep the export flow small and reproducible. + +## Train the model + +Run the training script: + +```bash +cd /home/developer/models +python3 train_mnist.py --out /home/developer/models/mnist_model.pth --epochs 3 +``` + +The output shows the test accuracy after each epoch: + +```output +epoch=1 test_loss=... test_acc=... +epoch=2 test_loss=... test_acc=... +epoch=3 test_loss=... test_acc=... +saved /home/developer/models/mnist_model.pth +``` + +Verify the checkpoint: + +```bash +ls -lh /home/developer/models/mnist_model.pth +``` + +## Export to ExecuTorch + +Export the trained model to `.pte` format for Ethos-U85: + +```bash +cd $ET_HOME +MNIST_LOAD_CHECKPOINT=1 python3 -m examples.arm.aot_arm_compiler --model_name=/home/developer/models/mnist_model.py --delegate --quantize --target=ethos-u85-256 --system_config=Ethos_U85_SYS_DRAM_Mid --memory_mode=Shared_Sram --output=/home/developer/output/mnist_ethos_u85.pte +``` + +Argument definitions: +- `MNIST_LOAD_CHECKPOINT=1`: loads the trained weights from /home/developer/models/mnist_model.pth. +- `--model_name`: points to the PyTorch model definition. +- `--delegate`: partitions supported operators for execution on the Ethos-U NPU. +- `--quantize`: converts the model for int8 inference. +- `--target=ethos-u85-256`: targets an Ethos-U85 configuration with 256 MACs per cycle. +- `--system_config` and `--memory_mode`: selects the Vela memory configuration used when compiling the NPU command stream. +- `--output`: writes the exported ExecuTorch program. + +{{% notice Important %}} +Use full paths such as `/home/developer/models/mnist_model.py`. Do not use `~` in the export command. +{{% /notice %}} + +## Build ExecuTorch static libraries + +The firmware links against ExecuTorch runtime libraries. Build the bare-metal Cortex-M libraries inside the container: + +```bash +cd $ET_HOME +source ~/executorch-venv/bin/activate +rm -rf arm_test/cmake-out +bash backends/arm/scripts/build_executorch.sh +``` + +This step can take several minutes. + +When the build finishes, list the generated static libraries: + +```bash +find arm_test/cmake-out -type f -name "*.a" | sort +``` + +You should see libraries including: + +```output +libexecutorch.a +libexecutorch_core.a +libexecutorch_delegate_ethos_u.a +libcortex_m_ops_lib.a +libcmsis-nn.a +``` + +## Package headers and libraries + +Bundle the ExecuTorch headers and libraries into `et_bundle.tar.gz`: + +```bash +rm -rf /home/developer/output/et_bundle +mkdir -p /home/developer/output/et_bundle +cp -a arm_test/cmake-out/include /home/developer/output/et_bundle/ +cp -a arm_test/cmake-out/lib /home/developer/output/et_bundle/ +CMSIS_NN_LIB=$(find arm_test/cmake-out -type f -name "libcmsis-nn.a" | head -n 1) +cp "$CMSIS_NN_LIB" /home/developer/output/et_bundle/lib/ +tar -C /home/developer/output -czf /home/developer/output/et_bundle.tar.gz et_bundle +``` + +Because `/home/developer/output` is mounted from your host machine, `et_bundle.tar.gz` is now available on your host at: + +```text +~/mnist_alif/executorch-alif/output/et_bundle.tar.gz +``` + +You may now exit Docker: +```bash +exit +``` \ No newline at end of file diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/5-prepare-firmware-artifacts.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/5-prepare-firmware-artifacts.md new file mode 100644 index 0000000000..e0e39f1585 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/5-prepare-firmware-artifacts.md @@ -0,0 +1,113 @@ +--- +title: Prepare firmware artifacts +weight: 6 +layout: learningpathall +--- + +## Overview + +This section prepares the generated model and the ExecuTorch libraries needed for the project. + +## Prerequisites +The firmware project needs two artifacts: + +- `mnist_ethos_u85.pte`: the ExecuTorch model compiled for Ethos-U85 +- `et_bundle.tar.gz`: ExecuTorch headers and static libraries for the bare-metal Cortex-M build + +If you completed the optional export section, these files are already in `~/mnist_alif/executorch-alif/output/` + + + +If you skipped the optional export section, create the output directory and download the provided artifacts: + +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash" >}} +mkdir -p ~/mnist_alif/executorch-alif/output +cd ~/mnist_alif/executorch-alif/output +curl -L -o et_bundle.tar.gz https://raw.githubusercontent.com/arm-education/alif-ethos-u85-npu-mnist/main/et_bundle.tar.gz +curl -L -o mnist_ethos_u85.pte https://raw.githubusercontent.com/arm-education/alif-ethos-u85-npu-mnist/main/mnist_ethos_u85.pte + {{< /tab >}} + + {{< tab header="Windows (PowerShell)" language="powershell" >}} +New-Item -ItemType Directory -Force -Path "$HOME\mnist_alif\executorch-alif\output" +cd "$HOME\mnist_alif\executorch-alif\output" +curl.exe -L -o et_bundle.tar.gz https://raw.githubusercontent.com/arm-education/alif-ethos-u85-npu-mnist/main/et_bundle.tar.gz +curl.exe -L -o mnist_ethos_u85.pte https://raw.githubusercontent.com/arm-education/alif-ethos-u85-npu-mnist/main/mnist_ethos_u85.pte + {{< /tab >}} +{{< /tabpane >}} + +Verify all artifacts are present: + +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash" >}} +ls -lh ~/mnist_alif/executorch-alif/output/mnist_ethos_u85.pte +ls -lh ~/mnist_alif/executorch-alif/output/et_bundle.tar.gz + {{< /tab >}} + + {{< tab header="Windows (PowerShell)" language="powershell" >}} +Get-Item ~\mnist_alif\executorch-alif\output\mnist_ethos_u85.pte +Get-Item ~\mnist_alif\executorch-alif\output\et_bundle.tar.gz + {{< /tab >}} +{{< /tabpane >}} + +## Convert the model to a C header + +The firmware embeds the `.pte` model as a byte array in flash memory. Use `xxd` to generate a C header: + +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash" >}} +cd ~/mnist_alif/executorch-alif/output +xxd -i mnist_ethos_u85.pte > mnist_model_data.h + {{< /tab >}} + +{{< tab header="Windows (PowerShell)" language="powershell" >}} +cd ~\mnist_alif\executorch-alif\output +& "$env:ProgramFiles\Git\usr\bin\xxd.exe" -i mnist_ethos_u85.pte | Set-Content -Encoding ascii mnist_model_data.h +{{< /tab >}} +{{< /tabpane >}} + +Open the generated header (`mnist_model_data.h`) and change the first array declaration to this: + +```c +#include +const uint8_t __attribute__((aligned(16))) mnist_ethos_u85_pte[] = { +``` + +{{% notice Important %}} +The `aligned(16)` attribute is required because the Ethos-U85 needs the Vela command stream data aligned to 16 bytes. Without it, the NPU driver will report an alignment error at runtime. +{{% /notice %}} + +## Extract the Executorch bundle + +Extract the ExecuTorch headers (from `et_bundle.tar.gz`) into the VS Code template project: + +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash" >}} +cd ~/mnist_alif/alif_vscode-template +mkdir -p third_party/executorch +tar -C third_party/executorch -xzf ~/mnist_alif/executorch-alif/output/et_bundle.tar.gz + {{< /tab >}} + + {{< tab header="Windows (PowerShell)" language="powershell" >}} +cd ~\mnist_alif\alif_vscode-template +New-Item -ItemType Directory -Force -Path .\third_party\executorch +tar -C .\third_party\executorch -xzf "$HOME\mnist_alif\executorch-alif\output\et_bundle.tar.gz" + {{< /tab >}} +{{< /tabpane >}} + +Verify the headers are in place: +```bash +ls third_party/executorch/et_bundle/include/executorch/ +``` +You should see `runtime/` and other directories. + +You are now ready to integrate the model into the VS Code project. + +## Summary +In this section, you prepared the following firmware inputs: + +- `mnist_model_data.h` contains the embedded ExecuTorch model +- `third_party/executorch/et_bundle/include` contains ExecuTorch headers +- `third_party/executorch/et_bundle/lib` contains the static libraries used by the firmware build + +The next section creates the MNIST firmware project by duplicating the Blinky example and replacing relevant files to fit our application. \ No newline at end of file diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/6-alif-cmsis-project.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/6-alif-cmsis-project.md new file mode 100644 index 0000000000..8b0e861f81 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/6-alif-cmsis-project.md @@ -0,0 +1,346 @@ +--- +title: Create Alif E8 CMSIS project +weight: 7 +layout: learningpathall +--- + +## Overview + +In this section, you duplicate the existing Blinky example to create a new CMSIS project called mnist_executorch, configured to include ExecuTorch libraries, the compiled model, and SEGGER RTT for debug output. + +## Duplicate the Blinky project + +Start by copying the working Blinky project as a template, and renaming the project file inside the new directory: + +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash" >}} +cd ~/mnist_alif/alif_vscode-template +cp -R blinky/ mnist_executorch +mv mnist_executorch/blinky.cproject.yml mnist_executorch/mnist_executorch.cproject.yml + {{< /tab >}} + + {{< tab header="Windows (PowerShell)" language="powershell" >}} +cd ~\mnist_alif\alif_vscode-template +Copy-Item .\blinky .\mnist_executorch -Recurse +Rename-Item .\mnist_executorch\blinky.cproject.yml mnist_executorch.cproject.yml + {{< /tab >}} +{{< /tabpane >}} + +Next, replace all internal references from `blinky` to `mnist_executorch`: +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash" >}} +perl -pi -e 's/\bblinky\b/mnist_executorch/g' $(grep -RIl "blinky" mnist_executorch) + {{< /tab >}} + + {{< tab header="Windows (PowerShell)" language="powershell" >}} +Get-ChildItem .\mnist_executorch -Recurse -File | Where-Object { Select-String -Path $_.FullName -Pattern '\bblinky\b' -Quiet } | ForEach-Object { $text = Get-Content $_.FullName -Raw; $text = $text -replace '\bblinky\b', 'mnist_executorch'; Set-Content $_.FullName $text -NoNewline } + {{< /tab >}} +{{< /tabpane >}} + +## Rename main.c to main.cpp + +ExecuTorch is a C++ library, so the source file needs a `.cpp` extension: +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash" >}} +mv mnist_executorch/main.c mnist_executorch/main.cpp + {{< /tab >}} + {{< tab header="Windows (PowerShell)" language="powershell" >}} +Rename-Item .\mnist_executorch\main.c main.cpp + {{< /tab >}} +{{< /tabpane >}} + +## Copy model assets + +Create an assets directory and copy the model header into the project: +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash" >}} +mkdir -p mnist_executorch/assets +cp ~/mnist_alif/executorch-alif/output/mnist_model_data.h mnist_executorch/assets/ + {{< /tab >}} + + {{< tab header="Windows (PowerShell)" language="powershell" >}} +New-Item -ItemType Directory -Force -Path .\mnist_executorch\assets +Copy-Item ~\mnist_alif\executorch-alif\output\mnist_model_data.h .\mnist_executorch\assets\ + {{< /tab >}} +{{< /tabpane >}} + +## Create the SEGGER RTT configuration + +RTT (Real-Time Transfer) works through the J-Link debug probe, reading and writing a memory buffer through the debug interface. It's faster than UART and doesn't require extra wiring. Within `mnist_executorch/`, create the configuration file `SEGGER_RTT_Conf.h`: + +```c +#ifndef SEGGER_RTT_CONF_H +#define SEGGER_RTT_CONF_H +#define SEGGER_RTT_MAX_NUM_UP_BUFFERS (1) +#define SEGGER_RTT_MAX_NUM_DOWN_BUFFERS (1) +#define SEGGER_RTT_BUFFER_SIZE_UP (4096) +#define SEGGER_RTT_BUFFER_SIZE_DOWN (16) +#define SEGGER_RTT_MODE_DEFAULT SEGGER_RTT_MODE_NO_BLOCK_SKIP +#define SEGGER_RTT_PRINTF_BUFFER_SIZE (256) +#endif +``` + +## Update the solution file + +The `alif.csolution.yml` file describes the whole CMSIS workspace: available projects, targets, devices, packs, and flash settings. + +Open it and make the following changes. + +Update the `created-for` field to match your CMSIS Toolbox version: + +```yaml + created-for: CMSIS-Toolbox@2.12.0 +``` + +Add the required packs under the `packs:` section: + +```yaml + packs: + - pack: AlifSemiconductor::Ensemble@2.0.4 + - pack: ARM::CMSIS@6.0.0 + - pack: ARM::CMSIS-Compiler@2.1.0 + - pack: Keil::MDK-Middleware@8.2.0 + - pack: ARM::ethos-u-core-driver +``` + +Add a `target-set` block to the `E8-HP` target type so the Security Toolkit knows which binary to flash. Find the `type: E8-HP` section and add: + +```yaml + - type: E8-HP + device: Alif Semiconductor::AE822FA0E5597LS0:M55_HP + board: Alif Semiconductor::DevKit-E8 + define: + - "CORE_M55_HP" + target-set: + - set: + images: + - project-context: mnist_executorch.debug +``` + +For all other target types (E7-HE, E7-HP, E1C-HE, E8-HE), add a `target-set` pointing to `blinky.debug` instead. + +Add `mnist_executorch` to the projects list. It's at the bottom of the file: + +```yaml + projects: + - project: blinky/blinky.cproject.yml + - project: hello/hello.cproject.yml + - project: hello_rtt/hello_rtt.cproject.yml + - project: mnist_executorch/mnist_executorch.cproject.yml +``` + +## Configure the project file + +The `mnist_executorch/mnist_executorch.cproject.yml` file describes our MNIST application: its source files, components, include paths, and build options. +Locate this file and replace the contents with the following configuration: + +```yaml +# yaml-language-server: $schema=https://raw.githubusercontent.com/Open-CMSIS-Pack/devtools/tools/projmgr/2.6.0/tools/projmgr/schemas/cproject.schema.json +project: + groups: + - group: App + files: + - file: main.cpp + - file: executorch_runner.cpp + - file: executorch_runner.h + - file: SEGGER_RTT_Conf.h + - file: assets/mnist_model_data.h + + - group: SEGGER_RTT + files: + - file: ../libs/SEGGER_RTT_V796h/RTT/SEGGER_RTT.c + - file: ../libs/SEGGER_RTT_V796h/RTT/SEGGER_RTT_printf.c + - file: ../libs/SEGGER_RTT_V796h/Syscalls/SEGGER_RTT_Syscalls_GCC.c + + output: + base-name: $Project$ + type: + - elf + - bin + + layers: + - layer: ../device/ensemble/alif-ensemble.clayer.yml + + packs: + - pack: AlifSemiconductor::Ensemble + - pack: ARM::ethos-u-core-driver + + components: + - component: ARM::Machine Learning:NPU Support:Ethos-U Driver&Generic U85 + + define: + - C10_USING_CUSTOM_GENERATED_MACROS + - ETHOSU + - ETHOSU85 + + add-path: + - . + - assets + - ../libs/SEGGER_RTT_V796h/RTT + - ../third_party/executorch/et_bundle/include + - ../third_party/executorch/et_bundle/include/executorch/runtime/core/portable_type/c10 + + misc: + - for-compiler: GCC + C-CPP: + - -std=c++17 + - -fno-rtti + - -fno-exceptions + - -DETHOSU_ARCH=u85 + - -DETHOSU_MACS=256 + - -DETHOSU_MODEL=1 + Link: + - -L/absolute/path/to/third_party/executorch/et_bundle/lib + - -Wl,--whole-archive + - -lexecutorch + - -lexecutorch_core + - -lexecutorch_delegate_ethos_u + - -lcortex_m_ops_lib + - -Wl,--no-whole-archive + - -Wl,--start-group + - -lextension_runner_util + - -lcortex_m_kernels + - -lportable_ops_lib + - -lportable_kernels + - -lquantized_ops_lib + - -lquantized_kernels + - -lkernels_util_all_deps + - -lcmsis-nn + - -lflatccrt + - -Wl,--end-group +``` + +{{% notice Warning %}} +Update the `-L` path to match the absolute path to your `third_party/executorch/et_bundle/lib` directory. For example: `-L/Users/username/mnist_alif/alif_vscode-template/third_party/executorch/et_bundle/lib`. +{{% /notice %}} + +There are several important details in this configuration: + +- **`--whole-archive`** is required for `libexecutorch`, `libexecutorch_core`, `libexecutorch_delegate_ethos_u`, and `libcortex_m_ops_lib`. These libraries contain static registration constructors (for operator registration and PAL symbols) that the linker would otherwise discard as unused. +- **Don't add** `portable_ops_lib` or `quantized_ops_lib` to `--whole-archive`. They are large and will overflow the microcontroller's ITCM/MRAM. +- **`--start-group`/`--end-group`** resolves circular dependencies among the remaining libraries. +- **`C10_USING_CUSTOM_GENERATED_MACROS`** tells ExecuTorch to skip looking for a `cmake_macros.h` header that doesn't exist in the bare-metal build. +- The **c10 include path** provides the tensor type definitions that ExecuTorch's headers depend on. + +## Add the source files + +The application code is split across three source files: + +- `main.cpp` is the firmware entry point. It initializes the board, loads the embedded model from `mnist_model_data.h`, passes the MNIST input image to the runner, and prints the predicted digit and output scores. +- `executorch_runner.h` declares the small C-style interface used by `main.cpp` to initialize ExecuTorch and run inference. +- `executorch_runner.cpp` implements the ExecuTorch integration. It initializes the Ethos-U NPU driver, loads the `.pte` program, prepares input and output tensors, allocates runtime buffers, and executes the model. + +Run these commands to download the source files and place them in `mnist_executorch/`: + +```bash +cd ~/mnist_alif/alif_vscode-template/mnist_executorch +curl -o main.cpp https://raw.githubusercontent.com/arm-education/alif-ethos-u85-npu-mnist/refs/heads/main/mnist_executorch/main.cpp +curl -o executorch_runner.cpp https://raw.githubusercontent.com/arm-education/alif-ethos-u85-npu-mnist/refs/heads/main/mnist_executorch/executorch_runner.cpp +curl -o executorch_runner.h https://raw.githubusercontent.com/arm-education/alif-ethos-u85-npu-mnist/refs/heads/main/mnist_executorch/executorch_runner.h +``` +After downloading the files, open them in VS Code and briefly review how `main.cpp` calls into `executorch_runner.cpp` to initialize ExecuTorch, pass in the MNIST input tensor, and read back the model output. + +## Update the linker script + +Open: + +```text +device/ensemble/RTE/Device/AE822FA0E5597LS0_M55_HP/linker_gnu_mram.ld.src +``` + +On Cortex-M systems, the **DTCM (Data Tightly Coupled Memory)** is a small, fast RAM region connected closely to the CPU. By default, the linker script sends most `.bss` data to DTCM. + +ExecuTorch uses large memory pools, so this data must be routed to SRAM0 and SRAM1 instead. Without this change, the linker reports that `.bss` does not fit in DTCM. + +Make the following changes. + +### Add SRAM1 to the zero-initialization table + +Find this block in `.zero.table`: + +```ld +#if __HAS_BULK_SRAM + LONG (ADDR(.bss.at_sram0)) + LONG (SIZEOF(.bss.at_sram0)/4) +#endif +``` + +Replace it with: + +```ld +#if __HAS_BULK_SRAM + LONG (ADDR(.bss.at_sram0)) + LONG (SIZEOF(.bss.at_sram0)/4) + LONG (ADDR(.bss.at_sram1)) + LONG (SIZEOF(.bss.at_sram1)/4) +#endif +``` + +This tells the startup code to clear both SRAM0 and SRAM1 `.bss` sections before `main()` runs. + +### Add GOT sections to `.data.at_dtcm` + +Find this part of `.data.at_dtcm`: + +```ld + KEEP(*(.jcr*)) + . = ALIGN(8); +``` + +Replace it with: + +```ld + KEEP(*(.jcr*)) + *(.got) + *(.got.plt) + . = ALIGN(8); +``` + +The ExecuTorch C++ libraries can use a Global Offset Table (GOT). These entries must be copied into RAM at startup. + +### Route ExecuTorch buffers to SRAM0 and SRAM1 + +Find this existing block: + +```ld +#if __HAS_BULK_SRAM + .bss.at_sram0 (NOLOAD) : ALIGN(8) + { + *(.bss.lcd_crop_and_interpolate_buf) + *(.bss.lcd_frame_buf) + *(.bss.camera_frame_buf) + *(.bss.camera_frame_bayer_to_rgb_buf) + } > SRAM0 +#endif +``` + +Replace it with: + +```ld +#if __HAS_BULK_SRAM + .bss.at_sram0 (NOLOAD) : ALIGN(8) + { + *(.bss.lcd_crop_and_interpolate_buf) + *(.bss.lcd_frame_buf) + *(.bss.camera_frame_buf) + *(.bss.camera_frame_bayer_to_rgb_buf) + *(.bss.at_sram0) + *(.bss.at_sram0.*) + } > SRAM0 + .bss.at_sram1 (NOLOAD) : ALIGN(8) + { + *(.bss.at_sram1) + *(.bss.at_sram1.*) + } > SRAM1 +#endif +``` + +This must appear before the generic `.bss` section. Otherwise, sections such as `.bss.at_sram0` and `.bss.at_sram1` are caught by `*(.bss.*)` and placed in DTCM. + +The project structure is ready. The next sections cover image preparation before you build and flash. + +## What you've learned and what's next + +You've created the mnist_executorch firmware project, configured CMSIS packs, and set up the linker to include ExecuTorch libraries with the correct archive flags. + +You've also added the application code that loads the model and runs inference on the Ethos-U85 NPU. \ No newline at end of file diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/7-load-test-image.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/7-load-test-image.md new file mode 100644 index 0000000000..bef7575f0b --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/7-load-test-image.md @@ -0,0 +1,148 @@ +--- +title: Load test sample +weight: 8 +layout: learningpathall +--- + +## Overview + +This section converts a handwritten digit image into a C header that the firmware can include at build time. + +The MNIST model expects one grayscale 28 x 28 image. The script in this section resizes the image, converts it to grayscale, scales pixel values to the range 0 to 127, and writes the result to `input_mnist.h`. + +## Set up Python image tools + +Create a Python virtual environment for image preprocessing: + +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash" >}} +cd ~/mnist_alif +python3 -m venv venv_image_prep +source venv_image_prep/bin/activate +python -m pip install --upgrade pip +python -m pip install numpy pillow + {{< /tab >}} + + {{< tab header="Windows (PowerShell)" language="powershell" >}} +cd ~\mnist_alif +py -m venv venv_image_prep +.\venv_image_prep\Scripts\Activate.ps1 +python -m pip install --upgrade pip +python -m pip install numpy pillow + {{< /tab >}} +{{< /tabpane >}} + +## Save your image + +Create a directory for the input image and preprocessing script: + +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash" >}} +mkdir -p ~/mnist_alif/image +cd ~/mnist_alif/image + {{< /tab >}} + + {{< tab header="Windows (PowerShell)" language="powershell" >}} +New-Item -ItemType Directory -Force -Path ~\mnist_alif\image +cd ~\mnist_alif\image + {{< /tab >}} +{{< /tabpane >}} + +Place a PNG or JPEG image of a handwritten digit in this directory and name it: + +```text +mnist_image.jpg +``` + +{{% notice Note %}} +Use a simple, centered, high-contrast digit image. The script supports black-on-white and white-on-black images, and automatically converts black-on-white images to MNIST-style white-on-black format. + + +Feel free to use any of the images within this dataset: https://huggingface.co/datasets/ylecun/mnist. +{{% /notice %}} + +## Download the preprocessing script + +Download `prepare_mnist_image.py` into the image directory: + +```bash +cd ~/mnist_alif/image +curl -o prepare_mnist_image.py https://raw.githubusercontent.com/arm-education/alif-ethos-u85-npu-mnist/main/prepare_mnist_image.py +``` + +The script converts the image to 28 x 28 grayscale pixels, scales each pixel into the int8 range used by the firmware input, then writes the values as a C array: +```python +pixels = np.clip(np.rint(pixels * 127.0 / 255.0), 0, 127).astype(np.int8) +flat = pixels.reshape(-1) +``` +The generated header will contain a total of 784 values, one for each pixel in the 28 x 28 input image. + +## Generate the input header + +Run the preprocessing script: + +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash" >}} +cd ~/mnist_alif/image +python prepare_mnist_image.py mnist_image.jpg --output input_mnist.h + {{< /tab >}} + + {{< tab header="Windows (PowerShell)" language="powershell" >}} +cd ~\mnist_alif\image +python .\prepare_mnist_image.py .\mnist_image.jpg --output .\input_mnist.h + {{< /tab >}} +{{< /tabpane >}} + +Verify the generated file: + +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash" >}} +ls -lh input_mnist.h +head -n 8 input_mnist.h + {{< /tab >}} + + {{< tab header="Windows (PowerShell)" language="powershell" >}} +Get-Item .\input_mnist.h +Get-Content .\input_mnist.h -TotalCount 8 + {{< /tab >}} +{{< /tabpane >}} + +Open `input_mnist.h` and look at the generated `input_mnist` array. This is the numeric pixel data that `main.cpp` passes into the ExecuTorch runner. + +## Copy the header into the firmware project + +Copy `input_mnist.h` into the application assets directory: + +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash" >}} +cp ~/mnist_alif/image/input_mnist.h ~/mnist_alif/alif_vscode-template/mnist_executorch/assets/ + {{< /tab >}} + + {{< tab header="Windows (PowerShell)" language="powershell" >}} +Copy-Item "$HOME\mnist_alif\image\input_mnist.h" "$HOME\mnist_alif\alif_vscode-template\mnist_executorch\assets\" + {{< /tab >}} +{{< /tabpane >}} + +Verify that both firmware assets are present: + +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash" >}} +ls -lh ~/mnist_alif/alif_vscode-template/mnist_executorch/assets/ + {{< /tab >}} + + {{< tab header="Windows (PowerShell)" language="powershell" >}} +Get-ChildItem "$HOME\mnist_alif\alif_vscode-template\mnist_executorch\assets" + {{< /tab >}} +{{< /tabpane >}} + +You should see: + +```output +input_mnist.h +mnist_model_data.h +``` + +## Summary + +You converted a test digit image into `input_mnist.h` and copied it into the firmware project. The next section builds and flashes the application to the Alif Ensemble E8 DevKit. + diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/8-flash-and-run.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/8-flash-and-run.md new file mode 100644 index 0000000000..1c1df6847d --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/8-flash-and-run.md @@ -0,0 +1,109 @@ +--- +title: Flash and run on hardware +weight: 9 +layout: learningpathall +--- + +## Overview + +This section covers programming the Alif Ensemble E8 DevKit using VS Code CMSIS Extensions, viewing debug output via SEGGER RTT, and verifying successful deployment. + +## Build the project using VS Code CMSIS Extension + +Firstly, clear any cached build files present from previous runs (eg: Blinky project). CMSIS Toolbox caches aggressively and won’t pick up YAML configuration changes unless you clean first: + +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash" >}} +cd ~/mnist_alif/alif_vscode-template +rm -rf tmp/ out/ + {{< /tab >}} + + {{< tab header="Windows (PowerShell)" language="powershell" >}} +cd "$HOME\mnist_alif\alif_vscode-template" +Remove-Item -Recurse -Force .\tmp, .\out -ErrorAction SilentlyContinue + {{< /tab >}} +{{< /tabpane >}} + +Next, follow these steps to build: +- Select the **CMSIS** icon in the left sidebar. +- Select the gear icon. +- Set **Active Target** to **E8-HP**. +- Set **Active Project** to **mnist_executorch**. +- Select the **Build** hammer icon. + +A successful build prints a memory report similar to: + +```output +Memory region Used Size Region Size %age Used + ITCM: 149232 B 256 KB 56.93% + DTCM: 256 KB 256 KB 100.00% + SRAM0: 2576 KB 4 MB 62.89% + SRAM1: 2 MB 4 MB 50.00% + MRAM: 291008 B 2 MB 13.88% +``` + +## Flash the application + +Open the Command Palette (`Ctrl+Shift+P` on Windows/Linux or `Cmd+Shift+P` on macOS), select **Tasks: Run Task**, then select **Program with Security Toolkit (select COM port)**. Choose the DevKit's port when prompted. +Flashing takes about 30 seconds. + +## Start the J-Link RTT server + +Open a new terminal and start J-Link Commander. + +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash" >}} +JLinkExe -device AE822FA0E5597LS0_M55_HE -if SWD -speed 4000 + {{< /tab >}} + + {{< tab header="Windows (PowerShell)" language="powershell" >}} +& "C:\Program Files\SEGGER\JLink_V954\JLink.exe" -device AE822FA0E5597LS0_M55_HP -if SWD -speed 4000 + {{< /tab >}} +{{< /tabpane >}} + +At the `J-Link>` prompt, run: + +```text +connect +r +g +``` + +Leave this terminal open. It acts as the RTT server. + +## Start the RTT client + +Open another terminal and start the RTT client. + +{{< tabpane code=true >}} + {{< tab header="macOS / Linux" language="bash" >}} +JLinkRTTClient + {{< /tab >}} + + {{< tab header="Windows (PowerShell)" language="powershell" >}} +& "C:\Program Files\SEGGER\JLink_V954\JLinkRTTClient.exe" + {{< /tab >}} +{{< /tabpane >}} + +You should see output similar to: + +```output +ExecuTorch MNIST NPU Demo +Alif Ensemble E8 - Cortex-M55 HP + +Initializing SRAM0 power... +SRAM0 enabled successfully + +Loading model ... +Running inference... +Inference completed! +Predicted digit: ... +``` + +The predicted digit depends on the image you converted in the previous section. + +## Summary + +You have built and flashed a CMSIS-based firmware application that embeds an ExecuTorch `.pte` model, runs MNIST inference on the Ethos-U85 NPU, and reports the result through SEGGER RTT. + +You can extend this project by trying different MNIST images, retraining the model, or replacing MNIST with a different model, such as one trained to classify handwritten letters instead of digits. \ No newline at end of file diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/_index.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/_index.md new file mode 100644 index 0000000000..e65d727930 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/_index.md @@ -0,0 +1,85 @@ +--- +title: Run OCR (MNIST) on Alif E8 Ensemble DevKit using ExecuTorch and Ethos-U85 + +minutes_to_complete: 45 + +who_is_this_for: This is an introductory topic for embedded developers and ML engineers who want to run TinyML inference on physical hardware with Arm Ethos-U85 NPU acceleration. + +learning_objectives: + - Set up the Alif Ensemble E8 development kit for ML applications + - (Optional) Train, compile and export an MNIST PyTorch model to ExecuTorch .pte format using a Docker container + - Configure CMSIS project files, memory layout, and linker scripts for an ML workload on the Alif Ensemble E8 + - Build and flash firmware to the Alif Ensemble E8 DevKit + - Run MNIST digit classification on Ethos-U85 NPU + - Monitor inference results via SEGGER Real-Time Transfer (RTT) + +prerequisites: + - Experience with C/C++ and embedded development concepts + - Alif [Ensemble E8 Series Development Kit](https://alifsemi.com/ensemble-e8-series/) (contact [Alif Sales](https://alifsemi.com/support/sales-support/)) + - USB Type-C cable for programming + - A SEGGER J-Link debug probe (included in the DevKit) + - A development machine running Windows, Linux, or macOS with Visual Studio Code installed + +author_primary: Waheed Brown + +author: + - Waheed Brown + - Fidel Makatia Omusilibwa + - Kwashie Andoh + +### Tags +skilllevels: Introductory +subjects: ML +armips: + - Cortex-M + - Ethos-U + +operatingsystems: + - Linux + - macOS + - Windows + +tools_software_languages: + - C + - CMSIS + - TensorFlow Lite + - SEGGER JLink + - SEGGER RTT + - GCC + - Arm Compiler + +further_reading: + - resource: + title: Introduction to TinyML on Arm using PyTorch and ExecuTorch + link: /learning-paths/embedded-and-microcontrollers/introduction-to-tinyml-on-arm/ + type: documentation + - resource: + title: Run image classification on an Alif Ensemble E8 DevKit using ExecuTorch and Ethos-U85 + link: /learning-paths/embedded-and-microcontrollers/alif-image-classification/ + type: documentation + - resource: + title: Visualize Ethos-U NPU performance with ExecuTorch on Arm FVPs + link: /learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/ + type: documentation + - resource: + title: Alif Semiconductor Ensemble E8 Series + link: https://alifsemi.com/ensemble-e8-series/ + type: website + - resource: + title: Arm Ethos-U85 NPU + link: https://www.arm.com/products/silicon-ip-cpu/ethos/ethos-u85 + type: website + - resource: + title: Arm Developers Guide for Cortex-M Processors and Ethos-U NPU + link: https://developer.arm.com/documentation/109267/0101 + type: documentation + + + + +### FIXED, DO NOT MODIFY +# ================================================================================ +weight: 1 # _index.md always has weight of 1 to order correctly +layout: "learningpathall" # All files under learning paths have this same wrapper +learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content. +--- \ No newline at end of file diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/_next-steps.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/_next-steps.md new file mode 100644 index 0000000000..adccf10c38 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/_next-steps.md @@ -0,0 +1,8 @@ +--- +# ================================================================================ +# FIXED, DO NOT MODIFY THIS FILE +# ================================================================================ +weight: 21 # Set to always be larger than the content in this path to be at the end of the navigation. +title: "Next Steps" # Always the same, html page title. +layout: "learningpathall" # All files under learning paths have this same wrapper for Hugo processing. +--- \ No newline at end of file diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/alif-ensemble-e8-board-soc-highlighted.jpg b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/alif-ensemble-e8-board-soc-highlighted.jpg new file mode 100644 index 0000000000..fbd86f0a3a Binary files /dev/null and b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/alif-ensemble-e8-board-soc-highlighted.jpg differ diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/e8-board-connected.mp4 b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/e8-board-connected.mp4 new file mode 100644 index 0000000000..4fc714b9d7 Binary files /dev/null and b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/e8-board-connected.mp4 differ diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/macos-allow-setools.jpg b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/macos-allow-setools.jpg new file mode 100644 index 0000000000..d04af386aa Binary files /dev/null and b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/macos-allow-setools.jpg differ diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/macos-not-opened-warning.jpg b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/macos-not-opened-warning.jpg new file mode 100644 index 0000000000..565f953577 Binary files /dev/null and b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/macos-not-opened-warning.jpg differ diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/prg-usb-port.png b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/prg-usb-port.png new file mode 100644 index 0000000000..9f896d2ffc Binary files /dev/null and b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-alif/prg-usb-port.png differ