diff --git a/assets/contributors.csv b/assets/contributors.csv index 8de0eb0454..e871b8fcb1 100644 --- a/assets/contributors.csv +++ b/assets/contributors.csv @@ -91,3 +91,4 @@ Gian Marco Iodice,Arm,,,, Aude Vuilliomenet,Arm,,,, Andrew Kilroy,Arm,,,, Peter Harris,Arm,,,, +Waheed Brown,Arm,https://github.com/armwaheed,https://www.linkedin.com/in/waheedbrown/,, diff --git a/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/1-overview.md b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/1-overview.md new file mode 100644 index 0000000000..cbfad5ce57 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/1-overview.md @@ -0,0 +1,25 @@ +--- +title: Overview +weight: 2 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Visualizing ML on Embedded Devices + +Selecting the best hardware for machine learning (ML) models depends on effective tools. You can visualize ML performance early in the development cycle by using Arm [Fixed Virtual Platforms](https://developer.arm.com/Tools%20and%20Software/Fixed%20Virtual%20Platforms) (FVPs). + +## TinyML + +This Learning Path uses TinyML. TinyML is machine learning tailored to function on devices with limited resources, constrained memory, low power, and fewer processing capabilities. + +For a learning path focused on creating and deploying your own TinyML models, please see [Introduction to TinyML on Arm using PyTorch and ExecuTorch](/learning-paths/embedded-and-microcontrollers/introduction-to-tinyml-on-arm/) + +## Benefits and applications + +New products, like Arm's [Ethos-U85](https://www.arm.com/products/silicon-ip-cpu/ethos/ethos-u85) NPU are available on FVPs earlier than on physical devices. FVPs also have a graphical user iterface (GUI), which is useful for for ML performance visualization due to: +- visual confirmation that your ML model is running on the desired device, +- clearly indicated instruction counts, +- confirmation of total execution time and +- visually appealing output for prototypes and demos. diff --git a/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/2-env-setup.md b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/2-env-setup.md new file mode 100644 index 0000000000..ed0a605229 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/2-env-setup.md @@ -0,0 +1,78 @@ +--- +# User change +title: "Install ExecuTorch" + +weight: 3 + +# Do not modify these elements +layout: "learningpathall" +--- + +In this section, you will prepare a development environment to compile a machine learning model. + +## Introduction to ExecuTorch + +ExecuTorch is a lightweight runtime designed for efficient execution of PyTorch models on resource-constrained devices. It enables machine learning inference on embedded and edge platforms, making it well-suited for Arm-based hardware. Since Arm processors are widely used in mobile, IoT, and embedded applications, ExecuTorch leverages Arm's efficient CPU architectures to deliver optimized performance while maintaining low power consumption. By integrating with Arm's compute libraries, it ensures smooth execution of AI workloads on Arm-powered devices, from Cortex-M microcontrollers to Cortex-A application processors. + +## Install dependencies + +These instructions have been tested on Ubuntu 22.04, 24.04, and on Windows Subsystem for Linux (WSL). + +Python3 is required and comes installed with Ubuntu, but some additional packages are needed: + +```bash +sudo apt update +sudo apt install python-is-python3 python3-dev python3-venv gcc g++ make -y +``` + +## Create a virtual environment + +Create a Python virtual environment using `python venv`: + +```console +python3 -m venv $HOME/executorch-venv +source $HOME/executorch-venv/bin/activate +``` +The prompt of your terminal now has `(executorch)` as a prefix to indicate the virtual environment is active. + + +## Install Executorch + +From within the Python virtual environment, run the commands below to download the ExecuTorch repository and install the required packages: + +``` bash +cd $HOME +git clone https://github.com/pytorch/executorch.git +cd executorch +``` + +Run the commands below to set up the ExecuTorch internal dependencies: + +```bash +git submodule sync +git submodule update --init +./install_executorch.sh +``` + +{{% notice Note %}} +If you run into an issue of `buck` running in a stale environment, reset it by running the following instructions: + +```bash +ps aux | grep buck +pkill -f buck +``` +{{% /notice %}} + +After running the commands, `executorch` should be listed upon running `pip list`: + +```bash +pip list | grep executorch +``` + +```output +executorch 0.6.0a0+3eea1f1 +``` + +## Next Steps + +Proceed to the next section to learn about and set up the virtualized hardware. diff --git a/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/3-env-setup-fvp.md b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/3-env-setup-fvp.md new file mode 100644 index 0000000000..43662d7525 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/3-env-setup-fvp.md @@ -0,0 +1,52 @@ +--- +# User change +title: "Set up the Corstone-320 FVP on Linux" + +weight: 4 # 1 is first, 2 is second, etc. + +# Do not modify these elements +layout: "learningpathall" +--- + +In this section, you will run scripts to set up the Corstone-320 reference package. + +The Corstone-320 Fixed Virtual Platform (FVP) is a pre-silicon software development environment for Arm-based microcontrollers. It provides a virtual representation of hardware, allowing developers to test and optimize software before actual hardware is available. Designed for AI and machine learning workloads, it includes support for Arm's Ethos-U NPU and Cortex-M processors, making it ideal for embedded AI applications. The FVP accelerates development by enabling early software validation and performance tuning in a flexible, simulation-based environment. + +The Corstone reference system is provided free of charge, although you will have to accept the license in the next step. For more information on Corstone-320, check out the [official documentation](https://developer.arm.com/documentation/109761/0000?lang=en). + +## Corstone-320 FVP Setup for ExecuTorch + +{{% notice macOS %}} + +Setting up FVPs on MacOS requires some extra steps, outlined in GitHub repo [VPs-on-Mac](https://github.com/Arm-Examples/FVPs-on-Mac/). macOS users must do this first, before setting up the Corestone-320 FVP. + +{{% /notice %}} + +Navigate to the Arm examples directory in the ExecuTorch repository. Run the following command. + +```bash +cd $HOME/executorch/examples/arm +./setup.sh --i-agree-to-the-contained-eula +``` + +After the script has finished running, it prints a command to run to finalize the installation. This step adds the FVP executables to your system path. + +```bash +source $HOME/executorch/examples/arm/ethos-u-scratch/setup_path.sh +``` + +Test that the setup was successful by running the `run.sh` script for Ethos-U85, which is the target device for Corstone-320: + +{{% notice macOS %}} + +**Start Docker:** on macOS, FVPs run inside a Docker container. + +{{% /notice %}} + +```bash + ./examples/arm/run.sh --target=ethos-u85-256 +``` + +You will see a number of examples run on the FVP. + +This confirms the installation, so you can now proceed to the Learning Path [Build a Simple PyTorch Model](/learning-paths/embedded-and-microcontrollers/introduction-to-tinyml-on-arm/4-build-model/). \ No newline at end of file diff --git a/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/4-how-executorch-works.md b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/4-how-executorch-works.md new file mode 100644 index 0000000000..e10107404d --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/4-how-executorch-works.md @@ -0,0 +1,57 @@ +--- +# User change +title: "How ExecuTorch Works" + +weight: 5 # 1 is first, 2 is second, etc. + +# Do not modify these elements +layout: "learningpathall" +--- + +Source: [How ExecuTorch Works](https://docs.pytorch.org/executorch/stable/intro-how-it-works.html) (official PyTorch Documentation) +1. **Export the model:** + * Generate a Graph + * A graph is series of operators (ReLU, quantize, etc.) eligible for delegation to an accelerator + * Your goal is to identify operators for acceleration on the Ethos-U NPU +2. **Compile to ExecuTorch:** + * This is the ahead-of-time compiler + * This is why ExecuTorch inference is faster than PyTorch inference + * Delegate operators to an accelerator, like the Ethos-U NPU +3. **Run on targeted device:** + * Deploy the ML model to the Fixed Virtual Platform (FVP) or physical device + * Execute operators on the CPU and delegated operators on the Ethos-U NPU + +**Diagram of How ExecuTorch Works** +![How ExecuTorch works](./how-executorch-works-high-level.png) + +## Deploy a TinyML Model + +With your development environment set up, you can deploy a simple PyTorch model. + +This example deploys the [MobileNet V2](https://pytorch.org/hub/pytorch_vision_mobilenet_v2/) computer vision model. The model is a convolutional neural network (CNN) that extracts visual features from an image. It is used for image classification and object detection. + +The actual Python code for the MobileNet V2 model is in your local `executorch` repo: [executorch/examples/models/mobilenet_v2/model.py](https://github.com/pytorch/executorch/blob/main/examples/models/mobilenet_v2/model.py). You can deploy it using [run.sh](https://github.com/pytorch/executorch/blob/main/examples/arm/run.sh), just like you did in the previous step, with some extra parameters: + +{{% notice macOS %}} + +**Start Docker:** on macOS, FVPs run inside a Docker container. + +{{% /notice %}} + +```bash +./examples/arm/run.sh \ +--aot_arm_compiler_flags="--delegate --quantize --intermediates mv2_u85/ --debug --evaluate" \ +--output=mv2_u85 \ +--target=ethos-u85-128 \ +--model_name=mv2 +``` + +**Explanation of run.sh Parameters** +|run.sh Parameter|Meaning / Context| +|--------------|-----------------| +|--aot_arm_compiler_flags|Passes a string of compiler options to the ExecuTorch Ahead-of-Time (AOT) compiler| +|--delegate|Enables backend delegation| +|--quantize|Converts the floating-point model to int8 quantized format using post-training quantization
**Essential for running on NPUs**| +|--intermediates mv2_u85/|Directory where intermediate files (e.g., TOSA, YAMLs, debug graphs) will be saved
Useful output files for **manual debugging**| +|--debug|Verbose debugging logging| +|--evaluate|Validates model output, provides timing estimates| diff --git a/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/5-configure-fvp-gui.md b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/5-configure-fvp-gui.md new file mode 100644 index 0000000000..7da0db60cf --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/5-configure-fvp-gui.md @@ -0,0 +1,94 @@ +--- +# User change +title: "Configure the FVP GUI (optional)" + +weight: 6 # 1 is first, 2 is second, etc. + +# Do not modify these elements +layout: "learningpathall" +--- + +## Find your IP address + +Note down your computer's IP address: +```bash +ip addr show +``` +To help you, here are some common WiFi interface names on Linux: +|Interface Name|Meaning / Context| +|--------------|-----------------| +|wlan0|Legacy name (common on older systems)| +|wlp2s0, wlp3s0|Predictable network naming scheme (modern systems)| +|wlx|Some systems name interfaces after MAC addresses| +|wifi0, ath0|Very rare, specific to certain drivers (e.g., Atheros)| + +{{% notice macOS %}} + +Note down your `en0` IP address (or whichever network adapter is active): + +```bash +ipconfig getifaddr en0 # Returns your Mac's WiFi IP address +``` + +{{% /notice %}} + +## Enable the FVP's GUI + +Edit the following parameters in [run_fvp.sh](https://github.com/pytorch/executorch/blob/d5fe5faadb8a46375d925b18827493cd65ec84ce/backends/arm/scripts/run_fvp.sh#L97-L102), to enable the Mobilenet V2 output on the FVP's GUI: + +```bash +-C mps4_board.subsystem.ethosu.num_macs=${num_macs} \ +-C mps4_board.visualisation.disable-visualisation=1 \ +-C vis_hdlcd.disable_visualisation=1 \ +-C mps4_board.telnetterminal0.start_telnet=0 \ +-C mps4_board.uart0.out_file='-' \ +-C mps4_board.uart0.shutdown_on_eot=1 \ +``` + +- Change `mps4_board.visualisation.disable-visualisation` to equal `0` +- Change `vis_hdlcd.disable_visualisation` to equal `0` +- Enter a `--display-ip` parameter and set it to your computer's IP address + +```bash +-C mps4_board.subsystem.ethosu.num_macs=${num_macs} \ +-C mps4_board.visualisation.disable-visualisation=0 \ +-C vis_hdlcd.disable_visualisation=0 \ +-C mps4_board.telnetterminal0.start_telnet=0 \ +-C mps4_board.uart0.out_file='-' \ +-C mps4_board.uart0.shutdown_on_eot=1 \ +--display-ip \ +``` + +## Deploy the model + +{{% notice macOS %}} + +- **Start Docker:** on macOS, FVPs run inside a Docker container. + + **Do not use Colima Docker!** + + - Make sure to use an [official version of Docker](https://www.docker.com/products/docker-desktop/) and not a free version like the [Colima](https://github.com/abiosoft/colima?tab=readme-ov-file) Docker container runtime + - `run.sh` assumes Docker Desktop style networking (`host.docker.internal`) which breaks with Colima + - Colima then breaks the FVP GUI + +- **Start XQuartz:** on macOS, the FVP GUI runs using XQuartz. + + Start the xquartz.app and then configure XQuartz so that the FVP will accept connections from your Mac and localhost: + ```bash + xhost + + xhost + 127.0.0.1 # The Docker container seems to proxy through localhost + ``` +{{% /notice %}} + +Now run the Mobilenet V2 computer vision model, using [executorch/examples/arm/run.sh](https://github.com/pytorch/executorch/blob/main/examples/arm/run.sh): +```bash +./examples/arm/run.sh \ +--aot_arm_compiler_flags="--delegate --quantize --intermediates mv2_u85/ --debug --evaluate" \ +--output=mv2_u85 \ +--target=ethos-u85-128 \ +--model_name=mv2 +``` + +Observe that the FVP loads the model file, compiles the PyTorch model to ExecuTorch `.pte` format and then shows an instruction count in the top right of the GUI: + +![Terminal and FVP output](./Terminal%20and%20FVP%20Output.jpg) diff --git a/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/6-evaluate-output.md b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/6-evaluate-output.md new file mode 100644 index 0000000000..b6567d743f --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/6-evaluate-output.md @@ -0,0 +1,163 @@ +--- +# User change +title: "Evaluate Ethos-U Performance" + +weight: 7 # 1 is first, 2 is second, etc. + +# Do not modify these elements +layout: "learningpathall" +--- + +## Observe Ahead-of-Time Compilation +- The below output snippet from [run.sh](https://github.com/pytorch/executorch/blob/main/examples/arm/run.sh) is how you can confirm ahead-of-time compilation +- Specifically you want to see that the original PyTorch model was converted to an ExecuTorch `.pte` file +- For the MobileNet V2 example, the compiled ExecuTorch file will be output as `mv2_arm_delegate_ethos-u85-128.pte` + +{{% notice Note %}} + +In the below sample outputs, the `executorch` directory path is indicated as `/path/to/executorch`. Your actual path will depend on where you cloned your local copy of the [executorch repo](https://github.com/pytorch/executorch/tree/main). + +{{% /notice %}} + +**Ahead-of-Time Compiler Start:** +```bash { output_lines = "1-4" } +-------------------------------------------------------------------------------- +Running e2e flow for model 'mv2' with flags '--delegate --quantize --delegate --quantize --intermediates mv2_u85/ --debug --evaluate' +-------------------------------------------------------------------------------- +CALL python3 -m examples.arm.aot_arm_compiler --model_name=mv2 --target=ethos-u85-128 --delegate --quantize --delegate --quantize --intermediates mv2_u85/ --debug --evaluate --intermediate=/path/to/executorch/mv2_u85 --output=/path/to/executorch/mv2_u85/mv2_arm_delegate_ethos-u85-128.pte --system_config=Ethos_U85_SYS_DRAM_Mid --memory_mode=Sram_Only +``` + +**.pte File Build Completion:** +```bash { output_lines = "1-3" } +PTE file saved as /path/to/executorch/mv2_u85/mv2_arm_delegate_ethos-u85-128.pte +pte_data_size: 3809584 /path/to/executorch/mv2_u85/mv2_arm_delegate_ethos-u85-128.pte +pte_file: /path/to/executorch/mv2_u85/mv2_arm_delegate_ethos-u85-128.pte +``` + +**Ethos-U Delegate Build Start:** +```bash{ output_lines = "1-5" } ++ backends/arm/scripts/build_executor_runner.sh --et_build_root=/path/to/executorch/arm_test --pte=/path/to/executorch/mv2_u85/mv2_arm_delegate_ethos-u85-128.pte --build_type=Release --target=ethos-u85-128 --system_config=Ethos_U85_SYS_DRAM_Mid --memory_mode=Sram_Only --extra_build_flags= --ethosu_tools_dir=/path/to/executorch/examples/arm/ethos-u-scratch +-------------------------------------------------------------------------------- +Build Arm Baremetal executor_runner for ethos-u85-128 with /path/to/executorch/mv2_u85/mv2_arm_delegate_ethos-u85-128.pte using Ethos_U85_SYS_DRAM_Mid Sram_Only to '/path/to/executorch/mv2_u85/mv2_arm_delegate_ethos-u85-128/cmake-out' +-------------------------------------------------------------------------------- +``` + +**Ethos-U Delegate Build Completion:** +```bash { output_lines = "1" } +[100%] Built target arm_executor_runner +``` + +## Observe Test Batch Performance +By default, `run.sh` (and the underlying ethos_u_runner) uses: +- A constant input tensor, usually filled with zeros, ones, or random-ish synthetic data +- Input shape matches MobileNet V2: typically `[1, 3, 224, 224]` (batch size 1, 3 RGB channels, 224×224 image) +- Input tensor size: `1 × 3 × 224 × 224 × 1 byte = 150528 bytes ≈ 147 KB` + ```bash { output_lines = "1" } + Input SRAM bandwidth = 15.49 MB/batch + ``` +- `Batch Inference time` gives you a single performance metric for the Ethos-U85 (versus other [Ethos-U NPUs](https://developer.arm.com/Processors#q=Ethos-U&aq=%40navigationhierarchiescategories%3D%3D%22Processor%20products%22%20AND%20%40navigationhierarchiescontenttype%3D%3D%22Product%20Information%22&numberOfResults=48)) + ```bash { output_lines = "1" } + Batch Inference time 4.94 ms, 202.34 inferences/s (batch size 1) + ``` + +**Test Batch Performance:** +```bash { output_lines = "1-34" } +Network summary for out +Accelerator configuration Ethos_U85_128 +System configuration Ethos_U85_SYS_DRAM_Mid +Memory mode Sram_Only +Accelerator clock 1000 MHz +Design peak SRAM bandwidth 29.80 GB/s + +Total SRAM used 5178.77 KiB + +CPU operators = 0 (0.0%) +NPU operators = 64 (100.0%) + +Average SRAM bandwidth 7.21 GB/s +Input SRAM bandwidth 15.49 MB/batch +Weight SRAM bandwidth 11.87 MB/batch +Output SRAM bandwidth 6.66 MB/batch +Total SRAM bandwidth 35.65 MB/batch +Total SRAM bandwidth per input 35.65 MB/inference (batch size 1) + +Neural network macs 300836992 MACs/batch + +Info: The numbers below are internal compiler estimates. +For performance numbers the compiled network should be run on an FVP Model or FPGA. + +Network Tops/s 0.12 Tops/s + +NPU cycles 4832315 cycles/batch +SRAM Access cycles 1168037 cycles/batch +DRAM Access cycles 0 cycles/batch +On-chip Flash Access cycles 0 cycles/batch +Off-chip Flash Access cycles 0 cycles/batch +Total cycles 4942076 cycles/batch + +Batch Inference time 4.94 ms, 202.34 inferences/s (batch size 1) +``` + +## Observe Operator Delegation +This output indicates which operators go to processors: +- **Ethos-U85 NPU:** `occurrences_in_delegated_graphs` +- **Cortex-M85 CPU:** `occurrences_in_non_delegated_graph` + +```bash { output_lines = "1-34" } +Total delegated subgraphs: 1 +Number of delegated nodes: 419 +Number of non-delegated nodes: 3 + +Delegation table: +╒════╤════════════════════════════════════════════════════╤═══════════════════════════════════╤═══════════════════════════════════════╕ +│ │ op_type │ occurrences_in_delegated_graphs │ occurrences_in_non_delegated_graphs │ +╞════╪════════════════════════════════════════════════════╪═══════════════════════════════════╪═══════════════════════════════════════╡ +│ 0 │ aten_add_tensor │ 10 │ 0 │ +├────┼────────────────────────────────────────────────────┼───────────────────────────────────┼───────────────────────────────────────┤ +│ 1 │ aten_clone_default │ 1 │ 0 │ +├────┼────────────────────────────────────────────────────┼───────────────────────────────────┼───────────────────────────────────────┤ +│ 2 │ aten_convolution_default │ 52 │ 0 │ +├────┼────────────────────────────────────────────────────┼───────────────────────────────────┼───────────────────────────────────────┤ +│ 3 │ aten_hardtanh_default │ 35 │ 0 │ +├────┼────────────────────────────────────────────────────┼───────────────────────────────────┼───────────────────────────────────────┤ +│ 4 │ aten_linear_default │ 1 │ 0 │ +├────┼────────────────────────────────────────────────────┼───────────────────────────────────┼───────────────────────────────────────┤ +│ 5 │ aten_mean_dim │ 1 │ 0 │ +├────┼────────────────────────────────────────────────────┼───────────────────────────────────┼───────────────────────────────────────┤ +│ 6 │ aten_view_copy_default │ 1 │ 0 │ +├────┼────────────────────────────────────────────────────┼───────────────────────────────────┼───────────────────────────────────────┤ +│ 7 │ cortex_m_dequantize_per_tensor_default │ 0 │ 1 │ +├────┼────────────────────────────────────────────────────┼───────────────────────────────────┼───────────────────────────────────────┤ +│ 8 │ cortex_m_quantize_per_tensor_default │ 0 │ 1 │ +├────┼────────────────────────────────────────────────────┼───────────────────────────────────┼───────────────────────────────────────┤ +│ 9 │ getitem │ 0 │ 1 │ +├────┼────────────────────────────────────────────────────┼───────────────────────────────────┼───────────────────────────────────────┤ +│ 10 │ quantized_decomposed_dequantize_per_tensor_default │ 217 │ 0 │ +├────┼────────────────────────────────────────────────────┼───────────────────────────────────┼───────────────────────────────────────┤ +│ 11 │ quantized_decomposed_quantize_per_tensor_default │ 101 │ 0 │ +├────┼────────────────────────────────────────────────────┼───────────────────────────────────┼───────────────────────────────────────┤ +│ 12 │ Total │ 419 │ 3 │ +╘════╧════════════════════════════════════════════════════╧═══════════════════════════════════╧═══════════════════════════════════════╛ +``` + +## Observe the Ethos-U Performance Monitoring Unit +This output shows Ethos-U performance, from the Performance Monitoring Unit (PMU) +```bash { output_lines = "1-7" } +I [executorch:arm_perf_monitor.cpp:180] Ethos-U PMU report: +I [executorch:arm_perf_monitor.cpp:181] ethosu_pmu_cycle_cntr : 4738932 +I [executorch:arm_perf_monitor.cpp:184] ethosu_pmu_cntr0 : 1447178 +I [executorch:arm_perf_monitor.cpp:184] ethosu_pmu_cntr1 : 420661 +I [executorch:arm_perf_monitor.cpp:184] ethosu_pmu_cntr2 : 0 +I [executorch:arm_perf_monitor.cpp:184] ethosu_pmu_cntr3 : 0 +I [executorch:arm_perf_monitor.cpp:184] ethosu_pmu_cntr4 : 130 +``` + +**Table of Ethos-U PMU Counters:** +|PMU Counter|Default Event Tracked|Description|Interpretation| +|-----------|---------------------|-----------|--------------| +|ethosu_pmu_cycle_cntr|Total NPU cycles|Counts the number of core clock cycles where the Ethos-U NPU was executing work.|High value = long runtime; use to compute throughput.| +|ethosu_pmu_cntr0|SRAM read data beats received(ETHOSU_PMU_SRAM_RD_DATA_BEAT_RECEIVED)|How many data beats (e.g., 64-bit words) the NPU read from local SRAM.|Indicates input + weight loading efficiency.| +|ethosu_pmu_cntr1|SRAM write data beats written(ETHOSU_PMU_SRAM_WR_DATA_BEAT_WRITTEN)|Number of data beats the NPU wrote back to SRAM (e.g., outputs or intermediate results).|Reflects output bandwidth usage.| +|ethosu_pmu_cntr2|External DRAM read beats(ETHOSU_PMU_EXT_RD_DATA_BEAT_RECEIVED)|Number of data beats read from off-chip memory (e.g., DRAM). Often 0 if Sram_Only is used.|If non-zero, may indicate cache misses or large model size.| +|ethosu_pmu_cntr3|External DRAM write beats(ETHOSU_PMU_EXT_WR_DATA_BEAT_WRITTEN)|Number of write data beats to external memory.|Helps detect offloading or insufficient SRAM.| +|ethosu_pmu_cntr4|Idle cycles(ETHOSU_PMU_NPU_IDLE)|Number of cycles where the NPU had no work scheduled (i.e., idle).|High idle count = possible pipeline stalls or bad scheduling.| diff --git a/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/Terminal and FVP Output.jpg b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/Terminal and FVP Output.jpg new file mode 100644 index 0000000000..7c6cde8116 Binary files /dev/null and b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/Terminal and FVP Output.jpg differ diff --git a/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/_index.md b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/_index.md new file mode 100644 index 0000000000..e57c5de281 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/_index.md @@ -0,0 +1,65 @@ +--- +title: Visualizing Ethos-U Performance on Arm FVPs + +minutes_to_complete: 120 + +who_is_this_for: This is an introductory topic for developers and data scientists new to Tiny Machine Learning (TinyML), who want to visualize ExecuTorch performance on a virtual device. + +learning_objectives: + - Identify suitable Arm-based devices for TinyML applications. + - Optionally, install Fixed Virtual Platforms (FVPs) on MacOS. + - Deploy a TinyML ExecuTorch model to a Corstone-320 FVP. + - Observe model execution on the FVP's graphical user interface (GUI). + +prerequisites: + - Basic knowledge of Machine Learning concepts + - A Linux or Mac computer + + +author: Waheed Brown + +### Tags +skilllevels: Introductory +subjects: ML +armips: + - Cortex-A + - Cortex-M + - Ethos-U + - Ethos-U85 + +operatingsystems: + - Linux + - macOS + +tools_software_languages: + - Arm Virtual Hardware + - Fixed Virtual Platform + - Python + - PyTorch + - ExecuTorch + - Arm Compute Library + - GCC + +further_reading: + - resource: + title: TinyML Brings AI to Smallest Arm Devices + link: https://newsroom.arm.com/blog/tinyml + type: blog + - resource: + title: Arm Machine Learning Resources + link: https://www.arm.com/developer-hub/embedded-and-microcontrollers/ml-solutions/getting-started + type: documentation + - resource: + title: Arm Developers Guide for Cortex-M Processors and Ethos-U NPU + link: https://developer.arm.com/documentation/109267/0101 + type: documentation + + + + +### FIXED, DO NOT MODIFY +# ================================================================================ +weight: 1 # _index.md always has weight of 1 to order correctly +layout: "learningpathall" # All files under learning paths have this same wrapper +learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content. +--- diff --git a/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/_next-steps.md b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/_next-steps.md new file mode 100644 index 0000000000..c3db0de5a2 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/_next-steps.md @@ -0,0 +1,8 @@ +--- +# ================================================================================ +# FIXED, DO NOT MODIFY THIS FILE +# ================================================================================ +weight: 21 # Set to always be larger than the content in this path to be at the end of the navigation. +title: "Next Steps" # Always the same, html page title. +layout: "learningpathall" # All files under learning paths have this same wrapper for Hugo processing. +--- diff --git a/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/how-executorch-works-high-level.png b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/how-executorch-works-high-level.png new file mode 100644 index 0000000000..58b7369d5f Binary files /dev/null and b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/how-executorch-works-high-level.png differ