Skip to content

Commit 9bc35b5

Browse files
tgonzalezorlandoarmtgonzalezorlandoarm
committed
Topo Template learning path: Deploy an ML app to heterogenous hardware
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
1 parent 47d3187 commit 9bc35b5

6 files changed

Lines changed: 458 additions & 0 deletions

File tree

‎content/learning-paths/cross-platform/deploy-ml-model-to-npu-with-topo/_index.md‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
---
2+
title: Deploy a machine learning model to an NPU-capable system with Topo
3+
4+
description: Use Topo to deploy a web application on Cortex-A that triggers a MobileNetV2 image classifier running as Cortex-M firmware with Ethos-U65 NPU acceleration.
5+
6+
minutes_to_complete: 60
7+
8+
who_is_this_for: This is an introductory topic for embedded, edge, and cloud software developers who want to deploy machine learning workloads to heterogeneous Arm-based Linux targets using Topo.
9+
10+
learning_objectives:
11+
- Explain how Topo deploys an application that spans Cortex-A, Cortex-M, and Ethos-U
12+
- Prepare an NXP FRDM i.MX 93 board for remoteproc-runtime and shared-memory inference
13+
- Clone and deploy the topo-imx93-npu-deployment template
14+
- Run image classification from a browser and verify that inference is executed by the Cortex-M33 firmware
15+
16+
prerequisites:
17+
- A host machine (x86 or Arm) with Linux, macOS, or Windows
18+
- An NXP FRDM i.MX 93 target board accessible over SSH with root access
19+
- Docker installed on the host and target. For installation steps, see [Install Docker](/install-guides/docker/).
20+
- lscpu installed on the target (pre-installed on most Linux distributions)
21+
- Topo installed on the host. For installation steps, see [Deploy containerized workloads to Arm-based Linux targets with Topo](/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/).
22+
- Basic familiarity with containers, SSH, and CLI tools
23+
24+
author: Tomas Agustin Gonzalez Orlando
25+
26+
### Tags
27+
skilllevels: Introductory
28+
subjects: Containers and Virtualization
29+
armips:
30+
- Cortex-A
31+
- Cortex-M
32+
- Ethos-U
33+
tools_software_languages:
34+
- Topo
35+
- Docker
36+
- SSH
37+
- ExecuTorch
38+
- remoteproc-runtime
39+
operatingsystems:
40+
- Linux
41+
- macOS
42+
- Windows
43+
44+
### Cross-platform metadata only
45+
shared_path: true
46+
shared_between:
47+
- servers-and-cloud-computing
48+
- laptops-and-desktops
49+
- embedded-and-microcontrollers
50+
51+
further_reading:
52+
- resource:
53+
title: Topo repository
54+
link: https://github.com/arm/topo
55+
type: documentation
56+
- resource:
57+
title: Topo template format
58+
link: https://github.com/arm/topo-template-format
59+
type: documentation
60+
- resource:
61+
title: Topo releases
62+
link: https://github.com/arm/topo/releases/latest
63+
type: website
64+
- resource:
65+
title: remoteproc-runtime
66+
link: https://github.com/arm/remoteproc-runtime
67+
type: documentation
68+
- resource:
69+
title: ExecuTorch
70+
link: https://docs.pytorch.org/executorch/stable/index.html
71+
type: documentation
72+
73+
### FIXED, DO NOT MODIFY
74+
# ================================================================================
75+
weight: 1 # _index.md always has weight of 1 to order correctly
76+
layout: "learningpathall" # All files under learning paths have this same wrapper
77+
learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content.
78+
---

‎content/learning-paths/cross-platform/deploy-ml-model-to-npu-with-topo/_next-steps.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
---
2+
# ================================================================================
3+
# FIXED, DO NOT MODIFY THIS FILE
4+
# ================================================================================
5+
weight: 21 # The weight controls the order of the pages. _index.md always has weight 1.
6+
title: "Next Steps" # Always the same, html page title.
7+
layout: "learningpathall" # All files under learning paths have this same wrapper for Hugo processing.
8+
---

‎content/learning-paths/cross-platform/deploy-ml-model-to-npu-with-topo/deploy.md‎

Lines changed: 212 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,212 @@
1+
---
2+
title: Deploy the project
3+
weight: 3
4+
5+
### FIXED, DO NOT MODIFY
6+
layout: learningpathall
7+
---
8+
9+
## Prepare the target
10+
11+
Before deploying the Template, confirm that the FRDM i.MX 93 board is reachable from your host and that it's ready for deployment:
12+
13+
```bash
14+
topo health --target <user>@<target-ip>
15+
```
16+
Replace `<target-ip>` with the IP address or hostname of your board.
17+
18+
Resolve any errors before continuing.
19+
20+
The target section should include successful checks similar to:
21+
22+
```output
23+
Host
24+
----
25+
Topo: ✅ (topo)
26+
SSH: ✅ (ssh)
27+
Curl: ✅ (curl)
28+
Container Engine: ✅ (docker)
29+
30+
Target
31+
------
32+
Destination: ssh://<target-ip>
33+
Connectivity: ✅
34+
Container Engine: ✅ (docker)
35+
Remoteproc Runtime: ✅ (remoteproc-runtime)
36+
Remoteproc Shim: ✅ (containerd-shim-remoteproc-v1)
37+
Hardware Info: ✅ (lscpu)
38+
Subsystem Driver (remoteproc): ✅ (imx-rproc)
39+
```
40+
41+
If `remoteproc-runtime` is missing, install it with Topo:
42+
43+
```bash
44+
topo install remoteproc-runtime --target <user>@<target-ip>
45+
```
46+
47+
Run the health check again:
48+
49+
```bash
50+
topo health --target <user>@<target-ip>
51+
```
52+
53+
## Reserve memory in the device tree
54+
55+
The web application and Cortex-M33 firmware exchange data through reserved physical memory. The target device tree must reserve memory for the model/input buffer and for Ethos-U65. We are now goint to modify the device tree and reboot the target so that this modifications take place.
56+
57+
{{% notice Warning %}}
58+
Back up the board's original device tree before modifying it. The exact boot partition can differ between Linux images, so check the paths on your board before copying files.
59+
{{< /notice >}}
60+
61+
On your host, create a working directory and dump the live device tree from the target:
62+
63+
```bash
64+
mkdir -p devicetree
65+
ssh <user>@<target-ip> 'cat /sys/firmware/fdt' > devicetree/live.dtb
66+
dtc -I dtb -O dts -o devicetree/live.dts devicetree/live.dtb
67+
```
68+
69+
Open `devicetree/live.dts` in an editor.
70+
71+
Under `remoteproc-cm33`, add the CM33 power domain if it is not already present:
72+
73+
```dts
74+
power-domains = <0x61>;
75+
```
76+
77+
Under `reserved-memory`, add the model memory range:
78+
79+
```dts
80+
model@c0000000 {
81+
reg = <0x00 0xc0000000 0x00 0x400000>;
82+
no-map;
83+
};
84+
```
85+
86+
Update the Ethos-U reserved-memory node so it is reserved and not reusable:
87+
88+
```dts
89+
ethosu_region@A8000000 {
90+
compatible = "shared-dma-pool";
91+
reg = <0x00 0xa8000000 0x00 0x8000000>;
92+
no-map;
93+
phandle = <0x60>;
94+
};
95+
```
96+
97+
Add `iomem=relaxed` to `chosen.bootargs`. For example:
98+
99+
```dts
100+
bootargs = "clk-imx93.mcore_booted console=ttyLP0,115200 earlycon root=/dev/mmcblk1p2 rootwait rw iomem=relaxed";
101+
```
102+
103+
Build the patched device tree:
104+
105+
```bash
106+
dtc -I dts -O dtb -o devicetree/patched.dtb devicetree/live.dts
107+
```
108+
109+
Copy it to the board:
110+
111+
```bash
112+
scp devicetree/patched.dtb <user>@<target-ip>:/tmp/patched.dtb
113+
```
114+
115+
Install it on the board. Adjust the boot partition path if your image uses a different location:
116+
117+
```bash
118+
ssh <user>@<target-ip>
119+
cp /run/media/boot-mmcblk1p1/imx93-11x11-frdm.dtb \
120+
/run/media/boot-mmcblk1p1/imx93-11x11-frdm.dtb.bak
121+
cp /tmp/patched.dtb \
122+
/run/media/boot-mmcblk1p1/imx93-11x11-frdm.dtb
123+
sync
124+
reboot
125+
```
126+
127+
After the board reboots, run the Topo health check again from the host:
128+
129+
```bash
130+
topo health --target <user>@<target-ip>
131+
```
132+
133+
## Clone the Template
134+
135+
Clone the Template onto your host:
136+
137+
```bash
138+
topo clone https://github.com/Arm-Examples/topo-imx93-npu-deployment.git
139+
```
140+
141+
Topo prompts for optional build cache image arguments:
142+
143+
```output
144+
EXECUTORCH_BASE_CACHE_IMAGE
145+
IMX93_RUNNER_BUILD_CACHE_IMAGE
146+
```
147+
148+
Accept the defaults unless you have your own cache images.
149+
150+
Enter the project directory:
151+
152+
```bash
153+
cd topo-imx93-npu-deployment
154+
```
155+
156+
## Deploy to the board
157+
158+
Deploy the project to your target:
159+
160+
```bash
161+
topo deploy --target <user>@<target-ip>
162+
```
163+
164+
If not pulling from the cache, the first build can take a long time and requires about 25 GB of free disk space. It downloads and builds ExecuTorch, the Arm GNU toolchain, MCUX SDK components, RPMsg-Lite, and the Cortex-M33 runner sources. Later builds are faster when Docker can reuse local cache layers or import the configured GHCR cache layers.
165+
166+
During deployment, Topo builds the required images, transfers them to the target, starts the Cortex-M33 firmware through `remoteproc-runtime`, and starts the web application.
167+
168+
When deployment succeeds, the output includes a successful service startup. You can also check the deployed services:
169+
170+
```bash
171+
topo ps --target <user>@<target-ip>
172+
```
173+
174+
## Open the web application
175+
176+
Open the web application in a browser:
177+
178+
```output
179+
http://<target-ip>:3001
180+
```
181+
182+
The application shows:
183+
184+
- an image selector
185+
- a **Classify** button
186+
- board prerequisite checks
187+
- classification results
188+
- an expandable analysis section with runtime details
189+
190+
Select an image from an ImageNet-supported class, then click **Classify**. A successful run returns top-1 and top-5 ImageNet classifications.
191+
192+
If you need to use a different target port, set `WEBAPP_PORT` when deploying:
193+
194+
```bash
195+
WEBAPP_PORT=3002 topo deploy --target <user>@<target-ip>
196+
```
197+
198+
Then open:
199+
200+
```output
201+
http://<target-ip>:3002
202+
```
203+
204+
You should see something similar to:
205+
206+
207+
![Screenshot of the web interface running on an Arm-based target, showing an image and the model response. This confirms successful deployment and provides a visual reference for the expected result.#center](topo_npu_classifier.png "Image classification as seen in the web app")
208+
209+
## What you've accomplished
210+
211+
You have prepared an FRDM i.MX 93 board for shared-memory NPU inference, deployed the `topo-imx93-npu-deployment` Template with Topo, started Cortex-M33 firmware through `remoteproc-runtime`, and used a browser-based application to run MobileNetV2 classification with Ethos-U65 acceleration.
212+
Next, you will review the toolchains used to build the model artifact, firmware runner, and web application.

‎content/learning-paths/cross-platform/deploy-ml-model-to-npu-with-topo/overview.md‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
---
2+
title: Deploy ExecuTorch firmware on NXP FRDM i.MX 93 for Ethos-U65 acceleration using Topo
3+
weight: 2
4+
5+
### FIXED, DO NOT MODIFY
6+
layout: learningpathall
7+
---
8+
9+
## Get started
10+
11+
Before getting started, complete the Learning Path [Deploy containerized workloads to Arm-based Linux targets with Topo](/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/) to learn how to install Topo, run host and target health checks, inspect a target, list compatible Templates, and deploy a containerized workload.
12+
13+
For more background on the underlying NPU example, read [Deploy ExecuTorch firmware on NXP FRDM i.MX 93 for Ethos-U65 acceleration](/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/). You do not need to complete that Learning Path before using this one, but it helps explain the model, firmware, and [Ethos-U65](https://www.arm.com/products/silicon-ip-cpu/ethos/ethos-u65) execution flow.
14+
15+
## What is Topo?
16+
17+
[Topo](https://github.com/arm/topo) is an open-source command-line tool developed by Arm used to deploy projects to an Arm-based Linux target over SSH. Topo builds container images on the host, transfers them to the target, and starts the services on the target. Topo can also build and deploy directly on the target.
18+
19+
## What you'll learn
20+
21+
In this Learning Path, you will deploy the [topo-imx93-npu-deployment](https://github.com/Arm-Examples/topo-imx93-npu-deployment) Topo Template to an NXP FRDM i.MX 93 board.
22+
23+
The Template builds and deploys a browser-based MobileNetV2 image classifier. The user interface runs on the Cortex-A Linux side of the SoC. The inference runner is packaged as Cortex-M33 firmware and is started by [remoteproc-runtime](https://github.com/arm/remoteproc-runtime). The model is exported to an [ExecuTorch](https://docs.pytorch.org/executorch/stable/index.html) `.pte` [file](https://docs.pytorch.org/executorch/stable/pte-file-format.html) for Ethos-U65 NPU acceleration.
24+
25+
### What does deploying the topo-imx93-npu-deployment Template do?
26+
27+
Deploying the Template starts two runtime services on the target:
28+
29+
- `webapp`: Web application running on the Cortex-A Linux host. It receives an image to run a classification on.
30+
- `cm33-runner`: Cortex-M33 firmware, receives the image to classify from the web application and runs the classification Machine Learning model on it.
31+
32+
When you select an image in the browser and click **Classify**, the web application:
33+
34+
1. Resizes and normalizes the image to classify into an input tensor compatible with the [MobileNetV2](https://arxiv.org/abs/1801.04381) model.
35+
2. Writes the ExecuTorch program and input tensor into reserved physical memory.
36+
3. Sends a `RUN` command to the Cortex-M33 runner over `RPMsg`.
37+
4. Waits for the Cortex-M33 firmware to run inference using Ethos-U65 acceleration.
38+
5. Displays the top-1 and top-5 ImageNet classification results in the browser.
39+
40+
## System Architecture
41+
42+
The deployed application spans three processing domains on the i.MX 93:
43+
44+
- **Cortex-A Linux host**: runs Docker, Topo-deployed containers, the Flask web app, and the Linux `remoteproc` and `RPMsg` interfaces.
45+
- **Cortex-M33 firmware domain**: runs the ExecuTorch runner firmware loaded by `remoteproc-runtime`.
46+
- **Ethos-U65 NPU**: accelerates delegated neural network operators from the ExecuTorch MobileNetV2 program.
47+
48+
The high-level data flow is:
49+
50+
```output
51+
Browser
52+
|
53+
v
54+
Flask web application on Cortex-A Linux
55+
|
56+
| writes .pte and input tensor to reserved memory
57+
| sends RUN over RPMsg
58+
v
59+
Cortex-M33 ExecuTorch runner firmware
60+
|
61+
| delegates supported operators
62+
v
63+
Ethos-U65 NPU
64+
|
65+
v
66+
Cortex-M33 returns classification results over RPMsg
67+
|
68+
v
69+
Browser displays ImageNet top-1 and top-5 results
70+
```
71+
72+
## What you've accomplished and what's next
73+
74+
You now understand what the Topo Template deploys and how the Cortex-A, Cortex-M33, and Ethos-U65 parts work together. Next, you will prepare the i.MX 93 target and deploy the Template with Topo.

‎content/learning-paths/cross-platform/deploy-ml-model-to-npu-with-topo/topo_npu_classifier.png‎

625 KB
Loading

0 commit comments

Comments
 (0)