Skip to content

Commit 7f412ae

Browse files
matt-cossinsmatt-cossins
committed
New Learning Path: Deploy containerized workloads on Arm Linux targets with Topo
1 parent bb71a7e commit 7f412ae

6 files changed

Lines changed: 444 additions & 0 deletions

File tree

‎content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/_index.md‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
---
2+
title: Deploy containerized workloads to Arm-based Linux targets with Topo
3+
4+
description: Learn how to use Topo to detect device capabilities, select a compatible template, and deploy containerized workloads to Arm-based Linux targets over SSH.
5+
6+
minutes_to_complete: 45
7+
8+
who_is_this_for: This is an introductory topic for embedded, edge, and cloud software developers who want to easily deploy containerized workloads to Arm-based Linux targets with Topo.
9+
10+
learning_objectives:
11+
- Understand what Topo is, and why you would use it to deploy containerixed workloads to Arm Linux targets
12+
- Use Topo to perform a health-check on the target, generate a target description to capture Arm processor capabilities
13+
- Clone a compatible Topo template for your hardware and deploy the workload
14+
- (Optional) Deploy containerized workloads across heterogeneous devices (Cortex-A + Cortex-M) with Topo and remoteproc-runtime
15+
- (Optional) Use Command-Line-Interface (CLI) Agents with Topo
16+
17+
prerequisites:
18+
- A host machine (x86 or Arm) with Linux, macOS, or Windows
19+
- An Arm-based Linux target you can access over SSH, for example AWS Graviton, Raspberry Pi, DGX Spark, i.MX 93
20+
- Docker installed on host and target. If needed, use [Install Docker](/install-guides/docker/)
21+
- lscpu installed on target (typically pre-installed with Linux)
22+
- SSH key-based authentication configured between host and target - if using password-based authentication, Topo can help you setup key-based authentication
23+
- Basic familiarity with containers and CLI tools
24+
25+
author: Matt Cossins
26+
27+
### Tags
28+
skilllevels: Introductory
29+
subjects: Containers and Virtualization
30+
armips:
31+
- Neoverse
32+
- Cortex-A
33+
- Cortex-M
34+
tools_software_languages:
35+
- Topo
36+
- Docker
37+
- SSH
38+
- remoteproc-runtime
39+
- remoteproc
40+
- CLI
41+
operatingsystems:
42+
- Linux
43+
- macOS
44+
- Windows
45+
46+
further_reading:
47+
- resource:
48+
title: Topo repository
49+
link: https://github.com/arm/topo
50+
type: documentation
51+
- resource:
52+
title: Topo template format
53+
link: https://github.com/arm/topo-template-format
54+
type: documentation
55+
- resource:
56+
title: Topo releases
57+
link: https://github.com/arm/topo/releases/latest
58+
type: website
59+
- resource:
60+
title: remoteproc-runtime
61+
link: https://github.com/arm/remoteproc-runtime
62+
type: documentation
63+
64+
65+
66+
### FIXED, DO NOT MODIFY
67+
# ================================================================================
68+
weight: 1 # _index.md always has weight of 1 to order correctly
69+
layout: "learningpathall" # All files under learning paths have this same wrapper
70+
learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content.
71+
---

‎content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/_next-steps.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
---
2+
# ================================================================================
3+
# FIXED, DO NOT MODIFY THIS FILE
4+
# ================================================================================
5+
weight: 21 # The weight controls the order of the pages. _index.md always has weight 1.
6+
title: "Next Steps" # Always the same, html page title.
7+
layout: "learningpathall" # All files under learning paths have this same wrapper for Hugo processing.
8+
---

‎content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/assess-compatibility.md‎

Lines changed: 165 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,165 @@
1+
---
2+
title: Use Topo to assess target compatibility
3+
weight: 3
4+
5+
### FIXED, DO NOT MODIFY
6+
layout: learningpathall
7+
---
8+
9+
## Run Topo health checks
10+
11+
We will now run a health check against your target. Run the following command from the terminal of your host device.
12+
13+
If you are using your host device simultaneously as your target, use `topo health --target localhost`.
14+
15+
```bash
16+
topo health --target user@my-target
17+
```
18+
19+
The output should appear similar to:
20+
21+
```output
22+
Host
23+
----
24+
SSH: ✅
25+
Container Engine: ✅
26+
27+
Target
28+
------
29+
Connectivity: ✅
30+
Container Engine: ✅
31+
Hardware Info: ✅
32+
```
33+
34+
A Topo health check confirms connectivity between the host and target, as well as the verifying the presence of dependencies such as docker.
35+
36+
You should resolve any `❌` errors before moving on. Warnings (`⚠️`) can indicate optional capabilities you can add later. `ℹ️` provides other information. A `✅` confirms the presence of dependencies and no warnings or errors.
37+
38+
If you are using password-based SSH, you will likely see the `❌` error below:
39+
40+
```output
41+
Connectivity: ❌ (key-based SSH authentication is not setup)
42+
→ run `topo setup-keys --target user@my-target` or manually setup SSH keys for the target
43+
```
44+
45+
This is because Topo requires key-based SSH. You can use the command specified above, and Topo will setup the key-based SSH for you. Ensure that if prompted to set a passphrase, you leave it empty. Afterwards, run `topo health` again to confirm it has correctly setup the key-based authentication.
46+
47+
## Optional: install remoteproc-runtime on heterogeneous devices
48+
49+
If using a Cortex-A + Cortex-M device, such as the i.MX 93, you may see a `⚠️` warning if `remoteproc-runtime` is not installed on the target.
50+
51+
[`remoteproc`](https://docs.kernel.org/staging/remoteproc.html) is a Linux kernel framework for managing remote / auxiliary processors in a heterogeneous SoC. It allows the main CPU (e.g., Cortex-A) to load firmware on to the auxiliary processors (e.g., Cortex-M), start and stop them, and to communicate with them (e.g. using `rpmsg`).
52+
53+
[`remoteproc-runtime`](https://github.com/arm/remoteproc-runtime) builds on this by adding a container-style (OCI) interface. This lets you package and manage firmware like container images using standard tools (e.g. Docker or containerd), even though the code runs as firmware on the Cortex-M. OCI (Open Container Initiative) defines open standards for container image formats and runtimes, ensuring compatibility across container tools.
54+
55+
You can use Topo to install `remoteproc-runtime`. Run the following command from the host device:
56+
57+
```bash
58+
topo install remoteproc-runtime --target user@my-target
59+
```
60+
61+
Run the health command again to verify installation. Topo uses `remoteproc-runtime` under the hood when deploying to heterogeneous devices.
62+
63+
## Generate a target description
64+
65+
In this step, you ask Topo to probe your target hardware and create a machine-readable YAML description.
66+
67+
On your host device, run:
68+
69+
```bash
70+
topo describe --target user@my-target
71+
```
72+
73+
This writes a `target-description.yaml` file in your current directory.
74+
75+
The file captures details such as CPU architecture features, which Topo uses to select compatible templates.
76+
77+
Open the file and have a look. An example snippet from an AWS Graviton instance is shown below, showing the main processor and its features, an absence of any remote / auxiliary processors, and the total memory:
78+
79+
```output
80+
host:
81+
- model: Neoverse-V1
82+
cores: 4
83+
features:
84+
- fp
85+
- asimd
86+
- evtstrm
87+
- aes
88+
...
89+
remoteprocs: []
90+
totalmemory_kb: 16044280
91+
```
92+
93+
## List templates compatible with your target
94+
95+
Now that Topo understand the capabilities of your target device, it can advise on the compatibility of templates.
96+
97+
Use the following command on your host device to to list templates according to the target description:
98+
99+
```bash
100+
topo templates --target-description target-description.yaml
101+
```
102+
103+
You can also query templates directly by specifying the target:
104+
105+
```bash
106+
topo templates --target user@my-target
107+
```
108+
109+
An example output for an AWS Graviton instance is shown below:
110+
111+
```output
112+
✅ topo-welcome | https://github.com/Arm-Examples/topo-welcome.git | main
113+
A minimal "Hello, World" web app for validating a Topo setup and deployment.
114+
It runs a single service that exposes a web page on the target,
115+
with the greeting text customizable via the GREETING_NAME parameter.
116+
117+
❌ topo-lightbulb-moment | https://github.com/Arm-Examples/topo-lightbulb-moment.git | main
118+
Features: remoteproc-runtime
119+
Reads a switch over GPIO pins on an M class cpu, reports switch state over
120+
Remoteproc Message, then a web application on the A class reads this and
121+
displays a lightbulb in either the on or off state. The lightbulb state is
122+
described by an LLM in any user-specified style.
123+
124+
✅ topo-v9-cpu-chat | https://github.com/Arm-Examples/topo-v9-cpu-chat.git | main
125+
Features: SVE, NEON
126+
Complete LLM chat application optimized for Arm CPU inference.
127+
128+
This project demonstrates running large language models on CPU
129+
using llama.cpp compiled with Arm baseline optimizations and
130+
accelerated using NEON SIMD and SVE (when supported and enabled).
131+
132+
The stack includes:
133+
- llama.cpp server with Arm NEON optimizations (SVE optional)
134+
- Quantized Qwen2.5-1.5B-Instruct model bundled in the image (~1.12 GB)
135+
- Simple web-based chat interface
136+
- No GPU required - pure CPU inference
137+
138+
Perfect for demos and testing! The bundled Qwen2.5-1.5B model allows the
139+
project to run immediately without downloading additional models.
140+
141+
Ideal for testing LLM workloads on Arm hardware without GPU dependencies,
142+
showcasing how far you can push NEON acceleration. Rebuild with SVE enabled
143+
when wider vectors are available.
144+
145+
✅ topo-simd-visual-benchmark | https://github.com/Arm-Examples/topo-simd-visual-benchmark.git | main
146+
Features: NEON, SVE
147+
Visual demonstration of SIMD performance benefits on Arm processors.
148+
Compare scalar (no SIMD), NEON (128-bit), and SVE (scalable vector)
149+
implementations running identical image processing workloads side-by-side.
150+
151+
This demo shows real hardware acceleration through three C++ services
152+
compiled with different architecture flags, processing the same box blur
153+
algorithm on images. Performance differences are measured in real-time
154+
and displayed in an interactive web dashboard.
155+
156+
Perfect for demonstrating to non-technical audiences the concrete benefits
157+
of SIMD optimizations, with visual results and quantified speedups.
158+
```
159+
160+
In the above example, `topo-lightbulb-moment` is marked as incompatible, since it requires an SoC with both a Cortex-A and a Cortex-M. The Graviton instance used contains Arm Neoverse cores only. All other templates are marked as compatible. You may see different results depending on the target hardware you use.
161+
162+
## What you've learned and what's next
163+
164+
You have performed a health check on your target device and generated a description of its hardware features. Topo has informed you which templates are compatible with your target. In the next step, you will choose and deploy a template containerized workload.
165+

‎content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/deploy-workload.md‎

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
---
2+
title: Deploy a compatible containerized workload with Topo
3+
weight: 4
4+
5+
### FIXED, DO NOT MODIFY
6+
layout: learningpathall
7+
---
8+
9+
## Choose a starter template and clone it
10+
11+
Choose one of the pre-existing templates that is compatible with your target hardware. This learning path will showcase deploying an LLM chatbot, but the same steps apply for the other templates.
12+
13+
To use a template, we clone it from git on our host device by running the following command:
14+
15+
```bash
16+
topo clone template:topo-v9-cpu-chat
17+
```
18+
19+
If a template asks for build arguments, Topo prompts you interactively.
20+
21+
This creates a project directory using the template. The directory will contain template source files and `compose.yaml`.
22+
23+
You may find it interesting to examine the `compose.yaml` file. An example for the LLM chatbot application is provided below:
24+
25+
```output
26+
services:
27+
llama-server:
28+
platform: linux/arm64
29+
build:
30+
context: ./llama-inference
31+
args:
32+
ENABLE_SVE: ON
33+
HF_MODEL: bartowski/Qwen_Qwen3.5-0.8B-GGUF
34+
HF_MODEL_FILE: ""
35+
ports:
36+
- "8080:8080"
37+
healthcheck:
38+
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
39+
interval: 10s
40+
timeout: 5s
41+
retries: 3
42+
start_period: 60s
43+
chat-ui:
44+
platform: linux/arm64
45+
build:
46+
context: ./simple-chat
47+
args:
48+
ENABLE_SVE: ON
49+
depends_on:
50+
llama-server:
51+
condition: service_healthy
52+
ports:
53+
- "3000:3000"
54+
x-topo:
55+
name: "Topo CPU AI Chat"
56+
description: "Complete LLM chat application optimized for Arm CPU inference.\n\nThis project demonstrates running large language models on CPU\nusing llama.cpp compiled with Arm baseline optimizations and \naccelerated using NEON SIMD and SVE (when supported and enabled).\n\nThe stack includes:\n- llama.cpp server with Arm NEON optimizations (SVE optional)\n- Quantized Qwen3.5-0.8B model bundled in the image\n- Simple web-based chat interface\n- No GPU required - pure CPU inference\n\nPerfect for demos and testing! The bundled Qwen3.5-0.8B model allows the\nproject to run immediately without downloading additional models.\n\nIdeal for testing LLM workloads on Arm hardware without GPU dependencies,\nshowcasing how far you can push NEON acceleration. Rebuild with SVE enabled\nwhen wider vectors are available.\n"
57+
features:
58+
- "SVE"
59+
- "NEON"
60+
args:
61+
HF_MODEL:
62+
description: "Hugging Face model repo ID containing a supported single-file GGUF model"
63+
default: "bartowski/Qwen_Qwen3.5-0.8B-GGUF"
64+
example: "unsloth/SmolLM2-135M-Instruct-GGUF"
65+
HF_MODEL_FILE:
66+
description: "Exact supported GGUF filename to download; sharded and mmproj files are rejected"
67+
default: ""
68+
example: "Qwen_Qwen3.5-0.8B-Q4_0.gguf"
69+
ENABLE_SVE:
70+
description: "Enables building with SVE instructions (OFF/ON)"
71+
default: "OFF"
72+
example: "ON"
73+
```
74+
75+
Changes can be made to the `compose.yaml` files to adjust arguments after the fact - for example, SVE can be turned ON or OFF for the LLM chatbot, and the LLM can be changed to use a different model.
76+
77+
## Deploy the app on the target
78+
79+
On your host device, enter the project directory created by the `topo clone` command. In the case of the LLM chatbot, this directory is `topo-v9-cpu-chat`:
80+
81+
```bash
82+
cd topo-v9-cpu-chat/
83+
```
84+
85+
Then use `topo deploy` to automatically build the container images on the host, transfer the images to the target via SSH, and start the application on the target:
86+
87+
```
88+
topo deploy --target user@my-target
89+
```
90+
91+
Once deployed, you can view the webpage at `http://<ip_address_of_target>:3000`. The port used depends on the template application chosen - you can see this in the `compose.yaml`.
92+
93+
The LLM chatbot application will appear as below:
94+
95+
![LLM Chatbot](llm_chatbot.png)
96+
97+
Depending on the permissions you have setup with your target device, you may not be able to use the IP address directly. In this case, you may need to forward to a local port and view at `http://localhost:port_number` instead:
98+
99+
```bash
100+
ssh -L 3000:localhost:3000 user@my-target
101+
```
102+
103+
## (Optional) Repeat this learning path, using a CLI Agent
104+
105+
Topo is packaged as a single executable with a `README.md` file. It is straightforward for agents to leverage.
106+
107+
Choose your preferred agent. If you do not have a CLI Agent pre-configured, you can follow our guides to [Install Codex](https://learn.arm.com/install-guides/codex-cli/), [Install Claude Code](https://learn.arm.com/install-guides/claude-code/), or [Install Gemini](https://learn.arm.com/install-guides/gemini/).
108+
109+
Once your agent is ready, ensure you have setup your host and target to have the required dependencies. Then ask your agent to leverage Topo to deploy an application to your target device.

‎content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/llm_chatbot.png‎

282 KB
Loading

0 commit comments

Comments
 (0)