Skip to content

Commit 55c54c7

Browse files
psiddhpsiddh
authored
Add Arduino library support for ExecuTorch (pytorch#20221)
Add tooling to package the ExecuTorch runtime as an Arduino library, enabling PyTorch model inference on Arduino microcontrollers. The library vendors ET runtime sources, CMSIS-NN kernels, and portable ops into a self-contained package that compiles under the Arduino build system. Key components: - `build_arduino_library.sh` assembles the distributable library from repository sources (no vendored copies checked in) - `ExecuTorchArduino.h` configures the build environment for Arduino (fixes for std::variant, cmake_macros.h stub, build defines) - `platform_stubs.c` provides C library stubs for the LLEXT environment - Example sketches using the native ExecuTorch C++ API (no wrapper layer) - Zephyr board config for Arduino Uno Q (STM32U585, Cortex-M33) Validated on Arduino Uno Q with DS-CNN keyword spotting model (int8, CMSIS-NN): 390+ source files compile, 106 KB flash (13%), 91 KB RAM.
1 parent 63b4c4d commit 55c54c7

22 files changed

Lines changed: 2511 additions & 0 deletions

‎examples/arduino/.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# Generated by build_arduino_library.sh — do not check in
2+
arduino_lib/

‎examples/arduino/ExecuTorchArduino.h‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
/*
2+
* Copyright (c) Meta Platforms, Inc. and affiliates.
3+
* All rights reserved.
4+
*
5+
* This source code is licensed under the BSD-style license found in the
6+
* LICENSE file in the root directory of this source tree.
7+
*/
8+
9+
#pragma once
10+
11+
// Arduino's custom <new> header omits <exception>, which breaks
12+
// std::bad_variant_access in <variant>. Include it first.
13+
#include <exception>
14+
15+
#ifndef C10_USING_CUSTOM_GENERATED_MACROS
16+
#define C10_USING_CUSTOM_GENERATED_MACROS
17+
#endif
18+
#ifndef ET_ENABLE_DEPRECATED_CONSTANT_BUFFER
19+
#define ET_ENABLE_DEPRECATED_CONSTANT_BUFFER 0
20+
#endif
21+
#ifndef FLATBUFFERS_MAX_ALIGNMENT
22+
#define FLATBUFFERS_MAX_ALIGNMENT 1024
23+
#endif
24+
25+
#include <executorch/extension/data_loader/buffer_data_loader.h>
26+
#include <executorch/runtime/core/memory_allocator.h>
27+
#include <executorch/runtime/executor/method.h>
28+
#include <executorch/runtime/executor/method_meta.h>
29+
#include <executorch/runtime/executor/program.h>
30+
#include <executorch/runtime/platform/runtime.h>

‎examples/arduino/README.md‎

Lines changed: 344 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,344 @@
1+
<!---
2+
Copyright (c) Meta Platforms, Inc. and affiliates.
3+
All rights reserved.
4+
5+
This source code is licensed under the BSD-style license found in the
6+
LICENSE file in the root directory of this source tree.
7+
--->
8+
9+
# ExecuTorch Arduino Library
10+
11+
Run PyTorch models on Arduino microcontrollers using ExecuTorch.
12+
13+
This directory contains everything needed to package ExecuTorch as an
14+
Arduino library. A build script vendors the runtime sources from this
15+
repository into a self-contained library that Arduino users install
16+
through the Library Manager or by copying into their libraries folder.
17+
18+
## How It Works
19+
20+
```
21+
PyTorch Model ──► torch.export ──► .pte file ──► model.h (C array)
22+
│
23+
Arduino Sketch (.ino)
24+
#include <ExecuTorchArduino.h>
25+
#include "model.h"
26+
│
27+
arduino-cli compile ──► Upload ──► Runs on board
28+
```
29+
30+
### The three pieces
31+
32+
1. **The library** (`arduino_lib/ExecuTorchArduino/`) — the ExecuTorch
33+
runtime, CMSIS-NN kernels, and portable ops packaged for the Arduino
34+
build system. Generated by `build_arduino_library.sh`; not checked in.
35+
36+
2. **The model** (`model.h`) — a `.pte` file converted to a C byte array.
37+
Each user brings their own model, exported from PyTorch with the
38+
Cortex-M backend.
39+
40+
3. **The sketch** (`.ino`) — a standard Arduino program that loads the
41+
model, feeds it input, and reads the output. Uses the native
42+
ExecuTorch C++ API (`Program::load`, `Method::execute`, etc.).
43+
44+
## Supported Boards
45+
46+
| Board | MCU | Status |
47+
|-------|-----|--------|
48+
| Arduino Uno Q | STM32U585 (Cortex-M33) | Tested |
49+
| Arduino Nano 33 BLE | nRF52840 (Cortex-M4F) | Planned (requires mbed PAL) |
50+
| Arduino Giga R1 WiFi | STM32H747 (Cortex-M7) | Planned (requires mbed PAL) |
51+
| Arduino Portenta H7 | STM32H747 (Cortex-M7) | Planned (requires mbed PAL) |
52+
53+
The library currently requires the Zephyr board core. Non-Zephyr boards
54+
(mbed) need a platform abstraction layer port before they can compile.
55+
CMSIS-NN accelerated ops work on any ARM Cortex-M with DSP extensions.
56+
Portable ops work on any architecture.
57+
58+
## Quick Start
59+
60+
### 1. Build the Arduino library
61+
62+
```bash
63+
cd examples/arduino
64+
./build_arduino_library.sh
65+
```
66+
67+
This copies the required ExecuTorch sources from the repository into
68+
`arduino_lib/ExecuTorchArduino/`, ready for Arduino.
69+
70+
### 2. Install the library
71+
72+
Copy the generated library into your Arduino libraries folder:
73+
74+
```bash
75+
# macOS:
76+
cp -r arduino_lib/ExecuTorchArduino ~/Documents/Arduino/libraries/
77+
# Linux:
78+
cp -r arduino_lib/ExecuTorchArduino ~/Arduino/libraries/
79+
```
80+
81+
Or with `arduino-cli`:
82+
83+
```bash
84+
cd arduino_lib && zip -r ExecuTorchArduino.zip ExecuTorchArduino && cd ..
85+
arduino-cli lib install --zip-path arduino_lib/ExecuTorchArduino.zip
86+
```
87+
88+
### 3. Export a model
89+
90+
Each sketch needs a `model.h` file — a `.pte` model converted to a C
91+
byte array. Use `pte_to_header.py` from the Arm examples to convert
92+
any `.pte` file:
93+
94+
```bash
95+
python examples/arm/executor_runner/pte_to_header.py \
96+
-p model.pte -d examples/arduino/examples/AddModel -o model.h
97+
```
98+
99+
**AddModel** — export a simple add model (no dataset needed):
100+
101+
```bash
102+
python -c "
103+
import torch
104+
from executorch.exir import to_edge
105+
from torch.export import export
106+
class Add(torch.nn.Module):
107+
def forward(self, x): return x + 1.0
108+
et = to_edge(export(Add().eval(), (torch.tensor([1.,2.,3.]),))).to_executorch()
109+
with open('add.pte','wb') as f: f.write(bytes(et.buffer))"
110+
111+
python examples/arm/executor_runner/pte_to_header.py \
112+
-p add.pte -d examples/arduino/examples/AddModel -o model.h
113+
```
114+
115+
**HelloExecuTorch** — uses any valid model; the AddModel `.pte` works:
116+
117+
```bash
118+
cp examples/arduino/examples/AddModel/model.h \
119+
examples/arduino/examples/HelloExecuTorch/model.h
120+
```
121+
122+
**KeywordSpotting** — requires a quantized DS-CNN model. Generate it
123+
with `export_model.py`:
124+
125+
```bash
126+
# Download the dataset (one time, ~2.3 GB) — run from repo root:
127+
python -c "import torchaudio; torchaudio.datasets.SPEECHCOMMANDS(
128+
root='outputs/speech_commands', download=True)"
129+
130+
# Train DS-CNN, quantize with CMSIS-NN, and export model.h:
131+
python examples/arduino/export_model.py \
132+
--output examples/arduino/examples/KeywordSpotting/model.h
133+
```
134+
135+
This trains DS-CNN on Google Speech Commands v2 (100 samples/class),
136+
quantizes to int8 via `CortexMQuantizer`, calibrates with real MFCC
137+
audio data, and exports a 54 KB `.pte` as a C header.
138+
139+
To export with a pre-trained checkpoint instead of training:
140+
141+
```bash
142+
python examples/arduino/export_model.py --checkpoint my_weights.pth \
143+
--output examples/arduino/examples/KeywordSpotting/model.h
144+
```
145+
146+
**Note:** `build_arduino_library.sh` requires schema headers from a prior
147+
cmake build. If you haven't built ExecuTorch yet, run
148+
`./install_executorch.sh` first.
149+
150+
### 4. Write a sketch
151+
152+
```cpp
153+
#include <ExecuTorchArduino.h>
154+
#include "model.h"
155+
156+
using executorch::extension::BufferDataLoader;
157+
using executorch::runtime::Error;
158+
using executorch::runtime::HierarchicalAllocator;
159+
using executorch::runtime::MemoryAllocator;
160+
using executorch::runtime::MemoryManager;
161+
using executorch::runtime::Method;
162+
using executorch::runtime::MethodMeta;
163+
using executorch::runtime::Program;
164+
using executorch::runtime::Result;
165+
using executorch::runtime::Span;
166+
167+
alignas(16) uint8_t method_pool[28 * 1024];
168+
169+
void setup() {
170+
Serial.begin(115200);
171+
delay(2000);
172+
173+
executorch::runtime::runtime_init();
174+
175+
auto loader = BufferDataLoader(model_pte, sizeof(model_pte));
176+
Result<Program> program = Program::load(&loader);
177+
if (!program.ok()) {
178+
Serial.println("Failed to load program");
179+
return;
180+
}
181+
182+
// ... load method, set inputs, execute, read outputs
183+
// See examples/ for complete working sketches.
184+
}
185+
186+
void loop() {
187+
// Run inference periodically
188+
delay(2000);
189+
}
190+
```
191+
192+
The sketch uses the **native ExecuTorch C++ API** — the same API used on
193+
Linux, Android, and bare-metal targets. No wrapper layer, no
194+
Arduino-specific abstractions.
195+
196+
### 5. Compile and upload
197+
198+
```bash
199+
arduino-cli compile --fqbn arduino:zephyr:unoq MySketch
200+
arduino-cli upload --fqbn arduino:zephyr:unoq -p /dev/cu.usbmodem* MySketch
201+
arduino-cli monitor -p /dev/cu.usbmodem* --config baudrate=115200
202+
```
203+
204+
## What is inside the library
205+
206+
The `build_arduino_library.sh` script assembles these components from
207+
the ExecuTorch repository:
208+
209+
| Component | Source in repo | Purpose |
210+
|-----------|---------------|---------|
211+
| ET Runtime | `runtime/executor/`, `runtime/core/`, `runtime/kernel/`, `runtime/platform/` | Model loading, memory management, op dispatch |
212+
| Portable Ops | `kernels/portable/` | Software op implementations (any CPU) |
213+
| Cortex-M Ops | `backends/cortex_m/ops/` | CMSIS-NN accelerated int8 ops |
214+
| CMSIS-NN | fetched by cmake / Zephyr module | ARM's optimized DSP kernels |
215+
| flatcc | `third-party/flatcc/` | .pte file parsing |
216+
| flatbuffers | `third-party/flatbuffers/` | Schema headers |
217+
| c10 | `runtime/core/portable_type/c10/` | Core type definitions |
218+
219+
The library uses no external dependencies beyond what the Arduino board
220+
core provides.
221+
222+
## Arduino-specific patches
223+
224+
The build script applies these patches to make ExecuTorch compile under
225+
Arduino's build system:
226+
227+
1. **`#include <exception>` before `<variant>`** — Arduino's custom
228+
`<new>` header omits `<exception>`, breaking `std::bad_variant_access`.
229+
230+
2. **`cmake_macros.h` stub** — c10/torch headers expect a cmake-generated
231+
file. The build script generates a stub; `C10_USING_CUSTOM_GENERATED_MACROS`
232+
is defined in `ExecuTorchArduino.h` to skip the include.
233+
234+
3. **`platform_stubs.c`** — provides weak stubs for `_Exit()`, `fprintf()`,
235+
and `__aeabi_f2lz` for the LLEXT environment on boards that lack them.
236+
237+
4. **Compile-time defines** — `ExecuTorchArduino.h` sets
238+
`ET_ENABLE_DEPRECATED_CONSTANT_BUFFER=0` (requires models exported with
239+
current ExecuTorch) and `FLATBUFFERS_MAX_ALIGNMENT=1024`.
240+
241+
## Development
242+
243+
### Updating the library
244+
245+
After modifying ExecuTorch sources, regenerate the library:
246+
247+
```bash
248+
./build_arduino_library.sh # rebuild
249+
./build_arduino_library.sh --clean # remove generated output
250+
```
251+
252+
### Testing
253+
254+
```bash
255+
arduino-cli compile --fqbn arduino:zephyr:unoq examples/HelloExecuTorch
256+
arduino-cli upload --fqbn arduino:zephyr:unoq -p /dev/cu.usbmodem* examples/HelloExecuTorch
257+
arduino-cli monitor -p /dev/cu.usbmodem* --config baudrate=115200
258+
```
259+
260+
### Publishing to Arduino Library Manager
261+
262+
The library is published by adding its repository URL to the
263+
[Arduino Library Registry](https://github.com/arduino/library-registry).
264+
After the initial registration, new git tags are picked up automatically.
265+
266+
## Build Validation
267+
268+
Tested on Arduino Uno Q (STM32U585, Cortex-M33 @ 160 MHz):
269+
270+
- **Portable ops**: Add model (`x + 1.0`) produces correct output
271+
`[1,2,3] + 1 = [2.0, 3.0, 4.0]`
272+
- **CMSIS-NN linear**: Quantized linear model (int8, 2.2 KB) runs
273+
`arm_fully_connected_s8` via `cortex_m::quantized_linear`
274+
- **CMSIS-NN keyword spotting**: DS-CNN (MLPerf Tiny KWS benchmark,
275+
54 KB, int8) correctly classifies real audio from Google Speech
276+
Commands dataset via 16 CMSIS-NN accelerated ops (conv2d, depthwise
277+
conv2d, avgpool, linear, quantize, dequantize, pad)
278+
279+
### Keyword Spotting Results
280+
281+
Verified with real audio on hardware:
282+
283+
```
284+
"yes" → [yes]=7.82 >>> Detected: yes CORRECT!
285+
"no" → [no]=1.60 >>> Detected: no CORRECT!
286+
```
287+
288+
To test different keywords, change one line in the sketch:
289+
290+
```cpp
291+
// In KeywordSpotting.ino, change this line:
292+
#include "mfcc_yes.h" // → detects "yes"
293+
// #include "mfcc_no.h" // → detects "no"
294+
// #include "mfcc_stop.h" // → detects "stop"
295+
// Available: mfcc_yes.h, mfcc_no.h, mfcc_up.h, mfcc_down.h,
296+
// mfcc_left.h, mfcc_right.h, mfcc_on.h, mfcc_off.h,
297+
// mfcc_stop.h, mfcc_go.h
298+
```
299+
300+
To test with your own audio recording:
301+
302+
```bash
303+
python generate_test_input.py --input my_recording.wav --output mfcc_custom.h
304+
# Then: #include "mfcc_custom.h" in the sketch
305+
```
306+
307+
## End-to-End Flow
308+
309+
```
310+
Google Speech Commands "yes" audio (.wav, 16kHz, 1 second)
311+
→ MFCC extraction (49 time frames × 10 coefficients)
312+
→ DS-CNN model (23K params, trained on MacBook CPU)
313+
→ CortexMQuantizer → int8 (calibrated with real MFCC data)
314+
→ CMSIS-NN ops (conv2d, depthwise_conv2d, avgpool, linear)
315+
→ Export to .pte (54 KB)
316+
→ Arduino library → arduino-cli compile → upload
317+
→ Cortex-M33 @ 160 MHz (Arduino Uno Q, STM32U585)
318+
→ Serial output: ">>> yes" ✅
319+
```
320+
321+
## Dataset
322+
323+
Training and test audio from [Google Speech Commands v2](https://arxiv.org/abs/1804.03209)
324+
— 65,000 one-second recordings of 35 words spoken by thousands of
325+
people. Standard dataset used by the MLPerf Tiny benchmark. Download
326+
via `torchaudio.datasets.SPEECHCOMMANDS` (2.3 GB).
327+
328+
The DS-CNN KWS benchmark uses 12 output classes (silence, unknown, plus
329+
10 keywords). The Arduino export script trains the 10 keyword classes:
330+
yes, no, up, down, left, right, on, off, stop, go.
331+
332+
## LLEXT Memory Budget
333+
334+
The Arduino Uno Q loads sketches as LLEXT (Loadable Extensions).
335+
Sizes reported by `arduino-cli compile` (Zephyr board core 0.55.2):
336+
337+
| Build | Code | Data | Total | Status |
338+
|-------|------|------|-------|--------|
339+
| HelloExecuTorch (portable ops) | 62 KB | 27 KB | 89 KB | ✅ |
340+
| Add model (portable ops) | 88 KB | 35 KB | 123 KB | ✅ |
341+
| DS-CNN (selective CMSIS-NN) | 87 KB | 57 KB | 144 KB | ✅ |
342+
343+
All CMSIS-NN sources are compiled, but the linker's
344+
`--gc-sections` discards unused functions from the final binary.

0 commit comments

Comments
 (0)