Skip to content

Commit 599a6e7

Browse files
committed
Extension updates
1 parent 2a1f99d commit 599a6e7

6 files changed

Lines changed: 87 additions & 89 deletions

File tree

content/learning-paths/cross-platform/explore-model-artifacts-with-model-explorer/1-what-is-model-explorer.md

Lines changed: 14 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -7,11 +7,11 @@ weight: 2
77
layout: "learningpathall"
88
---
99

10-
## Model Explorer and Arm Adapters
10+
## Model Explorer and Arm extensions
1111

1212
[Model Explorer](https://ai.google.dev/edge/model-explorer) is an open-source, web-based graph visualizer and debugger from Google AI Edge. It provides a hierarchical view of model graphs, lets you expand and collapse layers, search for nodes, inspect metadata, highlight inputs and outputs, compare models, and add overlays to graph nodes. The default version, [contained in this repository](https://github.com/google-ai-edge/model-explorer/) supports TFLite, TF, TFJS, MLIR, and PyTorch (Exported Program) formats.
1313

14-
Model Explorer uses adapters and data providers to load formats beyond the built-in model types. The Arm adapters used in this Learning Path transform `.pte`, `.tosa`, `.vgf`, and `.etrecord` files into graph data that Model Explorer can display. In the final section, you also add `.etdp` runtime trace data as an overlay on top of the exported graph.
14+
Model Explorer uses adapters and data providers to load formats beyond the built-in model types. The [ExecuTorch extension for Model Explorer](https://github.com/arm/executorch-extension-model-explorer) combines support for `.pte`, `.etrecord`, and `.etdp` files. Standalone `.tosa` and `.vgf` files use separate TOSA and VGF adapters. You install all three packages together in the next section.
1515

1616
## What you will do
1717

@@ -21,13 +21,13 @@ The artifacts covered in this learning path are:
2121

2222
| Artifact | Model Explorer support | Workflow layer | Use it to inspect |
2323
| --- | --- | --- | --- |
24-
| `.pte` | `pte-adapter-model-explorer` | ExecuTorch program | Delegate regions, backend partitioning, work outside delegates, and the deployed ExecuTorch graph |
25-
| `.tosa` | `tosa-adapter-model-explorer` | Compiler/backend intermediate representation | Lowered operators, tensor shapes, quantized types, graph splits, and missed optimization opportunities |
26-
| `.vgf` | `vgf-adapter-model-explorer` | Vulkan ML graph artifact | Inputs, outputs, constants, tensor metadata, graph connectivity, and SPIR-V graph modules |
27-
| `.etrecord` | ExecuTorch ETRecord adapter | Export-time profiling context | Graph structure, debug handles, operator names, and delegate metadata used to map runtime events back to graph nodes |
28-
| `.etdp` | ExecuTorch ETDump data provider | Runtime trace overlay | Timing data from a specific execution |
24+
| `.pte` | PTE adapter in the ExecuTorch extension | ExecuTorch program | Delegate regions, backend partitioning, work outside delegates, and the deployed ExecuTorch graph |
25+
| `.tosa` | Separate TOSA adapter | Compiler/backend intermediate representation | Lowered operators, tensor shapes, quantized types, graph splits, and missed optimization opportunities |
26+
| `.vgf` | Separate VGF adapter | Vulkan ML graph artifact | Inputs, outputs, constants, tensor metadata, graph connectivity, and SPIR-V graph modules |
27+
| `.etrecord` | ETRecord adapter in the ExecuTorch extension | Export-time profiling context | Graph structure, debug handles, operator names, and delegate metadata used to map runtime events back to graph nodes |
28+
| `.etdp` | ETDump data provider in the ExecuTorch extension | Runtime trace overlay | Timing data from a specific execution |
2929

30-
This learning path focusses on Arm adapters and extensions for Modle Explorer, but the repository of artifacts we use also includes some `.tflite` and `.pt2` files. You can optionally try these out as well, and they are natively supported by Model Explorer.
30+
This Learning Path focuses on Arm adapters and extensions for Model Explorer, but the artifacts repository also includes `.tflite` and `.pt2` files. You can optionally try these files because Model Explorer supports them without an additional adapter.
3131

3232
{{% notice Note %}}
3333
Model Explorer visualizes the specific artifact you generated or received. Small differences in the target the model has been delegated to, could result in a very different model graph. For example, delegating the same model to an Ethos-U55, may produce a very different model graph from delegating to an Ethos-U85.
@@ -80,10 +80,10 @@ For VGF, the ExecuTorch Arm VGF backend uses the [Arm ML SDK Model Converter](ht
8080

8181
When ExecuTorch is used for VGF, a `.pte` is emitted as well. Use that VGF-backend `.pte` when you want to run through ExecuTorch. Use the standalone `.vgf` when you want to inspect or integrate the Vulkan ML artifact directly, such as in a neural graphics workflow.
8282

83-
ETRecord and ETDump sit alongside these artifact views rather than replacing them. ETRecord is generated at export time and preserves the graph context needed for profiling attribution. ETDump is generated at runtime and records what actually happened when a `.pte` ran. Together, they let Model Explorer move from static inspection to runtime overlays: you can connect the graph structures you saw in the `.pte`, `.tosa`, and `.vgf` sections to operator and delegate events measured during execution.
84-
8583
If you have used the [Arm Neural Graphics Model Gym](https://github.com/arm/neural-graphics-model-gym), then under the hood you have been using ExecuTorch to export your neural graphics model to VGF. If you are interested in learning more, try out the [Fine-tune neural graphics using Model Gym](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/model-training-gym/#:~:text=Upon%20completion%20of%20this%20Learning,and%20train%20neural%20graphics%20models) learning path, which briefly introduces Model Explorer.
8684

85+
ETRecord and ETDump are additional, ExecuTorch-specific, artifacts. ETRecord is generated at export time and preserves the graph context needed for profiling attribution. ETDump is generated at runtime and records what actually happened when a `.pte` ran. Together, they let Model Explorer move from static inspection to runtime overlays: you can connect the graph structures you saw in the `.pte`, `.tosa`, and `.vgf` sections to operator and delegate events measured during execution.
86+
8787
## Terminology
8888

8989
A helpful glossary of different terms is provided below:
@@ -112,24 +112,14 @@ The hands-on sections will use a variety of pre-provided model artifacts. These
112112

113113
```output
114114
ml-model-artifacts/
115-
├── README.md
116-
├── LICENSE.md
117115
├── pte/
118116
│ ├── mv2_cortex_m.pte
119117
│ ├── opt125m_cortex_a_portable.pte
120118
│ ├── opt125m_cortex_a_xnnpack.pte
121119
│ ├── mv2_fp32_ethos_u85.pte
122120
│ ├── mv2_int8_ethos_u85.pte
123121
│ ├── mv2_lrn_int8_ethos_u85.pte
124-
│ ├── small_upscaler_ptq_vgf.pte
125-
│ ├── small_upscaler_qat_vgf.pte
126-
│ └── add_sigmoid_vgf.pte
127-
├── pt2/
128-
│ └── mv2_fp32.pt2
129-
├── tflite/
130-
│ ├── mv2_fp32.tflite
131-
│ ├── mv2_int8.tflite
132-
│ └── mv2_lrn_int8.tflite
122+
│ └── small_upscaler_ptq_vgf.pte
133123
├── tosa/
134124
│ ├── mv2_fp32.tosa
135125
│ ├── mv2_int8.tosa
@@ -138,9 +128,7 @@ ml-model-artifacts/
138128
│ ├── small_upscaler_ptq.tosa
139129
│ └── small_upscaler_qat.tosa
140130
├── vgf/
141-
│ ├── small_upscaler_ptq.vgf
142-
│ ├── small_upscaler_qat.vgf
143-
│ └── add_sigmoid.vgf
131+
│ └── small_upscaler_ptq.vgf
144132
├── etrecord/
145133
│ ├── opt125m_portable.etrecord
146134
│ ├── opt125m_xnnpack.etrecord
@@ -157,6 +145,6 @@ ml-model-artifacts/
157145

158146
## What you have learned
159147

160-
You have learned how Model Explorer uses adapters and data providers to load artifact formats beyond its built-in model types. You have also seen how `.pte`, `.tosa`, `.vgf`, `.etrecord`, and `.etdp` files fit into Cortex-A, Cortex-M, Ethos-U, Vulkan ML, and ExecuTorch profiling workflows. The artifacts repository also includes `.pt2` and `.tflite` files that you can inspect directly as optional extensions.
148+
You have learned how the combined ExecuTorch extension and the separate TOSA and VGF adapters add artifact formats to Model Explorer. You have also seen how `.pte`, `.tosa`, `.vgf`, `.etrecord`, and `.etdp` files fit into Cortex-A, Cortex-M, Ethos-U, Vulkan ML, and ExecuTorch profiling workflows.
161149

162-
Next, you will install Model Explorer, launch it with the Arm adapters, and open the first `.pte` artifact.
150+
Next, you will install Model Explorer and the Arm extensions, then open the first `.pte` artifact.

content/learning-paths/cross-platform/explore-model-artifacts-with-model-explorer/2-install-and-open-pte.md

Lines changed: 37 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: "Install Model Explorer and the adapters, and view a Cortex-M model graph"
2+
title: "Install Model Explorer extensions and view a Cortex-M model graph"
33

44
weight: 3
55

@@ -9,11 +9,29 @@ layout: "learningpathall"
99

1010
## Clone the repo of example models
1111

12-
In this section, you install Model Explorer in a clean Python virtual environment, along with the Arm adapters, and confirm it is working using the PTE adapter. First you will clone the repo of example models to use across the course of this learning path.
12+
In this section, you install Model Explorer and the Arm extensions in a clean Python virtual environment. You then confirm the installation by opening a Cortex-M `.pte` file. First, clone the repository of example models used throughout this Learning Path.
1313

1414
Use a machine capable of displaying a browser e.g, a laptop.
1515

16-
This repository uses Git LFS for model artifacts. After cloning, run `git lfs pull` to download the files.
16+
This repository uses Git LFS for model artifacts. Install and configure Git LFS for your operating system. You only need to run `git lfs install` once for your user account.
17+
18+
{{< tabpane code=true >}}
19+
{{< tab header="Linux" language="bash">}}
20+
sudo apt update
21+
sudo apt install -y git-lfs
22+
git lfs install
23+
{{< /tab >}}
24+
{{< tab header="macOS" language="bash">}}
25+
brew install git-lfs
26+
git lfs install
27+
{{< /tab >}}
28+
{{< tab header="Windows PowerShell" language="powershell">}}
29+
winget install -e --id GitHub.GitLFS
30+
git lfs install
31+
{{< /tab >}}
32+
{{< /tabpane >}}
33+
34+
Clone the artifacts repository, then use `git lfs pull` to download the model files:
1735

1836
```bash
1937
git clone https://github.com/arm-education/ml-model-artifacts.git
@@ -25,6 +43,8 @@ git lfs pull
2543

2644
Use a separate environment to avoid dependency conflicts with any ExecuTorch build, notebook, or application environment you already use:
2745

46+
Use Python 3.10, 3.11, or 3.12 for this environment.
47+
2848
If you use WSL on Windows, follow the Linux/macOS commands.
2949

3050
{{< tabpane code=true >}}
@@ -48,53 +68,32 @@ Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
4868
```
4969
{{% /notice %}}
5070

51-
## Install Model Explorer
71+
## Install Model Explorer and the Arm extensions
5272

53-
Install Model Explorer and PyTorch in the active virtual environment.
54-
55-
The Linux command pins PyTorch to the CPU wheel index because this Learning Path does not need CUDA. On macOS and Windows, the standard PyPI install is usually sufficient.
56-
57-
{{< tabpane code=true >}}
58-
{{< tab header="Linux" language="bash">}}
59-
pip install torch --index-url https://download.pytorch.org/whl/cpu
60-
pip install ai-edge-model-explorer
61-
{{< /tab >}}
62-
{{< tab header="macOS" language="bash">}}
63-
pip install torch ai-edge-model-explorer
64-
{{< /tab >}}
65-
{{< tab header="Windows PowerShell" language="powershell">}}
66-
pip install torch ai-edge-model-explorer
67-
{{< /tab >}}
68-
{{< /tabpane >}}
73+
Install the combined ExecuTorch extension with the separate TOSA and VGF adapters:
6974

70-
## Install the Arm adapters
75+
```bash
76+
python -m pip install executorch-extension-model-explorer tosa-adapter-model-explorer vgf-adapter-model-explorer
77+
```
7178

72-
{{% notice TODO before release %}}
73-
Update this installation section after the ExecuTorch Model Explorer extension is released.
79+
The ExecuTorch extension provides the PTE adapter, ETRecord adapter, and ETDump profiling data provider. The separate TOSA and VGF adapters open standalone `.tosa` and `.vgf` files.
7480

75-
The intended install flow is to install the PTE, ETRecord, and ETDump adapters/data provider as one ExecuTorch extension, then launch Model Explorer with that ExecuTorch extension alongside the TOSA and VGF adapters. Until that package is available, keep the ETRecord and ETDump install and launch instructions under review.
81+
{{% notice Note %}}
82+
For component development or focused debugging, you can install the ExecuTorch components separately as `pte-adapter-model-explorer`, `etrecord-adapter-model-explorer`, and `etdump-data-provider-model-explorer`. For this Learning Path, use the combined `executorch-extension-model-explorer` package.
7683
{{% /notice %}}
7784

78-
Install the PTE, TOSA, and VGF adapters used for the static artifact sections:
79-
80-
```bash
81-
pip install pte-adapter-model-explorer
82-
pip install tosa-adapter-model-explorer
83-
pip install vgf-adapter-model-explorer
84-
```
85-
8685
## Launch Model Explorer
8786

88-
Launch Model Explorer with the Arm adapters. This is the recommended route for the static artifact sections because you will open `.pte`, `.tosa`, and `.vgf` artifacts. Also shown are examples of running the base model explorer without adapters, and running with a single adapter.
87+
Launch Model Explorer with all extensions used in this Learning Path. The combined ExecuTorch extension opens `.pte` and `.etrecord` files and adds `.etdp` profiling data. The separate TOSA and VGF extensions open `.tosa` and `.vgf` files.
8988

9089
This will launch a webpage in your browser.
9190

9291
{{< tabpane code=true >}}
93-
{{< tab header="All adapters" language="bash">}}
94-
model-explorer --extensions=pte_adapter_model_explorer,tosa_adapter_model_explorer,vgf_adapter_model_explorer
92+
{{< tab header="All Learning Path extensions" language="bash">}}
93+
model-explorer --extensions=executorch_extension_model_explorer,tosa_adapter_model_explorer,vgf_adapter_model_explorer
9594
{{< /tab >}}
96-
{{< tab header="PTE adapter" language="bash">}}
97-
model-explorer --extensions=pte_adapter_model_explorer
95+
{{< tab header="ExecuTorch extension" language="bash">}}
96+
model-explorer --extensions=executorch_extension_model_explorer
9897
{{< /tab >}}
9998
{{< tab header="No adapters" language="bash">}}
10099
model-explorer
@@ -162,6 +161,6 @@ Click the `eye symbol` in the top left bar to select data to view on nodes and e
162161

163162
## What you have learned
164163

165-
You have installed Model Explorer, launched it with the Arm adapters, and opened your first `.pte` artifact. You have also learned how to inspect the graph overview, expand into the `forward` layer, read operator metadata, and interpret common low-level `.pte` fields such as `KernelCall`, `op_index`, and `args`.
164+
You have installed Model Explorer, launched it with the Arm extensions, and opened your first `.pte` artifact. You have also learned how to inspect the graph overview, expand into the `forward` layer, read operator metadata, and interpret common low-level `.pte` fields such as `KernelCall`, `op_index`, and `args`.
166165

167166
Next, you will keep the same browser tab open and compare portable and XNNPACK `.pte` artifacts to see how backend delegation changes the graph.

content/learning-paths/cross-platform/explore-model-artifacts-with-model-explorer/4-inspect-ethosu-pte.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -89,7 +89,7 @@ Look for:
8989
- Multiple delegate regions
9090
- Operators between delegate regions
9191
- Unsupported operations or graph patterns outside the NPU delegate
92-
- Extra tensor movement around backend boundaries
92+
- Quantize, dequantize, or non-delegated operators between delegate regions
9393

9494
![Screenshot of examining a fragmented INT8 Ethos-U PTE in Model Explorer.#center](ethos-u-int8-fragmented.png "Inspecting fragmented INT8 Ethos-U PTE with Model Explorer")
9595

@@ -104,7 +104,7 @@ In Model Explorer, compare it with the clean delegated artifact:
104104
- You should see quantize and dequantize nodes around the delegated regions.
105105
- You should see an `aten::avg_pool3d` node between the delegate regions. This is visible non-delegated work that breaks the otherwise contiguous NPU path.
106106

107-
This is what fragmentation looks like in a `.pte`: the NPU still accelerates supported regions, but unsupported work splits the graph and creates extra delegate boundaries. In a deployed runtime, non-delegated work can run on the CPU only if the runtime includes compatible kernels for those operators, dtypes, layouts, and shapes. Cortex-M bare-metal runtimes often include a narrower kernel set than Cortex-A runtimes, and both can use selective builds that include only the kernels required by the product.
107+
This is what fragmentation looks like in a `.pte`: the NPU still accelerates supported regions, but unsupported work splits the graph and creates extra delegate boundaries. Additional boundaries can introduce data conversion, synchronization, or tensor movement. Non-delegated operators can also dominate runtime, or fail to run if the deployed runtime does not include compatible kernels for their operators, data types, layouts, and shapes. The static graph shows where delegation splits, but it does not show which cost dominates.
108108

109109
## Compare targets only with target-specific artifacts
110110

content/learning-paths/cross-platform/explore-model-artifacts-with-model-explorer/6-inspect-tosa.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,8 @@ The second LRN TOSA file is the larger fragment. It starts from the original ima
9696

9797
The graph fragmentation has become visible as multiple backend-ready TOSA artifacts. That matches the fragmented `.pte` view, where two `EthosUBackend` delegate regions were separated by non-delegated work.
9898

99+
The two TOSA files show where the backend-supported graph was split. They do not measure the cost of that split or confirm that the non-delegated operators can run in the deployed runtime. Runtime profiling is needed to identify whether boundary-related work or non-delegated operators dominate execution.
100+
99101
## Use TOSA outside ExecuTorch
100102

101103
TOSA is not limited to ExecuTorch. ExecuTorch can lower supported graph partitions to TOSA, but a different framework, internal compiler, or proprietary model frontend could also produce TOSA directly.

0 commit comments

Comments
 (0)