You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/learning-paths/cross-platform/explore-model-artifacts-with-model-explorer/1-what-is-model-explorer.md
+29-26Lines changed: 29 additions & 26 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,6 @@
1
1
---
2
-
title: "What is Model Explorer and what will you learn?"
2
+
title: Understand Model Explorer and the artifacts you will inspect
3
+
description: Understand how Model Explorer and Arm extensions visualize PTE, TOSA, VGF, ETRecord, and ETDump artifacts across model deployment workflows.
3
4
4
5
weight: 2
5
6
@@ -9,43 +10,43 @@ layout: "learningpathall"
9
10
10
11
## Model Explorer and Arm extensions
11
12
12
-
[Model Explorer](https://ai.google.dev/edge/model-explorer) is an open-source, web-based graph visualizer and debugger from Google AI Edge. It provides a hierarchical view of model graphs, lets you expand and collapse layers, search for nodes, inspect metadata, highlight inputs and outputs, compare models, and add overlays to graph nodes. The default version, [contained in this repository](https://github.com/google-ai-edge/model-explorer/) supports TFLite, TF, TFJS, MLIR, and PyTorch (Exported Program) formats.
13
+
[Model Explorer](https://ai.google.dev/edge/model-explorer) is an open-source, web-based graph visualizer and debugger from Google AI Edge. The visualizer provides a hierarchical view of model graphs. You can expand and collapse layers, search for nodes, inspect metadata, highlight inputs and outputs, compare models, and add overlays to graph nodes. The default version, [contained in the Model Explorer GitHub repository](https://github.com/google-ai-edge/model-explorer/), supports TFLite, TF, TFJS, MLIR, and PyTorch (Exported Program) formats.
13
14
14
-
Model Explorer uses adapters and data providers to load formats beyond the built-in model types. The [ExecuTorch extension for Model Explorer](https://github.com/arm/executorch-extension-model-explorer) combines support for `.pte`, `.etrecord`, and `.etdp` files. Standalone `.tosa` and `.vgf` files use separate TOSA and VGF adapters. You install all three packages together in the next section.
15
+
Model Explorer uses adapters and data providers to load formats beyond the built-in model types. The [ExecuTorch extension for Model Explorer](https://github.com/arm/executorch-extension-model-explorer) combines support for `.pte`, `.etrecord`, and `.etdp` files. Standalone `.tosa` and `.vgf` files use separate Tensor Operator Set Architecture (TOSA) and VGF adapters. You'll install all three packages together in the next section.
15
16
16
17
## What you will do
17
18
18
-
This learning path is about model artifacts, not model training or export. You start from small pre-generated files and use each to gain an understanding of when you would use Model Explorer, and the insights you can gain.
19
+
This Learning Path is about model artifacts, not model training or export. You'll start from small pre-generated files and use each to understand when you'd use Model Explorer, and what insights you can gain.
19
20
20
-
The artifacts covered in this learning path are:
21
+
You'll use the following artifacts:
21
22
22
23
| Artifact | Model Explorer support | Workflow layer | Use it to inspect |
23
24
| --- | --- | --- | --- |
24
25
|`.pte`| PTE adapter in the ExecuTorch extension | ExecuTorch program | Delegate regions, backend partitioning, work outside delegates, and the deployed ExecuTorch graph |
|`.tosa`| Separate TOSA adapter | Compiler or backend intermediate representation | Lowered operators, tensor shapes, quantized types, graph splits, and missed optimization opportunities |
26
27
|`.vgf`| Separate VGF adapter | Graph artifact for the ML extensions for Vulkan | Inputs, outputs, constants, tensor metadata, graph connectivity, and SPIR-V graph modules |
27
28
|`.etrecord`| ETRecord adapter in the ExecuTorch extension | Export-time profiling context | Graph structure, debug handles, operator names, and delegate metadata used to map runtime events back to graph nodes |
28
29
|`.etdp`| ETDump data provider in the ExecuTorch extension | Runtime trace overlay | Timing data from a specific execution |
29
30
30
-
This Learning Path focuses on Arm adapters and extensions for Model Explorer, but the artifacts repository also includes `.tflite` and `.pt2` files. You can optionally try these files because Model Explorer supports them without an additional adapter.
31
+
You'll focus on Arm adapters and extensions for Model Explorer, but the artifacts repository also includes `.tflite` and `.pt2` files. You can optionally try these files because Model Explorer supports them without an additional adapter.
31
32
32
33
{{% notice Note %}}
33
-
Model Explorer visualizes the specific artifact you generated or received. Small differences in the target the model has been delegated to, could result in a very different model graph. For example, delegating the same model to an Ethos-U55, may produce a very different model graph from delegating to an Ethos-U85.
34
+
Model Explorer visualizes the specific artifact you generated or received. Small differences in the target the model has been delegated to can result in a very different model graph. For example, delegating the same model to an Ethos-U55, might produce a very different model graph from delegating to an Ethos-U85.
34
35
{{% /notice %}}
35
36
36
37
## Understand the model artifact flow
37
38
38
-
Not every graph in this learning path needs to start with PyTorch and ExecuTorch. Let's run through what is specific to the ExecuTorch flow, and what is applicable more broadly.
39
+
Not every graph in this Learning Path needs to start with PyTorch and ExecuTorch.
39
40
40
41
PTE is ExecuTorch-specific. A `.pte` file is the serialized ExecuTorch program loaded by the ExecuTorch runtime. Baseline portable-kernel, XNNPACK, Cortex-M, Ethos-U, and VGF-backend examples are all ExecuTorch deployment artifacts.
41
42
42
43
The portable-kernel and XNNPACK routes are Cortex-A CPU routes. Cortex-M uses a separate ExecuTorch flow that applies CMSIS-NN-oriented passes for supported quantized operators.
43
44
44
-
For Ethos-U, the ExecuTorch backend uses the Ethos-U Vela compiler to compile TOSA flatbuffers into an Ethos-U command stream that is packaged into the final `.pte`.
45
+
For Ethos-U, the ExecuTorch backend uses the Ethos-U Vela compiler to compile TOSA flatbuffers into an Ethos-U command stream that's packaged into the final `.pte`.
45
46
46
-
TOSA is not inherently ExecuTorch-specific. TOSA is an intermediate representation (IR) that can sit between a model frontend and an Arm backend compiler or converter. ExecuTorch can lower supported graph partitions to TOSA, but another framework could also provide its own TOSA exporter and generate `.tosa` files.
47
+
TOSA is not inherently ExecuTorch-specific. TOSA is an intermediate representation (IR) that can sit between a model frontend and an Arm backend compiler or converter. ExecuTorch can lower supported graph partitions to TOSA, but another framework can also provide its own TOSA exporter and generate `.tosa` files.
47
48
48
-
```output
49
+
```text
49
50
PyTorch model
50
51
|
51
52
v
@@ -61,7 +62,7 @@ TOSA is not inherently ExecuTorch-specific. TOSA is an intermediate representati
61
62
|
62
63
|
63
64
|
64
-
Alternative (non PT/ET) v
65
+
Alternative (non PyTorch/ExecuTorch) v
65
66
frontend with TOSA export ---> TOSA (.tosa)
66
67
|
67
68
|
@@ -76,19 +77,21 @@ TOSA is not inherently ExecuTorch-specific. TOSA is an intermediate representati
76
77
for workflows using the ML extensions for Vulkan
77
78
```
78
79
79
-
For VGF, the ExecuTorch Arm VGF backend uses the [Arm ML SDK Model Converter](https://github.com/arm/ai-ml-sdk-model-converter) to produce a VGF backend payload from TOSA. But the Arm ML SDK Model Converter could be used to convert `.tosa` files generated from a different flow, so VGF is also not specific to just ExecuTorch.
80
+
For VGF, the ExecuTorch Arm VGF backend uses the [Arm ML SDK Model Converter](https://github.com/arm/ai-ml-sdk-model-converter) to produce a VGF backend payload from TOSA. But the Arm ML SDK Model Converter can be used to convert `.tosa` files generated from a different flow, so VGF is also not specific to ExecuTorch.
80
81
81
82
When ExecuTorch is used for VGF, a `.pte` is emitted as well. Use that VGF-backend `.pte` when you want to run through ExecuTorch. Use the standalone `.vgf` when you want to inspect or integrate the VGF artifact used with the ML extensions for Vulkan, such as in a neural graphics workflow.
82
83
83
-
If you have used the [Arm Neural Graphics Model Gym](https://github.com/arm/neural-graphics-model-gym), then under the hood you have been using ExecuTorch to export your neural graphics model to VGF. If you are interested in learning more, try out the [Fine-tune neural graphics using Model Gym](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/model-training-gym/#:~:text=Upon%20completion%20of%20this%20Learning,and%20train%20neural%20graphics%20models) learning path, which briefly introduces Model Explorer.
84
+
If you've used the [Arm Neural Graphics Model Gym](https://github.com/arm/neural-graphics-model-gym), then you've been using ExecuTorch to export your neural graphics model to VGF. To learn more, see the [Fine-tune neural graphics using Model Gym](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/model-training-gym/#:~:text=Upon%20completion%20of%20this%20Learning,and%20train%20neural%20graphics%20models) Learning Path, which briefly introduces Model Explorer.
85
+
86
+
ETRecord and ETDump are additional ExecuTorch-specific artifacts. ETRecord is generated at export time and preserves the graph context needed for profiling attribution. ETDump is generated at runtime and records what happened when a `.pte` ran. Together, these artifacts let Model Explorer move from static inspection to runtime overlays.
84
87
85
-
ETRecord and ETDump are additional, ExecuTorch-specific, artifacts. ETRecord is generated at export time and preserves the graph context needed for profiling attribution. ETDump is generated at runtime and records what actually happened when a `.pte` ran. Together, they let Model Explorer move from static inspection to runtime overlays: you can connect the graph structures you saw in the `.pte`, `.tosa`, and `.vgf` sections to operator and delegate events measured during execution.
88
+
You can connect the graph structures that you'll see in the `.pte`, `.tosa`, and `.vgf` sections to operator and delegate events measured during execution.
86
89
87
90
## Terminology
88
91
89
-
A helpful glossary of different terms is provided below:
92
+
The following is a glossary of different terms used in this Learning Path:
90
93
91
-
| Term | Meaning in this learning path|
94
+
| Term | Meaning in this Learning Path|
92
95
| --- | --- |
93
96
| Model | The neural network you want to deploy, before or after transformation by export and compiler tools. |
94
97
| Framework | The software used to define or train the model, such as PyTorch. |
@@ -98,17 +101,17 @@ A helpful glossary of different terms is provided below:
98
101
| Compiler | A tool that transforms an intermediate representation into a lower-level target representation, such as Vela compiling TOSA for Ethos-U. |
99
102
| Lower | To transform a model or graph from a higher-level representation into a lower-level representation closer to a target backend. |
100
103
| Convert | To change one artifact format into another, such as TOSA to VGF. |
101
-
| Intermediate representation | A representation between the original model and the final target artifact. TOSA is the main intermediate representation in this learning path. |
104
+
| Intermediate representation | A representation between the original model and the final target artifact. TOSA is the main intermediate representation in this Learning Path. |
102
105
| Delegate | A backend-specific execution path that handles supported parts of a graph. Unsupported parts can remain outside the delegate, and can run on a CPU path only when the deployed runtime includes compatible kernels. |
103
106
| Kernel | The code that executes a model operator for a specific runtime or backend, such as a portable ExecuTorch kernel or a CMSIS-NN kernel. |
104
-
| Flatbuffer | A compact binary serialization format. TOSA flatbuffers store TOSA graphs;`.pte` files use a FlatBuffer-based ExecuTorch program format. |
105
-
| SPIR-V | Standard Portable Intermediate Representation - Vulkan. SPIR-V modules inside VGF files describe the data graph used by the ML extensions for Vulkan. |
107
+
| Flatbuffer | A compact binary serialization format. TOSA flatbuffers store TOSA graphs, and`.pte` files use a FlatBuffer-based ExecuTorch program format. |
108
+
| SPIR-V | Standard Portable Intermediate Representation - Vulkan (SPIR-V) modules inside VGF files describe the data graph used by the ML extensions for Vulkan. |
106
109
| ETRecord | An ExecuTorch export-time debug artifact that preserves graph and delegate metadata for profiling attribution. |
107
110
| ETDump | An ExecuTorch runtime trace artifact that can record operator, delegate, backend, timing, and cycle-count events from execution. The first Model Explorer overlay used in this Learning Path focuses on timing data. |
108
111
109
-
## What models will I use?
112
+
## What models you will use
110
113
111
-
The hands-on sections will use a variety of pre-provided model artifacts. These are provided purely for educational purposes.
114
+
You'll use the following pre-provided model artifactsprovided purely for educational purposes:
112
115
113
116
```output
114
117
ml-model-artifacts/
@@ -143,8 +146,8 @@ ml-model-artifacts/
143
146
└── mobilenetv2_lrn_int8_ethosu.etdp
144
147
```
145
148
146
-
## What you have learned
149
+
## What you've learned and what's next
147
150
148
-
You have learned how the combined ExecuTorch extension and the separate TOSA and VGF adapters add artifact formats to Model Explorer. You have also seen how `.pte`, `.tosa`, `.vgf`, `.etrecord`, and `.etdp` files support Cortex-A, Cortex-M, Ethos-U, the ML extensions for Vulkan, and ExecuTorch profiling workflows.
151
+
You've now learned how the combined ExecuTorch extension and the separate TOSA and VGF adapters add artifact formats to Model Explorer. You've also seen how `.pte`, `.tosa`, `.vgf`, `.etrecord`, and `.etdp` files support Cortex-A, Cortex-M, Ethos-U, the ML extensions for Vulkan, and ExecuTorch profiling workflows.
149
152
150
-
Next, you will install Model Explorer and the Arm extensions, then open the first `.pte` artifact.
153
+
Next, you'll install Model Explorer and the Arm extensions.
0 commit comments