You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/learning-paths/cross-platform/explore-model-artifacts-with-model-explorer/1-what-is-model-explorer.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -46,7 +46,7 @@ For Ethos-U, the ExecuTorch backend uses the Ethos-U Vela compiler to compile TO
46
46
47
47
TOSA is not inherently ExecuTorch-specific. TOSA is an intermediate representation (IR) that can sit between a model frontend and an Arm backend compiler or converter. ExecuTorch can lower supported graph partitions to TOSA, but another framework can also provide its own TOSA exporter and generate `.tosa` files.
48
48
49
-
```output
49
+
```text
50
50
PyTorch model
51
51
|
52
52
v
@@ -62,7 +62,7 @@ TOSA is not inherently ExecuTorch-specific. TOSA is an intermediate representati
Copy file name to clipboardExpand all lines: content/learning-paths/cross-platform/explore-model-artifacts-with-model-explorer/3-open-and-inspect-cortex-m-pte.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -48,17 +48,17 @@ To inspect the Cortex-M PTE, follow these steps:
48
48
49
49
1. In Model Explorer, open `ml-model-artifacts/pte/mv2_cortex_m.pte`.
50
50
51
-

51
+

52
52
53
53
2. Select **View selected models**.
54
54
55
-

55
+

56
56
57
57
The graph information panel shows the **op node count** and **layer count**. The **op node count** is the number of operator nodes in the graph. The **layer count** is the number of hierarchical graph components represented in the current view, not necessarily the number of neural network layers in the original model.
58
58
59
59
3. Expand the `forward` layer to view its operators. Select an operator, such as `cortex_m::quantize_per_tensor`, to see its attributes, inputs, and outputs in the node information panel.
60
60
61
-

61
+

62
62
63
63
When you inspect a `.pte` for the first time, focus on the higher-level graph information first:
Copy file name to clipboardExpand all lines: content/learning-paths/cross-platform/explore-model-artifacts-with-model-explorer/4-compare-portable-xnnpack-pte.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -57,7 +57,7 @@ Inspect the graph and look for the following:
57
57
58
58
The following is a small snippet image:
59
59
60
-

60
+

61
61
62
62
In the artifact, notice the following:
63
63
@@ -83,7 +83,7 @@ Inspect the graph and look for the following:
83
83
84
84
The backend has changed the execution plan:
85
85
86
-

86
+

87
87
88
88
- The top-level graph is smaller than the portable graph, with about 335 operator nodes instead of about 600.
89
89
- The graph contains many `XnnpackBackend` nodes. In this artifact, these represent the delegated regions that'll execute through XNNPACK.
@@ -92,7 +92,7 @@ The backend has changed the execution plan:
92
92
- Some `aten::` operators still remain at the top level, including shape, masking, normalization, and elementwise operations. These are the parts of the graph that stayed on the default ExecuTorch path.
93
93
- The graph isn't one single XNNPACK region. OPT-125M is a transformer with attention, masking, reshapes, and layout changes, so delegation is useful but fragmented into many backend regions.
94
94
95
-

95
+

96
96
97
97
This is the key difference to notice: XNNPACK doesn't replace the whole `.pte`. It captures supported subgraphs and leaves the rest of the program in ExecuTorch. In performance work, the balance between large delegated regions and remaining default-path operators is often more important than the raw number of delegate nodes.
Copy file name to clipboardExpand all lines: content/learning-paths/cross-platform/explore-model-artifacts-with-model-explorer/5-inspect-ethosu-pte.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -40,7 +40,7 @@ Inspect the graph and look for the following:
40
40
- The input and output shapes
41
41
- What this tells you about targeting Ethos-U without quantization
42
42
43
-

43
+

44
44
45
45
Ethos-U execution expects quantized integer workloads. This artifact was generated from an FP32 MobileNetV2 model, so the graph isn't in the form Ethos-U needs for NPU execution. As a result, the work remains outside an Ethos-U delegate region.
46
46
@@ -65,7 +65,7 @@ Inspect the graph and look for the following:
65
65
- Inputs and outputs that cross the delegate boundary
66
66
- Whether any visible work remains outside the delegated region
67
67
68
-

68
+

69
69
70
70
A clean delegated example should have most supported quantized work inside the Ethos-U region. In this example, the compute-heavy quantized CNN operators are suitable for Ethos-U and appear as one large delegated region.
71
71
@@ -91,7 +91,7 @@ Inspect the graph and look for the following:
91
91
- Unsupported operations or graph patterns outside the NPU delegate
92
92
- Quantize, dequantize, or non-delegated operators between delegate regions
93
93
94
-

94
+

95
95
96
96
Fragmentation often means that the model was only partly suitable for the target backend. Common causes include unsupported operators, unsupported tensor shapes, quantization issues, or target-specific compiler constraints.
Copy file name to clipboardExpand all lines: content/learning-paths/cross-platform/explore-model-artifacts-with-model-explorer/6-inspect-tosa.md
+5-5Lines changed: 5 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -39,7 +39,7 @@ Inspect the graph and look for the following:
39
39
- Whether the tensor types are FP32
40
40
- What convolution, add, clamp, and pooling patterns are visible
41
41
42
-

42
+

43
43
44
44
The artifact shows that the FP32 model can be represented in TOSA. In Model Explorer, you'll see a large graph with about 800 nodes. Most of the graph is made from constants and arithmetic around the MobileNetV2 operator structure: `CONV2D`, `DEPTHWISE_CONV2D`, `MUL`, `ADD`, `SUB`, `CLAMP`, one `AVG_POOL2D`, and a final `RESHAPE`.
45
45
@@ -54,7 +54,7 @@ Compare it with the FP32 TOSA graph:
54
54
- You'll see many `RESCALE` operations, which are common in quantized graphs.
55
55
- You'll still see the core CNN structure, including `CONV2D`, `DEPTHWISE_CONV2D`, `ADD`, `AVG_POOL2D`, and `RESHAPE`.
56
56
57
-

57
+

58
58
59
59
The INT8 graph is still a full MobileNetV2 graph, but the operator mix changes. You'll see fewer floating-point arithmetic nodes and many `RESCALE` nodes. These are used in quantized graphs to move values between quantization scales after integer operations. The convolution and depthwise convolution operators use INT32 accumulation, which is typical for INT8 convolution workloads.
60
60
@@ -74,9 +74,9 @@ Inspect both TOSA files and look for the following:
74
74
- The fragment that represents the graph region after the inserted LRN-related work
75
75
- Whether the fragment boundaries are consistent with the two `EthosUBackend` regions in the `.pte`
76
76
77
-

77
+

78
78
79
-

79
+

80
80
81
81
The first LRN TOSA file is the smaller fragment. It has two inputs, with shapes `[1, 1280, 7, 7]` and `[1, 1, 1280, 7, 7]`, and one `[1, 1000]` output. The file contains the later part of the graph after the inserted LRN-related work, including a small number of `RESCALE`, `TABLE`, `MUL`, `AVG_POOL2D`, and `CONV2D` operations.
82
82
@@ -133,7 +133,7 @@ These small upscaler graphs are INT8 TOSA artifacts. In Model Explorer, look for
133
133
-`RESCALE` operations from quantized arithmetic
134
134
- Similarities and differences between the PTQ and QAT artifacts
135
135
136
-

136
+

137
137
138
138
Don't expect a major visual difference between the PTQ and QAT TOSA graphs. They represent the same small upscaler architecture and were lowered to the same visible TOSA structure: 41 nodes, including one bilinear `RESIZE`, three `CONV2D` operations, four `RESCALE` operations, three `CONST_SHAPE` nodes, and constants for weights and quantization parameters.
Copy file name to clipboardExpand all lines: content/learning-paths/cross-platform/explore-model-artifacts-with-model-explorer/7-inspect-vgf.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -38,15 +38,15 @@ Inspect the graph and look for the following:
38
38
- Whether the top-level graph shows the internal upscaler operators
39
39
- What this view tells you about the ExecuTorch runtime path
40
40
41
-

41
+

42
42
43
43
In Model Explorer, the `.pte` looks compact. You'll see a `quantized_decomposed::quantize_per_tensor` node, a single `VgfBackend` delegate node, a `quantized_decomposed::dequantize_per_tensor` node, and graph inputs and outputs.
44
44
45
45
This is similar to the Ethos delegation. Apart from inputs and outputs and the quantize and dequantize operators, the model is completely delegated to the backend.
46
46
47
47
Expand the `VgfBackend` delegate graph:
48
48
49
-

49
+

50
50
51
51
Now, open the matched standalone VGF artifact `ml-model-artifacts/vgf/small_upscaler_ptq.vgf`.
Copy file name to clipboardExpand all lines: content/learning-paths/cross-platform/explore-model-artifacts-with-model-explorer/8-etrecord-etdump-overlays.md
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -191,7 +191,7 @@ Inspect the graph and profiling overlay, then look for the following:
191
191
- Repeated operators that dominate the profile
192
192
- How the runtime view compares with the portable `.pte` view that you inspected
193
193
194
-

194
+

195
195
196
196
This is a pure portable CPU run. The ETDump contains about 1,199 events, with `Method::execute` around 9,082 ms. There are no delegate calls. Almost all runtime is in native calls, with repeated `aten.addmm` events forming the main hotspot.
197
197
@@ -208,7 +208,7 @@ Compare it with the portable profile and look for the following:
208
208
- Native operators that still run outside the delegate
209
209
- How much faster this run is than the portable CPU baseline
210
210
211
-

211
+

212
212
213
213
The XNNPACK ETDump contains about 813 events, with `Method::execute` around 125 ms. The profile includes 98 delegate calls to `XnnpackBackend`. Delegate calls account for about 116.7 ms, while native calls account for about 8.5 ms.
214
214
@@ -227,7 +227,7 @@ Look for the following:
227
227
- A large `Method::execute` total
228
228
- Runtime behavior that matches the FP32 `.pte` view from the Ethos-U section
229
229
230
-

230
+

231
231
232
232
The ETDump contains about 309 events, with `Method::execute` around 395 million cycles. There are no delegate calls, and the native call sum accounts for almost all of the runtime. The profile is dominated by CPU-side convolution work.
233
233
@@ -242,7 +242,7 @@ Inspect the overlay and look for the following:
242
242
- Whether the largest runtime cost comes from the NPU event or from CPU-side quantization
243
243
- How this compares with the clean INT8 `.pte` view
244
244
245
-

245
+

246
246
247
247
This is the clean Ethos-U delegation case. The ETDump is small, with about 13 events. `Method::execute` is around 6.59 million cycles. There's one `EthosUBackend` delegate call, and only a small number of native calls.
248
248
@@ -259,7 +259,7 @@ Compare it with the clean INT8 profile and look for the following:
259
259
- How large the non-delegated native cost is compared with the NPU cost
260
260
- Whether quantize and dequantize events appear around delegate boundaries
261
261
262
-

262
+

263
263
264
264
Fragmentation can affect performance in different ways. Additional delegate boundaries can introduce data conversion, synchronization, or tensor movement. Non-delegated operators can also dominate runtime when their execution cost is much larger than the delegated work. Which effect matters most depends on the model, backend, and runtime.
265
265
@@ -273,4 +273,4 @@ You've learned that ETRecord and ETDump add runtime context to static graphs. ET
273
273
274
274
The OPT-125M profiles made the CPU case clear: the portable run stayed on native operators, while the XNNPACK run moved most of the work into delegate calls. The MobileNetV2 profiles showed the same pattern for Ethos-U. Native CPU execution, clean Ethos-U delegation, and fragmented delegation are much easier to tell apart when the runtime data is overlaid on the exported graph.
275
275
276
-
You can now explore deeper workflows for generating, running, profiling, and optimizing your own models.
276
+
You can now explore deeper workflows for generating, running, profiling, and optimizing your own models with Model Explorer.
0 commit comments