Skip to content

Commit cb3c7d1

Browse files
committed
Updated Vulkan ML to ML Extensions for Vulkan
1 parent 599a6e7 commit cb3c7d1

5 files changed

Lines changed: 19 additions & 19 deletions

File tree

content/learning-paths/cross-platform/explore-model-artifacts-with-model-explorer/1-what-is-model-explorer.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ The artifacts covered in this learning path are:
2323
| --- | --- | --- | --- |
2424
| `.pte` | PTE adapter in the ExecuTorch extension | ExecuTorch program | Delegate regions, backend partitioning, work outside delegates, and the deployed ExecuTorch graph |
2525
| `.tosa` | Separate TOSA adapter | Compiler/backend intermediate representation | Lowered operators, tensor shapes, quantized types, graph splits, and missed optimization opportunities |
26-
| `.vgf` | Separate VGF adapter | Vulkan ML graph artifact | Inputs, outputs, constants, tensor metadata, graph connectivity, and SPIR-V graph modules |
26+
| `.vgf` | Separate VGF adapter | Graph artifact for the ML extensions for Vulkan | Inputs, outputs, constants, tensor metadata, graph connectivity, and SPIR-V graph modules |
2727
| `.etrecord` | ETRecord adapter in the ExecuTorch extension | Export-time profiling context | Graph structure, debug handles, operator names, and delegate metadata used to map runtime events back to graph nodes |
2828
| `.etdp` | ETDump data provider in the ExecuTorch extension | Runtime trace overlay | Timing data from a specific execution |
2929

@@ -73,12 +73,12 @@ TOSA is not inherently ExecuTorch-specific. TOSA is an intermediate representati
7373
-> VGF payload
7474
-> VGF-backend .pte
7575
-> standalone .vgf
76-
for Vulkan ML workflows
76+
for workflows using the ML extensions for Vulkan
7777
```
7878

7979
For VGF, the ExecuTorch Arm VGF backend uses the [Arm ML SDK Model Converter](https://github.com/arm/ai-ml-sdk-model-converter) to produce a VGF backend payload from TOSA. But the Arm ML SDK Model Converter could be used to convert `.tosa` files generated from a different flow, so VGF is also not specific to just ExecuTorch.
8080

81-
When ExecuTorch is used for VGF, a `.pte` is emitted as well. Use that VGF-backend `.pte` when you want to run through ExecuTorch. Use the standalone `.vgf` when you want to inspect or integrate the Vulkan ML artifact directly, such as in a neural graphics workflow.
81+
When ExecuTorch is used for VGF, a `.pte` is emitted as well. Use that VGF-backend `.pte` when you want to run through ExecuTorch. Use the standalone `.vgf` when you want to inspect or integrate the VGF artifact used with the ML extensions for Vulkan, such as in a neural graphics workflow.
8282

8383
If you have used the [Arm Neural Graphics Model Gym](https://github.com/arm/neural-graphics-model-gym), then under the hood you have been using ExecuTorch to export your neural graphics model to VGF. If you are interested in learning more, try out the [Fine-tune neural graphics using Model Gym](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/model-training-gym/#:~:text=Upon%20completion%20of%20this%20Learning,and%20train%20neural%20graphics%20models) learning path, which briefly introduces Model Explorer.
8484

@@ -102,7 +102,7 @@ A helpful glossary of different terms is provided below:
102102
| Delegate | A backend-specific execution path that handles supported parts of a graph. Unsupported parts can remain outside the delegate, and can run on a CPU path only when the deployed runtime includes compatible kernels. |
103103
| Kernel | The code that executes a model operator for a specific runtime or backend, such as a portable ExecuTorch kernel or a CMSIS-NN kernel. |
104104
| Flatbuffer | A compact binary serialization format. TOSA flatbuffers store TOSA graphs; `.pte` files use a FlatBuffer-based ExecuTorch program format. |
105-
| SPIR-V | Standard Portable Intermediate Representation - Vulkan. SPIR-V modules inside VGF files describe the Vulkan ML data graph used by the runtime. |
105+
| SPIR-V | Standard Portable Intermediate Representation - Vulkan. SPIR-V modules inside VGF files describe the data graph used by the ML extensions for Vulkan. |
106106
| ETRecord | An ExecuTorch export-time debug artifact that preserves graph and delegate metadata for profiling attribution. |
107107
| ETDump | An ExecuTorch runtime trace artifact that can record operator, delegate, backend, timing, and cycle-count events from execution. The first Model Explorer overlay used in this Learning Path focuses on timing data. |
108108

@@ -145,6 +145,6 @@ ml-model-artifacts/
145145

146146
## What you have learned
147147

148-
You have learned how the combined ExecuTorch extension and the separate TOSA and VGF adapters add artifact formats to Model Explorer. You have also seen how `.pte`, `.tosa`, `.vgf`, `.etrecord`, and `.etdp` files fit into Cortex-A, Cortex-M, Ethos-U, Vulkan ML, and ExecuTorch profiling workflows.
148+
You have learned how the combined ExecuTorch extension and the separate TOSA and VGF adapters add artifact formats to Model Explorer. You have also seen how `.pte`, `.tosa`, `.vgf`, `.etrecord`, and `.etdp` files support Cortex-A, Cortex-M, Ethos-U, the ML extensions for Vulkan, and ExecuTorch profiling workflows.
149149

150150
Next, you will install Model Explorer and the Arm extensions, then open the first `.pte` artifact.

content/learning-paths/cross-platform/explore-model-artifacts-with-model-explorer/6-inspect-tosa.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -128,9 +128,9 @@ This matters because TOSA provides a contract between the model frontend and the
128128

129129
## Inspect TOSA for VGF conversion
130130

131-
TOSA can also feed Vulkan ML workflows. The Arm ML SDK Model Converter takes TOSA as input, applies transforms and optimizations, lowers to SPIR-V graph IR, and packages the result into a VGF file.
131+
TOSA can also feed workflows that use the ML extensions for Vulkan. The Arm ML SDK Model Converter takes TOSA as input, applies transforms and optimizations, lowers to SPIR-V graph IR, and packages the result into a VGF file.
132132

133-
The examples in this section use a small neural upscaling model. It takes a low-resolution image-like tensor and produces a higher-resolution output, which makes it a useful compact example for Vulkan ML and neural graphics workflows.
133+
The examples in this section use a small neural upscaling model. It takes a low-resolution image-like tensor and produces a higher-resolution output, which makes it a useful compact example for workflows using the ML extensions for Vulkan and for neural graphics.
134134

135135
These artifacts were generated in the [Quantize neural upscaling models with ExecuTorch](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/quantize-neural-upscaling-models/) learning path. Go through that learning path if you want to learn how to generate the `.tosa` and `.vgf` files yourself, and how to apply post-training quantization (PTQ) and quantization-aware training (QAT) before export.
136136

@@ -158,10 +158,10 @@ The important difference is how the quantized model parameters were produced. PT
158158

159159
In Model Explorer, that difference is not likely to appear as a different graph shape. Look instead at tensor metadata, constants, scales, shifts, zero-points, and any downstream accuracy or runtime behavior.
160160

161-
These files are useful because they connect the TOSA view to the VGF view. TOSA shows the backend-neutral graph representation. The next section shows the VGF artifact produced for Vulkan ML and neural graphics integration.
161+
These files are useful because they connect the TOSA view to the VGF view. TOSA shows the backend-neutral graph representation. The next section shows the VGF artifact produced for integration with the ML extensions for Vulkan and neural graphics.
162162

163163
## What you have learned
164164

165165
You have inspected TOSA as the intermediate representation between model lowering and backend compilation or conversion. For Ethos-U, TOSA helps explain why FP32 does not delegate, why INT8 can produce a compact NPU region, and why an inserted unsupported operation can fragment the graph. You have also seen that TOSA is not ExecuTorch-specific and can be produced by other frontends.
166166

167-
Next, you will inspect VGF artifacts and see what the TOSA-to-VGF conversion produces for Vulkan ML workflows.
167+
Next, you will inspect VGF artifacts and see what the TOSA-to-VGF conversion produces for workflows using the ML extensions for Vulkan.

content/learning-paths/cross-platform/explore-model-artifacts-with-model-explorer/7-inspect-vgf.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,9 @@ weight: 7
77
layout: "learningpathall"
88
---
99

10-
## Inspect Vulkan ML artifacts
10+
## Inspect VGF artifacts for the ML extensions for Vulkan
1111

12-
VGF is the artifact used by Arm ML SDK for Vulkan workflows. It is useful in Vulkan ML and neural graphics workflows where you need to inspect the graph consumed by the Vulkan ML side of an application.
12+
VGF is the artifact used by Arm ML SDK for Vulkan workflows. It is useful in workflows using the ML extensions for Vulkan and in neural graphics applications where you need to inspect the graph consumed by the Vulkan runtime.
1313

1414
In the previous section, you inspected the TOSA artifacts for a small neural upscaling model, obtained from the [Quantize neural upscaling models with ExecuTorch](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/quantize-neural-upscaling-models/) learning path. In this section, you inspect the VGF and PTE artifacts produced from that flow.
1515

@@ -22,9 +22,9 @@ There are two related but different Model Explorer views in a VGF workflow:
2222
| View | Open with | Layer inspected | What to look out for |
2323
| --- | --- | --- | --- |
2424
| VGF-backend `.pte` | PTE adapter | ExecuTorch program and deployment container | Where does the VGF backend call appear? Is there surrounding quantize, dequantize, or CPU work? |
25-
| Standalone `.vgf` | VGF adapter | Vulkan ML backend graph | What operators, tensors, shapes, constants, descriptors, and graph connectivity will the Vulkan ML side consume? |
25+
| Standalone `.vgf` | VGF adapter | Backend graph for the ML extensions for Vulkan | What operators, tensors, shapes, constants, descriptors, and graph connectivity will the Vulkan runtime consume? |
2626

27-
Use the `.pte` file when you want to understand how ExecuTorch wraps and calls the VGF backend. Use the `.vgf` file when you want to inspect the Vulkan ML artifact directly. You can also open the VGF backend in model explorer from the `.pte` file and you will see it is the same view as opening the `.vgf` file directly.
27+
Use the `.pte` file when you want to understand how ExecuTorch wraps and calls the VGF backend. Use the `.vgf` file when you want to inspect the VGF artifact used with the ML extensions for Vulkan. You can also open the VGF backend in Model Explorer from the `.pte` file and see the same view as opening the `.vgf` file directly.
2828

2929
## Open the VGF-backend PTE
3030

@@ -67,7 +67,7 @@ Inspect the graph and answer:
6767
- Where do `Rescale` operations appear?
6868
- Which details are visible here that were hidden behind the `VgfBackend` node in the `.pte` view?
6969

70-
The VGF graph shows the backend-level structure consumed by the Vulkan ML workflow. For the PTQ upscaler, you should see a small graph with the same high-level structure you saw in TOSA:
70+
The VGF graph shows the backend-level structure consumed by the workflow using the ML extensions for Vulkan. For the PTQ upscaler, you should see a small graph with the same high-level structure you saw in TOSA:
7171

7272
- `Resize`
7373
- `Rescale`
@@ -80,7 +80,7 @@ The VGF graph shows the backend-level structure consumed by the Vulkan ML workfl
8080

8181
You should also see Vulkan tensor descriptor nodes such as `VK_DESCRIPTOR_TYPE_TENSOR_ARM`. The input descriptor has shape `[1, 16, 16, 3]` and format `VK_FORMAT_R8_SINT`. The graph produces an INT8 output with shape `[1, 32, 32, 3]`.
8282

83-
This is the view to use when you care about the Vulkan ML integration contract: tensor shapes, tensor formats, graph connectivity, quantized operators, and backend-visible layout choices.
83+
This is the view to use when you care about integration with the ML extensions for Vulkan: tensor shapes, tensor formats, graph connectivity, quantized operators, and backend-visible layout choices.
8484

8585
## Look at other artifacts
8686

@@ -96,8 +96,8 @@ You can use Model Explorer to inspect these graphs in the same way.
9696

9797
## What you have learned
9898

99-
You have inspected the same VGF workflow from two angles. The `.pte` view shows the ExecuTorch program and where it calls the VGF backend. The standalone `.vgf` view shows the Vulkan ML backend graph: tensor descriptors, graph connectivity, operator structure, quantization-related rescale operations, and the input/output contract used by a neural graphics application.
99+
You have inspected the same VGF workflow from two angles. The `.pte` view shows the ExecuTorch program and where it calls the VGF backend. The standalone `.vgf` view shows the backend graph for the ML extensions for Vulkan: tensor descriptors, graph connectivity, operator structure, quantization-related rescale operations, and the input/output contract used by a neural graphics application.
100100

101-
At this point, you have used Model Explorer to inspect the main static artifacts in this learning path: `.pte` files for the deployed ExecuTorch program, `.tosa` files for backend-ready intermediate graphs, and `.vgf` files for Vulkan ML integration. These views answer what was exported, lowered, converted, and packaged.
101+
At this point, you have used Model Explorer to inspect the main static artifacts in this learning path: `.pte` files for the deployed ExecuTorch program, `.tosa` files for backend-ready intermediate graphs, and `.vgf` files for integration with the ML extensions for Vulkan. These views answer what was exported, lowered, converted, and packaged.
102102

103103
The final section adds runtime profiling context. You will load ETRecord and ETDump data to see how exported graph structure connects to measured operator and delegate events during execution.

content/learning-paths/cross-platform/explore-model-artifacts-with-model-explorer/8-etrecord-etdump-overlays.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -290,4 +290,4 @@ ETRecord and ETDump add runtime context to static graphs. ETRecord gives you the
290290

291291
The OPT-125M profiles made the CPU case clear: the portable run stayed on native operators, while the XNNPACK run moved most of the work into delegate calls. The MobileNetV2 profiles showed the same pattern for Ethos-U. Native CPU execution, clean Ethos-U delegation, and fragmented delegation are much easier to tell apart once the runtime data is overlaid on the exported graph.
292292

293-
You have now completed the full artifact-inspection flow in this learning path. You started with `.pte` files to understand deployed ExecuTorch programs, moved through TOSA and VGF to inspect backend and Vulkan ML artifacts, and finished with ETRecord and ETDump overlays to connect the graph to runtime cost. The next steps page points to deeper workflows for generating, running, profiling, and optimizing your own models.
293+
You have now completed the full artifact-inspection flow in this learning path. You started with `.pte` files to understand deployed ExecuTorch programs, moved through TOSA and VGF to inspect backend artifacts for the ML extensions for Vulkan, and finished with ETRecord and ETDump overlays to connect the graph to runtime cost. The next steps page points to deeper workflows for generating, running, profiling, and optimizing your own models.

content/learning-paths/cross-platform/explore-model-artifacts-with-model-explorer/_index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ learning_objectives:
1313
- Open ExecuTorch .pte files and compare portable CPU, XNNPACK CPU, and Ethos-U artifacts
1414
- Use PTE visualization to reason about delegate regions, work outside delegates, graph fragmentation, and backend-specific changes
1515
- Inspect TOSA flatbuffers as an intermediate representation used by Arm compiler and backend workflows
16-
- Inspect VGF artifacts for Vulkan ML and neural graphics workloads
16+
- Inspect VGF artifacts used with the ML extensions for Vulkan and neural graphics workloads
1717
- Use ETRecord and ETDump overlays to connect exported graph structure with runtime profiling data
1818

1919
prerequisites:

0 commit comments

Comments
 (0)