You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/learning-paths/mobile-graphics-and-gaming/vulkan-ml-sample/1-introduction.md
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,11 +8,11 @@ layout: learningpathall
8
8
9
9
## What is neural graphics, and why does it matter for real-time rendering?
10
10
11
-
Neural graphics combines real-time rendering with the power of machine learning to enhance visual quality and performance. By integrating ML techniques like neural upscaling directly into the GPU pipeline, developers can achieve next-gen fidelity and efficiency. This is especially valuable on mobile and embedded devices, where power efficiency is critical.
11
+
Neural graphics combines real-time rendering with machine learning to improve visual quality and performance. Integrating ML techniques, such as neural upscaling, directly into the GPU pipelinecan improve visual fidelity and efficiency. This approach is especially valuable on mobile and embedded devices, where power efficiency is critical.
12
12
13
13
## How do ML Extensions for Vulkan support neural graphics workloads?
14
14
15
-
Vulkan's data graph pipelines, introduced through the `VK_ARM_data_graph` and `VK_ARM_tensors` extensions, bring structured compute graph execution to the Vulkan API by introducing support for processing tensors. These pipelines are designed to execute ML inference workloads efficiently using SPIR-V-defined graphs.
15
+
The `VK_ARM_data_graph` and `VK_ARM_tensors` extensions bring structured compute graph execution and tensor processing to the Vulkan API. Their data graph pipelines use SPIR-V-defined graphs to execute ML inference workloads efficiently.
16
16
17
17
To help developers adopt these features, the Tensor and Data Graph Vulkan Samples offer hands-on demonstrations.
18
18
@@ -21,18 +21,18 @@ These samples address key challenges in ML integration, such as:
21
21
-**Understanding graph-based compute with Vulkan**: See how compute workloads can be structured using explicit graph topologies.
22
22
-**Demystifying ML inference in real-time rendering**: Learn how ML fits into the graphics pipeline.
23
23
24
-
Samples range from basic setups to more advanced features like tensor aliasing and compute shader integration.
24
+
The samples range from basic configurations to more advanced features, such as tensor aliasing and compute shader integration.
25
25
26
26
This Learning Path walks you through setting up and running the first sample, `simple_tensor_and_data_graph`.
27
27
28
-
29
-
### Why use ML Extensions for Vulkan for game and graphics development?
28
+
### Why use ML Extensions for Vulkan in game and graphics development?
30
29
31
30
As a game developer, you've probably noticed the rising demand for smarter, more immersive graphics — but also the increasing strain on GPU resources, especially on mobile. Vulkan's traditional pipelines give you fine-grained control, but finding the right tooling to integrate machine learning has been a challenge. That’s where the new ML extensions for Vulkan come in.
32
31
33
32
Arm’s `VK_ARM_tensors` and `VK_ARM_data_graph` extensions give you native Vulkan support for executing neural networks on the GPU — using structured tensors and data graph pipelines. Instead of chaining compute shaders to simulate ML models, you can now express them as dataflow graphs in SPIR-V and run them more efficiently. This opens the door to using AI techniques right alongside the graphics pipeline.
34
33
35
-
And while ML has found success in image classification and LLMs, these extensions are designed from the ground up for gaming and graphics workloads — prioritizing predictable execution, GPU compatibility, and memory efficiency. With built-in support for tensor formats and pipeline sessions, the extensions are optimized for developers looking to blend traditional rendering with machine learning on Vulkan.
34
+
35
+
Although ML is widely used for image classification and large language models (LLMs), these extensions target game and graphics workloads. They prioritize predictable execution, GPU compatibility, and memory efficiency. Built-in support for tensor formats and pipeline sessions helps you combine traditional Vulkan rendering with machine learning.
36
36
37
37
Arm provides emulation layers for development on any modern Vulkan-capable hardware, and PyTorch support is available for model conversion workflows.
To verify your installation, run the following commands:
27
51
28
52
```bash
29
53
cmake --version
30
-
python3 --version
54
+
python --version
31
55
git --version
56
+
python -m pip show ai-ml-emulation-layer-for-vulkan
32
57
```
33
58
34
-
Each command should print the installed version of the tool.
59
+
Each command prints the installed version of the tool or shows where the appropriate files are located.
35
60
36
-
### Install Vulkan Software Development Kit
61
+
### Install the Vulkan Software Development Kit
37
62
38
-
Go to the [Getting Started with the Windows Vulkan SDK](https://vulkan.lunarg.com/sdk/home) and download the SDK Installer for Windows. This installs **Vulkan Configurator** which is used to run the emulation layers.
63
+
Download the Windows installer from [Getting Started with the Windows Vulkan SDK](https://vulkan.lunarg.com/sdk/home). The Vulkan Software Development Kit (SDK) includes **Vulkan Configurator**, which you use to run the emulation layers.
39
64
40
65
{{% notice Note %}}
41
-
You must use a version >= 1.4.321 for the Vulkan SDK.
42
-
{{% /notice %}}
43
66
44
-
## Download the emulation layers
67
+
Use Vulkan SDK version 1.4.321 or later.
45
68
46
-
For this Learning Path, a pre-built of package of the emulation layers is available. Download them by clicking the link.
47
-
48
-
[ML Emulation Layer for Vulkan](https://www.arm.com/-/media/Files/developer/MLEmulationLayerForVulkan20251107)
69
+
{{% /notice %}}
49
70
50
-
Extract the downloaded file in a location of your choice. You’re now ready to enable the emulation layers in Vulkan Configurator.
51
71
52
72
## Enable the emulation layers in Vulkan Configurator
53
73
54
-
Next, enable the emulation layers using the Vulkan Configurator to simulate the `VK_ARM_data_graph` and `VK_ARM_tensors` extensions. Open **Vulkan Configurator**.
74
+
Open **Vulkan Configurator** to enable the layers that emulate the `VK_ARM_data_graph` and `VK_ARM_tensors` extensions.
55
75
56
-
Under the **Vulkan Layers Location** tab, add the path to your `MLEmulationLayerForVulkan` folder.
76
+
On the **Vulkan Layers Available** tab, browse to the emulation layer folder in your virtual environment's `site-packages` directory:
57
77
58
-
Switch back to the **Vulkan Loader Management** tab. In the **Apply a Vulkan Loader Configuration** list, right-click and choose **Create a new Configuration**. You can give the new configuration any name, for example `tensor_and_data_graph`.

84
+
On the **Vulkan Layers Configuration** tab, set **Vulkan Loader Configuration Scope** to **Any Running Vulkan Executable**. Select **New Configuration**, then enable the **Graph Emulation** and **Tensor Emulation** layers.
61
85
62
86
Ensure that the **Graph** layer is listed above the **Tensor** layer.
63
87
64
-

88
+

65
89
66
90
{{% notice Important %}}
91
+
67
92
Keep Vulkan Configurator running while you run the Vulkan samples.
68
-
{{% /notice %}}
69
93
70
-
With the emulation layers configured, you're ready to build the Vulkan Samples. Continue to the next section to get started.
94
+
{{% /notice %}}
71
95
96
+
With the emulation layers configured, you're ready to build the Vulkan Samples.
Copy file name to clipboardExpand all lines: content/learning-paths/mobile-graphics-and-gaming/vulkan-ml-sample/3-first-sample.md
+18-15Lines changed: 18 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,36 +8,39 @@ layout: learningpathall
8
8
9
9
## Understand how the Simple Tensor and Data Graph sample works
10
10
11
-
The **Simple Tensor and Data Graph** sample is your starting point for working with the ML extensions for Vulkan. It demonstrates how to execute a simple neural network using a data graph pipeline — specifically, a 2D average pooling operation.
11
+
The **Simple Tensor and Data Graph** sample is a starting point for working with ML Extensions for Vulkan. It demonstrates how to execute a simple neural network with a data graph pipeline, specifically a 2D average pooling operation.
12
12
13
13
## Clone the Vulkan Samples
14
14
15
-
With the environment set up, you can now grab the sample code. These examples sit in the the Khronos Group's repository.
15
+
With the environment set up, clone the sample code from the Khronos Group repository:
This repository includes the framework and samples showcasing the ML extensions for Vulkan.
22
+
The repository includes the framework and samples that demonstrate ML Extensions for Vulkan.
23
23
24
24
## Build the Vulkan Samples
25
25
26
-
You're now ready to compile the project. From the root of the repository:
26
+
You're now ready to compile the project.
27
27
28
28
{{% notice Note %}}
29
-
Be sure to run the commands in [Developer Mode](https://learn.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development#activate-developer-mode) to avoid permission issues.
29
+
Enable [Developer Mode](https://learn.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development#activate-developer-mode) before running the commands to avoid permission issues.
30
30
{{% /notice %}}
31
31
32
32
Generate Visual Studio project files using CMake:
33
+
33
34
```bash
34
35
cmake -G "Visual Studio 17 2022" -A x64 -S . -Bbuild/windows
35
36
```
36
-
Finally, compile the `vulkan_samples` target in Release mode:
37
+
38
+
Compile the `vulkan_samples` target in Release mode:
This should open a new window visualizing the operation. In this sample, a minimal Vulkan application sets up a data graph pipeline configured to process a small neural network.
52
+
A new window opens and visualizes the operation. The sample uses a minimal Vulkan application to create a data graph pipeline that processes a small neural network.
50
53
51
-
The sample creates input and output tensors, binds them using descriptor sets and pipeline layouts, and supplies a SPIR-V module that defines the network operation. Finally, it records and dispatches commands to execute the pipeline — and visualizes the results in real time. More details about what's going on under the hood can be found in the [documentation](https://arm-software.github.io/Vulkan-Samples/samples/extensions/tensor_and_data_graph/simple_tensor_and_data_graph/README.html).
54
+
The sample creates input and output tensors, binds them with descriptor sets and pipeline layouts, and supplies a SPIR-V module that defines the network operation. It then records and dispatches commands to execute the pipeline and visualize the results in real time. For implementation details, see the [Simple Tensor and Data Graph documentation](https://arm-software.github.io/Vulkan-Samples/samples/extensions/tensor_and_data_graph/simple_tensor_and_data_graph/README.html).
52
55
53
56
## Summary and next steps
54
57
55
-
By running this sample, you’ve stepped through a complete Vulkan data graph pipeline powered by the ML extensions for Vulkan. You’ve created tensors, set up descriptors, built a SPIR-V-encoded ML graph, and dispatched inference — all without needing custom shaders. This sets the foundation for neural graphics. As you explore the remaining samples, you’ll see how this core pattern extends into real-world graphics scenarios.
58
+
By running this sample, you've stepped through a complete Vulkan data graph pipeline powered by ML Extensions for Vulkan. You've created tensors, set up descriptors, built a SPIR-V-encoded ML graph, and dispatched inference without custom shaders. This workflow provides a foundation for neural graphics and extends to more complex graphics scenarios.
56
59
57
-
As a next step, you can explore the remaining samples for the data graph pipeline. The documentation sits in each sample's directory, available under `samples/extensions/tensor_and_data_graph/` in the repository.
60
+
You can also explore the remaining data graph pipeline samples. Each sample's documentation is in its directoryunder `samples/extensions/tensor_and_data_graph/`.
|**Graph Constants**| Shows how to include constants like weights and biases into the data graph pipeline using tensors| Constant tensor injection |
66
+
|**Graph Constants**| Shows how to add constants, such as weights and biases, to the data graph pipeline using tensors| Constant tensor injection |
64
67
|**Compute Shaders with Tensors**| Demonstrates how to feed tensor data into or out of data graph pipelines using compute shaders | Shader interoperability |
65
68
|**Tensor Image Aliasing**| Demonstrates tensor aliasing with Vulkan images to enable zero-copy workflows | Memory-efficient data sharing |
66
-
|**Postprocessing with VGF**| Explores using VGF format, which contains SPIR-V, input, output and constant data used to run a data graph pipeline. | Neural network model |
69
+
|**Postprocessing with VGF**| Explores how a VGF file packages SPIR-V with input, output, and constant data for a data graph pipeline | Neural network model |
67
70
68
-
Next, you'll review additional tools to help you work with ML extensions for Vulkan in your own development environment.
71
+
Next, you'll review additional tools for working with ML Extensions for Vulkan in your development environment.
0 commit comments