Skip to content

Commit a4a8fbe

Browse files
committed
Final updates for Vulkan ML Sample LP
1 parent a01cc48 commit a4a8fbe

12 files changed

Lines changed: 86 additions & 105 deletions

File tree

content/learning-paths/mobile-graphics-and-gaming/vulkan-ml-sample/1-introduction.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,11 @@ layout: learningpathall
88

99
## What is neural graphics, and why does it matter for real-time rendering?
1010

11-
Neural graphics combines real-time rendering with the power of machine learning to enhance visual quality and performance. By integrating ML techniques like neural upscaling directly into the GPU pipeline, developers can achieve next-gen fidelity and efficiency. This is especially valuable on mobile and embedded devices, where power efficiency is critical.
11+
Neural graphics combines real-time rendering with machine learning to improve visual quality and performance. Integrating ML techniques, such as neural upscaling, directly into the GPU pipeline can improve visual fidelity and efficiency. This approach is especially valuable on mobile and embedded devices, where power efficiency is critical.
1212

1313
## How do ML Extensions for Vulkan support neural graphics workloads?
1414

15-
Vulkan's data graph pipelines, introduced through the `VK_ARM_data_graph` and `VK_ARM_tensors` extensions, bring structured compute graph execution to the Vulkan API by introducing support for processing tensors. These pipelines are designed to execute ML inference workloads efficiently using SPIR-V-defined graphs.
15+
The `VK_ARM_data_graph` and `VK_ARM_tensors` extensions bring structured compute graph execution and tensor processing to the Vulkan API. Their data graph pipelines use SPIR-V-defined graphs to execute ML inference workloads efficiently.
1616

1717
To help developers adopt these features, the Tensor and Data Graph Vulkan Samples offer hands-on demonstrations.
1818

@@ -21,18 +21,18 @@ These samples address key challenges in ML integration, such as:
2121
- **Understanding graph-based compute with Vulkan**: See how compute workloads can be structured using explicit graph topologies.
2222
- **Demystifying ML inference in real-time rendering**: Learn how ML fits into the graphics pipeline.
2323

24-
Samples range from basic setups to more advanced features like tensor aliasing and compute shader integration.
24+
The samples range from basic configurations to more advanced features, such as tensor aliasing and compute shader integration.
2525

2626
This Learning Path walks you through setting up and running the first sample, `simple_tensor_and_data_graph`.
2727

28-
29-
### Why use ML Extensions for Vulkan for game and graphics development?
28+
### Why use ML Extensions for Vulkan in game and graphics development?
3029

3130
As a game developer, you've probably noticed the rising demand for smarter, more immersive graphics — but also the increasing strain on GPU resources, especially on mobile. Vulkan's traditional pipelines give you fine-grained control, but finding the right tooling to integrate machine learning has been a challenge. That’s where the new ML extensions for Vulkan come in.
3231

3332
Arm’s `VK_ARM_tensors` and `VK_ARM_data_graph` extensions give you native Vulkan support for executing neural networks on the GPU — using structured tensors and data graph pipelines. Instead of chaining compute shaders to simulate ML models, you can now express them as dataflow graphs in SPIR-V and run them more efficiently. This opens the door to using AI techniques right alongside the graphics pipeline.
3433

35-
And while ML has found success in image classification and LLMs, these extensions are designed from the ground up for gaming and graphics workloads — prioritizing predictable execution, GPU compatibility, and memory efficiency. With built-in support for tensor formats and pipeline sessions, the extensions are optimized for developers looking to blend traditional rendering with machine learning on Vulkan.
34+
35+
Although ML is widely used for image classification and large language models (LLMs), these extensions target game and graphics workloads. They prioritize predictable execution, GPU compatibility, and memory efficiency. Built-in support for tensor formats and pipeline sessions helps you combine traditional Vulkan rendering with machine learning.
3636

3737
Arm provides emulation layers for development on any modern Vulkan-capable hardware, and PyTorch support is available for model conversion workflows.
3838

Lines changed: 24 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Setting up the ML Emulation Layers for Vulkan
2+
title: Set up the ML Emulation Layers for Vulkan
33
weight: 3
44

55
### FIXED, DO NOT MODIFY
@@ -12,32 +12,33 @@ To run the Vulkan Samples, you first need to set up your development environment
1212

1313
This setup involves two main steps:
1414

15-
* Install the required tools on your development machine
16-
* Download the ML emulation layers for Vulkan, which simulate the `VK_ARM_data_graph` and `VK_ARM_tensors` extensions
15+
- Install the required tools on your development machine.
16+
- Download the ML Emulation Layers for Vulkan, which simulate the `VK_ARM_data_graph` and `VK_ARM_tensors` extensions.
1717

1818
## Install required tools for development
1919

20-
Before building and running the samples, ensure the following tools are installed on your development machine:
20+
Before building and running the samples, install these tools on your development machine:
2121

2222
- CMake (version 3.25 or later)
2323
- Python 3
2424
- Git
2525

26-
Git can be installed with the following command:
26+
Install Git with the following command:
2727

2828
```bash
2929
winget install --id Git.Git -e
3030
```
3131

32-
In VSCode create a new folder for this project then make sure you have the following extensions:
33-
* C/C++ From Microsoft
34-
* CMake Tools From Microsoft
32+
In Visual Studio Code, create a folder for the project and install these Microsoft extensions:
33+
34+
- C/C++
35+
- CMake Tools
3536

3637
![C/C++ tools extension](images/c-ext.png)
3738

38-
![Cmake tools extension](images/cmake-ext.png)
39+
![CMake Tools extension](images/cmake-ext.png)
3940

40-
The following commands will make a venv in your local version of python and install the required packages.
41+
Create a Python virtual environment and install the required packages:
4142

4243
```bash
4344
py -m venv .venv
@@ -55,39 +56,41 @@ git --version
5556
python -m pip show ai-ml-emulation-layer-for-vulkan
5657
```
5758

58-
Each command should print the installed version of the tool or show where the appropriate files are located.
59+
Each command prints the installed version of the tool or shows where the appropriate files are located.
5960

60-
### Install Vulkan Software Development Kit
61+
### Install the Vulkan Software Development Kit
6162

62-
Go to the [Getting Started with the Windows Vulkan SDK](https://vulkan.lunarg.com/sdk/home) and download the SDK Installer for Windows. This installs **Vulkan Configurator** which is used to run the emulation layers.
63+
Download the Windows installer from [Getting Started with the Windows Vulkan SDK](https://vulkan.lunarg.com/sdk/home). The Vulkan Software Development Kit (SDK) includes **Vulkan Configurator**, which you use to run the emulation layers.
6364

6465
{{% notice Note %}}
6566

66-
You must use a version >= 1.4.321 for the Vulkan SDK.
67+
Use Vulkan SDK version 1.4.321 or later.
6768

6869
{{% /notice %}}
6970

7071

7172
## Enable the emulation layers in Vulkan Configurator
7273

73-
Next, enable the emulation layers using the Vulkan Configurator to simulate the `VK_ARM_data_graph` and `VK_ARM_tensors` extensions. Open **Vulkan Configurator**.
74+
Open **Vulkan Configurator** to enable the layers that emulate the `VK_ARM_data_graph` and `VK_ARM_tensors` extensions.
7475

75-
Under the **Vulkan Layers Available** tab, add the path to your `Emulation layer` folder that can be found in the site packages of your venv folder select the bin folder.
76+
On the **Vulkan Layers Available** tab, browse to the emulation layer folder in your virtual environment's `site-packages` directory:
7677

78+
```output
79+
.venv/Lib/site-packages/emulation_layer/deploy/bin
80+
```
7781

78-
![VL available](images/vl-availble.png)
82+
![VL available](images/vl-available.png)
7983

80-
Under the **Vulkan layers Configuration** tab go through each setting in the top rigt except disable all Vulkan Layers and enable the Graph and Tensor Emulation
84+
On the **Vulkan Layers Configuration** tab, set **Vulkan Loader Configuration Scope** to **Any Running Vulkan Executable**. Select **New Configuration**, then enable the **Graph Emulation** and **Tensor Emulation** layers.
8185

8286
Ensure that the **Graph** layer is listed above the **Tensor** layer.
8387

84-
![VL Config](images/vl-config.png)
88+
![VL Config](images/verify_layers.png)
8589

8690
{{% notice Important %}}
8791

8892
Keep Vulkan Configurator running while you run the Vulkan samples.
8993

9094
{{% /notice %}}
9195

92-
With the emulation layers configured, you're ready to build the Vulkan Samples. Continue to the next section to get started.
93-
96+
With the emulation layers configured, you're ready to build the Vulkan Samples.

content/learning-paths/mobile-graphics-and-gaming/vulkan-ml-sample/3-first-sample.md

Lines changed: 18 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -8,59 +8,39 @@ layout: learningpathall
88

99
## Understand how the Simple Tensor and Data Graph sample works
1010

11-
The **Simple Tensor and Data Graph** sample is your starting point for working with the ML extensions for Vulkan. It demonstrates how to execute a simple neural network using a data graph pipelinespecifically, a 2D average pooling operation.
11+
The **Simple Tensor and Data Graph** sample is a starting point for working with ML Extensions for Vulkan. It demonstrates how to execute a simple neural network with a data graph pipeline, specifically a 2D average pooling operation.
1212

1313
## Clone the Vulkan Samples
1414

15-
With the environment set up, you can now grab the sample code. These examples sit in the the Khronos Group's repository.
15+
With the environment set up, clone the sample code from the Khronos Group repository:
1616

1717
```bash
1818
git clone --recurse-submodules https://github.com/KhronosGroup/Vulkan-Samples.git
1919
cd Vulkan-Samples
2020
```
2121

22-
This repository includes the framework and samples showcasing the ML extensions for Vulkan.
23-
24-
## Setting up Visual studio build tools
25-
26-
To build the Vulkan Samples on Windows using the Visual Studio generator, you still need the Microsoft C++ compiler and build tools.
27-
28-
Install the Visual Studio 2022 Build Tools with:
29-
30-
```bash
31-
winget install -e --id Microsoft.VisualStudio.2022.BuildTools
32-
```
33-
34-
When the installer opens, select:
35-
36-
Desktop development with C++
37-
38-
![image for the desktop tool required visual studio](images/vsmod.png)
39-
40-
Make sure these components are included:
41-
42-
MSVC v143 C++ build tools
43-
Windows 10/11 SDK
44-
C++ CMake tools for Windows
45-
22+
The repository includes the framework and samples that demonstrate ML Extensions for Vulkan.
4623

4724
## Build the Vulkan Samples
4825

49-
You're now ready to compile the project. From the root of the repository:
26+
You're now ready to compile the project.
5027

5128
{{% notice Note %}}
52-
Be sure to run the commands in [Developer Mode](https://learn.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development#activate-developer-mode) to avoid permission issues.
29+
Enable [Developer Mode](https://learn.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development#activate-developer-mode) before running the commands to avoid permission issues.
5330
{{% /notice %}}
5431

5532
Generate Visual Studio project files using CMake:
33+
5634
```bash
5735
cmake -G "Visual Studio 17 2022" -A x64 -S . -Bbuild/windows
5836
```
59-
Finally, compile the `vulkan_samples` target in Release mode:
37+
38+
Compile the `vulkan_samples` target in Release mode:
6039

6140
```bash
6241
cmake --build build/windows --config Release --target vulkan_samples
6342
```
43+
6444
## Run the Simple Tensor and Data Graph sample
6545

6646
Run the built executable using the following command:
@@ -69,23 +49,23 @@ Run the built executable using the following command:
6949
build\windows\app\bin\Release\AMD64\vulkan_samples.exe sample simple_tensor_and_data_graph
7050
```
7151

72-
This should open a new window visualizing the operation. In this sample, a minimal Vulkan application sets up a data graph pipeline configured to process a small neural network.
52+
A new window opens and visualizes the operation. The sample uses a minimal Vulkan application to create a data graph pipeline that processes a small neural network.
7353

74-
The sample creates input and output tensors, binds them using descriptor sets and pipeline layouts, and supplies a SPIR-V module that defines the network operation. Finally, it records and dispatches commands to execute the pipeline and visualizes the results in real time. More details about what's going on under the hood can be found in the [documentation](https://arm-software.github.io/Vulkan-Samples/samples/extensions/tensor_and_data_graph/simple_tensor_and_data_graph/README.html).
54+
The sample creates input and output tensors, binds them with descriptor sets and pipeline layouts, and supplies a SPIR-V module that defines the network operation. It then records and dispatches commands to execute the pipeline and visualize the results in real time. For implementation details, see the [Simple Tensor and Data Graph documentation](https://arm-software.github.io/Vulkan-Samples/samples/extensions/tensor_and_data_graph/simple_tensor_and_data_graph/README.html).
7555

7656
## Summary and next steps
7757

78-
By running this sample, youve stepped through a complete Vulkan data graph pipeline powered by the ML extensions for Vulkan. Youve created tensors, set up descriptors, built a SPIR-V-encoded ML graph, and dispatched inference — all without needing custom shaders. This sets the foundation for neural graphics. As you explore the remaining samples, you’ll see how this core pattern extends into real-world graphics scenarios.
58+
By running this sample, you've stepped through a complete Vulkan data graph pipeline powered by ML Extensions for Vulkan. You've created tensors, set up descriptors, built a SPIR-V-encoded ML graph, and dispatched inference without custom shaders. This workflow provides a foundation for neural graphics and extends to more complex graphics scenarios.
7959

80-
As a next step, you can explore the remaining samples for the data graph pipeline. The documentation sits in each sample's directory, available under `samples/extensions/tensor_and_data_graph/` in the repository.
60+
You can also explore the remaining data graph pipeline samples. Each sample's documentation is in its directory under `samples/extensions/tensor_and_data_graph/`.
8161

82-
## Overview of Additional Samples
62+
## Overview of additional samples
8363

84-
| Sample Name | Description | Focus Area |
64+
| Sample name | Description | Focus area |
8565
|-------------------------------------|----------------------------------------------------------------------------------------------|------------------------------------------|
86-
| **Graph Constants** | Shows how to include constants like weights and biases into the data graph pipeline using tensors| Constant tensor injection |
66+
| **Graph Constants** | Shows how to add constants, such as weights and biases, to the data graph pipeline using tensors | Constant tensor injection |
8767
| **Compute Shaders with Tensors** | Demonstrates how to feed tensor data into or out of data graph pipelines using compute shaders | Shader interoperability |
8868
| **Tensor Image Aliasing** | Demonstrates tensor aliasing with Vulkan images to enable zero-copy workflows | Memory-efficient data sharing |
89-
| **Postprocessing with VGF** | Explores using VGF format, which contains SPIR-V, input, output and constant data used to run a data graph pipeline. | Neural network model |
69+
| **Postprocessing with VGF** | Explores how a VGF file packages SPIR-V with input, output, and constant data for a data graph pipeline | Neural network model |
9070

91-
Next, you'll review additional tools to help you work with ML extensions for Vulkan in your own development environment.
71+
Next, you'll review additional tools for working with ML Extensions for Vulkan in your development environment.

0 commit comments

Comments
 (0)