Skip to content

Commit f9e4820

Browse files
authored
Merge pull request #3492 from V3x-0E/main
Open PR for Vulkan update
2 parents 6963894 + 2909fc3 commit f9e4820

13 files changed

Lines changed: 109 additions & 83 deletions

File tree

content/learning-paths/mobile-graphics-and-gaming/vulkan-ml-sample/1-introduction.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,11 @@ layout: learningpathall
88

99
## What is neural graphics, and why does it matter for real-time rendering?
1010

11-
Neural graphics combines real-time rendering with the power of machine learning to enhance visual quality and performance. By integrating ML techniques like neural upscaling directly into the GPU pipeline, developers can achieve next-gen fidelity and efficiency. This is especially valuable on mobile and embedded devices, where power efficiency is critical.
11+
Neural graphics combines real-time rendering with machine learning to improve visual quality and performance. Integrating ML techniques, such as neural upscaling, directly into the GPU pipeline can improve visual fidelity and efficiency. This approach is especially valuable on mobile and embedded devices, where power efficiency is critical.
1212

1313
## How do ML Extensions for Vulkan support neural graphics workloads?
1414

15-
Vulkan's data graph pipelines, introduced through the `VK_ARM_data_graph` and `VK_ARM_tensors` extensions, bring structured compute graph execution to the Vulkan API by introducing support for processing tensors. These pipelines are designed to execute ML inference workloads efficiently using SPIR-V-defined graphs.
15+
The `VK_ARM_data_graph` and `VK_ARM_tensors` extensions bring structured compute graph execution and tensor processing to the Vulkan API. Their data graph pipelines use SPIR-V-defined graphs to execute ML inference workloads efficiently.
1616

1717
To help developers adopt these features, the Tensor and Data Graph Vulkan Samples offer hands-on demonstrations.
1818

@@ -21,18 +21,18 @@ These samples address key challenges in ML integration, such as:
2121
- **Understanding graph-based compute with Vulkan**: See how compute workloads can be structured using explicit graph topologies.
2222
- **Demystifying ML inference in real-time rendering**: Learn how ML fits into the graphics pipeline.
2323

24-
Samples range from basic setups to more advanced features like tensor aliasing and compute shader integration.
24+
The samples range from basic configurations to more advanced features, such as tensor aliasing and compute shader integration.
2525

2626
This Learning Path walks you through setting up and running the first sample, `simple_tensor_and_data_graph`.
2727

28-
29-
### Why use ML Extensions for Vulkan for game and graphics development?
28+
### Why use ML Extensions for Vulkan in game and graphics development?
3029

3130
As a game developer, you've probably noticed the rising demand for smarter, more immersive graphics — but also the increasing strain on GPU resources, especially on mobile. Vulkan's traditional pipelines give you fine-grained control, but finding the right tooling to integrate machine learning has been a challenge. That’s where the new ML extensions for Vulkan come in.
3231

3332
Arm’s `VK_ARM_tensors` and `VK_ARM_data_graph` extensions give you native Vulkan support for executing neural networks on the GPU — using structured tensors and data graph pipelines. Instead of chaining compute shaders to simulate ML models, you can now express them as dataflow graphs in SPIR-V and run them more efficiently. This opens the door to using AI techniques right alongside the graphics pipeline.
3433

35-
And while ML has found success in image classification and LLMs, these extensions are designed from the ground up for gaming and graphics workloads — prioritizing predictable execution, GPU compatibility, and memory efficiency. With built-in support for tensor formats and pipeline sessions, the extensions are optimized for developers looking to blend traditional rendering with machine learning on Vulkan.
34+
35+
Although ML is widely used for image classification and large language models (LLMs), these extensions target game and graphics workloads. They prioritize predictable execution, GPU compatibility, and memory efficiency. Built-in support for tensor formats and pipeline sessions helps you combine traditional Vulkan rendering with machine learning.
3636

3737
Arm provides emulation layers for development on any modern Vulkan-capable hardware, and PyTorch support is available for model conversion workflows.
3838

Lines changed: 48 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Setting up the ML Emulation Layers for Vulkan
2+
title: Set up the ML Emulation Layers for Vulkan
33
weight: 3
44

55
### FIXED, DO NOT MODIFY
@@ -12,60 +12,85 @@ To run the Vulkan Samples, you first need to set up your development environment
1212

1313
This setup involves two main steps:
1414

15-
* Install the required tools on your development machine
16-
* Download the ML emulation layers for Vulkan, which simulate the `VK_ARM_data_graph` and `VK_ARM_tensors` extensions
15+
- Install the required tools on your development machine.
16+
- Download the ML Emulation Layers for Vulkan, which simulate the `VK_ARM_data_graph` and `VK_ARM_tensors` extensions.
1717

1818
## Install required tools for development
1919

20-
Before building and running the samples, ensure the following tools are installed on your development machine:
20+
Before building and running the samples, install these tools on your development machine:
2121

22-
- CMake (version 3.12 or later)
22+
- CMake (version 3.25 or later)
2323
- Python 3
2424
- Git
2525

26+
Install Git with the following command:
27+
28+
```bash
29+
winget install --id Git.Git -e
30+
```
31+
32+
In Visual Studio Code, create a folder for the project and install these Microsoft extensions:
33+
34+
- C/C++
35+
- CMake Tools
36+
37+
![C/C++ tools extension](images/c-ext.png)
38+
39+
![CMake Tools extension](images/cmake-ext.png)
40+
41+
Create a Python virtual environment and install the required packages:
42+
43+
```bash
44+
py -m venv .venv
45+
.\.venv\Scripts\Activate.ps1
46+
python -m pip install --upgrade pip
47+
python -m pip install ai-ml-emulation-layer-for-vulkan cmake
48+
```
49+
2650
To verify your installation, run the following commands:
2751

2852
```bash
2953
cmake --version
30-
python3 --version
54+
python --version
3155
git --version
56+
python -m pip show ai-ml-emulation-layer-for-vulkan
3257
```
3358

34-
Each command should print the installed version of the tool.
59+
Each command prints the installed version of the tool or shows where the appropriate files are located.
3560

36-
### Install Vulkan Software Development Kit
61+
### Install the Vulkan Software Development Kit
3762

38-
Go to the [Getting Started with the Windows Vulkan SDK](https://vulkan.lunarg.com/sdk/home) and download the SDK Installer for Windows. This installs **Vulkan Configurator** which is used to run the emulation layers.
63+
Download the Windows installer from [Getting Started with the Windows Vulkan SDK](https://vulkan.lunarg.com/sdk/home). The Vulkan Software Development Kit (SDK) includes **Vulkan Configurator**, which you use to run the emulation layers.
3964

4065
{{% notice Note %}}
41-
You must use a version >= 1.4.321 for the Vulkan SDK.
42-
{{% /notice %}}
4366

44-
## Download the emulation layers
67+
Use Vulkan SDK version 1.4.321 or later.
4568

46-
For this Learning Path, a pre-built of package of the emulation layers is available. Download them by clicking the link.
47-
48-
[ML Emulation Layer for Vulkan](https://www.arm.com/-/media/Files/developer/MLEmulationLayerForVulkan20251107)
69+
{{% /notice %}}
4970

50-
Extract the downloaded file in a location of your choice. You’re now ready to enable the emulation layers in Vulkan Configurator.
5171

5272
## Enable the emulation layers in Vulkan Configurator
5373

54-
Next, enable the emulation layers using the Vulkan Configurator to simulate the `VK_ARM_data_graph` and `VK_ARM_tensors` extensions. Open **Vulkan Configurator**.
74+
Open **Vulkan Configurator** to enable the layers that emulate the `VK_ARM_data_graph` and `VK_ARM_tensors` extensions.
5575

56-
Under the **Vulkan Layers Location** tab, add the path to your `MLEmulationLayerForVulkan` folder.
76+
On the **Vulkan Layers Available** tab, browse to the emulation layer folder in your virtual environment's `site-packages` directory:
5777

58-
Switch back to the **Vulkan Loader Management** tab. In the **Apply a Vulkan Loader Configuration** list, right-click and choose **Create a new Configuration**. You can give the new configuration any name, for example `tensor_and_data_graph`.
78+
```output
79+
.venv/Lib/site-packages/emulation_layer/deploy/bin
80+
```
81+
82+
![VL available](images/vl-available.png)
5983

60-
![Screenshot of the Vulkan Configurator showing the Vulkan Layers Location tab, where the emulation layer path (MLEmulationLayerForVulkan) is added to enable VK_ARM_data_graph and VK_ARM_tensors alt-text#center](./images/load_layers.png "Add emulation layers in Vulkan Configurator")
84+
On the **Vulkan Layers Configuration** tab, set **Vulkan Loader Configuration Scope** to **Any Running Vulkan Executable**. Select **New Configuration**, then enable the **Graph Emulation** and **Tensor Emulation** layers.
6185

6286
Ensure that the **Graph** layer is listed above the **Tensor** layer.
6387

64-
![Screenshot showing the Graph layer listed above the Tensor layer in the Vulkan Configurator. alt-text#center](./images/verify_layers.png "Reorder layers in Vulkan Configurator")
88+
![VL Config](images/vl-layers.png)
6589

6690
{{% notice Important %}}
91+
6792
Keep Vulkan Configurator running while you run the Vulkan samples.
68-
{{% /notice %}}
6993

70-
With the emulation layers configured, you're ready to build the Vulkan Samples. Continue to the next section to get started.
94+
{{% /notice %}}
7195

96+
With the emulation layers configured, you're ready to build the Vulkan Samples.

content/learning-paths/mobile-graphics-and-gaming/vulkan-ml-sample/3-first-sample.md

Lines changed: 18 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -8,36 +8,39 @@ layout: learningpathall
88

99
## Understand how the Simple Tensor and Data Graph sample works
1010

11-
The **Simple Tensor and Data Graph** sample is your starting point for working with the ML extensions for Vulkan. It demonstrates how to execute a simple neural network using a data graph pipelinespecifically, a 2D average pooling operation.
11+
The **Simple Tensor and Data Graph** sample is a starting point for working with ML Extensions for Vulkan. It demonstrates how to execute a simple neural network with a data graph pipeline, specifically a 2D average pooling operation.
1212

1313
## Clone the Vulkan Samples
1414

15-
With the environment set up, you can now grab the sample code. These examples sit in the the Khronos Group's repository.
15+
With the environment set up, clone the sample code from the Khronos Group repository:
1616

1717
```bash
1818
git clone --recurse-submodules https://github.com/KhronosGroup/Vulkan-Samples.git
1919
cd Vulkan-Samples
2020
```
2121

22-
This repository includes the framework and samples showcasing the ML extensions for Vulkan.
22+
The repository includes the framework and samples that demonstrate ML Extensions for Vulkan.
2323

2424
## Build the Vulkan Samples
2525

26-
You're now ready to compile the project. From the root of the repository:
26+
You're now ready to compile the project.
2727

2828
{{% notice Note %}}
29-
Be sure to run the commands in [Developer Mode](https://learn.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development#activate-developer-mode) to avoid permission issues.
29+
Enable [Developer Mode](https://learn.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development#activate-developer-mode) before running the commands to avoid permission issues.
3030
{{% /notice %}}
3131

3232
Generate Visual Studio project files using CMake:
33+
3334
```bash
3435
cmake -G "Visual Studio 17 2022" -A x64 -S . -Bbuild/windows
3536
```
36-
Finally, compile the `vulkan_samples` target in Release mode:
37+
38+
Compile the `vulkan_samples` target in Release mode:
3739

3840
```bash
3941
cmake --build build/windows --config Release --target vulkan_samples
4042
```
43+
4144
## Run the Simple Tensor and Data Graph sample
4245

4346
Run the built executable using the following command:
@@ -46,23 +49,23 @@ Run the built executable using the following command:
4649
build\windows\app\bin\Release\AMD64\vulkan_samples.exe sample simple_tensor_and_data_graph
4750
```
4851

49-
This should open a new window visualizing the operation. In this sample, a minimal Vulkan application sets up a data graph pipeline configured to process a small neural network.
52+
A new window opens and visualizes the operation. The sample uses a minimal Vulkan application to create a data graph pipeline that processes a small neural network.
5053

51-
The sample creates input and output tensors, binds them using descriptor sets and pipeline layouts, and supplies a SPIR-V module that defines the network operation. Finally, it records and dispatches commands to execute the pipeline and visualizes the results in real time. More details about what's going on under the hood can be found in the [documentation](https://arm-software.github.io/Vulkan-Samples/samples/extensions/tensor_and_data_graph/simple_tensor_and_data_graph/README.html).
54+
The sample creates input and output tensors, binds them with descriptor sets and pipeline layouts, and supplies a SPIR-V module that defines the network operation. It then records and dispatches commands to execute the pipeline and visualize the results in real time. For implementation details, see the [Simple Tensor and Data Graph documentation](https://arm-software.github.io/Vulkan-Samples/samples/extensions/tensor_and_data_graph/simple_tensor_and_data_graph/README.html).
5255

5356
## Summary and next steps
5457

55-
By running this sample, youve stepped through a complete Vulkan data graph pipeline powered by the ML extensions for Vulkan. Youve created tensors, set up descriptors, built a SPIR-V-encoded ML graph, and dispatched inference — all without needing custom shaders. This sets the foundation for neural graphics. As you explore the remaining samples, you’ll see how this core pattern extends into real-world graphics scenarios.
58+
By running this sample, you've stepped through a complete Vulkan data graph pipeline powered by ML Extensions for Vulkan. You've created tensors, set up descriptors, built a SPIR-V-encoded ML graph, and dispatched inference without custom shaders. This workflow provides a foundation for neural graphics and extends to more complex graphics scenarios.
5659

57-
As a next step, you can explore the remaining samples for the data graph pipeline. The documentation sits in each sample's directory, available under `samples/extensions/tensor_and_data_graph/` in the repository.
60+
You can also explore the remaining data graph pipeline samples. Each sample's documentation is in its directory under `samples/extensions/tensor_and_data_graph/`.
5861

59-
## Overview of Additional Samples
62+
## Overview of additional samples
6063

61-
| Sample Name | Description | Focus Area |
64+
| Sample name | Description | Focus area |
6265
|-------------------------------------|----------------------------------------------------------------------------------------------|------------------------------------------|
63-
| **Graph Constants** | Shows how to include constants like weights and biases into the data graph pipeline using tensors| Constant tensor injection |
66+
| **Graph Constants** | Shows how to add constants, such as weights and biases, to the data graph pipeline using tensors | Constant tensor injection |
6467
| **Compute Shaders with Tensors** | Demonstrates how to feed tensor data into or out of data graph pipelines using compute shaders | Shader interoperability |
6568
| **Tensor Image Aliasing** | Demonstrates tensor aliasing with Vulkan images to enable zero-copy workflows | Memory-efficient data sharing |
66-
| **Postprocessing with VGF** | Explores using VGF format, which contains SPIR-V, input, output and constant data used to run a data graph pipeline. | Neural network model |
69+
| **Postprocessing with VGF** | Explores how a VGF file packages SPIR-V with input, output, and constant data for a data graph pipeline | Neural network model |
6770

68-
Next, you'll review additional tools to help you work with ML extensions for Vulkan in your own development environment.
71+
Next, you'll review additional tools for working with ML Extensions for Vulkan in your development environment.

0 commit comments

Comments
 (0)