You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/learning-paths/mobile-graphics-and-gaming/nss-unreal/1-install-plugin.md
+2-18Lines changed: 2 additions & 18 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
---
2
2
title: Introduction to neural graphics and Neural Super Sampling (NSS)
3
-
weight: 2
3
+
weight: 3
4
4
5
5
### FIXED, DO NOT MODIFY
6
6
layout: learningpathall
@@ -27,20 +27,4 @@ Under the hood, Neural Super Sampling for Unreal Engine (NSS for UE) runs its ne
27
27
28
28
With these resources, you can seamlessly integrate NSS into any Unreal Engine project. The setup is designed to work with Vulkan as your rendering backend, and you don’t need to overhaul your workflow - just plug it in and start leveraging ML-powered upscaling right away. The technology is available as a source-code implementation that you will build with Visual Studio.
29
29
30
-
## Download required artifacts
31
-
32
-
Before you begin, download the required plugins and dependencies. These two repositories contain everything you need to set up NSS for Unreal Engine, including the VGF model file, and the ML Emulations Layers for Vulkan.
33
-
34
-
### 1. Download the NSS plugin
35
-
36
-
[**Neural Super Sampling Unreal Engine Plugin** → GitHub Repository](https://github.com/arm/neural-graphics-for-unreal)
37
-
38
-
Download the latest release package and extract it on your Windows machine. Use the folder corresponding to your Unreal version.
39
-
40
-
41
-
### 2. Download the runtime for ML Extensions for Vulkan
42
-
[**Unreal NNE Runtime RDG for ML Extensions for Vulkan** → GitHub Repository](https://github.com/arm/ml-extensions-for-vulkan-unreal-plugin).
43
-
44
-
Download and extract the release package on your Windows machine.
45
-
46
-
Once you’ve extracted both repositories, proceed to the next section to set up your development environment and enable the NSS plugin.
30
+
Proceed to the next section to set up your development environment and enable the NSS plugin.
To run NSS in your Unreal Engine project, install and configure the following:
11
+
To run NSS in your Unreal Engine project, you will need to install and configure the following:
12
12
13
13
-**Vulkan SDK**: Required for development of applications that use Vulkan, and to enable the Vulkan Configurator. The latter sets up the emulation layers used for running ML extensions for Vulkan workloads.
14
-
-**ML Emulation Layer for Vulkan**: These layers allows neural inference to run in emulation through Vulkan’s compute backend. They are activated by Vulkan Configurator to run with the Unreal Engine plugin. The layers are included in the `NNERuntimeRDGMLExtensionsForVulkan` zip you downloaded in a previous step. The Vulkan layer configuration activates the ML Emulation Layer for Vulkan, which implements the ML extensions for Vulkan.
15
-
-**NSS for Unreal Engine plugins**: These include `NSS` (the inference and model interface) and `NNERuntimeRDGMLExtensionsForVulkan` (which connects Unreal’s Render Dependency Graph to the ML extensions for Vulkan).
14
+
-**ML Emulation Layer for Vulkan**: These layers allows neural inference to run in emulation through Vulkan’s compute backend. They are activated by Vulkan Configurator to run with the Unreal Engine plugin. The Vulkan layer configuration activates the ML Emulation Layer for Vulkan, which implements the ML extensions for Vulkan.
15
+
-**NSS for Unreal Engine plugin**: You will download and integrate the plugin in the next section.
16
16
17
17
These components allow you to run NSS in Unreal Engine, using ML emulation layers for Vulkan for development and testing.
18
18
19
19
## Install Vulkan Software Development Kit
20
20
21
21
Go to the [Vulkan SDK landing page](https://vulkan.lunarg.com/sdk/home) and download the SDK Installer for Windows. After you have run the installer, you can move on to the next step.
22
22
23
+
## Download the emulation layers
24
+
25
+
For this Learning Path, a pre-built of package of the emulation layers is available. Download them by clicking the link.
26
+
27
+
[**ML Emulation Layer for Vulkan** → Arm Developer Downloads](https://www.arm.com/-/media/Files/developer/MLEmulationLayerForVulkan20251107)
28
+
29
+
Extract the downloaded file in a location of your choice.
30
+
31
+
23
32
## Configure Vulkan Layers
24
33
25
34
Vulkan Configurator is a program that will run the emulation layers in the background when you want to utilize them with Unreal Engine.
@@ -28,66 +37,20 @@ To emulate the ML extensions for Vulkan:
28
37
1. Launch the **Vulkan Configurator** (bundled with the Vulkan SDK) from the Windows **Start** menu.
29
38
2. In the **Apply a Vulkan Loader Configuration** list, right-click and choose **Create a new Configuration**. You can give the new configuration any name, for example `NSS`.
30
39
3. Navigate to the **Vulkan Layers Location** tab.
31
-
4. Append a user-defined path pointing to the emulation layers you downloaded in the previous section:
40
+
4. Append a user-defined path pointing to the emulation layers you downloaded in the previous step:
5. Switch back to the **Vulkan Loader Management** tab. Ensure the Graph layer is listed *above* the Tensor layer, and that you've set up the configuration scope as shown in the image.
Keep the Vulkan Configurator running to enable the emulation layers during engine execution.
44
52
{{% /notice %}}
45
53
46
-
## Enable NSS for Unreal Engine
47
-
48
-
1. Open Unreal Engine and create a new **Third Person** template project using the **C++** option.
49
-
50
-
![Unreal Engine project selection screen showing C++ Third Person template#center](./images/unreal_startup.webp"Figure 3: Create a new C++ project in Unreal Engine.")
51
-
52
-
2. Open the project in **Visual Studio**. Build it from source through **Build** > **Build Solution** or with `Ctrl+Shift+B`.
53
-
54
-
After the build is finished, open your project in Unreal Engine.
55
-
56
-
## Change Unreal’s Rendering Interface to Vulkan
57
-
58
-
By default, Unreal uses DirectX. Instead, you need to choose Vulkan as the default RHI:
![Project Settings with Vulkan selected as Default RHI under Targeted RHIs#center](./images/targeted_rhis.png"Figure 4: Set Vulkan as the default RHI.")
67
-
68
-
## Create the Plugins directory
69
-
70
-
Open your project directory in Windows explorer, and create a new folder called `Plugins`.
71
-
72
-
Enabling the plugin will look slightly different depending on what Unreal version you are using. Follow the steps corresponding to your setup.
73
-
74
-
## For Unreal 5.4 and 4.27
75
-
76
-
1. Copy the downloaded and extracted `.zip` archive into the new `Plugins` directory:
77
-
-`UE5.4` or `UE4.27`
78
-
2. Re-open Unreal Engine. When prompted, confirm plugin integration.
79
-
3. Rebuild your project in Visual Studio from source.
80
-
4. Verify the installation by opening the Plugins view in Unreal Engine, and making sure the checkbox is selected for `NSS`. Restart Unreal Engine if prompted.
81
-
82
-
## For Unreal 5.5
83
-
84
-
1. Copy the downloaded and extracted `.zip` archives into the new `Plugins` directory:
85
-
-`UE5.5`
86
-
-`NNERuntimeRDGMLExtensionsForVulkan`
87
-
2. Re-open Unreal Engine. When prompted, confirm plugin integration.
88
-
3. Rebuild your project in Visual Studio from source.
89
-
4. Verify the installation by opening the Plugins view in Unreal Engine, and making sure the checkbox is selected for both `NSS` and `NNERuntimeRDGMLExtensionsForVulkan` as shown. Restart Unreal Engine if prompted.
90
-
91
-
![Unreal Engine plugins window showing NSS and NNERuntimeRDGMLExtensionsForVulkan enabled#center](./images/verify_plugin_enabled.png"Figure 5: Verify plugin installation in Unreal Engine.")
54
+
With the ML emulation layers configured, Vulkan is now able to run machine learning workloads through the ML extensions for Vulkan. This enables neural inference to execute alongside the graphics pipeline during development, without requiring access to hardware with dedicated neural accelerators.
92
55
93
-
With the emulation layers and plugins configured, you're ready to run Neural Super Sampling in Unreal Engine. Continue to the next section to test the integration.
56
+
The next step is to integrate Neural Super Sampling into an Unreal Engine project. You’ll do this by installing the NSS plugin and creating a simple example game that lets you verify the setup and visualize the upscaling in action.
This repository's release package contains everything you need to set up NSS for Unreal Engine, including the plugin and the VGF model file.
11
+
12
+
[**Neural Super Sampling Unreal Engine Plugin** → GitHub Repository](https://github.com/arm/neural-graphics-for-unreal)
13
+
14
+
Download the latest release `.zip` and extract it on your Windows machine.
15
+
16
+
## Enable NSS for Unreal Engine
17
+
18
+
1. Open Unreal Engine and create a new **Third Person** template project using the **C++** option.
19
+
20
+
![Unreal Engine project selection screen showing C++ Third Person template#center](./images/unreal_startup.webp"Figure 3: Create a new C++ project in Unreal Engine.")
21
+
22
+
2. Open the project in **Visual Studio**. Build it from source through **Build** > **Build Solution** or with `Ctrl+Shift+B`.
23
+
24
+
After the build is finished, open your project in Unreal Engine.
25
+
26
+
## Change Unreal’s Rendering Interface to Vulkan
27
+
28
+
By default, Unreal uses DirectX. Instead, you need to choose Vulkan as the default RHI:
![Project Settings with Vulkan selected as Default RHI under Targeted RHIs#center](./images/targeted_rhis.png"Figure 4: Set Vulkan as the default RHI.")
37
+
38
+
## Create the Plugins directory
39
+
40
+
Open your project directory in Windows explorer, and create a new folder called `Plugins`.
41
+
42
+
![Windows File Explorer showing project directory with newly created Plugins folder alongside other project directories#center](./images/plugins_dir.png"Figure 5: The new Plugins directory")
43
+
44
+
## Enable the plugin
45
+
46
+
The plugin is included in the release package you downloaded in the previous section. The package contains a separate folder for each supported Unreal Engine version. Make sure you use the folder that matches your engine version (for example, UE5.5 for Unreal Engine 5.5).
47
+
48
+
1. Copy the appropriate engine-version folder from the extracted .zip archive into your project's Plugins directory.
49
+
2. Reopen Unreal Engine. When prompted, confirm that you want to enable the plugin.
50
+
3. Rebuild your project from source in Visual Studio.
51
+
4. Verify the installation by opening Edit → Plugins in Unreal Engine and confirming that the NSS plugin is enabled. Restart Unreal Engine if prompted.
52
+
![Unreal Engine plugins window showing NSS and NNERuntimeRDGMLExtensionsForVulkan enabled#center](./images/verify_plugin_enabled.png"Figure 5: Verify plugin installation in Unreal Engine.")
53
+
54
+
With the emulation layers and plugins configured, you're ready to run Neural Super Sampling in Unreal Engine. Continue to the next section to test the integration.
Copy file name to clipboardExpand all lines: content/learning-paths/mobile-graphics-and-gaming/nss-unreal/3-run-example.md
+32-2Lines changed: 32 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
---
2
2
title: Run the example
3
-
weight: 4
3
+
weight: 6
4
4
5
5
### FIXED, DO NOT MODIFY
6
6
layout: learningpathall
@@ -20,13 +20,43 @@ In **Project Settings > Plugins > Neural Super Sampling**, you can view and conf
20
20
21
21
Run `ShowFlag.VisualizeTemporalUpscaler 0` to disable the overview. To visualize the NSS model output in real-time, run the following command:
22
22
```
23
-
r.NSS.Debug 2
23
+
r.NSS.Debug 1
24
24
```
25
25
26
26
This will add real-time views showing the model’s processed outputs, such as predicted filter coefficients and feedback, as below. In the [Wrapping up section](/learning-paths/mobile-graphics-and-gaming/nss-unreal/6-wrapping-up), you will find links to learn more about what the debug outputs mean.
27
27
28
28
![Debug view of Neural Super Sampling model output in Unreal Engine#center](./images/nss_debug.png"Figure 6: Visualize NSS model debug output in real time.")
29
29
30
+
## Troubleshooting tips
31
+
32
+
If the example does not behave as expected, check the following common issues before continuing.
33
+
34
+
### Check for build issues in Visual Studio
35
+
- Build failures related to `AutomationTool`, `Gauntlet`, or other `*.Automation` projects can be ignored.
36
+
- Focus on whether the project itself, named as `<Your Project Name>Editor`, builds successfully.
37
+
38
+
### Check you Unreal Engine configuration
39
+
- Verify that Vulkan is selected as the **Default RHI**.
40
+
- Confirm the NSS plugin is enabled and that Unreal Engine was restarted after enabling it.
41
+
- Check **Project Settings → Plugins → Neural Super Sampling** to confirm a model is selected and active.
42
+
43
+
If the NSS plugin is enabled but appears to have no effect:
44
+
- Ensure Vulkan Configurator is running.
45
+
- Verify that the correct layer configuration is selected and active.
46
+
- Double-check that:
47
+
- The emulation layer path is correct
48
+
- The Graph layer is ordered above the Tensor layer
49
+
50
+
Refer back to the [emulation layer section](/learning-paths/mobile-graphics-and-gaming/nss-unreal/2-emulation-layer/) for the full Vulkan Configurator setup and validation steps.
51
+
52
+
### Check the software and hardware setup
53
+
- Confirm that the plugin version exactly matches your Unreal Engine version.
54
+
- Verify that your GPU driver supports Vulkan.
55
+
- Verify that your Visual Studio version aligns with the Unreal Engine version you are using.
56
+
- Return to the Visual Studio build output and inspect the logs carefully to identify the first reported error
57
+
58
+
Build or startup failures are often caused by version mismatches or missing dependencies.
59
+
30
60
## NSS model on Hugging Face
31
61
32
62
The model that powers NSS is published on Hugging Face in the [VGF format](https://github.com/arm/ai-ml-sdk-vgf-library). This format is optimized for inference via ML extensions for Vulkan.
0 commit comments