You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/learning-paths/mobile-graphics-and-gaming/neural-graphics-playbook-evaluate/01-from-neural-graphics-to-megalights.md
+8-8Lines changed: 8 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,19 +10,19 @@ layout: learningpathall
10
10
11
11
When we first started talking about neural graphics at Arm, the idea was pretty simple: if some parts of rendering can be done more efficiently with machine learning, that frees up budget for everything else. At SIGGRAPH 2025, we announced that [Arm GPUs will have dedicated neural accelerators (NX) in 2026](https://newsroom.arm.com/news/arm-announces-arm-neural-technology). Since then, we’ve worked hard to build the tools that developers will need to seamlessly evaluate and integrate neural graphics use cases.
12
12
13
-
On mobile, every rendering decision is a tradeoff. Resolution, lighting, effects, frame rate, thermals, battery life—they all compete for the same limited resources. And ultimately, those technical tradeoffs affect something even more important: the player experience. Visual quality matters, but so does responsiveness, stable performance, and how long a player can stay immersed in the game without the device heating up or throttling.
13
+
On mobile, every rendering decision is a tradeoff. Resolution, lighting, effects, frame rate, thermals, battery life all compete for the same limited resources. Ultimately, those technical tradeoffs affect something even more important: the player experience. Visual quality matters, but so does responsiveness, stable performance, and how long a player can stay immersed in the game without their device heating up or throttling.
14
14
15
-
[Neural Super Sampling (NSS)](https://developer.arm.com/community/arm-community-blogs/b/mobile-graphics-and-gaming-blog/posts/how-to-access-arm-neural-super-sampling) was our first attempt at shifting that balance. Upscaling itself isn’t new. Shader-based upscalers are already widely used across game engines and mobile games because rendering at lower resolution is one of the most effective ways to save performance. What interested us was whether machine learning could push that idea further.
15
+
[Neural Super Sampling (NSS)](https://developer.arm.com/community/arm-community-blogs/b/mobile-graphics-and-gaming-blog/posts/how-to-access-arm-neural-super-sampling) was our first attempt at shifting that balance. Upscaling isn’t new. Shader-based upscalers are already widely used across game engines and mobile games because rendering at lower resolution is one of the most effective ways to save performance. What interested us was whether machine learning could push that idea further.
16
16
17
-
In practice, NSS allowed us to work with more aggressive upscale ratios, like rendering at 540p and reconstructing to 1080p, while maintaining image quality that would be difficult to achieve with more traditional approaches. It also opened up better NX utilization in Arm GPUs, while reducing pressure on bandwidth and energy consumption, which are often just as important to mobile developers as raw frame rate.
17
+
In practice, NSS allowed us to work with more aggressive upscale ratios, such as rendering at 540p and reconstructing to 1080p, while maintaining image quality that would be difficult to achieve with more traditional approaches. It also opened up better NX utilization in Arm GPUs, while reducing pressure on bandwidth and energy consumption, which are often just as important to mobile developers as raw frame rate.
18
18
19
19
On mobile, bandwidth is expensive. Moving less data around the system can have a major impact on power efficiency and sustained performance. In many cases, that becomes just as valuable as the rendering speedup itself.
20
20
21
21
NSS was really a first step in exploring a bigger question: what happens if machine learning becomes a normal part of the rendering pipeline?
22
22
23
23
## From a single technique to something you can actually build on
24
24
25
-
This has resulted in the Neural Graphics Development Kit - a set of Vulkan based tools to support early experimentation of different neural graphics use cases that are production ready and efficient for mobile.
25
+
Exploring this question has resulted in the Neural Graphics Development Kit - a set of Vulkan based tools to support early experimentation of different neural graphics use cases that are production ready and efficient for mobile.
26
26
27
27
| Resource |
28
28
|----------|
@@ -33,19 +33,19 @@ This has resulted in the Neural Graphics Development Kit - a set of Vulkan based
33
33
|[ML Emulation Layer for Vulkan](https://github.com/arm/ai-ml-emulation-layer-for-vulkan)|
34
34
|[Neural Graphics Model Gym](https://github.com/arm/neural-graphics-model-gym)|
35
35
36
-
The development kit, the Unreal plugin, the sample content—those are provided so you can get started with minimal effort and validate how it behaves with your own content. And that’s really the point. It’s one thing to show a clean before-and-after in a controlled scene, and something else entirely when you’re dealing with dynamic lighting, fast camera movement, complex materials, and all the edge cases that come with real production conditions. That’s where things either hold up or start to break, and where you actually learn if a technique is useful.
36
+
The development kit, the Unreal plugin, and sample content are provided so you can get started with minimal effort and validate how it behaves with your own content. And that’s really the point. It’s one thing to show a clean before-and-after in a controlled scene, and something else entirely when you’re dealing with dynamic lighting, fast camera movement, complex materials, and all the edge cases that come with real production conditions. That’s where things either hold up or start to break, and where you actually learn if a technique is useful.
37
37
38
-
Since then, we’ve been expanding neural graphics use cases beyond NSS into applications of frame generation and ray denoising, while also making sure it can be integrated in real projects. This playbook will focus on the techniques we’ve worked on since then, but NSS remains relevant as the most straightforward way to start evaluating neural graphics in a real project.
38
+
Since then, we’ve been expanding neural graphics use cases beyond NSS into applications of frame generation and ray denoising, while also making sure it can be integrated in real projects. This playbook focuses on the techniques we’ve worked on since then, but NSS remains relevant as the most straightforward way to start evaluating neural graphics in a real project.
39
39
40
40
## Why this matters now
41
41
42
-
As engines like Unreal adopt more advanced techniques like ray tracing, new neural graphics use cases start to make sense in a practical way. Features like MegaLights make it possible to work with far more dynamic lights than before, without the cost scaling directly with the number of lights. The challenge is that these approaches rely on very low sample counts, which quickly introduce noise—especially on mobile.
42
+
As engines such as Unreal adopt more advanced techniques such as ray tracing, new neural graphics use cases start to make sense in a practical way. Features such as MegaLights make it possible to work with far more dynamic lights than before, without the cost scaling directly with the number of lights. The challenge is that these approaches rely on very low sample counts, which quickly introduce noise, especially on mobile.
43
43
44
44
Traditional denoisers and shader-based upscalers already help solve part of that problem. But as rendering pushes toward lower resolutions, fewer samples, and more dynamic lighting, reconstruction quality becomes harder to maintain. We’ve found that neural reconstruction allows us to push more aggressive upscale ratios, work with noisier inputs, and still produce stable, high-quality output. It also helps reduce bandwidth and energy pressure, which matters just as much as raw frame rate on mobile devices. More importantly, it starts making some of these newer rendering approaches practical on mobile in the first place.
45
45
46
46
## Enter Neural Dawn
47
47
48
-
Neural Dawn is how we validate the use cases are production ready.
48
+
Neural Dawn is how we validate that the use cases are production ready.
49
49
50
50
It’s a mobile game created with Unreal Engine using Megalights, combined with the neural graphics work we’ve been developing. The goal was to build a real game and see what actually happens when you try to use this end to end. Where the integration feels natural, where it doesn’t, what kind of gains you get, and what trade-offs show up along the way.
Copy file name to clipboardExpand all lines: content/learning-paths/mobile-graphics-and-gaming/neural-graphics-playbook-evaluate/02-neural-dawn.md
+9-9Lines changed: 9 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,5 @@
1
1
---
2
-
title: "About Neural Dawn"
2
+
title: About Neural Dawn
3
3
weight: 3
4
4
5
5
### FIXED, DO NOT MODIFY
@@ -18,14 +18,14 @@ First, it had to look good, like _really_ good. The lighting fidelity and scene
18
18
19
19
Second, it had to run efficiently. We targeted 60 FPS to meet modern expectations for smooth, responsive mobile gameplay.
20
20
21
-
And third, it had to show what becomes possible when neural graphics and Arm NX are used together with approaches like MegaLights, on mobile.
21
+
And third, it had to show what becomes possible when neural graphics and Arm NX are used together with approaches such as MegaLights, on mobile.
22
22
23
23
Instead of optimizing a traditional graphics pipeline, we built one that depends on neural reconstruction. Neural Dawn is built around MegaLights. We render at low resolution tracing a low amount of rays per pixel, then rely on:
24
24
25
25
- Neural Super Sampling Denoising (NSSD) to denoise and upscale the image
26
26
- Neural Frame Rate Upscaling (NFRU) to generate intermediate frames
27
27
28
-
So, the final result looks like a clean, high-resolution, smooth experience, even though we only rendered about 1/8 of the pixels directly.
28
+
So, the final result looks like a clean, high-resolution, smooth experience, even though we rendered only about one-eighth of the pixels directly.
29
29
30
30
Ultimately, Neural Dawn shows that neural graphics on mobile are production ready.
31
31
@@ -47,17 +47,17 @@ It also stacks well with other techniques. Overall, NFRU is a fairly predictable
47
47
48
48
### NSSD
49
49
50
-
NSSD is about making noisy, low-cost rendering usable. If you’ve worked with ray tracing, you’ve seen the issue. At real-time budgets, you can only afford a few rays per pixel. That gives you a very noisy image. Traditional shader- based denoisers help, but can struggle in challenging scenarios (for example, translucent particles like fire or mist).
50
+
NSSD is about making noisy, low-cost rendering usable. If you’ve worked with ray tracing, you’ve seen the issue. At real-time budgets, you can only afford a few rays per pixel. That gives you a very noisy image. Traditional shader- based denoisers help, but can struggle in challenging scenarios (for example, translucent particles such as fire or mist).
51
51
52
-
Unlike shader-based approaches, which often process diffuse, specular, shadows, and other signals separately, NSSD uses a neural network to reconstruct the final lighting result directly. This allows NSSD to perform denoising and upscaling together in a single pass, reducing the need for multiple specialized denoising stages.
52
+
Unlike shader-based approaches, which often process diffuse, specular, shadows, and other signals separately, NSSD uses a neural network to reconstruct the final lighting result directly. Doing so allows NSSD to perform denoising and upscaling together in a single pass, reducing the need for multiple specialized denoising stages.
53
53
54
-
In practical terms, NSSD lets you achieve high-quality rendering with Raytracing. We validated NSSD with Project Dawn with the following setup:
54
+
In practical terms, with NSSD, you can achieve high-quality rendering with Raytracing. We validated NSSD with Project Dawn with the following setup:
55
55
56
56
Rendering at 540p and upscaling to 1080p, tracing 1 ray per pixel with MegaLights.
57
57
58
58
Normally, lighting cost scales with scene complexity. More lights = more cost. With stochastic techniques, you can keep the cost roughly constant, but the image gets noisy. NSSD cleans up that noise after the fact and makes the content usable.
59
59
60
-
That’s what enables things like:
60
+
That’s what enables things such as:
61
61
62
62
- lots of dynamic lights in a scene
63
63
- soft shadows from all lights
@@ -76,6 +76,6 @@ NFRU and NSSD don’t compete—they sit at different levels:
76
76
77
77
You can use NFRU without touching your content.
78
78
79
-
You can’t really use NSSD that way—it depends on how your game looks and how much you’re willing to invest in getting good results.
79
+
You can’t really use NSSD that way — it depends on how your game looks and how much you’re willing to invest in getting good results.
80
80
81
-
That’s why most teams will start with NSS and NFRU, and treat NSSD as something to explore once they’re ready to go deeper. In the next section, you will learn what is really behind these technologies, and how we enable them through tooling.
81
+
That’s why most teams will start with NSS and NFRU, and treat NSSD as something to explore when they’re ready to go deeper. In the next section, you'll learn what is really behind these technologies, and how we enable them through tooling.
0 commit comments