WebGPU: Support for custom materials; simple glTF-compliant material

[TheBlek]

• Adds webgpu_viewerTest.html example which is almost 1 to 1 with viewerTest.html example
• Adds pbr material bsdf which is ported from glsl version
• Packs color, normal, tangent and uv into attributes buffer
• Adds support for sampling material attributes from textures
• Wavefront: pcgState is passed between stages for good distribution of pseudo random numbers

Transmissive materials need support from the light transport algorithm and will not work correctly for now. I think they can be addressed later.

@gkjohnson Should I divide this PR into multiple for better readability?

[gkjohnson]

Thanks for this - with the last couple PRs there will be quite a few conflicts here. I'm not sure how easy it is but if possible it would be nice to break this up into smaller chunks for review if possible.

A few things on my mind as we move into BRDF implementation - there are definitely some improvements that can be made to the handling of BRDF blending from the previous implementation. GLTF, which is at least the base model I'd like to follow here, specifies the BRDF blending hierarchy here and some of the extensions like irridescence specify their terms internally, as well. It's been a long time since I've looked at the material code but it might be a good opportunity to look at that and try to match some of the gltf notation for consistency. I'm wondering if it makes sense to try to get a lot of the material qualities and BRDFs passing some of the more complicated tests (like the furnace test) before moving onto things like next event sampling and bi directional path tracing? Something to think about.

These aren't things I'm expected in a first PR, to be clear. But just things that are on my mind and to consider to what would come next.

[TheBlek]

Decided to split this into multiple PRs:

1. Passing material information + constructing SurfaceRecord + bsdfSample interface. See WebGPU: Pass full materials into shaders #722
2. Texture support for materials WebGPU: Add support for textures #732
3. BRDF implementation
4. Randomness fix in wavefront. See WebGPU: Carry pcg generator state through stage #728

Let me know if you think it could/should be done differently.

Once (1) merges, other PRs will be unblocked. This PR's focus will be about actual BRDF implementation. Do you think its better to build up the implementation over multiple iterations? First, base GLTF spec, then adding extensions?

Could you expand on testing? Do you usually use some other implementation (blender?) as ground truth? I've read about furnace test but was unable to find them in examples. Does it need to be constructed manually?

[gkjohnson]

Do you think its better to build up the implementation over multiple iterations? First, base GLTF spec, then adding extensions?

This seems good to me. I think this is a good opportunity to build this up again and validate it against a known specification while developing.

Could you expand on testing? Do you usually use some other implementation (blender?) as ground truth?

The viewer test demo (which you've already done the leg work for migrating here) is probably the best way to test these things. For some background, the viewer test page allows for loading and rendering the litany of gltf-sample-assets models which should cover the broad set of gltf features.

For comparisons, there's an update-screenshots command in the package.json that will render and save out every screenshot from the model viewer page which I had been previously manually committing to the "screenshots" branch (Github CI had some issues rendering screenshots awhile go but I haven't tested in a while). These screenshots are then viewable in the screenshotList example page which you compare against the model-viewer CI screenshots.

You'll notice that there is a dropdown with other comparison options (babylon, stellar etc - some of which are path tracers as well) that are currently broken. The model-viewer repo used to include a list of comparisons from a number of different projects (including this one). These screenshots have since been moved to the glTF-Render-Fidelity repo (related modelviewer PR: google/model-viewer#4779) so we'll want to update those pointers, as well.

This set of screenshots is good litmus test for ensuring we're covering the right features and should be a good way to catch any inconsistencies. A good long term goal would be to see if we can get our screenshots updated in the Khronos gltf-render-fidelity repo, as well, but one step at a time 😁

[gkjohnson]

I forgot to mention that the material database demo can also be useful for testing against materials and Blender screenshots from physicallybased.info, which can be good for evaluating against something like cycles.

[TheBlek]

Also, regarding this comment, note that because the spheres will be reflecting off of each other, it's possible that one sphere may look like it has more energy loss than is present because it's reflecting a sphere next to it that has more significant energy loss, if that makes sense.

I understand that and tried to check fully metallic and rough sphere by itself and it still showed some loss.

Rereading this, you might be referring to these artifacts? (I thought you were referring to some artifact from fresnel blending). I have accentuated them here by reducing the geometric resolution of the spheres.

Yes, sorry for not attaching an image to showcase them. First noticed them on a model, as very bright white highlights. It seems that this was a problem with precision of complex trigonometric formulas. It took simplifying all of them to get rid of those bright highlights. Went back to webgl implementation to check if there were any problems. Turning off nan checks turns a lot of the same pixels that were white in webgpu into black (presumably NaNs). My guess is that trig is handled a little differently in webgl vs webgpu and causes different behavior.

Initially I though filteredRoughness may help here, but it should not, so it could wait for another PR imo.

Also: saved turquin compensation texture into an image file to load. Since its static for a material, we can package the default texture to not calculate it on the device. Let me know if you would like that to be handled differently.

I'm thinking we can handle some of these comments and get the rest of the non-transparent / transmissive PBR pipeline working (test emission, iridescence, etc) and get this merged.

To be clear: Do you want other gltf extensions handled in this PR? Like iridescence or clearcoat?

[gkjohnson]

To be clear: Do you want other gltf extensions handled in this PR? Like iridescence or clearcoat?

Either way! Lets handle these in subsequent PRs since this is just about ready. Very excited to see this 😁

One thing I will note is that I am seeing these artifacts when toggling between wavefront and megakernel path tracers. Do you see it on your machine?

--

And just to make a list of remaining PBR features to tackle - is there anything missing here?

• Clearcoat support
• Sheen support
• Transmission support
• Iridescence
• Volume attenuation
• Opacity (handled in #751)
• Alpha cutout
• Anisotropy
• Dielectric / Transmission multiscatter compensation
• Material sided-ness

[gkjohnson]

This isn't a method we can use for importing textures - it may work for vite but it won't work across the board with other bundlers. A typical method is to use a data url or to use something like this which is pretty reliable across bundlers I believe:

const TURQUIN_URL = new URL( '../../textures/turquinMetal.png', import.meta.url ).toString();

[gkjohnson]

Do we need to add "constants" to the dependencies of this function?

[gkjohnson]

We should add "pcgRand2" to the dependencies here, as well.

[gkjohnson]

I'm thinking this should be "break" rather than "return"? Otherwise we don't increment that we took a sample or save any accumulated emission.

[gkjohnson]

I'm also thinking this should be moved before the above "invalid scatter" check. Even if the path is ultimately terminated we should still accumulate the emission contribution.

[gkjohnson]

Small catch that may be hard to find later - this should be "clearcoatNormal" rather than "normal"

[TheBlek]

Good catch!

[TheBlek]

One thing I will note is that I am seeing these artifacts when toggling between wavefront and megakernel path tracers. Do you see it on your machine?

Yes. That is weird, looks like there are some problems with low res samples and/or problem with blending them. Looking into it.

[gkjohnson]

Yes. That is weird, looks like there are some problems with low res samples and/or problem with blending them. Looking into it.

It looks like this issue starts to happen in 8fed465. I suspect it's because the turquin texture loads asynchronously but is not awaited, so it's being used before it's fully initialized, resulting in some invalid calculations.

Why don't we just load the texture once in an awaited global variable so we can ensure it's available and then we can revisit the swappable material design later on to be more resilient to these cases.

[TheBlek]

Why don't we just load the texture once in an awaited global variable so we can ensure it's available and then we can revisit the swappable material design later on to be more resilient to these cases.

Did that.

While trying to test with a debug material I noticed that the material change for a pathtracer does not work properly. Fixed that. I think it could be helpful to expose an example with editable debug texture. So you could inspect different values. Maybe even inside the viewerTest example?

Another thing I noticed is normal map seems to have less effect in webgpu.

WebGL:

WebGPU:

[TheBlek]

Normal map issue turned out to be missing tangent attribute on the mesh. Added an option to generate common attributes as in StaticGeometryGenerator.

[TheBlek]

And just to make a list of remaining PBR features to tackle - is there anything missing here?

• Clearcoat support
• Sheen support
• Transmission support
• Iridescence
• Volume attenuation
• Opacity (handled in WebGPUPathTracer: Add support for opacity during BVH traversal #751)
• Alpha cutout
• Anisotropy
• Dielectric / Transmission multiscatter compensation
• Material sided-ness

Thin materials would also need special handling afaik

[gkjohnson]

"load" returns a Texture instance so await won't do anything here. We should use "loadAsync" if we want to await a promise.

[gkjohnson]

I think it could be helpful to expose an example with editable debug texture. So you could inspect different values. Maybe even inside the viewerTest example?

What do you mean by "inspect different values"? Do you mean like hovering over a pixel to see intermediate calculations? Either way we will need a better few test scenes. We can probably start converting some of the other examples to use WebGPUPathtracer if it's helpful, as well.

[TheBlek]

What do you mean by "inspect different values"?

I mean having some debug overlays that would display intermediate values as color. This is a limited view that would allow to see only one value at a time.

Do you mean like hovering over a pixel to see intermediate calculations?

That would be very useful. Do you know how that could be achieved? Only thing I can think of is writing intermediate values into storage textures but there is only so many texture binding slots.

Another way to trace calculations is to debug shaders in native tools like RenderDoc or just Metal Debugger. So maybe we should not resort to very complex means of tracing if such debuggers useful enough.

I tried debugging with RenderDoc myself but faced problems emitting hlsl shaders without symbol renaming. Although even without that translated shaders are recognizable and debuggable. And buffer contents are inspectable which is especially useful in wavefront case.

Note: useful flags for debugging in chrome: https://dawn.googlesource.com/dawn/+/HEAD/docs/dawn/debugging.md

[gkjohnson]

That would be very useful. Do you know how that could be achieved? Only thing I can think of is writing intermediate values into storage textures but there is only so many texture binding slots.

Yeah this or storage buffers more or less. The good thing is that we can use any extra slots allowed by our hardware since this is for debugging only rather than being limited by the default WebGPU limitations.

I tried debugging with RenderDoc myself but faced problems emitting hlsl shaders without symbol renaming.

The new three.js inspector may have utilities for stuff like this but I'm less familiar with it and it's unclear how well it works outside the traditional rendering paths.

--

And again - very nice work on the PR. I'll go ahead and merge this along with some of the other opacity-related ones.

[gkjohnson]

One thing I've noticed is that are a lot of black pixels showing up when using the wave front path - I'm think it might be from NaNs? Not sure if this is something you might have some ideas about?

Megakernel

Wavefront

[TheBlek]

lot of black pixels showing up when using the wave front path

Yes, I've also noticed that when working on clearcoat. The problem was that wavefront path tracer did not have a proper guard against scatter records with pdf <= 0. Fixed in #758