Skip to content

fix: volume rendering picking issue with other transparent objects#1003

Open
seankmartin wants to merge 3 commits into
google:masterfrom
MetaCell:fix/volume-rendering-pick-issues
Open

fix: volume rendering picking issue with other transparent objects#1003
seankmartin wants to merge 3 commits into
google:masterfrom
MetaCell:fix/volume-rendering-pick-issues

Conversation

@seankmartin
Copy link
Copy Markdown
Contributor

@seankmartin seankmartin commented May 21, 2026

This corrects a picking issue with volume rendering. It meant that if you had another transparent object in the same panel as the volume rendering, the volume rendering could take precedence over the transparent object when it shouldn't because it was writing the wrong depth of the full screen quad instead of the depth from the max projection.

Edit: reverted an initial suspect other bug. It's not a bug, but points at unclear responsibilities of the volume render layer. Will make separate issue.

one issue is that the max projection to pick copy could happen with the depthFunc being
either GREATER or LESS depending on whether in MAX or ON mode, leading to inconsistent
pick combining with multiple volume rendering layers. Further, OIT blending state is on,
so we have to ensure that this max projection copy to the picking buffer does not use
blending or else it could blend pickIDs
previously the depth of the screen filling quadrant was getting written instead
@seankmartin seankmartin changed the title Fix: volume rendering picking issues fix: volume rendering picking issues May 21, 2026
@chrisj
Copy link
Copy Markdown
Contributor

chrisj commented May 22, 2026

looks very safe, could you easily share some example links of both scenarios?

is using GREATER depth test function just about consistency? If so I would add a comment there
and is 1.0 - depth.r; done because it is set to GREATER?

@seankmartin
Copy link
Copy Markdown
Contributor Author

looks very safe, could you easily share some example links of both scenarios?

is using GREATER depth test function just about consistency? If so I would add a comment there and is 1.0 - depth.r; done because it is set to GREATER?

Yes I think some links can help for this. Here is the first scenario, many volume renderings

It's a four channel rendering, all channels with max projection
image

This is used to show what should be happening in the picking buffer. With all layers in max projection, the color and the picking correspond very well. When some layers instead have volume rendering just on, a second max projection pass if performed and so the color and picking can match a little bit less, but the hope is that it still feels pretty good.
I noticed from using this link that both the current neuroglancer version and the client preview from this PR didn't feel any different, which was surprising to me. And then I noticed that the volume rendering layer is actually handling one part of this PR:

    const restoreDrawingBuffersAndState = () => {
      const performedSecondPassForPicking =
        !isProjectionMode(this.mode.value) &&
        !renderContext.isContinuousCameraMotionInProgress;
      // If the layer is in projection mode or the second pass for picking has been performed,
      // the max projection state is needed
      // the max projection buffer is not bound, because it is immediately read back
      // in the perspective panel to update the max projection picking buffer
      if (isProjectionMode(this.mode.value) || performedSecondPassForPicking) {
        gl.depthMask(true);
        gl.disable(WebGL2RenderingContext.BLEND);
        gl.depthFunc(WebGL2RenderingContext.GREATER);
      } else {
        // Otherwise, the regular OIT buffer is needed along with the state
        gl.depthMask(false);
        gl.enable(WebGL2RenderingContext.BLEND);
        gl.depthFunc(WebGL2RenderingContext.LESS);
        renderContext.bindVolumeRenderingBuffer!();
      }
    };

So I'll revert the piece of this PR that disables blending and sets the depth test function to GREATER. It's fully needed. I just didn't realise the volume rendering render layer was already doing it when I was reading back through the perspective panel code.

The above does point to an increasing feeling I've been getting to refactor the volume rendering to make the responsibilities of the volume rendering render layer and the perspective panel clearer. They have a bit of an intermingled back and forth right now which can be hard to reason about.

One thing I didn't answer here yet is why GREATER. We're really faking a depth value here to allow max intensity tracking across chunks and across layers. So depth that is stored in the offscreen buffer is the real depth of the voxel with max intensity, but the gl_FragDepth depth value for the current fragment is really the intensity value itself. So GREATER is used to take the samples with the max intensity. In this case for the picking, it's used when multiple layers use volume rendering to track the result at a particular fragment with the greatest intensity across all the layers.

@seankmartin
Copy link
Copy Markdown
Contributor Author

For the second scenario I've setup a pretty extreme example, because that's the easiest to see it with

image

We have two transparent objects in the scene. The mesh is almost completely in front of the image, yet if you try the picking before the fix here, you will end up picking the volume. The debug vis I have in a different PR #1002 shows this very well:
Old:
image

New:
image

The 1.0 - depth.r is because the depth stored in the offscreen framebuffer and the glFragDepth fragment depth need to be converted between. The regular emit shows this kind of idea as well. So without the 1.0 -, if you just used depth you'd be "sorting" in the wrong order.
image

export const glsl_perspectivePanelEmit = `
void emit(vec4 color, highp uint pickId) {
  out_color = color;
  float zValue = 1.0 - gl_FragCoord.z;
  out_z = vec4(zValue, zValue, zValue, 1.0);
  float pickIdFloat = float(pickId);
  out_pickId = vec4(pickIdFloat, pickIdFloat, pickIdFloat, 1.0);
}
`;

This reverts commit 6a2ede4.
This is because the volume rendering layer already handles this. It could use a
later refactor though.
@seankmartin seankmartin changed the title fix: volume rendering picking issues fix: volume rendering picking issue with other transparent objects May 22, 2026
@seankmartin
Copy link
Copy Markdown
Contributor Author

See #1008 regarding the refactor.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants