Skip to content

fix(2dgs): use ray-plane intersection depth instead of Gaussian center depth#932

Open
siddhu-pikachu wants to merge 3 commits into
nerfstudio-project:mainfrom
siddhu-pikachu:fix/2dgs-intersection-depth
Open

fix(2dgs): use ray-plane intersection depth instead of Gaussian center depth#932
siddhu-pikachu wants to merge 3 commits into
nerfstudio-project:mainfrom
siddhu-pikachu:fix/2dgs-intersection-depth

Conversation

@siddhu-pikachu
Copy link
Copy Markdown

Summary

This PR fixes 2DGS depth rendering so that the depth channel uses the ray-splat intersection depth instead of the Gaussian center depth.

The original issue was that rasterize_to_pixels_2dgs blended the packed Gaussian center depth in the last channel, which produced incorrect depth for tilted splats. In the repro from #863, this showed up as nearly constant depth across each splat instead of the expected gradient.

What changed

  • Added a has_depth_channel flag through the 2DGS stack so the correction is only applied when the last color channel actually stores depth:
    • Python wrapper
    • torch bindings / schemas
    • C++ declarations / definitions
    • CUDA forward / backward paths
  • Updated the 2DGS forward kernel to replace the packed center depth with the ray-splat intersection depth.
  • Updated the 2DGS backward kernel so gradients for the depth channel flow through the intersection-depth computation instead of back into the packed center-depth color channel.
  • Updated distortion / median-depth handling to use the same corrected depth quantity.
  • Fixed the 2DGS binding/declaration mismatches needed to expose the new argument correctly.

Validation

I validated this with the issue #863 repro scene (two tilted splats) and compared the gsplat 2DGS output against the local Inria diff-surfel-rasterization reference backend.

Before this change:

  • depth_2dgs was using Gaussian center depth and did not match the expected ray-plane/ray-splat intersection behavior.

After this change:

  • depth_2dgs shows the expected spatially varying depth gradient.
  • The output now matches the Inria reference visually on the repro scene.

Notes

There are still tiny residual rasterization-footprint differences between gsplat and the Inria backend, but those appear to be pre-existing and separate from the depth bug fixed here. This PR is focused on correcting the depth quantity used by the 2DGS path.

Closes #863

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: 2DGS rasterization uses Gaussian center depth instead of ray-plane intersection depth

1 participant