3DGS PR 6: 3dgs-examples-tests-docs: examples, unit tests, pybind, tutorial#7490
Open
ssheorey wants to merge 9 commits into
Open
3DGS PR 6: 3dgs-examples-tests-docs: examples, unit tests, pybind, tutorial#7490ssheorey wants to merge 9 commits into
ssheorey wants to merge 9 commits into
Conversation
|
Thanks for submitting this pull request! The maintainers of this repository would appreciate if you could update the CHANGELOG.md based on your changes. |
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
…ling - Add AGENTS.md with coding and PR review guidelines - GLEW upgraded to v2.3.1 (adds GL_EXT_memory_object, GL_EXT_semaphore for Vulkan-OpenGL interop needed by Gaussian splatting) - Filament version bump for shared-context platform header support - Style tooling: check_style.py and check_cpp_style.cmake updated for .comp (GLSL compute) and .mm (Objective-C++) file handling
… transforms - t::PointCloud: IsGaussianSplat(), Rotate/Scale/Translate/Transform for GS attributes (positions, quaternion rotations, linear scales, DC/SH coefficients) - Ivanic-Ruedenberg SH rotation algorithm for f_rest attributes (degrees 1-3) - Point inversion parity handling for odd-degree SH blocks - PLY IO (legacy + tensor): read/write GS attributes (pos, rot, scale, opacity, f_dc, f_rest); exponentiate log-scales from PLY files at load time - SPLAT file IO: linear scales, packed quaternions, rgb8 DC color - FileFormatIO: register .splat extension - MaterialRecord: add GS-specific render flags (sh_degree, antialias, etc.) - Renderer/View: virtual interface additions for GS output queries
…erop) Third-party additions: - vkmemalloc: Vulkan Memory Allocator (VMA) for device-local buffer allocation - vulkan_headers: Khronos Vulkan headers - find_dependencies.cmake: add Vulkan + vkmemalloc package discovery - cmake/Open3DAddComputeShaders.cmake: compile GLSL .comp -> SPIR-V (glslc) and cross-compile to MSL (spirv-cross) for Apple; installs to resources/ cpp/open3d/CMakeLists.txt: link OpenGL::EGL and X11 for the GL shared-context backend on Linux/Windows. New source files (wired into the build in the next PR): - ComputeGPU.h: ComputeProgramId enum, GaussianSplatGpuContext abstract base, GpuComputeFrame/GpuComputePass RAII helpers - ComputeGPUVulkan: headless Vulkan instance + device, SSBO/UBO binding, command buffer lifecycle, fence-based geometry sync - ComputeGPUMetal.mm: Metal GaussianSplatGpuContext implementation - GaussianSplatOpenGLContext: GLFW-owned GL 4.6 helper window for shared-context creation (GLX on Linux, WGL on Windows) before Engine::create() - GaussianSplatVulkanInteropContext: exports VkImage memory to OpenGL via GL_EXT_memory_object for zero-copy Filament-Vulkan texture sharing
…ation
Compute shaders (GLSL -> SPIR-V/MSL via cmake/Open3DAddComputeShaders.cmake):
- gaussian_project.comp: project splats to 2D, emit tile sort entries with
subgroup-batched atomics; SH evaluation degree 1-2; visibility mask test
- gaussian_composite.comp: depth-aware rasterization; work-stealing tile
dispatch; binary-search tile boundaries; front-to-back alpha blend with
early transmittance exit; writes RGBA16F + depth
- gaussian_radix_sort_{histograms,scatter}.comp: 4-pass LSD radix sort using
subgroup prefix-sum (GL_KHR_shader_subgroup_arithmetic)
- gaussian_compute_dispatch_args.comp: GPU-side indirect dispatch args to
eliminate CPU readback stall after projection
GS rendering core (gaussian_splat/):
- GaussianSplatDataPacking: CPU->GPU packing (fp16 scales, snorm8 rotations,
sigmoid-preapplied opacity, bit-packed visibility mask)
- GaussianSplatPassRunner: backend-agnostic geometry + composite pass sequence
- GaussianSplatRenderer: per-view output lifecycle, Backend abstraction,
BeginFrame/RenderGeometryStage/RenderCompositeStage
- GaussianSplatVulkanBackend: Vulkan backend (Linux/Windows)
- GaussianSplatMetalBackend.mm: Metal backend (macOS)
- GaussianSplatDesign.md: design document
Filament integration (rendering/filament/):
- FilamentEngine: create GL shared context before Engine::create(); force GLX
on Linux (XWayland for Wayland sessions); pass native handle to Filament
- FilamentRenderer: GaussianSplatRenderer lifecycle; BeginFrame fire-and-forget
geometry stage; post-depth composite stage; Apple PostRedraw callback
- FilamentScene: per_object_gs_attrs_ / merged_gs_attrs_; RebuildMergedGaussianData();
HasNonGaussianVisibleGeometry(); ShowGeometry patches visibility mask in-place
- FilamentView: EnableViewCaching(); GetRenderTargetHandle()
- FilamentResourceManager: CreateImportedTexture() / CreateImportedMTLTexture()
for zero-copy GL/Metal texture import
- FilamentNativeInterop.mm: retrieve MTLDevice/MTLCommandQueue from PlatformMetal
- FilamentRenderToBuffer.h: forward-declare GaussianSplatRenderer member
(used by offscreen path in next PR)
- Remove GaussianSplatBuffers.cpp (replaced by GaussianSplatDataPacking)
GUI / visualizer:
- Application: pre-Engine GL context init; GLFW platform X11 force
- Window: Apple composite-complete callback -> PostRedraw()
- GuiVisualizer / O3DVisualizer: GS scene load, help dialog, fps HUD
- Draw.h/.cpp: GS-specific draw() options (sh_degree, antialias, RenderConfig)
- gui/CMakeLists.txt: open3d_add_compute_shaders() for GS shaders; remove
gaussianSplat.mat; add img_blit.mat
- visualization/CMakeLists.txt: add all GS source files to visualization_impl
gaussian_depth_merge.comp: - New compute shader: merges GS composite depth (R32F) and Filament scene depth (GL_DEPTH_COMPONENT32F, reversed-Z) into a normalised R16UI texture - Per-pixel min-depth merge in linear space; outputs [0,65535] uint16 - Bound at composite stage binding 14 (reuses radix UBO slot which is idle) FilamentRenderToBuffer::Render(): - Mirror FilamentRenderer interactive pipeline for offscreen captures: BeginFrame -> flushAndWait -> RenderGeometryStage -> render() -> flushAndWait -> RenderCompositeStage - Color readback: two parallel readPixels (Filament RGBA+UBYTE base, GS RGBA+FLOAT overlay) then CPU BlendPremultipliedSplatOverRgb8() - Depth readback: ReadMergedDepthToUint16Cpu() reads the R16UI merged texture via glGetTexImage; converts uint16 -> float [0,1] for the RenderToDepthImage callback; fallback to Filament-only depth if GS depth unavailable - Metal: readPixels always uses RGBA+UBYTE (no native RGB Metal format); alpha stripped when n_channels_==3 - EnableViewCaching(true) in SetDimensions when GS geometry present (required for render-target-based readPixels)
Examples: - examples/cpp/GaussianSplat.cpp: interactive viewer with red sphere for depth compositing validation; command-line options for sh_degree, antialias, RenderConfig tuning - examples/python/visualization/draw_from_csv.py: load and visualize 3DGS scenes from CSV files with draw() Unit tests: - cpp/tests/visualization/rendering/GaussianSplatRender.cpp: RenderToImage golden PNG test (36x20 viewport, AllClose atol=5); OPEN3D_TEST_GENERATE_REFERENCE=1 regenerates reference image - cpp/tests/io/FileFormatIO.cpp: .ply and .splat round-trip IO tests for GS attributes (positions, rotations, scales, opacity, f_dc, f_rest) - cpp/tests/t/geometry/PointCloud.cpp: Rotate/Scale/Translate/Transform correctness tests for GS PointClouds including SH rotation validation Python bindings: - cpp/pybind/t/io/class_io.cpp: expose GS IO options (sh_degree etc.) - cpp/pybind/visualization/rendering/rendering.cpp: expose RenderConfig, GS texture query, RenderToDepthImage for 3DGS - cpp/pybind/io/class_io.cpp: .splat file extension registration Tutorial: - docs/tutorial/visualization/gaussian_splatting.ipynb: end-to-end notebook demonstrating load, visualize, RenderToImage, RenderToDepthImage - docs/tutorial/visualization/index.rst: add gaussian_splatting to toctree Minor cpp example cleanup (missing include fix in OffscreenRendering, OnlineSLAM*, TICP examples)
ssheorey
force-pushed
the
ss/3dgs-6-examples-tests-docs
branch
from
ab2a180 to
1b90638
Compare
Member
Author
Screen.Recording.2026-04-28.at.2.56.52.PM.mov |
benjaminum
approved these changes
May 7, 2026
…upported now. Install glslang in all 3 platforms. fix incomplete type error. Co-authored-by: Copilot <copilot@github.com>
ssheorey
force-pushed
the
ss/3dgs-6-examples-tests-docs
branch
from
98e0902 to
ce3432a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Type
Motivation and Context
Checklist:
python util/check_style.py --applyto apply Open3D code styleto my code.
updated accordingly.
results (e.g. screenshots or numbers) here.
Description
Examples:
depth compositing validation; command-line options for sh_degree, antialias,
RenderConfig tuning
scenes from CSV files with draw()
Unit tests:
RenderToImage golden PNG test (36x20 viewport, AllClose atol=5);
OPEN3D_TEST_GENERATE_REFERENCE=1 regenerates reference image
GS attributes (positions, rotations, scales, opacity, f_dc, f_rest)
correctness tests for GS PointClouds including SH rotation validation
Python bindings:
GS texture query, RenderToDepthImage for 3DGS
Tutorial:
demonstrating load, visualize, RenderToImage, RenderToDepthImage
Minor cpp example cleanup (missing include fix in OffscreenRendering,
OnlineSLAM*, TICP examples)