Skip to content

Commit 6dce267

Browse files
ssheoreyssheorey
committed
3dgs-render: interactive 3DGS rendering pipeline, Filament+GUI integration
Compute shaders (GLSL -> SPIR-V/MSL via cmake/Open3DAddComputeShaders.cmake): - gaussian_project.comp: project splats to 2D, emit tile sort entries with subgroup-batched atomics; SH evaluation degree 1-2; visibility mask test - gaussian_composite.comp: depth-aware rasterization; work-stealing tile dispatch; binary-search tile boundaries; front-to-back alpha blend with early transmittance exit; writes RGBA16F + depth - gaussian_radix_sort_{histograms,scatter}.comp: 4-pass LSD radix sort using subgroup prefix-sum (GL_KHR_shader_subgroup_arithmetic) - gaussian_compute_dispatch_args.comp: GPU-side indirect dispatch args to eliminate CPU readback stall after projection GS rendering core (gaussian_splat/): - GaussianSplatDataPacking: CPU->GPU packing (fp16 scales, snorm8 rotations, sigmoid-preapplied opacity, bit-packed visibility mask) - GaussianSplatPassRunner: backend-agnostic geometry + composite pass sequence - GaussianSplatRenderer: per-view output lifecycle, Backend abstraction, BeginFrame/RenderGeometryStage/RenderCompositeStage - GaussianSplatVulkanBackend: Vulkan backend (Linux/Windows) - GaussianSplatMetalBackend.mm: Metal backend (macOS) - GaussianSplatDesign.md: design document Filament integration (rendering/filament/): - FilamentEngine: create GL shared context before Engine::create(); force GLX on Linux (XWayland for Wayland sessions); pass native handle to Filament - FilamentRenderer: GaussianSplatRenderer lifecycle; BeginFrame fire-and-forget geometry stage; post-depth composite stage; Apple PostRedraw callback - FilamentScene: per_object_gs_attrs_ / merged_gs_attrs_; RebuildMergedGaussianData(); HasNonGaussianVisibleGeometry(); ShowGeometry patches visibility mask in-place - FilamentView: EnableViewCaching(); GetRenderTargetHandle() - FilamentResourceManager: CreateImportedTexture() / CreateImportedMTLTexture() for zero-copy GL/Metal texture import - FilamentNativeInterop.mm: retrieve MTLDevice/MTLCommandQueue from PlatformMetal - FilamentRenderToBuffer.h: forward-declare GaussianSplatRenderer member (used by offscreen path in next PR) - Remove GaussianSplatBuffers.cpp (replaced by GaussianSplatDataPacking) GUI / visualizer: - Application: pre-Engine GL context init; GLFW platform X11 force - Window: Apple composite-complete callback -> PostRedraw() - GuiVisualizer / O3DVisualizer: GS scene load, help dialog, fps HUD - Draw.h/.cpp: GS-specific draw() options (sh_degree, antialias, RenderConfig) - gui/CMakeLists.txt: open3d_add_compute_shaders() for GS shaders; remove gaussianSplat.mat; add img_blit.mat - visualization/CMakeLists.txt: add all GS source files to visualization_impl
1 parent 6b969dd commit 6dce267

63 files changed

Lines changed: 6870 additions & 807 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎CMakeLists.txt‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -223,6 +223,17 @@ if(ENABLE_HEADLESS_RENDERING AND BUILD_GUI)
223223
message(WARNING "Headless rendering disables the Open3D GUI")
224224
set(BUILD_GUI OFF)
225225
endif()
226+
if(NOT BUILD_FILAMENT_FROM_SOURCE AND NOT FILAMENT_PRECOMPILED_ROOT)
227+
message(WARNING
228+
"Gaussian splatting requires Filament >= 1.57.2 backend platform headers. Set FILAMENT_PRECOMPILED_ROOT to a compatible release package, or enable BUILD_FILAMENT_FROM_SOURCE.")
229+
endif()
230+
if(APPLE)
231+
find_program(XCRUN_EXECUTABLE xcrun)
232+
if(NOT XCRUN_EXECUTABLE)
233+
message(WARNING
234+
"xcrun was not found. Metal shader compilation for Gaussian splatting will require Xcode command line tools.")
235+
endif()
236+
endif()
226237
if(ENABLE_HEADLESS_RENDERING AND (USE_SYSTEM_GLEW OR USE_SYSTEM_GLFW))
227238
message(WARNING "Headless rendering requires customized GLEW and GLFW builds")
228239
set(USE_SYSTEM_GLEW OFF)

‎cpp/apps/CMakeLists.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ macro(open3d_add_app_gui SRC_DIR APP_NAME TARGET_NAME)
4747
add_custom_command(TARGET "${TARGET_NAME}"
4848
POST_BUILD
4949
COMMAND ${CMAKE_COMMAND} -E make_directory "${APP_DIR}/${RESOURCE_DIR_NAME}"
50-
COMMAND ${CMAKE_COMMAND} -E copy ${RESOURCE_FILES} "${APP_DIR}/${RESOURCE_DIR_NAME}"
50+
COMMAND ${CMAKE_COMMAND} -E copy_directory "${GUI_RESOURCE_DIR}" "${APP_DIR}/${RESOURCE_DIR_NAME}"
5151
)
5252
if (UNIX)
5353
install(DIRECTORY "${APP_DIR}"

‎cpp/apps/Open3DViewer/Open3DViewer_mac.mm‎

Lines changed: 39 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@
2727

2828
class Open3DVisualizer : public open3d::visualization::GuiVisualizer {
2929
using Super = GuiVisualizer;
30+
3031
public:
3132
Open3DVisualizer()
3233
: open3d::visualization::GuiVisualizer("Open3D", WIDTH, HEIGHT) {
@@ -49,55 +50,62 @@ void OnMenuItemSelected(Menu::ItemId item_id) override {
4950
// This will set the users personal default to use Open3D for
5051
// the file types below. THIS SHOULD ONLY BE CALLED
5152
// AFTER THE USER EXPLICITLY CONFIRMS THAT THEY WANT TO DO THIS!
52-
CFStringRef open3dBundleId = (__bridge CFStringRef)@"com.isl-org.open3d.Open3D";
53+
CFStringRef open3dBundleId =
54+
(__bridge CFStringRef) @"com.isl-org.open3d.Open3D";
5355
// The UTIs should match what we declare in Info.plist
5456
LSSetDefaultRoleHandlerForContentType(
55-
(__bridge CFStringRef)@"public.gl-transmission-format",
56-
kLSRolesAll, open3dBundleId);
57+
(__bridge CFStringRef) @"public.gl-transmission-format",
58+
kLSRolesAll, open3dBundleId);
5759
LSSetDefaultRoleHandlerForContentType(
58-
(__bridge CFStringRef)@"public.gl-binary-transmission-format",
59-
kLSRolesAll, open3dBundleId);
60+
(__bridge CFStringRef) @"public.gl-binary-transmission-"
61+
@"format",
62+
kLSRolesAll, open3dBundleId);
6063
LSSetDefaultRoleHandlerForContentType(
61-
(__bridge CFStringRef)@"public.geometry-definition-format",
62-
kLSRolesAll, open3dBundleId);
64+
(__bridge CFStringRef) @"public.geometry-definition-"
65+
@"format",
66+
kLSRolesAll, open3dBundleId);
6367
LSSetDefaultRoleHandlerForContentType(
64-
(__bridge CFStringRef)@"public.object-file-format",
65-
kLSRolesAll, open3dBundleId);
68+
(__bridge CFStringRef) @"public.object-file-format",
69+
kLSRolesAll, open3dBundleId);
6670
LSSetDefaultRoleHandlerForContentType(
67-
(__bridge CFStringRef)@"public.point-cloud-library-file",
68-
kLSRolesAll, open3dBundleId);
71+
(__bridge CFStringRef) @"public.point-cloud-library-"
72+
@"file",
73+
kLSRolesAll, open3dBundleId);
6974
LSSetDefaultRoleHandlerForContentType(
70-
(__bridge CFStringRef)@"public.polygon-file-format",
71-
kLSRolesAll, open3dBundleId);
75+
(__bridge CFStringRef) @"public.polygon-file-format",
76+
kLSRolesAll, open3dBundleId);
7277
LSSetDefaultRoleHandlerForContentType(
73-
(__bridge CFStringRef)@"public.3d-points-format",
74-
kLSRolesAll, open3dBundleId);
78+
(__bridge CFStringRef) @"public.3d-points-format",
79+
kLSRolesAll, open3dBundleId);
7580
LSSetDefaultRoleHandlerForContentType(
76-
(__bridge CFStringRef)@"public.standard-tesselated-geometry-format",
77-
kLSRolesAll, open3dBundleId);
81+
(__bridge CFStringRef) @"public.standard-tesselated-"
82+
@"geometry-format",
83+
kLSRolesAll, open3dBundleId);
7884
LSSetDefaultRoleHandlerForContentType(
79-
(__bridge CFStringRef)@"public.xyz-points-format",
80-
kLSRolesAll, open3dBundleId);
85+
(__bridge CFStringRef) @"public.xyz-points-format",
86+
kLSRolesAll, open3dBundleId);
8187
LSSetDefaultRoleHandlerForContentType(
82-
(__bridge CFStringRef)@"public.xyzn-points-format",
83-
kLSRolesAll, open3dBundleId);
88+
(__bridge CFStringRef) @"public.xyzn-points-format",
89+
kLSRolesAll, open3dBundleId);
8490
LSSetDefaultRoleHandlerForContentType(
85-
(__bridge CFStringRef)@"public.xyzrgb-points-format",
86-
kLSRolesAll, open3dBundleId);
91+
(__bridge CFStringRef) @"public.xyzrgb-points-format",
92+
kLSRolesAll, open3dBundleId);
8793

8894
this->CloseDialog();
8995
});
9096

9197
auto vert = std::make_shared<Vert>(0, Margins(em));
9298
vert->AddChild(std::make_shared<Label>(
93-
"This will make Open3D the default application for the "
94-
"following file types:"));
99+
"This will make Open3D the default application for the "
100+
"following file types:"));
95101
vert->AddFixed(em);
96102
auto table = std::make_shared<VGrid>(2, 0, Margins(em, 0, 0, 0));
97103
table->AddChild(std::make_shared<Label>("Mesh:"));
98-
table->AddChild(std::make_shared<Label>(".gltf, .glb, .obj, .off, .ply, .stl"));
104+
table->AddChild(std::make_shared<Label>(
105+
".gltf, .glb, .obj, .off, .ply, .stl"));
99106
table->AddChild(std::make_shared<Label>("Point clouds:"));
100-
table->AddChild(std::make_shared<Label>(".pcd, .ply, .pts, .xyz, .xyzn, .xyzrgb"));
107+
table->AddChild(std::make_shared<Label>(
108+
".pcd, .ply, .pts, .xyz, .xyzn, .xyzrgb"));
101109
vert->AddChild(table);
102110
vert->AddFixed(em);
103111
auto buttons = std::make_shared<Horiz>(0.5 * em);
@@ -129,11 +137,10 @@ static void LoadAndCreateWindow(const char *path) {
129137
@interface AppDelegate : NSObject <NSApplicationDelegate>
130138
@end
131139

132-
@interface AppDelegate ()
133-
{
140+
@interface AppDelegate () {
134141
bool open_empty_window_;
135142
}
136-
@property (retain) NSTimer *timer;
143+
@property(retain) NSTimer *timer;
137144
@end
138145

139146
@implementation AppDelegate
@@ -182,7 +189,7 @@ int main(int argc, const char *argv[]) {
182189
open3d::utility::filesystem::ChangeWorkingDirectory(homedir);
183190
}
184191

185-
Application::GetInstance().Initialize(argc, argv);
192+
Application::GetInstance().Initialize();
186193

187194
// Note: if NSApp is created (which happens in +sharedApplication)
188195
// then GLFW will use our NSApp with our delegate instead of its
@@ -211,4 +218,4 @@ int main(int argc, const char *argv[]) {
211218
// ----
212219
}
213220

214-
#endif // __APPLE__
221+
#endif // __APPLE__

‎cpp/open3d/visualization/CMakeLists.txt‎

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,9 @@ if (BUILD_GUI)
7373

7474
target_sources(visualization_impl PRIVATE
7575
rendering/filament/FilamentCamera.cpp
76+
rendering/gaussian_splat/GaussianSplatRenderer.cpp
77+
rendering/gaussian_splat/GaussianSplatDataPacking.cpp
78+
rendering/gaussian_splat/GaussianSplatPassRunner.cpp
7679
rendering/filament/FilamentEntitiesMods.cpp
7780
rendering/filament/FilamentGeometryBuffersBuilder.cpp
7881
rendering/filament/FilamentRenderToBuffer.cpp
@@ -82,9 +85,31 @@ if (BUILD_GUI)
8285
rendering/filament/LineSetBuffers.cpp
8386
rendering/filament/PointCloudBuffers.cpp
8487
rendering/filament/TriangleMeshBuffers.cpp
85-
rendering/filament/GaussianSplatBuffers.cpp
8688
)
8789

90+
if (APPLE)
91+
target_sources(visualization_impl PRIVATE
92+
rendering/gaussian_splat/ComputeGPUMetal.mm
93+
rendering/gaussian_splat/GaussianSplatMetalBackend.mm
94+
rendering/filament/FilamentNativeInterop.mm
95+
rendering/filament/FilamentResourceManagerApple.mm
96+
)
97+
set(O3D_GS_METAL_ARC_SOURCES
98+
rendering/gaussian_splat/ComputeGPUMetal.mm
99+
rendering/gaussian_splat/GaussianSplatMetalBackend.mm
100+
rendering/filament/FilamentResourceManagerApple.mm
101+
)
102+
set_source_files_properties(${O3D_GS_METAL_ARC_SOURCES}
103+
PROPERTIES COMPILE_OPTIONS "-fobjc-arc")
104+
else()
105+
target_sources(visualization_impl PRIVATE
106+
rendering/gaussian_splat/ComputeGPUVulkan.cpp
107+
rendering/gaussian_splat/GaussianSplatOpenGLContext.cpp
108+
rendering/gaussian_splat/GaussianSplatVulkanBackend.cpp
109+
rendering/gaussian_splat/GaussianSplatVulkanInteropContext.cpp
110+
)
111+
endif()
112+
88113
target_sources(visualization PRIVATE
89114
utility/Draw.cpp
90115
)

‎cpp/open3d/visualization/app/Viewer.cpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ void RunViewer(int argc, const char *argv[]) {
3636
}
3737

3838
auto &app = gui::Application::GetInstance();
39-
app.Initialize(argc, argv);
39+
app.Initialize();
4040

4141
auto vis = std::make_shared<open3d::visualization::GuiVisualizer>(
4242
"Open3D", width, height);

0 commit comments

Comments
 (0)