VkSplat Training Viewport Stability

[kmeirlaen]

Summary

This change fixes an intermittent CUDA illegal-address failure seen on a 2080 Ti when training starts while the VkSplat viewport is active, especially when dragging or moving the viewport around training startup.

The fix keeps hardware decoding and the Vulkan viewport enabled. It does not exclude specific GPUs, disable nvImageCodec, or increase the VkSplat shared-scratch allocation estimate.

Root Cause

The failure was initially reported as nvImageCodec decode failures, but the detailed logs showed those errors were often downstream of an already-poisoned CUDA context. The first CUDA illegal address usually came from FastGS rasterization or related training work, while nvImageCodec later observed the same CUDA error during decode or sync.

The useful signal came from the Vulkan/VkSplat path:

• VkSplat creates/imports a shared rasterizer scratch arena for viewport rendering.
• Training also uses the CUDA rasterizer arena.
• At training start, the render model pointer changes from the pre-training model to the training model.
• The old render path treated that model change as a full scene switch and reset VksplatViewportRenderer.
• That reset dropped the renderer state and shared-scratch import right as the training thread was starting.
• The first interactive viewport frames then tried to recreate or bind VkSplat shared scratch while FastGS training work was active.
• On the affected setup this could corrupt CUDA state and surface as cudaErrorIllegalAddress in FastGS, tensor ops, nvImageCodec, or Vulkan interop.

So the core issue was not "hardware decode is bad" and not "the 2080 Ti must be disabled". The crash came from unsafe lifecycle/ownership around the VkSplat shared scratch during training startup and interactive viewport work.

What Changed

Preserve VkSplat Renderer During Training Model Swap

When renderVulkanFrame() detects a model pointer change while training with the VkSplat backend, it now preserves the existing VksplatViewportRenderer instead of resetting it.

This keeps the already-created handshake and shared-scratch state alive across the normal training-start model transition.

For non-training model changes, the previous reset behavior is kept.

Prime VkSplat Shared Scratch Before Training Starts

Training startup now asks the rendering manager to prepare VkSplat training shared scratch before the training thread begins.

This happens after trainer initialization and memory-pool trimming, but before training_complete_ is cleared and before the training thread starts. The goal is to have the renderer's shared scratch installed before FastGS starts using the CUDA arena.

Fail Viewport Contention Safely

During training renders, VkSplat now checks that shared scratch is already installed and tries to acquire the rasterizer arena guard before doing input upload or other CUDA/Vulkan render preparation.

If the arena is busy, the viewport returns the existing retryable "shared scratch unavailable" path. The rendering manager can keep or return the cached viewport image and retry later, instead of performing late scratch setup while training owns the arena.

Prevent Camera Pause Lease Stacking

A second issue appeared after the crash was fixed: training sometimes did not resume after viewport/camera movement.

Logs showed repeated temporary camera pause requests at the same iteration while the underlying trainer was already paused. Those requests stacked temporary pause leases, so a later camera timeout could log "resuming" without actually reaching a zero pause depth.

Camera movement now only requests a temporary training pause when the trainer manager is running and the underlying trainer is not already paused.

Files Changed

• src/visualizer/rendering/rendering_manager.hpp
• src/visualizer/rendering/rendering_manager_vulkan.cpp
• src/visualizer/rendering/vksplat_viewport_renderer.hpp
• src/visualizer/rendering/vksplat_viewport_renderer.cpp
• src/visualizer/training/training_manager.hpp
• src/visualizer/training/training_manager.cpp
• src/visualizer/input/input_controller.cpp

Validation

Tested manually on the affected Windows/2080 Ti setup with:

build\LichtFeld-Studio.exe -o D:\temp\gs_tests\dual_camera\colmap\output -d D:\temp\gs_tests\dual_camera\colmap --log-file output.txt

Observed results after the patch:

• No CUDA illegal-address crashes while dragging/moving the viewport during training startup.
• nvImageCodec remains enabled.
• Vulkan viewport remains enabled.
• Training resumes correctly after camera/viewport movement.
• No GPU-specific exclusion was added.
• No larger shared-scratch capacity multiplier was kept.

Build validation:

cmake --build build -j 8

The build completed successfully.

Notes For Reviewers

The key behavioral change is that training-start model pointer changes are not treated the same as ordinary scene switches for VkSplat renderer lifetime. In training, preserving the renderer is intentional because the renderer owns CUDA/Vulkan interop state that must remain stable across the handoff into live training.

The retry path for busy shared scratch is also intentional. A missed viewport frame is acceptable; racing the trainer for shared rasterizer arena ownership is not.

[Copilot]

Pull request overview

This PR improves stability when starting training while the Vulkan/VkSplat viewport is active by preventing unsafe lifecycle churn of the VkSplat renderer/shared-scratch and by avoiding temporary camera-pause lease stacking.

Changes:

• Preserve VksplatViewportRenderer across the training-start model pointer swap (while still resetting on non-training scene/model changes).
• Prime VkSplat training shared scratch before the training thread starts, and make training-time viewport renders fail fast (retry/cached image) when the shared arena is unavailable.
• Prevent repeated camera movement from stacking temporary training pause leases by skipping the pause request when the underlying trainer is already paused.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/visualizer/rendering/rendering_manager.hpp	Adds API to pre-prime VkSplat training shared scratch from outside the render loop.
src/visualizer/rendering/rendering_manager_vulkan.cpp	Preserves VkSplat renderer across training model swaps; implements shared-scratch priming helper.
src/visualizer/rendering/vksplat_viewport_renderer.hpp	Declares ensureTrainingSharedScratchReady for pre-start setup.
src/visualizer/rendering/vksplat_viewport_renderer.cpp	Implements shared-scratch priming; tightens training-time arena-guard acquisition and “retryable” failure behavior.
src/visualizer/training/training_manager.hpp	Exposes isTrainerPaused() to distinguish UI state vs. underlying trainer pause.
src/visualizer/training/training_manager.cpp	Primes VkSplat training shared scratch between trainer init and training thread start.
src/visualizer/input/input_controller.cpp	Avoids stacking temporary pause leases during camera movement by checking underlying trainer pause state.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[MrNeRF]

this will fail because line 6609 creates the arena guard which prevents the arena from growing.

[MrNeRF]

Currently this will only break on massive resize of the viewport or in case we had just a huge densification.

I think this order must be ensured in the code:

ensureSharedScratchArena(...);  // may grow, requires no active arena frame
shared_arena_guard.emplace()

[MrNeRF]

I think this is confusing here and does not fit properly into the list of methods:

How about something like:

  bool TrainerManager::pauseTrainingTemporaryIfActive() {
      if (!isRunning() || !trainer_ || trainer_->is_paused()) {
          return false;
      }

      pauseTrainingTemporary();
      return true;
  }

[MrNeRF]

thanks! Looks good.