Hydra 2.0: Support for non-rtx renderer

[BenjaminYde]

Small question. Is it possible to plug in a non-RTX hydra renderer to Isaac Sim? I know the Omniverse / Isaac Sim renderers are RTX but these are very heavy to run.

For example, when building OpenUSD you have usdview which can use the HgiVulkan which is perfect for fast visualizations.
Realistic rendering is less important in the simulations we run because the robotics logic is more important thus RTX is not needed. Omniverse uses a lot of GPU resources for rendering, but we also need GPU resources for our business logic when doing co-simulation (omniverse <> business app).

When just working with native USD, it is nice to be able to just configure /compose the scene. It's like using the real-time vulkan renderer in Houdini. There I'm also not going to use the KarmaXPU renderer where I'm doing layouting / updating transforms / ... Same with Blender.

It's very nice to see Omniverse / Isaac Sim building on top of USD, but the non-RTX feature (also while simulating, for example with lidars or updating objects/xforms) would be very welcome.

Don't know if this is already possible / is on the roadmap / is a feature that is commonly requested.

[vick-yu]

Hi, @BenjaminYde

Isaac Sim doesn't currently support plugging in an alternative Hydra renderer (like Storm/HgiVulkan). The sensor simulation stack — cameras, RTX LiDAR, Replicator annotators — is tightly coupled to the RTX
renderer, so swapping in a rasterizer would disable most of that functionality.

That said, the Performance Optimization Handbook covers a range of options to reduce rendering
cost, from "full RTX" down to "almost nothing":

For physics-only / co-simulation workflows:

• Headless mode with viewport updates disabled — the lightest option, zero rendering overhead:

    app = SimulationApp({"headless": True, "disable_viewport_updates": True})                                                                                                                                       

• Selective rendering — advance physics without rendering every step:

    world.step(render=False)  # physics only, no render pass                                                                                                                                                        
    world.step(render=True)   # render when you need a frame                                                                                                                                                        

If you need some visualization but want to minimize GPU cost:

• RTX Real-Time 2.0 with reduced retrace threshold — trades a small amount of accuracy for significant speedup:

    --/rtx/pathtracing/cached/retrace=0.1                                                                                                                                                                           
  

• Strip rendering features — disable reflections, transparency, fractional cutout opacity, etc. As the docs note: "There is no non-RTX rendering mode in the Isaac Sim GUI application, but you can disable almost
  everything to increase execution speed."
• Skip material loading for maximum scene load speed:

    --/app/renderer/skipMaterialLoading=true                                                                                                                                                                        
  

• DLSS Performance mode (default) to minimize render time:

    --/rtx/post/dlss/execMode=0                                                                                                                                                                                     
  

The Houdini/Blender analogy is apt — those tools have lightweight viewport rasterizers for layout and heavy renderers for final output. Isaac Sim doesn't have that split, but between headless mode, selective
rendering, and stripping down RTX features, you can get fairly close to a "minimal rendering" configuration for co-simulation workflows.

[BenjaminYde]

@vick-yu, Thanks for the quick and detailed reply!
I totally understand, just wanted some clarification on if this was something that was possible already or not and if it's on the roadmap.

When starting isaac sim (editor), we run a python exec script to set all the rtx settings to their lowest so we can focus on simulation.
Also looking forward to the new real-time 2.0 renderer, have been waiting for this. All optimizations are welcome.