perf: add PrepareForReuse to FabricFrameView, remove sync_usd_on_fabric_write (#5380)

## Summary

Replace the `sync_usd_on_fabric_write` workaround in `FabricFrameView`
with proper `PrepareForReuse()` calls on the Fabric `PrimSelection`.
This tells the renderer (FSD/Storm) that Fabric data has changed, so the
next rendered frame reflects updated transforms — eliminating the need
to copy Fabric writes back to USD.

## Motivation

The existing `sync_usd_on_fabric_write` flag worked by mirroring every
Fabric write back to USD, which defeated the performance benefits of
Fabric. With `PrepareForReuse()`, the rendering pipeline is properly
notified of Fabric data changes without any USD writeback.

Additionally, the old code incorrectly fell back to USD for CPU devices
— Warp handles CPU Fabric buffers correctly, so the fallback was
unnecessary.

This addresses two of the issues raised in @pbarejko Piotr's review of
PR #4923:
- **Issue #1** (USD write-back): Fabric writes no longer sync back to
USD
- **Issue #4** (PrepareForReuse): Renderer notification via
`PrepareForReuse()` instead of USD writeback

## Changes

### Core (FabricFrameView)
- Call `_prepare_for_reuse()` in write paths (`set_world_poses`,
`set_scales`) to notify the renderer
- Remove `sync_usd_on_fabric_write` parameter (accepted via `**kwargs`
for backward compat)
- Remove incorrect CPU/device fallback warnings — Warp handles CPU
Fabric buffers correctly
- Add `_rebuild_fabric_arrays()` for topology change recovery when
`PrepareForReuse()` returns True, with assertion guarding the prim-count
invariant

### Camera
- Remove `sync_usd_on_fabric_write=True` from FrameView construction in
`camera.py`

## Benchmark Results

1024 prims, 50 iterations, NVIDIA L40 GPU:

| Operation | USD (ms) | Fabric (ms) | Speedup |
|---|---|---|---|
| Get World Poses | 14.71 | 0.07 | **203x** |
| Set World Poses | 40.75 | 0.16 | **259x** |
| Interleaved Set→Get | 55.90 | 0.24 | **232x** |
| Get Local Poses | 11.08 | 11.12 | 1.0x |
| Set Local Poses | 16.14 | 16.28 | 1.0x |

Local poses fall back to USD (expected — Fabric only accelerates world
poses via `omni:fabric:worldMatrix`).

## Tests Added

| Test | What it validates |
|------|------------------|
| `test_camera_pose_update_reflected_in_render` | Camera pose changes
propagate to rendered depth (close vs far) for CPU/GPU, tiled/non-tiled
|
| `test_fabric_set_world_does_not_write_back_to_usd` | Fabric writes
stay in Fabric, USD prim unchanged |
| `test_set_world_updates_local` (xfail) | Documents Issue #5:
`set_world_poses` doesn't update local pose in Fabric mode |

## Test Results

| Test Suite | Passed | Skipped | Xfailed | Total |
|---|---|---|---|---|
| Fabric contract tests (`test_views_xform_prim_fabric.py`) | 17 | 16 |
1 | 34 |
| USD contract tests (`test_views_xform_prim.py`) | 45 | 0 | 0 | 45 |
| Camera render test (`test_tiled_camera.py`) | 8 | 0 | 0 | 8 |

## Type of change

- Performance improvement (removes redundant USD writeback on Fabric
operations)

## Checklist

- [x] I have read and understood the [contribution
guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html)
- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

*No doc changes needed (parameter wasn't referenced in any docs)*

Author: pv-nvidia

CONTRIBUTORS.md

@@ -144,6 +144,7 @@ Guidelines for modifications:
 * Patrick Yin
 * Paul Reeves
 * Peter Du
+* Peter Verswyvelen
 * Philipp Reist
 * Piotr Barejko
 * Pulkit Goyal

source/isaaclab/changelog.d/fix-fabric-prepare-for-reuse.rst

@@ -0,0 +1,8 @@
+Changed
+^^^^^^^
+
+* Updated :class:`~isaaclab.sensors.camera.Camera` to construct its internal
+  :class:`~isaaclab.sim.views.FrameView` without the now-removed
+  ``sync_usd_on_fabric_write`` kwarg.  USD attributes on camera prims are
+  no longer kept in sync with Fabric writes; read poses through the view's
+  getters instead.

source/isaaclab/isaaclab/sensors/camera/camera.py

@@ -18,8 +18,7 @@
 import isaaclab.sim as sim_utils
 import isaaclab.utils.sensors as sensor_utils
 from isaaclab.app.settings_manager import get_settings_manager
-from isaaclab.renderers import BaseRenderer
-from isaaclab.renderers.camera_render_spec import CameraRenderSpec
+from isaaclab.renderers import BaseRenderer, CameraRenderSpec
 from isaaclab.sim.views import FrameView
 from isaaclab.utils import to_camel_case
 from isaaclab.utils.math import (
@@ -380,9 +379,7 @@ def _initialize_impl(self):
         # references to prims located in the stage.
         sim_ctx.render_context.ensure_prepare_stage(self.stage, self._num_envs)
 
-        # Create a view for the sensor with Fabric enabled for fast pose queries.
-        # TODO: remove sync_usd_on_fabric_write=True once the GPU Fabric sync bug is fixed.
-        self._view = FrameView(self.cfg.prim_path, device=self._device, stage=self.stage, sync_usd_on_fabric_write=True)
+        self._view = FrameView(self.cfg.prim_path, device=self._device, stage=self.stage)
         # Check that sizes are correct
         if self._view.count != self._num_envs:
             raise RuntimeError(

source/isaaclab/isaaclab/sim/views/usd_frame_view.py

@@ -72,8 +72,7 @@ def __init__(
             stage: USD stage to search for prims. Defaults to None, in which case the current
                 active stage from the simulation context is used.
             **kwargs: Additional keyword arguments (ignored). Allows forward-compatible
-                construction when callers pass backend-specific options like
-                ``sync_usd_on_fabric_write``.
+                construction when callers pass backend-specific options.
 
         Raises:
             ValueError: If any matched prim is not Xformable or doesn't have standardized

source/isaaclab/test/sensors/test_camera.py

@@ -1161,6 +1161,72 @@ def cleanup(self, render_data):
             Renderer._registry.pop(backend, None)
 
 
+@pytest.mark.parametrize("device", ["cuda:0", "cpu"])
+@pytest.mark.isaacsim_ci
+def test_camera_pose_update_reflected_in_render(setup_camera_device, device):
+    """Camera pose changes via FrameView should be visible in rendered depth.
+
+    Moves the camera close then far, renders depth, and verifies that the mean
+    valid depth from the far position is significantly larger (>1.5×) than the
+    close position.  This validates that Fabric-side pose writes (via
+    PrepareForReuse) and USD writes are correctly propagated to the RTX
+    renderer.
+    """
+    sim, _unused_cam_cfg, dt = setup_camera_device
+
+    cam_cfg = CameraCfg(
+        prim_path="/World/PoseTestCam",
+        height=128,
+        width=256,
+        update_period=0,
+        update_latest_camera_pose=True,
+        data_types=["distance_to_camera"],
+        spawn=sim_utils.PinholeCameraCfg(
+            focal_length=24.0,
+            focus_distance=400.0,
+            horizontal_aperture=20.955,
+            clipping_range=(0.1, 1.0e5),
+        ),
+    )
+    camera = Camera(cam_cfg)
+    try:
+        sim.reset()
+
+        target = torch.tensor([[0.0, 0.0, 0.0]], dtype=torch.float32, device=camera.device)
+        max_range = cam_cfg.spawn.clipping_range[1]
+
+        # -- close position --
+        eyes_close = torch.tensor([[2.0, 2.0, 2.0]], dtype=torch.float32, device=camera.device)
+        camera.set_world_poses_from_view(eyes_close, target)
+        sim.step()
+        camera.update(dt)
+        depth_close = camera.data.output["distance_to_camera"].clone()
+
+        # -- far position --
+        eyes_far = torch.tensor([[8.0, 8.0, 8.0]], dtype=torch.float32, device=camera.device)
+        camera.set_world_poses_from_view(eyes_far, target)
+        sim.step()
+        camera.update(dt)
+        depth_far = camera.data.output["distance_to_camera"].clone()
+
+        # -- validate --
+        valid_close = depth_close[depth_close < max_range]
+        valid_far = depth_far[depth_far < max_range]
+
+        assert valid_close.numel() > 0, "No valid close-range depth pixels"
+        assert valid_far.numel() > 0, "No valid far-range depth pixels"
+
+        mean_close = valid_close.mean().item()
+        mean_far = valid_far.mean().item()
+
+        assert mean_far > mean_close * 1.5, (
+            f"Far depth ({mean_far:.2f}) should be > 1.5× close depth ({mean_close:.2f}). "
+            "Camera pose change may not be reaching the renderer."
+        )
+    finally:
+        del camera
+
+
 def _populate_scene():
     """Add prims to the scene."""
     # Ground-plane

source/isaaclab/test/sensors/test_multi_mesh_ray_caster_camera.py

@@ -752,11 +752,11 @@ def test_output_equal_to_usd_camera_when_intrinsics_set(setup_simulation):
 
     # set camera position
     camera_warp.set_world_poses_from_view(
-        eyes=torch.tensor([[0.0, 0.0, 5.0]], device=camera_warp.device),
+        eyes=torch.tensor([[0.1, 0.0, 5.0]], device=camera_warp.device),
         targets=torch.tensor([[0.0, 0.0, 0.0]], device=camera_warp.device),
     )
     camera_usd.set_world_poses_from_view(
-        eyes=torch.tensor([[0.0, 0.0, 5.0]], device=camera_usd.device),
+        eyes=torch.tensor([[0.1, 0.0, 5.0]], device=camera_usd.device),
         targets=torch.tensor([[0.0, 0.0, 0.0]], device=camera_usd.device),
     )
 

source/isaaclab/test/sensors/test_ray_caster_camera.py

@@ -898,11 +898,11 @@ def test_output_equal_to_usd_camera_when_intrinsics_set(setup_sim, focal_length_
 
     # set camera position
     camera_warp.set_world_poses_from_view(
-        eyes=torch.tensor([[0.001, 0.0, 5.0]], device=camera_warp.device),
+        eyes=torch.tensor([[0.1, 0.0, 5.0]], device=camera_warp.device),
         targets=torch.tensor([[0.0, 0.0, 0.0]], device=camera_warp.device),
     )
     camera_usd.set_world_poses_from_view(
-        eyes=torch.tensor([[0.001, 0.0, 5.0]], device=camera_usd.device),
+        eyes=torch.tensor([[0.1, 0.0, 5.0]], device=camera_usd.device),
         targets=torch.tensor([[0.0, 0.0, 0.0]], device=camera_usd.device),
     )
 

source/isaaclab_physx/changelog.d/fix-fabric-prepare-for-reuse.rst

@@ -0,0 +1,12 @@
+Changed
+^^^^^^^
+
+* **Breaking:** Removed the ``sync_usd_on_fabric_write`` keyword argument from
+  :class:`~isaaclab_physx.sim.views.FabricFrameView`.  Fabric writes
+  (``set_world_poses``, ``set_scales``) now notify the renderer via
+  ``PrepareForReuse()`` on the underlying ``PrimSelection`` instead of writing
+  back to USD, which is ~200x faster and avoids the stale USD shadow state the
+  old path produced.  Callers passing ``sync_usd_on_fabric_write=True`` should
+  remove the argument; if they relied on USD reflecting Fabric writes, they
+  should now read Fabric poses directly via the view's getters or refresh USD
+  explicitly.