Skip to content

Commit b1aba48

Browse files
authored
Move TiledCamera implementation into Camera and deprecate TiledCamera. (#5162)
# Description Unify `Camera` and `TiledCamera` by moving TiledCamera's renderer-based implementation into `Camera` and making `TiledCamera` a deprecated subclass alias. ### What changed **`base_renderer.py` (interface change):** - `write_output(render_data, output_name, output_data)` replaced by `read_output(render_data, camera_data)`. The renderer now receives the full `CameraData` instance and owns the per-output iteration loop. **`camera.py`:** - `_initialize_impl` now uses `Renderer(cfg.renderer_cfg)` + `renderer.prepare_stage()` + `renderer.create_render_data()` instead of replicator imports and per-camera render product loops - `_create_buffers` uses eager pre-allocation + `renderer.set_outputs()` instead of lazy allocation - Removed: `_rep_registry`, `_render_product_paths`, `_create_annotator_data()`, `_process_annotator_output()`, `render_product_paths` property, `omni.replicator.core` import, `SyntheticData` import **`camera_data.py`: `info` type change:** - `CameraData.info` changed from `list[dict[str, Any]]` to `dict[str, Any]`, matching the Replicator annotator output structure. The old `list[dict]` was an artifact of the original `Camera`'s per-camera-annotator architecture. With tiled rendering, all cameras share a single annotator pipeline, so the per-camera list dimension carried no information. Access is now `camera.data.info["semantic_segmentation"]` directly. **`tiled_camera.py`:** - Replaced entire class with a thin `class TiledCamera(Camera)` subclass that emits a `DeprecationWarning` on instantiation **`tiled_camera_cfg.py`:** - Added `DeprecationWarning` in `__post_init__` **Test suite:** - Ported 6 unique tests from `test_tiled_camera.py` → `test_camera.py` (multi-regex init, all annotators, segmentation non-colorize, normals unit length, data types ordering, frame offset) - Slimmed `test_tiled_camera.py` to 4 focused deprecation/compatibility tests - Updated `info` assertions across all test files to use flat `dict` access pattern (`info["key"]` instead of `info[0]["key"]`) ### Breaking changes - `render_product_paths` property removed (confirmed zero external usage across entire repo — only referenced internally in `camera.py`, `isaac_rtx_renderer.py`, and `ovrtx_renderer.py`) - `CameraData.info` changed from `list[dict[str, Any]]` to `dict[str, Any]`. `Camera` users (old): replace `camera.data.info[cam_idx][data_type]` with `camera.data.info[data_type]`. `TiledCamera` users (old): access pattern `camera.data.info[data_type]` is unchanged. - `BaseRenderer.write_output` replaced by `BaseRenderer.read_output(render_data, camera_data)`. Custom renderer implementations must update their signature. ## Type of change - Breaking change (existing functionality will not work without user modification) ## Screenshots ## Checklist - [x] I have read and understood the [contribution guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html) - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./isaaclab.sh --format` - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] I have updated the changelog and the corresponding version in the extension's `config/extension.toml` file - [x] I have added my name to the `CONTRIBUTORS.md` or my name already exists there
1 parent abbf838 commit b1aba48

21 files changed

Lines changed: 636 additions & 2307 deletions

File tree

CONTRIBUTORS.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,7 @@ Guidelines for modifications:
7373
* Dhyan Thakkar
7474
* Dongxuan Fan
7575
* Dorsa Rohani
76+
* Ege Sekkin
7677
* Emily Sturman
7778
* Emmanuel Ferdman
7879
* Fabian Jenelten

docs/source/how-to/save_camera_output.rst

Lines changed: 5 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,10 @@ directory.
2121
Saving using Replicator Basic Writer
2222
------------------------------------
2323

24+
.. note::
25+
The BasicWriter is part of the Omniverse Replicator ecosystem and is specific to the default
26+
Isaac RTX renderer backend. Other renderer backends may require different save workflows.
27+
2428
To save camera outputs, we use the basic write class from Omniverse Replicator. This class allows us to save the
2529
images in a numpy format. For more information on the basic writer, please check the
2630
`documentation <https://docs.omniverse.nvidia.com/extensions/latest/ext_replicator/writer_examples.html>`_.
@@ -32,18 +36,11 @@ images in a numpy format. For more information on the basic writer, please check
3236

3337
While stepping the simulator, the images can be saved to the defined folder. Since the BasicWriter only supports
3438
saving data using NumPy format, we first need to convert the PyTorch sensors to NumPy arrays before packing
35-
them in a dictionary.
39+
them in a dictionary and writing with the BasicWriter.
3640

3741
.. literalinclude:: ../../../scripts/tutorials/04_sensors/run_usd_camera.py
3842
:language: python
3943
:start-at: # Save images from camera at camera_index
40-
:end-at: single_cam_info = camera.data.info[camera_index]
41-
42-
After this step, we can save the images using the BasicWriter.
43-
44-
.. literalinclude:: ../../../scripts/tutorials/04_sensors/run_usd_camera.py
45-
:language: python
46-
:start-at: # Pack data back into replicator format to save them using its writer
4744
:end-at: rep_writer.write(rep_output)
4845

4946

scripts/tutorials/04_sensors/run_ray_caster_camera.py

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -132,12 +132,10 @@ def run_simulator(sim: sim_utils.SimulationContext, scene_entities: dict):
132132
single_cam_data = convert_dict_to_backend(
133133
{k: v[camera_index] for k, v in camera.data.output.items()}, backend="numpy"
134134
)
135-
# Extract the other information
136-
single_cam_info = camera.data.info[camera_index]
137-
138135
# Pack data back into replicator format to save them using its writer
139136
rep_output = {"annotators": {}}
140-
for key, data, info in zip(single_cam_data.keys(), single_cam_data.values(), single_cam_info.values()):
137+
for key, data in single_cam_data.items():
138+
info = camera.data.info.get(key)
141139
if info is not None:
142140
rep_output["annotators"][key] = {"render_product": {"data": data, **info}}
143141
else:

scripts/tutorials/04_sensors/run_usd_camera.py

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -232,12 +232,10 @@ def run_simulator(sim: sim_utils.SimulationContext, scene_entities: dict):
232232
{k: v[camera_index] for k, v in camera.data.output.items()}, backend="numpy"
233233
)
234234

235-
# Extract the other information
236-
single_cam_info = camera.data.info[camera_index]
237-
238235
# Pack data back into replicator format to save them using its writer
239236
rep_output = {"annotators": {}}
240-
for key, data, info in zip(single_cam_data.keys(), single_cam_data.values(), single_cam_info.values()):
237+
for key, data in single_cam_data.items():
238+
info = camera.data.info.get(key)
241239
if info is not None:
242240
rep_output["annotators"][key] = {"render_product": {"data": data, **info}}
243241
else:

source/isaaclab/config/extension.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
[package]
22

33
# Note: Semantic Versioning is used: https://semver.org/
4-
version = "4.5.33"
4+
version = "4.6.0"
55

66
# Description
77
title = "Isaac Lab framework for Robot Learning"

source/isaaclab/docs/CHANGELOG.rst

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,45 @@
11
Changelog
22
---------
33

4+
4.6.0 (2026-04-13)
5+
~~~~~~~~~~~~~~~~~~
6+
7+
Changed
8+
^^^^^^^
9+
10+
* Unified :class:`~isaaclab.sensors.camera.Camera` and :class:`~isaaclab.sensors.camera.TiledCamera`
11+
into a single implementation. :class:`Camera` now delegates all rendering to the
12+
:class:`~isaaclab.renderers.Renderer` abstraction (same approach :class:`TiledCamera` used).
13+
The public API is unchanged for :class:`Camera` users.
14+
* **Breaking:** :attr:`~isaaclab.sensors.camera.CameraData.info` is now a flat
15+
``dict[str, Any]`` keyed by data type (e.g. ``camera.data.info["semantic_segmentation"]``).
16+
The metadata is shared across all cameras and identical to what the underlying renderer returns.
17+
18+
- **Camera users (old):** replace ``camera.data.info[cam_idx][data_type]`` with
19+
``camera.data.info[data_type]``.
20+
- **TiledCamera users (old):** access pattern ``camera.data.info[data_type]`` is unchanged.
21+
22+
* **Breaking:** :meth:`~isaaclab.renderers.BaseRenderer.write_output` has been replaced by
23+
:meth:`~isaaclab.renderers.BaseRenderer.read_output`. The new method receives the full
24+
:class:`~isaaclab.sensors.camera.CameraData` instance and iterates output types internally.
25+
Custom renderer implementations must replace ``write_output(render_data, output_name, output_data)``
26+
with ``read_output(render_data, camera_data)``.
27+
28+
Deprecated
29+
^^^^^^^^^^
30+
31+
* :class:`~isaaclab.sensors.camera.TiledCamera` is deprecated. Use
32+
:class:`~isaaclab.sensors.camera.Camera` directly — it now supports all renderer backends.
33+
* :class:`~isaaclab.sensors.camera.TiledCameraCfg` is deprecated. Use
34+
:class:`~isaaclab.sensors.camera.CameraCfg` directly.
35+
36+
Removed
37+
^^^^^^^
38+
39+
* Removed :attr:`~isaaclab.sensors.camera.Camera.render_product_paths`. Render products are
40+
now managed internally by the renderer backend and are not part of the public API.
41+
42+
443
4.5.33 (2026-04-13)
544
~~~~~~~~~~~~~~~~~~~
645

source/isaaclab/isaaclab/renderers/base_renderer.py

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
import torch
1515

1616
from isaaclab.sensors import SensorBase
17+
from isaaclab.sensors.camera.camera_data import CameraData
1718

1819

1920
class BaseRenderer(ABC):
@@ -93,13 +94,13 @@ def render(self, render_data: Any) -> None:
9394
pass
9495

9596
@abstractmethod
96-
def write_output(self, render_data: Any, output_name: str, output_data: torch.Tensor) -> None:
97-
"""Write a specific output type to the given buffer.
97+
def read_output(self, render_data: Any, camera_data: CameraData) -> None:
98+
"""Read rendered outputs from the renderer into the camera data container.
9899
99100
Args:
100101
render_data: The render data object from :meth:`create_render_data`.
101-
output_name: Name of the output (e.g. ``"rgba"``, ``"depth"``).
102-
output_data: Pre-allocated tensor to write the output into.
102+
camera_data: The :class:`~isaaclab.sensors.camera.camera_data.CameraData`
103+
instance to populate.
103104
"""
104105
pass
105106

0 commit comments

Comments
 (0)