Skip to content

Commit b1120e0

Browse files
Moves tactile sensor to contrib folder (#4481)
# Description As per our internal discussions, we are moving the tactile sensor implementation to the `isaaclab_contrib` module. We will move it back to the core module once the sensor receives sufficient testing, validation, and API stabilization. ## Type of change - Breaking change (existing functionality will not work without user modification) - Documentation update ## Checklist - [ ] I have read and understood the [contribution guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html) - [ ] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./isaaclab.sh --format` - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have updated the changelog and the corresponding version in the extension's `config/extension.toml` file - [ ] I have added my name to the `CONTRIBUTORS.md` or my name already exists there --------- Signed-off-by: Kelly Guo <kellyg@nvidia.com> Co-authored-by: Kelly Guo <kellyg@nvidia.com>
1 parent 2f192f2 commit b1120e0

24 files changed

Lines changed: 444 additions & 86 deletions

File tree

docs/source/_static/refs.bib

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -194,3 +194,14 @@ @article{kulkarni2025aerialgym
194194
pages={4093-4100},
195195
doi={10.1109/LRA.2025.3548507}
196196
}
197+
198+
@article{si2022taxim,
199+
title={Taxim: An example-based simulation model for gelsight tactile sensors},
200+
author={Si, Zilin and Yuan, Wenzhen},
201+
journal={IEEE Robotics and Automation Letters},
202+
volume={7},
203+
number={2},
204+
pages={2361--2368},
205+
year={2022},
206+
publisher={IEEE}
207+
}

docs/source/api/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,7 @@ The following modules are available in the ``isaaclab_contrib`` extension:
7979
actuators
8080
assets
8181
mdp
82+
sensors
8283

8384
isaaclab_tasks extension
8485
------------------------

docs/source/api/lab/isaaclab.sensors.rst

Lines changed: 0 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -38,9 +38,6 @@
3838
MultiMeshRayCasterCameraCfg
3939
Imu
4040
ImuCfg
41-
VisuoTactileSensor
42-
VisuoTactileSensorCfg
43-
VisuoTactileSensorData
4441

4542
Sensor Base
4643
-----------
@@ -207,22 +204,3 @@ Inertia Measurement Unit
207204
:inherited-members:
208205
:show-inheritance:
209206
:exclude-members: __init__, class_type
210-
211-
Visuo-Tactile Sensor
212-
--------------------
213-
214-
.. autoclass:: VisuoTactileSensor
215-
:members:
216-
:inherited-members:
217-
:show-inheritance:
218-
219-
.. autoclass:: VisuoTactileSensorData
220-
:members:
221-
:inherited-members:
222-
:exclude-members: __init__
223-
224-
.. autoclass:: VisuoTactileSensorCfg
225-
:members:
226-
:inherited-members:
227-
:show-inheritance:
228-
:exclude-members: __init__, class_type
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
isaaclab_contrib.sensors
2+
========================
3+
4+
.. automodule:: isaaclab_contrib.sensors
5+
6+
.. rubric:: Classes
7+
8+
.. autosummary::
9+
10+
VisuoTactileSensor
11+
VisuoTactileSensorCfg
12+
VisuoTactileSensorData
13+
GelSightRenderCfg
14+
15+
16+
Visuo-Tactile Sensor
17+
--------------------
18+
19+
.. autoclass:: VisuoTactileSensor
20+
:members:
21+
:inherited-members:
22+
:show-inheritance:
23+
24+
.. autoclass:: VisuoTactileSensorData
25+
:members:
26+
:inherited-members:
27+
:exclude-members: __init__
28+
29+
.. autoclass:: VisuoTactileSensorCfg
30+
:members:
31+
:inherited-members:
32+
:show-inheritance:
33+
:exclude-members: __init__, class_type
34+
35+
.. autoclass:: GelSightRenderCfg
36+
:members:
37+
:inherited-members:
38+
:show-inheritance:
39+
:exclude-members: __init__

docs/source/overview/core-concepts/sensors/visuo_tactile_sensor.rst

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,11 +22,12 @@ Tactile sensors require specific configuration parameters to define their behavi
2222

2323
.. code-block:: python
2424
25-
from isaaclab.sensors.tacsl_sensor import VisuoTactileSensorCfg
2625
from isaaclab.sensors import TiledCameraCfg
2726
from isaaclab_assets.sensors import GELSIGHT_R15_CFG
2827
import isaaclab.sim as sim_utils
2928
29+
from isaaclab_contrib.sensors.tacsl_sensor import VisuoTactileSensorCfg
30+
3031
# Tactile sensor configuration
3132
tactile_sensor = VisuoTactileSensorCfg(
3233
prim_path="{ENV_REGEX_NS}/Robot/elastomer/tactile_sensor",

scripts/demos/sensors/tacsl_sensor.py

Lines changed: 23 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
Example script demonstrating the TacSL tactile sensor implementation in IsaacLab.
88
99
This script shows how to use the TactileSensor for both camera-based and force field
10-
tactile sensing with the gelsight finger setup.
10+
tactile sensing with the GelSight finger setup.
1111
1212
.. code-block:: bash
1313
@@ -80,15 +80,15 @@
8080
import isaaclab.sim as sim_utils
8181
from isaaclab.assets import ArticulationCfg, AssetBaseCfg, RigidObjectCfg
8282
from isaaclab.scene import InteractiveScene, InteractiveSceneCfg
83-
84-
# Import our TactileSensor
85-
from isaaclab.sensors import TiledCameraCfg, VisuoTactileSensorCfg
86-
from isaaclab.sensors.tacsl_sensor.visuotactile_render import compute_tactile_shear_image
87-
from isaaclab.sensors.tacsl_sensor.visuotactile_sensor_data import VisuoTactileSensorData
83+
from isaaclab.sensors import TiledCameraCfg
8884
from isaaclab.utils import configclass
8985
from isaaclab.utils.assets import ISAACLAB_NUCLEUS_DIR
9086
from isaaclab.utils.timer import Timer
9187

88+
from isaaclab_contrib.sensors.tacsl_sensor import VisuoTactileSensorCfg
89+
from isaaclab_contrib.sensors.tacsl_sensor.visuotactile_render import compute_tactile_shear_image
90+
from isaaclab_contrib.sensors.tacsl_sensor.visuotactile_sensor_data import VisuoTactileSensorData
91+
9292
from isaaclab_assets.sensors import GELSIGHT_R15_CFG
9393

9494

@@ -224,10 +224,13 @@ def mkdir_helper(dir_path: str) -> tuple[str, str]:
224224
"""
225225
tactile_img_folder = dir_path
226226
os.makedirs(tactile_img_folder, exist_ok=True)
227+
# create a subdirectory for the force field data
227228
tactile_force_field_dir = os.path.join(tactile_img_folder, "tactile_force_field")
228229
os.makedirs(tactile_force_field_dir, exist_ok=True)
230+
# create a subdirectory for the RGB image data
229231
tactile_rgb_image_dir = os.path.join(tactile_img_folder, "tactile_rgb_image")
230232
os.makedirs(tactile_rgb_image_dir, exist_ok=True)
233+
231234
return tactile_force_field_dir, tactile_rgb_image_dir
232235

233236

@@ -268,9 +271,13 @@ def save_viz_helper(
268271
tactile_shear_force[1, :, :].detach().cpu().numpy(),
269272
)
270273
combined_image = np.vstack([tactile_image, tactile_image_1])
271-
cv2.imwrite(os.path.join(tactile_force_field_dir, f"{count}.png"), (combined_image * 255).astype(np.uint8))
274+
cv2.imwrite(
275+
os.path.join(tactile_force_field_dir, f"{count:04d}.png"), (combined_image * 255).astype(np.uint8)
276+
)
272277
else:
273-
cv2.imwrite(os.path.join(tactile_force_field_dir, f"{count}.png"), (tactile_image * 255).astype(np.uint8))
278+
cv2.imwrite(
279+
os.path.join(tactile_force_field_dir, f"{count:04d}.png"), (tactile_image * 255).astype(np.uint8)
280+
)
274281

275282
if tactile_data.tactile_rgb_image is not None:
276283
tactile_rgb_data = tactile_data.tactile_rgb_image.cpu().numpy()
@@ -284,7 +291,7 @@ def save_viz_helper(
284291
if tactile_rgb_tiled.max() <= 1.0
285292
else tactile_rgb_tiled.astype(np.uint8)
286293
)
287-
cv2.imwrite(os.path.join(tactile_rgb_image_dir, f"{count}.png"), tactile_rgb_tiled)
294+
cv2.imwrite(os.path.join(tactile_rgb_image_dir, f"{count:04d}.png"), tactile_rgb_tiled)
288295

289296

290297
def run_simulator(sim: sim_utils.SimulationContext, scene: InteractiveScene):
@@ -299,6 +306,7 @@ def run_simulator(sim: sim_utils.SimulationContext, scene: InteractiveScene):
299306

300307
if args_cli.save_viz:
301308
# Create output directories for tactile data
309+
print(f"[INFO]: Saving tactile data to: {args_cli.save_viz_dir}...")
302310
dir_path_list = mkdir_helper(args_cli.save_viz_dir)
303311

304312
# Create constant downward force
@@ -337,7 +345,7 @@ def run_simulator(sim: sim_utils.SimulationContext, scene: InteractiveScene):
337345
even_mask = env_indices % 2 == 0
338346
torque_tensor[odd_mask, 0, 2] = 10 # rotation for odd environments
339347
torque_tensor[even_mask, 0, 2] = -10 # rotation for even environments
340-
scene["contact_object"].set_external_force_and_torque(force_tensor, torque_tensor)
348+
scene["contact_object"].permanent_wrench_composer.set_forces_and_torques(force_tensor, torque_tensor)
341349

342350
# Step simulation
343351
scene.write_data_to_sim()
@@ -380,7 +388,7 @@ def main():
380388
sim = sim_utils.SimulationContext(sim_cfg)
381389

382390
# Set main camera
383-
sim.set_camera_view(eye=[1.5, 1.5, 1.5], target=[0.0, 0.0, 0.0])
391+
sim.set_camera_view(eye=[0.5, 0.6, 1.0], target=[-0.1, 0.1, 0.5])
384392

385393
# Create scene based on contact object type
386394
if args_cli.contact_object_type == "cube":
@@ -395,6 +403,10 @@ def main():
395403
scene_cfg.tactile_sensor.contact_object_prim_path_expr = None
396404
# this flag is to visualize the tactile sensor points
397405
scene_cfg.tactile_sensor.debug_vis = True
406+
else:
407+
raise ValueError(
408+
f"Invalid contact object type: '{args_cli.contact_object_type}'. Must be 'none', 'cube', or 'nut'."
409+
)
398410

399411
scene = InteractiveScene(scene_cfg)
400412

source/isaaclab/config/extension.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
[package]
22

33
# Note: Semantic Versioning is used: https://semver.org/
4-
version = "0.54.1"
4+
version = "0.54.2"
55

66
# Description
77
title = "Isaac Lab framework for Robot Learning"

source/isaaclab/docs/CHANGELOG.rst

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,16 @@
11
Changelog
22
---------
33

4+
0.54.2 (2026-01-28)
5+
~~~~~~~~~~~~~~~~~~~
6+
7+
Changed
8+
^^^^^^^
9+
10+
* Moved :mod:`isaaclab.sensors.tacsl_sensor` to :mod:`isaaclab_contrib.sensors.tacsl_sensor` module,
11+
since it is not completely ready for release yet.
12+
13+
414
0.54.1 (2026-01-25)
515
~~~~~~~~~~~~~~~~~~~
616

source/isaaclab/isaaclab/scene/interactive_scene.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,13 +27,17 @@
2727
SurfaceGripper,
2828
SurfaceGripperCfg,
2929
)
30-
from isaaclab.sensors import ContactSensorCfg, FrameTransformerCfg, SensorBase, SensorBaseCfg, VisuoTactileSensorCfg
30+
from isaaclab.sensors import ContactSensorCfg, FrameTransformerCfg, SensorBase, SensorBaseCfg
3131
from isaaclab.sim import SimulationContext
3232
from isaaclab.sim.utils.stage import get_current_stage, get_current_stage_id
3333
from isaaclab.sim.views import XformPrimView
3434
from isaaclab.terrains import TerrainImporter, TerrainImporterCfg
3535
from isaaclab.utils.version import get_isaac_sim_version
3636

37+
# Note: This is a temporary import for the VisuoTactileSensorCfg class.
38+
# It will be removed once the VisuoTactileSensor class is added to the core Isaac Lab framework.
39+
from isaaclab_contrib.sensors.tacsl_sensor import VisuoTactileSensorCfg
40+
3741
from .interactive_scene_cfg import InteractiveSceneCfg
3842

3943
# import logger

source/isaaclab/isaaclab/sensors/__init__.py

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -32,8 +32,6 @@
3232
+---------------------+---------------------------+---------------------------------------------------------------+
3333
| Imu | /World/robot/base | Leaf exists and is a physics body (Rigid Body) |
3434
+---------------------+---------------------------+---------------------------------------------------------------+
35-
| Visuo-Tactile Sensor| /World/robot/base | Leaf exists and is a physics body (Rigid Body) |
36-
+---------------------+---------------------------+---------------------------------------------------------------+
3735
3836
"""
3937

@@ -44,4 +42,3 @@
4442
from .ray_caster import * # noqa: F401, F403
4543
from .sensor_base import SensorBase # noqa: F401
4644
from .sensor_base_cfg import SensorBaseCfg # noqa: F401
47-
from .tacsl_sensor import * # noqa: F401, F403

0 commit comments

Comments
 (0)