Skip to content

Commit 0ca998c

Browse files
authored
Merge branch 'develop' into antoiner/feat/ovphysx_articulation
2 parents 311604b + 9989d27 commit 0ca998c

127 files changed

Lines changed: 6824 additions & 1384 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/actions/run-tests/action.yml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -214,9 +214,19 @@ runs:
214214
mkdir -p tests
215215
rm _isaac_sim || true
216216
ln -s /isaac-sim _isaac_sim
217+
# Allow OmniHub to start in the test container. Some base images
218+
# set this detect-only flag, which makes cold asset downloads
219+
# fall back to slow repeated retries.
220+
unset HUB__ARGS__DETECT_ONLY
217221
if [ -n \"\${TEST_EXTRA_PIP_PACKAGES:-}\" ]; then
218222
echo \"Installing extra pip packages: \${TEST_EXTRA_PIP_PACKAGES}\"
219223
./isaaclab.sh -p -m pip install \${TEST_EXTRA_PIP_PACKAGES}
224+
case \" \${TEST_EXTRA_PIP_PACKAGES} \" in
225+
*\" leapp\"*)
226+
echo \"Resolved LEAPP package:\"
227+
./isaaclab.sh -p -m pip show leapp || true
228+
;;
229+
esac
220230
fi
221231
echo 'Starting pytest with path: $test_path'
222232
./isaaclab.sh -p -m pytest --ignore=tools/conftest.py --ignore=source/isaaclab/test/install_ci $test_path $pytest_options -v --junitxml=tests/$result_file

CONTRIBUTORS.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,7 @@ Guidelines for modifications:
8080
* Emily Sturman
8181
* Emmanuel Ferdman
8282
* Fabian Jenelten
83+
* Fatima Anes
8384
* Felipe Mohr
8485
* Felix Yu
8586
* Frank Lai
@@ -188,6 +189,7 @@ Guidelines for modifications:
188189
* Xiaodi Yuan
189190
* Xinjie Yao
190191
* Xinpeng Liu
192+
* Xu Li
191193
* Yang Jin
192194
* Yanzi Zhu
193195
* Yijie Guo

docs/source/experimental-features/bleeding-edge.rst

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ To address this, some major features will be released as Experimental Feature Br
1111
This way, the community can experiment with and contribute to the feature before it's fully integrated, reducing the likelihood of being derailed by unexpected and new errors.
1212

1313
RL Post-Training for VLA Models
14-
---------------------------------
14+
-------------------------------
1515

1616
`RLinf <https://github.com/RLinf/RLinf.git>`_ is a flexible and scalable open-source RL infrastructure designed for
1717
Embodied and Agentic AI. This integration enables **reinforcement learning fine-tuning of Vision-Language-Action
@@ -83,6 +83,9 @@ From the Isaac Lab root directory:
8383
pip install -e .[base] --no-deps
8484
cd ../
8585
86+
# Install flash-attn (must be built against the correct PyTorch)
87+
pip install --no-build-isolation flash-attn==2.8.3
88+
8689
Quick Start
8790
~~~~~~~~~~~
8891

docs/source/features/reproducibility.rst

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,29 @@ simulation results are reproducible across different runs. The seed is set into
2020
parameters :attr:`isaaclab.envs.ManagerBasedEnvCfg.seed` or :attr:`isaaclab.envs.DirectRLEnvCfg.seed`
2121
depending on the manager-based or direct environment implementation respectively.
2222

23+
App-level deterministic rendering via ``AppLauncher``
24+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
25+
26+
The ``--deterministic`` flag is provided by :meth:`isaaclab.app.AppLauncher.add_app_launcher_args`.
27+
After the simulation app starts, :class:`~isaaclab.app.app_launcher.AppLauncher` applies RTX/RTPT carb
28+
settings via :meth:`~isaaclab.app.app_launcher.AppLauncher.apply_rtx_determinism_settings`.
29+
30+
**Strict PyTorch determinism** (calling :meth:`~isaaclab.utils.seed.configure_seed` with
31+
``torch_deterministic=True`` when you pass ``--deterministic``) is wired into the RL training scripts
32+
for **RL-Games**, **skrl**, **RSL-RL**, and **Stable-Baselines3**: each calls
33+
:meth:`~isaaclab.utils.seed.configure_seed` after constructing its framework runner or agent object
34+
so library initialization is not disturbed, then training proceeds with the requested global RNG and
35+
optional PyTorch deterministic algorithms. Whether you need ``--deterministic`` at the app level
36+
depends on the workload: **physics-only** simulation does not require it; **RTX** rendering
37+
(non-minimal mode) does require it for reproducible imagery; **Newton** rendering does not require it.
38+
39+
To enable deterministic RTX settings from the app launcher, pass ``--deterministic``.
40+
41+
.. code-block:: bash
42+
43+
./isaaclab.sh -p scripts/reinforcement_learning/rl_games/train.py \
44+
--task Isaac-Cartpole-RGB-v0 --enable_cameras --headless --deterministic
45+
2346
For results on our determinacy testing for RL training, please check the GitHub Pull Request `#940`_.
2447

2548
.. tip::

docs/source/how-to/import_new_asset.rst

Lines changed: 110 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -46,11 +46,12 @@ is then passed to the :class:`~sim.converters.UrdfConverter` class.
4646
The URDF importer has various configuration parameters that can be set to control the behavior of the importer.
4747
The default values for the importer's configuration parameters are specified are in the :class:`~sim.converters.UrdfConverterCfg` class, and they are listed below. We made a few commonly modified settings to be available as command-line arguments when calling the ``convert_urdf.py``, and they are marked with ``*`` in the list. For a comprehensive list of the configuration parameters, please check the the documentation at `URDF importer`_.
4848

49+
Articulation and joint structure
50+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
51+
4952
* :attr:`~sim.converters.UrdfConverterCfg.fix_base` * - Whether to fix the base of the robot.
5053
This depends on whether you have a floating-base or fixed-base robot. The command-line flag is
5154
``--fix-base`` where when set, the importer will fix the base of the robot, otherwise it will default to floating-base.
52-
* :attr:`~sim.converters.UrdfConverterCfg.root_link_name` - The link on which the PhysX articulation root is placed.
53-
**Deprecated in URDF importer 3.0** — this option is ignored.
5455
* :attr:`~sim.converters.UrdfConverterCfg.merge_fixed_joints` * - Whether to merge the fixed joints.
5556
Usually, this should be set to ``True`` to reduce the asset complexity. The command-line flag is
5657
``--merge-joints`` where when set, the importer will merge the fixed joints, otherwise it will default to not merging the fixed joints.
@@ -65,10 +66,53 @@ The default values for the importer's configuration parameters are specified are
6566
We support two ways to set the gains:
6667

6768
* :attr:`~sim.converters.UrdfConverterCfg.JointDriveCfg.PDGainsCfg` - To directly set the stiffness and damping.
69+
Both ``stiffness`` and ``damping`` accept a single float (applied uniformly).
6870
* :attr:`~sim.converters.UrdfConverterCfg.JointDriveCfg.NaturalFrequencyGainsCfg` - To set the gains using the
6971
desired natural frequency response of the system. **Deprecated in URDF importer 3.0** — use
7072
``PDGainsCfg`` instead.
7173

74+
Geometry, collisions, and materials
75+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
76+
77+
* :attr:`~sim.converters.UrdfConverterCfg.collision_from_visuals` - Whether to create collision geometry
78+
from visual geometry when no explicit ``<collision>`` is defined for a link. Defaults to ``False``.
79+
* :attr:`~sim.converters.UrdfConverterCfg.collision_type` - The collision shape simplification to apply.
80+
One of ``"Convex Hull"`` (default), ``"Convex Decomposition"``, ``"Bounding Sphere"``, or ``"Bounding Cube"``.
81+
* :attr:`~sim.converters.UrdfConverterCfg.self_collision` - Whether to activate self-collisions between
82+
links of the articulation. Defaults to ``False``.
83+
* :attr:`~sim.converters.UrdfConverterCfg.merge_mesh` - Whether to merge meshes where possible to optimize
84+
the model. Defaults to ``False``.
85+
* :attr:`~sim.converters.UrdfConverterCfg.link_density` - Default density in ``kg/m^3`` for links whose
86+
``<inertial>`` properties are missing. ``0.0`` (default) leaves densities unchanged.
87+
88+
Asset resolution and output
89+
~~~~~~~~~~~~~~~~~~~~~~~~~~~
90+
91+
* :attr:`~sim.converters.UrdfConverterCfg.ros_package_paths` - List of ROS package name/path mappings used
92+
to resolve ``package://`` URLs in the URDF. Each entry is a dict with keys ``name`` and ``path``.
93+
* :attr:`~sim.converters.UrdfConverterCfg.robot_type` - Robot type applied by the USD robot schema.
94+
Defaults to ``"Default"``. Must be one of: ``"Default"``, ``"End Effector"``, ``"Manipulator"``,
95+
``"Humanoid"``, ``"Wheeled"``, ``"Holonomic"``, ``"Quadruped"``, ``"Mobile Manipulators"``, ``"Aerial"``.
96+
* :attr:`~sim.converters.UrdfConverterCfg.run_asset_transformer` - Run the asset transformer to convert
97+
the flattened USD into a layered USD (interface USD + payloads). Defaults to ``True``.
98+
* :attr:`~sim.converters.UrdfConverterCfg.run_multi_physics_conversion` - Also emit MuJoCo-compatible joint
99+
attributes alongside PhysX. Defaults to ``True``.
100+
* :attr:`~sim.converters.UrdfConverterCfg.debug_mode` - Write intermediate conversion artifacts next to the
101+
output USD for inspection. Defaults to ``False``.
102+
103+
Deprecated (no-op in URDF importer 3.0)
104+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
105+
106+
The following options are retained for backwards compatibility but are ignored by the URDF importer 3.0.
107+
A warning is logged when they are set.
108+
109+
* :attr:`~sim.converters.UrdfConverterCfg.root_link_name` - The link on which the PhysX articulation root
110+
was previously placed.
111+
* :attr:`~sim.converters.UrdfConverterCfg.convert_mimic_joints_to_normal_joints` - Convert mimic joints to
112+
normal joints during conversion.
113+
* :attr:`~sim.converters.UrdfConverterCfg.replace_cylinders_with_capsules` - Replace cylinder shapes with
114+
capsule shapes during conversion.
115+
72116
For more detailed information on the configuration parameters, please check the documentation for :class:`~sim.converters.UrdfConverterCfg`.
73117

74118
Example Usage
@@ -139,6 +183,14 @@ is derived automatically from the robot name in the URDF):
139183

140184
* ``anymal.usda`` - This is the main asset file.
141185

186+
.. note::
187+
The URDF importer auto-deduplicates the per-robot subdirectory when it already exists.
188+
If you re-run the converter against the same ``usd_dir`` with a changed configuration
189+
(for example, flipping ``fix_base``), the importer writes to a new numbered folder
190+
(``anymal_1/``, ``anymal_2/``, …) rather than overwriting the previous output.
191+
:attr:`~sim.converters.UrdfConverter.usd_path` reflects whichever folder the importer
192+
actually used. Delete stale subdirectories manually (or wipe ``usd_dir``) if you do not
193+
want them to accumulate on disk.
142194

143195
To run the script headless, you can add the ``--headless`` flag. This will not open the GUI and
144196
exit the script after the conversion is complete.
@@ -169,22 +221,64 @@ parameters, please check the the documentation at `MJCF importer`_.
169221

170222
.. note::
171223
The MJCF importer was rewritten in Isaac Sim 5.0 to use the ``mujoco-usd-converter`` library.
172-
Settings such as ``fix_base``, ``import_sites``, ``import_inertia_tensor``, and ``make_instanceable``
173-
are no longer needed — the converter now handles these automatically based on the MJCF file content.
224+
Settings such as ``import_sites``, ``import_inertia_tensor``, and ``make_instanceable`` are no
225+
longer needed — the converter now handles these automatically based on the MJCF file content.
226+
227+
Geometry, collisions, and materials
228+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
174229

175230
* :attr:`~sim.converters.MjcfConverterCfg.merge_mesh` * - Whether to merge meshes where possible to
176231
optimize the model. The command-line flag is ``--merge-mesh``.
177232
* :attr:`~sim.converters.MjcfConverterCfg.collision_from_visuals` * - Whether to generate collision
178233
geometry from visual geometries. The command-line flag is ``--collision-from-visuals``.
179-
* :attr:`~sim.converters.MjcfConverterCfg.collision_type` * - Type of collision geometry to use
180-
(e.g. ``"default"``, ``"Convex Hull"``, ``"Convex Decomposition"``). The command-line flag is
181-
``--collision-type``.
234+
* :attr:`~sim.converters.MjcfConverterCfg.collision_type` * - The collision shape simplification to
235+
apply. One of ``"Convex Hull"`` (default), ``"Convex Decomposition"``, ``"Bounding Sphere"``, or
236+
``"Bounding Cube"``. The command-line flag is ``--collision-type``.
182237
* :attr:`~sim.converters.MjcfConverterCfg.self_collision` * - Whether to activate self-collisions
183238
between links of the articulation. The command-line flag is ``--self-collision``.
239+
240+
Articulation and physics
241+
~~~~~~~~~~~~~~~~~~~~~~~~
242+
243+
* :attr:`~sim.converters.MjcfConverterCfg.fix_base` - Whether to add a fixed joint between the world
244+
and the root rigid-body link. Defaults to ``False``.
245+
* :attr:`~sim.converters.MjcfConverterCfg.link_density` - Default density in ``kg/m^3`` for links whose
246+
``<inertial>`` properties are missing in the MJCF. ``0.0`` (default) leaves densities unchanged.
184247
* :attr:`~sim.converters.MjcfConverterCfg.import_physics_scene` * - Import physics scene properties
185248
(gravity, time step, etc.) from the MJCF file. Defaults to ``False``. The command-line flag is
186249
``--import-physics-scene``.
187250

251+
Actuator overrides
252+
~~~~~~~~~~~~~~~~~~
253+
254+
MuJoCo models actuators as an affine transformation ``tau = gain @ control + bias``. The following
255+
options override the values parsed from the MJCF on a per-actuator basis. Each defaults to ``None``,
256+
which leaves the parsed values unchanged.
257+
258+
* :attr:`~sim.converters.MjcfConverterCfg.override_gain_type` - The actuator gain type override (e.g.
259+
``"fixed"``).
260+
* :attr:`~sim.converters.MjcfConverterCfg.override_bias_type` - The actuator bias type override (e.g.
261+
``"affine"``).
262+
* :attr:`~sim.converters.MjcfConverterCfg.override_gain_prm` - The actuator gain parameter array override.
263+
Example for position control: ``[kp, 0, 0, 0, 0, 0, 0, 0, 0, 0]``.
264+
* :attr:`~sim.converters.MjcfConverterCfg.override_bias_prm` - The actuator bias parameter array override.
265+
Example for position control: ``[0, -kp, -kd, 0, 0, 0, 0, 0, 0, 0]``.
266+
267+
Asset resolution and output
268+
~~~~~~~~~~~~~~~~~~~~~~~~~~~
269+
270+
* :attr:`~sim.converters.MjcfConverterCfg.robot_type` - Robot type applied by the USD robot schema.
271+
Defaults to ``"Default"``. Must be one of: ``"Default"``, ``"End Effector"``, ``"Manipulator"``,
272+
``"Humanoid"``, ``"Wheeled"``, ``"Holonomic"``, ``"Quadruped"``, ``"Mobile Manipulators"``, ``"Aerial"``.
273+
* :attr:`~sim.converters.MjcfConverterCfg.run_asset_transformer` - Run the asset transformer to convert
274+
the flattened USD into a layered USD (interface USD + payloads). Defaults to ``True``.
275+
* :attr:`~sim.converters.MjcfConverterCfg.run_multi_physics_conversion` - Convert compatible MuJoCo
276+
attributes to PhysX attributes (e.g. actuator gains). Defaults to ``True``.
277+
* :attr:`~sim.converters.MjcfConverterCfg.debug_mode` - Write intermediate conversion artifacts next to
278+
the output USD for inspection. Defaults to ``False``.
279+
280+
For more detailed information on the configuration parameters, please check the documentation for :class:`~sim.converters.MjcfConverterCfg`.
281+
188282

189283
Example Usage
190284
~~~~~~~~~~~~~
@@ -234,6 +328,15 @@ Executing the above script will create the USD file inside the
234328

235329
* ``h1.usd`` - This is the converted USD asset file.
236330

331+
.. note::
332+
The MJCF importer auto-deduplicates the per-robot subdirectory when it already exists,
333+
matching the URDF importer's behavior. If you re-run the converter against the same
334+
``usd_dir`` with a changed configuration, the importer writes to a new numbered folder
335+
(``h1_1/``, ``h1_2/``, …) rather than overwriting the previous output.
336+
:attr:`~sim.converters.MjcfConverter.usd_path` reflects whichever folder the importer
337+
actually used. Delete stale subdirectories manually (or wipe ``usd_dir``) if you do not
338+
want them to accumulate on disk.
339+
237340
.. figure:: ../_static/tutorials/tutorial_convert_mjcf.jpg
238341
:align: center
239342
:figwidth: 100%

docs/source/overview/imitation-learning/humanoids_imitation.rst

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -636,6 +636,22 @@ Then, from the **Isaac-GR00T** directory, install GR00T N1.5 and its dependencie
636636
MAX_JOBS=4 uv pip install --no-build-isolation 'git+https://github.com/facebookresearch/pytorch3d.git@v0.7.9'
637637
uv pip install diffusers decord zmq
638638
639+
.. note::
640+
641+
**If you cannot install or use flash-attn**, an optional patch is provided that switches the
642+
bundled Eagle 2.5 VL model to PyTorch SDPA. Use this if ``flash-attn`` fails to build for your
643+
environment, or if it installs but raises a runtime error such as
644+
``RuntimeError: FlashAttention only supports Ampere GPUs or newer`` (for example on Blackwell
645+
GPUs, which ``flash-attn==2.7.1.post4`` does not have prebuilt kernels for). After the patch,
646+
finetune and rollout run on any CUDA arch supported by your PyTorch build, at the cost of
647+
flash-attn's training speedup. Skip the ``flash-attn`` install line above, then apply the
648+
patch from the **Isaac-GR00T** directory (the sibling layout above means the IsaacLab
649+
checkout is at ``../IsaacLab``):
650+
651+
.. code:: bash
652+
653+
git apply ../IsaacLab/scripts/imitation_learning/locomanipulation_sdg/gr00t/no_flash_attn.patch
654+
639655
Convert dataset to LeRobot format
640656
"""""""""""""""""""""""""""""""""
641657

docs/source/overview/reinforcement-learning/rl_existing_scripts.rst

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -360,15 +360,17 @@ large VLA models that don't fit on a single GPU.
360360
.. code:: bash
361361
362362
# Step 1: Install RLinf and its dependencies (from isaaclab_contrib)
363-
pip install -e "source/isaaclab_contrib[rlinf]"
363+
pip install -e "source/isaaclab_contrib[rlinf]" --ignore-requires-python
364364
365-
# Step 2: Clone and install Isaac-GR00T (for VLA model support)
366-
cd scripts/reinforcement_learning/rlinf
365+
# Step 2: Clone and install Isaac-GR00T (pinned version, for VLA model support)
367366
git clone https://github.com/NVIDIA/Isaac-GR00T.git
368-
pip install -e Isaac-GR00T/.[base] --no-deps
367+
cd Isaac-GR00T
368+
git checkout 4af2b622892f7dcb5aae5a3fb70bcb02dc217b96
369+
pip install -e .[base] --no-deps
370+
cd ../
369371
370372
# Step 3: Install flash-attn (must be built against the correct PyTorch)
371-
pip install --no-build-isolation flash-attn==2.7.1.post4
373+
pip install --no-build-isolation flash-attn==2.8.3
372374
373375
- Training a VLA agent with RLinf:
374376

docs/source/setup/installation/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -237,3 +237,4 @@ Please follow the steps :doc:`asset_caching` to enable asset caching and speed u
237237
source_installation
238238
isaaclab_pip_installation
239239
asset_caching
240+
uv run (experimental) <uv_run>

docs/source/setup/installation/kitless_installation.rst

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,8 @@ with MJWarp physics and the Newton visualizer:
4444
.. code-block:: batch
4545
4646
:: Install Isaac Lab (Newton backend, no Isaac Sim required)
47-
isaaclab.bat --install :: or isaaclab.bat -i
47+
:: or: isaaclab.bat -i
48+
isaaclab.bat --install
4849
4950
:: Kickoff training with MJWarp physics and Newton visualizer
5051
isaaclab.bat -p scripts\reinforcement_learning\rsl_rl\train.py ^
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
.. _uv-run-training:
2+
3+
``uv run`` Training and Play (Experimental)
4+
============================================
5+
6+
.. warning::
7+
8+
This feature is experimental and subject to change in future releases.
9+
10+
Install ``uv`` if you do not have it already:
11+
12+
.. code-block:: bash
13+
14+
curl -LsSf https://astral.sh/uv/install.sh | sh
15+
16+
Clone the repo and start training immediately — no virtual environment setup required:
17+
18+
.. code-block:: bash
19+
20+
git clone https://github.com/isaac-sim/IsaacLab.git
21+
cd IsaacLab
22+
23+
# Newton backend training without Isaac Sim
24+
uv run train --rl_library rsl_rl \
25+
--task Isaac-Cartpole-Direct-v0 --headless presets=newton_mjwarp
26+
27+
# Add OVRTX/OVPhysX extras only when the workflow needs them
28+
uv run --extra ov --extra rtx train --rl_library rsl_rl \
29+
--task Isaac-Cartpole-Direct-v0 --headless presets=newton_mjwarp
30+
31+
``uv`` resolves and manages the environment automatically on each invocation. Supported
32+
libraries for ``--rl_library`` are: ``rsl_rl``, ``rl_games``, ``skrl``, ``sb3``, and ``rlinf``.
33+
34+
Play / Evaluation
35+
-----------------
36+
37+
.. code-block:: bash
38+
39+
uv run play --rl_library rsl_rl --task <any_task>

0 commit comments

Comments
 (0)