You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Enhances Pink IK controller with null-space posture control and improv… (isaac-sim#3149)
# Description
<!--
Thank you for your interest in sending a pull request. Please make sure
to check the contribution guidelines.
Link:
https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html
-->
Enhance Pink IK Controller with Null Space Posture Control
This PR improves the Pink IK controller integration for better humanoid
robot control and more natural postures.
**Note**: Original this PR was staged in the internal repo (isaac-sim#547). It
has been moved here due to new Github workflow.
## Key Changes
### New Null Space Posture Task
- Added NullSpacePostureTask to enforce postural constraints on
shoulder/waist joints while prioritizing end-effector tasks
- Maintains natural robot poses during manipulation
### Controller Improvements
- Tuned low level PD controller gains
- Support mixed task types (FrameTask + NullSpacePostureTask)
### Testing & Environment Updates
- Redesigned pink controller test script to use JSON-based
configurations to program test motions.
- Updated all environments (PickPlace, NutPour, ExhaustPipe) with null
space control, damping tasks, and improved tracking
- Added `Isaac-PickPlace-GR1T2-WaistEnabled-Abs-v0` env that is
identical to `Isaac-PickPlace-GR1T2-Abs-v0` but enables the Waist DOFs.
- Added target_eef_link_names mapping for clearer link specification
Fixes # (issue)
These changes help fix the following problems from [VDR
feedback](https://docs.google.com/document/d/1saB1QA5r_WlD1l17q7C04WWNltnBW-K0ydI2UB8jxAs/edit?tab=t.0)
- [Enable Waist DOF](https://nvbugspro.nvidia.com/bug/5235527)
- Discourage elbow flare
- Make controller low-latency and low-jerk. **We improved the unit test
for the pink controller and reduced our position and rotation accuracy
tolerance from 30 mm, 10 degrees to 1 mm, 1 degree.**
- Develop metric for controller performance
- Added a flag to disable failure due to joint limits. Previously, any
commanded pose that ended in joint limit violation would result in no
solution and the controlled robot freezing in place. This change gets
the solver to still provide a solution and instead issue a warning for
joint limit violations.
## Screenshots
These controller changes have been tested through the Mimic pipeline
(teleop_se3_agent.py, record/replay_demos.py). Here are videos showing
teleoperation of all three environments working.
### PickPlace-GR1T2-Abs
### NutPour-GR1T2
### ExhaustPipe-GR1T2
### Successfully Trained Robomimic Model Rollout on PickPlace task
For the two robomimic tasks: `Isaac-PickPlace-GR1T2-Abs-v0` and
`Isaac-NutPour-GR1T2-Pink-IK-Abs-v0`, if we collect a new dataset, we
achieve a success rate of 96 and 92% respectively.
## Checklist
- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there
<!--
As you go through the checklist above, you can mark something as done by
putting an x character in it
For example,
- [x] I have done this task
- [ ] I have not done this task
-->
---------
Co-authored-by: Kelly Guo <kellyg@nvidia.com>
Copy file name to clipboardExpand all lines: docs/source/overview/teleop_imitation.rst
+4-1Lines changed: 4 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -333,7 +333,7 @@ Collect human demonstrations
333
333
334
334
Data collection for the GR-1 humanoid robot environment requires use of an Apple Vision Pro headset. If you do not have access to
335
335
an Apple Vision Pro, you may skip this step and continue on to the next step: `Generate the dataset`_.
336
-
A pre-recorded annotated dataset is provided in the next step.
336
+
A pre-recorded annotated dataset is provided in the next step.
337
337
338
338
.. tip::
339
339
The GR1 scene utilizes the wrist poses from the Apple Vision Pro (AVP) as setpoints for a differential IK controller (Pink-IK).
@@ -380,6 +380,9 @@ Collect five demonstrations by running the following command:
380
380
--dataset_file ./datasets/dataset_gr1.hdf5 \
381
381
--num_demos 5 --enable_pinocchio
382
382
383
+
.. note::
384
+
We also provide a GR-1 pick and place task with waist degrees-of-freedom enabled ``Isaac-PickPlace-GR1T2-WaistEnabled-Abs-v0`` (see :ref:`environments` for details on the available environments, including the GR1 Waist Enabled variant). The same command above applies but with the task name changed to ``Isaac-PickPlace-GR1T2-WaistEnabled-Abs-v0``.
385
+
383
386
.. tip::
384
387
If a demo fails during data collection, the environment can be reset using the teleoperation controls panel in the XR teleop client
385
388
on the Apple Vision Pro or via voice control by saying "reset". See :ref:`teleoperate-apple-vision-pro` for more details.
Copy file name to clipboardExpand all lines: source/isaaclab/docs/CHANGELOG.rst
+22Lines changed: 22 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,28 @@
1
1
Changelog
2
2
---------
3
3
4
+
0.45.8 (2025-07-25)
5
+
~~~~~~~~~~~~~~~~~~~
6
+
7
+
Added
8
+
^^^^^
9
+
10
+
* Created :attr:`~isaaclab.controllers.pink_ik.PinkIKControllerCfg.target_eef_link_names` to :class:`~isaaclab.controllers.pink_ik.PinkIKControllerCfg`
11
+
to specify the target end-effector link names for the pink inverse kinematics controller.
12
+
13
+
Changed
14
+
^^^^^^^
15
+
16
+
* Updated pink inverse kinematics controller configuration for the following tasks (Isaac-PickPlace-GR1T2, Isaac-NutPour-GR1T2, Isaac-ExhaustPipe-GR1T2)
17
+
to increase end-effector tracking accuracy and speed. Also added a null-space regularizer that enables turning on of waist degrees-of-freedom.
18
+
* Improved the test_pink_ik script to more comprehensive test on controller accuracy. Also, migrated to use pytest. With the current IK controller
19
+
improvements, our unit tests pass position and orientation accuracy test within **(1 mm, 1 degree)**. Previously, the position accuracy tolerances
20
+
were set to **(30 mm, 10 degrees)**.
21
+
* Included a new config parameter :attr:`fail_on_ik_error` to :class:`~isaaclab.controllers.pink_ik.PinkIKControllerCfg`
22
+
to control whether the IK controller raise an exception if robot joint limits are exceeded. In the case of an exception, the controller will hold the
23
+
last joint position. This adds to stability of the controller and avoids operator experiencing what is perceived as sudden large delays in robot control.
0 commit comments