Skip to content

Commit a92da7b

Browse files
committed
fix: Upgrade teleop dependency during install
Read per-extension pip upgrade opt-ins from extension.toml so install.py can refresh selected install_requires dependencies without duplicating version specs. Use uv --upgrade-package for targeted upgrades and keep pip behavior narrow, which lets CI pick up newer compatible isaacteleop releases without broad dependency churn.
1 parent ac0a8a9 commit a92da7b

7 files changed

Lines changed: 312 additions & 7 deletions

File tree

docs/source/overview/developer-guide/development.rst

Lines changed: 16 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -81,18 +81,21 @@ Custom Extension Dependency Management
8181
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
8282

8383
Certain extensions may have dependencies which require the installation of additional packages before the extension
84-
can be used. While Python dependencies are handled by the `setuptools <https://setuptools.pypa.io/en/latest/>`__
85-
package and specified in the ``setup.py`` file, non-Python dependencies such as `ROS <https://www.ros.org/>`__
86-
packages or `apt <https://en.wikipedia.org/wiki/APT_(software)>`__ packages are not handled by setuptools.
87-
Handling these kinds of dependencies requires an additional procedure.
84+
can be used. Python dependencies are handled by the `setuptools <https://setuptools.pypa.io/en/latest/>`__
85+
package and specified in the ``setup.py`` file. Non-Python dependencies such as
86+
`ROS <https://www.ros.org/>`__ packages or `apt <https://en.wikipedia.org/wiki/APT_(software)>`__
87+
packages are not handled by setuptools. Handling these kinds of dependencies requires an additional procedure.
8888

89-
There are two types of dependencies that can be specified in the ``extension.toml`` file
89+
There are three types of dependencies that can be specified in the ``extension.toml`` file
9090
under the ``isaac_lab_settings`` section:
9191

9292
1. **apt_deps**: A list of apt packages that need to be installed. These are installed using the
9393
`apt <https://ubuntu.com/server/docs/package-management>`__ package manager.
9494
2. **ros_ws**: The path to the ROS workspace that contains the ROS packages. These are installed using
9595
the `rosdep <https://docs.ros.org/en/humble/Tutorials/Intermediate/Rosdep.html>`__ dependency manager.
96+
3. **pip_upgrade_dependencies**: A list of ``install_requires`` dependency names that should be explicitly
97+
upgraded after installing the extension with ``./isaaclab.sh --install``. List package names only. Version
98+
ranges, extras, and platform markers are read from the installed extension metadata generated from ``setup.py``.
9699

97100
As an example, the following ``extension.toml`` file specifies the dependencies for the extension:
98101

@@ -106,8 +109,11 @@ As an example, the following ``extension.toml`` file specifies the dependencies
106109
# note: if this path is relative, it is relative to the extension directory's root
107110
ros_ws = "/home/user/catkin_ws"
108111
109-
These dependencies are installed using the ``install_deps.py`` script provided in the ``tools`` directory.
110-
To install all dependencies for all extensions, run the following command:
112+
# Python dependency names to upgrade after installing this extension
113+
pip_upgrade_dependencies = ["example_package"]
114+
115+
The ``apt_deps`` and ``ros_ws`` dependencies are installed using the ``install_deps.py`` script provided in the
116+
``tools`` directory. To install all apt and ROS dependencies for all extensions, run the following command:
111117

112118
.. code-block:: bash
113119
@@ -121,6 +127,9 @@ To install all dependencies for all extensions, run the following command:
121127
and ``Dockerfile.ros2``. This ensures that all the 'apt' and 'rosdep' dependencies are installed
122128
before building the extensions respectively.
123129

130+
The ``pip_upgrade_dependencies`` entries are handled by ``./isaaclab.sh --install`` after the extension's editable
131+
pip install completes.
132+
124133

125134
Standalone applications
126135
~~~~~~~~~~~~~~~~~~~~~~~
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
Fixed
2+
^^^^^
3+
4+
* Fixed extension installation to honor ``pip_upgrade_dependencies`` declared
5+
in ``config/extension.toml``.

source/isaaclab/isaaclab/cli/commands/install.py

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,13 @@
44
# SPDX-License-Identifier: BSD-3-Clause
55

66
import os
7+
import re
78
import shutil
89
import sys
910
from pathlib import Path
1011

12+
import tomllib
13+
1114
from ..utils import (
1215
ISAACLAB_ROOT,
1316
extract_isaacsim_path,
@@ -286,6 +289,111 @@ def _ensure_cuda_torch() -> None:
286289
NVIDIA_INDEX_URL = "https://pypi.nvidia.com"
287290

288291

292+
def _normalize_package_name(name: str) -> str:
293+
"""Normalize a Python package name for metadata comparisons."""
294+
return re.sub(r"[-_.]+", "-", name).lower()
295+
296+
297+
def _requirement_name(requirement: str) -> str:
298+
"""Extract the distribution name from a requirement string."""
299+
requirement = requirement.split(";", 1)[0].strip()
300+
return re.split(r"\s|<|>|=|!|~|\[|@", requirement, maxsplit=1)[0]
301+
302+
303+
def _get_installed_distribution_requirements(python_exe: str, distribution_name: str) -> list[str]:
304+
"""Return installed ``Requires-Dist`` requirements for a distribution."""
305+
probe = """import importlib.metadata
306+
import sys
307+
308+
try:
309+
dist = importlib.metadata.distribution(sys.argv[1])
310+
except importlib.metadata.PackageNotFoundError:
311+
sys.exit(1)
312+
313+
for requirement in dist.requires or []:
314+
print(requirement)
315+
"""
316+
result = run_command(
317+
[python_exe, "-c", probe, distribution_name],
318+
capture_output=True,
319+
text=True,
320+
check=False,
321+
)
322+
if result.returncode != 0:
323+
print_warning(f"Could not read installed metadata for {distribution_name}; skipping dependency upgrades.")
324+
return []
325+
return [line.strip() for line in result.stdout.splitlines() if line.strip()]
326+
327+
328+
def _get_extension_pip_upgrade_dependencies(extension_dir: Path) -> list[str]:
329+
"""Read dependency names opted into targeted pip upgrades from ``extension.toml``."""
330+
extension_toml = extension_dir / "config" / "extension.toml"
331+
if not extension_toml.is_file():
332+
return []
333+
334+
try:
335+
with extension_toml.open("rb") as fd:
336+
extension_data = tomllib.load(fd)
337+
except tomllib.TOMLDecodeError as exc:
338+
print_warning(f"Could not parse {extension_toml}: {exc}; skipping targeted dependency upgrades.")
339+
return []
340+
341+
isaac_lab_settings = extension_data.get("isaac_lab_settings", {})
342+
if not isinstance(isaac_lab_settings, dict):
343+
print_warning(
344+
f"Ignoring invalid isaac_lab_settings in {extension_toml}; expected a table with pip_upgrade_dependencies."
345+
)
346+
return []
347+
348+
upgrade_dependencies = isaac_lab_settings.get("pip_upgrade_dependencies", [])
349+
if not isinstance(upgrade_dependencies, list) or not all(isinstance(item, str) for item in upgrade_dependencies):
350+
print_warning(f"Ignoring invalid pip_upgrade_dependencies in {extension_toml}; expected a list of strings.")
351+
return []
352+
353+
return upgrade_dependencies
354+
355+
356+
def _get_pip_upgrade_command(pip_cmd: list[str], dependency_name: str, requirement: str) -> list[str]:
357+
"""Return a pip command that upgrades one dependency requirement."""
358+
if pip_cmd[0] == "uv":
359+
return pip_cmd + ["install", "--upgrade-package", dependency_name, requirement]
360+
return pip_cmd + ["install", "--upgrade", requirement]
361+
362+
363+
def _upgrade_extension_pip_dependencies(
364+
python_exe: str,
365+
pip_cmd: list[str],
366+
distribution_name: str,
367+
dependency_names: list[str],
368+
) -> None:
369+
"""Upgrade selected dependencies using installed distribution metadata requirements."""
370+
if not dependency_names:
371+
return
372+
373+
requirements = _get_installed_distribution_requirements(python_exe, distribution_name)
374+
seen_dependency_names = set()
375+
376+
for dependency_name in dependency_names:
377+
normalized_dependency_name = _normalize_package_name(dependency_name)
378+
if normalized_dependency_name in seen_dependency_names:
379+
continue
380+
seen_dependency_names.add(normalized_dependency_name)
381+
382+
matching_requirements = [
383+
req for req in requirements if _normalize_package_name(_requirement_name(req)) == normalized_dependency_name
384+
]
385+
if not matching_requirements:
386+
print_warning(
387+
f"Could not find dependency '{dependency_name}' in installed metadata for {distribution_name}; "
388+
"skipping targeted upgrade."
389+
)
390+
continue
391+
392+
for requirement in matching_requirements:
393+
print_info(f"Upgrading {dependency_name} for {distribution_name}: {requirement}")
394+
run_command(_get_pip_upgrade_command(pip_cmd, dependency_name, requirement))
395+
396+
289397
def _install_isaacsim() -> None:
290398
"""Install Isaac Sim pip package if not already present."""
291399
python_exe = extract_python_exe()
@@ -414,6 +522,12 @@ def _install_isaaclab_submodules(
414522
editable = (submodule_extras or {}).get(item.name, "")
415523
install_target = f"{item}{editable}"
416524
run_command(pip_cmd + ["install", "--editable", install_target])
525+
_upgrade_extension_pip_dependencies(
526+
python_exe,
527+
pip_cmd,
528+
item.name,
529+
_get_extension_pip_upgrade_dependencies(item),
530+
)
417531

418532

419533
def _install_extra_frameworks(framework_name: str = "all") -> None:

source/isaaclab/test/cli/test_install_commands.py

Lines changed: 162 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717

1818
import pytest
1919

20+
import isaaclab.cli.commands.install as install_cmd
2021
from isaaclab.cli.commands.install import (
2122
_PREBUNDLE_REPOINT_PACKAGES,
2223
_ensure_cuda_torch,
@@ -68,6 +69,167 @@ def _make_site_packages(
6869
return site_pkgs
6970

7071

72+
# ---------------------------------------------------------------------------
73+
# _install_isaaclab_submodules targeted dependency upgrades
74+
# ---------------------------------------------------------------------------
75+
76+
77+
class TestInstallSubmodulesTargetedDependencyUpgrades:
78+
"""Tests for extension.toml-driven dependency upgrades."""
79+
80+
def _make_extension(self, tmp_path, extension_toml: str) -> Path:
81+
"""Create a minimal installable extension fixture."""
82+
source_dir = tmp_path / "source"
83+
extension_dir = source_dir / "isaaclab_teleop"
84+
config_dir = extension_dir / "config"
85+
config_dir.mkdir(parents=True)
86+
(extension_dir / "setup.py").write_text("# test fixture\n", encoding="utf-8")
87+
(config_dir / "extension.toml").write_text(extension_toml, encoding="utf-8")
88+
return extension_dir
89+
90+
def test_installs_editable_then_upgrades_declared_dependency_from_metadata(self, tmp_path):
91+
"""An opted-in dependency is upgraded using the requirement recorded in installed metadata."""
92+
extension_dir = self._make_extension(
93+
tmp_path,
94+
'[isaac_lab_settings]\npip_upgrade_dependencies = ["isaacteleop"]\n',
95+
)
96+
97+
python_exe = str(tmp_path / "python")
98+
pip_cmd = [python_exe, "-m", "pip"]
99+
isaacteleop_req = 'isaacteleop[cloudxr,retargeters,ui] ~=1.2.0; platform_system == "Linux"'
100+
101+
with (
102+
mock.patch("isaaclab.cli.commands.install.ISAACLAB_ROOT", tmp_path),
103+
mock.patch("isaaclab.cli.commands.install.extract_python_exe", return_value=python_exe),
104+
mock.patch("isaaclab.cli.commands.install.get_pip_command", return_value=pip_cmd),
105+
mock.patch(
106+
"isaaclab.cli.commands.install._get_installed_distribution_requirements",
107+
return_value=[isaacteleop_req],
108+
),
109+
mock.patch("isaaclab.cli.commands.install.run_command") as mock_run,
110+
):
111+
install_cmd._install_isaaclab_submodules(["isaaclab_teleop"])
112+
113+
assert [call.args[0] for call in mock_run.call_args_list] == [
114+
pip_cmd + ["install", "--editable", str(extension_dir)],
115+
pip_cmd + ["install", "--upgrade", isaacteleop_req],
116+
]
117+
118+
def test_uv_install_uses_upgrade_package_for_declared_dependency(self, tmp_path):
119+
"""uv upgrades only the declared package rather than using a global upgrade."""
120+
extension_dir = self._make_extension(
121+
tmp_path,
122+
'[isaac_lab_settings]\npip_upgrade_dependencies = ["isaacteleop"]\n',
123+
)
124+
125+
python_exe = str(tmp_path / "python")
126+
pip_cmd = ["uv", "pip"]
127+
isaacteleop_req = 'isaacteleop[cloudxr,retargeters,ui] ~=1.2.0; platform_system == "Linux"'
128+
129+
with (
130+
mock.patch("isaaclab.cli.commands.install.ISAACLAB_ROOT", tmp_path),
131+
mock.patch("isaaclab.cli.commands.install.extract_python_exe", return_value=python_exe),
132+
mock.patch("isaaclab.cli.commands.install.get_pip_command", return_value=pip_cmd),
133+
mock.patch(
134+
"isaaclab.cli.commands.install._get_installed_distribution_requirements",
135+
return_value=[isaacteleop_req],
136+
),
137+
mock.patch("isaaclab.cli.commands.install.run_command") as mock_run,
138+
):
139+
install_cmd._install_isaaclab_submodules(["isaaclab_teleop"])
140+
141+
assert [call.args[0] for call in mock_run.call_args_list] == [
142+
pip_cmd + ["install", "--editable", str(extension_dir)],
143+
pip_cmd + ["install", "--upgrade-package", "isaacteleop", isaacteleop_req],
144+
]
145+
146+
def test_upgrades_all_matching_metadata_requirements(self, tmp_path):
147+
"""Duplicate metadata entries are preserved instead of collapsing to one requirement."""
148+
python_exe = str(tmp_path / "python")
149+
pip_cmd = [python_exe, "-m", "pip"]
150+
linux_req = 'example-package>=1.0; platform_system == "Linux"'
151+
windows_req = 'example_package>=2.0; platform_system == "Windows"'
152+
153+
with (
154+
mock.patch(
155+
"isaaclab.cli.commands.install._get_installed_distribution_requirements",
156+
return_value=[linux_req, windows_req],
157+
),
158+
mock.patch("isaaclab.cli.commands.install.run_command") as mock_run,
159+
):
160+
install_cmd._upgrade_extension_pip_dependencies(
161+
python_exe,
162+
pip_cmd,
163+
"isaaclab_teleop",
164+
["example-package"],
165+
)
166+
167+
assert [call.args[0] for call in mock_run.call_args_list] == [
168+
pip_cmd + ["install", "--upgrade", linux_req],
169+
pip_cmd + ["install", "--upgrade", windows_req],
170+
]
171+
172+
def test_skips_duplicate_declared_dependency_names(self, tmp_path):
173+
"""Duplicate TOML dependency names do not trigger duplicate pip commands."""
174+
python_exe = str(tmp_path / "python")
175+
pip_cmd = [python_exe, "-m", "pip"]
176+
req = "isaacteleop~=1.2.0"
177+
178+
with (
179+
mock.patch(
180+
"isaaclab.cli.commands.install._get_installed_distribution_requirements",
181+
return_value=[req],
182+
),
183+
mock.patch("isaaclab.cli.commands.install.run_command") as mock_run,
184+
):
185+
install_cmd._upgrade_extension_pip_dependencies(
186+
python_exe,
187+
pip_cmd,
188+
"isaaclab_teleop",
189+
["isaacteleop", "IsaacTeleop"],
190+
)
191+
192+
mock_run.assert_called_once_with(pip_cmd + ["install", "--upgrade", req])
193+
194+
def test_skips_when_toml_has_no_upgrade_dependencies(self, tmp_path):
195+
"""Extensions without pip upgrade opt-ins do not trigger metadata probes."""
196+
extension_dir = self._make_extension(tmp_path, "[isaac_lab_settings]\n")
197+
198+
assert install_cmd._get_extension_pip_upgrade_dependencies(extension_dir) == []
199+
200+
def test_warns_and_skips_invalid_upgrade_dependency_names(self, tmp_path):
201+
"""Invalid TOML value types warn and disable targeted upgrades."""
202+
extension_dir = self._make_extension(
203+
tmp_path,
204+
'[isaac_lab_settings]\npip_upgrade_dependencies = "isaacteleop"\n',
205+
)
206+
207+
with mock.patch("isaaclab.cli.commands.install.print_warning") as mock_warning:
208+
assert install_cmd._get_extension_pip_upgrade_dependencies(extension_dir) == []
209+
210+
mock_warning.assert_called_once()
211+
212+
def test_warns_when_declared_dependency_missing_from_metadata(self, tmp_path):
213+
"""A declared dependency name must exist in installed package metadata."""
214+
with (
215+
mock.patch(
216+
"isaaclab.cli.commands.install._get_installed_distribution_requirements",
217+
return_value=["dex-retargeting==0.5.0"],
218+
),
219+
mock.patch("isaaclab.cli.commands.install.print_warning") as mock_warning,
220+
mock.patch("isaaclab.cli.commands.install.run_command") as mock_run,
221+
):
222+
install_cmd._upgrade_extension_pip_dependencies(
223+
str(tmp_path / "python"),
224+
[str(tmp_path / "python"), "-m", "pip"],
225+
"isaaclab_teleop",
226+
["isaacteleop"],
227+
)
228+
229+
mock_warning.assert_called_once()
230+
mock_run.assert_not_called()
231+
232+
71233
# ---------------------------------------------------------------------------
72234
# _torch_first_on_sys_path_is_prebundle
73235
# ---------------------------------------------------------------------------

source/isaaclab_teleop/changelog.d/hougantc-pipelined-retargeting.rst

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,3 +15,9 @@ Changed
1515
Isaac Lab's step Python. Set
1616
``retargeting_execution=RetargetingExecutionConfig(mode="sync")`` to restore
1717
exact current-frame retargeting.
18+
19+
Fixed
20+
^^^^^
21+
22+
* Fixed installation to upgrade to the latest compatible ``isaacteleop``
23+
package when installing ``isaaclab_teleop``.

source/isaaclab_teleop/config/extension.toml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,10 @@ keywords = ["kit", "robotics", "teleoperation", "xr", "isaaclab"]
1313
[dependencies]
1414
"isaaclab" = {}
1515

16+
[isaac_lab_settings]
17+
# Names only. Version ranges, extras, and platform markers come from setup.py metadata.
18+
pip_upgrade_dependencies = ["isaacteleop"]
19+
1620
[core]
1721
reloadable = false
1822

0 commit comments

Comments
 (0)