Skip to content

Commit 73ea431

Browse files
yinsong1986yinsong1986
authored
docs(isaac): backend reference + Phase 1 status banner (5/5 of #31 split) (#48)
* docs(isaac): backend reference + Phase 1 status banner (5/5 of #31 split) Part 5 / 5 of the split of #31 — tracked by #42. Branched off PR-1 (`pr1/isaac-foundation` -> #44); rebases cleanly onto `main` once PR-1 merges. Parallel-mergeable with PR-4 (#47, simulation) -- this slice has no code dependency on simulation.py. Adds the user-facing reference doc for the Isaac backend plus the companion test file that pins R2's "Phase 1 status" disclosure banner: - `docs/backends/isaac.md` (208 LOC): Install + Quick Start + API reference + environment variables + R2's Phase 1 status banner before the Installation section. The banner discloses that the Phase 1 skeleton silently no-ops the data plane (`add_robot` procedural branch, `_load_usd_robot`, `_load_urdf_robot`, `add_object`, `add_camera`, `replicate` all return `status: "success"` without instantiating the underlying USD prim or articulation handle, with the observable downstream effect that `get_observation()` returns `{}` and `render()` returns blank frames). R2 reviewer asked for this explicitly so a user following the Quick Start sees the disclaimer before the silent path. - `strands_robots_sim/isaac/tests/test_phase1_doc_banner.py` (~75 LOC, 3 tests): Carved out of cagataycali's original `test_phase1_doc_honesty.py` -- the `TestIsaacDocsPhase1Banner` class only. The companion `TestG1DOFCount` class lives in PR-3 (#46) alongside `procedural.py`. Pin tests: - `test_isaac_docs_file_exists`: truth-source pin for the doc file's location. - `test_phase1_banner_present_before_installation`: pins that "Phase 1 status" appears AND precedes the `## Installation` heading. - `test_phase1_banner_names_the_silent_methods`: pins that the banner enumerates `add_robot`, `replicate`, `get_observation` by name (so a future maintainer who reads only the banner knows which API surfaces are affected). CI signal: lint clean (black / isort / flake8); the 3 doc-banner pin tests pass standalone (`pytest test_phase1_doc_banner.py -v` -> 3 passed). Why split `test_phase1_doc_honesty.py` into G1-DOF (PR-3) + doc- banner (this PR): keeps each parallel-mergeable slice CI-green standalone. Shipping the original combined file in either PR would couple them (PR-3 would CI-red until PR-5 lands, or vice versa). The combined coverage equals the original file. Original work by @cagataycali in #31 (`413ff15..85e180f`); this slice cherry-picks `docs/backends/isaac.md` plus the carved-out banner test file. R2's banner addition (commit `85e180f`) is already baked into this doc. * docs(isaac): align with PR #47 + PR #51 merged to main Updates docs/backends/isaac.md to reflect the actual state of strands_robots_sim.isaac after PR #44 / #45 / #46 / #47 / #51 all landed. The Phase 1 status banner had become stale (claimed all data- plane methods were silent no-op when in fact the procedural builders work, the loaders module ships, and several SimEngine methods now have concrete implementations). Three substantive changes: 1. Phase 1 status banner — split into 'Working today' (config, is_available, lifecycle, procedural builders, loaders module) vs 'Still no-op' (add_object, add_camera, replicate, the per-class _load_usd_robot / _load_urdf_robot stubs, articulation-touching paths under get_observation / send_action / render). The loaders module is the working URDF / MJCF / USD ingestion path today; the IsaacSimulation private no-op stubs remain Phase 2 work. 2. New 'Loading External Description Files (URDF / MJCF / USD)' section between Procedural Robots and Comparison with Newton. Documents the loaders module (PR #51): three load_*() functions producing ProceduralRobot dataclass instances, shared failure semantics (FileNotFoundError / ValueError, never silent phantom robot), and the parity-test pin against the seven robosuite- bundled MJCFs the strands-robots LIBERO adapter consumes. 3. Procedural Robots — added the G1 intermediate-link-bodies explanation and the fail-first kinematic-tree guard contract (PR #51's commit). No env-var escape hatch. Plus housekeeping in the Architecture file-list and Testing section: - Architecture: added _install.py (PR #47's commit — centralised install hints), loaders.py (PR #51), and the actual test files now in main (test_get_observation_diagnostic_logs.py, test_procedural_g1_dof.py, test_procedural_kinematic_guard.py, test_loaders.py) instead of the placeholder list. - Testing: pytest invocations updated to cover the five no-GPU test files now shipped, plus a one-liner for running them all at once (the --ignore pattern that the hatch script uses). The doc was already accurate about the *direction* of Phase 1 vs Phase 2; the update just brings the surface lists in line with what actually shipped. Verification: - pytest strands_robots_sim/isaac/tests/test_phase1_doc_banner.py → 3 passed (the banner-content pin still matches; the rewrite keeps 'Phase 1 status' + the same enumerated method names). - diff stat: docs/backends/isaac.md, +51 / -7 LOC. This change lands inside PR #48 (5/5 of #31 split) since it's the canonical home for docs/backends/isaac.md updates. Top-level README and examples/MIGRATION.md drift (Stage-3-no-longer-future framing) is a separate doc-alignment concern and lands in a follow-up PR rather than widening this slice.
1 parent 06fce4d commit 73ea431

2 files changed

Lines changed: 332 additions & 0 deletions

File tree

‎docs/backends/isaac.md‎

Lines changed: 252 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,252 @@
1+
# Isaac Sim Backend
2+
3+
GPU-native photorealistic simulation backend for `strands-robots-sim` using NVIDIA Isaac Sim.
4+
5+
## Overview
6+
7+
`IsaacSimulation` provides:
8+
9+
- **Photorealistic rendering** via Omniverse RTX (path-traced, ground-truth depth, semantic segmentation)
10+
- **Asset pipeline**: USD-native scenes, NVIDIA Nucleus assets
11+
- **Sensors**: RTX cameras, LiDAR, depth, contact, IMU (GPU-batched)
12+
- **Synthetic data generation**: Replicator pipeline for domain randomization
13+
- **Fleet replication**: parallel environments via `omni.isaac.cloner.Cloner`
14+
- **Isaac Lab integration**: GPU-accelerated RL environments
15+
16+
17+
> **Phase 1 status (skeleton).** This release ships the SimEngine-shaped surface, lazy-import scaffolding, the procedural-robot dataclass + builders (SO-100 / Panda / G1), and the URDF / MJCF / USD loader module. **Working today**: `IsaacConfig`, `IsaacSimulation.is_available()`, world / lifecycle (`create_world` / `destroy` / `cleanup`), procedural builders via `add_robot("so100" | "panda" | "unitree_g1")`, and the `isaac.loaders.load_urdf` / `load_mjcf` / `load_usd` functions for ingesting external robot description files into a `ProceduralRobot` dataclass. **Still no-op in this phase**: the data-plane wiring on `IsaacSimulation` itself — `add_object`, `add_camera`, `replicate`, the per-`IsaacSimulation` `_load_usd_robot` / `_load_urdf_robot` private methods, and articulation-touching paths under `get_observation` / `send_action` / `render` — currently return `status: "success"` without instantiating the underlying USD prim or articulation handle. Following the documented Quick Start on a real Isaac Sim install will therefore observe `get_observation()` returning `{}` for those paths and `render()` returning blank frames — no exception is raised. The data-plane wiring (USD stage management, articulation construction, sensor / replicator integration) lands in Phase 2 and later. Treat the Phase-1 surface as an integration contract for the still-no-op methods; the loaders module is the working path for URDF / MJCF / USD ingestion today.
18+
19+
## Installation
20+
21+
Isaac Sim is **not installable from PyPI**. It is an NVIDIA Omniverse Kit application that must be installed separately.
22+
23+
### Option 1: NVIDIA Omniverse Launcher (recommended)
24+
25+
1. Download [NVIDIA Omniverse Launcher](https://developer.nvidia.com/omniverse)
26+
2. Install **Isaac Sim 2024.x** (or newer) from the Exchange tab
27+
3. Install Python dependencies:
28+
29+
```bash
30+
pip install 'strands-robots-sim[isaac]'
31+
```
32+
33+
### Option 2: Isaac Lab
34+
35+
```bash
36+
git clone https://github.com/isaac-sim/IsaacLab.git
37+
cd IsaacLab
38+
./isaaclab.sh -i
39+
pip install 'strands-robots-sim[isaac]'
40+
```
41+
42+
### Option 3: Docker
43+
44+
```bash
45+
docker pull nvcr.io/nvidia/isaac-sim:4.5.0
46+
docker run --gpus all -it nvcr.io/nvidia/isaac-sim:4.5.0
47+
# Inside container:
48+
pip install 'strands-robots-sim[isaac]'
49+
```
50+
51+
## Requirements
52+
53+
- NVIDIA GPU (RTX 2070+ or A100/H100 for fleet training)
54+
- CUDA 12.0+
55+
- Isaac Sim 2024.x or newer
56+
- Linux (Ubuntu 22.04+ recommended)
57+
- Python 3.10+
58+
59+
## Quick Start
60+
61+
```python
62+
from strands_robots_sim.isaac import IsaacSimulation, IsaacConfig
63+
64+
# Check availability
65+
available, reason = IsaacSimulation.is_available()
66+
if not available:
67+
print(f"Isaac Sim not available: {reason}")
68+
exit(1)
69+
70+
# Create simulation
71+
config = IsaacConfig(
72+
num_envs=1,
73+
headless=True,
74+
render_mode="rtx_realtime",
75+
)
76+
sim = IsaacSimulation(config)
77+
78+
# Create world and add robot
79+
sim.create_world()
80+
sim.add_robot("so100")
81+
sim.add_camera("front_cam", position=[1.0, 0.0, 0.5])
82+
83+
# Step and render
84+
sim.step(100)
85+
result = sim.render("front_cam")
86+
rgb = result["rgb"] # (H, W, 3) uint8
87+
88+
# Clean up
89+
sim.destroy()
90+
```
91+
92+
## Fleet Training (Multi-Env)
93+
94+
```python
95+
config = IsaacConfig(num_envs=1024, headless=True)
96+
sim = IsaacSimulation(config)
97+
sim.create_world()
98+
sim.add_robot("so100")
99+
sim.replicate(1024) # 1024 parallel environments
100+
101+
for step in range(10000):
102+
sim.step(1)
103+
obs = sim.get_observation("so100") # batched across all envs
104+
# ... RL training loop ...
105+
106+
sim.destroy()
107+
```
108+
109+
## Entry-Point Discovery
110+
111+
Isaac Sim registers as a `strands_robots.backends` entry point:
112+
113+
```python
114+
from importlib.metadata import entry_points
115+
116+
for ep in entry_points(group='strands_robots.backends'):
117+
print(ep.name, '->', ep.value)
118+
# isaac -> strands_robots_sim.isaac.simulation:IsaacSimulation
119+
# isaac_sim -> strands_robots_sim.isaac.simulation:IsaacSimulation
120+
```
121+
122+
## Configuration
123+
124+
### `IsaacConfig` Parameters
125+
126+
| Parameter | Default | Description |
127+
|-----------|---------|-------------|
128+
| `num_envs` | 1 | Number of parallel environments |
129+
| `device` | "cuda:0" | CUDA device |
130+
| `headless` | True | Run without GUI |
131+
| `physics_dt` | 1/120 s | Physics timestep |
132+
| `rendering_dt` | 1/30 s | Rendering timestep |
133+
| `render_mode` | "headless" | "headless", "rtx_realtime", or "rtx_pathtracing" |
134+
| `gravity` | (0, 0, -9.81) | Gravity vector (Z-up) |
135+
| `camera_width` | 640 | Default camera width |
136+
| `camera_height` | 480 | Default camera height |
137+
| `enable_rtx_sensors` | True | Enable RTX-accelerated sensors |
138+
139+
### Environment Variables
140+
141+
| Variable | Default | Description |
142+
|----------|---------|-------------|
143+
| `STRANDS_ISAAC_HEADLESS` | - | Override headless mode ("true"/"false") |
144+
| `STRANDS_ISAAC_RTX_PATHTRACING` | - | Enable RTX pathtracing ("true"/"false") |
145+
| `STRANDS_ISAAC_NUCLEUS_URL` | - | Override Omniverse Nucleus server URL |
146+
147+
## Procedural Robots
148+
149+
The following robots can be added without any asset files:
150+
151+
- `so100` (aliases: `so-100`, `so_100`, `so101`) -- 6-DOF tabletop arm
152+
- `panda` (aliases: `franka`, `franka_panda`) -- 7-DOF manipulator
153+
- `unitree_g1` (aliases: `g1`) -- 21-DOF humanoid (simplified). The six 2-DOF compound joints (hips / ankles / shoulder-yaw + elbow on each arm) are split through massless intermediate `*_link` bodies so the kinematic graph is a valid tree by construction.
154+
155+
```python
156+
sim.add_robot("so100") # procedural, no asset files needed
157+
sim.add_robot("panda")
158+
sim.add_robot("g1", data_config="unitree_g1")
159+
```
160+
161+
Every procedural builder validates the kinematic graph at construction time via `_validate_kinematic_tree`: a robot whose joint set has a duplicate `(parent_body, child_body)` edge fails fast with `ValueError` listing the offending bodies + joint names. Validation is **fail-first by default** with no env-var escape hatch — shipping a knowingly-broken robot has no good use case in this package.
162+
163+
## Loading External Description Files (URDF / MJCF / USD)
164+
165+
The `strands_robots_sim.isaac.loaders` module produces `ProceduralRobot` dataclass instances from existing robot description files, so callers don't have to add a new `_build_*` function for every robot they need. Three formats are supported:
166+
167+
```python
168+
from strands_robots_sim.isaac.loaders import load_urdf, load_mjcf, load_usd
169+
170+
# URDF -- stdlib XML; no third-party deps required.
171+
panda_urdf = load_urdf("/path/to/panda.urdf")
172+
173+
# MJCF (MuJoCo XML) -- stdlib XML; LIBERO scenes, robosuite assets.
174+
panda_mjcf = load_mjcf("/opt/conda/.../robosuite/models/assets/robots/panda/robot.xml")
175+
176+
# USD -- requires `pxr` (ships in the [isaac] extra).
177+
panda_usd = load_usd("/path/to/panda.usda")
178+
179+
# All three return the same dataclass shape.
180+
print(panda_urdf.num_joints, panda_urdf.joint_names)
181+
```
182+
183+
The loaders share failure semantics: missing path raises `FileNotFoundError`, malformed document raises `ValueError` with the offending element + path, and an empty document (zero links / joints / bodies) also raises `ValueError`. Loaders never silently return a phantom robot.
184+
185+
The hardcoded `_build_*` functions in `procedural.py` remain as a zero-dep, testable fallback used when no description file is configured. Loaders layer on top.
186+
187+
The loader module is verified against the seven robosuite-bundled MJCFs that the `strands-robots` LIBERO adapter consumes (`panda` / `iiwa` / `kinova3` / `jaco` / `sawyer` / `ur5e` / `baxter`); the parity tests live in `strands_robots_sim/isaac/tests/test_loaders.py::TestRobosuiteMjcfParity`.
188+
189+
## Comparison with Newton Backend
190+
191+
| Feature | Newton (Warp) | Isaac Sim |
192+
|---------|:---:|:---:|
193+
| Physics parallelism | 4096+ envs | 1024 envs |
194+
| Rendering | OpenGL/null | RTX photorealistic |
195+
| USD native | Partial | Full |
196+
| Sensors (camera, LiDAR) | Basic | RTX GPU-batched |
197+
| Synthetic data gen | No | Replicator |
198+
| Soft body/cloth | Yes (VBD) | Yes (PhysX) |
199+
| Differentiable sim | Yes (Warp tape) | No |
200+
| Install size | ~500MB | ~30GB |
201+
| Use case | Fast RL training | Photorealistic sim2real |
202+
203+
## Architecture
204+
205+
```
206+
strands_robots_sim/isaac/
207+
__init__.py PEP 562 lazy exports (zero omni overhead on import)
208+
_install.py Single source of truth for Isaac Sim install metadata
209+
(docker image tag, Omniverse Launcher hint, Isaac Lab
210+
bootstrap) — composes ImportError messages and the
211+
is_available() reason string from these constants
212+
config.py IsaacConfig dataclass + validation
213+
simulation.py IsaacSimulation(SimEngine) -- main backend class
214+
procedural.py SO-100 / Panda / G1 builders + kinematic-tree guard
215+
loaders.py URDF / MJCF / USD -> ProceduralRobot loaders
216+
stages.py USD stage management (Phase 2)
217+
sensors.py RTX camera, LiDAR wrappers (Phase 3)
218+
replicator.py Domain randomization (Phase 3)
219+
tests/
220+
test_unit.py Mocked tests (no GPU)
221+
test_entrypoint.py Entry-point + lazy-import surface
222+
test_get_observation_diagnostic_logs.py WARNING/DEBUG level pins
223+
test_procedural_g1_dof.py G1 DOF-count drift pin
224+
test_procedural_kinematic_guard.py Fail-first kinematic-tree pin
225+
test_loaders.py URDF / MJCF / USD round-trip +
226+
robosuite real-asset parity tests
227+
test_gpu_integ.py GPU tests (STRANDS_GPU_TEST=1)
228+
```
229+
230+
## Thread Safety
231+
232+
- All mutable state protected by `threading.RLock`
233+
- `step()` must not run concurrently with `add_robot()`
234+
- `SimulationApp` is a process-wide singleton (never create more than one)
235+
- `destroy()` clears the World but does NOT shut down `SimulationApp`
236+
237+
## Testing
238+
239+
```bash
240+
# Unit tests (no GPU required)
241+
pytest strands_robots_sim/isaac/tests/test_unit.py -v
242+
pytest strands_robots_sim/isaac/tests/test_entrypoint.py -v
243+
pytest strands_robots_sim/isaac/tests/test_loaders.py -v
244+
pytest strands_robots_sim/isaac/tests/test_procedural_g1_dof.py -v
245+
pytest strands_robots_sim/isaac/tests/test_procedural_kinematic_guard.py -v
246+
247+
# Or all of the above in one go (skips the GPU file by default):
248+
pytest strands_robots_sim/isaac/tests/ --ignore=strands_robots_sim/isaac/tests/test_gpu_integ.py
249+
250+
# GPU integration tests (requires Isaac Sim)
251+
STRANDS_GPU_TEST=1 pytest strands_robots_sim/isaac/tests/test_gpu_integ.py -v
252+
```

‎strands_robots_sim/isaac/tests/test_phase1_doc_banner.py‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
"""Documentation honesty pin: Phase 1 status banner in docs/backends/isaac.md.
2+
3+
R2 review on #31 (``simulation.py:627`` thread) asked for a doc note in
4+
``docs/backends/isaac.md``'s Quick Start that the Phase 1 skeleton
5+
silently no-ops the data plane (``add_robot`` on the procedural branch,
6+
``_load_usd_robot``, ``_load_urdf_robot``, ``add_object``, ``add_camera``,
7+
``replicate`` all return ``status: "success"`` without instantiating the
8+
underlying USD prim or articulation handle). Reviewer's "at minimum"
9+
ask: a banner before the Installation section, so the disclosure precedes
10+
the documented call sequence.
11+
12+
Companion pin for the G1 DOF count lives in ``test_procedural_g1_dof.py``
13+
(landed in PR-3 alongside ``procedural.py`` itself). This file pins only
14+
the docs-side concerns.
15+
"""
16+
17+
from __future__ import annotations
18+
19+
from pathlib import Path
20+
21+
_ISAAC_DOCS = Path(__file__).resolve().parent.parent.parent.parent / "docs" / "backends" / "isaac.md"
22+
23+
24+
class TestIsaacDocsPhase1Banner:
25+
"""Pin: ``docs/backends/isaac.md`` must disclose Phase 1 data-plane no-ops."""
26+
27+
def test_isaac_docs_file_exists(self) -> None:
28+
assert _ISAAC_DOCS.is_file(), f"missing Isaac doc page at {_ISAAC_DOCS}"
29+
30+
def test_phase1_banner_present_before_installation(self) -> None:
31+
"""Banner must appear before the Installation / Quick Start sections.
32+
33+
Reviewer (R2 on PR #31, ``simulation.py:627`` thread): "At minimum,
34+
please add a note to ``docs/backends/isaac.md`` Quick Start that the
35+
Phase-1 skeleton silently no-ops the data plane."
36+
"""
37+
text = _ISAAC_DOCS.read_text(encoding="utf-8")
38+
39+
# The banner must appear AND must appear before the Installation
40+
# section header (so the disclosure precedes any procedural docs the
41+
# user would otherwise execute).
42+
banner_marker = "Phase 1 status"
43+
install_marker = "## Installation"
44+
45+
assert banner_marker in text, (
46+
"docs/backends/isaac.md missing the Phase 1 status disclosure "
47+
"banner. R2 reviewer asked for it explicitly because the doc's "
48+
"Quick Start otherwise executes a code path that silently no-ops "
49+
"on a real Isaac Sim install."
50+
)
51+
assert (
52+
install_marker in text
53+
), "docs/backends/isaac.md missing the Installation section -- doc structure has changed; pin needs review."
54+
assert text.find(banner_marker) < text.find(install_marker), (
55+
"Phase 1 banner must precede the Installation section so the "
56+
"user sees the disclosure before following the install / quick-"
57+
"start steps."
58+
)
59+
60+
def test_phase1_banner_names_the_silent_methods(self) -> None:
61+
"""Banner must enumerate the Phase-1 silent-success methods.
62+
63+
Without naming the methods, a future maintainer who reads only the
64+
banner won't know which API surfaces are affected, and the disclosure
65+
becomes a vague hedge.
66+
"""
67+
text = _ISAAC_DOCS.read_text(encoding="utf-8")
68+
# Slice the banner block (`> **Phase 1 status...**` paragraph).
69+
banner_start = text.find("Phase 1 status")
70+
# Banner is one paragraph; cut at the next `## ` heading.
71+
banner_end = text.find("##", banner_start)
72+
assert banner_end > banner_start, "could not locate end of banner block"
73+
banner = text[banner_start:banner_end]
74+
75+
for needed in ("add_robot", "replicate", "get_observation"):
76+
assert needed in banner, (
77+
f"Phase 1 banner does not mention `{needed}`; the disclosure "
78+
f"must enumerate the silent-success methods so users know "
79+
f"which call sites are affected."
80+
)

0 commit comments

Comments
 (0)