Skip to content

Commit 9c18e83

Browse files
yinsong1986yinsong1986
committed
docs(isaac): backend reference + Phase 1 status banner (5/5 of #31 split)
Part 5 / 5 of the split of #31 — tracked by #42. Branched off PR-1 (`pr1/isaac-foundation` -> #44); rebases cleanly onto `main` once PR-1 merges. Parallel-mergeable with PR-4 (#47, simulation) -- this slice has no code dependency on simulation.py. Adds the user-facing reference doc for the Isaac backend plus the companion test file that pins R2's "Phase 1 status" disclosure banner: - `docs/backends/isaac.md` (208 LOC): Install + Quick Start + API reference + environment variables + R2's Phase 1 status banner before the Installation section. The banner discloses that the Phase 1 skeleton silently no-ops the data plane (`add_robot` procedural branch, `_load_usd_robot`, `_load_urdf_robot`, `add_object`, `add_camera`, `replicate` all return `status: "success"` without instantiating the underlying USD prim or articulation handle, with the observable downstream effect that `get_observation()` returns `{}` and `render()` returns blank frames). R2 reviewer asked for this explicitly so a user following the Quick Start sees the disclaimer before the silent path. - `strands_robots_sim/isaac/tests/test_phase1_doc_banner.py` (~75 LOC, 3 tests): Carved out of cagataycali's original `test_phase1_doc_honesty.py` -- the `TestIsaacDocsPhase1Banner` class only. The companion `TestG1DOFCount` class lives in PR-3 (#46) alongside `procedural.py`. Pin tests: - `test_isaac_docs_file_exists`: truth-source pin for the doc file's location. - `test_phase1_banner_present_before_installation`: pins that "Phase 1 status" appears AND precedes the `## Installation` heading. - `test_phase1_banner_names_the_silent_methods`: pins that the banner enumerates `add_robot`, `replicate`, `get_observation` by name (so a future maintainer who reads only the banner knows which API surfaces are affected). CI signal: lint clean (black / isort / flake8); the 3 doc-banner pin tests pass standalone (`pytest test_phase1_doc_banner.py -v` -> 3 passed). Why split `test_phase1_doc_honesty.py` into G1-DOF (PR-3) + doc- banner (this PR): keeps each parallel-mergeable slice CI-green standalone. Shipping the original combined file in either PR would couple them (PR-3 would CI-red until PR-5 lands, or vice versa). The combined coverage equals the original file. Original work by @cagataycali in #31 (`413ff15..85e180f`); this slice cherry-picks `docs/backends/isaac.md` plus the carved-out banner test file. R2's banner addition (commit `85e180f`) is already baked into this doc.
1 parent 3a0a854 commit 9c18e83

2 files changed

Lines changed: 288 additions & 0 deletions

File tree

‎docs/backends/isaac.md‎

Lines changed: 208 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
# Isaac Sim Backend
2+
3+
GPU-native photorealistic simulation backend for `strands-robots-sim` using NVIDIA Isaac Sim.
4+
5+
## Overview
6+
7+
`IsaacSimulation` provides:
8+
9+
- **Photorealistic rendering** via Omniverse RTX (path-traced, ground-truth depth, semantic segmentation)
10+
- **Asset pipeline**: USD-native scenes, NVIDIA Nucleus assets
11+
- **Sensors**: RTX cameras, LiDAR, depth, contact, IMU (GPU-batched)
12+
- **Synthetic data generation**: Replicator pipeline for domain randomization
13+
- **Fleet replication**: parallel environments via `omni.isaac.cloner.Cloner`
14+
- **Isaac Lab integration**: GPU-accelerated RL environments
15+
16+
17+
> **Phase 1 status (skeleton).** This release ships the SimEngine-shaped surface and lazy-import scaffolding only. Several methods on `IsaacSimulation` (`add_robot` on the procedural branch, `_load_usd_robot`, `_load_urdf_robot`, `add_object`, `add_camera`, `replicate`) currently return `status: "success"` without instantiating the underlying USD prim or articulation handle. Following the documented Quick Start on a real Isaac Sim install will therefore observe `get_observation()` returning `{}` and `render()` returning blank frames -- no exception is raised. The data-plane wiring (USD stage management, articulation construction, sensor / replicator integration) lands in Phase 2 and later. Treat the Phase-1 surface as an integration contract, not as a working physics path.
18+
19+
## Installation
20+
21+
Isaac Sim is **not installable from PyPI**. It is an NVIDIA Omniverse Kit application that must be installed separately.
22+
23+
### Option 1: NVIDIA Omniverse Launcher (recommended)
24+
25+
1. Download [NVIDIA Omniverse Launcher](https://developer.nvidia.com/omniverse)
26+
2. Install **Isaac Sim 2024.x** (or newer) from the Exchange tab
27+
3. Install Python dependencies:
28+
29+
```bash
30+
pip install 'strands-robots-sim[isaac]'
31+
```
32+
33+
### Option 2: Isaac Lab
34+
35+
```bash
36+
git clone https://github.com/isaac-sim/IsaacLab.git
37+
cd IsaacLab
38+
./isaaclab.sh -i
39+
pip install 'strands-robots-sim[isaac]'
40+
```
41+
42+
### Option 3: Docker
43+
44+
```bash
45+
docker pull nvcr.io/nvidia/isaac-sim:4.5.0
46+
docker run --gpus all -it nvcr.io/nvidia/isaac-sim:4.5.0
47+
# Inside container:
48+
pip install 'strands-robots-sim[isaac]'
49+
```
50+
51+
## Requirements
52+
53+
- NVIDIA GPU (RTX 2070+ or A100/H100 for fleet training)
54+
- CUDA 12.0+
55+
- Isaac Sim 2024.x or newer
56+
- Linux (Ubuntu 22.04+ recommended)
57+
- Python 3.10+
58+
59+
## Quick Start
60+
61+
```python
62+
from strands_robots_sim.isaac import IsaacSimulation, IsaacConfig
63+
64+
# Check availability
65+
available, reason = IsaacSimulation.is_available()
66+
if not available:
67+
print(f"Isaac Sim not available: {reason}")
68+
exit(1)
69+
70+
# Create simulation
71+
config = IsaacConfig(
72+
num_envs=1,
73+
headless=True,
74+
render_mode="rtx_realtime",
75+
)
76+
sim = IsaacSimulation(config)
77+
78+
# Create world and add robot
79+
sim.create_world()
80+
sim.add_robot("so100")
81+
sim.add_camera("front_cam", position=[1.0, 0.0, 0.5])
82+
83+
# Step and render
84+
sim.step(100)
85+
result = sim.render("front_cam")
86+
rgb = result["rgb"] # (H, W, 3) uint8
87+
88+
# Clean up
89+
sim.destroy()
90+
```
91+
92+
## Fleet Training (Multi-Env)
93+
94+
```python
95+
config = IsaacConfig(num_envs=1024, headless=True)
96+
sim = IsaacSimulation(config)
97+
sim.create_world()
98+
sim.add_robot("so100")
99+
sim.replicate(1024) # 1024 parallel environments
100+
101+
for step in range(10000):
102+
sim.step(1)
103+
obs = sim.get_observation("so100") # batched across all envs
104+
# ... RL training loop ...
105+
106+
sim.destroy()
107+
```
108+
109+
## Entry-Point Discovery
110+
111+
Isaac Sim registers as a `strands_robots.backends` entry point:
112+
113+
```python
114+
from importlib.metadata import entry_points
115+
116+
for ep in entry_points(group='strands_robots.backends'):
117+
print(ep.name, '->', ep.value)
118+
# isaac -> strands_robots_sim.isaac.simulation:IsaacSimulation
119+
# isaac_sim -> strands_robots_sim.isaac.simulation:IsaacSimulation
120+
```
121+
122+
## Configuration
123+
124+
### `IsaacConfig` Parameters
125+
126+
| Parameter | Default | Description |
127+
|-----------|---------|-------------|
128+
| `num_envs` | 1 | Number of parallel environments |
129+
| `device` | "cuda:0" | CUDA device |
130+
| `headless` | True | Run without GUI |
131+
| `physics_dt` | 1/120 s | Physics timestep |
132+
| `rendering_dt` | 1/30 s | Rendering timestep |
133+
| `render_mode` | "headless" | "headless", "rtx_realtime", or "rtx_pathtracing" |
134+
| `gravity` | (0, 0, -9.81) | Gravity vector (Z-up) |
135+
| `camera_width` | 640 | Default camera width |
136+
| `camera_height` | 480 | Default camera height |
137+
| `enable_rtx_sensors` | True | Enable RTX-accelerated sensors |
138+
139+
### Environment Variables
140+
141+
| Variable | Default | Description |
142+
|----------|---------|-------------|
143+
| `STRANDS_ISAAC_HEADLESS` | - | Override headless mode ("true"/"false") |
144+
| `STRANDS_ISAAC_RTX_PATHTRACING` | - | Enable RTX pathtracing ("true"/"false") |
145+
| `STRANDS_ISAAC_NUCLEUS_URL` | - | Override Omniverse Nucleus server URL |
146+
147+
## Procedural Robots
148+
149+
The following robots can be added without any asset files:
150+
151+
- `so100` (aliases: `so-100`, `so_100`, `so101`) -- 6-DOF tabletop arm
152+
- `panda` (aliases: `franka`, `franka_panda`) -- 7-DOF manipulator
153+
- `unitree_g1` (aliases: `g1`) -- 21-DOF humanoid (simplified)
154+
155+
```python
156+
sim.add_robot("so100") # procedural, no asset files needed
157+
sim.add_robot("panda")
158+
sim.add_robot("g1", data_config="unitree_g1")
159+
```
160+
161+
## Comparison with Newton Backend
162+
163+
| Feature | Newton (Warp) | Isaac Sim |
164+
|---------|:---:|:---:|
165+
| Physics parallelism | 4096+ envs | 1024 envs |
166+
| Rendering | OpenGL/null | RTX photorealistic |
167+
| USD native | Partial | Full |
168+
| Sensors (camera, LiDAR) | Basic | RTX GPU-batched |
169+
| Synthetic data gen | No | Replicator |
170+
| Soft body/cloth | Yes (VBD) | Yes (PhysX) |
171+
| Differentiable sim | Yes (Warp tape) | No |
172+
| Install size | ~500MB | ~30GB |
173+
| Use case | Fast RL training | Photorealistic sim2real |
174+
175+
## Architecture
176+
177+
```
178+
strands_robots_sim/isaac/
179+
__init__.py PEP 562 lazy exports (zero omni overhead on import)
180+
config.py IsaacConfig dataclass
181+
simulation.py IsaacSimulation(SimEngine) -- main backend class
182+
procedural.py SO-100 / Panda / G1 USD prim builders
183+
stages.py USD stage management (Phase 2)
184+
sensors.py RTX camera, LiDAR wrappers (Phase 3)
185+
replicator.py Domain randomization (Phase 3)
186+
tests/
187+
test_unit.py Mocked tests (no GPU)
188+
test_entrypoint.py Entry-point verification
189+
test_gpu_integ.py GPU tests (STRANDS_GPU_TEST=1)
190+
```
191+
192+
## Thread Safety
193+
194+
- All mutable state protected by `threading.RLock`
195+
- `step()` must not run concurrently with `add_robot()`
196+
- `SimulationApp` is a process-wide singleton (never create more than one)
197+
- `destroy()` clears the World but does NOT shut down `SimulationApp`
198+
199+
## Testing
200+
201+
```bash
202+
# Unit tests (no GPU required)
203+
pytest strands_robots_sim/isaac/tests/test_unit.py -v
204+
pytest strands_robots_sim/isaac/tests/test_entrypoint.py -v
205+
206+
# GPU integration tests (requires Isaac Sim)
207+
STRANDS_GPU_TEST=1 pytest strands_robots_sim/isaac/tests/test_gpu_integ.py -v
208+
```

‎strands_robots_sim/isaac/tests/test_phase1_doc_banner.py‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
"""Documentation honesty pin: Phase 1 status banner in docs/backends/isaac.md.
2+
3+
R2 review on #31 (``simulation.py:627`` thread) asked for a doc note in
4+
``docs/backends/isaac.md``'s Quick Start that the Phase 1 skeleton
5+
silently no-ops the data plane (``add_robot`` on the procedural branch,
6+
``_load_usd_robot``, ``_load_urdf_robot``, ``add_object``, ``add_camera``,
7+
``replicate`` all return ``status: "success"`` without instantiating the
8+
underlying USD prim or articulation handle). Reviewer's "at minimum"
9+
ask: a banner before the Installation section, so the disclosure precedes
10+
the documented call sequence.
11+
12+
Companion pin for the G1 DOF count lives in ``test_procedural_g1_dof.py``
13+
(landed in PR-3 alongside ``procedural.py`` itself). This file pins only
14+
the docs-side concerns.
15+
"""
16+
17+
from __future__ import annotations
18+
19+
from pathlib import Path
20+
21+
_ISAAC_DOCS = Path(__file__).resolve().parent.parent.parent.parent / "docs" / "backends" / "isaac.md"
22+
23+
24+
class TestIsaacDocsPhase1Banner:
25+
"""Pin: ``docs/backends/isaac.md`` must disclose Phase 1 data-plane no-ops."""
26+
27+
def test_isaac_docs_file_exists(self) -> None:
28+
assert _ISAAC_DOCS.is_file(), f"missing Isaac doc page at {_ISAAC_DOCS}"
29+
30+
def test_phase1_banner_present_before_installation(self) -> None:
31+
"""Banner must appear before the Installation / Quick Start sections.
32+
33+
Reviewer (R2 on PR #31, ``simulation.py:627`` thread): "At minimum,
34+
please add a note to ``docs/backends/isaac.md`` Quick Start that the
35+
Phase-1 skeleton silently no-ops the data plane."
36+
"""
37+
text = _ISAAC_DOCS.read_text(encoding="utf-8")
38+
39+
# The banner must appear AND must appear before the Installation
40+
# section header (so the disclosure precedes any procedural docs the
41+
# user would otherwise execute).
42+
banner_marker = "Phase 1 status"
43+
install_marker = "## Installation"
44+
45+
assert banner_marker in text, (
46+
"docs/backends/isaac.md missing the Phase 1 status disclosure "
47+
"banner. R2 reviewer asked for it explicitly because the doc's "
48+
"Quick Start otherwise executes a code path that silently no-ops "
49+
"on a real Isaac Sim install."
50+
)
51+
assert (
52+
install_marker in text
53+
), "docs/backends/isaac.md missing the Installation section -- doc structure has changed; pin needs review."
54+
assert text.find(banner_marker) < text.find(install_marker), (
55+
"Phase 1 banner must precede the Installation section so the "
56+
"user sees the disclosure before following the install / quick-"
57+
"start steps."
58+
)
59+
60+
def test_phase1_banner_names_the_silent_methods(self) -> None:
61+
"""Banner must enumerate the Phase-1 silent-success methods.
62+
63+
Without naming the methods, a future maintainer who reads only the
64+
banner won't know which API surfaces are affected, and the disclosure
65+
becomes a vague hedge.
66+
"""
67+
text = _ISAAC_DOCS.read_text(encoding="utf-8")
68+
# Slice the banner block (`> **Phase 1 status...**` paragraph).
69+
banner_start = text.find("Phase 1 status")
70+
# Banner is one paragraph; cut at the next `## ` heading.
71+
banner_end = text.find("##", banner_start)
72+
assert banner_end > banner_start, "could not locate end of banner block"
73+
banner = text[banner_start:banner_end]
74+
75+
for needed in ("add_robot", "replicate", "get_observation"):
76+
assert needed in banner, (
77+
f"Phase 1 banner does not mention `{needed}`; the disclosure "
78+
f"must enumerate the silent-success methods so users know "
79+
f"which call sites are affected."
80+
)

0 commit comments

Comments
 (0)