Commit c3ad4f8
committed
feat(simpler_env): real-policy success videos + fix drawer/place/widowx upstream fidelity
The doc gallery was a scripted parity demo (one generic reach-close-lift motion for
all 25 tasks) — it never actually solved anything. Replace it with REAL pretrained
policies driving our MetaSim-native env, and fix genuine fidelity bugs found in the
process.
Real-policy gallery (scripts/render_policy_gallery.py):
- RT-1 (rt_1_tf_trained_for_000400120) for the 21 Google-robot tasks, Octo-base for
the 4 WidowX tasks — matching what SimplerEnv evaluates per embodiment. Records the
first episode the task's own success checker marks solved.
- 13/25 solved on our MetaSim-native env (5 picks + 3 move-near + 5 drawers). place
(0/5) and widowx (0/4) are genuine policy limits — verified identical on upstream
simpler_env (RT-1 place ~0, Octo bridge ~0 there too), not an integration defect.
Fidelity bugs fixed (the "fake-parity trap": our native-vs-_native parity was Δ=0 but
both sides shared the same deviation from upstream, so it hid these):
- drawer + place never implemented SimplerEnv's visual-matching STATION system
(per-episode robot base xy+yaw from the fixed table + the matching real-image
overlay open_drawer_<a0..c2>.png); they used a coke-style random base and no overlay.
- the cabinet used mk_station.urdf (white) instead of mk_station_recolor.urdf (tan,
matches the overlay) — upstream station_name is "mk_station_recolor".
- widowx put-on never applied the bridge overlay (bridge_real_eval_1 / bridge_sink).
Result: initial obs now matches upstream by mean-abs <= ~2/255 across all 6 families
(coke/pick/move-near bitwise; drawer 0.01, place 1.94, widowx 0.0). This upstream-obs
check supersedes the native-vs-reference parity in the Status table.
_assets.py: add cabinet_recolor_urdf (vendored + on HF). Doc gallery regenerated
data-driven from the rollout results (scripts/gen_simpler_doc_gallery.py): 25 videos
with honest per-task ✓ steps / ✗ captions. Old scripted parity clips removed from the
doc (regenerable via scripts/render_metasim_1to1_gallery.py).1 parent 970e5cd commit c3ad4f8
57 files changed
Lines changed: 479 additions & 79 deletions
File tree
- docs/source
- _static/integrations/simpler_env
- dataset_benchmark/integrations
- roboverse_pack/tasks/simpler_env
- _metasim
- _native
- scripts
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
0 commit comments