Skip to content

Add G1 Factory environments#887

Draft
TuurStuyck wants to merge 4 commits into
mainfrom
tstuyck/feature/g1-factory-port
Draft

Add G1 Factory environments#887
TuurStuyck wants to merge 4 commits into
mainfrom
tstuyck/feature/g1-factory-port

Conversation

@TuurStuyck

@TuurStuyck TuurStuyck commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

Ports the G1 Factory environment set onto current IsaacLab-Arena main, keeping G1-specific task/robot code isolated under:

isaaclab_arena_environments/G1_Factory/

Includes:

  • 9 G1 Factory environment entrypoints: LMDrillLift, LMDrillPnP, LMPushButton, LMBoxLift, LMBoxLiftFloor, LMBoxTableToShelfPnP, LMDrillLiftObs, LMPickDrillFromHolder, LMPushShelfForward
  • G1-local assets/events/terminations/task wrappers/pose utilities
  • Per-task GR00T closed-loop Arena-side translation configs under policy_configs/benchmark/
  • G1 Factory README with checkpoint references and run commands
  • Schema tests that load all G1 GR00T configs against current Gr00tClosedloopPolicyCfg

Design notes

The port targets feature parity while reusing Arena main and avoiding shared Arena core changes. G1-specific behavior is contained under isaaclab_arena_environments/G1_Factory/.

The legacy G1 policy YAMLs included model_path. Arena main's Gr00tRemoteClosedloopPolicy now uses YAML only for Arena-side observation/action translation. The checkpoint is loaded by the external GR00T server, so this PR documents checkpoint references in the G1 Factory README instead of adding unsupported YAML keys.

Important preserved legacy policy values:

  • action_horizon: 16
  • action_chunk_length: 16
  • pov_cam_name_sim: robot_head_cam_rgb
  • task_mode_name: g1_locomanipulation
  • G1 GR00T modality/joint config paths
  • legacy task language instructions

The only intentional schema normalization is embodiment_tag: NEW_EMBODIMENT.

Validation performed

  • pre-commit run --files on changed README/test/YAML files: passed
  • G1 config tests in Docker/Isaac Sim Python: 10 passed
  • All 9 G1 Factory GR00T configs load through Arena's runtime create_config_from_yaml(..., Gr00tClosedloopPolicyCfg) path
  • GR00T client import path validated with submodules/Isaac-GR00T on PYTHONPATH
  • LMDrillLift zero-action smoke:
    • headless, no cameras: completed 2 rollout steps
    • headless, --enable_cameras: completed 1 rollout step and confirmed camera_obs.robot_head_cam_rgb shape (480, 640, 3)

The zero_action policy is only an environment/action-space smoke test. It is not expected to produce stable learned G1 Factory behavior.

Example: zero-action smoke test

PYTHONPATH=/workspaces/isaaclab_arena:${PYTHONPATH:-} \
submodules/IsaacLab/isaaclab.sh -p \
  isaaclab_arena/evaluation/policy_runner.py \
  --policy_type zero_action \
  --num_steps 20 \
  --viz none \
  --external_environment_class_path isaaclab_arena_environments.G1_Factory.LMDrillLift:LMDrillLift \
  LMDrillLift

Example: G1 Factory GR00T checkpoint run path

Initialize the GR00T submodule first:

git -c url.https://github.com/.insteadOf=git@github.com: submodule update --init submodules/Isaac-GR00T

Start the GR00T server with a locally available G1 Factory checkpoint. The legacy checkpoint references are listed in isaaclab_arena_environments/G1_Factory/README.md; cloning arena_data provides assets, not these model checkpoints.

cd submodules/Isaac-GR00T
export G1_FACTORY_CHECKPOINT=/path/to/g1_benchmark_LMDrillLiftD1_100_gn1_5_output/checkpoint-20000
test -d "${G1_FACTORY_CHECKPOINT}"

uv run python gr00t/eval/run_gr00t_server.py \
  --model-path "${G1_FACTORY_CHECKPOINT}" \
  --embodiment-tag NEW_EMBODIMENT \
  --device cuda \
  --host 127.0.0.1 \
  --port 5555

From the Arena repository root in a second terminal, run the Arena client:

PYTHONPATH="$(pwd):$(pwd)/submodules/Isaac-GR00T:${PYTHONPATH:-}" \
submodules/IsaacLab/isaaclab.sh -p \
  isaaclab_arena/evaluation/policy_runner.py \
  --policy_type isaaclab_arena_gr00t.policy.gr00t_remote_closedloop_policy.Gr00tRemoteClosedloopPolicy \
  --policy_config_yaml_path isaaclab_arena_environments/G1_Factory/policy_configs/benchmark/LMDrillLift_gr00t_closedloop.yaml \
  --remote_host 127.0.0.1 \
  --remote_port 5555 \
  --num_steps 800 \
  --viz kit \
  --enable_cameras \
  --external_environment_class_path isaaclab_arena_environments.G1_Factory.LMDrillLift:LMDrillLift \
  LMDrillLift

Real-policy validation requires the external GR00T server and a locally available G1 Factory checkpoint, so this PR validates the Arena-side schema, scene construction, action-space path, camera observation path, and GR00T client import path.

Signed-off-by: Tuur Stuyck <tstuyck@nvidia.com>
Signed-off-by: Tuur Stuyck <tstuyck@nvidia.com>
Signed-off-by: Tuur Stuyck <tstuyck@nvidia.com>
Signed-off-by: Tuur Stuyck <tstuyck@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant