Add G1 Factory environments#887
Draft
TuurStuyck wants to merge 4 commits into
Draft
Conversation
Signed-off-by: Tuur Stuyck <tstuyck@nvidia.com>
Signed-off-by: Tuur Stuyck <tstuyck@nvidia.com>
Signed-off-by: Tuur Stuyck <tstuyck@nvidia.com>
Signed-off-by: Tuur Stuyck <tstuyck@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ports the G1 Factory environment set onto current IsaacLab-Arena
main, keeping G1-specific task/robot code isolated under:Includes:
LMDrillLift,LMDrillPnP,LMPushButton,LMBoxLift,LMBoxLiftFloor,LMBoxTableToShelfPnP,LMDrillLiftObs,LMPickDrillFromHolder,LMPushShelfForwardpolicy_configs/benchmark/Gr00tClosedloopPolicyCfgDesign notes
The port targets feature parity while reusing Arena
mainand avoiding shared Arena core changes. G1-specific behavior is contained underisaaclab_arena_environments/G1_Factory/.The legacy G1 policy YAMLs included
model_path. Arena main'sGr00tRemoteClosedloopPolicynow uses YAML only for Arena-side observation/action translation. The checkpoint is loaded by the external GR00T server, so this PR documents checkpoint references in the G1 Factory README instead of adding unsupported YAML keys.Important preserved legacy policy values:
action_horizon: 16action_chunk_length: 16pov_cam_name_sim: robot_head_cam_rgbtask_mode_name: g1_locomanipulationThe only intentional schema normalization is
embodiment_tag: NEW_EMBODIMENT.Validation performed
pre-commit run --fileson changed README/test/YAML files: passed10 passedcreate_config_from_yaml(..., Gr00tClosedloopPolicyCfg)pathsubmodules/Isaac-GR00TonPYTHONPATHLMDrillLiftzero-action smoke:--enable_cameras: completed 1 rollout step and confirmedcamera_obs.robot_head_cam_rgbshape(480, 640, 3)The
zero_actionpolicy is only an environment/action-space smoke test. It is not expected to produce stable learned G1 Factory behavior.Example: zero-action smoke test
PYTHONPATH=/workspaces/isaaclab_arena:${PYTHONPATH:-} \ submodules/IsaacLab/isaaclab.sh -p \ isaaclab_arena/evaluation/policy_runner.py \ --policy_type zero_action \ --num_steps 20 \ --viz none \ --external_environment_class_path isaaclab_arena_environments.G1_Factory.LMDrillLift:LMDrillLift \ LMDrillLiftExample: G1 Factory GR00T checkpoint run path
Initialize the GR00T submodule first:
Start the GR00T server with a locally available G1 Factory checkpoint. The legacy checkpoint references are listed in
isaaclab_arena_environments/G1_Factory/README.md; cloningarena_dataprovides assets, not these model checkpoints.From the Arena repository root in a second terminal, run the Arena client:
PYTHONPATH="$(pwd):$(pwd)/submodules/Isaac-GR00T:${PYTHONPATH:-}" \ submodules/IsaacLab/isaaclab.sh -p \ isaaclab_arena/evaluation/policy_runner.py \ --policy_type isaaclab_arena_gr00t.policy.gr00t_remote_closedloop_policy.Gr00tRemoteClosedloopPolicy \ --policy_config_yaml_path isaaclab_arena_environments/G1_Factory/policy_configs/benchmark/LMDrillLift_gr00t_closedloop.yaml \ --remote_host 127.0.0.1 \ --remote_port 5555 \ --num_steps 800 \ --viz kit \ --enable_cameras \ --external_environment_class_path isaaclab_arena_environments.G1_Factory.LMDrillLift:LMDrillLift \ LMDrillLiftReal-policy validation requires the external GR00T server and a locally available G1 Factory checkpoint, so this PR validates the Arena-side schema, scene construction, action-space path, camera observation path, and GR00T client import path.