UR5 Robot Integration Questions and Considerations

[ol-s-cloud]

Discussion: OpenPI Integration with UR5

Hello OpenPI team,

I noticed interest in using OpenPI with UR5 robots (issue #289) and would like to better understand the requirements before attempting an implementation. I have some specific questions after examining the codebase:

Current Understanding and Questions

1. Action Dimensions

The UR5 has 6 DOF, whereas I see ALOHA uses 14 DOF (lines 103-111 in src/openpi/policies/aloha_policy.py):

def _joint_flip_mask() -> np.ndarray:
    """Used to convert between ALOHA and PI joint angles."""
    return np.array([1, -1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, 1, 1])

• How should the action dimensions be mapped for a single-arm robot like UR5?
• Would the configuration need action_dim=6, or does the model expect a specific format?

2. Camera Setup

The DROID model uses multiple cameras, including a wrist camera:

# From src/openpi/policies/droid_policy.py
CAMERAS = {
    "exterior_image_1_left": "observation/exterior_image_1_left",
    "exterior_image_2_right": "observation/exterior_image_2_right",
    "wrist_image_left": "observation/wrist_image_left",
}

• Is a wrist camera required for UR5 integration?
• What’s the minimum viable camera setup (number and positioning)?

3. PI Internal Runtime

I see references to a "PI internal runtime" in issues #324 and #326, where different robots are standardized:

# The adapt_to_pi transformation appears critical
adapt_to_pi=self.adapt_to_pi

• How should the UR5 joint space be transformed to align with OpenPI’s internal representation?
• Are there specific joint conventions or coordinate frames I should follow?

4. Using Existing Checkpoints

Since both UR5 and DROID are single-arm robots:

• Could UR5 leverage the existing DROID checkpoints?
• Or would fine-tuning from the base model be necessary?

Next Steps

Based on the responses, I'd like to work on implementing UR5 support by:

• Creating a ur5_policy.py module with appropriate transforms
• Adding UR5 configurations in config.py
• Documenting the setup requirements

Any guidance from the team would be greatly appreciated before I start work on this integration.
Thanks!

[Kelym]

You could see in the UR5 tutorial how to set up action dim and cameras for finetuning UR5.

[xllau]

I have the same question. Since I am using a robot arm that is not exactly same with UR5, only part alike. I want to test the model on our platform. Any thing more than the UR5 tutorial ?

[Saurav1700]

Did you manage to get it running?
I am using libero as it looks like the example script used that. I am using one shoulder camera and one wrist camera, do i need to perform tf2 for camera before i feed that in the VLA model ?

I am using ROS2 humble. With the base model, the robot moves randomly. I think I am doing something wrong here. Any help would be appreciated.

[ankitupadhyay0]

Good questions on UR5 integration. Here's what I think:

Action Dimensions
The dimension mismatch needs fixing. For UR5's 6 DOF:

Set action_dim=6 in your UR5 policy config
Make a custom joint flip mask: np.array([1, -1, -1, 1, 1, 1]) (adjust signs for your UR5's joint setup)
The model should handle 6D actions fine since the policy head is usually a linear layer

Camera Setup
A wrist camera isn't required based on DROID. But multi-view helps:

Minimum: 1-2 external cameras covering the workspace well
Better: Add wrist cam if you can for more precise manipulation
Update the CAMERAS dict in your ur5_policy.py to match your cameras

PI Internal Runtime
The adapt_to_pi transform matters for joint space alignment. You need:

UR5 joint limits and home position defined
Coordinate frame transforms (base_link conventions, etc.)
Joint velocities/positions in OpenPI's expected ranges

Existing Checkpoints
Starting with DROID checkpoints makes sense since both are single-arm. But you'll need:

Fine-tuning on UR5 data because of different kinematics
Maybe adjust policy head dimensions if action spaces differ a lot

Implementation approach:

Start with minimal ur5_policy.py based on droid_policy.py
Test your camera setup before adding complexity
Collect small dataset for validation before full training