Question: How did you implement RoboBrain 2.0 on the Unitree G1? Is there a way to connect model outputs to the robot?

[mohdammari97]

Hi and thank you for the amazing work on RoboBrain 2.0!

I noticed in your videos that you were able to run RoboBrain on a Unitree G1 humanoid robot, performing tasks like locomotion, grasping and object interaction. I'm currently working with a G1 as well, and I would love to replicate or build on your results.

Would it be possible to clarify the following:

How did you extract the model’s output and translate it into actual robot commands?
Did you use joint-level actions, end-effector poses, or high-level commands?

Is any of the code available for this robot-side integration?

What action format did you use for the G1?

I assume, from what I read, that RoboBrain outputs symbolic actions or task plans.

Any guidance or examples would be greatly appreciated.

Thanks again for the great research and open-source release!

[bbbbbbbbly]

对，我也很好奇，为什么输出是图片中的像素，怎么转为机器人的动作

[Hitjss]

I have the same question. Have you figured it out?

[XieWeikai]

I have the same question.

[tanhuajie]

We re-target 2D to 3D position with depth info from camera.

[wjydym]

We re-target 2D to 3D position with depth info from camera.

Then use traditional motion plan? Or an end2end way that use the labeled image as input?

[tanhuajie]

RoboBrain 2.5 series now supports direct 3D trajectory prediction. Both 2.0 and 2.5 continue to use traditional motion planning with 3D trajectories.

[wjydym]

RoboBrain 2.5 series now supports direct 3D trajectory prediction. Both 2.0 and 2.5 continue to use traditional motion planning with 3D trajectories.

Thanks.