Skip to content

Commit f8dbe3a

Browse files
DozgulbasDozgulbas
committed
2 parents 887e990 + d9cec0a commit f8dbe3a

4 files changed

Lines changed: 238 additions & 73 deletions

File tree

‎README.md‎

Lines changed: 163 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ This repository contains a modified fork of LeRobot with OpenArm-specific enhanc
88

99
- **Bimanual OpenArm Support**: Control two OpenArm follower robots simultaneously
1010
- **Gamepad Teleoperation**: PlayStation/Xbox controller support for intuitive joint control
11+
- **Leader Arm Teleoperation**: Bimanual leader arm control for high-quality demonstration collection
1112
- **Hardware Calibration Sync**: Proper integration with OpenArm's hardware calibration system
1213
- **Velocity & Torque Support**: Full action space including position, velocity, and torque
1314
- **Recording & Replay**: Compatible with LeRobot's dataset recording and policy training
@@ -88,48 +89,87 @@ If the arms move to awkward or asymmetric positions, proceed with recalibration.
8889
**WARNING**: This will overwrite the existing calibration. Only do this if verification above failed.
8990

9091
```bash
91-
# Calibrate right arm (can0)
92+
# Calibrate follower right arm (can0)
9293
openarm-can-zero-position-calibration --canport can0 --arm-side right_arm
9394

94-
# Calibrate left arm (can1)
95+
# Calibrate follower left arm (can1)
9596
openarm-can-zero-position-calibration --canport can1 --arm-side left_arm
96-
```
9797

98-
This tool:
99-
1. Moves each joint to its mechanical stops
100-
2. Calculates the center position
101-
3. Sets that as zero in motor memory
102-
4. Stores the calibration permanently
98+
# Calibrate leader right arm (can2)
99+
openarm-can-zero-position-calibration --canport can2 --arm-side right_arm
100+
101+
# Calibrate leader left arm (can3)
102+
openarm-can-zero-position-calibration --canport can3 --arm-side left_arm
103+
```
103104

104105
### Sync Calibration with LeRobot
105106

106-
**REQUIRED**: After verifying or recalibrating hardware, sync LeRobot's calibration files with the hardware zero positions:
107+
**REQUIRED**: After verifying or recalibrating hardware, sync LeRobot's calibration files:
107108

108109
```bash
109110
# Delete old LeRobot calibration files (if any)
110-
rm -rf ~/.cache/huggingface/lerobot/calibration/robots/bi_openarm_follower/
111+
rm -rf ~/.cache/huggingface/lerobot/calibration/
112+
113+
# Calibrate follower arms
114+
lerobot-calibrate \
115+
--robot.type=openarm_follower \
116+
--robot.port=can0 \
117+
--robot.side=right \
118+
--robot.id=my_openarm_follower_right
119+
120+
lerobot-calibrate \
121+
--robot.type=openarm_follower \
122+
--robot.port=can1 \
123+
--robot.side=left \
124+
--robot.id=my_openarm_follower_left
125+
126+
# Calibrate leader arms
127+
lerobot-calibrate \
128+
--teleop.type=openarm_leader \
129+
--teleop.port=can2 \
130+
--teleop.id=my_openarm_leader_right
131+
132+
lerobot-calibrate \
133+
--teleop.type=openarm_leader \
134+
--teleop.port=can3 \
135+
--teleop.id=my_openarm_leader_left
136+
```
111137

112-
# Run calibration sync script
113-
# This moves arms to hardware zero and creates LeRobot calibration files
114-
python scripts/sync_calibration.py
138+
### Verify CAN Interfaces
139+
140+
After plugging in USB-CAN adapters, verify they're configured:
141+
142+
```bash
143+
ip link show can0 can1 can2 can3
115144
```
116145

117-
**What this does:**
118-
1. Uses OpenArm C++ library to move arms to hardware-calibrated zero
119-
2. Connects LeRobot and sets current position as LeRobot's zero
120-
3. Saves calibration files to `~/.cache/huggingface/lerobot/calibration/`
146+
All four interfaces should show as UP. If not, unplug and replug the USB-CAN adapters.
121147

148+
**CAN port mapping:**
149+
- `can0` — follower right arm
150+
- `can1` — follower left arm
151+
- `can2` — leader right arm
152+
- `can3` — leader left arm
122153

123-
### Verify CAN Interfaces
154+
### Camera Setup
124155

125-
After plugging in USB-CAN adapters, verify they're configured:
156+
USB wrist cameras are identified by physical USB port using udev symlinks. After plugging in cameras:
126157

127158
```bash
128-
# Check CAN interfaces
129-
ip link show can0 can1
159+
# Verify symlinks are created
160+
ls -la /dev/video-wrist-*
161+
# Expected:
162+
# /dev/video-wrist-left -> videoX
163+
# /dev/video-wrist-right -> videoX
164+
```
165+
166+
If symlinks are missing, check the udev rules file at `/etc/udev/rules.d/99-openarm-cameras.rules`.
130167

131-
# Should show both as UP and RUNNING
132-
# If not, unplug and replug USB-CAN adapters (auto-configures via udev)
168+
Get the RealSense chest camera serial number:
169+
170+
```bash
171+
source ~/humanoids/lerobot_env/bin/activate
172+
python3 -c "import pyrealsense2 as rs; ctx = rs.context(); print([d.get_info(rs.camera_info.serial_number) for d in ctx.devices])"
133173
```
134174

135175
### Manual CAN Configuration (if needed)
@@ -187,6 +227,55 @@ lerobot-teleoperate \
187227
- **Triangle/Y**: Print current arm position
188228
- **Circle/B**: Exit teleoperation
189229

230+
### Leader Arm Teleoperation
231+
232+
Leader arm teleoperation produces significantly smoother demonstrations than gamepad control and is recommended for high-quality data collection.
233+
234+
**CAN port mapping for leader arm setup:**
235+
- `can0/can1` — follower right/left arms
236+
- `can2/can3` — leader right/left arms
237+
238+
**Set camera formats before starting:**
239+
```bash
240+
v4l2-ctl --device=/dev/video-wrist-left --set-fmt-video=width=640,height=480,pixelformat=MJPG
241+
v4l2-ctl --device=/dev/video-wrist-right --set-fmt-video=width=640,height=480,pixelformat=MJPG
242+
```
243+
244+
**Teleoperate with cameras:**
245+
```bash
246+
lerobot-teleoperate \
247+
--robot.type=bi_openarm_follower \
248+
--robot.left_arm_config.port=can1 \
249+
--robot.left_arm_config.side=left \
250+
--robot.left_arm_config.cameras="{ \
251+
chest: {type: intelrealsense, serial_number_or_name: YOUR_SERIAL, width: 848, height: 480, fps: 30}, \
252+
wrist_left: {type: opencv, index_or_path: /dev/video-wrist-left, width: 640, height: 480, fps: 30, fourcc: MJPG} \
253+
}" \
254+
--robot.right_arm_config.port=can0 \
255+
--robot.right_arm_config.side=right \
256+
--robot.right_arm_config.cameras="{ \
257+
wrist_right: {type: opencv, index_or_path: /dev/video-wrist-right, width: 640, height: 480, fps: 30, fourcc: MJPG} \
258+
}" \
259+
--robot.id=my_bimanual_follower \
260+
--teleop.type=bi_openarm_leader \
261+
--teleop.left_arm_config.port=can3 \
262+
--teleop.right_arm_config.port=can2 \
263+
--teleop.id=my_bimanual_leader \
264+
--teleop.left_arm_config.position_kp="[120,120,60,20,12,15,12,2]" \
265+
--teleop.left_arm_config.position_kd="[2,2,1.0,0.5,0.1,0.1,0.1,0.02]" \
266+
--teleop.right_arm_config.position_kp="[120,120,60,20,12,15,12,2]" \
267+
--teleop.right_arm_config.position_kd="[2,2,1.0,0.5,0.1,0.1,0.1,0.02]" \
268+
--robot.left_arm_config.position_kp="[240,240,120,40,24,31,25,5]" \
269+
--robot.left_arm_config.position_kd="[5,5,1.5,0.3,0.3,0.3,0.3,0.05]" \
270+
--robot.right_arm_config.position_kp="[240,240,120,40,24,31,25,5]" \
271+
--robot.right_arm_config.position_kd="[5,5,1.5,0.3,0.3,0.3,0.3,0.05]" \
272+
--display_data=true
273+
```
274+
275+
**Tuning notes:**
276+
- `teleop` kp/kd values control leader arm stiffness — lower values make the leader easier to backdrive
277+
- `robot` kp/kd values control follower arm responsiveness
278+
190279
### Common Issues
191280

192281
**Controller not responding:**
@@ -201,6 +290,8 @@ lerobot-teleoperate \
201290

202291
## Recording Demonstrations
203292

293+
### Gamepad Recording
294+
204295
```bash
205296
export HF_HUB_OFFLINE=1
206297

@@ -214,18 +305,62 @@ lerobot-record \
214305
--teleop.joint_velocity_scale=60.0 \
215306
--dataset.repo_id=local/my_task \
216307
--dataset.single_task="Task description" \
217-
--dataset.fps=60 \
308+
--dataset.fps=30 \
218309
--dataset.num_episodes=50 \
219310
--dataset.episode_time_s=30 \
220311
--dataset.reset_time_s=10 \
221312
--dataset.push_to_hub=false
222313
```
223314

315+
### Leader Arm Recording
316+
317+
```bash
318+
export HF_HUB_OFFLINE=1
319+
320+
v4l2-ctl --device=/dev/video-wrist-left --set-fmt-video=width=640,height=480,pixelformat=MJPG
321+
v4l2-ctl --device=/dev/video-wrist-right --set-fmt-video=width=640,height=480,pixelformat=MJPG
322+
323+
lerobot-record \
324+
--robot.type=bi_openarm_follower \
325+
--robot.left_arm_config.port=can1 \
326+
--robot.left_arm_config.side=left \
327+
--robot.left_arm_config.cameras="{ \
328+
chest: {type: intelrealsense, serial_number_or_name: YOUR_SERIAL, width: 848, height: 480, fps: 30, use_depth: true}, \
329+
wrist_left: {type: opencv, index_or_path: /dev/video-wrist-left, width: 640, height: 480, fps: 30, fourcc: MJPG} \
330+
}" \
331+
--robot.right_arm_config.port=can0 \
332+
--robot.right_arm_config.side=right \
333+
--robot.right_arm_config.cameras="{ \
334+
wrist_right: {type: opencv, index_or_path: /dev/video-wrist-right, width: 640, height: 480, fps: 30, fourcc: MJPG} \
335+
}" \
336+
--robot.id=my_bimanual_follower \
337+
--teleop.type=bi_openarm_leader \
338+
--teleop.left_arm_config.port=can3 \
339+
--teleop.right_arm_config.port=can2 \
340+
--teleop.id=my_bimanual_leader \
341+
--teleop.left_arm_config.position_kp="[120,120,60,20,12,15,12,2]" \
342+
--teleop.left_arm_config.position_kd="[2,2,1.0,0.5,0.1,0.1,0.1,0.02]" \
343+
--teleop.right_arm_config.position_kp="[120,120,60,20,12,15,12,2]" \
344+
--teleop.right_arm_config.position_kd="[2,2,1.0,0.5,0.1,0.1,0.1,0.02]" \
345+
--dataset.repo_id=local/my_task \
346+
--dataset.single_task="Task description" \
347+
--dataset.fps=30 \
348+
--dataset.num_episodes=100 \
349+
--dataset.episode_time_s=40 \
350+
--dataset.reset_time_s=10 \
351+
--dataset.push_to_hub=false \
352+
--display_data=false
353+
```
354+
355+
To resume adding episodes to an existing dataset add `--resume=true` to the command.
356+
224357
**Recording Parameters:**
225-
- `--dataset.fps`: Recording framerate (60 fps recommended for smooth playback)
226-
- `--dataset.num_episodes`: Number of demonstrations
227-
- `--dataset.episode_time_s`: Maximum episode duration
228-
- `--dataset.reset_time_s`: Time to reset between episodes
358+
- `--dataset.fps`: Recording framerate (30 fps recommended for leader arm)
359+
- `--dataset.num_episodes`: Number of demonstrations to collect
360+
- `--dataset.episode_time_s`: Maximum episode duration in seconds
361+
- `--dataset.reset_time_s`: Time between episodes for scene reset
362+
- `--dataset.push_to_hub`: Set to false for local-only storage
363+
- `--resume`: Set to true to append episodes to an existing dataset
229364

230365
**Dataset location:** `~/.cache/huggingface/lerobot/local/`
231366

‎scripts/sync_calibration.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,8 @@
77
# Step 1: Move to hardware zero
88
print("\nStep 1: Moving arms to hardware-calibrated zero...")
99

10-
right_arm = oa.OpenArm("can0", True)
11-
left_arm = oa.OpenArm("can1", True)
10+
right_arm = oa.OpenArm("can2", True)
11+
left_arm = oa.OpenArm("can3", True)
1212

1313
motor_types = [
1414
oa.MotorType.DM8009, oa.MotorType.DM8009,

‎setup_system.sh‎

100644100755
Lines changed: 61 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -1,66 +1,88 @@
11
#!/bin/bash
2-
# One-time system setup for OpenArm module
3-
# Installs system packages and configures auto CAN setup
4-
2+
# System setup for OpenArm bimanual + leader arm module
53
set -e
64

75
echo "=== OpenArm System Setup ==="
86

9-
# 1. Add OpenArm PPA and install system packages
107
echo "Installing OpenArm system packages..."
118
sudo apt-get update
129
sudo apt-get install -y software-properties-common
1310
sudo add-apt-repository -y ppa:openarm/main
1411
sudo apt-get update
1512
sudo apt-get install -y \
16-
can-utils \
17-
iproute2 \
18-
libopenarm-can-dev \
19-
openarm-can-utils
13+
can-utils \
14+
iproute2 \
15+
libopenarm-can-dev \
16+
openarm-can-utils
2017

21-
echo "✓ OpenArm CLI tools installed (openarm-can-zero-position-calibration, etc.)"
18+
echo "OpenArm CLI tools installed"
2219

23-
# 2. Setup udev rules for automatic CAN configuration on USB connection
24-
echo ""
25-
echo "Setting up automatic CAN configuration..."
20+
echo "Setting up systemd-networkd CAN configuration..."
21+
22+
sudo tee /etc/systemd/network/can0.network > /dev/null <<'EOF'
23+
[Match]
24+
Name=can0
25+
26+
[CAN]
27+
BitRate=1000000
28+
DataBitRate=5000000
29+
FDMode=yes
30+
RestartSec=100ms
31+
EOF
2632

27-
sudo tee /etc/udev/rules.d/99-openarm-can.rules > /dev/null <<'EOF'
28-
# OpenArm CAN interfaces - auto-configure when USB devices are connected
29-
# This triggers whenever can0 or can1 interface is added to the system
33+
sudo tee /etc/systemd/network/can1.network > /dev/null <<'EOF'
34+
[Match]
35+
Name=can1
3036
31-
ACTION=="add", SUBSYSTEM=="net", KERNEL=="can0", RUN+="/bin/bash -c 'sleep 1 && /usr/bin/ip link set can0 type can bitrate 1000000 dbitrate 5000000 fd on && /usr/bin/ip link set up can0'"
37+
[CAN]
38+
BitRate=1000000
39+
DataBitRate=5000000
40+
FDMode=yes
41+
RestartSec=100ms
42+
EOF
43+
44+
sudo tee /etc/systemd/network/can2.network > /dev/null <<'EOF'
45+
[Match]
46+
Name=can2
3247
33-
ACTION=="add", SUBSYSTEM=="net", KERNEL=="can1", RUN+="/bin/bash -c 'sleep 1 && /usr/bin/ip link set can1 type can bitrate 1000000 dbitrate 5000000 fd on && /usr/bin/ip link set up can1'"
48+
[CAN]
49+
BitRate=1000000
50+
DataBitRate=5000000
51+
FDMode=yes
52+
RestartSec=100ms
3453
EOF
3554

36-
# Reload udev rules
37-
sudo udevadm control --reload-rules
38-
sudo udevadm trigger
55+
sudo tee /etc/systemd/network/can3.network > /dev/null <<'EOF'
56+
[Match]
57+
Name=can3
3958
40-
echo "✓ udev rules installed - CAN interfaces will auto-configure on USB connection"
59+
[CAN]
60+
BitRate=1000000
61+
DataBitRate=5000000
62+
FDMode=yes
63+
RestartSec=100ms
64+
EOF
4165

42-
# 3. Configure CAN now (for current session)
43-
echo ""
44-
echo "Configuring CAN interfaces for current session..."
45-
sudo ip link set can0 type can bitrate 1000000 dbitrate 5000000 fd on 2>/dev/null || true
46-
sudo ip link set up can0 2>/dev/null || true
47-
sudo ip link set can1 type can bitrate 1000000 dbitrate 5000000 fd on 2>/dev/null || true
48-
sudo ip link set up can1 2>/dev/null || true
66+
echo "Enabling systemd-networkd..."
67+
sudo systemctl enable systemd-networkd
68+
sudo systemctl start systemd-networkd
69+
70+
echo "Reloading networkd config..."
71+
sudo networkctl reload
4972

50-
# Check status
5173
echo ""
5274
echo "=== CAN Interface Status ==="
53-
ip link show can0 2>/dev/null && echo "✓ can0 is up" || echo "⚠ can0 not found (plug in USB adapter)"
54-
ip link show can1 2>/dev/null && echo "✓ can1 is up" || echo "⚠ can1 not found (plug in USB adapter)"
75+
for iface in can0 can1 can2 can3; do
76+
ip link show $iface 2>/dev/null && echo "$iface is up" || echo "$iface not found (plug in USB adapter)"
77+
done
5578

5679
echo ""
57-
echo "✓ System setup complete!"
80+
echo "Setup complete! CAN interfaces will now auto-configure on every boot and USB plug."
5881
echo ""
5982
echo "Next steps:"
60-
echo " 1. Plug in OpenArm USB-CAN adapters (if not already connected)"
61-
echo " 2. Verify CAN: ip link show can0 can1"
62-
echo " 3. Calibrate hardware:"
63-
echo " openarm-can-zero-position-calibration --canport can0 --arm-side right_arm"
64-
echo " openarm-can-zero-position-calibration --canport can1 --arm-side left_arm"
65-
echo " 4. Sync with LeRobot:"
66-
echo " python scripts/sync_calibration.py"
83+
echo " 1. Verify CAN: ip link show can0 can1 can2 can3"
84+
echo " 2. Calibrate hardware:"
85+
echo " openarm-can-zero-position-calibration --canport can0 --arm-side right_arm # follower right"
86+
echo " openarm-can-zero-position-calibration --canport can1 --arm-side left_arm # follower left"
87+
echo " openarm-can-zero-position-calibration --canport can2 --arm-side right_arm # leader right"
88+
echo " openarm-can-zero-position-calibration --canport can3 --arm-side left_arm # leader left"

0 commit comments

Comments
 (0)