You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: PW45_2026_Boston/Projects/SlicerToActionForSurgicalRobotImitationLearning/README.md
+18-9Lines changed: 18 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -64,26 +64,35 @@ Furthermore, it will allow us to build **custom simulation environments to gener
64
64
<!-- Update this section as you make progress, describing of what you have ACTUALLY DONE.
65
65
If there are specific steps that you could not complete then you can describe them here, too. -->
66
66
67
-
68
-
1. Describe specific steps you **have actually done**.
69
-
70
-
67
+
**1. Synthetic Training Data Generation**
68
+
Randomly generate diverse bone fragment configurations.
69
+
Generate and apply a reduction trajectory that reduces each configuration.
70
+
During the reduction process, capture images from four views (Axial, Lateral, Medial, ISO) in 3D Slicer, and store the corresponding strut lengths as an episode.
71
+
Save the data as a dataset of 100+ episodes.
72
+
**2. Training Policy**
73
+
Feed the stored dataset into the LeRobot visuomotor policy framework.
74
+
Train an ACT (Action Chunking with Transformers) policy to obtain the policy network (100K steps, loss < 0.03).
75
+
**3. Run Reduction**
76
+
Randomly generate an arbitrary bone fragment configuration.
77
+
(1) Capture the four views in Slicer and feed them, together with the corresponding strut lengths, into the policy as the observation.
78
+
(2) Infer strut lengths.
79
+
(3) Perform forward kinematics analysis and update the robot's pose.
80
+
(4) Return to (1) and iterate.
71
81
72
82
73
83
# Illustrations
74
84
75
85
<!-- Add pictures and links to videos that demonstrate what has been accomplished. -->
<!-- If you developed any software, include link to the source code repository.
85
93
If possible, also add links to sample data, and to any relevant publications. -->
94
+
References
95
+
[1] T. Z. Zhao, V. Kumar, S. Levine, and C. Finn, "Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware," in Proc. Robotics: Science and Systems (RSS), 2023. arXiv:2304.13705.
96
+
[2] R. Cadene, S. Alibert, A. Soare, Q. Gallouédec, A. Zouitine, T. Wolf, et al., "LeRobot: State-of-the-art Machine Learning for Real-World Robotics in PyTorch," 2024. [Online]. Available: https://github.com/huggingface/lerobot
0 commit comments