Skip to content

Commit dd1b531

Browse files
JSeo-KIMMsjh26
authored andcommitted
Update README with training data and policy details
Added detailed steps for synthetic training data generation, training policy, and reduction process. Included references to relevant publications.
1 parent 89d8070 commit dd1b531

1 file changed

Lines changed: 18 additions & 9 deletions

File tree

  • PW45_2026_Boston/Projects/SlicerToActionForSurgicalRobotImitationLearning

PW45_2026_Boston/Projects/SlicerToActionForSurgicalRobotImitationLearning/README.md

Lines changed: 18 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -64,26 +64,35 @@ Furthermore, it will allow us to build **custom simulation environments to gener
6464
<!-- Update this section as you make progress, describing of what you have ACTUALLY DONE.
6565
If there are specific steps that you could not complete then you can describe them here, too. -->
6666

67-
68-
1. Describe specific steps you **have actually done**.
69-
70-
67+
**1. Synthetic Training Data Generation**
68+
Randomly generate diverse bone fragment configurations.
69+
Generate and apply a reduction trajectory that reduces each configuration.
70+
During the reduction process, capture images from four views (Axial, Lateral, Medial, ISO) in 3D Slicer, and store the corresponding strut lengths as an episode.
71+
Save the data as a dataset of 100+ episodes.
72+
**2. Training Policy**
73+
Feed the stored dataset into the LeRobot visuomotor policy framework.
74+
Train an ACT (Action Chunking with Transformers) policy to obtain the policy network (100K steps, loss < 0.03).
75+
**3. Run Reduction**
76+
Randomly generate an arbitrary bone fragment configuration.
77+
(1) Capture the four views in Slicer and feed them, together with the corresponding strut lengths, into the policy as the observation.
78+
(2) Infer strut lengths.
79+
(3) Perform forward kinematics analysis and update the robot's pose.
80+
(4) Return to (1) and iterate.
7181

7282

7383
# Illustrations
7484

7585
<!-- Add pictures and links to videos that demonstrate what has been accomplished. -->
76-
77-
78-
_No response_
86+
<img width="1054" height="567" alt="image" src="https://github.com/user-attachments/assets/99518a87-4d88-4dd1-acc5-0b851af118d4" />
7987

8088

8189

8290
# Background and References
8391

8492
<!-- If you developed any software, include link to the source code repository.
8593
If possible, also add links to sample data, and to any relevant publications. -->
94+
References
95+
[1] T. Z. Zhao, V. Kumar, S. Levine, and C. Finn, "Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware," in Proc. Robotics: Science and Systems (RSS), 2023. arXiv:2304.13705.
96+
[2] R. Cadene, S. Alibert, A. Soare, Q. Gallouédec, A. Zouitine, T. Wolf, et al., "LeRobot: State-of-the-art Machine Learning for Real-World Robotics in PyTorch," 2024. [Online]. Available: https://github.com/huggingface/lerobot
8697

8798

88-
_No response_
89-

0 commit comments

Comments
 (0)