Skip to content

add benchmark for MyoSim MyoArm#937

Open
thowell wants to merge 1 commit intogoogle-deepmind:mainfrom
thowell:myo_sim
Open

add benchmark for MyoSim MyoArm#937
thowell wants to merge 1 commit intogoogle-deepmind:mainfrom
thowell:myo_sim

Conversation

@thowell
Copy link
Copy Markdown
Collaborator

@thowell thowell commented Dec 19, 2025 •
edited
Loading

add a benchmark for the MyoSim MyoArm musculoskeletal environment

run.py -f myoarm

myoarm.jit_duration: 104.49
myoarm.run_time: 7.69
myoarm.steps_per_second: 1,065,244
myoarm.converged_worlds: 8192
myoarm.ncon_mean: 2.22
myoarm.ncon_p95: 3.0
myoarm.nefc_mean: 15.58
myoarm.nefc_p95: 20.0
myoarm.solver_niter_mean: 1.24
myoarm.solver_niter_p95: 2.0
myoarm.step: 936.15 ns
  myoarm.step.euler: 120.46
  myoarm.step.forward: 815.14
    myoarm.step.forward.fwd_position: 460.56
      myoarm.step.forward.fwd_position.tendon: 279.02
      myoarm.step.forward.fwd_position.kinematics: 58.06
      myoarm.step.forward.fwd_position.transmission: 51.36
    myoarm.step.forward.solve: 123.76
    myoarm.step.forward.fwd_acceleration: 119.47
    myoarm.step.forward.fwd_velocity: 69.56
    myoarm.step.forward.fwd_actuation: 34.48

myoarm

previous (out of date)

a utility script benchmark/myosim/get_myosim.py is provided to clone MyoSim (pinned commit) from GitHub.

python get_myosim.py

testspeed

mjwarp-testspeed benchmark/myo_sim/arm/myoarm.xml --nworld=8192 --nconmax=16 --njmax=48 -o "opt.ccd_iterations=50" --event_trace=True --measure_alloc=True --memory=True

note: the default ccd_iterations (35) is too low and produces warnings

Loading model from: benchmark/myo_sim/arm/myoarm.xml...
  nbody: 40 nv: 38 ngeom: 161 nu: 63 is_sparse: True
  broadphase: NXN broadphase_filter: PLANE|SPHERE|OBB
  solver: NEWTON cone: PYRAMIDAL iterations: 100 iterative linesearch iterations: 50
  integrator: EULER graph_conditional: True
Data
  nworld: 8192 naconmax: 131072 njmax: 48

Rolling out 1000 steps at dt = 0.002...

Summary for 8192 parallel rollouts

Total JIT time: 0.59 s
Total simulation time: 11.17 s
Total steps per second: 733,257
Total realtime factor: 1,466.51 x
Total time per step: 1363.78 ns
Total converged worlds: 8192 / 8192

Event trace:

step: 1360.71
  forward: 1233.95
    fwd_position: 688.99
      kinematics: 80.03
      com_pos: 14.73
      camlight: 1.98
      flex: 0.17
      crb: 25.33
      tendon_armature: 9.97
      collision: 13.13
        nxn_broadphase: 4.10
        convex_narrowphase: 6.89
        primitive_narrowphase: 1.23
      make_constraint: 33.30
      transmission: 61.26
    sensor_pos: 0.17
    fwd_velocity: 145.89
      com_vel: 17.44
      passive: 15.31
      rne: 19.94
      tendon_bias: 27.18
    sensor_vel: 0.18
    fwd_actuation: 20.37
    fwd_acceleration: 128.34
      xfrc_accumulate: 4.74
    solve: 247.90
      mul_m: 13.04
    sensor_acc: 0.17
  euler: 126.21

ncon alloc:

mean     std      min    max  
------------------------------
16129.2  9024.78   8192  32768
2218.25  3018.07      2   8192
8850.36  1705.28   7770  15339
16338.3  164.723  15394  16384
  16384        0  16384  16384
13960.6  2936.51   8403  16384
8198.17  25.5985   8192   8367
   8192        0   8192   8192
   8192        0   8192   8192
   8192        0   8192   8192

nefc alloc:

mean   std       min  max
-------------------------
19.98   3.84182   16   32
 25.6  0.969536   23   27
25.81  0.868274   24   27
31.81    1.9733   26   33
30.69    1.0266   30   33
   30         0   30   30
30.02      0.14   30   31
   31         0   31   31
31.96   1.70833   31   35
   35         0   35   35

Total memory: 881.43 MB / 48640.12 MB (1.81%)
Model memory (2.01%):
 (no field >= 1% of utilized memory)
Data memory (97.99%):
 xmat: 11.25 MB (1.28%)
 ximat: 11.25 MB (1.28%)
 geom_xpos: 15.09 MB (1.71%)
 geom_xmat: 45.28 MB (5.14%)
 site_xpos: 47.53 MB (5.39%)
 site_xmat: 142.59 MB (16.18%)
 cinert: 12.50 MB (1.42%)
 ten_J: 79.56 MB (9.03%)
 wrap_obj: 29.00 MB (3.29%)
 wrap_xpos: 87.00 MB (9.87%)
 actuator_moment: 74.81 MB (8.49%)
 crb: 12.50 MB (1.42%)
 qM: 18.16 MB (2.06%)
 qLD: 18.16 MB (2.06%)
 efc.J: 72.00 MB (8.17%)

airspeed velocity

asv run myo_sim^! -b MyoArm

· Creating environments
· Discovering benchmarks
· Running 1 total benchmarks (1 commits * 1 environments * 1 benchmarks)
[ 0.00%] · For mujoco-warp commit 74371556 <main>:
[ 0.00%] ·· Benchmarking virtualenv-py3.13
[100.00%] ··· Setting up benchmark:144                                                                                                                                                                                                ok
[100.00%] ··· benchmark.MyoArm.track_metric                                                                                                                                                                                           ok
[100.00%] ··· =============================================== =====================
                                  function                                         
              ----------------------------------------------- ---------------------
                                jit_duration                    83.98443568497896  
                             solver_niter_mean                   1.3695615234375   
                              solver_niter_p95                         2.0         
                          device_memory_allocated                   941621248      
                                    step                        1363.148438744247  
                                step.forward                    1235.9470552764833 
                         step.forward.fwd_position              690.1426058029756  
                    step.forward.fwd_position.kinematics        80.12486679217545  
                     step.forward.fwd_position.com_pos          14.763125168428815 
                     step.forward.fwd_position.camlight         1.9792499891764237 
                       step.forward.fwd_position.crb            25.370499963173643 
                 step.forward.fwd_position.tendon_armature      9.984999742300715  
                    step.forward.fwd_position.collision         12.493625084971427 
                 step.forward.fwd_position.make_constraint      33.38687501491222  
                   step.forward.fwd_position.transmission        61.5145473384473  
                          step.forward.sensor_pos               0.1754999995569051 
                         step.forward.fwd_velocity              146.33549768768717 
                     step.forward.fwd_velocity.com_vel          17.502500037153368 
                     step.forward.fwd_velocity.passive          15.332375226535078 
                       step.forward.fwd_velocity.rne            20.019249903270975 
                   step.forward.fwd_velocity.tendon_bias        27.18824967450928  
                          step.forward.sensor_vel              0.17499999955816747 
                         step.forward.fwd_actuation             20.456375063076848 
                       step.forward.fwd_acceleration            128.8519998488482  
               step.forward.fwd_acceleration.xfrc_accumulate    4.746574093132949  
                          step.forward.sensor_acc               0.1781249995502776 
                             step.forward.solve                 247.7132497151615  
                                 step.euler                     126.66437595908064 
              =============================================== =====================

todo

  • benchmark/myosim/get_myosim._clone is a direct fork of benchmark/kitchen/populate_scene._clone, make _clone a shared utility function?

@thowell thowell requested a review from erikfrey December 19, 2025 10:55
erikfrey
erikfrey reviewed Jan 6, 2026
View reviewed changes
Copy link
Copy Markdown
Collaborator

@erikfrey erikfrey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's discuss how we want to design benchmarks going forward before we add this one. Might be time to make a few bigger changes.

Comment thread benchmark/benchmark.py Outdated
Comment thread benchmark/myosim/get_myosim.py Outdated
@thowell thowell mentioned this pull request Jan 9, 2026
Closed
@thowell thowell mentioned this pull request Mar 5, 2026
Merged
@thowell thowell force-pushed the myo_sim branch 3 times, most recently from 5b02970 to 96f5c97 Compare April 29, 2026 14:41
@thowell
Copy link
Copy Markdown
Collaborator Author

thowell commented Apr 29, 2026

@erikfrey this benchmark is updated to use the new pattern introduced in #1301

erikfrey
erikfrey approved these changes Apr 30, 2026
View reviewed changes
Copy link
Copy Markdown
Collaborator

@erikfrey erikfrey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, this is really clean. Just one nit.

Comment thread benchmarks/myosim/rollout.webp
Copy link
Copy Markdown
Collaborator

@erikfrey erikfrey Apr 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see the cam_distance flag in mjwarp-record

can you re-record this with a lower cam_distance so we can better see the musculoskeletal model itself?

Copy link
Copy Markdown
Collaborator Author

@thowell thowell Apr 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@erikfrey erikfrey erikfrey approved these changes

Assignees

No one assigned

Labels

benchmark

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@thowell @erikfrey