Skip to content

[Feature] Add process inference server control plane#3896

Merged
vmoens merged 13 commits into
gh/vmoens/291/basefrom
gh/vmoens/291/head
Jul 6, 2026
Merged

[Feature] Add process inference server control plane#3896
vmoens merged 13 commits into
gh/vmoens/291/basefrom
gh/vmoens/291/head

Conversation

@vmoens

@vmoens vmoens commented Jun 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

@vmoens
Update
8b85a5f 
[ghstack-poisoned]
@pytorch-bot

pytorch-bot Bot commented Jun 21, 2026 •
edited
Loading

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3896

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@github-actions github-actions Bot added the Feature New feature label Jun 21, 2026
@vmoens vmoens mentioned this pull request Jun 21, 2026
Merged
This was referenced Jun 21, 2026
Merged
Merged
Merged
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jun 21, 2026
@github-actions

github-actions Bot commented Jun 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmark Results: PR 6f81b6b0 vs main 3fdf0ab0

Benchmark run: https://github.com/pytorch/rl/actions/runs/28797325842

Higher ops/sec is better. Tables are sorted by largest absolute change.

CPU

Compared 216 benchmarks. Regressions over 5%: 11. Improvements over 5%: 8.

Benchmark main ops PR ops Change
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[False-None] 60.30 89.64 +48.65%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 437.50 536.44 +22.61%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3,470 2,931 -15.53%
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[4-same] 30.56 26.05 -14.75%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2,740 3,083 +12.51%
benchmarks/test_collectors_benchmark.py::test_single_with_rb 7.5550 6.7023 -11.29%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 2,938 3,265 +11.14%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 3,229 2,872 -11.04%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 3,147 2,832 -10.01%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 36.09 39.29 +8.86%
benchmarks/test_objectives_benchmarks.py::test_td3_speed[True-backward] 259.84 279.79 +7.68%
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[reduce-overhead-None] 360.82 335.77 -6.94%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 2,306 2,148 -6.87%
benchmarks/test_envs_benchmark.py::test_simple 1.8553 1.7372 -6.37%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2,300 2,156 -6.24%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 3,318 3,115 -6.14%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2,156 2,027 -6.00%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 2,099 2,214 +5.44%
benchmarks/test_objectives_benchmarks.py::test_sac_speed[True-backward] 238.28 250.33 +5.05%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3,133 2,977 -4.99%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[100-img_shape2-large_img] 179.72 170.78 -4.97%
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[True-backward] 117.11 122.87 +4.92%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-True-False] 33,846 32,195 -4.88%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-256-256-64] 10.60 10.08 -4.88%
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 49.46 47.05 -4.87%
benchmarks/test_collectors_benchmark.py::test_sync_preempt 16.21 16.99 +4.83%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[100-img_shape1-atari] 270.70 283.13 +4.59%
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[reduce-overhead-None] 253.64 265.01 +4.48%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-True-True] 19,758 18,881 -4.44%
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[safetensors] 24,265 23,241 -4.22%
benchmarks/test_objectives_benchmarks.py::test_iql_speed[reduce-overhead-None] 112.75 117.35 +4.08%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[100-img_shape1-atari] 699.02 727.39 +4.06%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3,135 3,260 +3.97%
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[pickle] 12,637 12,166 -3.72%
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[True-None] 1,735 1,799 +3.66%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 944.64 910.42 -3.62%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 546.09 565.28 +3.51%
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[True-None] 321.27 310.19 -3.45%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 199.26 192.74 -3.27%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-False-False] 46,799 45,268 -3.27%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-224-224-4] 70.12 72.41 +3.26%
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 25.18 25.99 +3.23%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-256-256-4] 47.73 49.21 +3.10%
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[reduce-overhead-None] 282.67 274.00 -3.06%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[100-img_shape1-atari] 646.08 665.60 +3.02%
benchmarks/test_objectives_benchmarks.py::test_td3_speed[False-None] 125.56 121.83 -2.97%
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[reduce-overhead-None] 277.34 285.36 +2.89%
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[True-backward] 131.83 135.61 +2.86%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-True-False] 33,764 32,805 -2.84%
benchmarks/test_objectives_benchmarks.py::test_redq_speed[True-None] 223.13 216.80 -2.84%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-480-640-64] 6.4602 6.2782 -2.82%
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 23.70 23.05 -2.74%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[200-img_shape3-large_batch] 762.61 783.36 +2.72%
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[reduce-overhead-None] 713.45 732.77 +2.71%
benchmarks/test_replaybuffer_benchmark.py::TestPrioritizedReplayBufferBenchmark::test_sampler_sample_scale[1000000-cpu] 99.22 96.58 -2.66%
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[False-None] 208.16 213.65 +2.64%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 822.42 800.94 -2.61%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-False-True] 35,800 34,871 -2.59%
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[4-constant] 4,369 4,257 -2.58%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[100-img_shape1-atari] 5,255 5,389 +2.56%
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[True-None] 266.75 273.42 +2.50%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-False-True] 39,782 38,797 -2.48%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-224-224-1] 636.40 652.15 +2.48%
benchmarks/test_envs_benchmark.py::test_parallel 0.9538 0.9770 +2.44%
benchmarks/test_objectives_benchmarks.py::test_iql_speed[False-backward] 33.74 32.93 -2.40%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-True-True] 18,997 18,554 -2.33%
benchmarks/test_objectives_benchmarks.py::test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 664.47 649.00 -2.33%
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-cudnn-False-0-gru] 1.4027 1.3708 -2.28%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[200-img_shape3-large_batch] 331.81 339.34 +2.27%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1,090 1,066 -2.25%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-False-True] 31,604 30,896 -2.24%
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 24.42 24.97 +2.24%
benchmarks/test_objectives_benchmarks.py::test_sac_speed[True-None] 451.64 461.57 +2.20%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-480-640-1] 468.63 478.94 +2.20%
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[reduce-overhead-None] 1,845 1,885 +2.17%
benchmarks/test_objectives_benchmarks.py::test_td3_speed[False-backward] 93.15 91.13 -2.17%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-False-True] 34,070 33,332 -2.17%
benchmarks/test_objectives_benchmarks.py::test_sac_speed[False-None] 122.80 125.46 +2.17%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-False-False] 58,750 60,014 +2.15%
benchmarks/test_objectives_benchmarks.py::test_cql_speed[reduce-overhead-None] 81.58 83.34 +2.15%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-True-False] 28,591 27,996 -2.08%
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[False-backward] 245.41 250.47 +2.06%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-256-256-16] 12.08 12.33 +2.01%
benchmarks/test_objectives_benchmarks.py::test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 2,318 2,271 -2.01%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[200-img_shape3-large_batch] 310.43 316.58 +1.98%
benchmarks/test_replaybuffer_benchmark.py::TestPrioritizedReplayBufferBenchmark::test_sample_mixed_devices[1000000-memmap_cpu_storage_cpu... 83.00 81.42 -1.90%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-False-False] 51,620 50,653 -1.87%
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-scan-False-0-lstm] 1.9478 1.9838 +1.85%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 768.00 754.23 -1.79%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-480-640-4] 19.72 20.06 +1.76%
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 52.71 51.79 -1.76%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-False-True] 29,860 29,337 -1.75%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-True-False] 30,674 30,143 -1.73%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1,095 1,076 -1.71%
benchmarks/test_objectives_benchmarks.py::test_iql_speed[False-None] 51.70 50.82 -1.70%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-256-256-16] 42.84 43.56 +1.67%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-True-True-False] 44,014 43,280 -1.67%
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[False-None] 164.07 161.34 -1.66%
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 52.99 53.87 +1.66%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 563.69 554.46 -1.64%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-True-False-True] 42,830 43,517 +1.60%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-True-True] 20,437 20,112 -1.59%
benchmarks/test_envs_benchmark.py::test_transformed 0.9109 0.9254 +1.59%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-480-640-16] 4.9731 5.0506 +1.56%
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[False-backward] 62.93 61.96 -1.55%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-480-640-4] 145.03 147.27 +1.54%
benchmarks/test_objectives_benchmarks.py::test_iql_speed[True-None] 116.18 117.94 +1.52%
benchmarks/test_objectives_benchmarks.py::test_values[td0_return_estimate-False-False] 7,925 8,045 +1.51%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-256-256-64] 3.0309 3.0765 +1.50%
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[16-same] 24.41 24.04 -1.50%
benchmarks/test_storage_write_benchmark.py::TestCollectorIntegrationBenchmark::test_collector_with_rb[100-img_shape0-atari] 20.79 21.10 +1.49%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-224-224-64] 4.4724 4.5389 +1.49%
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[untyped_storage] 9.2528 9.3897 +1.48%
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-scan-True-0-gru] 4.2088 4.2694 +1.44%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[50-img_shape0-small] 3,511 3,561 +1.42%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-224-224-16] 17.92 18.18 +1.41%
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[False-backward] 130.94 132.73 +1.36%
benchmarks/test_objectives_benchmarks.py::test_sac_speed[reduce-overhead-None] 459.86 466.02 +1.34%
benchmarks/test_objectives_benchmarks.py::test_redq_speed[reduce-overhead-None] 222.68 219.77 -1.31%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 195.21 192.67 -1.30%
... ... ... Showing 120 of 216 comparisons, sorted by absolute change.

GPU

Compared 226 benchmarks. Regressions over 5%: 8. Improvements over 5%: 17.

Benchmark main ops PR ops Change
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 43.12 194.70 +351.57%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 814.27 2,934 +260.30%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 188.90 48.36 -74.40%
benchmarks/test_objectives_benchmarks.py::test_cql_speed[reduce-overhead-None] 60.47 88.29 +46.00%
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[reduce-overhead-None] 85.60 106.70 +24.65%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3,123 3,763 +20.48%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2,877 3,461 +20.30%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1,800 2,111 +17.26%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1,969 2,279 +15.78%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3,168 2,674 -15.59%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 465.60 535.81 +15.08%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 445.94 509.42 +14.24%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2,823 2,506 -11.25%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 2,924 2,617 -10.48%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 713.57 787.18 +10.31%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 484.97 522.99 +7.84%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 2,842 3,056 +7.53%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 822.11 874.81 +6.41%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 978.90 1,039 +6.16%
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[True-backward] 278.81 262.91 -5.70%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 955.78 1,010 +5.67%
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[True-backward] 909.68 958.23 +5.34%
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[numpy] 376,368 356,900 -5.17%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[100-img_shape2-large_img] 547.48 519.83 -5.05%
benchmarks/test_objectives_benchmarks.py::test_td3_speed[False-backward] 82.55 78.39 -5.04%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 768.45 805.64 +4.84%
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[False-backward] 285.81 272.43 -4.68%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-480-640-1] 479.78 457.62 -4.62%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2,679 2,556 -4.61%
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[False-backward] 150.21 143.51 -4.46%
benchmarks/test_objectives_benchmarks.py::test_sac_speed[True-backward] 321.26 335.47 +4.42%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[200-img_shape3-large_batch] 138.08 131.98 -4.42%
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[safetensors] 23,274 24,301 +4.41%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[100-img_shape2-large_img] 382.36 398.04 +4.10%
benchmarks/test_objectives_benchmarks.py::test_cql_speed[True-backward] 231.69 222.26 -4.07%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-False-True] 29,509 30,664 +3.91%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[100-img_shape1-atari] 684.05 657.62 -3.86%
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-scan-True-0-gru] 48.07 49.91 +3.83%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[100-img_shape2-large_img] 172.04 165.55 -3.77%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-True-False-True] 42,549 44,149 +3.76%
benchmarks/test_envs_benchmark.py::test_simple 1.2662 1.2202 -3.63%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-256-256-16] 41.33 42.82 +3.60%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[50-img_shape0-small] 3,561 3,440 -3.39%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-False-False] 50,797 52,495 +3.34%
benchmarks/test_collectors_benchmark.py::test_sync_preempt 10.15 10.48 +3.29%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 193.66 200.04 +3.29%
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[True-None] 736.03 760.03 +3.26%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 170.66 176.03 +3.15%
benchmarks/test_objectives_benchmarks.py::test_td3_speed[False-None] 114.34 110.75 -3.14%
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[True-None] 817.13 841.98 +3.04%
benchmarks/test_collectors_benchmark.py::test_sync 10.28 10.58 +2.97%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[100-img_shape1-atari] 4,306 4,178 -2.96%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-True-True] 19,087 19,647 +2.93%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-True-False] 28,062 28,877 +2.90%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[100-img_shape1-atari] 627.82 609.60 -2.90%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-224-224-1] 608.99 626.10 +2.81%
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[True-None] 434.79 422.60 -2.80%
benchmarks/test_objectives_benchmarks.py::test_cql_speed[False-backward] 39.66 38.55 -2.79%
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-scan-False-0-gru] 22.82 22.20 -2.73%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-False-False] 45,481 46,715 +2.71%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2,192 2,250 +2.65%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 163.95 168.09 +2.53%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-True-True] 18,401 18,865 +2.52%
benchmarks/test_objectives_benchmarks.py::test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 1,305 1,338 +2.52%
benchmarks/test_objectives_benchmarks.py::test_iql_speed[True-None] 516.31 503.35 -2.51%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[100-img_shape1-atari] 267.19 273.83 +2.49%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 165.59 169.64 +2.45%
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[False-backward] 237.48 231.68 -2.44%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-False-True] 39,695 38,751 -2.38%
benchmarks/test_objectives_benchmarks.py::test_values[vec_generalized_advantage_estimate-True-True] 491.96 480.40 -2.35%
benchmarks/test_objectives_benchmarks.py::test_td3_speed[True-None] 729.47 746.57 +2.34%
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[reduce-overhead-None] 1,942 1,897 -2.32%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-224-224-1] 278.43 284.86 +2.31%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-480-640-4] 142.81 146.10 +2.30%
benchmarks/test_replaybuffer_benchmark.py::TestPrioritizedReplayBufferBenchmark::test_sampler_sample_scale[1000000-cuda] 2,212 2,161 -2.30%
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[True-backward] 366.57 358.37 -2.24%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 169.35 173.06 +2.19%
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 24.02 23.50 -2.15%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-True-False] 29,998 30,633 +2.12%
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[untyped_storage] 8.8312 8.6529 -2.02%
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[4-constant] 4,929 4,830 -2.02%
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[False-backward] 128.39 125.81 -2.01%
benchmarks/test_objectives_benchmarks.py::test_iql_speed[False-backward] 68.93 67.56 -1.98%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[50-img_shape0-small] 883.92 901.40 +1.98%
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[False-None] 224.52 228.87 +1.93%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-False-True] 31,036 31,635 +1.93%
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-serial-no-buffers-False] 0.6921 0.7054 +1.92%
benchmarks/test_objectives_benchmarks.py::test_td3_speed[reduce-overhead-None] 45.76 44.89 -1.91%
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[False-None] 345.04 338.50 -1.90%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1,984 2,021 +1.90%
benchmarks/test_objectives_benchmarks.py::test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 1,339 1,364 +1.86%
benchmarks/test_envs_benchmark.py::test_transformed 0.7238 0.7105 -1.84%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-224-224-64] 4.3624 4.4422 +1.83%
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[True-backward] 342.23 348.47 +1.82%
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-scan-False-0-lstm] 21.63 21.24 -1.80%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-False-False] 66,141 67,324 +1.79%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 170.40 173.44 +1.79%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-False-False] 56,114 57,111 +1.78%
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 48.15 48.97 +1.71%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[200-img_shape3-large_batch] 297.02 292.00 -1.69%
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 22.31 21.94 -1.68%
benchmarks/test_storage_write_benchmark.py::TestCollectorIntegrationBenchmark::test_collector_without_rb[200-img_shape1-large_batch] 14.98 15.22 +1.65%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-256-256-4] 154.91 157.46 +1.64%
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-serial-buffers-False] 0.6013 0.6111 +1.63%
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-True] 0.2163 0.2129 -1.60%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-True-True] 21,405 21,068 -1.57%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-True-True] 21,682 22,018 +1.55%
benchmarks/test_replaybuffer_benchmark.py::TestPrioritizedReplayBufferBenchmark::test_sample_mixed_devices[1000000-memmap_cpu_storage_cud... 976.89 992.00 +1.55%
benchmarks/test_replaybuffer_benchmark.py::TestPrioritizedReplayBufferBenchmark::test_sample_mixed_devices[1000000-cuda_storage_cuda_samp... 1,526 1,503 -1.54%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-True-True-False] 43,608 44,277 +1.53%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-False-False] 50,571 51,346 +1.53%
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[reduce-overhead-None] 864.84 878.04 +1.53%
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[16-constant] 4,664 4,732 +1.46%
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-False] 0.2241 0.2273 +1.43%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 165.04 167.36 +1.40%
benchmarks/test_objectives_benchmarks.py::test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1,271 1,253 -1.37%
benchmarks/test_collectors_benchmark.py::test_single_pixels 6.3395 6.2525 -1.37%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-224-224-64] 12.30 12.47 +1.36%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-False-True] 39,503 40,040 +1.36%
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-serial-buffers-True] 0.5232 0.5304 +1.36%
... ... ... Showing 120 of 226 comparisons, sorted by absolute change.

vmoens added 3 commits June 22, 2026 09:18
@vmoens
Update
53b0f0a 
[ghstack-poisoned]
@vmoens
Update
1f7039b 
[ghstack-poisoned]
@vmoens
Update
d3b7018 
[ghstack-poisoned]
This was referenced Jun 24, 2026
Merged
Closed
theap06
theap06 suggested changes Jun 27, 2026
View reviewed changes
Comment thread torchrl/modules/inference_server/_server.py
Comment thread torchrl/modules/inference_server/_server.py Outdated
@vmoens
Update
a9dcd31 
[ghstack-poisoned]
This was referenced Jul 6, 2026
Open
Merged
Merged
@vmoens vmoens marked this pull request as ready for review July 6, 2026 08:59
vmoens added 2 commits July 6, 2026 10:04
@vmoens
Update
4634b95 
[ghstack-poisoned]
@vmoens
Update
a5e8797 
[ghstack-poisoned]
This was referenced Jul 6, 2026
Merged
Merged
vmoens added 3 commits July 6, 2026 12:56
@vmoens
Update
9fafa63 
[ghstack-poisoned]
@vmoens
Update
a4aacdf 
[ghstack-poisoned]
@vmoens
Update
e8f2856 
[ghstack-poisoned]
vmoens added 3 commits July 6, 2026 13:37
@vmoens
Update
28e2341 
[ghstack-poisoned]
@vmoens
Update
c3dd450 
[ghstack-poisoned]
@vmoens
Update
6f81b6b 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Jul 6, 2026
@vmoens
[Feature] Add process inference server control plane
938d0ab 
ghstack-source-id: 230f559
Pull-Request: #3896
# Conflicts:
#	torchrl/modules/inference_server/_server.py
@vmoens vmoens merged commit 6f81b6b into gh/vmoens/291/base Jul 6, 2026
101 of 104 checks passed
@vmoens vmoens deleted the gh/vmoens/291/head branch July 6, 2026 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@theap06 theap06 theap06 requested changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Feature New feature Integrations/torch_geometric Integrations Modules

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@vmoens @theap06