[Feature] Propagate interaction types through inference-server clients#3941
Merged
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3941
Note: Links to docs will display an error until the docs builds have been completed. This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
Bot
added
the
CLA Signed
github-actions
Bot
added
Modules
Integrations/torch_geometric
Integrations
Feature
This was referenced Jul 6, 2026
Contributor
Benchmark Results: PR
|
| Benchmark | main ops | PR ops | Change |
|---|---|---|---|
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[False-None] |
59.38 | 87.28 | +46.98% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] |
2,616 | 3,542 | +35.40% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] |
3,168 | 2,633 | -16.87% |
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[4-same] |
28.29 | 24.14 | -14.66% |
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[16-same] |
19.71 | 22.33 | +13.29% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] |
2,950 | 3,341 | +13.25% |
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[True-backward] |
875.34 | 979.27 | +11.87% |
benchmarks/test_collectors_benchmark.py::test_single_with_rb |
7.3526 | 6.4890 | -11.74% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] |
809.31 | 717.71 | -11.32% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] |
2,080 | 2,309 | +11.01% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] |
760.81 | 685.92 | -9.84% |
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[True-backward] |
349.01 | 383.12 | +9.77% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-480-640-64] |
6.4715 | 7.0811 | +9.42% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] |
3,046 | 2,768 | -9.11% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[100-img_shape1-atari] |
4,787 | 5,217 | +8.99% |
benchmarks/test_objectives_benchmarks.py::test_sac_speed[True-backward] |
225.21 | 244.26 | +8.46% |
benchmarks/test_objectives_benchmarks.py::test_values[generalized_advantage_estimate-True-True] |
98.18 | 90.91 | -7.41% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] |
3,076 | 2,861 | -6.97% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] |
546.38 | 509.01 | -6.84% |
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[True-backward] |
105.58 | 112.77 | +6.81% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] |
426.30 | 454.83 | +6.69% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-256-256-1] |
186.46 | 198.16 | +6.28% |
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[safetensors] |
22,532 | 23,941 | +6.25% |
benchmarks/test_envs_benchmark.py::test_simple |
1.8089 | 1.6994 | -6.06% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] |
2,154 | 2,026 | -5.91% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] |
2,755 | 2,908 | +5.56% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[50-img_shape0-small] |
3,376 | 3,559 | +5.42% |
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[numpy] |
334,247 | 352,116 | +5.35% |
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[torch.save] |
7,206 | 6,830 | -5.22% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-False-False] |
65,575 | 62,338 | -4.94% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[100-img_shape2-large_img] |
578.54 | 551.05 | -4.75% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] |
3,170 | 3,318 | +4.66% |
benchmarks/test_objectives_benchmarks.py::test_gae_speed[vec_generalized_advantage_estimate-True-1-512] |
623.47 | 651.09 | +4.43% |
benchmarks/test_objectives_benchmarks.py::test_redq_speed[True-None] |
208.48 | 217.51 | +4.33% |
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-scan-False-0-gru] |
2.9002 | 3.0204 | +4.14% |
benchmarks/test_objectives_benchmarks.py::test_gae_speed[vec_generalized_advantage_estimate-True-32-512] |
27.97 | 29.12 | +4.11% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-256-256-1] |
494.09 | 514.10 | +4.05% |
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-scan-False-0-lstm] |
1.9224 | 1.9989 | +3.98% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[100-img_shape1-atari] |
277.48 | 266.63 | -3.91% |
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-cudnn-False-0-lstm] |
0.8705 | 0.8367 | -3.88% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-False-True] |
34,485 | 33,181 | -3.78% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-224-224-1] |
622.10 | 645.49 | +3.76% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-480-640-1] |
463.93 | 480.94 | +3.67% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] |
1,934 | 1,865 | -3.59% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] |
898.88 | 867.03 | -3.54% |
benchmarks/test_objectives_benchmarks.py::test_redq_speed[reduce-overhead-None] |
215.06 | 222.02 | +3.24% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[100-img_shape2-large_img] |
396.16 | 408.84 | +3.20% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-True-True-True] |
23,793 | 23,041 | -3.16% |
benchmarks/test_replaybuffer_benchmark.py::TestPrioritizedReplayBufferBenchmark::test_sample_mixed_devices[1000000-memmap_cpu_storage_cpu... |
80.60 | 83.05 | +3.04% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] |
2,013 | 2,074 | +3.04% |
benchmarks/test_objectives_benchmarks.py::test_redq_speed[False-backward] |
55.37 | 57.05 | +3.03% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-False-True] |
39,073 | 37,901 | -3.00% |
benchmarks/test_envs_benchmark.py::test_transformed |
0.9082 | 0.8813 | -2.96% |
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[untyped_storage] |
8.3244 | 8.0866 | -2.86% |
benchmarks/test_objectives_benchmarks.py::test_sac_speed[False-None] |
120.20 | 116.82 | -2.81% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[200-img_shape3-large_batch] |
773.05 | 751.60 | -2.77% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[100-img_shape2-large_img] |
418.46 | 429.36 | +2.60% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-True-False] |
32,378 | 31,536 | -2.60% |
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[4-constant] |
4,250 | 4,142 | -2.53% |
benchmarks/test_objectives_benchmarks.py::test_values[td_lambda_return_estimate-True-False] |
24.56 | 25.18 | +2.52% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-224-224-16] |
17.92 | 18.37 | +2.52% |
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[reduce-overhead-None] |
1,752 | 1,797 | +2.52% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] |
2,772 | 2,840 | +2.46% |
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[True-None] |
269.22 | 262.66 | -2.44% |
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[True-backward] |
122.85 | 125.81 | +2.41% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[100-img_shape1-atari] |
636.52 | 651.71 | +2.39% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] |
38.47 | 39.37 | +2.34% |
benchmarks/test_objectives_benchmarks.py::test_sac_speed[True-None] |
439.86 | 450.14 | +2.34% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[200-img_shape3-large_batch] |
307.51 | 314.61 | +2.31% |
benchmarks/test_objectives_benchmarks.py::test_gae_speed[vec_generalized_advantage_estimate-False-1-512] |
2,209 | 2,260 | +2.31% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-False-False] |
54,488 | 53,312 | -2.16% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-224-224-4] |
71.55 | 73.06 | +2.11% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-False-False] |
50,890 | 49,831 | -2.08% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-False-True] |
30,104 | 30,724 | +2.06% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] |
1,067 | 1,089 | +2.05% |
benchmarks/test_storage_write_benchmark.py::TestCollectorIntegrationBenchmark::test_collector_without_rb[100-img_shape0-atari] |
29.28 | 29.88 | +2.04% |
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[False-backward] |
241.73 | 236.88 | -2.01% |
benchmarks/test_objectives_benchmarks.py::test_sac_speed[False-backward] |
87.87 | 86.15 | -1.95% |
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-scan-True-0-lstm] |
3.0777 | 3.1374 | +1.94% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-224-224-1] |
268.39 | 273.57 | +1.93% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-True-True] |
19,720 | 19,339 | -1.93% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-256-256-4] |
48.27 | 49.17 | +1.87% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-True-False] |
34,709 | 34,064 | -1.86% |
benchmarks/test_objectives_benchmarks.py::test_values[td1_return_estimate-False-False] |
36.52 | 37.19 | +1.84% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-224-224-64] |
4.5170 | 4.5959 | +1.75% |
benchmarks/test_objectives_benchmarks.py::test_cql_speed[True-backward] |
58.20 | 59.18 | +1.68% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[100-img_shape2-large_img] |
175.50 | 172.57 | -1.67% |
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[False-None] |
342.27 | 336.61 | -1.65% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-True-False] |
38,240 | 37,612 | -1.64% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] |
500.60 | 508.73 | +1.63% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-True-True] |
22,334 | 21,975 | -1.61% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[100-img_shape1-atari] |
698.86 | 687.82 | -1.58% |
benchmarks/test_replaybuffer_benchmark.py::TestPrioritizedReplayBufferBenchmark::test_sampler_sample_scale[10000000-cpu] |
52.15 | 52.95 | +1.55% |
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[True-None] |
253.85 | 257.74 | +1.53% |
benchmarks/test_objectives_benchmarks.py::test_redq_speed[False-None] |
94.49 | 95.89 | +1.48% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[50-img_shape0-small] |
7,028 | 7,131 | +1.47% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] |
22.83 | 23.17 | +1.46% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-256-256-4] |
158.62 | 160.86 | +1.41% |
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[False-None] |
178.92 | 176.44 | -1.39% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-480-640-1] |
80.15 | 81.26 | +1.38% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-False-False] |
49,476 | 50,159 | +1.38% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-False-True] |
38,081 | 37,557 | -1.38% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-False-False] |
56,958 | 57,739 | +1.37% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-True-False] |
29,375 | 28,974 | -1.37% |
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[False-backward] |
82.75 | 83.88 | +1.36% |
benchmarks/test_objectives_benchmarks.py::test_gae_speed[generalized_advantage_estimate-False-1-512] |
110.35 | 111.84 | +1.35% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-True-False] |
31,492 | 31,068 | -1.35% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[50-img_shape0-small] |
859.38 | 870.93 | +1.34% |
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[False-None] |
213.01 | 210.16 | -1.34% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] |
47.06 | 47.68 | +1.33% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-256-256-16] |
12.17 | 12.33 | +1.32% |
benchmarks/test_objectives_benchmarks.py::test_cql_speed[False-backward] |
27.75 | 27.38 | -1.32% |
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[False-backward] |
60.60 | 61.38 | +1.30% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-False-True] |
32,706 | 33,126 | +1.28% |
benchmarks/test_objectives_benchmarks.py::test_values[vec_td_lambda_return_estimate-True-False] |
55.52 | 54.81 | -1.28% |
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[False-None] |
158.90 | 160.83 | +1.22% |
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-cudnn-True-0-gru] |
1.4602 | 1.4425 | -1.21% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] |
1,063 | 1,051 | -1.20% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-480-640-4] |
19.94 | 20.17 | +1.19% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-True-True] |
20,277 | 20,515 | +1.17% |
| ... | ... | ... | Showing 120 of 216 comparisons, sorted by absolute change. |
GPU
Compared 226 benchmarks. Regressions over 5%: 7. Improvements over 5%: 18.
| Benchmark | main ops | PR ops | Change |
|---|---|---|---|
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] |
44.44 | 196.24 | +341.54% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] |
1,952 | 517.34 | -73.49% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] |
185.22 | 50.91 | -72.51% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] |
743.15 | 999.47 | +34.49% |
benchmarks/test_objectives_benchmarks.py::test_iql_speed[reduce-overhead-None] |
76.48 | 102.74 | +34.34% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] |
2,645 | 3,542 | +33.91% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] |
740.85 | 966.42 | +30.45% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] |
2,634 | 3,274 | +24.29% |
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[reduce-overhead-None] |
85.63 | 105.77 | +23.51% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] |
3,579 | 2,751 | -23.12% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] |
695.14 | 855.53 | +23.07% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] |
2,536 | 3,110 | +22.64% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[100-img_shape2-large_img] |
404.61 | 460.41 | +13.79% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] |
2,946 | 3,344 | +13.51% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] |
2,094 | 1,818 | -13.16% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] |
3,162 | 3,576 | +13.07% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] |
2,146 | 1,975 | -7.94% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] |
708.50 | 762.02 | +7.55% |
benchmarks/test_objectives_benchmarks.py::test_td3_speed[True-backward] |
368.75 | 395.21 | +7.18% |
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[True-backward] |
447.42 | 478.41 | +6.93% |
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[True-backward] |
357.16 | 335.48 | -6.07% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[100-img_shape1-atari] |
660.28 | 697.70 | +5.67% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] |
746.48 | 786.16 | +5.32% |
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[True-backward] |
363.68 | 344.99 | -5.14% |
benchmarks/test_objectives_benchmarks.py::test_sac_speed[True-backward] |
319.56 | 335.84 | +5.09% |
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[False-backward] |
277.91 | 264.99 | -4.65% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] |
491.65 | 514.20 | +4.59% |
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[4-constant] |
4,815 | 4,598 | -4.51% |
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[safetensors] |
23,029 | 24,068 | +4.51% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-False-False] |
45,607 | 43,730 | -4.11% |
benchmarks/test_storage_write_benchmark.py::TestCollectorIntegrationBenchmark::test_collector_with_rb_cuda[200-img_shape1-large_batch] |
6.6853 | 6.4141 | -4.06% |
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[reduce-overhead-None] |
1,902 | 1,825 | -4.05% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] |
2,159 | 2,243 | +3.89% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[100-img_shape1-atari] |
658.10 | 632.75 | -3.85% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] |
499.33 | 518.37 | +3.81% |
benchmarks/test_objectives_benchmarks.py::test_gae_speed[vec_generalized_advantage_estimate-False-32-512] |
1,287 | 1,336 | +3.74% |
benchmarks/test_replaybuffer_benchmark.py::TestPrioritizedReplayBufferBenchmark::test_sample_mixed_devices[1000000-memmap_cpu_storage_cud... |
997.95 | 961.92 | -3.61% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[50-img_shape0-small] |
3,565 | 3,439 | -3.53% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] |
2,877 | 2,777 | -3.48% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[50-img_shape0-small] |
5,848 | 6,049 | +3.45% |
benchmarks/test_envs_benchmark.py::test_simple |
1.2377 | 1.1958 | -3.39% |
benchmarks/test_storage_write_benchmark.py::TestCollectorIntegrationBenchmark::test_collector_without_rb_cuda[200-img_shape1-large_batch] |
8.7162 | 8.4257 | -3.33% |
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[True-None] |
719.16 | 742.78 | +3.28% |
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[True-backward] |
902.88 | 873.79 | -3.22% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-224-224-64] |
12.13 | 12.51 | +3.17% |
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[False-None] |
392.61 | 380.21 | -3.16% |
benchmarks/test_objectives_benchmarks.py::test_values[td_lambda_return_estimate-True-False] |
12.05 | 12.43 | +3.15% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-True-True] |
20,954 | 20,304 | -3.10% |
benchmarks/test_envs_benchmark.py::test_parallel |
0.5279 | 0.5441 | +3.08% |
benchmarks/test_objectives_benchmarks.py::test_gae_speed[vec_generalized_advantage_estimate-False-1-512] |
1,335 | 1,376 | +3.07% |
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-scan-False-0-gru] |
22.46 | 21.77 | -3.06% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[50-img_shape0-small] |
4,308 | 4,438 | +3.03% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] |
48.59 | 47.14 | -2.99% |
benchmarks/test_objectives_benchmarks.py::test_values[vec_generalized_advantage_estimate-True-True] |
507.90 | 493.25 | -2.89% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-True-True] |
18,228 | 17,728 | -2.74% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-480-640-4] |
144.07 | 140.12 | -2.74% |
benchmarks/test_replaybuffer_benchmark.py::TestPrioritizedReplayBufferBenchmark::test_sample_mixed_devices[1000000-cuda_storage_cuda_samp... |
1,493 | 1,452 | -2.73% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[200-img_shape3-large_batch] |
139.62 | 143.35 | +2.67% |
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[False-backward] |
67.99 | 69.79 | +2.65% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-224-224-4] |
184.89 | 180.02 | -2.64% |
benchmarks/test_objectives_benchmarks.py::test_values[td1_return_estimate-False-False] |
19.95 | 20.47 | +2.61% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-256-256-64] |
10.58 | 10.86 | +2.58% |
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[False-None] |
343.25 | 334.58 | -2.53% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] |
2,765 | 2,697 | -2.47% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-480-640-4] |
20.08 | 19.59 | -2.44% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] |
52.06 | 50.79 | -2.43% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[100-img_shape2-large_img] |
180.25 | 175.91 | -2.40% |
benchmarks/test_envs_benchmark.py::test_transformed |
0.7100 | 0.6934 | -2.33% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[50-img_shape0-small] |
879.16 | 859.09 | -2.28% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-True-True-False] |
42,822 | 41,853 | -2.26% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-False-False] |
50,138 | 49,013 | -2.24% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] |
21.70 | 21.23 | -2.20% |
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[False-backward] |
130.77 | 128.00 | -2.11% |
benchmarks/test_storage_write_benchmark.py::TestCollectorIntegrationBenchmark::test_collector_with_rb_cuda[100-img_shape0-atari] |
13.32 | 13.04 | -2.10% |
benchmarks/test_storage_write_benchmark.py::TestCollectorIntegrationBenchmark::test_collector_without_rb[100-img_shape0-atari] |
30.48 | 29.86 | -2.06% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-False-True] |
36,981 | 37,733 | +2.03% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[200-img_shape3-large_batch] |
727.86 | 713.15 | -2.02% |
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-scan-True-0-gru] |
49.79 | 48.79 | -2.02% |
benchmarks/test_collectors_benchmark.py::test_sync |
10.22 | 10.43 | +2.00% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] |
23.27 | 22.82 | -1.93% |
benchmarks/test_replaybuffer_benchmark.py::TestPrioritizedReplayBufferBenchmark::test_sampler_sample_scale[1000000-cuda] |
2,234 | 2,191 | -1.92% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-True-True] |
19,623 | 19,996 | +1.90% |
benchmarks/test_envs_benchmark.py::test_serial |
0.4215 | 0.4295 | +1.90% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-True-False-False] |
74,688 | 76,089 | +1.88% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-False-True] |
28,589 | 28,053 | -1.87% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-480-640-1] |
473.64 | 464.81 | -1.87% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-True-False] |
34,753 | 34,113 | -1.84% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] |
497.39 | 488.49 | -1.79% |
benchmarks/test_objectives_benchmarks.py::test_td3_speed[reduce-overhead-None] |
44.27 | 45.05 | +1.75% |
benchmarks/test_replaybuffer_benchmark.py::TestPrioritizedReplayBufferBenchmark::test_sampler_sample_scale[10000000-cuda] |
1,064 | 1,046 | -1.71% |
benchmarks/test_objectives_benchmarks.py::test_values[td0_return_estimate-False-False] |
11,674 | 11,479 | -1.67% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[100-img_shape2-large_img] |
555.58 | 564.74 | +1.65% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-False-True] |
37,565 | 38,183 | +1.64% |
benchmarks/test_objectives_benchmarks.py::test_sac_speed[False-backward] |
75.85 | 77.09 | +1.63% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-480-640-64] |
1.2522 | 1.2319 | -1.63% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] |
167.66 | 170.36 | +1.61% |
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[pickle] |
11,993 | 11,800 | -1.61% |
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[16-constant] |
4,623 | 4,695 | +1.56% |
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[True-None] |
763.84 | 752.01 | -1.55% |
benchmarks/test_collectors_benchmark.py::test_async_pixels |
10.53 | 10.70 | +1.54% |
benchmarks/test_collectors_benchmark.py::test_sync_pixels |
10.15 | 10.30 | +1.54% |
benchmarks/test_collectors_benchmark.py::test_sync_preempt |
10.15 | 10.30 | +1.52% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-False-True] |
33,189 | 32,687 | -1.51% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-True-False-True] |
42,223 | 41,601 | -1.47% |
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[False-backward] |
233.71 | 230.29 | -1.46% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-False-True] |
30,604 | 30,166 | -1.43% |
benchmarks/test_objectives_benchmarks.py::test_cql_speed[reduce-overhead-None] |
87.11 | 88.32 | +1.39% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-224-224-4] |
69.92 | 70.88 | +1.38% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-256-256-4] |
158.39 | 156.20 | -1.38% |
benchmarks/test_objectives_benchmarks.py::test_iql_speed[True-backward] |
236.37 | 239.63 | +1.38% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-480-640-64] |
7.0593 | 7.1550 | +1.36% |
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[False-None] |
96.64 | 97.92 | +1.32% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-False-False] |
55,320 | 54,600 | -1.30% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-True-False] |
27,548 | 27,198 | -1.27% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-224-224-16] |
47.94 | 48.55 | +1.27% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-480-640-16] |
5.0215 | 4.9578 | -1.27% |
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-False] |
0.2222 | 0.2249 | +1.22% |
benchmarks/test_objectives_benchmarks.py::test_gae_speed[generalized_advantage_estimate-False-1-512] |
47.81 | 48.39 | +1.21% |
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[untyped_storage] |
8.8055 | 8.7057 | -1.13% |
benchmarks/test_objectives_benchmarks.py::test_values[generalized_advantage_estimate-True-True] |
47.61 | 48.15 | +1.13% |
| ... | ... | ... | Showing 120 of 226 comparisons, sorted by absolute change. |
vmoens
added a commit
that referenced
this pull request
Jul 6, 2026
PolicyClientModule(propagate_interaction_type=True) attaches the caller's active tensordict interaction type to each request as a compact int8 code; the server decodes it and runs the batched forward under the corresponding set_interaction_type context, so remote policies honor the client-side exploration context (e.g. RANDOM vs DETERMINISTIC). Mixed-type batches are rejected with a clear error. The server-side forward, output-device move, and version stamping now all run under the model lock so responses cannot be stamped with a version that does not match the weights used. ghstack-source-id: 5df2c8d Pull-Request: #3941
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack from ghstack (oldest at bottom):
PolicyClientModule(propagate_interaction_type=True) attaches the caller's
active tensordict interaction type to each request as a compact int8 code;
the server decodes it and runs the batched forward under the corresponding
set_interaction_type context, so remote policies honor the client-side
exploration context (e.g. RANDOM vs DETERMINISTIC). Mixed-type batches are
rejected with a clear error. The server-side forward, output-device move,
and version stamping now all run under the model lock so responses cannot
be stamped with a version that does not match the weights used.