Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] non-functional SAC loss #2393

Open
wants to merge 2 commits into
base: gh/vmoens/19/base
Choose a base branch
from
Open

[Feature] non-functional SAC loss #2393

wants to merge 2 commits into from

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Aug 13, 2024 •
edited
Loading

@vmoens
Update
3b10120 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Aug 13, 2024
Open
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Aug 13, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2393

Note: Links to docs will display an error until the docs builds have been completed.

❌ 12 New Failures, 4 Unrelated Failures

As of commit 569b6ba with merge base 25e8bd2 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vmoens added a commit that referenced this pull request Aug 13, 2024
@vmoens
[Feature] non-functional SAC loss
68ae620 
ghstack-source-id: f904e4739f31162498553495238a906262aa4b48
Pull Request resolved: #2393
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 13, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 13, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1084s 0.1075s 9.3012 Ops/s 9.3022 Ops/s $\color{#d91a1a}-0.01\%$
test_sync 95.1135ms 92.7047ms 10.7869 Ops/s 10.9403 Ops/s $\color{#d91a1a}-1.40\%$
test_async 0.2524s 89.9285ms 11.1199 Ops/s 11.1305 Ops/s $\color{#d91a1a}-0.09\%$
test_single_pixels 0.1185s 0.1181s 8.4660 Ops/s 8.4534 Ops/s $\color{#35bf28}+0.15\%$
test_sync_pixels 77.6523ms 76.1890ms 13.1253 Ops/s 13.2074 Ops/s $\color{#d91a1a}-0.62\%$
test_async_pixels 0.1321s 69.8217ms 14.3222 Ops/s 13.8976 Ops/s $\color{#35bf28}+3.05\%$
test_simple 0.7893s 0.7759s 1.2889 Ops/s 1.2494 Ops/s $\color{#35bf28}+3.16\%$
test_transformed 1.0938s 1.0302s 0.9707 Ops/s 1.0005 Ops/s $\color{#d91a1a}-2.99\%$
test_serial 2.2646s 2.2111s 0.4523 Ops/s 0.4538 Ops/s $\color{#d91a1a}-0.34\%$
test_parallel 1.9659s 1.8990s 0.5266 Ops/s 0.5355 Ops/s $\color{#d91a1a}-1.66\%$
test_step_mdp_speed[True-True-True-True-True] 0.1527ms 37.0060μs 27.0226 KOps/s 25.5989 KOps/s $\textbf{\color{#35bf28}+5.56\%}$
test_step_mdp_speed[True-True-True-True-False] 48.4510μs 21.3229μs 46.8980 KOps/s 46.1418 KOps/s $\color{#35bf28}+1.64\%$
test_step_mdp_speed[True-True-True-False-True] 47.4710μs 20.8448μs 47.9736 KOps/s 46.6856 KOps/s $\color{#35bf28}+2.76\%$
test_step_mdp_speed[True-True-True-False-False] 41.0310μs 12.1098μs 82.5780 KOps/s 83.2210 KOps/s $\color{#d91a1a}-0.77\%$
test_step_mdp_speed[True-True-False-True-True] 70.2720μs 39.6508μs 25.2202 KOps/s 24.3569 KOps/s $\color{#35bf28}+3.54\%$
test_step_mdp_speed[True-True-False-True-False] 46.7010μs 23.8942μs 41.8512 KOps/s 41.6819 KOps/s $\color{#35bf28}+0.41\%$
test_step_mdp_speed[True-True-False-False-True] 42.3510μs 23.4370μs 42.6675 KOps/s 42.4707 KOps/s $\color{#35bf28}+0.46\%$
test_step_mdp_speed[True-True-False-False-False] 35.3510μs 14.5391μs 68.7802 KOps/s 68.6378 KOps/s $\color{#35bf28}+0.21\%$
test_step_mdp_speed[True-False-True-True-True] 69.2420μs 42.2605μs 23.6628 KOps/s 23.0610 KOps/s $\color{#35bf28}+2.61\%$
test_step_mdp_speed[True-False-True-True-False] 54.2710μs 26.1548μs 38.2339 KOps/s 37.9405 KOps/s $\color{#35bf28}+0.77\%$
test_step_mdp_speed[True-False-True-False-True] 42.9320μs 23.4153μs 42.7072 KOps/s 41.7859 KOps/s $\color{#35bf28}+2.20\%$
test_step_mdp_speed[True-False-True-False-False] 34.6800μs 14.4970μs 68.9796 KOps/s 69.0353 KOps/s $\color{#d91a1a}-0.08\%$
test_step_mdp_speed[True-False-False-True-True] 69.0320μs 43.9005μs 22.7788 KOps/s 21.9138 KOps/s $\color{#35bf28}+3.95\%$
test_step_mdp_speed[True-False-False-True-False] 48.4210μs 27.9914μs 35.7252 KOps/s 34.5264 KOps/s $\color{#35bf28}+3.47\%$
test_step_mdp_speed[True-False-False-False-True] 51.5720μs 25.3070μs 39.5147 KOps/s 38.6145 KOps/s $\color{#35bf28}+2.33\%$
test_step_mdp_speed[True-False-False-False-False] 38.7410μs 16.6031μs 60.2298 KOps/s 59.6929 KOps/s $\color{#35bf28}+0.90\%$
test_step_mdp_speed[False-True-True-True-True] 61.7820μs 42.1615μs 23.7183 KOps/s 23.2260 KOps/s $\color{#35bf28}+2.12\%$
test_step_mdp_speed[False-True-True-True-False] 53.0220μs 26.0464μs 38.3931 KOps/s 38.1777 KOps/s $\color{#35bf28}+0.56\%$
test_step_mdp_speed[False-True-True-False-True] 53.5910μs 27.8920μs 35.8526 KOps/s 34.3961 KOps/s $\color{#35bf28}+4.23\%$
test_step_mdp_speed[False-True-True-False-False] 39.6810μs 16.2951μs 61.3683 KOps/s 60.3741 KOps/s $\color{#35bf28}+1.65\%$
test_step_mdp_speed[False-True-False-True-True] 74.3020μs 43.6175μs 22.9266 KOps/s 22.0123 KOps/s $\color{#35bf28}+4.15\%$
test_step_mdp_speed[False-True-False-True-False] 55.0510μs 27.8037μs 35.9664 KOps/s 34.8549 KOps/s $\color{#35bf28}+3.19\%$
test_step_mdp_speed[False-True-False-False-True] 52.5610μs 29.4364μs 33.9716 KOps/s 31.9280 KOps/s $\textbf{\color{#35bf28}+6.40\%}$
test_step_mdp_speed[False-True-False-False-False] 40.9210μs 18.3729μs 54.4280 KOps/s 52.2390 KOps/s $\color{#35bf28}+4.19\%$
test_step_mdp_speed[False-False-True-True-True] 3.7721ms 46.6828μs 21.4212 KOps/s 20.6056 KOps/s $\color{#35bf28}+3.96\%$
test_step_mdp_speed[False-False-True-True-False] 56.7220μs 30.6435μs 32.6333 KOps/s 32.1939 KOps/s $\color{#35bf28}+1.37\%$
test_step_mdp_speed[False-False-True-False-True] 52.3210μs 30.1513μs 33.1661 KOps/s 32.4087 KOps/s $\color{#35bf28}+2.34\%$
test_step_mdp_speed[False-False-True-False-False] 35.2410μs 18.5469μs 53.9175 KOps/s 52.5937 KOps/s $\color{#35bf28}+2.52\%$
test_step_mdp_speed[False-False-False-True-True] 66.3810μs 47.7469μs 20.9438 KOps/s 20.1850 KOps/s $\color{#35bf28}+3.76\%$
test_step_mdp_speed[False-False-False-True-False] 55.2410μs 32.7567μs 30.5281 KOps/s 30.1699 KOps/s $\color{#35bf28}+1.19\%$
test_step_mdp_speed[False-False-False-False-True] 55.6210μs 31.9113μs 31.3369 KOps/s 30.6028 KOps/s $\color{#35bf28}+2.40\%$
test_step_mdp_speed[False-False-False-False-False] 48.9510μs 21.1390μs 47.3059 KOps/s 47.2394 KOps/s $\color{#35bf28}+0.14\%$
test_values[generalized_advantage_estimate-True-True] 25.3765ms 24.7567ms 40.3931 Ops/s 40.4483 Ops/s $\color{#d91a1a}-0.14\%$
test_values[vec_generalized_advantage_estimate-True-True] 95.7513ms 2.8229ms 354.2416 Ops/s 351.6892 Ops/s $\color{#35bf28}+0.73\%$
test_values[td0_return_estimate-False-False] 92.2030μs 67.5230μs 14.8098 KOps/s 14.8990 KOps/s $\color{#d91a1a}-0.60\%$
test_values[td1_return_estimate-False-False] 55.9609ms 55.3849ms 18.0555 Ops/s 17.9070 Ops/s $\color{#35bf28}+0.83\%$
test_values[vec_td1_return_estimate-False-False] 1.2813ms 1.0968ms 911.7627 Ops/s 914.1290 Ops/s $\color{#d91a1a}-0.26\%$
test_values[td_lambda_return_estimate-True-False] 92.5529ms 89.4042ms 11.1852 Ops/s 11.2468 Ops/s $\color{#d91a1a}-0.55\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.2791ms 1.0942ms 913.9161 Ops/s 914.6490 Ops/s $\color{#d91a1a}-0.08\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 26.5730ms 26.2169ms 38.1434 Ops/s 40.4800 Ops/s $\textbf{\color{#d91a1a}-5.77\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9969ms 0.7636ms 1.3097 KOps/s 1.3713 KOps/s $\color{#d91a1a}-4.49\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7859ms 0.6840ms 1.4621 KOps/s 1.4753 KOps/s $\color{#d91a1a}-0.90\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5212ms 1.4804ms 675.5142 Ops/s 677.3582 Ops/s $\color{#d91a1a}-0.27\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8167ms 0.7262ms 1.3770 KOps/s 1.3919 KOps/s $\color{#d91a1a}-1.07\%$
test_dqn_speed 6.9783ms 1.3885ms 720.2032 Ops/s 728.4295 Ops/s $\color{#d91a1a}-1.13\%$
test_ddpg_speed 3.0019ms 2.7526ms 363.2916 Ops/s 359.3291 Ops/s $\color{#35bf28}+1.10\%$
test_sac_speed 99.9864ms 8.7150ms 114.7441 Ops/s 125.2051 Ops/s $\textbf{\color{#d91a1a}-8.36\%}$
test_redq_speed 12.2153ms 10.2876ms 97.2041 Ops/s 97.9761 Ops/s $\color{#d91a1a}-0.79\%$
test_redq_deprec_speed 12.2397ms 10.8037ms 92.5605 Ops/s 92.0631 Ops/s $\color{#35bf28}+0.54\%$
test_td3_speed 7.9699ms 7.8667ms 127.1187 Ops/s 126.3700 Ops/s $\color{#35bf28}+0.59\%$
test_cql_speed 25.2167ms 24.6547ms 40.5602 Ops/s 40.0035 Ops/s $\color{#35bf28}+1.39\%$
test_a2c_speed 5.8823ms 5.5603ms 179.8458 Ops/s 183.1772 Ops/s $\color{#d91a1a}-1.82\%$
test_ppo_speed 6.0920ms 5.8777ms 170.1351 Ops/s 171.7535 Ops/s $\color{#d91a1a}-0.94\%$
test_reinforce_speed 5.2514ms 4.4376ms 225.3468 Ops/s 225.2810 Ops/s $\color{#35bf28}+0.03\%$
test_iql_speed 19.9571ms 19.4218ms 51.4885 Ops/s 52.8610 Ops/s $\color{#d91a1a}-2.60\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.8239ms 6.6669ms 149.9947 Ops/s 151.3637 Ops/s $\color{#d91a1a}-0.90\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7750ms 0.5172ms 1.9336 KOps/s 1.9339 KOps/s $\color{#d91a1a}-0.02\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6880ms 0.4944ms 2.0226 KOps/s 2.0272 KOps/s $\color{#d91a1a}-0.23\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.8718ms 6.6126ms 151.2272 Ops/s 153.9297 Ops/s $\color{#d91a1a}-1.76\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.8624ms 0.5108ms 1.9579 KOps/s 1.9619 KOps/s $\color{#d91a1a}-0.20\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6705ms 0.4886ms 2.0466 KOps/s 2.0518 KOps/s $\color{#d91a1a}-0.25\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.1083ms 1.9588ms 510.5248 Ops/s 513.3389 Ops/s $\color{#d91a1a}-0.55\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 5.6874ms 1.8764ms 532.9244 Ops/s 540.5951 Ops/s $\color{#d91a1a}-1.42\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.8974ms 6.7711ms 147.6875 Ops/s 147.4717 Ops/s $\color{#35bf28}+0.15\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.5331ms 0.6652ms 1.5034 KOps/s 1.5036 KOps/s $\color{#d91a1a}-0.01\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8611ms 0.6412ms 1.5596 KOps/s 1.5631 KOps/s $\color{#d91a1a}-0.22\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.8359ms 6.7159ms 148.9014 Ops/s 151.5585 Ops/s $\color{#d91a1a}-1.75\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.6267ms 0.5149ms 1.9422 KOps/s 1.9430 KOps/s $\color{#d91a1a}-0.04\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7053ms 0.4950ms 2.0201 KOps/s 2.0128 KOps/s $\color{#35bf28}+0.36\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.8373ms 6.5831ms 151.9036 Ops/s 152.4624 Ops/s $\color{#d91a1a}-0.37\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6285ms 0.5121ms 1.9527 KOps/s 1.9521 KOps/s $\color{#35bf28}+0.03\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.3419ms 0.4964ms 2.0147 KOps/s 2.0419 KOps/s $\color{#d91a1a}-1.33\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.9396ms 6.8276ms 146.4647 Ops/s 147.8829 Ops/s $\color{#d91a1a}-0.96\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1965ms 0.6680ms 1.4971 KOps/s 1.5074 KOps/s $\color{#d91a1a}-0.68\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8049ms 0.6458ms 1.5484 KOps/s 1.5408 KOps/s $\color{#35bf28}+0.49\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1459s 8.1221ms 123.1212 Ops/s 125.1346 Ops/s $\color{#d91a1a}-1.61\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 18.8012ms 16.2689ms 61.4668 Ops/s 52.4150 Ops/s $\textbf{\color{#35bf28}+17.27\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 7.5739ms 1.5421ms 648.4604 Ops/s 724.5505 Ops/s $\textbf{\color{#d91a1a}-10.50\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1246s 10.1234ms 98.7809 Ops/s 127.9669 Ops/s $\textbf{\color{#d91a1a}-22.81\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 18.9922ms 16.2713ms 61.4579 Ops/s 60.4899 Ops/s $\color{#35bf28}+1.60\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.4631ms 1.3171ms 759.2457 Ops/s 671.8737 Ops/s $\textbf{\color{#35bf28}+13.00\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1233s 7.8860ms 126.8078 Ops/s 124.8090 Ops/s $\color{#35bf28}+1.60\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.9581ms 16.5022ms 60.5981 Ops/s 60.4705 Ops/s $\color{#35bf28}+0.21\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.7638ms 1.5388ms 649.8657 Ops/s 692.1019 Ops/s $\textbf{\color{#d91a1a}-6.10\%}$

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 13, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 58.9883ms 58.3442ms 17.1397 Ops/s 17.4848 Ops/s $\color{#d91a1a}-1.97\%$
test_sync 33.5389ms 31.7033ms 31.5425 Ops/s 31.2190 Ops/s $\color{#35bf28}+1.04\%$
test_async 0.1411s 30.9067ms 32.3555 Ops/s 32.6719 Ops/s $\color{#d91a1a}-0.97\%$
test_simple 0.4951s 0.4193s 2.3848 Ops/s 2.4055 Ops/s $\color{#d91a1a}-0.86\%$
test_transformed 0.6325s 0.5681s 1.7601 Ops/s 1.7303 Ops/s $\color{#35bf28}+1.72\%$
test_serial 1.3208s 1.2584s 0.7947 Ops/s 0.8057 Ops/s $\color{#d91a1a}-1.38\%$
test_parallel 1.1731s 1.1088s 0.9019 Ops/s 0.8918 Ops/s $\color{#35bf28}+1.14\%$
test_step_mdp_speed[True-True-True-True-True] 0.1622ms 24.9111μs 40.1428 KOps/s 40.2222 KOps/s $\color{#d91a1a}-0.20\%$
test_step_mdp_speed[True-True-True-True-False] 43.3200μs 14.4201μs 69.3475 KOps/s 69.4687 KOps/s $\color{#d91a1a}-0.17\%$
test_step_mdp_speed[True-True-True-False-True] 78.0020μs 14.3405μs 69.7326 KOps/s 70.3307 KOps/s $\color{#d91a1a}-0.85\%$
test_step_mdp_speed[True-True-True-False-False] 29.2250μs 8.2577μs 121.0986 KOps/s 120.4371 KOps/s $\color{#35bf28}+0.55\%$
test_step_mdp_speed[True-True-False-True-True] 62.0650μs 26.5525μs 37.6613 KOps/s 37.8136 KOps/s $\color{#d91a1a}-0.40\%$
test_step_mdp_speed[True-True-False-True-False] 67.1550μs 15.9767μs 62.5911 KOps/s 62.7561 KOps/s $\color{#d91a1a}-0.26\%$
test_step_mdp_speed[True-True-False-False-True] 41.1860μs 15.9011μs 62.8887 KOps/s 63.0957 KOps/s $\color{#d91a1a}-0.33\%$
test_step_mdp_speed[True-True-False-False-False] 48.6300μs 9.7380μs 102.6902 KOps/s 101.2067 KOps/s $\color{#35bf28}+1.47\%$
test_step_mdp_speed[True-False-True-True-True] 78.3460μs 28.1777μs 35.4891 KOps/s 35.5629 KOps/s $\color{#d91a1a}-0.21\%$
test_step_mdp_speed[True-False-True-True-False] 63.5280μs 17.4542μs 57.2928 KOps/s 56.9870 KOps/s $\color{#35bf28}+0.54\%$
test_step_mdp_speed[True-False-True-False-True] 40.7360μs 15.7938μs 63.3159 KOps/s 62.7686 KOps/s $\color{#35bf28}+0.87\%$
test_step_mdp_speed[True-False-True-False-False] 47.4890μs 9.7385μs 102.6849 KOps/s 101.0971 KOps/s $\color{#35bf28}+1.57\%$
test_step_mdp_speed[True-False-False-True-True] 69.5300μs 29.5460μs 33.8456 KOps/s 33.5331 KOps/s $\color{#35bf28}+0.93\%$
test_step_mdp_speed[True-False-False-True-False] 70.9920μs 18.8508μs 53.0480 KOps/s 52.6284 KOps/s $\color{#35bf28}+0.80\%$
test_step_mdp_speed[True-False-False-False-True] 65.0610μs 17.1412μs 58.3389 KOps/s 57.5414 KOps/s $\color{#35bf28}+1.39\%$
test_step_mdp_speed[True-False-False-False-False] 62.9670μs 11.1934μs 89.3381 KOps/s 88.4552 KOps/s $\color{#35bf28}+1.00\%$
test_step_mdp_speed[False-True-True-True-True] 79.4180μs 28.1192μs 35.5630 KOps/s 35.2134 KOps/s $\color{#35bf28}+0.99\%$
test_step_mdp_speed[False-True-True-True-False] 45.2250μs 17.5288μs 57.0489 KOps/s 56.7773 KOps/s $\color{#35bf28}+0.48\%$
test_step_mdp_speed[False-True-True-False-True] 68.4870μs 18.3392μs 54.5281 KOps/s 54.6518 KOps/s $\color{#d91a1a}-0.23\%$
test_step_mdp_speed[False-True-True-False-False] 45.8760μs 10.9452μs 91.3642 KOps/s 90.5430 KOps/s $\color{#35bf28}+0.91\%$
test_step_mdp_speed[False-True-False-True-True] 81.0920μs 29.5663μs 33.8223 KOps/s 33.6095 KOps/s $\color{#35bf28}+0.63\%$
test_step_mdp_speed[False-True-False-True-False] 68.1470μs 18.9284μs 52.8306 KOps/s 52.0856 KOps/s $\color{#35bf28}+1.43\%$
test_step_mdp_speed[False-True-False-False-True] 61.9160μs 19.6825μs 50.8067 KOps/s 50.2057 KOps/s $\color{#35bf28}+1.20\%$
test_step_mdp_speed[False-True-False-False-False] 51.4460μs 12.3915μs 80.7003 KOps/s 79.8009 KOps/s $\color{#35bf28}+1.13\%$
test_step_mdp_speed[False-False-True-True-True] 3.2767ms 31.0935μs 32.1610 KOps/s 31.9571 KOps/s $\color{#35bf28}+0.64\%$
test_step_mdp_speed[False-False-True-True-False] 68.9790μs 20.2818μs 49.3053 KOps/s 48.2659 KOps/s $\color{#35bf28}+2.15\%$
test_step_mdp_speed[False-False-True-False-True] 57.3270μs 19.7222μs 50.7043 KOps/s 50.9349 KOps/s $\color{#d91a1a}-0.45\%$
test_step_mdp_speed[False-False-True-False-False] 59.4610μs 12.3530μs 80.9519 KOps/s 80.4783 KOps/s $\color{#35bf28}+0.59\%$
test_step_mdp_speed[False-False-False-True-True] 87.1230μs 32.0618μs 31.1897 KOps/s 30.7362 KOps/s $\color{#35bf28}+1.48\%$
test_step_mdp_speed[False-False-False-True-False] 93.3540μs 21.3738μs 46.7863 KOps/s 45.7352 KOps/s $\color{#35bf28}+2.30\%$
test_step_mdp_speed[False-False-False-False-True] 74.7890μs 20.7500μs 48.1928 KOps/s 47.6774 KOps/s $\color{#35bf28}+1.08\%$
test_step_mdp_speed[False-False-False-False-False] 36.7590μs 13.6562μs 73.2268 KOps/s 72.0247 KOps/s $\color{#35bf28}+1.67\%$
test_values[generalized_advantage_estimate-True-True] 10.1386ms 9.5862ms 104.3166 Ops/s 105.0461 Ops/s $\color{#d91a1a}-0.69\%$
test_values[vec_generalized_advantage_estimate-True-True] 53.3047ms 36.4924ms 27.4030 Ops/s 27.6587 Ops/s $\color{#d91a1a}-0.92\%$
test_values[td0_return_estimate-False-False] 0.2315ms 0.1653ms 6.0499 KOps/s 6.1295 KOps/s $\color{#d91a1a}-1.30\%$
test_values[td1_return_estimate-False-False] 24.0688ms 23.6124ms 42.3506 Ops/s 41.7358 Ops/s $\color{#35bf28}+1.47\%$
test_values[vec_td1_return_estimate-False-False] 38.3068ms 36.4803ms 27.4120 Ops/s 27.4876 Ops/s $\color{#d91a1a}-0.27\%$
test_values[td_lambda_return_estimate-True-False] 34.9941ms 34.2213ms 29.2216 Ops/s 28.7953 Ops/s $\color{#35bf28}+1.48\%$
test_values[vec_td_lambda_return_estimate-True-False] 38.5710ms 36.5678ms 27.3465 Ops/s 27.7270 Ops/s $\color{#d91a1a}-1.37\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 9.4278ms 8.3660ms 119.5320 Ops/s 119.1194 Ops/s $\color{#35bf28}+0.35\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.5367ms 1.9618ms 509.7439 Ops/s 527.9388 Ops/s $\color{#d91a1a}-3.45\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4241ms 0.3563ms 2.8067 KOps/s 2.7793 KOps/s $\color{#35bf28}+0.99\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 49.0848ms 47.8531ms 20.8973 Ops/s 24.7227 Ops/s $\textbf{\color{#d91a1a}-15.47\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.1534ms 3.0308ms 329.9504 Ops/s 320.6426 Ops/s $\color{#35bf28}+2.90\%$
test_dqn_speed 5.9262ms 1.2720ms 786.1563 Ops/s 778.9092 Ops/s $\color{#35bf28}+0.93\%$
test_ddpg_speed 3.4936ms 2.6658ms 375.1241 Ops/s 375.2348 Ops/s $\color{#d91a1a}-0.03\%$
test_sac_speed 9.5269ms 7.8610ms 127.2097 Ops/s 126.9001 Ops/s $\color{#35bf28}+0.24\%$
test_redq_speed 14.0164ms 12.6271ms 79.1945 Ops/s 79.2430 Ops/s $\color{#d91a1a}-0.06\%$
test_redq_deprec_speed 14.6222ms 12.6664ms 78.9490 Ops/s 79.1956 Ops/s $\color{#d91a1a}-0.31\%$
test_td3_speed 8.0689ms 7.7869ms 128.4216 Ops/s 128.7659 Ops/s $\color{#d91a1a}-0.27\%$
test_cql_speed 45.7021ms 36.2331ms 27.5991 Ops/s 26.1842 Ops/s $\textbf{\color{#35bf28}+5.40\%}$
test_a2c_speed 7.8885ms 7.2275ms 138.3605 Ops/s 137.9896 Ops/s $\color{#35bf28}+0.27\%$
test_ppo_speed 9.1073ms 7.4882ms 133.5433 Ops/s 132.1877 Ops/s $\color{#35bf28}+1.03\%$
test_reinforce_speed 7.8421ms 6.3850ms 156.6174 Ops/s 156.0511 Ops/s $\color{#35bf28}+0.36\%$
test_iql_speed 33.2992ms 31.7737ms 31.4726 Ops/s 31.8265 Ops/s $\color{#d91a1a}-1.11\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.0849ms 4.7516ms 210.4544 Ops/s 205.9125 Ops/s $\color{#35bf28}+2.21\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8625ms 0.4736ms 2.1115 KOps/s 2.1088 KOps/s $\color{#35bf28}+0.13\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6995ms 0.4517ms 2.2138 KOps/s 2.2347 KOps/s $\color{#d91a1a}-0.93\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.6921ms 4.7729ms 209.5166 Ops/s 209.4966 Ops/s $+0.01\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7666ms 0.4710ms 2.1232 KOps/s 2.1365 KOps/s $\color{#d91a1a}-0.62\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7483ms 0.4464ms 2.2399 KOps/s 2.2460 KOps/s $\color{#d91a1a}-0.27\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.8635ms 1.6743ms 597.2694 Ops/s 593.9596 Ops/s $\color{#35bf28}+0.56\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.2777ms 1.5851ms 630.8860 Ops/s 622.8764 Ops/s $\color{#35bf28}+1.29\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 9.3461ms 4.9789ms 200.8459 Ops/s 201.7626 Ops/s $\color{#d91a1a}-0.45\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1400s 0.7164ms 1.3958 KOps/s 1.6442 KOps/s $\textbf{\color{#d91a1a}-15.11\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7960ms 0.5826ms 1.7163 KOps/s 1.7033 KOps/s $\color{#35bf28}+0.77\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.3019ms 4.8343ms 206.8540 Ops/s 203.1818 Ops/s $\color{#35bf28}+1.81\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7324ms 0.4728ms 2.1152 KOps/s 2.0992 KOps/s $\color{#35bf28}+0.77\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6551ms 0.4548ms 2.1987 KOps/s 2.2142 KOps/s $\color{#d91a1a}-0.70\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.5833ms 4.7939ms 208.5987 Ops/s 205.6462 Ops/s $\color{#35bf28}+1.44\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.3994ms 0.4683ms 2.1353 KOps/s 2.1332 KOps/s $\color{#35bf28}+0.10\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7545ms 0.4527ms 2.2088 KOps/s 2.2272 KOps/s $\color{#d91a1a}-0.83\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.1512ms 4.9134ms 203.5236 Ops/s 199.7219 Ops/s $\color{#35bf28}+1.90\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8167ms 0.6155ms 1.6248 KOps/s 1.6577 KOps/s $\color{#d91a1a}-1.99\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 7.2279ms 0.5938ms 1.6840 KOps/s 1.7223 KOps/s $\color{#d91a1a}-2.22\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1266s 6.2823ms 159.1780 Ops/s 157.9276 Ops/s $\color{#35bf28}+0.79\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1216s 14.9911ms 66.7061 Ops/s 76.4886 Ops/s $\textbf{\color{#d91a1a}-12.79\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 5.0740ms 1.3329ms 750.2192 Ops/s 787.4046 Ops/s $\color{#d91a1a}-4.72\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1069s 5.9330ms 168.5489 Ops/s 168.5273 Ops/s $\color{#35bf28}+0.01\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.0590ms 12.9688ms 77.1084 Ops/s 66.7561 Ops/s $\textbf{\color{#35bf28}+15.51\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.8110ms 1.2500ms 800.0193 Ops/s 750.4129 Ops/s $\textbf{\color{#35bf28}+6.61\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1131s 6.1724ms 162.0103 Ops/s 162.3397 Ops/s $\color{#d91a1a}-0.20\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.5600ms 12.9863ms 77.0040 Ops/s 75.9369 Ops/s $\color{#35bf28}+1.41\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.0869ms 1.3827ms 723.2155 Ops/s 656.8989 Ops/s $\textbf{\color{#35bf28}+10.10\%}$

@vmoens
Update
569b6ba 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Aug 13, 2024
@vmoens
[Feature] non-functional SAC loss
0dbb1a5 
ghstack-source-id: fd766d1a4f0868435920c8fe8caffb75425c2a05
Pull Request resolved: #2393
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot