Low accuracy for act policy on pushT env

[KongCDY]

The highest success rate is 44%, as n_decoder_layers=7. Are there any other tricks for this?

[pprett]

A success rate of 44% seems high to me for ACT on the pushT task [1]. Note that the original DP paper used a different notion of success for the task: they reported average maximum reward [2] whereas lerobot reports average success (where success is defined as 95% overlap reached). When I consider average success, DPs performance drops to 68% for me in the original DP and 62% when running mobile-aloha's DP on [1] .

Did you notice a big difference between n_decoder_layers=7 vs default ACT setting** of n_decoder_layers=1?

** default as in the original ACT codebase - not sure what lerobot uses as default.

[1] https://huggingface.co/datasets/lerobot/pusht
[2] https://github.com/real-stanford/diffusion_policy/blob/548a52bbb105518058e27bf34dcf90bf6f73681a/diffusion_policy/env_runner/pusht_image_runner.py#L232-L249