Bug fix: The atari_100k.yaml file had all env settings NOT under a env_config key, but on the top level. This lead to the Atari problem being:
- W/ stochastic actions
- frameskip=20(!) (5 (gym default) x 4 (RLlib's MaxAndSkip wrapper))
- full action space
See Pong-v5 results on 8 seed V-100 GPUs here:
https://wandb.ai/rllib-team/atari-100k-pong?workspace=user-sven_mika