[Error]: I got unexpected error using enjoy() with pretrain model

[tiencapham]

🐛 Bug

I just install as in README.md file and try to 'enjoy' the pretrained model with Atari games. But I got and error as below, it seems sb3-zoo fail to register the environment.

To Reproduce

python enjoy.py --algo a2c --env BreakoutNoFrameskip-v4 --folder rl-trained-agents/ -n 5000

Relevant log output / Error message

rl-baselines3-zoo\enjoy.py", line 4, in <module> enjoy()
rl-baselines3-zoo\rl_zoo3\enjoy.py", line 138, in enjoy
    is_atari = ExperimentManager.is_atari(env_name.gym_id)
rl-baselines3-zoo\rl_zoo3\exp_manager.py", line 533, in is_atari
    return "AtariEnv" in ExperimentManager.entry_point(env_id)
rl-baselines3-zoo\rl_zoo3\exp_manager.py", line 528, in entry_point
    return str(gym.envs.registry[env_id].entry_point)
KeyError: 'BreakoutNoFrameskip-v4'

System Info

• OS: Windows-10-10.0.22621-SP0 10.0.22621
• Python: 3.9.18
• Stable-Baselines3: 2.2.1
• PyTorch: 2.1.1+cpu
• GPU Enabled: False
• Numpy: 1.26.2
• Cloudpickle: 3.0.0
• Gymnasium: 0.29.1
• OpenAI Gym: 0.26.2

Checklist

• I have checked that there is no similar issue in the repo
• I have read the SB3 documentation
• I have read the RL Zoo documentation
• I have provided a minimal and working example to reproduce the bug
• I've used the markdown code blocks for both code and stack traces.

[qgallouedec]

Hi, make sure to install atari envs.

pip install stable-baselines3[extra]

Answer in the doc: https://stable-baselines3.readthedocs.io/en/master/guide/examples.html#id2

[tiencapham]

Hi, make sure to install atari envs.

pip install stable-baselines3[extra]

Answer in the doc: https://stable-baselines3.readthedocs.io/en/master/guide/examples.html#id2

Hi,
Thank for your response.

I actually modified the stable-baseline3 and installed it from source. Maybe that is the reason I missed this step.

How can I install the atari envs in this case?

Best,
Tien Pham

[qgallouedec]

The answer is still in the doc, in the exact same paragraph :)

pip install gymnasium[atari, accept-rom-license]

Command out-of-date in the doc, fixed by DLR-RM/stable-baselines3#1773

[tiencapham]

Silly me, but I solved it by:

pip install -e .[docs,tests,extra] in stable baseline.

Thanks for your help