PPO_RLLIB code improvement

[AvisP]

System information

• Grid2op version: 1.9.5
• l2rpn-baselines version: 0.8.0
• System: osx
• Baseline concerned: eg PPO_RLLIB

Bug description

The PPO_RLLIB code has been updated but there are couple of issues

1. Missing the following line self.env_glop.chronics_handler.reset() after https://github.com/rte-france/l2rpn-baselines/blob/ba346d347c85ac70cef6e4c73e3e60edc839490c/l2rpn_baselines/PPO_RLLIB/env_rllib.py#L103 and need to add it to make the train and eval script work.

2. There environment seems to be getting created twice. First one just to convert the environment observation and action space into gym format and then pass into the RLLIBAgent class where the environment is built again through rllib library. If I understand correctly this takes more memory for two environments and rewriting to just make one will help with memory.

3. The environment for the l2rpn_neurips_2020_track1_small take a very long time to do 100 iterations with train_batch_size of 20,000 added to env_config_ppo. These two parameters may even need to be higher to get good results. If something can be done to speed up the training that would be helpful for scaling to bigger networks.

How to reproduce

Execute the train and eval script here

Expected output

Train script should run without any issues and memory requirement is lower and faster training