Running TRL example doesn't work when using deepspeed on accelerate

[Superskyyy]

System Info

Just linking the issue back to accelerate so maybe someone can help to resolve it instead of at the TRL repo. huggingface/trl#2377 Currently we are stuck at a low version because of this bug.

error: initialize your accelerator via accelerator = Accelerator()

Information

• The official example scripts
• My own modified scripts

Tasks

• One of the scripts in the examples/ folder of Accelerate or an officially supported no_trainer script in the examples folder of the transformers repo (such as run_no_trainer_glue.py)
• My own task or dataset (give details below)

Reproduction

Please find in the linked issue

Expected behavior

Deepspeed can run properly on the example with accelerate.