[Question] Resume of training from saved model does not give similar result

[kmsgnnew]

Question

Does Resume training from saved model give similar results in stable baseline 3??

Additional context

A model is trained for 10 epochs, model is saved at 5 epochs then loaded using DQN.load() and then training is continued with model.learn for another 5 epoch. This continuation does not match for me. does it match for anyone or is it still not supported

Note: 1 epoch is over entire dataset

I have already seen similar issue comments in stable baseline at
hill-a/stable-baselines#692
hill-a/stable-baselines#301

[araffin]

Hello,
have you saved the replay buffer? (cf doc)
have you make sure that the exploration factor (epsilon) starts at the same value it ended?

It is hard to compare training policy, better to compare deterministic ones, usually used for evaluation (cf doc).

Please provide a minimal working example (cf issue template)

[kmsgnnew]

thanks araffin for quick reply.

as I understand adding following two changes should improve:

1. {have you saved the replay buffer?}

save_replay_buffer while training and then first load model using DQN.load() also load_replay_buffer from pickle file

2. { have you make sure that the exploration factor (epsilon) starts at the same value it ended? }

for ensuring exploration factor (epsilon) starts at the same value it ended.

mean_params = model.get_parameters()
save this dictonary.
DQN.load() also set_parameters(load_path_to_dictonary)

[araffin]

hill-a/stable-baselines#950 (comment)