How to save/load model

[yiqiang-zhao]

Hi, I want to save a model and load it using torch.load. I followed this and add a SaveCheckpointHook in the hook_list.

I can load the checkpointed model, but the shape of every parameter is reduced to 1/8 by one dimension. Besides, every parameter's name is prefixed with module.module..

I used a 8-v100-16G node.

Are there any tutorials on saving model and loading it by pure torch? Thanks.

[ver217]

What parallel training technique are you using?

[Excuses123]

Are you using tensor parallel? I also encounter this problem

[yangjianxin1]

I use tensor parallel, and I also encounter this problem

[yangjianxin1]

do you solve the problem @Excuses123 @ver217

[binmakeswell]

Hi @yiqiang-zhao @Excuses123 @yangjianxin1 We will update this part next week. #3339 Thanks.