How to continue training?

[dungcan2k5]

Hi Lukas, this project is really cool, but now I have a little question that needs your help. I trained the model based on your manual. Now I want to pause the training (due to the time limit on colab), how can I resume my work tomorrow? One more time, thank you very much

[lukas-blecher]

Hello,

the model should be saved in the colab at the path /content/pix2tex/model/checkpoints.
You can download it from there and upload it the next day or save it on gdrive which might be faster if your upload speed is not the best.
E.g.

from google.colab import drive
drive.mount('/content/gdrive')

To resume the next day you need to save the weights as weights.pth in the project root.

Hope that helps

[dungcan2k5]

Thank you very much for your enthusiastic reply. I've done it and it worked. By the way, I also wanted to share how everyone can sync and continue their progress on wandb.
First, you need to edit the file train.py at the path /usr/local/lib/python3.8/dist-packages/pix2tex/train.py:
On line 87:

     parser.add_argument('--resume', help='path to checkpoint folder', action='store_true')

Edited into:

     parser.add_argument('--resume', help='path to checkpoint folder', type=str)

Back in the notebook, we enter the following:
!python -m pix2tex.train --config /path/to/file/config.yaml --resume /path/to/checkpoint/folder

Hope they can help. Thank you so much for this project!!!