Generate the math expression LATEX sequence according to the handwritten math expression image.
git clone https://github.com/win5923/TrOCR-Handwritten-Mathematical-Expression-Recognition.git
pip install transformers
pip install datasets jiwer
pip install sentencepiece
for Ubuntu you can use screen and run train2.py
for Jupyter you can run train.ipynb
python train2.py
python train.ipynb
use predict.py or test.py to inference on new images.
On CHROME 2016 dataset CER = 0.193
On CHROME 2016 dataset Accuracy = 0.306
On CROHME 2016 test dataset the Accuracy is worst below image's model.
Thanks @NielsRogge's Notebook so much.It's very helpful.
https://github.com/NielsRogge/Transformers-Tutorials/blob/master/TrOCR/Fine_tune_TrOCR_on_IAM_Handwriting_Database_using_Seq2SeqTrainer.ipynb