Repository with all models in ONNX format

[Kromtar]

In this Drive folder you can find all EasyOCR models in ONNX format covering all currently available languages. Look at the text file in the folder to see which one corresponds to the language you are using.

ONNX is an agnostic and standardized format for storing models. Models in ONNX format can be imported and used in several Runtimes including PyTorch, TensorFlow and OpenVINO. The ONNX team also provides its own runtine, ONNX Runtime. This Runtime allows development in several languages, operating systems and acceleration hardware. By owning the EasyOCR models in ONNX it is possible to cover new scenarios where the use of Python and/or PyTorch is not adequate.

In my use case, I am using the EasyOCR and ONNX Runtime models to be able to develop C++ applications. This makes it much easier to bring EasyOCR functionalities into production.

In this issue you can find a general guide on how I have managed to modify EasyOCR to export the models, initially developed in Torch to ONNX. Note that the export process explained in the guide can only be achieved if you have CUDA acceleration. As not everyone has access to such hardware, I have decided to share a copy of the models already exported.

The models that I make available are a first version, I may optimize the exported models in the next weeks.

[RaiAmanRai]

Hi @Kromtar , I am trying to convert these onnx models to tensorrt formats using NVIDIA TensorRT containers.

Everything works fine with the detection model but when I am exporting the recognition model, it fails

Error[4]: [graphShapeAnalyzer.cpp::processCheck::581] Error Code 4: Internal Error (Conv_16: spatial dimension of convolution output cannot be negative (build-time output dimension of axis 3 is -1))

However, I suspect this is only limited to smaller models of size ~approx. 15mb . I didn't try everyone of them but I did manage to convert 10_recognition_model.onnx to tensorrt formt without any problem.

Do you suspect anything that is gong wrong?

Also could you please clarify if you are using two different model architectures ?

[DLumi]

Hi @Kromtar.
Any luck in reworking the original codebase?
I tried to mess around with your ONNX models, but it didn't get me far without proper pre/postprocessing steps. I tried to replicate them in numpy, but since the original code is rather messy and tangled, I wasn't able to get this thing working properly as I've probably missed something.

[alib022]

Hi @Kromtar
I managed to load your onnx models into the Triton inference server. I reworked the whole code to work and everything goes end to end. However, the results are not right!
for a speed limit of 35 I am getting

[([[6, 4], [83, 4], [83, 59], [6, 59]], 'ISAP7', 0.07690704349016575),
([[6, 46], [82, 46], [82, 101], [6, 101]], '351', 0.9431633933333133)]

'as' suppose to be 35

The result from easyOCR itself:
([[6, 4], [81, 4], [81, 58], [6, 58]], '[serp]', 0.024114014123917345) ([[8, 48], [76, 48], [76, 98], [8, 98]], '35', 0.9999992413568111)

And the bounding boxes are all correct, compare to running the same image with easyOCR.
However, seems like this implementation has a different config code or something that I can't figure out!

[nissansz]

How to use these onnx samples? Is it possible to use these onnx without EasyOcr?
Any dict for chars needed?

[light42]

@Kromtar not only conversion to onnx already discussed many times since two years ago, easyocr already provide the script to export the model into onnx format.

What's the difference between your approach with the one provided officially by JaidedAI?

[light42]

This is what chatgpt spews out when I asked it to also consider dict and configs:

import onnx

# Define the output ONNX file path
output_path = 'model.onnx'

# Convert the PyTorch model to ONNX format
torch.onnx.export(model, input_data, output_path, export_params=True, opset_version=11, input_names=['input'], output_names=['output'], dynamic_axes={'input': {0: 'batch_size'}, 'output': {0: 'batch_size'}})

# Update the ONNX model with the dictionary and configuration files
model_proto = onnx.load(output_path)
model_proto.graph.input[0].type.tensor_type.shape.dim[0].dim_param = 'batch_size'
model_proto.graph.output[0].type.tensor_type.shape.dim[0].dim_param = 'batch_size'

# Add the dictionary and configuration files as attributes to the ONNX model
dict_proto = onnx.helper.make_tensor("dictionary", onnx.TensorProto.STRING, [len(dictionary)], dictionary)
config_proto = onnx.helper.make_tensor("config", onnx.TensorProto.STRING, [1], [json.dumps(config)])
model_proto.graph.initializer.extend([dict_proto, config_proto])

I haven't checked the validity of this script yet, but I hope this will give inspirations for you @Kromtar.

[kadmor]

Привет@Kromtar Мне удалось загрузить ваши модели onnx на сервер вывода Triton. Я переработал весь код, чтобы он работал и все идет от и до. Тем не менее, результаты не являются правильными! для ограничения скорости 35 я получаю

[([[6, 4], [83, 4], [83, 59], [6, 59]], 'ISAP7', 0.07690704349016575), ([[6, 46], [82, 46], [ 82, 101], [6, 101]], '351', 0,9431633933333133)]

'как' должно быть 35

Результат самого easyOCR: ([[6, 4], [81, 4], [81, 58], [6, 58]], '[serp]', 0.024114014123917345) ([[8, 48], [76, 48], [76, 98], [8, 98]], '35', 0.9999992413568111)

И все ограничивающие рамки правильные, сравните с запуском того же изображения с помощью easyOCR. Однако похоже, что эта реализация имеет другой код конфигурации или что-то, что я не могу понять!

Hello @alib022 Did you manage to solve this problem? Can you show your config.pbtxt for detector.onnx and recognition.onnx models?

[alib022]

right now I have these in dev environment
`name: "ocr-detection-onnx"
platform: "onnxruntime_onnx"
max_batch_size : 0
input [
{
name: "input1"
data_type: TYPE_FP32
format: FORMAT_NCHW
dims: [ 3, -1, -1 ]
reshape { shape: [ 1, 3, -1, -1 ] }
}
]

output [
{
name: "output"
data_type: TYPE_FP32
dims: [1,-1,-1,2]

}
]
instance_group [
{
count: 1
kind: KIND_GPU

}
]
name: "ocr-recognition-onnx"
platform: "onnxruntime_onnx"
max_batch_size : 0
input [
{
name: "input1"
data_type: TYPE_FP32
dims: [1,1,64,-1]

}
]
output [
{
name: "output"
data_type: TYPE_FP32
dims: [1,-1,97]

}
]

instance_group [
{
count: 1
kind: KIND_GPU

}
]`

[kadmor]

right now I have these in dev environment `name: "ocr-detection-onnx" platform: "onnxruntime_onnx" max_batch_size : 0 input [ { name: "input1" data_type: TYPE_FP32 format: FORMAT_NCHW dims: [ 3, -1, -1 ] reshape { shape: [ 1, 3, -1, -1 ] } } ]

output [ { name: "output" data_type: TYPE_FP32 dims: [1,-1,-1,2]

} ] instance_group [ { count: 1 kind: KIND_GPU

} ] name: "ocr-recognition-onnx" platform: "onnxruntime_onnx" max_batch_size : 0 input [ { name: "input1" data_type: TYPE_FP32 dims: [1,1,64,-1]

} ] output [ { name: "output" data_type: TYPE_FP32 dims: [1,-1,97]

} ]

instance_group [ { count: 1 kind: KIND_GPU

} ]`

@alib022 Thank you very much!

[nissansz]

Any latest models in onnx format?

[masoudMZB]

@light42 you said:

@Kromtar not only conversion to onnx already discussed many times since two years ago, easyocr already provide the script to export the model into onnx format.

where is the script to convert my text recognition model to onnx?
there is export.py in easyocr but it is only for text detection. any code for recognition?

[as1605]

Hi @Kromtar @alib022 , I am trying to run EasyOCR on NVIDIA Triton with ONNX.

Currently, the ONNX model has loaded as

"input": [
    {
      "name": "input1",
      "data_type": "TYPE_FP32",
      "format": "FORMAT_NONE",
      "dims": [1, 1, 64, -1],
      "is_shape_tensor": false,
      "allow_ragged_batch": false,
      "optional": false
    }
  ],
  "output": [
    {
      "name": "output",
      "data_type": "TYPE_FP32",
      "dims": [1, -1, 97],
      "label_filename": "",
      "is_shape_tensor": false
    }
  ],

Can anyone help me with how to go about with making inferences and making sense of the output? What are the 64 and 97 in dimensions for?

[alib022]

Hi,
The EasyOCR is a four-step process, 1) pre-processing, 2) detection, 3) classification, 4) post-processing
I moved steps 2 and 3 to Tirton and 1 and 4 are left on the client. I will try to open a public repo in a few weeks with all the codes for everyone to use

[as1605]

Hi, The EasyOCR is a four-step process, 1) pre-processing, 2) detection, 3) classification, 4) post-processing I moved steps 2 and 3 to Tirton and 1 and 4 are left on the client. I will try to open a public repo in a few weeks with all the codes for everyone to use

@alib022
Thanks, this is exactly what I need. Actually, I need this a bit early so can you please share a preview with me, or maybe I can help in making the repo public?

[arcb01]

Hi,
The EasyOCR is a four-step process, 1) pre-processing, 2) detection, 3) classification, 4) post-processing
I moved steps 2 and 3 to Tirton and 1 and 4 are left on the client. I will try to open a public repo in a few weeks with all the codes for everyone to use

Looking forward to it!

[brooksyott]

Hi, The EasyOCR is a four-step process, 1) pre-processing, 2) detection, 3) classification, 4) post-processing I moved steps 2 and 3 to Tirton and 1 and 4 are left on the client. I will try to open a public repo in a few weeks with all the codes for everyone to use

Wondering if you are able to share your work?

[namogg]

For the regconition network, im getting this error.
RuntimeError: Only tuples, lists and Variables are supported as JIT inputs/outputs. Dictionaries and strings are also accepted, but their usage is not recommended. Here, received an input of unsupported type: torch._C.ScriptObject

[rm-asif-amin]

Hi, The EasyOCR is a four-step process, 1) pre-processing, 2) detection, 3) classification, 4) post-processing I moved steps 2 and 3 to Tirton and 1 and 4 are left on the client. I will try to open a public repo in a few weeks with all the codes for everyone to use

@alib022 thanks for sharing your insights in this thread, can I get access to your code please? I have a similar use case

[vidysqrx]

I am trying to access the link to google drive, but I get 404 page.
Maybe the link is dead now, could you help to share latest link