This folder contains four notebooks that show how to train, optimize, quantize and show live inference on a MONAI segmentation model with PyTorch Lightning and OpenVINO:
1. Data Preparation for 2D Segmentation of 3D Medical Data
2. Train a 2D-UNet Medical Imaging Model with PyTorch Lightning
3a. Convert and Quantize a UNet Model and Show Live Inference using POT
3b. Convert and Quantize a UNet Model and Show Live Inference using NNCF
The main difference between the POT and NNCF quantization notebooks is that NNCF performs quantization within the PyTorch framework, while POT performs quantization after the PyTorch model has been converted to OpenVINO IR format. There is a pre-trained model and a subset of the dataset provided for the quantization notebook, so it is not required to run the data preparation and training notebooks before running the quantization tutorial.
This quantization tutorial consists of the following steps:
- Converting an ONNX model to OpenVINO IR with Model Optimizer.
- Quantizing a model with the Post-Training Optimization Tool API in OpenVINO.
- Evaluating the F1 score metric of the original model and the quantized model.
- Benchmarking performance of the original model and the quantized model.
- Showing live inference with async API and MULTI plugin in OpenVINO.
In addition to the notebooks in this folder, the Live Inference and Benchmark CT-scan data demo notebook contains the live-inference part of the quantization tutorial. It includes a pre-quantized model.
If you have not installed all required dependencies, follow the Installation Guide.