Digit Recogition with Custom Model using CoreML

After iOS 11, Apple introduce CoreML, which enables developer to build up application with state-of-the-art AI technique.

In this project, you can recognize your handwritten digit with a machine learning model, specifically trained model based on digit (MNIST).

Apple provides ready-to-use Core ML models, or you can find some interesting models available here. What's more, you can use your own machine learning model!

---

Build up Machine Learning Model with Keras

As you may know, TensorFlow is the most popular machine learning library, and Keras is a high-level neural networks API, written in Python and capable of running on top of TensorFlow.

Environment Requiresments:

• python 2.7 (for coremltools)
• coremoltools
• tensorflow
• keras

My environment is currently in Python 3. To be pain-free, you have to create a python 2 environment for everything to work. For me I use Anaconda to manage packages and environments.

create one for python version 2.

conda create -n py2 python=2

and you can list all the environments you have

conda env list

# conda environments:
#
py2                      /Users/WeiJay/anaconda/envs/py2
tensorflow               /Users/WeiJay/anaconda/envs/tensorflow
root                  *  /Users/WeiJay/anaconda

activate py2 to set up the environment

source activate py2

pip install -U coremltools
pip install --upgrade tensorflow      # for Python 2.7
pip install keras
pip install h5py                      # required if you plan on saving Keras models to disk

Quick use

open your terminal, and go to the project directory.

$ cd ~/DigitRecognition/ML_model/
$ python mnist.py
$ python convert_to_coreml.py

Build my own deep MNIST model

I build an easy deep network with the Deep MNIST for Experts instruction. In brief, my network structure is as follows: Conv - Pool - Conv - Pool - FC - Dropout - FC(Softmax)

With Keras' API, you can see your network structure model.summary()

Also, you can evaluate you model model.evaluate(X_test, y_test)

Training a model is pretty time consuming, you can save your model to disk whenever you want!

# save your model! serialize model to JSON
model_json = model.to_json()
with open("keras_model.json", "w") as json_file:
    json_file.write(model_json)

# serialize weights to HDF5
model.save_weights("keras_mnist_model.h5")

Convert to CoreML Model

When you satisfy with your model, load your model from disk as well as the pretrained weights, then convert to coreml model.

That's all. Finally, drag and drop your model and happy coding 😀

import coremltools
from keras.models import model_from_json

# load model and weights
json_file = open('keras_model.json', 'r')
loaded_model_json = json_file.read()
json_file.close()
model = model_from_json(loaded_model_json)
model.load_weights("keras_mnist_model.h5")

# convert to .mlmodel
keras_model = coremltools.converters.keras.convert(model, 
                                                    input_names='image (28x28)', 
                                                    image_input_names='image (28x28)', 
                                                    output_names = ['prediction'],
                                                    class_labels=['0', '1', '2', '3', '4', '5', '6', '7', '8', '9'])
keras_model.author = 'Wei Chieh'
keras_model.license = 'nope'
keras_model.short_description = 'Predicts a handwritten digit'

keras_model.save('my_mnist.mlmodel')