DESCIT

Image Caption Generation

The project uses Convolution Neural Network(CNN) and Long Short Term Memory(LSTM) technique of Recurrent Neural Network(RNN) for predicting the captions for the image provided by the user through the Web App designed using Flask

Structure:

Dataset Used:-

Flickr8k Image Dataset (https://drive.google.com/drive/u/3/folders/1VNtUKzFWg0TFB5W3FkBOHlzX4qmmR2HA) Flickr8k Text Dataset (https://drive.google.com/drive/u/3/folders/1RGq8vhCVxH5TKeA7IaACjy0IXDZz8jTf)

Construction Process:

1. We prepared a feature map of 1500 images (out of 8k){because of hardware limitation} using the ResNet50 model of Tensorflow and saved it in a feature.pickle file.
2. Flickr8k text dataset contains 5 captions for every 8092(8k) images in it. We have prepared a dictionary of all the captions with image as key and caption as value.
3. Once every image feature is attached with the corresponding keyword from the caption, we trained or LSTM model to predict the caption.

How to install and use:

1. Clone the repo and open in Editor and change the paths
2. Run the predict.py file
3. Run flaskk.py file
4. A web app will start at localhost
5. Go to upload page and provide the image whose caption you want. Done, the model will provide you with the predicted caption.

Versions:

1.Numpy 1.18.5 2.Pandas - 1.1.4 3.Tensorflow - 2.3.1 4.Keras - 2.4.3 5.Python - 3.7.6

Screenshots:

For more:

About any issue or construction of model or anything. Mail Mayank Singh([email protected])