Polyglot

Polyglot is an extensible, configurable, experimenting automation for researching the performance of Recurrent Neural Nets language models.

Built as a need for our Bachelor of Science Software Engineering degree final project - Improving performance of RNN based LSTM language models using concurrent Machine Learning techniques with TensorFlow for Python.

Tasks

In addition to the basic task of predicting the next word of a sentence, theses are the additional tasks that we forced the language model to improve it's performance:

1. Part of Speech (POS) - Predicting what part of speech the next word should be (verb, noun, adjective etc.)
2. Generated Classifier - Given the original dataset and a dataset generated by the language model itself, the classifier should detect which sentences are generated.
3. Same as 2 but this time use an unlearing softmax function as the lost function.
4. Write your own.

Techniques

1. Multitask Learning - training the same language model with different tasks concurrently.
2. Transfer Learning - training the same language model with different tasks sequentially.
3. Write your own.

Installation

Using Docker

1. Install Docker https://docs.docker.com/install/

Manual

1. Install TensorFlow version: 1.12 - https://www.tensorflow.org/install
2. Install Python dependancies:

pip install -r requirements.txt

Open Python console and run:

import nltk
nltk.download('punkt')
nltk.download('averaged_perceptron_tagger')

How to Run

Using Docker

1. Build the image (notice the dot at the end):

docker build -t lstm_fast .

2. Run the image:

docker run lstm_fast

Manual

Make sure your current working directory is the project's root folder

python main.py

Limitations

We assume of the following:

1. Your vocab file fits in memory (train, test and validation datasets are unlimited)
2. We need the num batches of each {train, test, validation} to be inserted into hyperparameters.json this is due to the fact that tf.data.Dataset loads in mini batches your data without taking into how much data there is.

Configuring the JSON Schema

Using PyCharm

1. Open the experiment_config.json in Pycharm

2. Click on the "No Json Schema" button (marked red):

3. Choose "New Schema Mapping":

4. Give the mapping any name you wish

5. Choose the file schema - schema.json:

6. Choose Schema version - "JSON Schema Version 7"

7. Your final settings should look this:

Using The Web

1. Go to https://www.jsonschemavalidator.net/

2. Copy the content of schema.json to the left side of the web page

3. Copy the content of experiments_config.json to the right side of the web page

4. If your JSON is valid you should see the green message:

5. If your json is invalid you should see the red message indication the error: