NN on IMDB data.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# IMDB dataset \n",
    "\n",
    "* It's a binary classification problem on movie reviews\n",
    "* A set of 50,000 highly polarized reviews from the Internet Movie Database. They’re split into 25,000 reviews for training and 25,000 reviews for testing, each set consisting of 50% negative and 50% positive reviews.\n",
    "\n",
    "* IMDB dataset comes packaged with Keras. It has already been preprocessed: the reviews (sequences of words) have been turned into sequences of integers(ranks based on frequence), where each integer stands for a specific word in a dictionary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\vipul\\Anaconda3\\lib\\site-packages\\h5py\\__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n",
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from keras.datasets import imdb\n",
    "from keras import models\n",
    "from keras import layers\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.Loading the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=15000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The parameter **num_words=15000** means you’ll only keep the top 15,000 most frequently\n",
    " occurring words in the training data. Rare words will be discarded. This allows\n",
    " you to work with vector data of manageable size."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1,\n",
       " 14,\n",
       " 22,\n",
       " 16,\n",
       " 43,\n",
       " 530,\n",
       " 973,\n",
       " 1622,\n",
       " 1385,\n",
       " 65,\n",
       " 458,\n",
       " 4468,\n",
       " 66,\n",
       " 3941,\n",
       " 4,\n",
       " 173,\n",
       " 36,\n",
       " 256,\n",
       " 5,\n",
       " 25,\n",
       " 100,\n",
       " 43,\n",
       " 838,\n",
       " 112,\n",
       " 50,\n",
       " 670,\n",
       " 2,\n",
       " 9,\n",
       " 35,\n",
       " 480,\n",
       " 284,\n",
       " 5,\n",
       " 150,\n",
       " 4,\n",
       " 172,\n",
       " 112,\n",
       " 167,\n",
       " 2,\n",
       " 336,\n",
       " 385,\n",
       " 39,\n",
       " 4,\n",
       " 172,\n",
       " 4536,\n",
       " 1111,\n",
       " 17,\n",
       " 546,\n",
       " 38,\n",
       " 13,\n",
       " 447,\n",
       " 4,\n",
       " 192,\n",
       " 50,\n",
       " 16,\n",
       " 6,\n",
       " 147,\n",
       " 2025,\n",
       " 19,\n",
       " 14,\n",
       " 22,\n",
       " 4,\n",
       " 1920,\n",
       " 4613,\n",
       " 469,\n",
       " 4,\n",
       " 22,\n",
       " 71,\n",
       " 87,\n",
       " 12,\n",
       " 16,\n",
       " 43,\n",
       " 530,\n",
       " 38,\n",
       " 76,\n",
       " 15,\n",
       " 13,\n",
       " 1247,\n",
       " 4,\n",
       " 22,\n",
       " 17,\n",
       " 515,\n",
       " 17,\n",
       " 12,\n",
       " 16,\n",
       " 626,\n",
       " 18,\n",
       " 2,\n",
       " 5,\n",
       " 62,\n",
       " 386,\n",
       " 12,\n",
       " 8,\n",
       " 316,\n",
       " 8,\n",
       " 106,\n",
       " 5,\n",
       " 4,\n",
       " 2223,\n",
       " 5244,\n",
       " 16,\n",
       " 480,\n",
       " 66,\n",
       " 3785,\n",
       " 33,\n",
       " 4,\n",
       " 130,\n",
       " 12,\n",
       " 16,\n",
       " 38,\n",
       " 619,\n",
       " 5,\n",
       " 25,\n",
       " 124,\n",
       " 51,\n",
       " 36,\n",
       " 135,\n",
       " 48,\n",
       " 25,\n",
       " 1415,\n",
       " 33,\n",
       " 6,\n",
       " 22,\n",
       " 12,\n",
       " 215,\n",
       " 28,\n",
       " 77,\n",
       " 52,\n",
       " 5,\n",
       " 14,\n",
       " 407,\n",
       " 16,\n",
       " 82,\n",
       " 10311,\n",
       " 8,\n",
       " 4,\n",
       " 107,\n",
       " 117,\n",
       " 5952,\n",
       " 15,\n",
       " 256,\n",
       " 4,\n",
       " 2,\n",
       " 7,\n",
       " 3766,\n",
       " 5,\n",
       " 723,\n",
       " 36,\n",
       " 71,\n",
       " 43,\n",
       " 530,\n",
       " 476,\n",
       " 26,\n",
       " 400,\n",
       " 317,\n",
       " 46,\n",
       " 7,\n",
       " 4,\n",
       " 12118,\n",
       " 1029,\n",
       " 13,\n",
       " 104,\n",
       " 88,\n",
       " 4,\n",
       " 381,\n",
       " 15,\n",
       " 297,\n",
       " 98,\n",
       " 32,\n",
       " 2071,\n",
       " 56,\n",
       " 26,\n",
       " 141,\n",
       " 6,\n",
       " 194,\n",
       " 7486,\n",
       " 18,\n",
       " 4,\n",
       " 226,\n",
       " 22,\n",
       " 21,\n",
       " 134,\n",
       " 476,\n",
       " 26,\n",
       " 480,\n",
       " 5,\n",
       " 144,\n",
       " 30,\n",
       " 5535,\n",
       " 18,\n",
       " 51,\n",
       " 36,\n",
       " 28,\n",
       " 224,\n",
       " 92,\n",
       " 25,\n",
       " 104,\n",
       " 4,\n",
       " 226,\n",
       " 65,\n",
       " 16,\n",
       " 38,\n",
       " 1334,\n",
       " 88,\n",
       " 12,\n",
       " 16,\n",
       " 283,\n",
       " 5,\n",
       " 16,\n",
       " 4472,\n",
       " 113,\n",
       " 103,\n",
       " 32,\n",
       " 15,\n",
       " 16,\n",
       " 5345,\n",
       " 19,\n",
       " 178,\n",
       " 32]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data[0] #the train data is already preprocessed , that mean we only have the rank of the words based on frequency."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 2.Data preparation\n",
    "* You can’t feed lists of integers into a neural network. You have to turn your lists into tensors.We use One-hot encoding to convert the reviews into vectors of 0s and 1s."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def vectorizer(sequences, dimension=15000):\n",
    "    # Creates an all-zero matrix of shape (len(sequences), dimension)\n",
    "    results = np.zeros((len(sequences), dimension))\n",
    "    for i, sequence in enumerate(sequences):\n",
    "        results[i, sequence] = 1.\n",
    "    return results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x_train = vectorizer(train_data) # Vectorize training data\n",
    "x_test = vectorizer(test_data) # Vectorize test data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#vectorizing the labels \n",
    "y_train = np.asarray(train_labels).astype('float32')\n",
    "y_test = np.asarray(test_labels).astype('float32')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.Building the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model=models.Sequential()\n",
    "model.add(layers.Dense(64,activation='relu',input_shape=(15000,)))\n",
    "model.add(layers.Dense(32,activation='relu'))\n",
    "model.add(layers.Dense(1,activation='tanh'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# create a validation set by setting apart 10,000 samples from the original training data.\n",
    "x_val = x_train[:15000]\n",
    "partial_x_train = x_train[15000:]\n",
    "\n",
    "y_val = y_train[:15000]\n",
    "partial_y_train = y_train[15000:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 10000 samples, validate on 15000 samples\n",
      "Epoch 1/20\n",
      "10000/10000 [==============================] - 10s 972us/step - loss: 0.7668 - acc: 0.6109 - val_loss: 0.5013 - val_acc: 0.7859\n",
      "Epoch 2/20\n",
      "10000/10000 [==============================] - 4s 444us/step - loss: 0.3378 - acc: 0.8921 - val_loss: 0.3557 - val_acc: 0.8695\n",
      "Epoch 3/20\n",
      "10000/10000 [==============================] - 4s 420us/step - loss: 0.1883 - acc: 0.9414 - val_loss: 0.3429 - val_acc: 0.8781\n",
      "Epoch 4/20\n",
      "10000/10000 [==============================] - 4s 424us/step - loss: 0.1146 - acc: 0.9632 - val_loss: 0.4149 - val_acc: 0.8677\n",
      "Epoch 5/20\n",
      "10000/10000 [==============================] - 4s 440us/step - loss: 0.0709 - acc: 0.9501 - val_loss: 0.4558 - val_acc: 0.8346\n",
      "Epoch 6/20\n",
      "10000/10000 [==============================] - 4s 426us/step - loss: 0.0451 - acc: 0.9099 - val_loss: 0.4675 - val_acc: 0.7976\n",
      "Epoch 7/20\n",
      "10000/10000 [==============================] - 4s 416us/step - loss: 0.0310 - acc: 0.8741 - val_loss: 0.5376 - val_acc: 0.7493\n",
      "Epoch 8/20\n",
      "10000/10000 [==============================] - 4s 413us/step - loss: 0.0238 - acc: 0.8278 - val_loss: 0.5255 - val_acc: 0.7277\n",
      "Epoch 9/20\n",
      "10000/10000 [==============================] - 4s 420us/step - loss: 0.0190 - acc: 0.7803 - val_loss: 0.5997 - val_acc: 0.6670\n",
      "Epoch 10/20\n",
      "10000/10000 [==============================] - 5s 488us/step - loss: 0.0159 - acc: 0.7199 - val_loss: 0.6095 - val_acc: 0.6466\n",
      "Epoch 11/20\n",
      "10000/10000 [==============================] - 4s 443us/step - loss: 0.0140 - acc: 0.6897 - val_loss: 0.6483 - val_acc: 0.6211\n",
      "Epoch 12/20\n",
      "10000/10000 [==============================] - 5s 480us/step - loss: 0.0129 - acc: 0.6601 - val_loss: 0.6418 - val_acc: 0.6166\n",
      "Epoch 13/20\n",
      "10000/10000 [==============================] - 5s 465us/step - loss: 0.0120 - acc: 0.6441 - val_loss: 0.6738 - val_acc: 0.5998\n",
      "Epoch 14/20\n",
      "10000/10000 [==============================] - 5s 533us/step - loss: 0.0114 - acc: 0.6261 - val_loss: 0.6803 - val_acc: 0.5929\n",
      "Epoch 15/20\n",
      "10000/10000 [==============================] - 4s 446us/step - loss: 0.0107 - acc: 0.6182 - val_loss: 0.6914 - val_acc: 0.5871\n",
      "Epoch 16/20\n",
      "10000/10000 [==============================] - 4s 444us/step - loss: 0.0103 - acc: 0.6146 - val_loss: 0.6969 - val_acc: 0.5843\n",
      "Epoch 17/20\n",
      "10000/10000 [==============================] - 5s 473us/step - loss: 0.0100 - acc: 0.6040 - val_loss: 0.7226 - val_acc: 0.5727\n",
      "Epoch 18/20\n",
      "10000/10000 [==============================] - 4s 450us/step - loss: 0.0097 - acc: 0.5946 - val_loss: 0.7197 - val_acc: 0.5741\n",
      "Epoch 19/20\n",
      "10000/10000 [==============================] - 5s 478us/step - loss: 0.0096 - acc: 0.5911 - val_loss: 0.7363 - val_acc: 0.5711\n",
      "Epoch 20/20\n",
      "10000/10000 [==============================] - 5s 477us/step - loss: 0.0094 - acc: 0.5841 - val_loss: 0.7536 - val_acc: 0.5647\n"
     ]
    }
   ],
   "source": [
    "test_model = model.fit(partial_x_train,\n",
    "                    partial_y_train,\n",
    "                    epochs=20,\n",
    "                    batch_size=512,\n",
    "                    validation_data=(x_val, y_val))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. let's plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['val_loss', 'val_acc', 'loss', 'acc'])"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "history_dict = test_model.history\n",
    "history_dict.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl8FPX9x/HXRw4RQUFARVACSlVA\njpiiFisgavECb0GoYlWKlWpr+6tUrVJaWqtWLYoHtVIVlKKtShVFq3hWkXAqIIqIGkCuyiWoBD6/\nP76TZQmbZHNMNsf7+XjsIzuz35n5ZLOZz853voe5OyIiIgB7ZDoAERGpOpQUREQkQUlBREQSlBRE\nRCRBSUFERBKUFEREJEFJQSqUmdUxs81mdkhFls0kMzvMzCq87baZnWRmy5KWF5vZ99MpW4ZjPWhm\n15d1+2L2+3sz+3tF71cyp26mA5DMMrPNSYsNgW+A7dHyj919Ymn25+7bgUYVXbY2cPfDK2I/ZnY5\nMNjdeyXt+/KK2LfUfEoKtZy7J07K0TfRy939P0WVN7O67p5fGbGJSOVT9ZEUK6oe+IeZPW5mm4DB\nZnacmb1jZuvNbKWZjTGzelH5umbmZpYVLU+IXn/ezDaZ2dtm1ra0ZaPXTzWzD81sg5ndbWZvmdmQ\nIuJOJ8Yfm9kSM/vSzMYkbVvHzO40s3Vm9jHQt5j350Yzm1Ro3VgzuyN6frmZLYp+n4+jb/FF7SvP\nzHpFzxua2aNRbAuAo1Mcd2m03wVm1i9afxRwD/D9qGpubdJ7OzJp+2HR777OzJ42s5bpvDclMbOz\nonjWm9krZnZ40mvXm9kKM9toZh8k/a7HmtnsaP0qM7st3eNJDNxdDz1wd4BlwEmF1v0e+BY4k/Al\nYi/gu8AxhCvNdsCHwPCofF3AgaxoeQKwFsgB6gH/ACaUoez+wCagf/TatcA2YEgRv0s6MT4D7Atk\nAf8r+N2B4cACoDXQDHg9/KukPE47YDOwd9K+VwM50fKZURkDTgS2Ap2j104CliXtKw/oFT2/HXgV\naAq0ARYWKnsB0DL6m1wUxXBA9NrlwKuF4pwAjIyenxLF2BVoANwLvJLOe5Pi9/898Pfo+ZFRHCdG\nf6Pro/e9HtAR+BQ4MCrbFmgXPZ8JDIyeNwaOyfT/Qm1+6EpB0vGmu//b3Xe4+1Z3n+nuM9w9392X\nAuOAnsVs/6S757r7NmAi4WRU2rJnAHPd/ZnotTsJCSSlNGP8o7tvcPdlhBNwwbEuAO509zx3Xwfc\nUsxxlgLvE5IVwMnAenfPjV7/t7sv9eAV4GUg5c3kQi4Afu/uX7r7p4Rv/8nHnezuK6O/yWOEhJ6T\nxn4BBgEPuvtcd/8aGAH0NLPWSWWKem+KMwCY4u6vRH+jW4B9CMk5n5CAOkZVkJ9E7x2E5N7ezJq5\n+yZ3n5Hm7yExUFKQdHyevGBmR5jZc2b2hZltBEYBzYvZ/ouk51so/uZyUWUPSo7D3Z3wzTqlNGNM\n61iEb7jFeQwYGD2/iJDMCuI4w8xmmNn/zGw94Vt6ce9VgZbFxWBmQ8xsXlRNsx44Is39Qvj9Evtz\n943Al0CrpDKl+ZsVtd8dhL9RK3dfDPyC8HdYHVVHHhgVvRToACw2s3fN7LQ0fw+JgZKCpKNwc8wH\nCN+OD3P3fYCbCNUjcVpJqM4BwMyMXU9ihZUnxpXAwUnLJTWZ/QdwUvRNuz8hSWBmewFPAn8kVO00\nAV5MM44viorBzNoB9wFXAs2i/X6QtN+Sms+uIFRJFeyvMaGaankacZVmv3sQ/mbLAdx9grv3IFQd\n1SG8L7j7YncfQKgi/DPwTzNrUM5YpIyUFKQsGgMbgK/M7Ejgx5VwzGeBbDM708zqAtcALWKKcTLw\nMzNrZWbNgOuKK+zuq4A3gfHAYnf/KHppT6A+sAbYbmZnAH1KEcP1ZtbEQj+O4UmvNSKc+NcQ8uPl\nhCuFAquA1gU31lN4HLjMzDqb2Z6Ek/Mb7l7klVcpYu5nZr2iY/8f4T7QDDM70sx6R8fbGj22E36B\nH5pZ8+jKYkP0u+0oZyxSRkoKUha/AC4h/MM/QPimHKvoxHshcAewDjgUmEPoV1HRMd5HqPt/j3AT\n9Mk0tnmMcOP4saSY1wM/B54i3Kw9j5Dc0nEz4YplGfA88EjSfucDY4B3ozJHAMn18C8BHwGrzCy5\nGqhg+xcI1ThPRdsfQrjPUC7uvoDwnt9HSFh9gX7R/YU9gVsJ94G+IFyZ3BhtehqwyELrttuBC939\n2/LGI2VjoWpWpHoxszqE6orz3P2NTMcjUlPoSkGqDTPra2b7RlUQvyG0aHk3w2GJ1ChKClKdHA8s\nJVRB9AXOcveiqo9EpAxirT4ys77AXwgtDR5091sKvX4I8DDQJCozwt2nxhaQiIgUK7akENX5fkjo\nzJPHzl6LC5PKjAPmuPt9ZtYBmOruWbEEJCIiJYpzQLzuwJKCXovR+DD9Cd31CzihxyOELvUrStpp\n8+bNPSsrq2IjFRGp4WbNmrXW3Ytrxg3EmxRasWuPzDxCd/dkI4EXzeynwN6EJn3FysrKIjc3t6Ji\nFBGpFcyspJ75QLw3mlP12ixcVzWQMJhWa0Jb5UejXpC77shsqJnlmlnumjVrYghVREQg3qSQx67d\n9Fuze/XQZYRekLj724QBs3Ybv8Xdx7l7jrvntGhR4tWPiIiUUZxJYSZh5MO2ZlafaATFQmU+I+r2\nHw1F0IDQE1JERDIgtqTgYXau4cA0YBEw2d0XmNmogglBCEMRXGFm8wjjsQxxdbEWEcmYWKfjjPoc\nTC207qak5wuBHnHGICIi6asVPZonToSsLNhjj/BzYqmmohcRqT1ivVKoCiZOhKFDYcuWsPzpp2EZ\nYFC5x4UUEalZavyVwg037EwIBbZsCetFRGRXNT4pfPZZ6daLiNRmNT4pHFLERIpFrRcRqc1qfFIY\nPRoaNtx1XcOGYb2ISHWxYQN89VX8x6nxSWHQIBg3Dtq0AbPwc9w43WQWkaptxw6YMwf++Ec44QRo\n1gwmT47/uDW+9RGEBKAkICJV3bp18NJL8MIL4bFqVVifnQ3XXQfdu8cfQ61ICiIiVdH27ZCbGxLA\n88/Du++CO+y3H/zgB9C3b/h5wAGVF5OSgohIJVq1CqZNC0ngxRfhf/8LVdvHHAM33xwSQU4O1KmT\nmfiUFEREYvTtt/Df/4ZqoeefD/cJIHz7P/PMkAROPjncM6gKlBRERCqQOyxcGJLASy/Bq6+GDrN1\n6sD3vgd/+ENIBF26hKF3qholBRGRclq1Cv7zn52JYEU0c8x3vgOXXgqnnAK9esE++xS7mypBSUFE\npJS2boU33tiZBObNC+ubNYM+fUISOPnk6tlJVklBRGq0hQvh3nth0qTQ9r9RI2jcOL2fyc/r1t15\nb+CNN+Cbb6B+fejRI1QJnXIKdOtWNauESkNJQURqnPx8mDIF7rkHpk8PJ+9zzoEWLWDTJti8eefP\nNWt2Xd66tfh9d+oEP/lJSALf/z7svXfl/E6VRUlBRGqMVavgwQfh/vshLy9U3/zhD3D55SEhpCM/\nPwwnUTh5bN0KXbvCQQfF+ztkmpKCiFRr7vDOO+Gq4IknYNu2UJ9/zz1wxhmlb+9fty7su2941Eax\nJgUz6wv8BagDPOjutxR6/U6gd7TYENjf3ZvEGZOI1AxbtsDjj8PYsaHt/z77wJVXhqqdww/PdHTV\nV2xJwczqAGOBk4E8YKaZTYnmZQbA3X+eVP6nQLe44hGRmuHjj+G+++Chh+DLL0Md//33h/HNGjXK\ndHTVX5xXCt2BJe6+FMDMJgH9gYVFlB8I3BxjPCJSzbiHYSBWrYIPP4S//jX0Cq5TJ9w4vuqqcLPX\nLNOR1hxxJoVWwOdJy3nAMakKmlkboC3wShGvDwWGAhxSHRv+ikjCjh3hRP/FF+Fkn+pR8Nrq1eHG\nb4EDD4SbbgrzrNf0G76ZEmdSSJW7vYiyA4An3X17qhfdfRwwDiAnJ6eofYhIFbJqFcyfv/OxYEHo\n6bt6dRgdtLB69cJ4QAccAC1bhpY+BcsHHhjWHXdcaF4q8YkzKeQBBycttwZWFFF2AHBVjLGISEy+\n+QYWLQon/nnzdiaB1at3ljnooFD3n3yiL/xo2lTVQFVBnElhJtDezNoCywkn/osKFzKzw4GmwNsx\nxiIi5eQOy5fv+u1//nz44IOd3/wbNAgn/zPOgM6dw+Ooo6B588zGLumLLSm4e76ZDQemEZqkPuTu\nC8xsFJDr7lOiogOBSe6uaiGRDHEPLXmWLw+dvpYv3/35Z5/B+vU7t2nTJpz0zzprZwJo3z5z8wBI\nxbDqdi7Oycnx3NzcTIchUq24h7b8n3yS+oS/fPnuwzuYwf77Q6tW0Lp1eHTsGE7+nTpBE/UoqlbM\nbJa755RUTj2aRWq4l1+GG26AGTN2rqtfP5zsW7UKs3z177/z5F+wvmVL3dStjZQURGqot98OyWD6\n9HCyv/fe0HqnVatQx6+bupKKkoJIDTN3LvzmN/Dss6H656674Mc/DjeBRUpSzUf+FpECixfDhReG\nMf3ffBNGjw5DQlxzjRKCpE9XCiLV3LJl8NvfwiOPwF57hSqjX/5SN4KlbJQURKqplSvD1cC4cWG2\nr2uugREjQpWRSFkpKYhUgqVLw+QvjRrt2sKnVasw3WNprFsHf/pTmC9g2za47DK48cawX5HyUlIQ\nidG2bXDnnTByZBgOYseO3cs0brxrkih4JCeP/fcPs4HdeSf8+c9hJrBBg8J+Dz20sn8rqcmUFERi\nMnMmXHFFGA/orLPg7rthv/127TBW+DF9eqgWSh4ZFEIv4fr1Qwezc86BUaNCRzKRiqakIFLBNm0K\nTULvvjuM7vmvf8HZZ+98vX378CjKjh1hMLnCCWPjRrj44tDZTCQuSgoiFejf/w4Tv+TlhWkhR48u\n/Vy/e+wRksmBB8LRR8cTp0hR1E9BpAKsXAnnnw/9+oUk8NZb4UZwbZ38XaovJQWRctixAx54AI48\nMlwljB4Ns2aF4SREqiNVH4mU0cKFYVrIt96C3r1DcijuXoFIdaArBZFS+vpruPnmMIvYokUwfnwY\niVQJQWoCXSmIlMJrr4XB5RYvhsGD4Y47oEWLTEclUnGUFESKkZ8fJqZZtAiefjpcFbRtC9OmwSmn\nZDo6kYoXa1Iws77AXwjTcT7o7rekKHMBMBJwYJ677zaPs0jctm4N3/4XLdr5+OAD+PBD+PbbUKZO\nHfjVr0LVUcOGmY1XJC6xJQUzqwOMBU4G8oCZZjbF3RcmlWkP/Bro4e5fmpmG8pJYrVu384SfnAA+\n/TRMWQmhn0C7dqFF0amnhp8FDzUxlZouziuF7sASd18KYGaTgP7AwqQyVwBj3f1LAHdfHWM8Uou9\n9loYK2j58p3rGjSAww+HY4+FSy+FI44IJ/727TX/gNRecSaFVsDnSct5wDGFynwHwMzeIlQxjXT3\nFwrvyMyGAkMBDjnkkFiClZrr44/DeEHNm8Ptt4cT/xFHQJs2oUpIRHaKMymkmgHWUxy/PdALaA28\nYWad3H39Lhu5jwPGAeTk5BTeh0iRNm6EM88Mz6dO1YiiIiWJs59CHnBw0nJrYEWKMs+4+zZ3/wRY\nTEgSIuW2fTtcdBF89BE8+aQSgkg64kwKM4H2ZtbWzOoDA4Aphco8DfQGMLPmhOqkpTHGJLXIr38N\nzz0XRivt3TvT0YhUD7ElBXfPB4YD04BFwGR3X2Bmo8ysX1RsGrDOzBYC04H/c/d1ccUktccjj8Bt\nt4WRSocNy3Q0ItWHuVevKvqcnBzPzc3NdBhShb39NvTqBccfDy+8APXqZToikcwzs1nuXuJsHBr7\nSGqUzz8PE9ocfDA88YQSgkhpaZgLqTG++gr69w+9k6dPD1NfikjpKClIjbBjBwwZAnPnwrPPhr4I\nIlJ6SgpSI/zud6HZ6e23w2mnZToakepL9xSk2nviCRg5MlwpXHttpqMRqd6UFKRamzMHLrkEvvc9\nuP9+sFT96EUkbUoKUm198QX06xfGNPrXv2DPPTMdkUj1p3sKUi19/XVoevq//4U5kg84INMRidQM\nSgpS7biHKTHfeSfcXO7aNdMRidQcSgpSKT78EO68M3QqO/poyM4u+9zGt98ehrEYNQrOPbdi4xSp\n7ZQUJHZffAE/+AGsWLFzakvYNUEcfXR4lFQN9NxzcN11cMEFcOON8cYtUhspKUisvvoKzjgDVq8O\ndf+HHRZaDM2eDbNmhcfTT+8sf9BBOxNEQbI46KDw2oIFMHAgdOsG48erpZFIHJQUJDb5+TBgQEgC\nzzwDOdFQXL177zqU9caNoSfyrFk7k8Wzz+6cM/nAA0OCWLAA9t477Kthw8r/fURqAyUFiYU7XH11\nOLnfd1+4WijKPvvACSeER4HNm2HevJ1XE7Nnh30+9RS0bh1//CK1lZKCxOLWW0MyuO66ss1n0KgR\n9OgRHiJSedR5TSrc44/DiBGh/v8Pf8h0NCJSGkoKUqFeey2MQdSzZ7gZvIc+YSLVSqz/smbW18wW\nm9kSMxuR4vUhZrbGzOZGj8vjjEfitWgRnHUWHHpoqPvXsBMi1U9sScHM6gBjgVOBDsBAM+uQoug/\n3L1r9Hgwrni+/jpMvCLx+OILOPXUkAimToWmTTMdkYiURZxXCt2BJe6+1N2/BSYB/WM8XrFGj4aT\nT4a8vExFUHNt3gynnw5r1oTOZVlZmY5IRMoqzqTQCvg8aTkvWlfYuWY238yeNLODU+3IzIaaWa6Z\n5a5Zs6ZMwVx2WZid64EHyrS5FKGgL8LcuTB5cuhsJiLVV5xJIVV/Uy+0/G8gy907A/8BHk61I3cf\n5+457p7ToowD5mRlwZlnwrhx8M03ZdqFFOIOP/1puDq4995wtSAi1VucSSEPSP7m3xpYkVzA3de5\ne8Ep+q9ArN8zhw8Pwy08+WScR6k9br01TGwzYkQYtVREqr84k8JMoL2ZtTWz+sAAYEpyATNrmbTY\nD1gUYzz06QOHHw733BPnUWqHgr4IF10U7teISM0QW1Jw93xgODCNcLKf7O4LzGyUmfWLil1tZgvM\nbB5wNTAkrnggtJn/yU/COPyzZsV5pJotuS/CQw+pL4JITWLuhav5q7acnBzPzc0t8/YbNkCrVmHo\n5YceqsDAaolFi8J8yC1bhlFP1fRUpHows1nunlNSuVr3HW/ffeHii+Gxx2DdukxHU70U9EVo0ACe\nf14JQaQmqnVJAeCqq0ILpL/9LdORVA9ffQVvvx1aF61dG0Y+bdMm01GJSBxq5SipHTuG8fzvvRd+\n8QuoUyfTEVUda9eG+Q/mzAl9D+bMCVNp7tgBdeuGuQzUF0Gk5qqVSQFC89Rzzw1t7Pv1K7l8TeMO\nn366MwEUJIHkHt+HHAJdu8KFF4bZzrp3D/cSRKTmqrVJoV+/MFnLPffUjqSwZUsYpG7WrJ0JYP36\n8Noee8ARR4TWRF27hgTQtSs0a5bZmEWk8tXapFC3Llx5JdxwA3zwQTgp1lSvvw4/+hF8/HG4Sdy5\nc/j2X5AAjjpK01uKSFArbzQXuPxyqF8/3Fuoib76KkyJ2bNnuCcwbRps2gQzZoSeyMOGwTHHKCGI\nyE619koBYP/9Q3+Fv/899Mpt3DjTEVWc114LVwdLl4bxif74xzDpvUhpbdu2jby8PL7++utMhyJp\naNCgAa1bt6ZevXpl2r5WJwUIN5wnTIBHHw29nau7zZvh178O90ratYNXXw1XCiJllZeXR+PGjcnK\nysIs1TiXUlW4O+vWrSMvL4+2bduWaR+1uvoIQouanBwYOza0yKnOXn013C+4555QbTR/vhKClN/X\nX39Ns2bNlBCqATOjWbNm5bqqq/VJwSxcLSxcGE6q1dHmzeF36N07tCR67TX4y19UXSQVRwmh+ijv\n36rWJwUILXGaNaueo6dOnx5aD917L/zsZ+Hq4IQTMh2VSMVZt24dXbt2pWvXrhx44IG0atUqsfzt\nt9+mtY9LL72UxYsXF1tm7NixTJw4sSJC5vjjj2fu3LkVsq/KVuvvKUBopnnFFWF+gM8+C522qrrN\nm+FXv4L77oPDDgvNTo8/PtNRicDEiaGpd8H/0ujRMGhQ2ffXrFmzxAl25MiRNGrUiF/+8pe7lHF3\n3J09ihiyd/z48SUe56qrrip7kDWIrhQiw4aFn9Vhus5XXglXB/ffDz//Ocybp4QgVcPEiTB0aOgt\nX9BrfujQsL6iLVmyhE6dOjFs2DCys7NZuXIlQ4cOJScnh44dOzJq1KhE2YJv7vn5+TRp0oQRI0bQ\npUsXjjvuOFavXg3AjTfeyF133ZUoP2LECLp3787hhx/Of//7XwC++uorzj33XLp06cLAgQPJyckp\n8YpgwoQJHHXUUXTq1Inrr78egPz8fH74wx8m1o8ZMwaAO++8kw4dOtClSxcGDx5c4e9ZOpQUIm3a\n7Jyus6q2vNu0KXS469MH6tWDN96AO+5QPwOpOm64IfSeT7ZlS1gfh4ULF3LZZZcxZ84cWrVqxS23\n3EJubi7z5s3jpZdeYuHChbtts2HDBnr27Mm8efM47rjjeKiIMfTdnXfffZfbbrstkWDuvvtuDjzw\nQObNm8eIESOYM2dOsfHl5eVx4403Mn36dObMmcNbb73Fs88+y6xZs1i7di3vvfce77//PhdffDEA\nt956K3PnzmXevHnck6H67LSSgpkdamZ7Rs97mdnVZtYk3tAq3/DhYUC4J57IdCS7mz8/XB088ABc\ne20YpqJHj0xHJbKrzz4r3fryOvTQQ/nud7+bWH788cfJzs4mOzubRYsWpUwKe+21F6eeeioARx99\nNMuWLUu573POOWe3Mm+++SYDBgwAoEuXLnTs2LHY+GbMmMGJJ55I8+bNqVevHhdddBGvv/46hx12\nGIsXL+aaa65h2rRp7LvvvgB07NiRwYMHM3HixDL3MyivdK8U/glsN7PDgL8BbYHHYosqQ6rqdJ3L\nl8Npp0F+Prz5Jvz5z7o6kKqpqPtxcd2n2zupid1HH33EX/7yF1555RXmz59P3759UzbNrF+/fuJ5\nnTp1yM/PT7nvPffcc7cypZ2UrKjyzZo1Y/78+Rx//PGMGTOGH0eTnE+bNo1hw4bx7rvvkpOTw/bt\n20t1vIqQblLYEU2veTZwl7v/HChxvEwz62tmi81siZmNKKbceWbmZlbirEBxKmie+u674VEVbN4c\nqrU2bICpU8OsZyJV1ejRu39hadiwcubx3rhxI40bN2afffZh5cqVTJs2rcKPcfzxxzN58mQA3nvv\nvZRXIsmOPfZYpk+fzrp168jPz2fSpEn07NmTNWvW4O6cf/75/Pa3v2X27Nls376dvLw8TjzxRG67\n7TbWrFnDlsJ1cZUg3dZH28xsIHAJcGa0rthrGzOrA4wFTgbygJlmNsXdFxYq15gwP/OM0gQel4sv\nDj2Cx44NHdsyafv20Gpj3jz4979DxzSRqqyglVFFtj5KV3Z2Nh06dKBTp060a9eOHjHUr/70pz/l\n4osvpnPnzmRnZ9OpU6dE1U8qrVu3ZtSoUfTq1Qt358wzz+T0009n9uzZXHbZZbg7Zsaf/vQn8vPz\nueiii9i0aRM7duzguuuuo3Emxt4paMpV3APoAIwBBkbLbYERJWxzHDAtafnXwK9TlLsLOAN4Fcgp\nKZajjz7a43bVVe577um+enXshyrWL37hDu733JPZOKR2W7hwYaZDqDK2bdvmW7dudXf3Dz/80LOy\nsnzbtm0Zjmp3qf5mQK6ncb5Pq/rI3Re6+9Xu/riZNQUau/stJWzWCvg8aTkvWpdgZt2Ag9392eJ2\nZGZDzSzXzHLXrFmTTsjlUhWm63zggXDv4OqrQzwiknmbN2+mR48edOnShXPPPZcHHniAunVrVnev\ntH4bM3sV6BeVnwusMbPX3P3a4jZLsS5x18XM9gDuBIaUdHx3HweMA8jJyYl9hKIjj4QTTwwdw375\nyzD3QmV68cWQCE4/PTQ5FZGqoUmTJsyaNSvTYcQq3RvN+7r7RuAcYLy7Hw2cVMI2ecDBScutgRVJ\ny42BTsCrZrYMOBaYkumbzQWGDw91os8Wew1T8RYsgPPPD/NIP/645o8WkcqVblKoa2YtgQuAdE+T\nM4H2ZtbWzOoDA4ApBS+6+wZ3b+7uWe6eBbwD9HP33PTDj8+ZZ8LBB1du89RVq8LVQcOGIRnVpPkd\nRKR6SDcpjAKmAR+7+0wzawd8VNwGHpqwDo+2WwRMdvcFZjbKzKr8rMgF03W+/DIsWhT/8bZuhbPO\ngtWrQ0ujgw8ueRsRkYqW7o3mJ9y9s7tfGS0vdfdz09huqrt/x90PdffR0bqb3H1KirK9qspVQoGC\n6TrHjo33ODt2wJAhYZrMxx4L8zuIiGRCusNctDazp8xstZmtMrN/mlnruIPLtBYtYMAAePhh2Lgx\nvuPcdBNMnhxGaT3rrPiOI1Id9erVa7eOaHfddRc/KWGqxEaNGgGwYsUKzjvvvCL3nZtb/HfRu+66\na5dOZKeddhrr169PJ/RijRw5kttvv73c+6lo6VYfjSfcDziI0Kz039G6Gm/48NCr+JFH4tn/ww+H\nzj1XXAG/+EU8xxCpzgYOHMikSZN2WTdp0iQGDhyY1vYHHXQQTz75ZJmPXzgpTJ06lSZNatzQbwnp\nJoUW7j7e3fOjx9+BFjHGVWV897uhZ3Mc03W+9lpIBn36hP1rciuR3Z133nk8++yzfPPNNwAsW7aM\nFStWcPzxx7N582b69OlDdnY2Rx11FM8888xu2y9btoxOnToBsHXrVgYMGEDnzp258MIL2bp1a6Lc\nlVdemRh2++abbwZgzJgxrFixgt69e9O7d28AsrKyWLt2LQB33HEHnTp1olOnTolht5ctW8aRRx7J\nFVdcQceOHTnllFN2OU4qc+fO5dhjj6Vz586cffbZfPnll4njd+jQgc6dOycG4nvttdcSkwx169aN\nTZs2lfm9TSXdFvhrzWww8Hi0PBBYV6GRVGFXXQWXXBLmMejTp2L2+eGHcPbZcOih8OSTYShskaru\nZz8LI/RWpK5dITqfptSsWTO6d+/OCy+8QP/+/Zk0aRIXXnghZkaDBg146qmn2GeffVi7di3HHnss\n/fr1K3JKyvvuu4+GDRsyf/7TZOv7AAAS0UlEQVR85s+fT3Z2duK10aNHs99++7F9+3b69OnD/Pnz\nufrqq7njjjuYPn06zZs332Vfs2bNYvz48cyYMQN355hjjqFnz540bdqUjz76iMcff5y//vWvXHDB\nBfzzn/8sdn6Eiy++mLvvvpuePXty00038dvf/pa77rqLW265hU8++YQ999wzUWV1++23M3bsWHr0\n6MHmzZtp0KBBKd7tkqV7pfAjQnPUL4CVwHnApRUaSRV2wQXQvHn44BYxoGKprFsXmp7WrQvPPQc1\n+EpUpEIkVyElVx25O9dffz2dO3fmpJNOYvny5axatarI/bz++uuJk3Pnzp3pnDSg2OTJk8nOzqZb\nt24sWLCgxMHu3nzzTc4++2z23ntvGjVqxDnnnMMbb7wBQNu2benatStQ/PDcEOZ3WL9+PT179gTg\nkksu4fXXX0/EOGjQICZMmJDoOd2jRw+uvfZaxowZw/r16yu8R3Vae3P3zwg9mhPM7GeEcYtqvAYN\nwsxsv/897LMPdOsWqpUKqpYOOyz9qp9vvglXCJ9/Hq482rWLN3aRilTcN/o4nXXWWVx77bXMnj2b\nrVu3Jr7hT5w4kTVr1jBr1izq1atHVlZWyuGyk6W6ivjkk0+4/fbbmTlzJk2bNmXIkCEl7seLqU8u\nGHYbwtDbJVUfFeW5557j9ddfZ8qUKfzud79jwYIFjBgxgtNPP52pU6dy7LHH8p///IcjjjiiTPtP\npTwzrxU3xEWNc/PNoYfxj38cEsC4cTB4MHznO7DffnDyyXD99fD002H+g1Tcw9SEb7wB48drGGyR\ndDVq1IhevXrxox/9aJcbzBs2bGD//fenXr16TJ8+nU8//bTY/ZxwwglMjOYGff/995k/fz4Qht3e\ne++92XfffVm1ahXPP/98YpvGjRunrLc/4YQTePrpp9myZQtfffUVTz31FN///vdL/bvtu+++NG3a\nNHGV8eijj9KzZ0927NjB559/Tu/evbn11ltZv349mzdv5uOPP+aoo47iuuuuIycnhw8++KDUxyxO\nea47as1t0VQTkV94YRiSYubMnY9bbw3DXQO0bLnr1UROThhL6ZFHYNQoSLPhhIhEBg4cyDnnnLNL\nS6RBgwZx5plnkpOTQ9euXUv8xnzllVdy6aWX0rlzZ7p27Ur3aHz8Ll260K1bNzp27LjbsNtDhw7l\n1FNPpWXLlkyfPj2xPjs7myFDhiT2cfnll9OtW7diq4qK8vDDDzNs2DC2bNlCu3btGD9+PNu3b2fw\n4MFs2LABd+fnP/85TZo04Te/+Q3Tp0+nTp06dOjQITGLXEWx4i6Bit3Q7DN3j2k+paLl5OR4Se2K\nK1LBROTJc100bBiuFAqPEb91a7gJl5woFi/etczgwSExqKWRVBeLFi3iyCOPzHQYUgqp/mZmNsvd\nS+waW+yVgpltImlk0+SXgL1KE2R1VdxE5IWTwl57wXHHhUeB9eth1qyQILZtg1/9SglBRKquYpOC\nu9f6IdnKOxF5kyahGWtFNWUVEYlTeW401wqVPRG5iEgmKSmUIJMTkYtUFWW99yiVr7x/KyWFEgwa\nFG4qt2kT7gW0aZP6JrNITdWgQQPWrVunxFANuDvr1q0rVy/nMrc+ypTKbn0kUttt27aNvLy8Ejtz\nSdXQoEEDWrduTb1CY+dUSOsjEZF69erRtm3bTIchlUTVRyIikhBrUjCzvma22MyWmNmIFK8PM7P3\nzGyumb1pZh3ijEdERIoXW1IwszrAWOBUoAMwMMVJ/zF3P8rduwK3AnfEFY+IiJQsziuF7sCSaD7n\nb4FJQP/kAu6ePMnl3qTuPS0iIpUkzhvNrYDPk5bzgGMKFzKzqwgjrtYHTky1IzMbCgwFOES9xkRE\nYhPnlUKqEX52uxJw97HufihwHXBjqh25+zh3z3H3nBYtasUsoCIiGRFnUsgDDk5abg2sKKb8JOCs\nGOMREZESxJkUZgLtzaytmdUHBgBTkguYWfukxdOBj2KMR0REShDbPQV3zzez4cA0oA7wkLsvMLNR\nQK67TwGGm9lJwDbgS+CSuOIREZGSxdqj2d2nAlMLrbsp6fk1cR5fRERKRz2aRUQkQUlBREQSlBRE\nRCRBSUFERBKUFEREJEFJQUREEpQUREQkQUlBREQSlBRERCRBSUFERBKUFEREJEFJQUREEpQUREQk\nQUlBREQSlBRERCRBSUFERBKUFEREJCHWpGBmfc1ssZktMbMRKV6/1swWmtl8M3vZzNrEGY+IiBQv\ntqRgZnWAscCpQAdgoJl1KFRsDpDj7p2BJ4Fb44pHRERKFueVQndgibsvdfdvgUlA/+QC7j7d3bdE\ni+8ArWOMR0REShBnUmgFfJ60nBetK8plwPOpXjCzoWaWa2a5a9asqcAQK8fEiZCVBXvsEX5OnJjp\niEREUqsb474txTpPWdBsMJAD9Ez1uruPA8YB5OTkpNxHVTVxIgwdClui66FPPw3LAIMGZS4uEZFU\n4rxSyAMOTlpuDawoXMjMTgJuAPq5+zcxxpMRN9ywMyEU2LIlrBcRqWriTAozgfZm1tbM6gMDgCnJ\nBcysG/AAISGsjjGWjPnss9KtFxHJpNiSgrvnA8OBacAiYLK7LzCzUWbWLyp2G9AIeMLM5prZlCJ2\nV20dckjp1ouIZFKc9xRw96nA1ELrbkp6flKcx68KRo/e9Z4CQMOGYb2ISFWjHs0xGzQIxo2DNm3A\nLPwcN043mUWkaor1SkGCQYOUBESketCVgoiIJCgpiIhIgpKCiIgkKCmIiEiCkoKIiCQoKYiISIKS\ngoiIJCgpiIhIgpKCiIgkKCmIiEiCkoKIiCQoKVQDms5TRCqLBsSr4jSdp4hUJl0pVHGazlNEKpOS\nQhWn6TxFpDLFmhTMrK+ZLTazJWY2IsXrJ5jZbDPLN7Pz4oylutJ0niJSmWJLCmZWBxgLnAp0AAaa\nWYdCxT4DhgCPxRVHdTd6dJi+M5mm8xSRuMR5pdAdWOLuS939W2AS0D+5gLsvc/f5wI4Y46jWNJ2n\niFSmOFsftQI+T1rOA44py47MbCgwFOCQWlhvouk8RaSyxHmlYCnWeVl25O7j3D3H3XNatGhRzrBE\nRKQocSaFPODgpOXWwIoYjyciIuUUZ1KYCbQ3s7ZmVh8YAEyJ8XgiIlJOsSUFd88HhgPTgEXAZHdf\nYGajzKwfgJl918zygPOBB8xsQVzxiIhIyWLtp+DuU939O+5+qLuPjtbd5O5Toucz3b21u+/t7s3c\nvWOc8dRWGjtJRNKlsY9qOI2dJCKloWEuajiNnSQipaGkUMNp7CQRKQ0lhRpOYyeJSGkoKdRwGjtJ\nREpDSaGG09hJIlIaSgq1wKBBsGwZ7NgRfpY2IahJq0jtoSapUiw1aRWpXXSlIMVSk1aR2kVJQYpV\nEU1aVf0kUn0oKUixytuktaD66dNPwX1n9ZMSg0jVpKQgxSpvk9aKqH7SlYZI5VFSkGKVt0lreauf\nKuJKQ0lFJH1KClKi8jRpLW/1U3mvNJRUREpHSUFiVd7qp/JeadSUpFLefWR6e6lG3L1aPY4++miX\n6mXCBPc2bdzNws8JE9Lftk0b93A63vXRpk1625ul3t6sco4/YYJ7w4a7btuwYeneg/LuI9PbF+yj\nrJ8BbV/+7d3dgVxP4xyb8ZN8aR9KCrVLeU9I1T2pVMQ+Mr19ppNSbd++QJVICkBfYDGwBBiR4vU9\ngX9Er88Askrap5JC7VOeb0nVPalUxD4yvX2mk1Jt375AxpMCUAf4GGgH1AfmAR0KlfkJcH/0fADw\nj5L2q6QgpVWdk0pF7CPT22c6KdX27QukmxTivNHcHVji7kvd/VtgEtC/UJn+wMPR8yeBPmZmMcYk\ntVB5Wk+Vt0luRQxdXt59ZHr78rZA0/bl277U0skcZXkA5wEPJi3/ELinUJn3gdZJyx8DzVPsayiQ\nC+QecsghpUuPIhlWETcJM32jMpNXW9q+htxTAM5PkRTuLlRmQYqk0Ky4/ar6SKT6qc5JrSZs755+\nUrBQtuKZ2XHASHf/QbT86+jK5I9JZaZFZd42s7rAF0ALLyaonJwcz83NjSVmEZGaysxmuXtOSeXi\nvKcwE2hvZm3NrD7hRvKUQmWmAJdEz88DXikuIYiISLxim2TH3fPNbDgwjdAS6SF3X2BmowiXMVOA\nvwGPmtkS4H+ExCEiIhkS68xr7j4VmFpo3U1Jz78m3HsQEZEqQGMfiYhIgpKCiIgkxNb6KC5mtgb4\nNNNxFKE5sDbTQRRD8ZVPVY8Pqn6Miq98yhNfG3dvUVKhapcUqjIzy02nyVemKL7yqerxQdWPUfGV\nT2XEp+ojERFJUFIQEZEEJYWKNS7TAZRA8ZVPVY8Pqn6Miq98Yo9P9xRERCRBVwoiIpKgpCAiIglK\nCqVkZgeb2XQzW2RmC8zsmhRlepnZBjObGz1uSrWvGGNcZmbvRcfebUhZC8aY2RIzm29m2ZUY2+FJ\n78tcM9toZj8rVKbS3z8ze8jMVpvZ+0nr9jOzl8zso+hn0yK2vSQq85GZXZKqTAyx3WZmH0R/v6fM\nrEkR2xb7WYg5xpFmtjzp73haEdv2NbPF0edxRCXG94+k2JaZ2dwito31PSzqnJKxz18642vrscsc\nEC2B7Oh5Y+BDdp9mtBfwbAZjXEaKyYqSXj8NeB4w4FhgRobirEMYLr1Npt8/4AQgG3g/ad2tRHOL\nAyOAP6XYbj9gafSzafS8aSXEdgpQN3r+p1SxpfNZiDnGkcAv0/gMFDttb1zxFXr9z8BNmXgPizqn\nZOrzpyuFUnL3le4+O3q+CVgEtMpsVKXWH3jEg3eAJmbWMgNx9AE+dveM91B399cJI/UmS54u9mHg\nrBSb/gB4yd3/5+5fAi8BfeOOzd1fdPf8aPEdoHVFHrO0inj/0pHOtL3lVlx80RTAFwCPV/Rx01HM\nOSUjnz8lhXIwsyygGzAjxcvHmdk8M3vezDpWamDgwItmNsvMhqZ4vRXwedJyHplJbAMo+h8xk+9f\ngQPcfSWEf1xg/xRlqsJ7+SPClV8qJX0W4jY8quJ6qIjqj6rw/n0fWOXuHxXxeqW9h4XOKRn5/Ckp\nlJGZNQL+CfzM3TcWenk2oUqkC3A38HQlh9fD3bOBU4GrzOyEQq9bim0qtW2yhYmX+gFPpHg50+9f\naWT0vTSzG4B8YGIRRUr6LMTpPuBQoCuwklBFU1jGP4vAQIq/SqiU97CEc0qRm6VYV673T0mhDMys\nHuGPN9Hd/1X4dXff6O6bo+dTgXpm1ryy4nP3FdHP1cBThEv0ZHnAwUnLrYEVlRNdwqnAbHdfVfiF\nTL9/SVYVVKtFP1enKJOx9zK6qXgGMMijCubC0vgsxMbdV7n7dnffAfy1iGNn9LNoYRrgc4B/FFWm\nMt7DIs4pGfn8KSmUUlT/+DdgkbvfUUSZA6NymFl3wvu8rpLi29vMGhc8J9yQfL9QsSnAxVErpGOB\nDQWXqZWoyG9nmXz/CkmeLvYS4JkUZaYBp5hZ06h65JRoXazMrC9wHdDP3bcUUSadz0KcMSbfpzq7\niGOnM21vnE4CPnD3vFQvVsZ7WMw5JTOfv7juqNfUB3A84fJsPjA3epwGDAOGRWWGAwsILSneAb5X\nifG1i447L4rhhmh9cnwGjCW0+ngPyKnk97Ah4SS/b9K6jL5/hAS1EthG+PZ1GdAMeBn4KPq5X1Q2\nB3gwadsfAUuix6WVFNsSQl1ywWfw/qjsQcDU4j4Llfj+PRp9vuYTTnAtC8cYLZ9GaHHzcVwxpoov\nWv/3gs9dUtlKfQ+LOadk5POnYS5ERCRB1UciIpKgpCAiIglKCiIikqCkICIiCUoKIiKSoKQgEjGz\n7bbrCK4VNmKnmWUlj9ApUlXVzXQAIlXIVnfvmukgRDJJVwoiJYjG0/+Tmb0bPQ6L1rcxs5ejAd9e\nNrNDovUHWJjjYF70+F60qzpm9tdozPwXzWyvqPzVZrYw2s+kDP2aIoCSgkiyvQpVH12Y9NpGd+8O\n3APcFa27hzAEeWfCgHRjovVjgNc8DOiXTegJC9AeGOvuHYH1wLnR+hFAt2g/w+L65UTSoR7NIhEz\n2+zujVKsXwac6O5Lo4HLvnD3Zma2ljB0w7Zo/Up3b25ma4DW7v5N0j6yCOPet4+WrwPqufvvzewF\nYDNhNNinPRoMUCQTdKUgkh4v4nlRZVL5Jun5dnbe0zudMBbV0cCsaOROkYxQUhBJz4VJP9+Onv+X\nMKonwCDgzej5y8CVAGZWx8z2KWqnZrYHcLC7Twd+BTQBdrtaEaks+kYistNetuvk7S+4e0Gz1D3N\nbAbhi9TAaN3VwENm9n/AGuDSaP01wDgzu4xwRXAlYYTOVOoAE8xsX8LotXe6+/oK+41ESkn3FERK\nEN1TyHH3tZmORSRuqj4SEZEEXSmIiEiCrhRERCRBSUFERBKUFEREJEFJQUREEpQUREQk4f8Bv+4G\nNaan2acAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x25d041385c0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "train_loss_values = history_dict['loss']\n",
    "val_loss_values = history_dict['val_loss']\n",
    "\n",
    "epochs = range(1, len(history_dict['acc']) + 1)\n",
    "\n",
    "plt.plot(epochs, train_loss_values, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss_values, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Loss')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* as we can see the model give least loss when the no. of epochs are 3 after that it is overfitting."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.final model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/3\n",
      "25000/25000 [==============================] - 6s 248us/step - loss: 0.7514 - acc: 0.7044\n",
      "Epoch 2/3\n",
      "25000/25000 [==============================] - 6s 245us/step - loss: 0.2836 - acc: 0.9056\n",
      "Epoch 3/3\n",
      "25000/25000 [==============================] - 5s 218us/step - loss: 0.2001 - acc: 0.9381\n",
      "25000/25000 [==============================] - 11s 448us/step\n"
     ]
    }
   ],
   "source": [
    "model = models.Sequential()\n",
    "model.add(layers.Dense(64, activation='relu', input_shape=(15000,)))\n",
    "model.add(layers.Dense(32, activation='relu'))\n",
    "model.add(layers.Dense(1, activation='tanh'))\n",
    "\n",
    "model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])\n",
    "\n",
    "model.fit(x_train, y_train, epochs=3, batch_size=512)\n",
    "results = model.evaluate(x_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.48843979853630065, 0.8644]"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* as you can see the bit difference between training acc(93%) and testing acc(86%) which means the model is bit overfitting but this is by far the best acc. i got , you can tune it more by changing hyperparameters."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6. Prediction on test data\n",
    "After having trained a network, you’ll want to use it in a practical setting. You can generate the likelihood of reviews being positive by using the predict method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.43963727],\n",
       "       [0.99779546],\n",
       "       [0.3725262 ],\n",
       "       ...,\n",
       "       [0.12591572],\n",
       "       [0.01595698],\n",
       "       [0.87717104]], dtype=float32)"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.predict(x_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* As you can see, the network is confident for some samples (0.99 or 0.01 or less) but less confident for others (0.43, etc.)\n",
    "* which means the prob. of this review belonging to class 1 is 44% ,class 2 is 99% and class 3 is 37% , therefore this reviews is of class 2. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}