Abstractive_Text_Summerization.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Abstractive_Text_Summerization.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyO+admpiexLuHi7upnWK7Nu",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/SayanMitra-99/Abstractive_text_summarization/blob/master/Abstractive_Text_Summerization.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sGoWvEudp4UF",
        "colab_type": "text"
      },
      "source": [
        "# Installing the GPU"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3EHE8_XhJbYa",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 726
        },
        "outputId": "9fe5d688-79df-4f22-e33e-bcc73c7dc7f1"
      },
      "source": [
        "!pip install tensorflow-gpu"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting tensorflow-gpu\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/0f/11/763f55d3d15efd778ef24453f126e6c33635680e5a2bb346da3fab5997cb/tensorflow_gpu-2.3.0-cp36-cp36m-manylinux2010_x86_64.whl (320.4MB)\n",
            "\u001b[K     |████████████████████████████████| 320.4MB 49kB/s \n",
            "\u001b[?25hRequirement already satisfied: scipy==1.4.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (1.4.1)\n",
            "Requirement already satisfied: h5py<2.11.0,>=2.10.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (2.10.0)\n",
            "Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (0.10.0)\n",
            "Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (1.12.1)\n",
            "Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (1.32.0)\n",
            "Requirement already satisfied: google-pasta>=0.1.8 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (0.2.0)\n",
            "Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (0.35.1)\n",
            "Requirement already satisfied: numpy<1.19.0,>=1.16.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (1.18.5)\n",
            "Requirement already satisfied: astunparse==1.6.3 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (1.6.3)\n",
            "Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (3.3.0)\n",
            "Requirement already satisfied: tensorflow-estimator<2.4.0,>=2.3.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (2.3.0)\n",
            "Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (1.1.0)\n",
            "Requirement already satisfied: protobuf>=3.9.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (3.12.4)\n",
            "Requirement already satisfied: tensorboard<3,>=2.3.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (2.3.0)\n",
            "Requirement already satisfied: gast==0.3.3 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (0.3.3)\n",
            "Requirement already satisfied: keras-preprocessing<1.2,>=1.1.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (1.1.2)\n",
            "Requirement already satisfied: six>=1.12.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (1.15.0)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from protobuf>=3.9.2->tensorflow-gpu) (50.3.0)\n",
            "Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard<3,>=2.3.0->tensorflow-gpu) (2.23.0)\n",
            "Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard<3,>=2.3.0->tensorflow-gpu) (1.7.0)\n",
            "Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tensorboard<3,>=2.3.0->tensorflow-gpu) (1.0.1)\n",
            "Requirement already satisfied: google-auth<2,>=1.6.3 in /usr/local/lib/python3.6/dist-packages (from tensorboard<3,>=2.3.0->tensorflow-gpu) (1.17.2)\n",
            "Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.6/dist-packages (from tensorboard<3,>=2.3.0->tensorflow-gpu) (0.4.1)\n",
            "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<3,>=2.3.0->tensorflow-gpu) (3.2.2)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard<3,>=2.3.0->tensorflow-gpu) (3.0.4)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard<3,>=2.3.0->tensorflow-gpu) (2.10)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard<3,>=2.3.0->tensorflow-gpu) (1.24.3)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard<3,>=2.3.0->tensorflow-gpu) (2020.6.20)\n",
            "Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.6/dist-packages (from google-auth<2,>=1.6.3->tensorboard<3,>=2.3.0->tensorflow-gpu) (0.2.8)\n",
            "Requirement already satisfied: rsa<5,>=3.1.4; python_version >= \"3\" in /usr/local/lib/python3.6/dist-packages (from google-auth<2,>=1.6.3->tensorboard<3,>=2.3.0->tensorflow-gpu) (4.6)\n",
            "Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.6/dist-packages (from google-auth<2,>=1.6.3->tensorboard<3,>=2.3.0->tensorflow-gpu) (4.1.1)\n",
            "Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard<3,>=2.3.0->tensorflow-gpu) (1.3.0)\n",
            "Requirement already satisfied: importlib-metadata; python_version < \"3.8\" in /usr/local/lib/python3.6/dist-packages (from markdown>=2.6.8->tensorboard<3,>=2.3.0->tensorflow-gpu) (1.7.0)\n",
            "Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.6/dist-packages (from pyasn1-modules>=0.2.1->google-auth<2,>=1.6.3->tensorboard<3,>=2.3.0->tensorflow-gpu) (0.4.8)\n",
            "Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.6/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard<3,>=2.3.0->tensorflow-gpu) (3.1.0)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.6/dist-packages (from importlib-metadata; python_version < \"3.8\"->markdown>=2.6.8->tensorboard<3,>=2.3.0->tensorflow-gpu) (3.1.0)\n",
            "Installing collected packages: tensorflow-gpu\n",
            "Successfully installed tensorflow-gpu-2.3.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5mLkVOjRqYHj",
        "colab_type": "text"
      },
      "source": [
        "# Checking the GPU version :)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ovF1-Q6VJ3d2",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 353
        },
        "outputId": "ab57972d-ede4-41bc-912b-e37ec8d1395b"
      },
      "source": [
        "!nvidia-smi"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Sat Sep 19 03:34:39 2020       \n",
            "+-----------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 450.66       Driver Version: 418.67       CUDA Version: 10.1     |\n",
            "|-------------------------------+----------------------+----------------------+\n",
            "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                               |                      |               MIG M. |\n",
            "|===============================+======================+======================|\n",
            "|   0  Tesla P100-PCIE...  Off  | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   45C    P0    29W / 250W |      0MiB / 16280MiB |      0%      Default |\n",
            "|                               |                      |                 ERR! |\n",
            "+-------------------------------+----------------------+----------------------+\n",
            "                                                                               \n",
            "+-----------------------------------------------------------------------------+\n",
            "| Processes:                                                                  |\n",
            "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
            "|        ID   ID                                                   Usage      |\n",
            "|=============================================================================|\n",
            "|  No running processes found                                                 |\n",
            "+-----------------------------------------------------------------------------+\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LQUyDbdYqicn",
        "colab_type": "text"
      },
      "source": [
        "# Installing all the important libraries "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JSg43yNVL6GL",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import numpy as np  \n",
        "import pandas as pd \n",
        "import matplotlib.pyplot as plt           \n",
        "from bs4 import BeautifulSoup \n",
        "import re\n",
        "from nltk.corpus import stopwords  \n",
        "from keras.preprocessing.text import Tokenizer \n",
        "from keras.preprocessing.sequence import pad_sequences \n",
        "from tensorflow.keras.layers import Input, LSTM, Embedding, Dense, Concatenate, TimeDistributed, Bidirectional\n",
        "from tensorflow.keras.models import Model\n",
        "from tensorflow.keras.callbacks import EarlyStopping\n",
        "import warnings\n",
        "import io\n",
        "from google.colab import files\n",
        "pd.set_option(\"display.max_colwidth\", 200)\n",
        "warnings.filterwarnings(\"ignore\")"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Dh85OapNq02X",
        "colab_type": "text"
      },
      "source": [
        "# Importing the dataset "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0aYeTfZCZQZs",
        "colab_type": "code",
        "colab": {
          "resources": {
            "http://localhost:8080/nbextensions/google.colab/files.js": {
              "data": "Ly8gQ29weXJpZ2h0IDIwMTcgR29vZ2xlIExMQwovLwovLyBMaWNlbnNlZCB1bmRlciB0aGUgQXBhY2hlIExpY2Vuc2UsIFZlcnNpb24gMi4wICh0aGUgIkxpY2Vuc2UiKTsKLy8geW91IG1heSBub3QgdXNlIHRoaXMgZmlsZSBleGNlcHQgaW4gY29tcGxpYW5jZSB3aXRoIHRoZSBMaWNlbnNlLgovLyBZb3UgbWF5IG9idGFpbiBhIGNvcHkgb2YgdGhlIExpY2Vuc2UgYXQKLy8KLy8gICAgICBodHRwOi8vd3d3LmFwYWNoZS5vcmcvbGljZW5zZXMvTElDRU5TRS0yLjAKLy8KLy8gVW5sZXNzIHJlcXVpcmVkIGJ5IGFwcGxpY2FibGUgbGF3IG9yIGFncmVlZCB0byBpbiB3cml0aW5nLCBzb2Z0d2FyZQovLyBkaXN0cmlidXRlZCB1bmRlciB0aGUgTGljZW5zZSBpcyBkaXN0cmlidXRlZCBvbiBhbiAiQVMgSVMiIEJBU0lTLAovLyBXSVRIT1VUIFdBUlJBTlRJRVMgT1IgQ09ORElUSU9OUyBPRiBBTlkgS0lORCwgZWl0aGVyIGV4cHJlc3Mgb3IgaW1wbGllZC4KLy8gU2VlIHRoZSBMaWNlbnNlIGZvciB0aGUgc3BlY2lmaWMgbGFuZ3VhZ2UgZ292ZXJuaW5nIHBlcm1pc3Npb25zIGFuZAovLyBsaW1pdGF0aW9ucyB1bmRlciB0aGUgTGljZW5zZS4KCi8qKgogKiBAZmlsZW92ZXJ2aWV3IEhlbHBlcnMgZm9yIGdvb2dsZS5jb2xhYiBQeXRob24gbW9kdWxlLgogKi8KKGZ1bmN0aW9uKHNjb3BlKSB7CmZ1bmN0aW9uIHNwYW4odGV4dCwgc3R5bGVBdHRyaWJ1dGVzID0ge30pIHsKICBjb25zdCBlbGVtZW50ID0gZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgnc3BhbicpOwogIGVsZW1lbnQudGV4dENvbnRlbnQgPSB0ZXh0OwogIGZvciAoY29uc3Qga2V5IG9mIE9iamVjdC5rZXlzKHN0eWxlQXR0cmlidXRlcykpIHsKICAgIGVsZW1lbnQuc3R5bGVba2V5XSA9IHN0eWxlQXR0cmlidXRlc1trZXldOwogIH0KICByZXR1cm4gZWxlbWVudDsKfQoKLy8gTWF4IG51bWJlciBvZiBieXRlcyB3aGljaCB3aWxsIGJlIHVwbG9hZGVkIGF0IGEgdGltZS4KY29uc3QgTUFYX1BBWUxPQURfU0laRSA9IDEwMCAqIDEwMjQ7CgpmdW5jdGlvbiBfdXBsb2FkRmlsZXMoaW5wdXRJZCwgb3V0cHV0SWQpIHsKICBjb25zdCBzdGVwcyA9IHVwbG9hZEZpbGVzU3RlcChpbnB1dElkLCBvdXRwdXRJZCk7CiAgY29uc3Qgb3V0cHV0RWxlbWVudCA9IGRvY3VtZW50LmdldEVsZW1lbnRCeUlkKG91dHB1dElkKTsKICAvLyBDYWNoZSBzdGVwcyBvbiB0aGUgb3V0cHV0RWxlbWVudCB0byBtYWtlIGl0IGF2YWlsYWJsZSBmb3IgdGhlIG5leHQgY2FsbAogIC8vIHRvIHVwbG9hZEZpbGVzQ29udGludWUgZnJvbSBQeXRob24uCiAgb3V0cHV0RWxlbWVudC5zdGVwcyA9IHN0ZXBzOwoKICByZXR1cm4gX3VwbG9hZEZpbGVzQ29udGludWUob3V0cHV0SWQpOwp9CgovLyBUaGlzIGlzIHJvdWdobHkgYW4gYXN5bmMgZ2VuZXJhdG9yIChub3Qgc3VwcG9ydGVkIGluIHRoZSBicm93c2VyIHlldCksCi8vIHdoZXJlIHRoZXJlIGFyZSBtdWx0aXBsZSBhc3luY2hyb25vdXMgc3RlcHMgYW5kIHRoZSBQeXRob24gc2lkZSBpcyBnb2luZwovLyB0byBwb2xsIGZvciBjb21wbGV0aW9uIG9mIGVhY2ggc3RlcC4KLy8gVGhpcyB1c2VzIGEgUHJvbWlzZSB0byBibG9jayB0aGUgcHl0aG9uIHNpZGUgb24gY29tcGxldGlvbiBvZiBlYWNoIHN0ZXAsCi8vIHRoZW4gcGFzc2VzIHRoZSByZXN1bHQgb2YgdGhlIHByZXZpb3VzIHN0ZXAgYXMgdGhlIGlucHV0IHRvIHRoZSBuZXh0IHN0ZXAuCmZ1bmN0aW9uIF91cGxvYWRGaWxlc0NvbnRpbnVlKG91dHB1dElkKSB7CiAgY29uc3Qgb3V0cHV0RWxlbWVudCA9IGRvY3VtZW50LmdldEVsZW1lbnRCeUlkKG91dHB1dElkKTsKICBjb25zdCBzdGVwcyA9IG91dHB1dEVsZW1lbnQuc3RlcHM7CgogIGNvbnN0IG5leHQgPSBzdGVwcy5uZXh0KG91dHB1dEVsZW1lbnQubGFzdFByb21pc2VWYWx1ZSk7CiAgcmV0dXJuIFByb21pc2UucmVzb2x2ZShuZXh0LnZhbHVlLnByb21pc2UpLnRoZW4oKHZhbHVlKSA9PiB7CiAgICAvLyBDYWNoZSB0aGUgbGFzdCBwcm9taXNlIHZhbHVlIHRvIG1ha2UgaXQgYXZhaWxhYmxlIHRvIHRoZSBuZXh0CiAgICAvLyBzdGVwIG9mIHRoZSBnZW5lcmF0b3IuCiAgICBvdXRwdXRFbGVtZW50Lmxhc3RQcm9taXNlVmFsdWUgPSB2YWx1ZTsKICAgIHJldHVybiBuZXh0LnZhbHVlLnJlc3BvbnNlOwogIH0pOwp9CgovKioKICogR2VuZXJhdG9yIGZ1bmN0aW9uIHdoaWNoIGlzIGNhbGxlZCBiZXR3ZWVuIGVhY2ggYXN5bmMgc3RlcCBvZiB0aGUgdXBsb2FkCiAqIHByb2Nlc3MuCiAqIEBwYXJhbSB7c3RyaW5nfSBpbnB1dElkIEVsZW1lbnQgSUQgb2YgdGhlIGlucHV0IGZpbGUgcGlja2VyIGVsZW1lbnQuCiAqIEBwYXJhbSB7c3RyaW5nfSBvdXRwdXRJZCBFbGVtZW50IElEIG9mIHRoZSBvdXRwdXQgZGlzcGxheS4KICogQHJldHVybiB7IUl0ZXJhYmxlPCFPYmplY3Q+fSBJdGVyYWJsZSBvZiBuZXh0IHN0ZXBzLgogKi8KZnVuY3Rpb24qIHVwbG9hZEZpbGVzU3RlcChpbnB1dElkLCBvdXRwdXRJZCkgewogIGNvbnN0IGlucHV0RWxlbWVudCA9IGRvY3VtZW50LmdldEVsZW1lbnRCeUlkKGlucHV0SWQpOwogIGlucHV0RWxlbWVudC5kaXNhYmxlZCA9IGZhbHNlOwoKICBjb25zdCBvdXRwdXRFbGVtZW50ID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQob3V0cHV0SWQpOwogIG91dHB1dEVsZW1lbnQuaW5uZXJIVE1MID0gJyc7CgogIGNvbnN0IHBpY2tlZFByb21pc2UgPSBuZXcgUHJvbWlzZSgocmVzb2x2ZSkgPT4gewogICAgaW5wdXRFbGVtZW50LmFkZEV2ZW50TGlzdGVuZXIoJ2NoYW5nZScsIChlKSA9PiB7CiAgICAgIHJlc29sdmUoZS50YXJnZXQuZmlsZXMpOwogICAgfSk7CiAgfSk7CgogIGNvbnN0IGNhbmNlbCA9IGRvY3VtZW50LmNyZWF0ZUVsZW1lbnQoJ2J1dHRvbicpOwogIGlucHV0RWxlbWVudC5wYXJlbnRFbGVtZW50LmFwcGVuZENoaWxkKGNhbmNlbCk7CiAgY2FuY2VsLnRleHRDb250ZW50ID0gJ0NhbmNlbCB1cGxvYWQnOwogIGNvbnN0IGNhbmNlbFByb21pc2UgPSBuZXcgUHJvbWlzZSgocmVzb2x2ZSkgPT4gewogICAgY2FuY2VsLm9uY2xpY2sgPSAoKSA9PiB7CiAgICAgIHJlc29sdmUobnVsbCk7CiAgICB9OwogIH0pOwoKICAvLyBXYWl0IGZvciB0aGUgdXNlciB0byBwaWNrIHRoZSBmaWxlcy4KICBjb25zdCBmaWxlcyA9IHlpZWxkIHsKICAgIHByb21pc2U6IFByb21pc2UucmFjZShbcGlja2VkUHJvbWlzZSwgY2FuY2VsUHJvbWlzZV0pLAogICAgcmVzcG9uc2U6IHsKICAgICAgYWN0aW9uOiAnc3RhcnRpbmcnLAogICAgfQogIH07CgogIGNhbmNlbC5yZW1vdmUoKTsKCiAgLy8gRGlzYWJsZSB0aGUgaW5wdXQgZWxlbWVudCBzaW5jZSBmdXJ0aGVyIHBpY2tzIGFyZSBub3QgYWxsb3dlZC4KICBpbnB1dEVsZW1lbnQuZGlzYWJsZWQgPSB0cnVlOwoKICBpZiAoIWZpbGVzKSB7CiAgICByZXR1cm4gewogICAgICByZXNwb25zZTogewogICAgICAgIGFjdGlvbjogJ2NvbXBsZXRlJywKICAgICAgfQogICAgfTsKICB9CgogIGZvciAoY29uc3QgZmlsZSBvZiBmaWxlcykgewogICAgY29uc3QgbGkgPSBkb2N1bWVudC5jcmVhdGVFbGVtZW50KCdsaScpOwogICAgbGkuYXBwZW5kKHNwYW4oZmlsZS5uYW1lLCB7Zm9udFdlaWdodDogJ2JvbGQnfSkpOwogICAgbGkuYXBwZW5kKHNwYW4oCiAgICAgICAgYCgke2ZpbGUudHlwZSB8fCAnbi9hJ30pIC0gJHtmaWxlLnNpemV9IGJ5dGVzLCBgICsKICAgICAgICBgbGFzdCBtb2RpZmllZDogJHsKICAgICAgICAgICAgZmlsZS5sYXN0TW9kaWZpZWREYXRlID8gZmlsZS5sYXN0TW9kaWZpZWREYXRlLnRvTG9jYWxlRGF0ZVN0cmluZygpIDoKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgJ24vYSd9IC0gYCkpOwogICAgY29uc3QgcGVyY2VudCA9IHNwYW4oJzAlIGRvbmUnKTsKICAgIGxpLmFwcGVuZENoaWxkKHBlcmNlbnQpOwoKICAgIG91dHB1dEVsZW1lbnQuYXBwZW5kQ2hpbGQobGkpOwoKICAgIGNvbnN0IGZpbGVEYXRhUHJvbWlzZSA9IG5ldyBQcm9taXNlKChyZXNvbHZlKSA9PiB7CiAgICAgIGNvbnN0IHJlYWRlciA9IG5ldyBGaWxlUmVhZGVyKCk7CiAgICAgIHJlYWRlci5vbmxvYWQgPSAoZSkgPT4gewogICAgICAgIHJlc29sdmUoZS50YXJnZXQucmVzdWx0KTsKICAgICAgfTsKICAgICAgcmVhZGVyLnJlYWRBc0FycmF5QnVmZmVyKGZpbGUpOwogICAgfSk7CiAgICAvLyBXYWl0IGZvciB0aGUgZGF0YSB0byBiZSByZWFkeS4KICAgIGxldCBmaWxlRGF0YSA9IHlpZWxkIHsKICAgICAgcHJvbWlzZTogZmlsZURhdGFQcm9taXNlLAogICAgICByZXNwb25zZTogewogICAgICAgIGFjdGlvbjogJ2NvbnRpbnVlJywKICAgICAgfQogICAgfTsKCiAgICAvLyBVc2UgYSBjaHVua2VkIHNlbmRpbmcgdG8gYXZvaWQgbWVzc2FnZSBzaXplIGxpbWl0cy4gU2VlIGIvNjIxMTU2NjAuCiAgICBsZXQgcG9zaXRpb24gPSAwOwogICAgd2hpbGUgKHBvc2l0aW9uIDwgZmlsZURhdGEuYnl0ZUxlbmd0aCkgewogICAgICBjb25zdCBsZW5ndGggPSBNYXRoLm1pbihmaWxlRGF0YS5ieXRlTGVuZ3RoIC0gcG9zaXRpb24sIE1BWF9QQVlMT0FEX1NJWkUpOwogICAgICBjb25zdCBjaHVuayA9IG5ldyBVaW50OEFycmF5KGZpbGVEYXRhLCBwb3NpdGlvbiwgbGVuZ3RoKTsKICAgICAgcG9zaXRpb24gKz0gbGVuZ3RoOwoKICAgICAgY29uc3QgYmFzZTY0ID0gYnRvYShTdHJpbmcuZnJvbUNoYXJDb2RlLmFwcGx5KG51bGwsIGNodW5rKSk7CiAgICAgIHlpZWxkIHsKICAgICAgICByZXNwb25zZTogewogICAgICAgICAgYWN0aW9uOiAnYXBwZW5kJywKICAgICAgICAgIGZpbGU6IGZpbGUubmFtZSwKICAgICAgICAgIGRhdGE6IGJhc2U2NCwKICAgICAgICB9LAogICAgICB9OwogICAgICBwZXJjZW50LnRleHRDb250ZW50ID0KICAgICAgICAgIGAke01hdGgucm91bmQoKHBvc2l0aW9uIC8gZmlsZURhdGEuYnl0ZUxlbmd0aCkgKiAxMDApfSUgZG9uZWA7CiAgICB9CiAgfQoKICAvLyBBbGwgZG9uZS4KICB5aWVsZCB7CiAgICByZXNwb25zZTogewogICAgICBhY3Rpb246ICdjb21wbGV0ZScsCiAgICB9CiAgfTsKfQoKc2NvcGUuZ29vZ2xlID0gc2NvcGUuZ29vZ2xlIHx8IHt9OwpzY29wZS5nb29nbGUuY29sYWIgPSBzY29wZS5nb29nbGUuY29sYWIgfHwge307CnNjb3BlLmdvb2dsZS5jb2xhYi5fZmlsZXMgPSB7CiAgX3VwbG9hZEZpbGVzLAogIF91cGxvYWRGaWxlc0NvbnRpbnVlLAp9Owp9KShzZWxmKTsK",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "application/javascript"
                ]
              ],
              "status": 200,
              "status_text": ""
            }
          },
          "base_uri": "https://localhost:8080/",
          "height": 73
        },
        "outputId": "a80aeb0f-4a6a-42aa-e9bd-1197d73b0bf9"
      },
      "source": [
        "upload = files.upload()"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "\n",
              "     <input type=\"file\" id=\"files-88d365bb-0555-43be-b630-34509b33d679\" name=\"files[]\" multiple disabled\n",
              "        style=\"border:none\" />\n",
              "     <output id=\"result-88d365bb-0555-43be-b630-34509b33d679\">\n",
              "      Upload widget is only available when the cell has been executed in the\n",
              "      current browser session. Please rerun this cell to enable.\n",
              "      </output>\n",
              "      <script src=\"/nbextensions/google.colab/files.js\"></script> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "Saving Reviews.csv to Reviews.csv\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XI3eA24RMpKX",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "data = pd.read_csv(io.StringIO(upload['Reviews.csv'].decode('utf-8')))"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LLnIi0OnrWUo",
        "colab_type": "text"
      },
      "source": [
        "# DATA PREPROCESSING"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ybvDBP6Aq-l7",
        "colab_type": "text"
      },
      "source": [
        "# Exploring the dataset"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QZSHAWw-PgjM",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 380
        },
        "outputId": "cb88d2e0-f89a-4a9a-911b-39c50488c79d"
      },
      "source": [
        "data.head()"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Id</th>\n",
              "      <th>ProductId</th>\n",
              "      <th>UserId</th>\n",
              "      <th>ProfileName</th>\n",
              "      <th>HelpfulnessNumerator</th>\n",
              "      <th>HelpfulnessDenominator</th>\n",
              "      <th>Score</th>\n",
              "      <th>Time</th>\n",
              "      <th>Summary</th>\n",
              "      <th>Text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>B001E4KFG0</td>\n",
              "      <td>A3SGXH7AUHU8GW</td>\n",
              "      <td>delmartian</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>5</td>\n",
              "      <td>1303862400</td>\n",
              "      <td>Good Quality Dog Food</td>\n",
              "      <td>I have bought several of the Vitality canned dog food products and have found them all to be of good quality. The product looks more like a stew than a processed meat and it smells better. My Labr...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>B00813GRG4</td>\n",
              "      <td>A1D87F6ZCVE5NK</td>\n",
              "      <td>dll pa</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1346976000</td>\n",
              "      <td>Not as Advertised</td>\n",
              "      <td>Product arrived labeled as Jumbo Salted Peanuts...the peanuts were actually small sized unsalted. Not sure if this was an error or if the vendor intended to represent the product as \"Jumbo\".</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>B000LQOCH0</td>\n",
              "      <td>ABXLMWJIXXAIN</td>\n",
              "      <td>Natalia Corres \"Natalia Corres\"</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>4</td>\n",
              "      <td>1219017600</td>\n",
              "      <td>\"Delight\" says it all</td>\n",
              "      <td>This is a confection that has been around a few centuries.  It is a light, pillowy citrus gelatin with nuts - in this case Filberts. And it is cut into tiny squares and then liberally coated with ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>B000UA0QIQ</td>\n",
              "      <td>A395BORC6FGVXV</td>\n",
              "      <td>Karl</td>\n",
              "      <td>3</td>\n",
              "      <td>3</td>\n",
              "      <td>2</td>\n",
              "      <td>1307923200</td>\n",
              "      <td>Cough Medicine</td>\n",
              "      <td>If you are looking for the secret ingredient in Robitussin I believe I have found it.  I got this in addition to the Root Beer Extract I ordered (which was good) and made some cherry soda.  The fl...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>B006K2ZZ7K</td>\n",
              "      <td>A1UQRSCLF8GW1T</td>\n",
              "      <td>Michael D. Bigham \"M. Wassir\"</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>5</td>\n",
              "      <td>1350777600</td>\n",
              "      <td>Great taffy</td>\n",
              "      <td>Great taffy at a great price.  There was a wide assortment of yummy taffy.  Delivery was very quick.  If your a taffy lover, this is a deal.</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Id  ...                                                                                                                                                                                                     Text\n",
              "0   1  ...  I have bought several of the Vitality canned dog food products and have found them all to be of good quality. The product looks more like a stew than a processed meat and it smells better. My Labr...\n",
              "1   2  ...           Product arrived labeled as Jumbo Salted Peanuts...the peanuts were actually small sized unsalted. Not sure if this was an error or if the vendor intended to represent the product as \"Jumbo\".\n",
              "2   3  ...  This is a confection that has been around a few centuries.  It is a light, pillowy citrus gelatin with nuts - in this case Filberts. And it is cut into tiny squares and then liberally coated with ...\n",
              "3   4  ...  If you are looking for the secret ingredient in Robitussin I believe I have found it.  I got this in addition to the Root Beer Extract I ordered (which was good) and made some cherry soda.  The fl...\n",
              "4   5  ...                                                             Great taffy at a great price.  There was a wide assortment of yummy taffy.  Delivery was very quick.  If your a taffy lover, this is a deal.\n",
              "\n",
              "[5 rows x 10 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3myHqrSZ3qCe",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 302
        },
        "outputId": "5c2a43ac-7fa5-4e34-9d1f-cd8abf1b2442"
      },
      "source": [
        "data.info()"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 568454 entries, 0 to 568453\n",
            "Data columns (total 10 columns):\n",
            " #   Column                  Non-Null Count   Dtype \n",
            "---  ------                  --------------   ----- \n",
            " 0   Id                      568454 non-null  int64 \n",
            " 1   ProductId               568454 non-null  object\n",
            " 2   UserId                  568454 non-null  object\n",
            " 3   ProfileName             568438 non-null  object\n",
            " 4   HelpfulnessNumerator    568454 non-null  int64 \n",
            " 5   HelpfulnessDenominator  568454 non-null  int64 \n",
            " 6   Score                   568454 non-null  int64 \n",
            " 7   Time                    568454 non-null  int64 \n",
            " 8   Summary                 568427 non-null  object\n",
            " 9   Text                    568454 non-null  object\n",
            "dtypes: int64(5), object(5)\n",
            "memory usage: 43.4+ MB\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "J58DcikmrWQa",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "ec6fb96d-34a6-4658-8b8c-60832f7bb9ce"
      },
      "source": [
        "data.shape"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(568454, 10)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lZ9tQPOirEzC",
        "colab_type": "text"
      },
      "source": [
        "# Dropping the duplicates and the na values"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "h6t8-aSOriSi",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "data.drop_duplicates(subset=['Text'],inplace=True)  #dropping duplicates\n",
        "data.dropna(axis=0,inplace=True)   #dropping na"
      ],
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gnbPhkayrM1B",
        "colab_type": "text"
      },
      "source": [
        "# declaring the contraction mapping dictionary"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RQTpcQ8trsYj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "contraction_mapping = {\"ain't\": \"is not\", \"aren't\": \"are not\",\"can't\": \"cannot\", \"'cause\": \"because\", \"could've\": \"could have\", \"couldn't\": \"could not\",\n",
        "\n",
        "                           \"didn't\": \"did not\", \"doesn't\": \"does not\", \"don't\": \"do not\", \"hadn't\": \"had not\", \"hasn't\": \"has not\", \"haven't\": \"have not\",\n",
        "\n",
        "                           \"he'd\": \"he would\",\"he'll\": \"he will\", \"he's\": \"he is\", \"how'd\": \"how did\", \"how'd'y\": \"how do you\", \"how'll\": \"how will\", \"how's\": \"how is\",\n",
        "\n",
        "                           \"I'd\": \"I would\", \"I'd've\": \"I would have\", \"I'll\": \"I will\", \"I'll've\": \"I will have\",\"I'm\": \"I am\", \"I've\": \"I have\", \"i'd\": \"i would\",\n",
        "\n",
        "                           \"i'd've\": \"i would have\", \"i'll\": \"i will\",  \"i'll've\": \"i will have\",\"i'm\": \"i am\", \"i've\": \"i have\", \"isn't\": \"is not\", \"it'd\": \"it would\",\n",
        "\n",
        "                           \"it'd've\": \"it would have\", \"it'll\": \"it will\", \"it'll've\": \"it will have\",\"it's\": \"it is\", \"let's\": \"let us\", \"ma'am\": \"madam\",\n",
        "\n",
        "                           \"mayn't\": \"may not\", \"might've\": \"might have\",\"mightn't\": \"might not\",\"mightn't've\": \"might not have\", \"must've\": \"must have\",\n",
        "\n",
        "                           \"mustn't\": \"must not\", \"mustn't've\": \"must not have\", \"needn't\": \"need not\", \"needn't've\": \"need not have\",\"o'clock\": \"of the clock\",\n",
        "\n",
        "                           \"oughtn't\": \"ought not\", \"oughtn't've\": \"ought not have\", \"shan't\": \"shall not\", \"sha'n't\": \"shall not\", \"shan't've\": \"shall not have\",\n",
        "\n",
        "                           \"she'd\": \"she would\", \"she'd've\": \"she would have\", \"she'll\": \"she will\", \"she'll've\": \"she will have\", \"she's\": \"she is\",\n",
        "\n",
        "                           \"should've\": \"should have\", \"shouldn't\": \"should not\", \"shouldn't've\": \"should not have\", \"so've\": \"so have\",\"so's\": \"so as\",\n",
        "\n",
        "                           \"this's\": \"this is\",\"that'd\": \"that would\", \"that'd've\": \"that would have\", \"that's\": \"that is\", \"there'd\": \"there would\",\n",
        "\n",
        "                           \"there'd've\": \"there would have\", \"there's\": \"there is\", \"here's\": \"here is\",\"they'd\": \"they would\", \"they'd've\": \"they would have\",\n",
        "\n",
        "                           \"they'll\": \"they will\", \"they'll've\": \"they will have\", \"they're\": \"they are\", \"they've\": \"they have\", \"to've\": \"to have\",\n",
        "\n",
        "                           \"wasn't\": \"was not\", \"we'd\": \"we would\", \"we'd've\": \"we would have\", \"we'll\": \"we will\", \"we'll've\": \"we will have\", \"we're\": \"we are\",\n",
        "\n",
        "                           \"we've\": \"we have\", \"weren't\": \"were not\", \"what'll\": \"what will\", \"what'll've\": \"what will have\", \"what're\": \"what are\",\n",
        "\n",
        "                           \"what's\": \"what is\", \"what've\": \"what have\", \"when's\": \"when is\", \"when've\": \"when have\", \"where'd\": \"where did\", \"where's\": \"where is\",\n",
        "\n",
        "                           \"where've\": \"where have\", \"who'll\": \"who will\", \"who'll've\": \"who will have\", \"who's\": \"who is\", \"who've\": \"who have\",\n",
        "\n",
        "                           \"why's\": \"why is\", \"why've\": \"why have\", \"will've\": \"will have\", \"won't\": \"will not\", \"won't've\": \"will not have\",\n",
        "\n",
        "                           \"would've\": \"would have\", \"wouldn't\": \"would not\", \"wouldn't've\": \"would not have\", \"y'all\": \"you all\",\n",
        "\n",
        "                           \"y'all'd\": \"you all would\",\"y'all'd've\": \"you all would have\",\"y'all're\": \"you all are\",\"y'all've\": \"you all have\",\n",
        "\n",
        "                           \"you'd\": \"you would\", \"you'd've\": \"you would have\", \"you'll\": \"you will\", \"you'll've\": \"you will have\",\n",
        "\n",
        "                           \"you're\": \"you are\", \"you've\": \"you have\"}"
      ],
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a2oEssh8rzco",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 222
        },
        "outputId": "981472f4-583e-48d9-97a0-804124cb9650"
      },
      "source": [
        "data['Text'][:10]"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0    I have bought several of the Vitality canned dog food products and have found them all to be of good quality. The product looks more like a stew than a processed meat and it smells better. My Labr...\n",
              "1             Product arrived labeled as Jumbo Salted Peanuts...the peanuts were actually small sized unsalted. Not sure if this was an error or if the vendor intended to represent the product as \"Jumbo\".\n",
              "2    This is a confection that has been around a few centuries.  It is a light, pillowy citrus gelatin with nuts - in this case Filberts. And it is cut into tiny squares and then liberally coated with ...\n",
              "3    If you are looking for the secret ingredient in Robitussin I believe I have found it.  I got this in addition to the Root Beer Extract I ordered (which was good) and made some cherry soda.  The fl...\n",
              "4                                                               Great taffy at a great price.  There was a wide assortment of yummy taffy.  Delivery was very quick.  If your a taffy lover, this is a deal.\n",
              "5    I got a wild hair for taffy and ordered this five pound bag. The taffy was all very enjoyable with many flavors: watermelon, root beer, melon, peppermint, grape, etc. My only complaint is there wa...\n",
              "6    This saltwater taffy had great flavors and was very soft and chewy.  Each candy was individually wrapped well.  None of the candies were stuck together, which did happen in the expensive version, ...\n",
              "7                                                               This taffy is so good.  It is very soft and chewy.  The flavors are amazing.  I would definitely recommend you buying it.  Very satisfying!!\n",
              "8                                                                        Right now I'm mostly just sprouting this so my cats can eat the grass. They love it. I rotate it around with Wheatgrass and Rye too\n",
              "9                                                                  This is a very healthy dog food. Good for their digestion. Also good for small puppies. My dog eats her required amount at every feeding.\n",
              "Name: Text, dtype: object"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GnGwDMPnsEae",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 67
        },
        "outputId": "8bedb0fd-97bb-4904-d103-4f694d19dee9"
      },
      "source": [
        "import nltk\n",
        "nltk.download('stopwords')"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/stopwords.zip.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MZyctVEKrgXm",
        "colab_type": "text"
      },
      "source": [
        "## Text cleaning is done"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nAcQRJjur22L",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "stop_words = set(stopwords.words('english')) \n",
        "def text_cleaner(text):\n",
        "    newString = text.lower()\n",
        "    newString = BeautifulSoup(newString, \"lxml\").text\n",
        "    newString = re.sub(r'\\([^)]*\\)', '', newString)\n",
        "    newString = re.sub('\"','', newString)\n",
        "    newString = ' '.join([contraction_mapping[t] if t in contraction_mapping else t for t in newString.split(\" \")])    \n",
        "    newString = re.sub(r\"'s\\b\",\"\",newString)\n",
        "    newString = re.sub(\"[^a-zA-Z]\", \" \", newString) \n",
        "    tokens = [w for w in newString.split() if not w in stop_words]\n",
        "    long_words=[]\n",
        "    for i in tokens:\n",
        "        if len(i)>=3:                  #removing short word\n",
        "            long_words.append(i)   \n",
        "    return (\" \".join(long_words)).strip()\n",
        "\n",
        "cleaned_text = []\n",
        "for t in data['Text']:\n",
        "    cleaned_text.append(text_cleaner(t))"
      ],
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JRgUGb3Or9Bt",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 205
        },
        "outputId": "1df5f355-c3d8-4fa2-b7ea-083f4d4220c7"
      },
      "source": [
        "cleaned_text[:10]"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['bought several vitality canned dog food products found good quality product looks like stew processed meat smells better labrador finicky appreciates product better',\n",
              " 'product arrived labeled jumbo salted peanuts peanuts actually small sized unsalted sure error vendor intended represent product jumbo',\n",
              " 'confection around centuries light pillowy citrus gelatin nuts case filberts cut tiny squares liberally coated powdered sugar tiny mouthful heaven chewy flavorful highly recommend yummy treat familiar story lewis lion witch wardrobe treat seduces edmund selling brother sisters witch',\n",
              " 'looking secret ingredient robitussin believe found got addition root beer extract ordered made cherry soda flavor medicinal',\n",
              " 'great taffy great price wide assortment yummy taffy delivery quick taffy lover deal',\n",
              " 'got wild hair taffy ordered five pound bag taffy enjoyable many flavors watermelon root beer melon peppermint grape etc complaint bit much red black licorice flavored pieces kids husband lasted two weeks would recommend brand taffy delightful treat',\n",
              " 'saltwater taffy great flavors soft chewy candy individually wrapped well none candies stuck together happen expensive version fralinger would highly recommend candy served beach themed party everyone loved',\n",
              " 'taffy good soft chewy flavors amazing would definitely recommend buying satisfying',\n",
              " 'right mostly sprouting cats eat grass love rotate around wheatgrass rye',\n",
              " 'healthy dog food good digestion also good small puppies dog eats required amount every feeding']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7X4Uf1QKsNmI",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 202
        },
        "outputId": "a13c61fe-8cae-418d-df66-58f4e923affa"
      },
      "source": [
        "data['Summary'][:10]"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0                            Good Quality Dog Food\n",
              "1                                Not as Advertised\n",
              "2                            \"Delight\" says it all\n",
              "3                                   Cough Medicine\n",
              "4                                      Great taffy\n",
              "5                                       Nice Taffy\n",
              "6    Great!  Just as good as the expensive brands!\n",
              "7                           Wonderful, tasty taffy\n",
              "8                                       Yay Barley\n",
              "9                                 Healthy Dog Food\n",
              "Name: Summary, dtype: object"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AsA6cHe-roJx",
        "colab_type": "text"
      },
      "source": [
        "## Summary cleaning is done"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zN5U1xgosbil",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def summary_cleaner(text):\n",
        "    newString = re.sub('\"','', text)\n",
        "    newString = ' '.join([contraction_mapping[t] if t in contraction_mapping else t for t in newString.split(\" \")])    \n",
        "    newString = re.sub(r\"'s\\b\",\"\",newString)\n",
        "    newString = re.sub(\"[^a-zA-Z]\", \" \", newString)\n",
        "    newString = newString.lower()\n",
        "    tokens=newString.split()\n",
        "    newString=''\n",
        "    for i in tokens:\n",
        "        if len(i)>1:                                 \n",
        "            newString=newString+i+' '  \n",
        "    return newString\n",
        "\n",
        "#Call the above function\n",
        "cleaned_summary = []\n",
        "for t in data['Summary']:\n",
        "    cleaned_summary.append(summary_cleaner(t))\n",
        "\n",
        "data['cleaned_text']=cleaned_text\n",
        "data['cleaned_summary']=cleaned_summary\n",
        "data['cleaned_summary'].replace('', np.nan, inplace=True)\n",
        "data.dropna(axis=0,inplace=True)"
      ],
      "execution_count": 17,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "THOe97W9snq1",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 185
        },
        "outputId": "fff3f18f-e2b2-41c6-ad0e-88e98d34f783"
      },
      "source": [
        "cleaned_summary[:10]"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['good quality dog food ',\n",
              " 'not as advertised ',\n",
              " 'delight says it all ',\n",
              " 'cough medicine ',\n",
              " 'great taffy ',\n",
              " 'nice taffy ',\n",
              " 'great just as good as the expensive brands ',\n",
              " 'wonderful tasty taffy ',\n",
              " 'yay barley ',\n",
              " 'healthy dog food ']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IcQgdgLTsq4d",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 951
        },
        "outputId": "597b8a01-4362-4a31-dd2a-6678e2d839ea"
      },
      "source": [
        "data.head()"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Id</th>\n",
              "      <th>ProductId</th>\n",
              "      <th>UserId</th>\n",
              "      <th>ProfileName</th>\n",
              "      <th>HelpfulnessNumerator</th>\n",
              "      <th>HelpfulnessDenominator</th>\n",
              "      <th>Score</th>\n",
              "      <th>Time</th>\n",
              "      <th>Summary</th>\n",
              "      <th>Text</th>\n",
              "      <th>cleaned_text</th>\n",
              "      <th>cleaned_summary</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>B001E4KFG0</td>\n",
              "      <td>A3SGXH7AUHU8GW</td>\n",
              "      <td>delmartian</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>5</td>\n",
              "      <td>1303862400</td>\n",
              "      <td>Good Quality Dog Food</td>\n",
              "      <td>I have bought several of the Vitality canned dog food products and have found them all to be of good quality. The product looks more like a stew than a processed meat and it smells better. My Labr...</td>\n",
              "      <td>bought several vitality canned dog food products found good quality product looks like stew processed meat smells better labrador finicky appreciates product better</td>\n",
              "      <td>good quality dog food</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>B00813GRG4</td>\n",
              "      <td>A1D87F6ZCVE5NK</td>\n",
              "      <td>dll pa</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1346976000</td>\n",
              "      <td>Not as Advertised</td>\n",
              "      <td>Product arrived labeled as Jumbo Salted Peanuts...the peanuts were actually small sized unsalted. Not sure if this was an error or if the vendor intended to represent the product as \"Jumbo\".</td>\n",
              "      <td>product arrived labeled jumbo salted peanuts peanuts actually small sized unsalted sure error vendor intended represent product jumbo</td>\n",
              "      <td>not as advertised</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>B000LQOCH0</td>\n",
              "      <td>ABXLMWJIXXAIN</td>\n",
              "      <td>Natalia Corres \"Natalia Corres\"</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>4</td>\n",
              "      <td>1219017600</td>\n",
              "      <td>\"Delight\" says it all</td>\n",
              "      <td>This is a confection that has been around a few centuries.  It is a light, pillowy citrus gelatin with nuts - in this case Filberts. And it is cut into tiny squares and then liberally coated with ...</td>\n",
              "      <td>confection around centuries light pillowy citrus gelatin nuts case filberts cut tiny squares liberally coated powdered sugar tiny mouthful heaven chewy flavorful highly recommend yummy treat famil...</td>\n",
              "      <td>delight says it all</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>B000UA0QIQ</td>\n",
              "      <td>A395BORC6FGVXV</td>\n",
              "      <td>Karl</td>\n",
              "      <td>3</td>\n",
              "      <td>3</td>\n",
              "      <td>2</td>\n",
              "      <td>1307923200</td>\n",
              "      <td>Cough Medicine</td>\n",
              "      <td>If you are looking for the secret ingredient in Robitussin I believe I have found it.  I got this in addition to the Root Beer Extract I ordered (which was good) and made some cherry soda.  The fl...</td>\n",
              "      <td>looking secret ingredient robitussin believe found got addition root beer extract ordered made cherry soda flavor medicinal</td>\n",
              "      <td>cough medicine</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>B006K2ZZ7K</td>\n",
              "      <td>A1UQRSCLF8GW1T</td>\n",
              "      <td>Michael D. Bigham \"M. Wassir\"</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>5</td>\n",
              "      <td>1350777600</td>\n",
              "      <td>Great taffy</td>\n",
              "      <td>Great taffy at a great price.  There was a wide assortment of yummy taffy.  Delivery was very quick.  If your a taffy lover, this is a deal.</td>\n",
              "      <td>great taffy great price wide assortment yummy taffy delivery quick taffy lover deal</td>\n",
              "      <td>great taffy</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Id  ...         cleaned_summary\n",
              "0   1  ...  good quality dog food \n",
              "1   2  ...      not as advertised \n",
              "2   3  ...    delight says it all \n",
              "3   4  ...         cough medicine \n",
              "4   5  ...            great taffy \n",
              "\n",
              "[5 rows x 12 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9Gz6INnAsvl0",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "18078a3b-5566-4ebc-f3dd-0ab613a7776c"
      },
      "source": [
        "data.shape"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(393222, 12)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XPgv1HQOr8GT",
        "colab_type": "text"
      },
      "source": [
        "## adding <start> and <end> tokens to the summary"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "V5odTjj2s1fN",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "data['cleaned_summary'] = data['cleaned_summary'].apply(lambda x : '_START_ '+ x + ' _END_')"
      ],
      "execution_count": 21,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VHJpjJr8s6li",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 202
        },
        "outputId": "2cb047e3-46f4-4173-8170-02be5b2482fc"
      },
      "source": [
        "data['cleaned_summary'][:10]"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0                         _START_ good quality dog food  _END_\n",
              "1                             _START_ not as advertised  _END_\n",
              "2                           _START_ delight says it all  _END_\n",
              "3                                _START_ cough medicine  _END_\n",
              "4                                   _START_ great taffy  _END_\n",
              "5                                    _START_ nice taffy  _END_\n",
              "6    _START_ great just as good as the expensive brands  _END_\n",
              "7                         _START_ wonderful tasty taffy  _END_\n",
              "8                                    _START_ yay barley  _END_\n",
              "9                              _START_ healthy dog food  _END_\n",
              "Name: cleaned_summary, dtype: object"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HvOqhjN1s-YY",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 373
        },
        "outputId": "32265393-da85-4d92-c0e3-af794964fdde"
      },
      "source": [
        "for i in range(5):\n",
        "    print(\"Review:\",data['cleaned_text'][i])\n",
        "    print(\"Summary:\",data['cleaned_summary'][i])\n",
        "    print(\"\\n\")"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Review: bought several vitality canned dog food products found good quality product looks like stew processed meat smells better labrador finicky appreciates product better\n",
            "Summary: _START_ good quality dog food  _END_\n",
            "\n",
            "\n",
            "Review: product arrived labeled jumbo salted peanuts peanuts actually small sized unsalted sure error vendor intended represent product jumbo\n",
            "Summary: _START_ not as advertised  _END_\n",
            "\n",
            "\n",
            "Review: confection around centuries light pillowy citrus gelatin nuts case filberts cut tiny squares liberally coated powdered sugar tiny mouthful heaven chewy flavorful highly recommend yummy treat familiar story lewis lion witch wardrobe treat seduces edmund selling brother sisters witch\n",
            "Summary: _START_ delight says it all  _END_\n",
            "\n",
            "\n",
            "Review: looking secret ingredient robitussin believe found got addition root beer extract ordered made cherry soda flavor medicinal\n",
            "Summary: _START_ cough medicine  _END_\n",
            "\n",
            "\n",
            "Review: great taffy great price wide assortment yummy taffy delivery quick taffy lover deal\n",
            "Summary: _START_ great taffy  _END_\n",
            "\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ax9avConsCgs",
        "colab_type": "text"
      },
      "source": [
        "## Understanding the distribution of the sequences"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BbDiUuRKtDOa",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 281
        },
        "outputId": "52bd80dc-04a4-4bda-82e9-832fc384de90"
      },
      "source": [
        "text_word_count = []\n",
        "summary_word_count = []\n",
        "\n",
        "# populate the lists with sentence lengths\n",
        "for i in data['cleaned_text']:\n",
        "      text_word_count.append(len(i.split()))\n",
        "\n",
        "for i in data['cleaned_summary']:\n",
        "      summary_word_count.append(len(i.split()))\n",
        "\n",
        "length_df = pd.DataFrame({'text':text_word_count, 'summary':summary_word_count})\n",
        "length_df.hist(bins = 20)\n",
        "plt.show()"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAEICAYAAACqMQjAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dfZRV1Znn8e8voISYKKjpCkE70C1JGt9QGSUTZ1IjEUuSGdJrqa2dGdCwQmeCHTOxEzHd0yS+dOOaVluMoccEItokaBttmIghFbRWZpyAYiQiEIcKYigWgvKmaDBBn/nj7NLj5ZziVhV1b738PmudVfc+522fW+fe59599tlbEYGZmVmRd9W7AGZm1ns5SZiZWSknCTMzK+UkYWZmpZwkzMyslJOEmZmVcpIwM7NSThJm1idJ2izpk71lO/2Vk4R1m6TB9S6DmfUMJ4leQtI1krZKekXSs5ImSrpL0g25ZRolteWeb5b0VUlPS3pV0nxJDZIeTtv5qaThadlRkkLSFZK2SNot6QuS/k1af4+kb+W2/ceSHpG0U9JLkhZJGlax72skPQ28msrxw4pjmivpth594WxAknQP8IfA/5K0T9LXJE2Q9H/TufxLSY1p2X+bzuET0/PT0/n/0aLt1O2gequI8FTnCfgIsAX4YHo+Cvhj4C7ghtxyjUBb7vlmYCXQAIwEdgC/AM4A3g08AszObTOAf0rzJgH7gX8F/iC3/ifS8icB5wNDgPcDPwP+sWLfa4ATgaHACOBVYFiaPzht76x6v76e+ueUzsFPpscjgZ3AZLIvv+en5+9P829M74ehwFrgyqLteDp48i+J3uENsg/jsZKOiIjNEfHrKte9PSK2R8RW4H8DqyLiqYjYDzxIljDyro+I/RHxE7IP9R9ExI7c+mcARERrRDRHxOsR8SJwC/CJim3NjYgtEfHbiNhGlkguTvOagJci4slOvRJmXfOfgWURsSwi3oyIZmA1WdIA+AZwDPA4sBW4oy6l7IOcJHqBiGgFvkx2Iu+QtFjSB6tcfXvu8W8Lnr+3K8unaqvFqQrsZeCfgeMrtrWl4vlCsjcr6e89VR6DWXd9CLg4VTXtkbQHOJfsFy4R8XuyX+anADdH+glhh+Yk0UtExPcj4lyykz2Am8i+6b8nt9gHalikv0vlODUijib70FfFMpVvtH8FTpN0CvBpYFGPl9IGsvz5twW4JyKG5aajImIOgKSRwGzge8DNkoaUbMcqOEn0ApI+Ium8dOLuJ/tG/yZZnf9kScdK+gDZr41aeR+wD9ib3mBfPdQKqYrrfuD7wOMR8ZueLaINcNuBP0qP/xn4j5IukDRI0rtTQ48TJInsV8R8YDqwDbi+ZDtWwUmidxgCzAFeAl4gu5B8LVl1zS/JLqz9BLi3hmX6JnAmsBd4CHigyvUWAqfiqibreX8P/E2qWvozYArwdeBFsl8WXyX7jPsS2Xvqv6dqpiuAKyT9u8rtSPqrGh9DrydXzdnhJOkPgV8BH4iIl+tdHjPrHv+SsMNG0ruArwCLnSDM+gffKWuHhaSjyOp2nydr/mpm/YCrm8zMrJSrm8zMrFS/q246/vjjY9SoUfUuxiG9+uqrHHXUUfUuxmHRH4/lySeffCki3l/v8lSjo3O+P/1vDie/LgcrO+f7XZIYNWoUq1evrncxDqmlpYXGxsZ6F+Ow6I/HIun5epelWh2d8/3pf3M4+XU5WNk57+omMzMr5SRhZmalnCTMzKyUk4SZmZVykjAzs1JOEmZmVspJwszMSjlJmJlZKScJMzMr1e/uuO7IqFkPFcY3z/lUjUtiVhtrt+7l8oLz3ue8Vcu/JMzMrJSThJmZlXKSMDOzUk4SZmZWyknCzMxKOUmYmVkpJwkzMyvlJGFWYf/+/QB/IumXktZJ+iaApLskPSdpTZrGpbgkzZXUKulpSWe2b0vSNEkb0zQtFz9L0tq0zlxJSvFjJTWn5ZslDa/t0Zu9k5OEWYUhQ4YAPBsRpwPjgCZJE9Lsr0bEuDStSbELgTFpmgHMg+wDH5gNnAOcDczOfejPAz6fW68pxWcBKyJiDLAiPTerm0MmCUkLJO2Q9Ewu9j8k/Sp9a3pQ0rDcvGvTt6NnJV2QizelWKukWbn4aEmrUvxeSUem+JD0vDXNH3W4DtqsI+lL/Zvp6RFpig5WmQLcHZmVwDBJI4ALgOaI2BURu4FmsoQzAjg6IlZGRAB3A5/JbWtherwwFzeri2q65bgL+BbZidyuGbg2Ig5Iugm4FrhG0ljgUuBk4IPATyV9OK1zB3A+0AY8IWlpRKwHbgJujYjFkv4JmE72LWs6sDsiTpJ0aVruz7p3uGbVk7QGOAm4IyJWSfqvwI2S/pb0LT8iXgdGAltyq7alWEfxtoI4QENEbEuPXwAaSso2g+xXCw0NDbS0tBQeQ8NQuPrUAwfFy5YfKPbt2zfgX4NqHTJJRMTPKr/FR8RPck9XAhelx1OAxemN85ykVrKf2QCtEbEJQNJiYIqkDcB5wJ+nZRYC3yBLElPSY4D7gW9JUvrmZdbjImJc+pX8oKRTyL4MvQAcCdwJXANc14P7D0mF53tE3JnKwPjx46OxsbFwG7cvWsLNaw9+m2/+bPHyA0VLSwtlr5m90+Ho4O9zwL3p8UiypNEu/w2p8hvVOcBxwJ6IOFCw/FvfwtIvlr1p+ZcqC1Dtt6qib1RQn29V/embTH8+lojYI+lRoCki/iGFX5f0PeCv0vOtwIm5zZyQYluBxop4S4qfULA8wHZJIyJiW6qW2nEYDsusy7qVJCT9NXAAWHR4itM11X6rKuoNE+rzrao/fZPpb8dy8sknAwwCkDSUrJr0ptyHt8iuFbRfp1sKXJl+IZ8D7E3LLQf+LnexehJZNe0uSS+ni+GrgKnA7bltTQPmpL9LeviQzTrU5SQh6XLg08DEXBVQ2TcqSuI7yS7yDU6/JvLLt2+rTdJg4Ji0vFmP2rZtG8BHJD1N1rjjvoj4kaRHJL0fELAG+EJaZRkwGWgFXgOuAEjJ4HrgibTcdRGxKz3+Itn1vqHAw2mCLDncJ2k68DxwSU8dp1k1upQkJDUBXwM+ERGv5WYtBb4v6RayC9djgMfJ3lRjJI0m+/C/FPjzVOf6KNk1jcW885tT+zeqn6f5j/h6hNXCaaedBrA+Isbn4xFxXtHy6bycWTJvAbCgIL4aOKUgvhOY2PlSm/WMQyYJST8gq1c9XlIbWbvva4EhQHNqLrgyIr4QEesk3QesJ6uGmhkRb6TtXAksJ/sZvyAi1qVdXAMslnQD8BQwP8XnA/eki9+7yBKLmZnVUDWtmy4rCM8viLUvfyNwY0F8GdnP8sr4Jt5uAZWP7wcuPlT5zMys5/iOazMzK+UkYWZmpZwkzMyslJOEmZmVcpIwM7NSThJmZlbKScLMzEo5SZiZWSknCTMzK+UkYWZmpZwkzMyslJOEmZmVcpIwM7NSThJmZlbKScLMzEo5SZiZWSknCTMzK+UkYVZh//79AH8i6ZeS1kn6JoCk0ZJWSWqVdK+kI1N8SHremuaPat+WpGtT/FlJF+TiTSnWKmlWLl64D7N6cZIwqzBkyBCAZyPidGAc0CRpAnATcGtEnATsBqanVaYDu1P81rQcksaSjc1+MtAEfFvSIEmDgDuAC4GxwGVpWTrYh1ldOEmYVZAE8GZ6ekSaAjgPuD/FFwKfSY+npOek+ROVbWQKsDgiXo+I54BWsvHczwZaI2JTRPwOWAxMSeuU7cOsLgbXuwBmvZWkNcBJZN/6fw3siYgDaXYbMDI9HglsAYiIA5L2Asel+MrcJvPrbKmIn5PWKdtHZdlmADMAGhoaaGlpKTyGhqFw9akHDoqXLT9Q7Nu3b8C/BtVykjArERHjJA0DHgQ+Wu/y5EXEncCdAOPHj4/GxsbC5W5ftISb1x78Nt/82eLlB4qWlhbKXjN7J1c3mXUgIvYAjwIfA4ZJav/EPQHYmh5vBU4ESPOPAXbm4xXrlMV3drAPs7o4ZJKQtEDSDknP5GLHSmqWtDH9HZ7ikjQ3tcx4WtKZuXWmpeU3SpqWi58laW1aZ26qly3dh1lPe/HFFwEGAUgaCpwPbCBLFhelxaYBS9Ljpek5af4jEREpfmlq/TQaGAM8DjwBjEktmY4ku7i9NK1Ttg+zuqjml8RdZC0z8mYBKyJiDLAiPYestcaYNM0A5kH2gQ/MJqt3PRuYnfvQnwd8Prde0yH2Ydajtm3bBvARSU+TfaA3R8SPgGuAr0hqJbt+MD+tMh84LsW/QjpXI2IdcB+wHvgxMDMi3kjXHK4ElpMln/vSsnSwD7O6OOQ1iYj4Wb7ddzIFaEyPFwItZCf3FODu9I1opaRhkkakZZsjYheApGayZoUtwNERsTLF7yZrzfFwB/sw61GnnXYawPqIGJ+PR8Qmsi85VMT3AxcXbSsibgRuLIgvA5YVxAv3YVYvXb1w3RAR29LjF4CG9PitVh5Je+uMjuJtBfGO9nGQalt6FLXygPq09OhPrSt8LGb9V7dbN0VESIrDUZiu7qPalh6Xz3qoMF6Plh79qXWFj8Ws/+pq66btqRqJ9HdHine2NcfW9Lgy3tE+zMysRrqaJPKtOSpbeUxNrZwmAHtTldFyYJKk4emC9SRgeZr3sqQJqVXTVIpbjLiVh5lZHRyyuknSD8guIB8vqY2sldIc4D5J04HngUvS4suAyWTdD7wGXAEQEbskXU/WUgTguvaL2MAXyVpQDSW7YP1wipftw8zMaqSa1k2XlcyaWLBsADNLtrMAWFAQXw2cUhDfWbQPMzOrHd9xbWZmpZwkzMyslJOEmZmVcpIwM7NSThJmZlbKScLMzEo5SZiZWSknCTMzK+UkYWZmpZwkzMyslJOEmZmVcpIwM7NSThJmZlbKScKswpYtWwA+LGm9pHWSrgKQ9A1JWyWtSdPk9nUkXSupVdKzki7IxZtSrFXSrFx8tKRVKX6vpCNTfEh63prmj6rVcZsVcZIwqzB48GCAtogYC0wAZkoam2bfGhHj0rQMIM27FDgZaAK+LWmQpEHAHcCFwFjgstx2bkrbOgnYDUxP8enA7hS/NS1nVjdOEmYVRowYAdmgWUTEK8AGYGQHq0wBFkfE6xHxHNmgW2enqTUiNkXE74DFwJQ0CuN5wP1p/YXAZ3LbWpge3w9MTMub1cUhBx0yG8hSdc8ZwCrg48CVkqYCq4GrI2I3WQJZmVutjbeTypaK+DnAccCeiDhQsPzI9nUi4oCkvWn5lyrKNQOYAdDQ0EBLS0th+RuGwtWnHjgoXrb8QLFv374B/xpUy0nCrISk9wI/BL4cES9LmgdcD0T6ezPwuXqULSLuBO4EGD9+fDQ2NhYud/uiJdy89uC3+ebPFi8/ULS0tFD2mtk7ubrJrJjIEsSiiHgAICK2R8QbEfEm8B2y6iSArcCJuXVPSLGy+E5gmKTBFfF3bCvNPyYtb1YXThJmFbKh2vkQsCEibmmPSxqRW+xPgWfS46XApall0mhgDPA48AQwJrVkOpLs4vbSNBb8o8BFaf1pwJLctqalxxcBj6TlzerC1U1mFR577DHIrgOcJ2lNCn+drHXSOLLqps3AXwBExDpJ9wHrgQPAzIh4A0DSlcByYBCwICLWpe1dAyyWdAPwFDA/xecD90hqBXaRJRazunGSMKtw7rnnAjwZEeMrZi0rWycibgRuLIgvK1ovIjbxdnVVPr4fuLiTRTbrMa5uMjOzUk4SZmZWqltJQtJ/S90WPCPpB5Le3ZXuBjrbpYGZmdVGl5OEpJHAl4DxEXEK2YW5S+lkdwNd7NLAzMxqoLvVTYOBoak993uAbXS+u4FOdWnQzfKamVkndLl1U0RslfQPwG+A3wI/AZ6k890NdLZLg4NU20VBUfcEUJ8uCvpTtwA+FrP+q8tJQtJwsm/2o4E9wL+QVRfVXLVdFFw+66HCeD26KOhP3QL4WMz6r+5UN30SeC4iXoyI3wMPkHWA1tnuBjrbpYGZmdVId5LEb4AJkt6Tri1MJLvjtLPdDXSqS4NulNfMzDqpO9ckVkm6H/gFWVcET5FV+TxEJ7ob6GKXBmZmVgPd6pYjImYDsyvCne5uoLNdGpiZWW34jmszMyvlJGFmZqXcCywwqqRpLMDmOZ86bOuYmfU1/iVhZmalnCTMzKyUk4SZmZVykjAzs1JOEmZmVspJwqzCli1bAD4saX0aVOsqAEnHSmqWtDH9HZ7ikjQ3DY71tKQz27claVpafqOkabn4WZLWpnXmpq5tSvdhVi9OEmYVBg8eDNAWEWOBCcDMNODVLGBFRIwBVqTnkA2MNSZNM4B5kH3gk/VIcA5ZLwSzcx/684DP59Zr70G5bB9mdeEkYVZhxIgRAK8BRMQrwAayMU7yA2dVDqh1d2RWkvWEPAK4AGiOiF0RsRtoBprSvKMjYmXq5PJuigfnyu/DrC58M51ZB9JY7GcAq4CGiNiWZr0ANKTHbw2olbQPnNVRvK0gTgf7qCxXVQNtNQwtHmxroA+s5MGlquckYVZC0nuBHwJfjoiX02UDACIiJEVP7r+jfVQ70Nbti5Zw89qD3+b1GGirN/HgUtVzdZNZMZEliEUR8UCKbU9VRaS/O1K8swNnbU2PK+Md7cOsLpwkzCpklwn4ELAhIm7JzcoPnFU5oNbU1MppArA3VRktByZJGp4uWE8Clqd5L0uakFo1TaV4cK78PszqwtVNZhUee+wxgOOA8yStSeGvA3OA+yRNB54HLknzlgGTgVayC95XAETELknXk42yCHBdROxKj78I3AUMBR5OEx3sw6wunCTMKpx77rkAT0bE+ILZEysDqYXSzKJtRcQCYEFBfDVwSkF8Z9E+zOrF1U1mZlbKScLMzEo5SZiZWSknCTMzK+UkYWZmpZwkzMysVLeShKRhku6X9CtJGyR9rBbdKZuZWW109z6J24AfR8RFko4E3kN209GKiJgjaRZZV8fX8M7ulM8h6yr5nFx3yuOBAJ6UtDT1mtnenfIqshuWmnj7pqOaGDXroVruzsysV+nyLwlJxwD/HpgPEBG/i4g91KY7ZTMzq4Hu/JIYDbwIfE/S6cCTwFXUpjvld6i22+SiLpN7QjVdEPenrop9LGb9V3eSxGDgTOAvI2KVpNuoGEWrFt0pp/1U1W3y5TWqOqqmG+b+1FWxj8Ws/+rOhes2siEeV6Xn95MljVp0p2xmZjXQ5SQRES8AWyR9JIUmAuupTXfKZmZWA91t3fSXwKLUsmkTWRfJ76Lnu1M2M7Ma6FaSiIg1ZE1XK/Vod8pmZlYbvuPazMxKOUmYmVkpJwkzMyvlJGFmZqWcJMyKjZK0Q9Iz7QFJ35C0VdKaNE3Ozbs2dUT5rKQLcvGmFGtNfZm1x0dLWpXi96YWgkgakp63pvmjanO4ZsWcJMyKvUTWoWSlWyNiXJqWAUgaC1wKnJzW+bakQZIGAXeQdW45FrgsLQtwU9rWScBuYHqKTwd2p/itaTmzunGSMCu2D9h1yKUyU4DFEfF6RDxHdi/Q2WlqjYhNEfE7YDEwJd0ceh5ZLwVwcEeY7R1k3g9MdBf5Vk/dvZnObKC5UtJUYDVwdeq5eCSwMrdMvjPKys4rzwGOA/ZExIGC5d/q8DIiDkjam5Z/KV+Iaju1bBha3LHlQO/E0B05Vs9Jwqx684DrycY9uR64GfhcPQpSbaeWty9aws1rD36bV9MJZX/mjhyr5+omsypFxPaIeCMi3gS+Q1adBJ3vvHIn2Xgqgyvi79hWmn9MWt6sLpwkzKrU3rtx8qdAe8unpcClqWXSaLLRFx8n649sTGrJdCTZxe2lqYuaR4GL0vqVHWG2d5B5EfBIWt6sLlzdZFZsNPBz4HhJbWRD7DZKGkdW3bQZ+AuAiFgn6T6yXpAPADMj4g0ASVeS9XQ8CFgQEevS9q8BFku6AXiKNMJj+nuPpFayC+eX9vSBmnXEScKs2HMRUdl55fzCJYGIuBG4sSC+jKwH5Mr4Jt6ursrH9wMXd7q0Zj3E1U1mZlbKScLMzEo5SZiZWSknCTMzK+UkYWZmpZwkzMyslJOEmZmVcpIwM7NSThJmZlbKScLMzEp1O0mkEbiekvSj9LzTwzJ2duhHMzOrjcPxS+IqYEPueaeGZezi0I9mZlYD3UoSkk4APgV8Nz3vyrCMnRr6sTvlNTOzzuluL7D/CHwNeF963pVhGTs79ONBqh3KsWgYx55QzbCI/Wn4RB+LWf/V5SQh6dPAjoh4UlLj4StS51U7lOPlsx6qSXmqGRqyPw2f6GMx67+680vi48B/kjQZeDdwNHAbaVjG9GuiaFjGtophGcuGeKSDuJmZ1UCXr0lExLURcUJEjCK78PxIRHyWzg/L2KmhH7taXjMz67yeGJmuU8MydnHoRzMzq4HDkiQiogVoSY87PSxjZ4d+NDOz2vAd12bFRknaIemZ9oCkYyU1S9qY/g5PcUmam276fFrSmbl1pqXlN0qaloufJWltWmduag5eug+zenGSMCv2EtnNnXmzgBURMQZYkZ5DdsPnmDTNAOZB9oEPzCZrun02MDv3oT8P+HxuvaZD7MOsLpwkzIrtI7t2lpe/IbTyRtG7I7OSrIXfCOACoDkidkXEbqAZaErzjo6Ilanxxt0U33Sa34dZXfTEhWuz/qohIralxy8ADenxWzeKJu03hHYUbyuId7SPd6j2BtKGocU3kQ70GwZ902T1nCTMuiAiQlLUax/V3kB6+6Il3Lz24Ld5NTd89me+abJ6rm4yq972VFVE+rsjxctuCO0ofkJBvKN9mNWFk4RZ9fI3hFbeKDo1tXKaAOxNVUbLgUmShqcL1pOA5Wney5ImpFZNUym+6TS/D7O6cHWTWbHRwM+B4yW1kbVSmgPcJ2k68DxwSVp2GTCZrAfj14ArACJil6TryXoPALguItovhn8RuAsYCjycJjrYh1ldOEmYFXsuIsYXxCdWBlILpZlFG4mIBcCCgvhq4JSC+M6ifZjVi6ubzMyslJOEmZmVcpIwM7NSThJmZlbKScLMzEo5SZiZWSknCTMzK+UkYWZmpZwkzMyslJOEmZmVcpIwM7NSThJmZlbKScLMzEo5SZiZWakuJwlJJ0p6VNJ6SeskXZXix0pqlrQx/R2e4pI0V1KrpKclnZnb1rS0/EZJ03LxsyStTevMTQO0mJlZjXTnl8QB4OqIGAtMAGZKGgvMAlZExBhgRXoOcCEwJk0zgHmQJRWyAV3OAc4GZrcnlrTM53PrNXWjvGZm1kldThIRsS0ifpEevwJsAEYCU4CFabGFwGfS4ynA3ZFZCQxLY/heADRHxK6I2A00A01p3tERsTIN6nJ3bltmZlYDh+WahKRRwBnAKqAhjeEL8ALQkB6PBLbkVmtLsY7ibQVxMzOrkW4PXyrpvcAPgS9HxMv5ywYREZKiu/uoogwzyKqwaGhooKWlpXC5q0890NNFAeD2ReVj15868hgA9u3bV1rOvsbHYtZ/dStJSDqCLEEsiogHUni7pBERsS1VGe1I8a3AibnVT0ixrUBjRbwlxU8oWP4gEXEncCfA+PHjo7GxsWgxLp/1UJVH1nM2f7YRgJaWFsrK2dcMtGORtBl4BXgDOBAR49O1tXuBUcBm4JKI2J0aW9wGTAZeAy5vr6ZNjTT+Jm32hohYmOJnAXcBQ4FlwFWpytWs5rrTuknAfGBDRNySm7UUaG+hNA1YkotPTa2cJgB7U7XUcmCSpOHpgvUkYHma97KkCWlfU3PbMqu3/xAR4yJifHruBhvWL3XnmsTHgf8CnCdpTZomA3OA8yVtBD6ZnkP2jWgT0Ap8B/giQETsAq4HnkjTdSlGWua7aZ1fAw93o7xmPckNNqxf6nJ1U0T8H6DsvoWJBcsHMLNkWwuABQXx1cApXS2jWQ8J4Cfpetv/TNWdNW2wUe11uIahxdfiBvp1F197ql63L1ybDUDnRsRWSX8ANEv6VX5mLRpsVHsd7vZFS7h57cFv8/ZrYwNVf7qO1tPcLYdZJ0XE1vR3B/Ag2TWF7amqiE402CiLV9Vgw6wWnCTMOkHSUZLe1/6YrKHFM7jBhvVTrm4y65wG4MF0P9Bg4PsR8WNJTwD3SZoOPA9ckpZfRtb8tZWsCewVkDXYkNTeYAMObrBxF1kT2Idxgw2rIycJs06IiE3A6QXxnbjBhvVDThK9xKgObvTbPOdTNSyJmdnbnCRqrD0ZXH3qgV5xB7iZWUd84drMzEo5SZiZWSknCTMzK+UkYWZmpZwkzMyslJOEmZmVcpIwM7NSThJmZlbKScLMzEr5jmuzAcjdwFi1/EvCzMxKOUmYmVkpJwkzMyvlJGFmZqWcJMzMrJRbN/UBZS1R3ArFzHqaf0mYmVmpXv9LQlITcBswCPhuRMypc5F6Dbd177983ltv0auThKRBwB3A+UAb8ISkpRGxvr4l6/06SiBlnFh6h3qf9/7yYXm9OkkAZwOtEbEJQNJiYArgJNEDuvrhsHbr3sLxuv2B0mU+763X6O1JYiSwJfe8DTinciFJM4AZ6ek+Sc/WoGzd8iU4Hnip3uWolm7qcHbhsRxind6q/Vg+VMcyHPK878Q5f1jPsz76Py3Sp95/NVJ4zvf2JFGViLgTuLPe5egMSasjYny9y3E4+Fhqr9pzvq8cT635daleb2/dtBU4Mff8hBQz68983luv0duTxBPAGEmjJR0JXAosrXOZzHqaz3vrNXp1dVNEHJB0JbCcrCnggohYV+diHS59qnrsEHwsh9FhPu/rfjy9lF+XKiki6l0GMzPrpXp7dZOZmdWRk4SZmZVykqgBSQsk7ZD0TC52rKRmSRvT3+H1LGO1JJ0o6VFJ6yWtk3RVivep45H0bkmPS/plOo5vpvhoSasktUq6N1047nMkNUl6Nh3HrHqXp6d15j2mzNz02jwt6czcOtPS8hslTavHsfQ2ThK1cRfQVBGbBayIiDHAivS8LzgAXB0RY4EJwExJY+l7x/M6cF5EnA6MA5okTQBuAm6NiJOA3cD0OpaxS3LdelwIjAUuS/+j/uwuqn+PXQiMSdMMYB5kSQWYTXbj4tnA7N7+ZacWnCRqICJ+BuyqCE8BFqbHC4HP1LRQXRQR2yLiF+nxK1fJFEYAAAHgSURBVMAGsjuE+9TxRGZfenpEmgI4D7g/xXv9cZR4q1uPiPgd0N6tR7/VyffYFODudA6sBIZJGgFcADRHxK6I2A00c3DiGXCcJOqnISK2pccvAA31LExXSBoFnAGsog8ej6RBktYAO8g+EH4N7ImIA2mRNrIE2NcUdevRF4+ju8rOybLXx69bASeJXiCydsh9qi2ypPcCPwS+HBEv5+f1leOJiDciYhzZHc1nAx+tc5Gsh/SVc7I3cpKon+3pJy7p7446l6dqko4gSxCLIuKBFO6zxxMRe4BHgY+RVT2032TaV7vDcLcembJzsuz18etWwEmifpYC7a0npgFL6liWqkkSMB/YEBG35Gb1qeOR9H5Jw9LjoWRjN2wgSxYXpcV6/XGUcLcembJzcikwNbVymgDsTdVSy4FJkoanC9aTUmxgiwhPPTwBPwC2Ab8nq+ecDhxH1uJiI/BT4Nh6l7PKYzmX7Gf708CaNE3ua8cDnAY8lY7jGeBvU/yPgMeBVuBfgCH1LmsXj28y8P/IrrP8db3LU4Pjrfo9Bois9devgbXA+Nx2Ppf+963AFfU+rt4wuVsOMzMr5eomMzMr5SRhZmalnCTMzKyUk4SZmZVykjAzs1JOEmZmVspJwszMSv1/8VXZaPwdETwAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "du2hEJ_ptHLe",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 195
        },
        "outputId": "4f0e9757-5edb-4e75-a2d3-13c52c0b25dd"
      },
      "source": [
        "length_df.head()"
      ],
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>text</th>\n",
              "      <th>summary</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>23</td>\n",
              "      <td>6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>18</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>39</td>\n",
              "      <td>6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>17</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>13</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   text  summary\n",
              "0    23        6\n",
              "1    18        5\n",
              "2    39        6\n",
              "3    17        4\n",
              "4    13        4"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2hemvDLxtcMu",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "max_len_text=100\n",
        "max_len_summary=10"
      ],
      "execution_count": 26,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "If9THwUlsdoP",
        "colab_type": "text"
      },
      "source": [
        "## Splitting of the dataset into trainning and testing"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2dAz_sZktj2X",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "x_tr, x_val, y_tr, y_val = train_test_split(data['cleaned_text'],data['cleaned_summary'],test_size=0.1,random_state=0,shuffle=True)"
      ],
      "execution_count": 27,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dHTurzc9sm33",
        "colab_type": "text"
      },
      "source": [
        "## Preparing the Tokenizer\n",
        "A tokenizer builds the vocabulary and converts a word sequence to an integer sequence."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fnXG3f6dtqbN",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#prepare a tokenizer for reviews on training data\n",
        "x_tokenizer = Tokenizer()\n",
        "x_tokenizer.fit_on_texts(list(x_tr))\n",
        "\n",
        "#convert text sequences into integer sequences\n",
        "x_tr    =   x_tokenizer.texts_to_sequences(x_tr) \n",
        "x_val   =   x_tokenizer.texts_to_sequences(x_val)\n",
        "\n",
        "#padding zero upto maximum length\n",
        "x_tr    =   pad_sequences(x_tr,  maxlen=max_len_text, padding='post') \n",
        "x_val   =   pad_sequences(x_val, maxlen=max_len_text, padding='post')\n",
        "\n",
        "x_voc_size   =  len(x_tokenizer.word_index) +1"
      ],
      "execution_count": 28,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KJfVQAJ3twLe",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 134
        },
        "outputId": "05cf85e8-475a-4ef7-d1b3-3f77117f70f5"
      },
      "source": [
        "x_tr"
      ],
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[  33,   27, 2788, ...,    0,    0,    0],\n",
              "       [  13,    1,    3, ...,    0,    0,    0],\n",
              "       [ 139,   24, 2447, ...,    0,    0,    0],\n",
              "       ...,\n",
              "       [  43,  234, 1468, ...,    0,    0,    0],\n",
              "       [   9,  463,    1, ...,    0,    0,    0],\n",
              "       [ 141,    4,   57, ..., 1002,   85,  247]], dtype=int32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9_dkpMcTt0XR",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 134
        },
        "outputId": "cd8f993e-c215-45ed-f5be-2874067b3bc9"
      },
      "source": [
        "x_val"
      ],
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[   11,    69,   753, ...,     0,     0,     0],\n",
              "       [ 1164,    68,    67, ...,     0,     0,     0],\n",
              "       [  334,  2673,   162, ...,     0,     0,     0],\n",
              "       ...,\n",
              "       [  154,   641,  1528, ...,     0,     0,     0],\n",
              "       [  172,   120,    52, ...,     0,     0,     0],\n",
              "       [   50,   172,  1142, ..., 18673,   246,   428]], dtype=int32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 30
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WSoV9ScDt5Hl",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "99080988-533d-4862-ab26-732a04ee1403"
      },
      "source": [
        "x_voc_size"
      ],
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "114678"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "f_ReHFk4t65b",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#preparing a tokenizer for summary on training data \n",
        "y_tokenizer = Tokenizer()\n",
        "y_tokenizer.fit_on_texts(list(y_tr))\n",
        "\n",
        "#convert summary sequences into integer sequences\n",
        "y_tr    =   y_tokenizer.texts_to_sequences(y_tr) \n",
        "y_val   =   y_tokenizer.texts_to_sequences(y_val) \n",
        "\n",
        "#padding zero upto maximum length\n",
        "y_tr    =   pad_sequences(y_tr, maxlen=max_len_summary, padding='post')\n",
        "y_val   =   pad_sequences(y_val, maxlen=max_len_summary, padding='post')\n",
        "\n",
        "y_voc_size  =   len(y_tokenizer.word_index) +1"
      ],
      "execution_count": 32,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fjotWcFGuAGt",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 134
        },
        "outputId": "c4760e2c-b8a3-44ff-891e-24c58d784813"
      },
      "source": [
        "y_tr"
      ],
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[   1,  184,    2, ...,    0,    0,    0],\n",
              "       [   1, 3817, 5623, ...,    0,    0,    0],\n",
              "       [   1, 4885, 4667, ...,    0,    0,    0],\n",
              "       ...,\n",
              "       [   1,    9,   70, ...,    0,    0,    0],\n",
              "       [  10,  506,   73, ...,   39,  269,    2],\n",
              "       [   1,  153, 2242, ...,    0,    0,    0]], dtype=int32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zUgoWUyuuEQL",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 134
        },
        "outputId": "a9d457ae-b199-4939-f975-c6a4ea9840e4"
      },
      "source": [
        "y_val"
      ],
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[   1, 1038,    2, ...,    0,    0,    0],\n",
              "       [   1,   14,   41, ...,    0,    0,    0],\n",
              "       [   1,  395,   20, ...,    0,    0,    0],\n",
              "       ...,\n",
              "       [   1,  240,    2, ...,    0,    0,    0],\n",
              "       [   1,  746,    2, ...,    0,    0,    0],\n",
              "       [3831, 1514,  424, ...,    4, 2880,    2]], dtype=int32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 34
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dNRD1_uxuGAX",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "0f650b30-ca7e-4d5f-ce95-1c875996be38"
      },
      "source": [
        "y_voc_size"
      ],
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "31030"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pjz311DYsyU8",
        "colab_type": "text"
      },
      "source": [
        "# MODEL CREATION \n",
        "Here, I am using three stacked layer of Bidirectional LSTM as encoder and then a single layer LSTM as decoder.\n",
        "So, basically we are implementing Seq2Seq Model for text summarization."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "A_r1YKhtz1Z8",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 756
        },
        "outputId": "56cf8a52-7e16-4471-f734-1a0ed9294912"
      },
      "source": [
        "from keras import backend as K \n",
        "K.clear_session()\n",
        "\n",
        "latent_dim = 300\n",
        "embedding_dim=200\n",
        "\n",
        "# Encoder\n",
        "encoder_inputs = Input(shape=(max_len_text,))\n",
        "\n",
        "#embedding layer\n",
        "enc_emb =  Embedding(x_voc_size, embedding_dim,trainable=True)(encoder_inputs)\n",
        "\n",
        "#encoder lstm 1\n",
        "encoder_lstm1 = Bidirectional(LSTM(latent_dim,return_sequences=True,return_state=True,dropout=0.4,recurrent_dropout=0.4))\n",
        "encoder_output1, f_h1, f_c1, b_h1, b_c1 = encoder_lstm1(enc_emb)\n",
        "\n",
        "#encoder lstm 2\n",
        "encoder_lstm2 = Bidirectional(LSTM(latent_dim,return_sequences=True,return_state=True,dropout=0.4,recurrent_dropout=0.4))\n",
        "encoder_output2, f_h2, f_c2, b_h2, b_c2 = encoder_lstm2(encoder_output1)\n",
        "\n",
        "#encoder lstm 3\n",
        "encoder_lstm3 = Bidirectional(LSTM(latent_dim, return_state=True, return_sequences=True,dropout=0.4,recurrent_dropout=0.4))\n",
        "encoder_outputs, f_h, f_c, b_h, b_c = encoder_lstm3(encoder_output2)\n",
        "\n",
        "state_h = Concatenate()([f_h, b_h])\n",
        "state_c = Concatenate()([f_c, b_c])\n",
        "\n",
        "# Set up the decoder, using `encoder_states` as initial state.\n",
        "decoder_inputs = Input(shape=(None,))\n",
        "\n",
        "#embedding layer\n",
        "dec_emb_layer = Embedding(y_voc_size, embedding_dim, trainable=True)\n",
        "dec_emb = dec_emb_layer(decoder_inputs)\n",
        "\n",
        "decoder_lstm = LSTM(latent_dim*2, return_sequences=True, return_state=True,dropout=0.4,recurrent_dropout=0.2)\n",
        "decoder_outputs, decoder_fwd_state, decoder_back_state = decoder_lstm(dec_emb, initial_state= [state_h, state_c])\n",
        "\n",
        "#dense layer\n",
        "decoder_dense =  TimeDistributed(Dense(y_voc_size, activation='softmax'))\n",
        "decoder_outputs = decoder_dense(decoder_outputs)\n",
        "\n",
        "# Define the model \n",
        "model = Model([encoder_inputs, decoder_inputs], decoder_outputs)\n",
        "\n",
        "model.summary()"
      ],
      "execution_count": 36,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:Layer lstm will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU\n",
            "WARNING:tensorflow:Layer lstm will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU\n",
            "WARNING:tensorflow:Layer lstm will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU\n",
            "WARNING:tensorflow:Layer lstm_1 will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU\n",
            "WARNING:tensorflow:Layer lstm_1 will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU\n",
            "WARNING:tensorflow:Layer lstm_1 will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU\n",
            "WARNING:tensorflow:Layer lstm_2 will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU\n",
            "WARNING:tensorflow:Layer lstm_2 will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU\n",
            "WARNING:tensorflow:Layer lstm_2 will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU\n",
            "WARNING:tensorflow:Layer lstm_3 will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU\n",
            "Model: \"functional_1\"\n",
            "__________________________________________________________________________________________________\n",
            "Layer (type)                    Output Shape         Param #     Connected to                     \n",
            "==================================================================================================\n",
            "input_1 (InputLayer)            [(None, 100)]        0                                            \n",
            "__________________________________________________________________________________________________\n",
            "embedding (Embedding)           (None, 100, 200)     22935600    input_1[0][0]                    \n",
            "__________________________________________________________________________________________________\n",
            "bidirectional (Bidirectional)   [(None, 100, 600), ( 1202400     embedding[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "bidirectional_1 (Bidirectional) [(None, 100, 600), ( 2162400     bidirectional[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "input_2 (InputLayer)            [(None, None)]       0                                            \n",
            "__________________________________________________________________________________________________\n",
            "bidirectional_2 (Bidirectional) [(None, 100, 600), ( 2162400     bidirectional_1[0][0]            \n",
            "__________________________________________________________________________________________________\n",
            "embedding_1 (Embedding)         (None, None, 200)    6206000     input_2[0][0]                    \n",
            "__________________________________________________________________________________________________\n",
            "concatenate (Concatenate)       (None, 600)          0           bidirectional_2[0][1]            \n",
            "                                                                 bidirectional_2[0][3]            \n",
            "__________________________________________________________________________________________________\n",
            "concatenate_1 (Concatenate)     (None, 600)          0           bidirectional_2[0][2]            \n",
            "                                                                 bidirectional_2[0][4]            \n",
            "__________________________________________________________________________________________________\n",
            "lstm_3 (LSTM)                   [(None, None, 600),  1922400     embedding_1[0][0]                \n",
            "                                                                 concatenate[0][0]                \n",
            "                                                                 concatenate_1[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "time_distributed (TimeDistribut (None, None, 31030)  18649030    lstm_3[0][0]                     \n",
            "==================================================================================================\n",
            "Total params: 55,240,230\n",
            "Trainable params: 55,240,230\n",
            "Non-trainable params: 0\n",
            "__________________________________________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PJn1by3uuR8e",
        "colab_type": "text"
      },
      "source": [
        "## Compliling the model with RMSprop optimizer and loss function as sparse categorical cross entropy."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "27YNe_f134hF",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model.compile(optimizer='rmsprop', loss='sparse_categorical_crossentropy', metrics=['accuracy'])"
      ],
      "execution_count": 37,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "65wZCrxSudWc",
        "colab_type": "text"
      },
      "source": [
        "## using early stopping"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nol-GGyFXKT7",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "es = EarlyStopping(monitor='val_loss', mode='min', verbose=1)"
      ],
      "execution_count": 38,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FTE2SEo9ujTr",
        "colab_type": "text"
      },
      "source": [
        "## Trainning the model :) \n",
        "I have used 3 epochs as it's taking too time better to use 20 epochs."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RPZcwBZd4k0l",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 118
        },
        "outputId": "a4d100dc-abe7-4a8b-f600-d61136a2fcd7"
      },
      "source": [
        "history = model.fit([x_tr,y_tr[:,:-1]], y_tr.reshape(y_tr.shape[0],y_tr.shape[1], 1)[:,1:] ,epochs=3, batch_size= 256, validation_data=([x_val,y_val[:,:-1]], y_val.reshape(y_val.shape[0],y_val.shape[1], 1)[:,1:]))"
      ],
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/3\n",
            "1383/1383 [==============================] - 4908s 4s/step - loss: 2.6515 - accuracy: 0.6150 - val_loss: 2.3644 - val_accuracy: 0.6372\n",
            "Epoch 2/3\n",
            "1383/1383 [==============================] - 4863s 4s/step - loss: 2.3224 - accuracy: 0.6421 - val_loss: 2.1895 - val_accuracy: 0.6535\n",
            "Epoch 3/3\n",
            "1383/1383 [==============================] - 4866s 4s/step - loss: 2.2021 - accuracy: 0.6527 - val_loss: 2.1207 - val_accuracy: 0.6599\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ps6dByo3uxK4",
        "colab_type": "text"
      },
      "source": [
        "### model evaluation on testing data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dnmej6tD4oDd",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 50
        },
        "outputId": "b5e11315-38ef-4ac2-d2ef-ab5573079872"
      },
      "source": [
        "model.evaluate([x_val, y_val], y_val)"
      ],
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "1229/1229 [==============================] - 229s 186ms/step - loss: 5.2760 - accuracy: 0.4247\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[5.276016712188721, 0.42467257380485535]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 43
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0326OCGYu0df",
        "colab_type": "text"
      },
      "source": [
        "## Visualization of the loss curve "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WtwcTifjDw-o",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 483
        },
        "outputId": "220653cc-37fb-4054-8277-fd5641a02633"
      },
      "source": [
        "plt.figure(figsize=(15,8))\n",
        "plt.plot(history.history['loss'], label='train') \n",
        "plt.plot(history.history['val_loss'], label='test') \n",
        "plt.legend() \n",
        "plt.show()"
      ],
      "execution_count": 52,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2oAAAHSCAYAAACdLTg6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3SW9f3G8eubPQkJYa+wCQk7QZERBxtFhnvWVnELYp2trW1tHf01Ai7qqtW6GaKAGLCQsCEMIYGwEwgzkEH2vH9/3BFHAYMkuZ8k79c5nkPzPHlycY71cPG9P9+PsSxLAAAAAADX4eZ0AAAAAADAj1HUAAAAAMDFUNQAAAAAwMVQ1AAAAADAxVDUAAAAAMDFUNQAAAAAwMV4OPWDQ0NDrbCwMKd+PAAAAAA4auPGjScsy2p6ptccK2phYWFKTEx06scDAAAAgKOMMWlne41HHwEAAADAxVDUAAAAAMDFUNQAAAAAwMU4NqMGAAAAoGErLS1Venq6ioqKnI5So3x8fNSmTRt5enpW+XsoagAAAAAckZ6ersDAQIWFhckY43ScGmFZlk6ePKn09HR16NChyt/Ho48AAAAAHFFUVKQmTZrU25ImScYYNWnS5LxPDSlqAAAAABxTn0vad37J75GiBgAAAKBBys7O1muvvXbe3zdmzBhlZ2fXQKLvUdQAAAAANEhnK2plZWXn/L5FixapcePGNRVLEpeJAAAAAGignnjiCe3du1d9+vSRp6enfHx8FBwcrJSUFO3atUvjx4/XwYMHVVRUpClTpmjy5MmSpLCwMCUmJiovL0+jR4/W4MGDtXr1arVu3Vrz58+Xr6/vBWejqAEAAABw3J++TNb2w6eq9TN7tGqkP14VcdbXn3/+eSUlJWnLli1avny5xo4dq6SkpNO3M77zzjsKCQlRYWGhoqOjNWnSJDVp0uRHn7F792599NFHevPNN3Xddddpzpw5uuWWWy44O0UNAAAAACQNGDDgR1foz5w5U/PmzZMkHTx4ULt37/6fotahQwf16dNHktS/f3+lpqZWSxaKGgAAAADHnevkq7b4+/uf/vXy5cu1dOlSrVmzRn5+frr00kvPeMW+t7f36V+7u7ursLCwWrJwmQgAAACABikwMFC5ublnfC0nJ0fBwcHy8/NTSkqK1q5dW6vZOFEDAAAA0CA1adJEgwYNUmRkpHx9fdW8efPTr40aNUqzZs1SeHi4unXrposvvrhWsxnLsmr1B34nKirKSkxMdORnAwAAAHDejh07FB4e7nSMWnGm36sxZqNlWVFnej+PPv5AUWm5jp363+dOAQAAAKA2UdR+YOY3uzXsH/H6cN0BVVQ4c9IIAAAAABS1H7guqq0iWwfpqXnbdNNba5V6It/pSAAAAAAaIIraD4SF+uvDuy7SC5N6KvnwKY2cnqDXl+9VWXmF09EAAAAANCAUtZ8wxuj66Hb6ZlqMLuvWTC8sTtHVr65S0qEcp6MBAAAAaCAoamfRrJGPZt3aX7Nu6afjucW6+tVVev6rFBWVljsdDQAAAEA9R1H7GaMiW2rpwzG6pl8bzYrfq9EzVmjtvpNOxwIAAABwgbKzs/Xaa6/9ou+dPn26CgoKqjnR9yhqVRDk56kXrumlD++8SOUVlm54Y62enLtNOYWlTkcDAAAA8Au5clHzqLFProcu6Ryqr6cO1UtLd+mtFfv0zY5j+sv4SI2MaOF0NAAAAADn6YknntDevXvVp08fDR8+XM2aNdOnn36q4uJiTZgwQX/605+Un5+v6667Tunp6SovL9fTTz+tY8eO6fDhw7rssssUGhqqZcuWVXs2itp58vVy11NjwnVlr5Z6bPZW3f3+Ro3p2ULPjItQs0Afp+MBAAAAddNXT0hHt1XvZ7boKY1+/qwvP//880pKStKWLVsUFxen2bNna/369bIsS+PGjVNCQoIyMjLUqlUrLVy4UJKUk5OjoKAgxcbGatmyZQoNDa3ezJV49PEX6tWmsb58cLAeHdlNS3cc1/DYBH2aeFCWxaJsAAAAoK6Ji4tTXFyc+vbtq379+iklJUW7d+9Wz549tWTJEj3++ONasWKFgoKCaiUPJ2oXwNPdTfdf1lmjIlvoyTnb9Njsrfpiy2H9bUJPtWvi53Q8AAAAoO44x8lXbbAsS08++aTuvvvu/3lt06ZNWrRokX7/+9/riiuu0B/+8Icaz8OJWjXo1DRAH0++WM+Oj9SWg9kaOT1Bb63Yp/IKTtcAAAAAVxUYGKjc3FxJ0siRI/XOO+8oLy9PknTo0CEdP35chw8flp+fn2655RY9+uij2rRp0/98b03gRK2auLkZ3XJxe10R3kxPf56kZxfu0JffHtbzk3opvGUjp+MBAAAA+IkmTZpo0KBBioyM1OjRo3XTTTdp4MCBkqSAgAD95z//0Z49e/Too4/Kzc1Nnp6eev311yVJkydP1qhRo9SqVasauUzEODVTFRUVZSUmJjrys2uaZVlasPWInvkiWTmFpbr30k564PLO8vZwdzoaAAAA4DJ27Nih8PBwp2PUijP9Xo0xGy3LijrT+3n0sQYYY3RV71ZaOi1G4/q00sv/3aMxM1YoMTXT6WgAAAAA6gCKWg0K9vdS7HV99O9fD1BRaYWumbVGT3+epNwiFmUDAAAAODuKWi2I6dpUcQ8P1a8HddB/1qVpxEsJ+m/KMadjAQAAAHBRFLVa4u/toT9c1UNz771EgT4e+vW7iXroo806mVfsdDQAAADAMQ1hD/Ev+T1S1GpZ33bBWvDgED08rKu+SjqiYbHxmrc5vUH8CwoAAAD8kI+Pj06ePFmv/yxsWZZOnjwpHx+f8/o+bn100O5juXp8zlZtOpCtmK5N9dcJkWoTzKJsAAAANAylpaVKT09XUVGR01FqlI+Pj9q0aSNPT88fff1ctz5S1BxWXmHp/TWpevHrnZKkx0Z2060Dw+TuZpwNBgAAAKBGcT2/C3N3M/rVoA6Ke3ioosNC9MyX23XtrNXafazmtpwDAAAAcG0UNRfRJthP794RrZeu7639J/I1ZuYKTV+6SyVlFU5HAwAAAFDLKGouxBijCX3baOm0GI3p2VLTl+7WlS+v0KYDWU5HAwAAAFCLKGouqEmAt2bc0Ffv/CpKeUVlmvT6av3py2TlF5c5HQ0AAABALaCoubDLuzdX3LQY3Xpxe/1rVapGvJSghF0ZTscCAAAAUMMoai4uwNtDf746UrPvGSgfTzfd9s56Tft0i7LyS5yOBgAAAKCGUNTqiKiwEC18aIgevLyzvthyWMNi4/Xlt4fr9XJAAAAAoKGiqNUhPp7uemREN3354GC1CfbVgx9t1l3vJepITqHT0QAAAABUI4paHRTespHm3jdIvx8brpV7Tmh4bILeX5umigpO1wAAAID6gKJWR7m7Gd05pKPipsaoT9vGevrzJN3wxlrtzchzOhoAAACAC0RRq+PaNfHT+78ZoL9f00s7j+Vq9IwVenXZHpWWsygbAAAAqKsoavWAMUbXRrXVkmlDNTy8uf7+9U6Ne2WVtqZnOx0NAAAAwC9AUatHmgX66NWb++mNW/srM79Y419dpb8t2qHCknKnowEAAAA4DxS1emhERAstmRaj66Pb6Y2EfRo5PUGr95xwOhYAAACAKvrZomaMaWuMWWaM2W6MSTbGTDnL+y41xmypfE989UfF+Wjk46nnJvbUx5Mvlrub0U1vrdPjs7cqp6DU6WgAAAAAfob5uYXJxpiWklpalrXJGBMoaaOk8ZZlbf/BexpLWi1plGVZB4wxzSzLOn6uz42KirISExMv/HeAn1VUWq4Z3+zWGwn7FOLvpT+Pi9Doni2djgUAAAA0aMaYjZZlRZ3ptZ89UbMs64hlWZsqf50raYek1j95202S5lqWdaDyfecsaahdPp7uenxUd82/f5CaBXrr3g826e73E3X8VJHT0QAAAACcwXnNqBljwiT1lbTuJy91lRRsjFlujNlojLmteuKhOkW2DtL8+wfpidHdtXxnhq6IjdfH6w/o505VAQAAANSuKhc1Y0yApDmSplqWdeonL3tI6i9prKSRkp42xnQ9w2dMNsYkGmMSMzIyLiA2fikPdzfdE9NJi6cOVUSrRnpi7jbd9OY6pZ7IdzoaAAAAgEpVKmrGGE/ZJe0Dy7LmnuEt6ZK+tiwr37KsE5ISJPX+6Zssy3rDsqwoy7KimjZteiG5cYE6hPrrwzsv1nMTeyrpcI5GTk/QP+P3qoxF2QAAAIDjqnLro5H0tqQdlmXFnuVt8yUNNsZ4GGP8JF0ke5YNLszNzejGAe20dFqMYro21XNfpWj8a6uUfDjH6WgAAABAg1aVE7VBkm6VdHnl9ftbjDFjjDH3GGPukSTLsnZIWixpq6T1kt6yLCupxlKjWjVv5KN/3tpfr93cT0dzijXulVV6YXGKikpZlA0AAAA44Wev568pXM/vmrILSvS3RTv0aWK6OoT66/mJPXVRxyZOxwIAAADqnQu6nh8NS2M/L714TW99cOdFKq+wdP0ba/XUvG06VcSibAAAAKC2UNRwRoM6h2rx1CG6a0gHfbz+gEbEJmjJ9mNOxwIAAAAaBIoazsrPy0O/G9tD8+4bpMZ+nrrrvUTd/+EmZeQWOx0NAAAAqNcoavhZvds21pcPDtZvR3TVkuRjGhYbr9kb01mUDQAAANQQihqqxNPdTQ9c3kWLpgxR1+YB+u1n3+q2d9brYGaB09EAAACAeoeihvPSuVmAPpk8UH+5OkKb0rI04qUEvbVin8orOF0DAAAAqgtFDefNzc3o1oFhWjItRgM7NdGzC3do4uurlXL0lNPRAAAAgHqBooZfrFVjX719e5Rm3thX6ZkFunLmSsXG7VRxGYuyAQAAgAtBUcMFMcZoXO9WWjItRuN6t9LM/+7R2JkrtTEt0+loAAAAQJ1FUUO1CPH3Uuz1ffTuHdEqLCnXNbPW6I/zk5RXXOZ0NAAAAKDOoaihWl3arZniHh6q2weG6b21aRoRG69lKcedjgUAAADUKRQ1VDt/bw89My5Cs++5RP7eHrrj3Q2a8vFmncxjUTYAAABQFRQ11Jj+7YO14KHBmnJFFy3adkTDX0rQ55sPsSgbAAAA+BkUNdQobw93PTy8qxY+NETtm/hp6idb9Ot3N+hQdqHT0QAAAACXRVFDrejaPFCz77lEf7yqh9btz9SI2Hj9e3WqKliUDQAAAPwPihpqjbub0R2DOujrqUPVPyxEf/wiWdf+c432HM91OhoAAADgUihqqHVtQ/z07zuiFXtdb+3NyNOYGSs185vdKimrcDoaAAAA4BIoanCEMUYT+7XR0mkxGhnZQrFLdumql1dqy8Fsp6MBAAAAjqOowVGhAd56+ca+evv2KJ0qKtXE11bpLwu2q6CERdkAAABouChqcAlXhDdX3MNDddNF7fT2yv0a8VKCVuzOcDoWAAAA4AiKGlxGoI+nnh3fU5/ePVBe7m669e31euTTb5VdUOJ0NAAAAKBWUdTgcgZ0CNGiKUP0wGWdNX/LIQ2LjdeCrYdZlA0AAIAGg6IGl+Tj6a7fjuymLx4YrFaNffXAh5t113sbdTSnyOloAAAAQI2jqMGl9WjVSHPvvUS/GxOulXsyNDw2Xh+sS2NRNgAAAOo1ihpcnoe7m+4a2lFfTx2qnm2C9Lt5SbrhzbXal5HndDQAAACgRlDUUGe0b+KvD+68SC9O6qWUI6c0asYKvbZ8j0rLWZQNAACA+oWihjrFGKProttq6SMxGhbeTC8u3qmrX1mlbek5TkcDAAAAqg1FDXVSs0AfvXZzf826pb9O5BVr/Gur9NyiHSosKXc6GgAAAHDBKGqo00ZFttCSaTG6LqqN/pmwT6NmJGj13hNOxwIAAAAuCEUNdV6Qr6eem9hLH951kYykm95cpyfmbFVOYanT0QAAAIBfhKKGeuOSTqFaPHWo7o7pqM82pmt4bLwWJx11OhYAAABw3ihqqFd8PN315Ohwzb9/kEIDvHXPfzbq3v9s1PFcFmUDAACg7qCooV6KbB2k+Q8M0mOjuumblOMa9o94fbLhgCyLRdkAAABwfRQ11Fue7m6679LOWjxliLq3bKTH52zTzW+tU9rJfKejAQAAAOdEUUO917FpgD6+62L9bUJPbUvP0cjpCXojYa/KWJQNAAAAF0VRQ4Pg5mZ000XttGRajIZ0aaq/LUrRhNdWa/vhU05HAwAAAP4HRQ0NSosgH71xa3+9elM/Hckp1LhXVurvX6eoqJRF2QAAAHAdFDU0OMYYje3VUkunxWh839Z6ddlejZm5Quv3ZzodDQAAAJBEUUMD1tjPS/93bW+9/5sBKimr0HX/XKPfzdum3CIWZQMAAMBZFDU0eEO6NFXcw0P1m8Ed9NH6Axoem6Cl2485HQsAAAANGEUNkOTn5aGnr+yhufcNUpCvp+58L1EPfLhJJ/KKnY4GAACABoiiBvxAn7aN9eWDg/XI8K6KSz6mYbHxmrMxnUXZAAAAqFUUNeAnvDzc9OAVXbRoymB1ahqgRz77Vrf/a4MOZhY4HQ0AAAANBEUNOIvOzQL12d0D9eerI7QxNVMjpyfonZX7VV7B6RoAAABqFkUNOAc3N6PbBoYpblqMLuoQoj8v2K5Jr6/WrmO5TkcDAABAPUZRA6qgdWNfvfOraM24oY8OZBZo7MwVil2yS8VlLMoGAABA9aOoAVVkjNHVfVpr6bQYje3ZUjO/2a0rZ67UxrQsp6MBAACgnqGoAecpxN9L02/oq3/dEa384jJdM2u1nvkiWfnFZU5HAwAAQD1BUQN+ocu6NVPctBjdPjBM/16TqhEvJWj5zuNOxwIAAEA9QFEDLkCAt4eeGReh2fcMlK+Xu371rw16+JMtyswvcToaAAAA6jCKGlAN+rcP0cKHBuuhK7powdbDGhYbr/lbDrEoGwAAAL8IRQ2oJt4e7po2vKsWPDhEbUP8NOXjLfrNvxN1OLvQ6WgAAACoYyhqQDXr1iJQc++9RE9f2UNr9p7U8Nh4vbcmVRUsygYAAEAVUdSAGuDuZvSbwR0U9/BQ9WsfrD/MT9Z1/1yjPcfznI4GAACAOoCiBtSgtiF+eu/XA/R/1/bW7uN5GjNjhV7+ZrdKyiqcjgYAAAAXRlEDapgxRtf0b6Ol02I0IqK5/rFkl8a9slLfHsx2OhoAAABcFEUNqCVNA731yk399OZtUcouKNWE11bp2QXbVVDComwAAAD8GEUNqGXDezRX3LShunFAO721cr9GTk/Qyt0nnI4FAAAAF0JRAxzQyMdTf53QU59Mvliebm665e11evSzb5VdwKJsAAAAUNQAR13UsYkWTRmi+y7tpLmbD2lYbIIWbj3ComwAAIAGjqIGOMzH012PjequLx4YpBZB3rr/w02a/P5GHTtV5HQ0AAAAOISiBriIiFZB+vy+QXpydHcl7MrQsH/E68N1B1iUDQAA0ABR1AAX4uHuprtjOunrqUMV2TpIT83bphvfXKv9J/KdjgYAAIBaRFEDXFBYqL8+vOsivTCpp7YfOaVR0xP0+vK9Ki1nUTYAAEBDQFEDXJQxRtdHt9M302J0WbdmemFxisa/ukpJh3KcjgYAAIAaRlEDXFyzRj6adWt/zbqln47nFuvqV1fp+a9SVFRa7nQ0AAAA1JCfLWrGmLbGmGXGmO3GmGRjzJQzvOdSY0yOMWZL5T9/qJm4QMM1KrKllj4co2v6tdGs+L0aNT1Ba/aedDoWAAAAakBVTtTKJD1iWVYPSRdLut8Y0+MM71thWVafyn/+XK0pAUiSgvw89cI1vfThnRepwpJufHOtnpy7VTmFpU5HAwAAQDX62aJmWdYRy7I2Vf46V9IOSa1rOhiAs7ukc6i+njpUk4d21CcbDmp4bLy+Tj7qdCwAAABUk/OaUTPGhEnqK2ndGV4eaIz51hjzlTEm4izfP9kYk2iMSczIyDjvsAC+5+vlrqfGhGv+/YPVJMBbd7+/Ufd9sFHHc1mUDQAAUNcZy6raMl1jTICkeEl/tSxr7k9eaySpwrKsPGPMGEkzLMvqcq7Pi4qKshITE39hbAA/VFpeoTcS9mnGN7vl6+mu340N17X928gY43Q0AAAAnIUxZqNlWVFneq1KJ2rGGE9JcyR98NOSJkmWZZ2yLCuv8teLJHkaY0IvIDOA8+Dp7qb7L+usr6YMUbfmgXps9lbd8vY6HThZ4HQ0AAAA/AJVufXRSHpb0g7LsmLP8p4Wle+TMWZA5edyHR1Qyzo1DdDHky/Ws+Mj9e3BHI2YHq+3VuxTeUXVTs4BAADgGjyq8J5Bkm6VtM0Ys6Xya09JaidJlmXNknSNpHuNMWWSCiXdYFX1mUoA1crNzeiWi9vrivBmevrzJD27cIe+/Pawnp/US+EtGzkdDwAAAFVQ5Rm16saMGlDzLMvSgq1H9MwXycopLNU9MZ30wOWd5ePp7nQ0AACABu+CZ9QA1E3GGF3Vu5WWTovRuD6t9MqyPRo7c4U2pGY6HQ0AAADnQFEDGoBgfy/FXtdH//71ABWVVujaWWv09OdJyi1iUTYAAIAroqgBDUhM16aKe3iofj2og/6zLk0jXkrQf1OOOR0LAAAAP0FRAxoYf28P/eGqHpp77yUK9PHQr99N1EMfbdbJvGKnowEAAKASRQ1ooPq2C9aCB4fo4WFd9VXSEQ2LjdfcTeniwlYAAADnUdSABszLw01ThnXRooeGqEOov6Z9+q1+9a8NSs9iUTYAAICTKGoA1KV5oD675xI9c1UPbUjN1IiXEvSvVftZlA0AAOAQihoASZK7m9GvBnVQ3MNDFR0Woj99uV3XzFqtXcdynY4GAADQ4FDUAPxIm2A/vXtHtF66vrdST+Rr7MwVmr50l0rKKpyOBgAA0GBQ1AD8D2OMJvRto6XTYjSmZ0tNX7pbV768QpsOZDkdDQAAoEGgqAE4qyYB3ppxQ1+986so5RWVadLrq/WnL5OVX1zmdDQAAIB6jaIG4Gdd3r254qbF6NaL2+vd1aka8VKC4ndlOB0LAACg3qKoAaiSAG8P/fnqSH1290D5eLrp9nfWa9onW5SVX+J0NAAAgHqHogbgvESFhWjhQ0P04OWd9cW3hzUsNl5ffHuYRdkAAADViKIG4Lz5eLrrkRHd9OWDg9Um2FcPfbRZd/47UUdyCp2OBgAAUC9Q1AD8YuEtG2nufYP0+7HhWrX3hIbHJuj9tWmqYFE2AADABaGoAbgg7m5Gdw7pqLipMerTtrGe/jxJN7yxVnsz8pyOBgAAUGdR1ABUi3ZN/PT+bwbo79f00s5juRo9Y4VeXbZHpeUsygYAADhfFDUA1cYYo2uj2mrJtKEaHt5cf/96p656eaW2pmc7HQ0AAKBOoagBqHbNAn306s399Mat/ZVVUKLxr67SXxduV2FJudPRAAAA6gSKGoAaMyKihZZMi9H10e305or9Gjk9Qav2nHA6FgAAgMujqAGoUY18PPXcxJ76ePLFcnczuvmtdXps9rfKKSh1OhoAAIDLoqgBqBUXd2yir6YM0b2XdtKcTYc07KV4fbXtiNOxAAAAXBJFDUCt8fF01+Ojumv+/YPULNBb936wSXe/n6hjp4qcjgYAAOBSKGoAal1k6yDNv3+QnhjdXct3ZmhYbLw+Wn9AlsWibAAAAImiBsAhHu5uuiemkxZPHaqIVo305NxtuvHNtUo9ke90NAAAAMdR1AA4qkOovz6882I9N7Gnkg+f0sjpCZoVv1dlLMoGAAANGEUNgOPc3IxuHNBOS6fFKKZrUz3/VYrGv7ZKyYdznI4GAADgCIoaAJfRvJGP/nlrf71+cz8dzSnWuFdW6YXFKSoqZVE2AABoWChqAFyKMUaje7bUN9NiNKlfa72+fK9Gz1ihtftOOh0NAACg1lDUALikID9PvXhNb31w50Uqr7B0wxtr9dS8bTpVxKJsAABQ/1HUALi0QZ1DtXjqEN01pIM+Xn9Aw2PjFZd81OlYAAAANYqiBsDl+Xl56Hdje2jefYMU7Oelye9v1P0fbFJGbrHT0QAAAGoERQ1AndG7bWN9+eBg/XZEVy3ZfkzDYuP1WeJBFmUDAIB6h6IGoE7xdHfTA5d30aIpQ9S1eYAenb1Vt72zXgczC5yOBgAAUG0oagDqpM7NAvTJ5IH6y9UR2pSWpREvJeitFftUXsHpGgAAqPsoagDqLDc3o1sHhmnJtBgN7NREzy7coYmvr1bK0VNORwMAALggFDUAdV6rxr56+/Yozbyxr9IzC3TlzJX6R9xOFZexKBsAANRNFDUA9YIxRuN6t9KSaTEa17uVXv7vHo2duVIb0zKdjgYAAHDeKGoA6pUQfy/FXt9H794RrcKScl0za43+OD9JecVlTkcDAACoMooagHrp0m7NFPfwUN0+MEzvrU3TiNh4LUs57nQsAACAKqGoAai3/L099My4CM2+5xL5e3vojnc3aMrHm3Uyj0XZAADAtVHUANR7/dsHa8FDgzXlii5atO2IhsXG6/PNh1iUDQAAXBZFDUCD4O3hroeHd9XCh4YoLNRfUz/Zojve3aD0LBZlAwAA10NRA9CgdG0eqNn3XKI/XtVD6/dnasRLCXp31X5VsCgbAAC4EIoagAbH3c3ojkEd9PXUoYoKC9EzX27XNbNWa/exXKejAQAASKKoAWjA2ob46d93RCv2ut7adyJfY2eu1Iylu1VSVuF0NAAA0MBR1AA0aMYYTezXRkunxWhkZAu9tHSXrnp5pTYfyHI6GgAAaMAoagAgKTTAWy/f2Fdv3x6lU0Wlmvj6av35y+0qKGFRNgAAqH0UNQD4gSvCmyvu4aG6+aJ2emfVfo14KUEJuzKcjgUAABoYihoA/ESgj6eeHd9Tn90zUF4ebrrtnfV65NNvlV1Q4nQ0AADQQFDUAOAsosNCtOihIXrgss6av+WQhsXGa8HWwyzKBgAANY6iBgDn4OPprt+O7KYvHhisVo199cCHm3XXe4k6klPodDQAAFCPUdQAoAp6tGqkufdeot+NCdfKPSc0IjZB/1mbxqJsAABQIyhqAFBFHu5uumtoR309dah6tgnS7z9P0g1vrtW+jDynowEAgHqGogYA56l9E399cOdFenFSL6UcObWuZScAACAASURBVKVRM1bo1WV7VFrOomwAAFA9KGoA8AsYY3RddFstfSRGw8Kb6e9f79S4V1ZpW3qO09EAAEA9QFEDgAvQLNBHr93cX7Nu6a+TecW6+tWVem7RDhWWlDsdDQAA1GEUNQCoBqMiW2jJtBhdH91W/0zYp1EzErR6zwmnYwEAgDqKogYA1STI11PPTeylD++6SEbSTW+t0+OztyqnsNTpaAAAoI6hqAFANbukU6gWTx2qu2M6avamdA2LjdfipCNOxwIAAHUIRQ0AaoCPp7ueHB2u+fcPUtMAb93zn0265/2NOn6qyOloAACgDqCoAUANimwdpPkPDNJjo7rpvzuPa1hsvD7ZcECWxaJsAABwdhQ1AKhhnu5uuu/Szlo8ZYi6t2ykx+ds081vrVPayXynowEAABdFUQOAWtKxaYA+vuti/W1CT21Lz9HI6Ql6I2GvyliUDQAAfuJni5oxpq0xZpkxZrsxJtkYM+Uc7402xpQZY66p3pgAUD+4uRnddFE7LZkWoyFdmupvi1I04bXVSj7MomwAAPC9qpyolUl6xLKsHpIulnS/MabHT99kjHGX9IKkuOqNCAD1T4sgH71xa3+9elM/Hckp1LhXVunFxSkqKmVRNgAAqEJRsyzriGVZmyp/nStph6TWZ3jrg5LmSDperQkBoJ4yxmhsr5ZaOi1GE/q21mvL92rMjBVat++k09EAAIDDzmtGzRgTJqmvpHU/+XprSRMkvV5dwQCgoWjs56X/u7a33v/NAJWUV+j6N9bqd/O2KbeIRdkAADRUVS5qxpgA2SdmUy3LOvWTl6dLetyyrHNOxBtjJhtjEo0xiRkZGeefFgDqsSFdmiru4aH6zeAO+mj9AQ2PTdDS7cecjgUAABxgqrLLxxjjKWmBpK8ty4o9w+v7JZnK/xkqqUDSZMuyPj/bZ0ZFRVmJiYm/KDQA1HdbDmbr8dlbtfNYrq7s1VLPjItQaIC307EAAEA1MsZstCwr6kyvVeXWRyPpbUk7zlTSJMmyrA6WZYVZlhUmabak+85V0gAA59anbWN9+eBgPTK8q+KSj2lYbLzmbExnUTYAAA1EVR59HCTpVkmXG2O2VP4zxhhzjzHmnhrOBwANlpeHmx68oosWTRmszk0D9Mhn3+q2d9brYGaB09EAAEANq9KjjzWBRx8BoOoqKiz9Z12aXvgqRRWW9NuR3fSrS8Lk7mZ+/psBAIBLuqBHHwEAznNzM7ptYJjipsXo4o4h+suC7Zr0+mrtPJrrdDQAAFADKGoAUIe0buyrd34VrRk39NGBzAJd+fIKxS7ZpeIyFmUDAFCfUNQAoI4xxujqPq21dFqMxvZsqZnf7NbYmSu1MS3L6WgAAKCaUNQAoI4K8ffS9Bv66l93RKuguEzXzFqtZ75IVl5xmdPRAADABaKoAUAdd1m3ZoqbFqPbB4bp32tSNfKlBC3bedzpWAAA4AJQ1ACgHgjw9tAz4yI0+56B8vVy1x3/2qCpH29WZn6J09EAAMAvQFEDgHqkf/sQLXxosB66oosWbjuiYbHxmr/lEIuyAQCoYyhqAFDPeHu4a9rwrlrw4BC1DfHTlI+36NfvbtDh7EKnowEAgCqiqAFAPdWtRaDm3nuJnr6yh9buy9Tw2Hi9tyZVFRWcrgEA4OooagBQj7m7Gf1mcAfFPTxU/doH6w/zk3XdP9doz/E8p6MBAIBzoKgBQAPQNsRP7/16gP7v2t7afTxPY2as0Mvf7FZJWYXT0QAAwBlQ1ACggTDG6Jr+bbR0WoxGRDTXP5bs0rhXVmrLwWynowEAgJ+gqAFAA9M00Fuv3NRPb94WpeyCUk18bZX+smC7CkpYlA0AgKugqAFAAzW8R3PFTRuqGwe009sr92vk9ASt3H3C6VgAAEAUNQBo0Br5eOqvE3rqk8kXy9PNTbe8vU6//exbZRewKBsAACdR1AAAuqhjEy2aMkT3XdpJ8zYf0rDYBC3ceoRF2QAAOISiBgCQJPl4uuuxUd31xQOD1CLIW/d/uEmT39+oozlFTkcDAKDBoagBAH4kolWQPr9vkJ4c3V0JuzI0PDZeH647wKJsAABqEUUNAPA/PNzddHdMJ309dagiWwfpqXnbdOOba7Uvg0XZAADUBooaAOCswkL99eFdF+mFST21/cgpjZqxQq8t36PSchZlAwBQkyhqP5S5X0rfKDE8DwCnGWN0fXQ7fTMtRpd3a6YXF+/U+FdXKelQjtPRAACotyhqP7ThLemty6UZvaWlz0hHt1HaAKBSs0Y+mnVrf826pZ+O5xbr6ldX6bmvdqiotNzpaAAA1DvGqauXo6KirMTEREd+9lkVZkspC6XkudLeZZJVLjXpIkVOsv9p2tXphADgEnIKSvW3RTv0SeJBhTXx03MTe2lgpyZOxwIAoE4xxmy0LCvqjK9R1M4i/6S0Y76UNFdKXSnJkpr3lCInSBETpZAOTicEAMet3nNCT8zdpgOZBbpxQFs9MTpcQb6eTscCAKBOoKhdqNyjUvLn9knbwXX211r3twtbxAQpqLWz+QDAQYUl5Xpp6S69tWKfQgO89ZfxkRoZ0cLpWAAAuDyKWnXKPiAlz7NP2o5ssb/W7hIpcqLU42opoJmz+QDAIdvSc/TYnK3aceSUxvRsoWfGRahZoI/TsQAAcFkUtZpycq9d2JLnSse3S8ZNChtiz7OFXyX5hTidEABqVWl5hd5I2KcZ3+yWj4ebfj+2h66NaiNjjNPRAABwORS12nBsu13YkuZImfskNw+p0+V2aes2RvJp5HRCAKg1ezPy9OScbVqfmqlBnZvouQm91K6Jn9OxAABwKRS12mRZ0pFv7cKWPE/KOSi5e0tdhtulresoyYs/rACo/yoqLH24/oCe/ypFZRUVemR4N90xKEwe7myGAQBAoqg5x7Kk9A2Vpe1zKe+o5OkvdRtll7bOwyQPb6dTAkCNOpJTqKc/T9LSHcfVq02QXpjUS+EtecoAAACKmiuoKJfSVtulbft8qTBT8g6Swq+0b4/sGCO5c6U1gPrJsiwt2HpEz3yRrJzCUt0d01EPXt5FPp7uTkcDAMAxFDVXU14q7Y+3LyLZsUAqzpF8Q+xbIyMnSu0HSW784QVA/ZOVX6K/LNyuuZsOqWNTf70wqZeiw7h4CQDQMFHUXFlZsbTnG/ukbedXUmm+FNDc3s8WMVFqEy25Mc8BoH5J2JWhp+ZtU3pWoW69uL0eG9VNgT48VQAAaFgoanVFSYG0+2u7tO2Kk8qLpaC2UsR4e6atZR+JK64B1BP5xWX6R9wu/Wv1frVo5KNnx0fqivDmTscCAKDWUNTqoqJT9glb0hxp73+lilIppKN9yhY5SWrew+mEAFAtNh/I0uNztmrXsTxd1buV/nhVD4UGcNESAKD+o6jVdQWZUsoCu7TtT5CsCqlpuD3PFjFRCu3sdEIAuCAlZRV6fflevbJst/y9PfSHK3toQt/WLMoGANRrFLX6JO+4fWtk0lzpwGr7ay1726dsEROkxu2czQcAF2D3sVw9PmerNh3I1tCuTfW3CZFqE8zuSQBA/URRq69yDknbP7dP2g5ttL/WZoB90tZjvNSopbP5AOAXKK+w9P6aVL349U5J0qMju+m2gWFyd+N0DQBQv1DUGoLM/VLyPPuk7dg2SUYKG2yXtvCrJf8mTicEgPNyKLtQv5u3Tct3Zqhvu8Z6YVIvdW0e6HQsAACqDUWtocnYJSXPtU/aTuySjLvU8VK7tHW/UvJt7HRCAKgSy7L0+ZZD+vOX25VXXKb7Lu2s+y7rJG8Pdk0CAOo+ilpDZVnSsWS7sCXNkbLTJHcvqfMw+xKSbqMl7wCnUwLAzzqZV6w/L9iu+VsOq0uzAL1wTS/1axfsdCwAAC4IRQ12aTu8yX40MmmulHtY8vCVuo60T9q6jJA8fZ1OCQDn9N+UY/r9vCQdOVWk2weG6dGR3eTv7eF0LAAAfhGKGn6sokI6uM4+Zdv+uZSfIXkFSN3H2idtnS6XPLycTgkAZ5RXXKYXF6fo/bVpahXkq79N7KmYrk2djgUAwHmjqOHsysuktJWVpe0LqShb8mkshV9lX/kfNkRy52+rAbiexNRMPT5nq/Zm5Gti39Z6+soeCvbnL5kAAHUHRQ1VU1Yi7Vtul7aUhVJJruQXKkWMt0/a2g2U3NycTgkApxWVluvVZXv0+vK9CvL11B/HReiqXi1ZlA0AqBMoajh/pUXSniV2adu5WCorlAJb2Uu1IydJrftJ/EEIgIvYceSUnpizVd+m5+iK7s30l/GRatWYuVsAgGujqOHCFOdJuxbbl5DsWSKVl0iN29uXkERMlFr0pLQBcFx5haV/rdqv/4vbKQ83Nz0+qptuvqi93FiUDQBwURQ1VJ/CbGnnIvukbe8yySqXmnSxT9kiJ0pNuzmdEEADd+BkgZ6at00r95xQdFiwnpvYS52bsYoEAOB6KGqoGfknpR1f2KUtdaUkS2oe+f1JW0gHpxMCaKAsy9Lsjel6duEOFZaU66ErOuvumE7ydGfOFgDgOihqqHm5R6Xt8+3SdnCd/bVW/eyTtogJUlBrZ/MBaJCO5xbpT19s18JtR9S9RaBevKaXerVp7HQsAAAkUdRQ27IPSsnz7NJ2ZIv9tXYD7dLW42opoJmz+QA0OHHJR/X0/CRl5BbrN4M7aNrwbvL1cnc6FgCggaOowTkn90rJc+2LSI5vl4ybvZstcpK9q80vxOmEABqIU0Wlem5Rij5af0DtQvz03MSeGtQ51OlYAIAGjKIG13B8h13YkuZImXslNw+p0+V2aes2RvJp5HRCAA3A2n0n9eTcbdp/Il/dWwQqKixY0WEhig4L4Up/AECtoqjBtViWdOTb70/acg5K7t5Sl+F2aes6UvLydzolgHqsqLRc761J1co9J7UpLUt5xWWSpNaNfX9U3Lo0C+B6fwBAjaGowXVZlpS+wS5syfOkvKOSp5/UbbRd2joPkzy8nU4JoB4rK69QytFcJaZmakNqltanZiojt1iSFOTrqaj2wYoKC1F0WLB6tgmStwezbQCA6kFRQ91QUS6lrbZP2rbPlwpOSt6NpO5X2qWtY4zk7ul0SgD1nGVZOphZqPWpmZXlLVN7M/IlSV4eburTpvHpU7d+7YMV5Mt/lwAAvwxFDXVPeam0P15Kmift+FIqzpF8Q6Qe4+zS1n6Q5MbfagOoHSfzipWYlnX61C3pUI7KKiwZI3VrHmg/KtnBPnVrGcScGwCgaihqqNvKiqW9/7UvIUlZJJXmSwHNpR7j7dLWJlpyY4ktgNpTUFKmLQezlZiapQ2pmdqUlqX8knJJ9pxbdFhwZXELUeemzLkBAM6Moob6o6RA2v21PdO2O04qK5IatZEiJ9ilrWUfyfAHIgC167s5tw2Vj0puSM06PefW2O/Hc26RrZlzAwDYKGqon4pzpZ1f2Sdte76RKkql4A52YYucJDXv4XRCAA2UZVk6kFmgDalZ2rA/UxvSMrWvcs7N28NNvds2VnSYXd76tw9WIx/m3ACgIaKoof4ryJRSFtgnbfvjJatCatrdLmwRE6XQzk4nBNDAncgrVmJq5ZxbWpaSfzDn1r1Fo9PFbUBYiFoE+TgdFwBQCyhqaFjyMqTtn9vX/aetlmRJLXpVlrYJUnB7pxMCgD3ndiBbG1KzlJiWqY1pWSqonHNrE+x7epdbdFiwOjHnBgD1EkUNDVfOIbu0Jc2VDlX++9Ym2i5tPcZLjVo6mw8AKpWVV2jHkR/PuZ3I+/GcW3RYiKLCQtSzdZC8PLhECQDqOooaIElZqfYpW9Ic6eg2Sca+5j9yotTjask/1OmEAHCaZVlKO1lwurglpmZp34nv59z6tG1cWdyC1b99sAKZcwOAOoeiBvzUid32KVvSbOnELsm42wu1IyfZC7Z9GzudEAD+hz3nZp+2JaZmKunwKZVXWHL76ZxbhxA1b8ScGwC4OooacDaWJR1Ltk/Zkufap27uXlKnK+zS1m205B3gdEoAOKP8Ynuf23cnbpsOfD/n1jbEV9Htv1/E3alpgAzrSwDApVxQUTPGtJX0nqTmkixJb1iWNeMn77la0l8kVUgqkzTVsqyV5/pcihpcjmVJhzfZJ23J86RThyQPX6nrCLu0dRkhefo6nRIAzqq0vEI7jpw6vRYgMS1TJ/JKJEnBfp7q3z5EAzrYp26RrZhzAwCnXWhRaymppWVZm4wxgZI2ShpvWdb2H7wnQFK+ZVmWMaaXpE8ty+p+rs+lqMGlVVRIB9fZJ23bP5fyMySvAKnbGLu0dbpc8vByOiUAnJNlWUr9bs5tf6YS07K0v3LOzcfzh3NuIerXrjFzbgBQy6r10UdjzHxJr1iWteQsrw+U9I5lWeHn+hyKGuqM8jIpbWVlaftCKsqWfIKk8Kvs0hY2VHL3cDolAFRJRu4P5tzSMpX8gzm38JaNTl9QEh3GnBsA1LRqK2rGmDBJCZIiLcs69ZPXJkh6TlIzSWMty1pzhu+fLGmyJLVr165/WlpalX824BLKSqR9y+3SlrJQKsmV/ELtWyMjJ0ntBkpuPEoEoO7ILy7T5gOVc25pmdqUlq3CUnvOrV2I3+nSFh0Wok5N/ZlzA4BqVC1FrfLxxnhJf7Usa+453jdU0h8syxp2rs/jRA11XmmRtGeJPdO28yuprFAKbCVFjLdLW+v+En+gAVDHlJZXaPvhU6cvKNmQmqmT+facW4i/l/q3D9aAylO3CObcAOCCXHBRM8Z4Slog6WvLsmKr8P59kgZYlnXibO+hqKFeKc6Tdi22LyHZHSeVl0iN20kRE+3S1qInpQ1AnWRZlvafyFdiapbWp2YqMTVTqScLJNlzbn3bBp9eC9CvfbACvHkUHACq6kIvEzGS/i0p07KsqWd5T2dJeysvE+kn6UtJbaxzfDhFDfVWUY79WGTSXGnfMqmiTGrSxV6sHTlJatrN6YQAcEGO5xZp4+nilqXkwzmqsCQ3I/Vo1UhR7UMqH5cMVjPm3ADgrC60qA2WtELSNtnX70vSU5LaSZJlWbOMMY9Luk1SqaRCSY9yPT8gKf+ktOMLe0fb/hWSLKl5pBQxwS5uIR2dTggAFyyvuEybD2SdXsS9+cD3c27tm/hVFrdgRXcIUcdQ5twA4DssvAZcQe5Raft8+6Tt4Fr7a6362YUtYoIU1MbZfABQTUrLK5R8+FTl7ZL2qdsP59yi2gdrQAd7LUBEq0bydGfODUDDRFEDXE32QXueLWmOdGSL/bV2A+2Zth5XS4HNnc0HANXIsiztO5GvxNRMrd9vrwVIq5xz8/V0t/e5dbBP3fq2Y84NQMNBUQNc2cm99qORSXOl49sl4yaFDbFP2sLHSX4hTicEgGp3/FSREtOytH6/vRZg++FTqrAkdzejHi0bnV4LEBUWrGaBzLkBqJ8oakBdcXyHXdiS5kiZeyU3D6nT5fZJW/cx9qJtAKiH8orLtCkt6/Qy7s0Hs1RUao/GhzXxU1Tl5STRYSHqwJwbgHqCogbUNZYlHd1qF7akeVLOAcndW+oy3D5p6zpK8vJ3OiUA1JjS8golHco5vcstMS1LmZVzbk38vX60iLsHc24A6iiKGlCXWZaUnmiXtuR5Ut5RydNP6jbaPmnrPEzy5LEgAPWbZVnam5F/+sRtQ2qmDmR+P+fWt13j08Wtb7vG8mfODUAdQFED6ouKcunAGru0bZ8vFZyUvBtJ3a+0T9o6Xiq5ezqdEgBqxbFTRadP3DakZmrHke/n3CJO73Ozl3E3DfR2Oi4A/A+KGlAflZdJ++PtmbYdX0rFOZJviNRjnH3SFjZYcnN3OiUA1JrcolJtOpB9ei3A5gPZKi6z59w6hPorqn3l45IdQhTWxI85NwCOo6gB9V1ZsbT3v/ZJW8oiqTRfCmgu9Rhvn7S1GSC5Mb8BoGEpKatQ0uGc049LJqZmKqugVJIUGuClqPb2rZIDOoSoR8tG8mDODUAto6gBDUlJgbQ7zi5tu+OksiKpURspcoJ90taqr8TfIgNogCoqLO07kXd6xm1DaqYOZhZKkvy8fjzn1qctc24Aah5FDWioinOlnV/ZpW3PN1JFqRTcQYqcZJ+0NetBaQPQoB3NKVJiWqY27LdP3XYcPSWrcs4tslWj02sBosJCFBrAnBuA6kVRAyAVZkk7FtilbX+CZJVLTbvbpS1iohTa2emEAOC4U0Wl2nwgu7K4ZWrLwe/n3DqG+v9oLUB75twAXCCKGoAfy8uQdsy3LyJJWy3Jklr0qixtE6Tg9k4nBACXUFJWoW2HfjDnlpap7NNzbt6nl3BHh4UovGUgc24AzgtFDcDZnTosJX9un7Qdqvz/ZJtou7T1GC81aulsPgBwIRUVlvZm5J2+nGR9aqbSs+w5N38vd/VrH3x6LUCfdo3l58WcG4Czo6gBqJqsVHupdtIc6eg2SUZqP8ieZ+txteQf6nRCAHA5R3IKf7DPLUsplXNuHm5GEa2DFN0++PSsWxPm3AD8AEUNwPk7sdt+NDJptnRil2TcpY4x9jxb+JWSb7DTCQHAJZ0qKtWmtO+L25aD2Sr5bs6tqb+if7AWoF0Ic25AQ0ZRA/DLWZZ0LFlKnmuftGWlSm6eUudh9uOR3UZJ3oFOpwQAl1VcVq6kQzmnH5fckJqlnEJ7zq1p4I/n3Lq3YM4NaEgoagCqh2VJhzfbhS15nnTqkOThI3UdaZ+0dR0pefo6nRIAXFpFhaU9GXnakJqpxNQsrd+fqUPZP55ziw6zT936tg2Wr5e7w4kB1BSKGoDqV1EhHVxnn7Qlz5PyMySvAKnbGPukrdPlkoeX0ykBoE44nF2oxLTKC0r2Z2rnsdzTc26RrYNO73KLDgtRiD//bQXqC4oagJpVUS6lrrRP2nZ8Ye9s8wmSwq+yS1vYUMmdm88AoKpyCku16UCWNuy3T922pH8/59apqX/liVuIBoSFqG2IL3NuQB1FUQNQe8pLpX3LK0vbAqkkV/ILtW+NjJwktRsouTF/AQDno7isXNvSv59zS0z7fs6tWaB35YybfeoW3rKR3N0obkBdQFED4IzSImnPUru07VoslRZIgS3tpdqRk6TW/SX+FhgAzltFhaXdx7+bc7MvKPluzi3A20N92zXWgMpTtz5tGzPnBrgoihoA55Xk22Utaa60O04qL5Eat7MvIYmcKLXoRWkDgAtwKLvQPm2r3On23Zybp/t3c24hiqrc6cacG+AaKGoAXEtRjpSyyD5p27dMqiiTmnS2T9kiJkrNujudEADqvJwCe85tfeWp27cHc1RSbs+5dW4WYD8q2T5EAzqEqE0wc26AEyhqAFxXQaZ9AUnSHGn/CkmW1CzCPmWLnCiFdHQ6IQDUC0Wl9j639ZWnbompmTpVVCZJat7I275Vsn2wojuEqHsL5tyA2kBRA1A35B6Vts+3H488uNb+Wqu+lSdtE6SgNs7mA4B6pKLC0q7jud8v4t6fqcP/396dBtd53fcd/x4sxELs3DcskrWRlGxJpLwpduy43hJLspQX7qRZOs24ky6Jp02nTTqTdtrpTGc6k047mYyTSTKTdFLbjUVJlnfZkq04liVSkiVxlSgCIClK4gKQAAkS6+mL8wC4uLigQIHAvcD9fmaeEfA8B5fnHj665I/n+Z9z4QqQ6tzu6mjlnmyBkvdta6G22jo36XozqElafs6fgIOPppm2Uy+mc9s+kELb9vuhcUNx+ydJK9Bkndvenj72dvdz5O1BINW53T5Z55bVurVa5yYtmEFN0vJ27vW0qfb+PXD6AIQK6Lw3hbbb7oP6tmL3UJJWpAtDozx/vI/nutOs28snp+vcblrfkG3C3cruTuvcpHfDoCZp5Th9GA7sSTNt545CRRXc8LEU2m79bNpoW5K0KK6MjvPKGxd4rrtvaj+3wazObWNTLbuy0La7s41bNjZa5ya9A4OapJUnRnjr5TTLtn8PXDgOlTVw0z9Ki5Dc/GlYtbrYvZSkFW1iInLk7cGpvdz29vTxZlbn1jhZ59aVHpV8r3Vu0iwGNUkrW4xwcl8207YHLr4F1fUprO18CN7zCaiuLXYvJaksnOwfmtrLbW9PH6++fRHIqXPramN3Rxu7OltpqbfOTeXNoCapfEyMw/FnUmA7+CgMnYOaJrj1l1Nou+EXobK62L2UpLJxfmiE53v7p2bcXj55ntHx9PfPmzfMrHPb0mKdm8qLQU1SeRofg+4fp5m2Q4+njbbrWtMCJDsfSguSVPgYjiQtpSuj47x88sLUjNvzPf0MDqc6t03NtTOC280brHPTymZQk6SxYXj9yTTTduTbMHIRVq+HHQ+k0Lb1HqioKHYvJansjE9Ejrw1yL7erM6tu4+3BrI6t9oq7u6YXqDkjq3N1rlpRTGoSVKukSF47ftppu3V78HYFWjakjbV3vlQ2mTbR28kqShijJzsvzwjuL12OtW5raqs4PatzVlwa+XuDuvctLwZ1CRpLsODcOQ7aabt6A9gYhRau9LKkTsfgvXbDW2SVGT9l7I6t94+9nb38cobF6bq3G7Z0Di1LcCuzla2ttYXubfS/BnUJGk+LvfDoW+mmbZjP4Y4DmtvSYFt54Ow9qZi91CSRKpze+nE+azOrZ8Xeqfr3Dbn1rl1tXHz+kYqrHNTiTKoSdK1ungGDj0G+x+B3n8AImy8PYW2HQ9Ca0exeyhJykzWuU0uULK3p4+3B4YBaMrq3HZ1tnFPVxu3b7HOTaXDoCZJCzFwCg48mmbaTu5N57bsykLbA9C0ubj9kyTNMFnnNjnjtq9nZp3bHVuz/dw6W7m7vY3merdtUXEY1CTpeunvgQOPpJq2t14GAnR8KD0auf0BWL222D2UJBXQl9W57evp47mePvZndW4hzKxz293ZxuaWumJ3V2XCoCZJi+Hsaymw7X8Yzh6BUAldH0kzbbf9StqzTZJUki6PjPPSyfPs7e5jb2+qc7uY1bltaaljGGJZhQAAG0FJREFUV2f2uGRnGzetb7DOTYvCoCZJiylGOH0wBbb9e6C/Gyqq4T2fSDNtt3wGahqL3UtJ0lWMT0QOvzUwFdz2dvdxenC6zm1XNtu2u7OV27c2U1NlnZsWzqAmSUslRjj1YgptBx6FgZNQVQs3fyotQnLzp6DaR2okqdTFGDnRl+rcJvd0OzpZ51ZVwfu2tkw9LnlXRyvNdda56doZ1CSpGCYm4ORz06Ht0mlY1ZBm2HY+BDd+HKpqit1LSdI8nbs4nOrcevt5rjvVuY1NTNe5Te7ldk9XG5ua/Uc5vTODmiQV28Q49PwkhbZD30h7ttU2w62fS49Hdn0UKquK3UtJ0jW4PDLOz0+cn1qg5IXefi6NjAOpzm135/S2AO9ZZ52bZjOoSVIpGR+FYz9Koe3wt2B4AOrXwvb70kxb+wehwtoHSVpuxsYnOJzt57avp5/nevo4k9W5NddVs6ujdWpbgJ1brHOTQU2SStfoFTj6gxTaXv0ujA5B46a01P/Oh2DrLgj+C6wkLUcxRo73DU3t5ba3p4/Xz1wCpuvcdnelWbe72q1zK0cGNUlaDkYupbC2fw+89gSMD0NLO+z4fAptG+8wtEnSMnfu4jD7pvZz6+dATp3brRubph+X7GxjY3NtsburRWZQk6Tl5soFOPztNNN27CmYGIM170mBbceDsP7WYvdQknQdDI2MZXVu/ezNq3Pb2lo3vUBJZxs3Wue24hjUJGk5G+pLC5DsfzgtSBInYP0O2Pn5FNrW3FjsHkqSrpPJOrfnutO2AM9193P2Yqpza6nP6tw629jV2cbtW5pZVVVR5B5rIQxqkrRSDL4NBx9Loe3Ez9K5zXemwLbj89Cyrbj9kyRdV5N1bs91pwVK9vb2cSyrc6upquC921q4J5t1u6ujlaZa69yWE4OaJK1EF07CgUdSaDv1Yjq37QNpuf/tD0DjhuL2T5K0KM5eHGbf5AIlvf3sf+MC4xORirw6t93WuZU8g5okrXR9x9IiJPv3wOkDECqg894007b9fqhvK3YPJUmLZGhkjJ8fP8/eyTq34/0MZXVu29rq2N3Rlu3n1sqN6xoILkxVMgxqklROTh+GA3vSTNu5o1BRBTd8LM203frLaaNtSdKKNTY+waE3B3mupy/bFmC6zq21vpq7O9Jebru72ti52Tq3YjKoSVI5ihHeeiUFtv174MJxqFwFN30y1bPd8hlYtbrYvZQkLbIYI73nhqaC276efo6dna5ze9+2Fu7pasv2c2uh0Tq3JWNQk6RyFyO88XwKbQcegcE3oboebv50WvL/PZ+AausYJKlcnBkc5vnevqnNuPefGpiqc7ttU9PUtgC7O9vY0OSfD4vFoCZJmjYxAcefSaHt4KMwdA5qmtJjkTsehBs/BpX+a6oklZNLw2k/t709fdl+bue5PJrq3Nrb6qdC2+7ONm5ct9o6t+vEoCZJKmx8DHqeTqHt0ONpo+26VrjtvlTT1vkLUFFZ7F5KkpbY6PgEh94cSAuUZHu6nb04AqQ6t7SqZApvO6xze9cMapKkdzY2Aq8/mULbkW/DyEVYvR52PJBm2ra9Hyr8g1iSylGMkZ5zQ+ztTjNu+3r76c7q3Gqrszq3bCPuO61zmzeDmiTp2oxehte+n0Lbq9+DsSvQtCUtQrLzQdh8F/jYiySVtTODw1OrSu7r7eNAgTq33dnM23rr3AoyqEmS3r3hQTjy3RTajv4AJkahtTMtQrLjQdiww9AmSeLi8OR+bmnW7cXj03VuHWvq2ZVtC7DLOrcpBjVJ0vVxuR8OfyuFtmM/hjgOa29JoW3ng7D2pmL3UJJUIkbHJzh4amAquO3r6efcpVTn1rZ6Fbs6sgVKutrYsbmJ6srye7zeoCZJuv4unYWDj6U92nr/AYiw8fbpmbbWjmL3UJJUQmKMdJ+9lAW3tC1Az7khINW53bktbcK9u7OVO9tbaaipKnKPF59BTZK0uAbeTEv9738YTu5N57bsykLbA9C0ubj9kySVpNODV9jX0z8143bg1AUmIlQE2L55us5tV2cr6xtXXp2bQU2StHT6e9Om2vsfhrdeBgJ0fCg9Gnnb/dCwrtg9lCSVqIvDY7x4vH9qxi23zq1zTf3UtgC7Otu4Ye3yr3MzqEmSiuPsUTiwB175Opw9AqECuj6aZtpu+5W0Z5skSXMYHZ/gwKkB9vX08Vx32hagL6tzW7N61YyNuLcvwzq3BQW1EMI24G+ADUAE/jzG+L/y2vwa8O+BAAwCvxNjfOlqr2tQk6QyEiOcPpjq2fY/DP3dUFEN7/mlFNpu+QzUNBa7l5KkEhdj5NjZS9l+bmlbgN6szq2uupI721vY1dnGPdl+bqtLvM5toUFtE7ApxvhCCKEReB54IMZ4MKfNh4BDMcb+EMJngP8cY3z/1V7XoCZJZSpGOPVimmnb/wgMnISqWrjpkym03fRJWFVf7F5KkpaJ0wNX2DtZ59bbx8FTA0xEqKwIbJ/az62VT+3YSEVFaT0qeV0ffQwhPAb8SYzxiTmutwL7Y4xbrvY6BjVJEhMTcPK5NNN24BG4dBqqV8Otn02h7caPQ1VNsXspSVpGBq+M8uLx81Obcb94op+WulU88wcfL7matusW1EIIncDTwM4Y48AcbX4fuDXG+NtXey2DmiRpholx6PlJmmk7+Fjas62mGW77XFqIpOujUFnaj7BIkkrPyNgEp85fpnPt6mJ3ZZbrEtRCCA3Aj4H/FmPcM0ebjwF/CtwbYzxX4PoXgS8CtLe3393b2zu/dyBJKi/jo3DsR2mm7fA3YXgA6tfA9vvTTFv7B6Gisti9lCRpQRYc1EII1cA3ge/FGP94jjZ3AI8An4kxvvpOr+mMmiRpXkavwOs/TIuQHPkOjA5Bw0bY8fkU2rbughJ7lEWSpPlY6GIiAfhroC/G+KU52rQDTwK/EWP86Xw6ZVCTJF2zkUvw6nfTTNtrT8D4MDS3w84stG28w9AmSVo2FhrU7gX+HngFmMhO/yHQDhBj/HII4S+Ah4DJZxnH5voFJxnUJEkLcuUCHP52qml7/UmYGIO2G1Ng2/kQrL+12D2UJOmq3PBakrSyDfXBoW+kmbaev4c4Aeu3p0VIdjwIa24sdg8lSZrFoCZJKh+Db6dVIw/sgePPpHOb3pdm2XZ8Hlq2Fbd/kiRlDGqSpPJ04WTan23/Hjj1Qjq37f0ptG1/ABo3FLd/kqSyZlCTJKnv2PTG2m/vB0KaXWvthNau9N+2runv61qK219J0opnUJMkKdfpw3D4cThzBPp7oK8bhs7ObFPbMjO45Qa5pi3u4yZJWrCrBbWqpe6MJElFt/7W2atCDg+m0DYZ3Pp7oL8b3nwJDj2eVpWcVFENLe2Fg1xLB9Q0LNlbkSStTAY1SZIAahph4+3pyDc+BgNvpOCWH+RO7IXhCzPbr14/+1HK1s50NG50rzdJ0jsyqEmS9E4qq6C1Ix2FDPVNB7fcINf7DLzyd2m7gElVddlrFaiLa2mH6tpFfzuSpNJnUJMkaaHq29Kx5a7Z18ZG4MKJLLx1z3y8svtpGL2U0zhA0+a5Fzipb3M2TpLKhEFNkqTFVLUqbbhdaNPtGOHSmdmPU/b3wNEfwMW3Zravacpm4zpnB7nmbVBZvdjvRpK0RAxqkiQVSwjQsD4d2+6ZfX1kCM73zg5yZ47Aq9+H8eGc16qE5q1zr1RZ27wkb0mSdH0Y1CRJKlWr6mH9benINzEBg28WXuDk0OMwdG5m+7q2wguctHVB42aoqFj0tyNJmj+DmiRJy1FFBTRvSUfnvbOvXxmYrofLDXNvvAAHHoU4Pt22clXaVqDgSpUdsGr1UrwjSVIOg5okSStRbRNsuiMd+cbHYODk7Lq4vm448SwMD8xs37Bh7gVOGta7wIkkLQKDmiRJ5aayanpft3wxwuX+FN5mBLle6PkJvPw1IE63r66ffq38INfSDlU1i/9+JGkFMqhJkqRpIeRsN3D37Otjw3D+eOGVKo/9CEaHcl8MmrZkwS1/77guqGt1Nk6S5mBQkyRJ81dVA2tvSke+GOHi6cILnLz2BFx8e2b7muYU4AotcNK0Nc38SVKZ8hNQkiRdHyFA44Z0tH9g9vWRS+kRyvyNv98+CEe+A+Mj020rqtLecAUXOOlMNXiStIIZ1CRJ0tJYtRo2bE9HvonxtN1AoQVODjwKl/tmtq9fU2Bxk850rnGT2w1IWvYMapIkqfgqsg27m7dC1y/Mvn7lQuG6uJN74cAjedsN1GQ1cZ2Fw1x13RK8IUlaGIOaJEkqfbXNsOm96cg3PgoXThQOcr3PwMjgzPYNGwvXxbV2wup1LnAiqSQY1CRJ0vJWWQ1tN6TjxrxrMcJQX+EFTrqfhpe+MrN99erCj1O2daWauapVS/GOJMmgJkmSVrAQYPWadGzdNfv66JVsu4G8IHfudTj6Qxi7nPNaFWk1yrlWqqxrXZr3JKksGNQkSVL5qq6FdTenI1+MaUuBQgucHPkuXDo9s31tc4GauOy/TVvcbkDSNfETQ5IkqZAQoHFjOjo+OPv68EU43zs7yL31Chz+FkyMTretqIKW9rkXOKlpXIp3JGkZMahJkiS9GzUNsGFHOvJNjMPAG4UXOHnjBbhyfmb7+rVzL3DSsNHtBqQyZFCTJEm63ioq0wxaSzt0fWT29cv9Mzf9ngxyJ56F/Q9DnJhuW1ULLXPUxbW0u92AtEIZ1CRJkpZaXWs6Nt85+9r4aLbASU/eIie90PMTGLk4s33j5sJ1cW1daWNwtxuQliWDmiRJUimprIY1N6YjX4wwdK7wAievPwWDp2a2X9WQBbcCM3It7enXklSSDGqSJEnLRQiwem06tu2efX30cpqNyw9yZ1+D156A8eGc16qA5q3TwS1/Vq6uZUnekqTCDGqSJEkrRXUdrLslHfkmJuDiW4UXODn8LRg6O7N9bcvcC5w0bUl1eJIWjUFNkiSpHFRUQNPmdHR8aPb14cECC5z0wJsvwaHHYWIs57Wq06OTBRc46UgrYkpaEIOaJEmS0l5uG29PR77xsentBmYscNIDJ/fClQsz269eP/cCJw0bXOBEmgeDmiRJkq6usiotSNLaAXx09vXL/YUXOOl9Bl75u7ztBuoK18RNLnBSXbsEb0gqfQY1SZIkLUxdK2xphS13zb42NgIXTmRBrnvm45XdT8PopZzGIT2aOTUD15mz2EkX1Lc5G6eyYVCTJEnS4qladfXtBi6dKbzAydEfpMVPctU0ZTN7nbMXOGne5nYDWlEMapIkSSqOEKBhfTq23TP7+sgQnO+dHeTOHIFXv5+33UBl2m5grpUqa5uX5C1J14tBTZIkSaVpVT2svy0d+SYmYPDNwgucHHo8bQyeq65t7gVOGjenVTGlEmJQkyRJ0vJTUQHNW9LR+eHZ168MTNfD5Qa5N16Ag4/N3G6gclXaVqBQkGvtTIFRWmIGNUmSJK08tU2w6Y505Bsfg4GThVeqPPEsDA/MbN+wYfajlJPfN6x3gRMtCoOaJEmSyktl1fRsWb4Y03YD/d15Qa4Xen4CL38NiNPtq+tnz8C15Ww3UFWz+O9HK5JBTZIkSZoUQtoGoL4Nttw9+/rYMJw/XmClym449hSMDuW+GDRtyYJbR96sXFfa1sDZOM3BoCZJkiTNV1UNrL0pHflihIunCy9w8toTcPHtme1rmlOAK7TASdPWNPOnsuXvviRJknQ9hACNG9LR/v7Z10cupUco84Pc2wfhyHdgfGS6bUVV2hturgVOapuW4h2piAxqkiRJ0lJYtRo2bE9HvonxtN1AoQVODjwKl/tmtq9fM/cCJ42b3G5gBTCoSZIkScVWkW3Y3bwVun5h9vUrFwrUxfXAyb1w4BGI49NtK2sK1MR1Th/VdUvwhrRQBjVJkiSp1NU2w6b3piPf+ChcODG9b1xumOv9KYwMzmzfsLFwXVxrJ6xe5wInJcKgJkmSJC1nldXQdkM68sUIQ30zV6fs60nfdz8NL32VmdsNrC4wC9eVvm/eBlWrluANCQxqkiRJ0soVAqxek46tBbYbGL0yvd1Abl3cudfh6A9h7HLOa1Wk1SjnWqmyrnVp3lOZMKhJkiRJ5aq6FtbdnI58MaYtBQotcHLku3Dp9Mz2tc1zL3DSvDXV4WneDGqSJEmSZgsBGjemo+ODs68PX4TzvbOD3FuvwOFvwcTodNuKKmhpnzvI1TQsyVtaTgxqkiRJkq5dTQNs2JGOfBPjMPBG3uImPSnMnXoBLvfPbL963ey94ibDXMPGstxuwKAmSZIk6fqqqEwzaC3t0PWR2dcvn5/9OGV/D5z4Gez/OsSJ6bZVtdAyR11cS/uK3W7AoCZJkiRpadW1QN37YPP7Zl8bHy28wEl/L/T8BEYuzmzfuLnw45RtXWlj8GW63YBBTZIkSVLpqKyGNTemI1+MMHSu8AInrz8Fg6dmtl/VmELbupvhV/9qCTp//RjUJEmSJC0PIcDqtenYtnv29dHLaTYuP8gND85uW+IMapIkSZJWhuo6WHdLOpa58ls+RZIkSZJKnEFNkiRJkkqMQU2SJEmSSoxBTZIkSZJKjEFNkiRJkkqMQU2SJEmSSoxBTZIkSZJKjEFNkiRJkkqMQU2SJEmSSoxBTZIkSZJKzDsGtRDCthDCUyGEgyGEAyGE3yvQ5tYQwjMhhOEQwu8vTlclSZIkqTxUzaPNGPBvY4wvhBAagedDCE/EGA/mtOkDfhd4YDE6KUmSJEnl5B1n1GKMb8YYX8i+HgQOAVvy2pyOMe4FRhell5IkSZJURq6pRi2E0AncCTz7bn6xEMIXQwj7Qgj7zpw5825eQpIkSZJWvHkHtRBCA/Aw8KUY48C7+cVijH8eY9wVY9y1bt26d/MSkiRJkrTizSuohRCqSSHtb2OMexa3S5IkSZJU3uaz6mMA/hI4FGP848XvkiRJkiSVt/ms+vhh4NeBV0IIP8/O/SHQDhBj/HIIYSOwD2gCJkIIXwK2v9tHJCVJkiSpnIUYY3F+4RDOAL1F+cWvbi1wttidKFOOffE49sXl+BePY188jn3xOPbF49gXT6mOfUeMseDiHUULaqUqhLAvxrir2P0oR4598Tj2xeX4F49jXzyOffE49sXj2BfPchz7a1qeX5IkSZK0+AxqkiRJklRiDGqz/XmxO1DGHPviceyLy/EvHse+eBz74nHsi8exL55lN/bWqEmSJElSiXFGTZIkSZJKTFkFtRDCp0MIR0IIR0MI/6HA9ZoQwtey68+GEDpzrv1Bdv5ICOFTS9nvlWAeY/9vQggHQwgvhxB+GELoyLk2HkL4eXZ8Y2l7vvzNY+x/K4RwJmeMfzvn2m+GEF7Ljt9c2p4vf/MY+/+ZM+6vhhDO51zzvl+AEMJfhRBOhxD2z3E9hBD+d/Z783II4a6ca973CzCPsf+1bMxfCSH8NITw3pxrPdn5n4cQ9i1dr1eGeYz9L4YQLuR8tvxRzrWrfl7p6uYx9v8uZ9z3Z5/xbdk17/sFCCFsCyE8lf098kAI4fcKtFmen/kxxrI4gErgdeAGYBXwEmlT7tw2/wL4cvb1F4CvZV9vz9rXAF3Z61QW+z0tl2OeY/8xoD77+ncmxz77/mKx38NyPeY59r8F/EmBn20DjmX/bc2+bi32e1oux3zGPq/9vwb+Kud77/uFjf9HgLuA/XNc/yzwHSAAHwCezc573y/+2H9ockyBz0yOffZ9D7C22O9huR7zGPtfBL5Z4Pw1fV55XPvY57X9HPBkzvfe9wsb+03AXdnXjcCrBf6usyw/88tpRu0e4GiM8ViMcQT4KnB/Xpv7gb/Ovv468EshhJCd/2qMcTjG2A0czV5P8/OOYx9jfCrGOJR9+zNg6xL3caWaz30/l08BT8QY+2KM/cATwKcXqZ8r0bWO/T8GvrIkPSsDMcangb6rNLkf+JuY/AxoCSFswvt+wd5p7GOMP83GFvy8v67mcd/PZSF/VohrHns/76+jGOObMcYXsq8HgUPAlrxmy/Izv5yC2hbgRM73J5n9mzjVJsY4BlwA1szzZzW3ax2/f0b6V49JtSGEfSGEn4UQHliMDq5g8x37h7JHAb4eQth2jT+rwuY9ftmjvl3Akzmnve8X11y/P973Syv/8z4C3w8hPB9C+GKR+rTSfTCE8FII4TshhB3ZOe/7JRJCqCcFgYdzTnvfXychlS3dCTybd2lZfuZXFbsDUq4Qwj8BdgEfzTndEWN8I4RwA/BkCOGVGOPrxenhivQ48JUY43AI4Z+TZpU/XuQ+lZsvAF+PMY7nnPO+14oWQvgYKajdm3P63uy+Xw88EUI4nM1U6Pp4gfTZcjGE8FngUeCmIvep3HwO+IcYY+7sm/f9dRBCaCAF4C/FGAeK3Z/roZxm1N4AtuV8vzU7V7BNCKEKaAbOzfNnNbd5jV8I4RPAfwTuizEOT56PMb6R/fcY8CPSv5Roft5x7GOM53LG+y+Au+f7s7qqaxm/L5D3GIz3/aKb6/fH+34JhBDuIH3e3B9jPDd5Pue+Pw08gmUG11WMcSDGeDH7+ttAdQhhLd73S+lqn/fe9+9SCKGaFNL+Nsa4p0CTZfmZX05BbS9wUwihK4SwivQ/Sv5Kat8AJld7+VVSoWfMzn8hpFUhu0j/+vTcEvV7JXjHsQ8h3An8GSmknc453xpCqMm+Xgt8GDi4ZD1f/uYz9ptyvr2P9Gw3wPeAT2a/B63AJ7Nzmp/5fOYQQriVVMD8TM457/vF9w3gN7KVwD4AXIgxvon3/aILIbQDe4BfjzG+mnN+dQihcfJr0tgXXEFP704IYWNWe08I4R7S3wPPMc/PKy1MCKGZ9MTQYznnvO8XKLun/xI4FGP84zmaLcvP/LJ59DHGOBZC+Fekwa8kra52IITwX4B9McZvkH6T/08I4SipIPQL2c8eCCH8P9JflMaAf5n3iJKuYp5j/z+ABuDvsj9DjscY7wNuA/4shDBB+gPlv8cY/QvrPM1z7H83hHAf6d7uI60CSYyxL4TwX0l/gAP8l7xHNXQV8xx7SJ8zX83+UWiS9/0ChRC+Qlrhbm0I4STwn4BqgBjjl4Fvk1YBOwoMAf80u+Z9v0DzGPs/ItV//2n2eT8WY9wFbAAeyc5VAf83xvjdJX8Dy9g8xv5Xgd8JIYwBl4EvZJ89BT+vivAWlq15jD3A54Hvxxgv5fyo9/3CfRj4deCVEMLPs3N/CLTD8v7MDzP/biBJkiRJKrZyevRRkiRJkpYFg5okSZIklRiDmiRJkiSVGIOaJEmSJJUYg5okSZIklRiDmiRJkiSVGIOaJEmSJJUYg5okSZIklZj/D0yNUIF0gnrAAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 1080x576 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OvKwgeCvL4CK",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "reverse_target_word_index = y_tokenizer.index_word\n",
        "reverse_source_word_index = x_tokenizer.index_word\n",
        "target_word_index = y_tokenizer.word_index"
      ],
      "execution_count": 64,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7Hh9rglmu6b8",
        "colab_type": "text"
      },
      "source": [
        "## Inference\n",
        "\n",
        "Set up the inference for the encoder and decoder:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JOrf-RokNCyA",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Encode the input sequence to get the feature vector\n",
        "encoder_model = Model(inputs=encoder_inputs,outputs=[encoder_outputs, state_h, state_c])\n",
        "\n",
        "# Decoder setup\n",
        "# Below tensors will hold the states of the previous time step\n",
        "decoder_state_input_h = Input(shape=(latent_dim*2,))\n",
        "decoder_state_input_c = Input(shape=(latent_dim*2,))\n",
        "decoder_hidden_state_input = Input(shape=(max_len_text,latent_dim))\n",
        "\n",
        "# Get the embeddings of the decoder sequence\n",
        "dec_emb2= dec_emb_layer(decoder_inputs) \n",
        "# To predict the next word in the sequence, set the initial states to the states from the previous time step\n",
        "decoder_outputs2, state_h2, state_c2 = decoder_lstm(dec_emb2, initial_state=[decoder_state_input_h, decoder_state_input_c])\n",
        "\n",
        "# A dense softmax layer to generate prob dist. over the target vocabulary\n",
        "decoder_outputs2 = decoder_dense(decoder_outputs2) \n",
        "\n",
        "# Final decoder model\n",
        "decoder_model = Model(\n",
        "    [decoder_inputs] + [decoder_hidden_state_input,decoder_state_input_h, decoder_state_input_c],\n",
        "    [decoder_outputs2] + [state_h2, state_c2])"
      ],
      "execution_count": 47,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FjKq35p4NGt9",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def decode_sequence(input_seq):\n",
        "    # Encode the input as state vectors.\n",
        "    out, eh, ec = encoder_model.predict(input_seq)\n",
        "\n",
        "    # Generate empty target sequence of length 1.\n",
        "    target_seq = np.zeros((1,1))\n",
        "\n",
        "    # Chose the 'start' word as the first word of the target sequence\n",
        "    target_seq[0, 0] = target_word_index['start']\n",
        "\n",
        "    stop_condition = False\n",
        "    decoded_sentence = ''\n",
        "    while not stop_condition:\n",
        "        output_tokens, h, c = decoder_model.predict([target_seq] + [out, eh, ec])\n",
        "\n",
        "        # Sample a token\n",
        "        sampled_token_index = np.argmax(output_tokens[0, -1, :])\n",
        "        sampled_token = reverse_target_word_index[sampled_token_index]\n",
        "\n",
        "        if(sampled_token!='end'):\n",
        "            decoded_sentence += ' '+sampled_token\n",
        "\n",
        "            # Exit condition: either hit max length or find stop word.\n",
        "            if (sampled_token == 'end' or len(decoded_sentence.split()) >= (max_len_summary-1)):\n",
        "                stop_condition = True\n",
        "\n",
        "        # Update the target sequence (of length 1).\n",
        "        target_seq = np.zeros((1,1))\n",
        "        target_seq[0, 0] = sampled_token_index\n",
        "\n",
        "        # Update internal states\n",
        "        eh, ec = h, c\n",
        "\n",
        "    return decoded_sentence"
      ],
      "execution_count": 60,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QkR39hCdvK7G",
        "colab_type": "text"
      },
      "source": [
        "Let us define the functions to convert an integer sequence to a word sequence for summary as well as the reviews:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "r1glqSNENKH8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def seq2summary(input_seq):\n",
        "    newString=''\n",
        "    for i in input_seq:\n",
        "      if((i!=0 and i!=target_word_index['start']) and i!=target_word_index['end']):\n",
        "        newString=newString+reverse_target_word_index[i]+' '\n",
        "    return newString\n",
        "\n",
        "def seq2text(input_seq):\n",
        "    newString=''\n",
        "    for i in input_seq:\n",
        "      if(i!=0):\n",
        "        newString=newString+reverse_source_word_index[i]+' '\n",
        "    return newString"
      ],
      "execution_count": 61,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OguoNEIKNNjd",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 877
        },
        "outputId": "c9ed6170-3a53-40f9-efc3-3497c2c00a0f"
      },
      "source": [
        "for i in range(0,10):\n",
        "  print(\"Review:\",seq2text(x_val[i]))\n",
        "  print(\"Original summary:\",seq2summary(y_val[i]))\n",
        "  print(\"Predicted summary:\",decode_sequence(x_val[i].reshape(1,max_len_text)))\n",
        "  print(\"\\n\")"
      ],
      "execution_count": 63,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Review: love hot sauces rediculous thought like good hot sauce would good tacos nachos etc nice heat stuff stupid hot bought three pack regreted moment tried standard insanity sauce cannot even tolerate dashes tacos good \n",
            "Original summary: seriously \n",
            "Predicted summary:  me me me me ok great start start start\n",
            "\n",
            "\n",
            "Review: whenever want add bit cream soup sauce never seem around use cream often enough even buy smallest container stuff kept long time really add creamy flavor glad found \n",
            "Original summary: this stuff is great \n",
            "Predicted summary:  the and chicken strong great start start start start\n",
            "\n",
            "\n",
            "Review: glad opportunity purchase last boxes amazing tea local herb stores tell sold anymore cannot sold unless ground powder form hope buy overseas batch runs \n",
            "Original summary: last of the best \n",
            "Predicted summary:  the product great start start start start start start\n",
            "\n",
            "\n",
            "Review: tried peanut butter creme version first impression fudge coating looker rather soft would messy left warm day tasting overwhelming sweetness predominant aside sweetness flavor butterfinger reeses slight crunchiness cookie portion also made texturally similar butterfinger chocolate flavor present drowned sugar fudge coating seems chocolate flavored actual chocolate excess sweetness cookies makes hard eat one good thing considering nutritional information overall cookies much sweet taste like candy bar \n",
            "Original summary: too sweet \n",
            "Predicted summary:  for tasting this for the great start start start\n",
            "\n",
            "\n",
            "Review: love product kids drink instead sports drinks play soccer much healthier twice received amazon times product poorly packaged times padding cartons got crushed shipping cartons leak amazon responded well shipping replacement cartons still padding leaks would like continue buy amazon figure package cartons get crushed \n",
            "Original summary: poorly packaged \n",
            "Predicted summary:  the of this great start start start start start\n",
            "\n",
            "\n",
            "Review: delicious coffee ever drink hot cold drink cold litle cream needs booze \n",
            "Original summary: as good as it gets \n",
            "Predicted summary:  taste great start start start start start start start\n",
            "\n",
            "\n",
            "Review: things work well trouble leave one machine overnight sticks hard get otherwise great product \n",
            "Original summary: pretty cool \n",
            "Predicted summary:  not of great start start start start start start\n",
            "\n",
            "\n",
            "Review: received product described minus one boxes received three four boxes cereal beware inept service \n",
            "Original summary: pack of four \n",
            "Predicted summary:  for with priced great start start start start start\n",
            "\n",
            "\n",
            "Review: love chili mix like another reviewer said earlier use little half chili powder packet provided also use rotel instead tomato sauce water comes great give try \n",
            "Original summary: great chili mix \n",
            "Predicted summary:  the yes great start start start start start start\n",
            "\n",
            "\n",
            "Review: hear people complain aftertaste toothpaste probably toothpaste manufacturers overdo saccharin product works great sweetening tea coffee among beverages one grain tablet like teaspoon sugar sweetness need sweeten gallon gallon something adding pellets way easier opening bunch little packets even spooning sweetener pour pellets bottle cap count need whatever reason well kool aid brand soft drink mix mix well wyler flavor aid tastes find grain pellets sweetens quart soft drink packet well really feel like still getting aftertaste try pellets two packets splenda always add either nectasweet splenda prefer sweeter adding directly already chilled beverages give pellets minute dissolve stir well \n",
            "Original summary: nectasweet grain sugar substitute pellets \n",
            "Predicted summary:  the very best white great start start start start\n",
            "\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}