Sentiment analysis (prediction).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "\n",
    "\n",
    "# Fundamentals of Data Science - Week 2 and Week 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  <span style='color: green'>Scroll down to the bottom of the notebook to see your assignment</span> \n",
    "<p></p>\n",
    "<span style='color: red'>Deadline: **27.09.2017 (Wednesday) at 23:55 CEST**</span>\n",
    "\n",
    "\n",
    "In this notebook we are going to cover the following practical aspects of data science:\n",
    "+ Gathering data (scraping the Twitter Streaming API)\n",
    "+ Storing and organizing it (store to file or a database)\n",
    "+ Preprocess the data\n",
    "+ Perform sentiment, topical and correlation analysis\n",
    "+ Visualize\n",
    "\n",
    "To complete this assignment you need to have a running Anaconda installation with Python 2.7 on your device. If this is not the case, refer back to Week 1. Python package prerequisites include:\n",
    "+  **Twitter API Client** [Tweepy](https://github.com/tweepy/tweepy) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; [Install command: **pip install tweepy**]\n",
    "+  **Python Data Analysis Library** [Pandas](https://pandas.pydata.org/pandas-docs/stable/install.html)  &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;  [Install command: **pip install pandas**]\n",
    "+  **Python Visualization Library** [MatPlotLib](https://matplotlib.org/)   &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; [Install command: **python -m pip install matplotlib**]\n",
    "+  **Python-Mongo Database Client** [PyMongo](https://api.mongodb.com/python/current/)  &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;  [Install command: **python -m pip install pymongo**]\n",
    "+  **Python Topic Modelling Library** [GENISM](https://radimrehurek.com/gensim/install.html) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;  [Install command: **pip install --upgrade gensim**]\n",
    "\n",
    "An additional requirement if **you would like to use a database** is MongoDB:\n",
    "+ MongoDb database server instance [MongoDb Installation Instructions](https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/#install-mongodb-community-edition)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** MongoDB stuff **"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If MongoDB is installed on your device and a database named Twitter is created, the tweets can be stored as database entries using the following code:\n",
    "\n",
    "**Note**: If Mongo is not installed on your device it will yield a Connection Refused exception."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from pymongo import MongoClient\n",
    "\n",
    "client = MongoClient()\n",
    "db = client.Twitter\n",
    "#db = client.tweets_sample\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from pprint import pprint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import geopandas as gpd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Load necessary data from MongoDB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19.6721830368\n"
     ]
    }
   ],
   "source": [
    "start_time = time.time()\n",
    "#we are filtering out tweets of different languages and outside of the US\n",
    "filter_query = { \n",
    "    \"$and\":[ {\"place.country_code\":\"US\"}, { \"lang\": \"en\" } ]\n",
    "    }\n",
    "#we are keeping only our fields of interest\n",
    "columns_query = {\n",
    "    'text':1,\n",
    "    'entities.hashtags':1,\n",
    "    'entities.user_mentions':1,\n",
    "    'place.full_name':1,\n",
    "    'place.bounding_box':1\n",
    "}\n",
    "\n",
    "tweets = pd.DataFrame(list(db.tweets.find(\n",
    "    filter_query, \n",
    "    columns_query\n",
    "    )#.limit()\n",
    "                               )\n",
    "                          )\n",
    "elapsed_time = time.time() - start_time\n",
    "print elapsed_time\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "tweets.drop(['_id'],axis=1,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>entities</th>\n",
       "      <th>place</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>{u'user_mentions': [{u'id': 813286, u'indices'...</td>\n",
       "      <td>{u'bounding_box': {u'type': u'Polygon', u'coor...</td>\n",
       "      <td>@BarackObama \\n@FBI\\n@LORETTALYNCH \\nALL IN CO...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>{u'user_mentions': [], u'hashtags': [{u'indice...</td>\n",
       "      <td>{u'bounding_box': {u'type': u'Polygon', u'coor...</td>\n",
       "      <td>#CNN #newday clear #Trump deliberately throwin...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>{u'user_mentions': [{u'id': 4852163069, u'indi...</td>\n",
       "      <td>{u'bounding_box': {u'type': u'Polygon', u'coor...</td>\n",
       "      <td>@mike4193496 @realDonaldTrump I TOTALLY CONCUR...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>{u'user_mentions': [{u'id': 1339835893, u'indi...</td>\n",
       "      <td>{u'bounding_box': {u'type': u'Polygon', u'coor...</td>\n",
       "      <td>@HillaryClinton you ARE the co-founder of ISIS...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>{u'user_mentions': [{u'id': 25073877, u'indice...</td>\n",
       "      <td>{u'bounding_box': {u'type': u'Polygon', u'coor...</td>\n",
       "      <td>@realDonaldTrump, you wouldn't recognize a lie...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            entities  \\\n",
       "0  {u'user_mentions': [{u'id': 813286, u'indices'...   \n",
       "1  {u'user_mentions': [], u'hashtags': [{u'indice...   \n",
       "2  {u'user_mentions': [{u'id': 4852163069, u'indi...   \n",
       "3  {u'user_mentions': [{u'id': 1339835893, u'indi...   \n",
       "4  {u'user_mentions': [{u'id': 25073877, u'indice...   \n",
       "\n",
       "                                               place  \\\n",
       "0  {u'bounding_box': {u'type': u'Polygon', u'coor...   \n",
       "1  {u'bounding_box': {u'type': u'Polygon', u'coor...   \n",
       "2  {u'bounding_box': {u'type': u'Polygon', u'coor...   \n",
       "3  {u'bounding_box': {u'type': u'Polygon', u'coor...   \n",
       "4  {u'bounding_box': {u'type': u'Polygon', u'coor...   \n",
       "\n",
       "                                                text  \n",
       "0  @BarackObama \\n@FBI\\n@LORETTALYNCH \\nALL IN CO...  \n",
       "1  #CNN #newday clear #Trump deliberately throwin...  \n",
       "2  @mike4193496 @realDonaldTrump I TOTALLY CONCUR...  \n",
       "3  @HillaryClinton you ARE the co-founder of ISIS...  \n",
       "4  @realDonaldTrump, you wouldn't recognize a lie...  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tweets.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "517724\n"
     ]
    }
   ],
   "source": [
    "print len(tweets)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "** Extract data we need into their own columns (links, mentions, hashtags) **"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Deal with hyperlinks "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import re\n",
    "\n",
    "# A function that extracts the hyperlinks from the tweet's content.\n",
    "def extract_link(text):\n",
    "    regex = r'https?://[^\\s<>\"]+|www\\.[^\\s<>\"]+'\n",
    "    match = re.search(regex, text)\n",
    "    if match:\n",
    "        return match.group()\n",
    "    return ''\n",
    "\n",
    "# A function that checks whether a word is included in the tweet's content\n",
    "def word_in_text(word, text):\n",
    "    word = word.lower()\n",
    "    text = text.lower()\n",
    "    match = re.search(word, text)\n",
    "    if match:\n",
    "        return True\n",
    "    return False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "tweets['link'] = tweets['text'].apply(lambda tweet: extract_link(tweet))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#remove links\n",
    "tweets['text'] = tweets['text'].apply(lambda tweet: re.sub(r\"http\\S+\", \"\", tweet))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Deal with hashtags & mentions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Functions to extract hashtags and mentions from entities\n",
    "def extract_hashtags(ent):\n",
    "    a=[]\n",
    "    [a.append(hasht['text'].lower()) for hasht in ent['hashtags']]\n",
    "    #[a.append(hasht['text']) for hasht in ent['hashtags']]\n",
    "    return a\n",
    "def extract_mentions(ent):\n",
    "    users=[]\n",
    "    [users.append(usr_ment['screen_name'].lower()) for usr_ment in ent['user_mentions']]\n",
    "    #[users.append(usr_ment['screen_name']) for usr_ment in ent['user_mentions']]\n",
    "    return users"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "tweets['hashtags'] = map(extract_hashtags,tweets['entities'])\n",
    "tweets['mentions'] = map(extract_mentions,tweets['entities'])\n",
    "tweets.drop(['entities'],axis=1,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "tweets['state'] = map(lambda place_dict: place_dict['full_name'][-2:] ,tweets['place'])\n",
    "tweets['geography'] = map(lambda place_dict: place_dict['bounding_box'] ,tweets['place'])\n",
    "tweets.drop(['place'],axis=1,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#make all text lowercase\n",
    "tweets['text'] = tweets.text.apply(lambda x: x.lower())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index([u'text', u'link', u'hashtags', u'mentions', u'state', u'geography'], dtype='object')"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tweets.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# efforts with SentiStrength"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#SentiStrength\n",
    "import subprocess\n",
    "jar_path = \"/home/antonis/sentistrength/SentiStrength.jar\"\n",
    "senti_data_path = \"/home/antonis/sentistrength/SentiData/\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#define a SentiStrength function which takes a string or tokenized input and returns sentiment scores\n",
    "## TODO : caution: may be slow! (using ''.join )\n",
    "sample_text = 'this is something'\n",
    "def SentiStrength(sample_text):\n",
    "    '''Returns a list of [positive, negative] values'''\n",
    "    if type(sample_text) is str:\n",
    "        return subprocess.check_output(['java', '-jar', jar_path, 'sentidata', senti_data_path ,'text',sample_text]).split()\n",
    "    else:\n",
    "        return subprocess.check_output(['java', '-jar', jar_path, 'sentidata', senti_data_path ,'text','+'.join(sample_text)]).split()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Use externally trained model\n",
    "\n",
    "### IMPORTANT:\n",
    "#### we should use the same vectorizer as in the trained instance of the classifier our training and prediction matrices have to be the same\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from nltk.stem.snowball import SnowballStemmer\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "import random\n",
    "\n",
    "def stemmed_words(doc):\n",
    "    return (stemmer.stem(w) for w in analyzer(doc))\n",
    "\n",
    "\n",
    "stemmer = SnowballStemmer('english')\n",
    "analyzer = CountVectorizer().build_analyzer()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Externally trained NaiveBayes classifier on 67k tweets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "#import the model\n",
    "from sklearn.externals import joblib\n",
    "\n",
    "import pickle\n",
    "clf = joblib.load('trained models/NaiveBayes67k_chi2descr9-26_22,52.pkl') #the model itself\n",
    "selector = joblib.load(\"trained models/selector_chi267k_chi2descr9-26_22,52.pkl\") #chi2 feature selector\n",
    "vect = joblib.load('trained models/vectdescr9-26_22,52.pkl') #Countvectorizer along with its vocabulary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from nltk import tokenize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "102.259329081\n"
     ]
    }
   ],
   "source": [
    "#rtransform our \n",
    "start_time = time.time()\n",
    "\n",
    "results = clf.predict(selector.transform(vect.transform(tweets['text'])))\n",
    "\n",
    "elapsed_time = time.time() - start_time\n",
    "print elapsed_time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "112.642943859\n"
     ]
    }
   ],
   "source": [
    "#try with probability!\n",
    "start_time = time.time()\n",
    "\n",
    "results_prob = clf.predict_proba(selector.transform(vect.transform(tweets['text'])))\n",
    "\n",
    "elapsed_time = time.time() - start_time\n",
    "print elapsed_time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "results = pd.Series(results)\n",
    "results_prob = pd.DataFrame(results_prob,columns=['negative','positive'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "tweets['NB'] = pd.Series(results)\n",
    "tweets['NB_prob+'] = pd.Series(results_prob['positive'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text</th>\n",
       "      <th>link</th>\n",
       "      <th>hashtags</th>\n",
       "      <th>mentions</th>\n",
       "      <th>state</th>\n",
       "      <th>geography</th>\n",
       "      <th>NB</th>\n",
       "      <th>NB_prob+</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>@barackobama \\n@fbi\\n@lorettalynch \\nall in co...</td>\n",
       "      <td>https://t.co/5GMNZq40V3</td>\n",
       "      <td>[nojustice, trumppence]</td>\n",
       "      <td>[barackobama, fbi, lorettalynch, realdonaldtrump]</td>\n",
       "      <td>LA</td>\n",
       "      <td>{u'type': u'Polygon', u'coordinates': [[[-91.2...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.268989</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>#cnn #newday clear #trump deliberately throwin...</td>\n",
       "      <td></td>\n",
       "      <td>[cnn, newday, trump, isis]</td>\n",
       "      <td>[]</td>\n",
       "      <td>MD</td>\n",
       "      <td>{u'type': u'Polygon', u'coordinates': [[[-76.7...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.200862</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>@mike4193496 @realdonaldtrump i totally concur...</td>\n",
       "      <td></td>\n",
       "      <td>[]</td>\n",
       "      <td>[mike4193496, realdonaldtrump]</td>\n",
       "      <td>MD</td>\n",
       "      <td>{u'type': u'Polygon', u'coordinates': [[[-76.5...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.059660</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>@hillaryclinton you are the co-founder of isis...</td>\n",
       "      <td></td>\n",
       "      <td>[]</td>\n",
       "      <td>[hillaryclinton]</td>\n",
       "      <td>TX</td>\n",
       "      <td>{u'type': u'Polygon', u'coordinates': [[[-97.0...</td>\n",
       "      <td>1</td>\n",
       "      <td>0.965413</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>@realdonaldtrump, you wouldn't recognize a lie...</td>\n",
       "      <td>https://t.co/pKSQM8yikm</td>\n",
       "      <td>[nevertrump]</td>\n",
       "      <td>[realdonaldtrump]</td>\n",
       "      <td>CA</td>\n",
       "      <td>{u'type': u'Polygon', u'coordinates': [[[-116....</td>\n",
       "      <td>1</td>\n",
       "      <td>0.936727</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                text                     link  \\\n",
       "0  @barackobama \\n@fbi\\n@lorettalynch \\nall in co...  https://t.co/5GMNZq40V3   \n",
       "1  #cnn #newday clear #trump deliberately throwin...                            \n",
       "2  @mike4193496 @realdonaldtrump i totally concur...                            \n",
       "3  @hillaryclinton you are the co-founder of isis...                            \n",
       "4  @realdonaldtrump, you wouldn't recognize a lie...  https://t.co/pKSQM8yikm   \n",
       "\n",
       "                     hashtags  \\\n",
       "0     [nojustice, trumppence]   \n",
       "1  [cnn, newday, trump, isis]   \n",
       "2                          []   \n",
       "3                          []   \n",
       "4                [nevertrump]   \n",
       "\n",
       "                                            mentions state  \\\n",
       "0  [barackobama, fbi, lorettalynch, realdonaldtrump]    LA   \n",
       "1                                                 []    MD   \n",
       "2                     [mike4193496, realdonaldtrump]    MD   \n",
       "3                                   [hillaryclinton]    TX   \n",
       "4                                  [realdonaldtrump]    CA   \n",
       "\n",
       "                                           geography  NB  NB_prob+  \n",
       "0  {u'type': u'Polygon', u'coordinates': [[[-91.2...   0  0.268989  \n",
       "1  {u'type': u'Polygon', u'coordinates': [[[-76.7...   0  0.200862  \n",
       "2  {u'type': u'Polygon', u'coordinates': [[[-76.5...   0  0.059660  \n",
       "3  {u'type': u'Polygon', u'coordinates': [[[-97.0...   1  0.965413  \n",
       "4  {u'type': u'Polygon', u'coordinates': [[[-116....   1  0.936727  "
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tweets.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "@hughhewitt @noltenc @realdonaldtrump hugh, you're right. #basketoofdeplorables he'll hit again, and again. \n",
      "0.86193296115  positive\n",
      "\n",
      "@karoli @davidshuster @hillaryclinton david might have deleted that after properly put him in line \n",
      "0.320181959525  positive\n",
      "\n",
      "@realdonaldtrump and, btw, no need to look at my tax returns \n",
      "0.331246620146  positive\n",
      "\n",
      "@realdonaldtrump do we need driver license at bank? yes. do we need driver license to take exams? yes. do we need driver license to vote? no \n",
      "0.416220810491  positive\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# manually inspect some results\n",
    "for i in range(1,5):\n",
    "    j = random.randint(0,len(tweets))\n",
    "    tw = tweets['text'][j]\n",
    "    print tw, '\\n',tweets.ix[j,'NB_prob+'],' positive'\n",
    "    print ''"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAD8CAYAAACcjGjIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFVhJREFUeJzt3X+Q3PV93/Hn28jYqmIsMOZGI6kRmVxSK2hs4xusjGfS\ni8mIA2cs/jAZaBNkRu3NUOxxG00b3GZGKdgd0g6lhjokalEkeUiAkDjSuCKqRubGbQcwIjjIQBgu\nmMJVipVEQuHM2O457/6xH9lbffZ0e3t3u3e3z8fMzn33/f18v/t566R76ftj9yIzkSSp2dt6PQFJ\n0uJjOEiSKoaDJKliOEiSKoaDJKliOEiSKoaDJKliOEiSKoaDJKmyotcT6NSll16aGzZs6Gjb73zn\nO6xatWp+J7TI2fPy12/9gj3P1jPPPPPXmfnedsYu2XDYsGEDR48e7WjbsbExhoeH53dCi5w9L3/9\n1i/Y82xFxP9ud6ynlSRJFcNBklQxHCRJFcNBklQxHCRJFcNBklQxHCRJFcNBklQxHCRJlSX7Dmlp\nsdpw+3/ryevuGemvj5HQwvLIQZJUMRwkSRXDQZJUMRwkSZW2wiEiVkfEoxHx5xHxYkT8bERcEhGH\nI+Ll8vXiMjYi4t6IGI+I5yLiyqb9bCvjX46IbU31D0XEsbLNvRER89+qJKld7R45fAH4k8z8B8D7\ngReB24EjmTkIHCnPAa4FBstjFLgfICIuAXYCHwauAnaeDZQyZrRpu5G5tSVJmosZwyEiLgJ+DngA\nIDO/n5lvAFuBvWXYXuD6srwV2JcNTwKrI2INcA1wODNPZeZp4DAwUtZdlJlPZGYC+5r2JUnqgXaO\nHH4C+CvgdyPi2Yj4rxGxChjIzBMA5etlZfxa4PWm7SdK7Xz1iRZ1SVKPtPMmuBXAlcCnM/OpiPgC\nPzqF1Eqr6wXZQb3eccQojdNPDAwMMDY2dp5pTG9ycrLjbZcqe+6eHZumuv6a4Pe4X3Sr53bCYQKY\nyMynyvNHaYTDtyNiTWaeKKeGTjaNX9+0/TrgeKkPn1MfK/V1LcZXMnMXsAtgaGgoO/09qv7e2f7Q\nq54/2cN3SPs9Xv661fOM4ZCZfxkRr0fET2fmS8DVwAvlsQ24q3zdXzY5AHwqIh6icfH5TAmQQ8C/\na7oIvQX4bGaeiog3I2Iz8BRwM3DfPPYoaYH5kSHLT7ufrfRp4MGIuBB4BbiFxvWKRyJiO/AacEMZ\nexC4DhgH3ipjKSFwJ/B0GXdHZp4qy7cCe4CVwGPlIUnqkbbCITO/AQy1WHV1i7EJ3DbNfnYDu1vU\njwJXtDMXSdLC8x3SkqSK4SBJqhgOkqSKv+xHWiaO/Z8zPbuNVsuPRw6SpIrhIEmqGA6SpIrXHCQt\nWb28zvLqXR/ryet2i0cOkqSK4SBJqnhaSZI6sNw/bNAjB0lSxXCQJFUMB0lSxXCQJFUMB0lSxXCQ\nJFUMB0lSxXCQJFUMB0lSxXCQJFUMB0lSxXCQJFUMB0lSpa1PZY2IV4E3gR8AU5k5FBGXAA8DG4BX\ngV/KzNMREcAXgOuAt4BPZuaflv1sA3697PZzmbm31D8E7AFWAgeBz2RmzkN/6mO9/EUw0lI3myOH\nn8/MD2TmUHl+O3AkMweBI+U5wLXAYHmMAvcDlDDZCXwYuArYGREXl23uL2PPbjfScUeSpDmby2ml\nrcDesrwXuL6pvi8bngRWR8Qa4BrgcGaeyszTwGFgpKy7KDOfKEcL+5r2JUnqgXbDIYH/HhHPRMRo\nqQ1k5gmA8vWyUl8LvN607USpna8+0aIuSeqRdn8T3Ecy83hEXAYcjog/P8/YaFHLDur1jhvBNAow\nMDDA2NjYeSc9ncnJyY63Xar6seeBlbBj01Svp9E1/dYv9GfP3fq33FY4ZObx8vVkRHyZxjWDb0fE\nmsw8UU4NnSzDJ4D1TZuvA46X+vA59bFSX9difKt57AJ2AQwNDeXw8HCrYTMaGxuj022Xqn7s+b4H\n93P3sf75Tbg7Nk31Vb/Qnz3vGVnVlX/LM55WiohVEfGus8vAFuCbwAFgWxm2Ddhflg8AN0fDZuBM\nOe10CNgSEReXC9FbgENl3ZsRsbnc6XRz074kST3QTuQOAF9u/NxmBfB7mfknEfE08EhEbAdeA24o\n4w/SuI11nMatrLcAZOapiLgTeLqMuyMzT5XlW/nRrayPlYckqUdmDIfMfAV4f4v63wBXt6gncNs0\n+9oN7G5RPwpc0cZ8JUld4DukJUkVw0GSVDEcJEkVw0GSVDEcJEkVw0GSVDEcJEkVw0GSVOmvDyUp\nevVLYF6962Ndf01J6oRHDpKkSl8eOah7NvTw13Tu2NSzl5aWPI8cJEkVw0GSVDEcJEkVw0GSVDEc\nJEkV71bqE716b4ekpckjB0lSxXCQJFUMB0lSxXCQJFUMB0lSxXCQJFUMB0lSpe1wiIgLIuLZiPhK\neX55RDwVES9HxMMRcWGpv6M8Hy/rNzTt47Ol/lJEXNNUHym18Yi4ff7akyR1YjZHDp8BXmx6/pvA\nPZk5CJwGtpf6duB0Zv4kcE8ZR0RsBG4EfgYYAX6rBM4FwBeBa4GNwE1lrCSpR9p6h3RErAM+Bnwe\n+NWICOCjwD8qQ/YCvwHcD2wtywCPAv+5jN8KPJSZ3wO+FRHjwFVl3HhmvlJe66Ey9oU5dbYI+bsN\nJC0V7R45/CfgXwF/V56/B3gjM6fK8wlgbVleC7wOUNafKeN/WD9nm+nqkqQemfHIISJ+ETiZmc9E\nxPDZcouhOcO66eqtAipb1IiIUWAUYGBggLGxseknfh4DK2HHpqmZBy4j9rz89Vu/0J89T05Odvyz\nbzbaOa30EeDjEXEd8E7gIhpHEqsjYkU5OlgHHC/jJ4D1wERErADeDZxqqp/VvM109f9PZu4CdgEM\nDQ3l8PBwG9Ov3ffgfu4+1l+fObhj05Q9L3P91i/0Z897RlbR6c++2ZjxtFJmfjYz12XmBhoXlL+a\nmf8YeBz4RBm2Ddhflg+U55T1X83MLPUby91MlwODwNeBp4HBcvfTheU1DsxLd5Kkjswlcn8NeCgi\nPgc8CzxQ6g8AXyoXnE/R+GFPZj4fEY/QuNA8BdyWmT8AiIhPAYeAC4Ddmfn8HOYlSZqjWYVDZo4B\nY2X5FX50t1HzmO8CN0yz/edp3PF0bv0gcHA2c5EkLRzfIS1JqhgOkqSK4SBJqhgOkqSK4SBJqhgO\nkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK\n4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqTKjOEQEe+MiK9HxJ9FxPMR8W9L/fKIeCoiXo6IhyPi\nwlJ/R3k+XtZvaNrXZ0v9pYi4pqk+UmrjEXH7/LcpSZqNdo4cvgd8NDPfD3wAGImIzcBvAvdk5iBw\nGthexm8HTmfmTwL3lHFExEbgRuBngBHgtyLigoi4APgicC2wEbipjJUk9ciM4ZANk+Xp28sjgY8C\nj5b6XuD6sry1PKesvzoiotQfyszvZea3gHHgqvIYz8xXMvP7wENlrCSpR9q65lD+h/8N4CRwGPgL\n4I3MnCpDJoC1ZXkt8DpAWX8GeE9z/ZxtpqtLknpkRTuDMvMHwAciYjXwZeB9rYaVrzHNuunqrQIq\nW9SIiFFgFGBgYICxsbHzT3waAythx6apmQcuI/a8/PVbv9CfPU9OTnb8s2822gqHszLzjYgYAzYD\nqyNiRTk6WAccL8MmgPXARESsAN4NnGqqn9W8zXT1c19/F7ALYGhoKIeHh2cz/R+678H93H1sVq0v\neTs2TdnzMtdv/UJ/9rxnZBWd/uybjXbuVnpvOWIgIlYCvwC8CDwOfKIM2wbsL8sHynPK+q9mZpb6\njeVupsuBQeDrwNPAYLn76UIaF60PzEdzkqTOtBO5a4C95a6itwGPZOZXIuIF4KGI+BzwLPBAGf8A\n8KWIGKdxxHAjQGY+HxGPAC8AU8Bt5XQVEfEp4BBwAbA7M5+ftw4lSbM2Yzhk5nPAB1vUX6Fxp9G5\n9e8CN0yzr88Dn29RPwgcbGO+kqQu8B3SkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJ\nqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgO\nkqSK4SBJqhgOkqSK4SBJqswYDhGxPiIej4gXI+L5iPhMqV8SEYcj4uXy9eJSj4i4NyLGI+K5iLiy\naV/byviXI2JbU/1DEXGsbHNvRMRCNCtJak87Rw5TwI7MfB+wGbgtIjYCtwNHMnMQOFKeA1wLDJbH\nKHA/NMIE2Al8GLgK2Hk2UMqY0abtRubemiSpUzOGQ2aeyMw/LctvAi8Ca4GtwN4ybC9wfVneCuzL\nhieB1RGxBrgGOJyZpzLzNHAYGCnrLsrMJzIzgX1N+5Ik9cCsrjlExAbgg8BTwEBmnoBGgACXlWFr\ngdebNpsotfPVJ1rUJUk9sqLdgRHxY8AfAv88M//2PJcFWq3IDuqt5jBK4/QTAwMDjI2NzTDr1gZW\nwo5NUx1tu1TZ8/LXb/1Cf/Y8OTnZ8c++2WgrHCLi7TSC4cHM/KNS/nZErMnME+XU0MlSnwDWN22+\nDjhe6sPn1MdKfV2L8ZXM3AXsAhgaGsrh4eFWw2Z034P7uftY27m4LOzYNGXPy1y/9Qv92fOekVV0\n+rNvNtq5WymAB4AXM/M/Nq06AJy942gbsL+pfnO5a2kzcKacdjoEbImIi8uF6C3AobLuzYjYXF7r\n5qZ9SZJ6oJ3I/QjwK8CxiPhGqf1r4C7gkYjYDrwG3FDWHQSuA8aBt4BbADLzVETcCTxdxt2RmafK\n8q3AHmAl8Fh5SJJ6ZMZwyMz/SevrAgBXtxifwG3T7Gs3sLtF/ShwxUxzkSR1h++QliRVDAdJUsVw\nkCRVDAdJUsVwkCRVDAdJUsVwkCRVDAdJUsVwkCRVDAdJUsVwkCRVDAdJUsVwkCRVDAdJUsVwkCRV\nDAdJUsVwkCRVDAdJUsVwkCRVDAdJUsVwkCRVDAdJUsVwkCRVDAdJUmXGcIiI3RFxMiK+2VS7JCIO\nR8TL5evFpR4RcW9EjEfEcxFxZdM228r4lyNiW1P9QxFxrGxzb0TEfDcpSZqddo4c9gAj59RuB45k\n5iBwpDwHuBYYLI9R4H5ohAmwE/gwcBWw82yglDGjTdud+1qSpC6bMRwy82vAqXPKW4G9ZXkvcH1T\nfV82PAmsjog1wDXA4cw8lZmngcPASFl3UWY+kZkJ7GvalySpRzq95jCQmScAytfLSn0t8HrTuIlS\nO199okVdktRDK+Z5f62uF2QH9dY7jxilcQqKgYEBxsbGOpgiDKyEHZumOtp2qbLn5a/f+oX+7Hly\ncrLjn32z0Wk4fDsi1mTmiXJq6GSpTwDrm8atA46X+vA59bFSX9difEuZuQvYBTA0NJTDw8PTDT2v\n+x7cz93H5jsXF7cdm6bseZnrt36hP3veM7KKTn/2zUanp5UOAGfvONoG7G+q31zuWtoMnCmnnQ4B\nWyLi4nIhegtwqKx7MyI2l7uUbm7alySpR2aM3Ij4fRr/6780IiZo3HV0F/BIRGwHXgNuKMMPAtcB\n48BbwC0AmXkqIu4Eni7j7sjMsxe5b6VxR9RK4LHykCT10IzhkJk3TbPq6hZjE7htmv3sBna3qB8F\nrphpHpKk7vEd0pKkiuEgSaoYDpKkiuEgSaoYDpKkiuEgSaoYDpKkiuEgSaoYDpKkiuEgSaoYDpKk\niuEgSaoYDpKkiuEgSaoYDpKkiuEgSaoYDpKkiuEgSaoYDpKkiuEgSaoYDpKkiuEgSaoYDpKkiuEg\nSaosmnCIiJGIeCkixiPi9l7PR5L62aIIh4i4APgicC2wEbgpIjb2dlaS1L8WRTgAVwHjmflKZn4f\neAjY2uM5SVLfWizhsBZ4ven5RKlJknogMrPXcyAibgCuycx/Up7/CnBVZn76nHGjwGh5+tPASx2+\n5KXAX3e47VJlz8tfv/UL9jxbP56Z721n4IoOX2C+TQDrm56vA46fOygzdwG75vpiEXE0M4fmup+l\nxJ6Xv37rF+x5IS2W00pPA4MRcXlEXAjcCBzo8ZwkqW8tiiOHzJyKiE8Bh4ALgN2Z+XyPpyVJfWtR\nhANAZh4EDnbp5eZ8amoJsuflr9/6BXteMIvigrQkaXFZLNccJEmLyLIOh5k+kiMi3hERD5f1T0XE\nhu7Pcv600e+vRsQLEfFcRByJiB/vxTznU7sfuxIRn4iIjIglf2dLOz1HxC+V7/XzEfF73Z7jfGvj\n7/bfj4jHI+LZ8vf7ul7Mc75ExO6IOBkR35xmfUTEveXP47mIuHLeJ5GZy/JB48L2XwA/AVwI/Bmw\n8Zwx/wz47bJ8I/Bwr+e9wP3+PPD3yvKtS7nfdnsu494FfA14Ehjq9by78H0eBJ4FLi7PL+v1vLvQ\n8y7g1rK8EXi11/OeY88/B1wJfHOa9dcBjwEBbAaemu85LOcjh3Y+kmMrsLcsPwpcHRHRxTnOpxn7\nzczHM/Ot8vRJGu8nWcra/diVO4F/D3y3m5NbIO30/E+BL2bmaYDMPNnlOc63dnpO4KKy/G5avE9q\nKcnMrwGnzjNkK7AvG54EVkfEmvmcw3IOh3Y+kuOHYzJzCjgDvKcrs5t/s/0Iku00/uexlM3Yc0R8\nEFifmV/p5sQWUDvf558Cfioi/ldEPBkRI12b3cJop+ffAH45IiZo3PX4aZa3Bf/IoUVzK+sCaHUE\ncO6tWe2MWSra7iUifhkYAv7hgs5o4Z2354h4G3AP8MluTagL2vk+r6BxammYxtHh/4iIKzLzjQWe\n20Jpp+ebgD2ZeXdE/CzwpdLz3y389HpiwX92Lecjh3Y+kuOHYyJiBY3D0fMdyi1mbX0ESUT8AvBv\ngI9n5ve6NLeFMlPP7wKuAMYi4lUa52YPLPGL0u3+vd6fmf83M79F4zPIBrs0v4XQTs/bgUcAMvMJ\n4J00PoNouWrr3/tcLOdwaOcjOQ4A28ryJ4CvZrnaswTN2G85xfI7NIJhqZ+Hhhl6zswzmXlpZm7I\nzA00rrN8PDOP9ma686Kdv9d/TOPmAyLiUhqnmV7p6iznVzs9vwZcDRAR76MRDn/V1Vl21wHg5nLX\n0mbgTGaemM8XWLanlXKaj+SIiDuAo5l5AHiAxuHnOI0jhht7N+O5abPf/wD8GPAH5br7a5n58Z5N\neo7a7HlZabPnQ8CWiHgB+AHwLzPzb3o367lps+cdwH+JiH9B4/TKJ5fwf/SIiN+ncVrw0nIdZSfw\ndoDM/G0a11WuA8aBt4Bb5n0OS/jPT5K0QJbzaSVJUocMB0lSxXCQJFUMB0lSxXCQJFUMB0lSxXCQ\nJFUMB0lS5f8Bui0OWOvCHiYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f920d649b90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#check the distribution of our probability estimates of the tweets\n",
    "tweets['NB_prob+'].hist()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### investigate the possibility of introducing a 'neutral' sentiment class\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "sa_results = map(lambda x: 'positive' if x>0.7 else 'negative' if x<0.3 else 'neutral' , tweets['NB_prob+'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sa_results = pd.Series(sa_results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbYAAAEqCAYAAAB5rXVIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm8HFWZ//HPN4RNIBAgICRAWKKAyBqTuI1RFMJmcEFB\nhcDgLwziKIpLZFRWR3ABZdRoFAZQhOGHg6AiGJFNASEssiMRIgkgBJKQhE2WZ/4455Ki6e3ekHT3\n4ft+vfp1u06dqnqquqqeqlOnbysiMDMzK8WgTgdgZmb2SnJiMzOzojixmZlZUZzYzMysKE5sZmZW\nFCc2MzMrSkcSm6T1JV0paZGkb3cihleapEMlPSxpsaR1JIWkLTodV5Wk8ZLmNBm/WNJmyzOmZiRd\nLunjnY6jStLGeTut0OlY+kPSLEnv7nQcy4qk0yUd3+k4upmkkfm8NHg5LGvAx26zfbXVOaxPvxKb\npOskjZK0maQb+zNtjcnAo8CQiDiiznKW+07a7gZrMO2KwEnALhGxekQ89spGt3zk2O/tdBzdpPYg\ni4j783Z6vgOxdN3FUjOSpkmaPIDpmq6npAMl/XHpomu6/G9Imi1poaS/S/qPmvHbS7pB0pP57/ZN\n5vXRfCHU93oyr99OebwknSjpsfz6hiTlcWtKukTSAklnVS+mJP1Y0vtarEfRFzPNtJ3Y8sl7E2Am\nsBOwNIltE+COKOfb4esDqwC3dzoQsy4yAbio00EMwKnAlhExBHgL8BFJ7weQtBJwAfAzYChwBnBB\nLn+ZiDgrXwitHhGrA58A7mXJ+XMysDewHbAtsCdwSB53CHAT6fwyEnhfjuHNwAYRcf4rudK1lsed\n3TITEW29gB2Ay/L7E4FPtKj/FuB64PH89y25/HTgWeCfwGLg3TXTTa4Z/yvgIOBXlTozgXMrw7OB\n7fP7LYHpwDzgbuBDlXorA98C7gceBn4IrAqsBjwFvJCXuRjYEBgDzAAW5von1VnP1wFPAJGn+0Mu\nD2CL/H4P0g66MMd6dM08DgD+DjwGfAWY1bddcnxnAPOBO4EvAHMq024I/AKYC9wHfKoybtW8vecD\ndwCfr05bZ12qMZ8OfB/4DbAI+DOweZNpxwFXAwuAvwDjK+MOyrEvIh3Uh9RMOxG4OW+fvwETcvnl\nwHHAn/K0vwPWbbD8dYFf5+XPA64CBrWxjY4GzgXOzMu4HRidx/007xNP5c/2C6QTTACDKzEen9e9\nb39dBzgrr8/1wMjK8prtnw23OXBlXu4TeTkfbrAd/l9lW98B7JjLZ7FknxoDXJO31UPA94CV8jgB\nJwOPkI7dW4Bt8rjd8zwXAQ8An2uyP2wL3NJgXLPlN11PYCvgaeD5PH5BO/trs+3e4jw2HLgV+EIe\n3iWvuyp17ifvs23M7zLgqMrw1cDkyvDBwLX5/VRg1/z+hLz/rQBcS5NjsY19d1KO+VHgP2qOhfNI\nSXsh8HHSzc8U0nH5GOlYWTvXXyXXfSx/ltcD67dz7ALvJR1rC3LdrSrjZvHS89/ptHkOe3EebXwQ\nB+WFP5l3qAXAcznYBcCmdaZZOweyPzAY2C8Pr1PZCY9vssyXjAc2y8saBGxASgIPVMbNz+NWIyWO\ng/Jyd8wf3hty3e8AF+b41iCdhL6ex42v3WCkg2///H51YFyDePt2mMGVsmqSGA+8Mce4LSlJ7p3H\nbU3a8d4GrERKvM9WPtgTgCtIV4cjSCeaOXncIOAG4Kt52s1IiaN6MFyV13cj4LZmOwUvT2zzSCeh\nwaQT9TlNDv7HSCe+QcB78vCwPH4PYHPSSfMdeV/qO+GOIZ1A35OnHU66Woa0w/+NdPGwah4+oUEM\nXyddqKyYX2/Py2u1jY4m7de7k04aXyefWGoPsnqfdY5pZl6/NUkH31+Bd+ftdibw37luq/2z6Tav\nfj4NtsE+pJPum/K6bwFsUudksRPpQmRwXp87gcPzuF3z9lorz2Mr0t0BpCT09vx+aN9n2CCWKeRj\nq864hstvcz0PBP5Y55xRd9u12u5N4l+cY7kXGJHLPwP8tqbur4Ej2jiXbkJKyJtWyh4HxlaGRwOL\n8vvDgG+S9v0/kY6jz1BJjC2W9+JnXrPv/jjPczvgGXJSIR0Lz5LuIAflOoeTEukI0o3Bj4Czc/1D\nSOfQ15COnZ1Ij5egybHLkpuB95CO1S+QjqGVauOmn+ewF9e1nQ2UF3AVsD2wMenqWk3q7g9cV1N2\nDXBgZSdsO7Hlstl5h9wXmAZcR7oKOwi4MNf5MHBVzXQ/Ao4iHaRP8NKruDcD9+X342s3GOnq8Rga\n3CXU2WHqJrY69b8DnJzff7VvR8nDryHdrfZ9sC+ehPPwx1mS2MYC99fM+0ssOZHeS+VKknQ33J/E\n9pPKuN2BuxpM90XgpzVllwCTGtT/JfDpyudzcoN6lwNfrgx/Ari4Qd1jSU1EW9SUt9pGRwO/r4zb\nGniqMjyL1omtetX7bSonPmAv4OZW+2c727zZPlXZ5p9uMO4l61Ez7nDg/Pz+XaTEPI58x1updz/p\nZDak2fGQ615FToJt1H1x+W2u54HUT2x1t12r7d5kOSK1VB0DrJHLvkLNBR4piR7dxnp+Bbi8pux5\n8oVcHh6V11+kO6JppIvZE0jJ5UbSBdRU0vmp2Xm00b47olJ2HbBv5Vi4smYedwI7V4Y3ICW/wcC/\nku44t62z7MtpcOzm7VBtcRtEuiAbXxs3/TyH9b2aPmOTtHZ+cPk4qWnxctJt/OuB+ZIObzDphqS7\nqqq/k67GB+oKUvL5l/z+ctLV/zvyMKQrorE55gWSFgAfBV4LDCMljRsq4y7O5Y0cTLq6uEvS9ZL2\nHEjgksZKukzS3Lwt/43UdAZpW83uqxsRT5Ludqg3vub9JsCGNet7JKlNvt60tZ9JK/+ovH+SdNda\nzybAPjVxvI10ECBpN0nXSpqXx+3OkvXfiHRlt7QxfJN01fc7SfdKmlKJrdk2qreMVfr5fOHhyvun\n6gz3xdxs/2wUS6P1rafVtgRA0usk/VrSPyQtBP6T/HlExB9ITYPfBx7OHUCG5Ek/QPrs/i7pivys\np9781yJddF7d3+UvpUbbrp3t/jKR3ET6DI/JxYuBITVVhwCLtKTH7GJJi+vM8gDSY4Wq2vkNARbn\nZT8dEZMjYtuImEJqIj4yx74C6dw3VtKEZutRR7N9bHZN3U2A8yvb7U5SMl6f1Nx5CXCOpAdzx5cV\n21jOS/JDRLyQl1svPwzoHNY0sUXEvIhYi3SV9pP8/mJgr4hYKyK+02DSB0kbpGpjUlZuR9Qp60ts\nb8/vr+DliW02cEWOre+1ekQcSmp6eIrU/NA3bs1ID3TrLjMi7omI/YD1SM8Vz5O0WpvrUPVzUhPo\nRhGxJqnJTHncQ6QrMQAkrUp6RkO98aSTV5/ZpDvO6vquERG7V6at1t94ALG3Yzbpjq0ax2oRcYKk\nlUnPt75Fan9fi9ShQJVpN1/aACJiUUQcERGbke6SPitpZ1pvo5azXtrYKprtn6/U/NvZllOBu4BR\nkTpIHMmSz4OIOCUidgLeQLqw+3wuvz4iJpKOh1+SnrfUsytwaTTuOdp0+W3o72eytNt9MEu26+3A\ntn09F7NtgdtjSY/Zvo4iL5L0VtJJ+ryaed9OahLssx11OqHl5KWIuJj0WGNGpFuYGXn59Qxk362d\nZjawW822WyUiHoiIZyPimIjYmnTjsycpebfykvyQt+VG1M8PAzqHtdsrstoLcgdSG3wzFwGvk/QR\nSYMlfZjUxPPrNpf3MOlZSNUVwDuBVSNiDqmpYwIpCdyU6/w6L3d/SSvm15skbZWvCn4MnCxpPQBJ\nwyXtWlnmOpLW7FugpI9JGpanXZCLB9LNew1gXkQ8LWkM8JHKuPOAvSS9JfesOoaXHuTnAl+SNFTS\ncOCTlXHXAQslfVHSqpJWkLSNpDfVmXYE8O8DiL0dP8vrsGuOYRWlr0+MID3XWpnUceM5SbuRHsD3\nORU4SNLOkgblz2TL/gYgaU9JW+SDZCHpc3qe1tuolXr74kA13D9foVh+AnxO0k5KtpBUe4EJaX9c\nCCzO2/rFE3yOZ2y+8n6C3FFD0kpKXdfXjIhnWbKN69mD5r0hGy6/zfV8GBihBj0R62h7u+d98JB8\nzCgfr4cBl+Yql5PW+1OSVpbUdzz+oUUMk4BfRMSimvIzSRdhwyVtCBxBalatxrQKqSnyM7noPmB8\nXv+3kprr6nkl9t0fAl/r248kDZM0Mb9/p6Q3Kn0NYSGpibKd8+O5wB75mF+RtM7PUP8Of0DnsH4l\nNknrAM9HxPxmlSN9j2vPHPBjpIeDe0bEo20u71Rg63z7+8s8z7+SbtuvysMLSR/on/quDPNOswvp\nOdyDpFvhE0knVkjPgmYC1+YmkN+TmlWJiLuAs4F783I3JCXO23OzwndJbdFPt7kOVZ8AjpW0iPRM\n7cUr3Yi4nfRhnUO6OllE6pH2TK5yLDCHtDP/npQIn8nTPk+6O9k+j3+UdHLrS87HkG7d7yP1Svrp\nAGJvKSJmk3o2HklKYLNJV/mD8mfyKdI6zycl9Qsr015Hek56MulB+hW8/G6/HaNI22cx6XnuDyLi\n8ja2UStfB76c94nPDSCuF7Wxf7ZyNHBGjuVDdeb//4GvkVoIFpHuqtauM5/PkT6HRaSLvf+pjBuS\ny+azpKfut/K4/YFZ+dj5N+BjtTPOFxbvIbXsNNJs+S3Xk5REbgf+IanlOWUA2/19pCbdRaSLtv/K\nLyLin6TOFQeQLnb/ldQR7J+Nlp8T04d4eTMkpGd9vyL1vLyN1KvzRzV1jgTOysdZ3zTrko61OUCj\nbv+vxL77XdLx+rt8/rqW9NwaUlPueaSkdifp2P1ZqxlGxN2kfee/SMfjXqRWwHrbcEDnMOUHctYl\nJK1OOmBGRcR9dcYfSkqw71juwZm1kO9wvhcRYzodi716+X9FdgFJe0l6jdLzu2+Rrt5m5XEbSHpr\nbiJ5PekueJl+MdNsKR3V6QDs1a13v1lelomkW2yRHgbvG0tupVciNT1sSrqTOwf4QSeCNGslNy2b\ndZSbIs3MrChuijQzs6I4sZmZWVGKe8a27rrrxsiRIzsdhplZT7nhhhsejYhm/4mpZxSX2EaOHMmM\nGTM6HYaZWU+R1N9/ude13BRpZmZFcWIzM7OiOLGZmVlRnNjMzKwoTmxmZlYUJzYzMyuKE5uZmRXF\nic3MzIpS3Be0l7eRU37T6RCWqVkn7NHpEMzM+sV3bGZmVhQnNjMzK4oTm5mZFcWJzczMiuLEZmZm\nRXFiMzOzojixmZlZUZzYzMysKE5sZmZWFCc2MzMrihObmZkVxYnNzMyK0jKxSdpI0mWS7pR0u6RP\n5/K1JU2XdE/+OzSXS9IpkmZKukXSjpV5Tcr175E0qVK+k6Rb8zSnSFKzZZiZmTXSzh3bc8AREbEV\nMA44TNLWwBTg0ogYBVyahwF2A0bl12RgKqQkBRwFjAXGAEdVEtXUXLdvugm5vNEyzMzM6mqZ2CLi\noYi4Mb9fBNwJDAcmAmfkamcAe+f3E4EzI7kWWEvSBsCuwPSImBcR84HpwIQ8bkhEXBMRAZxZM696\nyzAzM6urX8/YJI0EdgD+DKwfEQ9BSn7AernacGB2ZbI5uaxZ+Zw65TRZRm1ckyXNkDRj7ty5/Vkl\nMzMrTNuJTdLqwC+AwyNiYbOqdcpiAOVti4hpETE6IkYPGzasP5OamVlh2kpsklYkJbWzIuJ/c/HD\nuRmR/PeRXD4H2Kgy+QjgwRblI+qUN1uGmZlZXe30ihRwKnBnRJxUGXUh0NezcRJwQaX8gNw7chzw\neG5GvATYRdLQ3GlkF+CSPG6RpHF5WQfUzKveMszMzOoa3EadtwL7A7dKujmXHQmcAJwr6WDgfmCf\nPO4iYHdgJvAkcBBARMyTdBxwfa53bETMy+8PBU4HVgV+m180WYaZmVldLRNbRPyR+s/BAHauUz+A\nwxrM6zTgtDrlM4Bt6pQ/Vm8ZZmZmjfg/j5iZWVGc2MzMrChObGZmVhQnNjMzK4oTm5mZFcWJzczM\niuLEZmZmRXFiMzOzojixmZlZUZzYzMysKE5sZmZWFCc2MzMrihObmZkVxYnNzMyK4sRmZmZFcWIz\nM7OiOLGZmVlRnNjMzKwoTmxmZlYUJzYzMyuKE5uZmRVlcKcDMOuUkVN+0+kQlqlZJ+zR6RDMOsJ3\nbGZmVhQnNjMzK4oTm5mZFcWJzczMiuLEZmZmRXFiMzOzojixmZlZUZzYzMysKE5sZmZWFCc2MzMr\nihObmZkVxYnNzMyK4sRmZmZFcWIzM7OiOLGZmVlRnNjMzKwoLRObpNMkPSLptkrZ0ZIekHRzfu1e\nGfclSTMl3S1p10r5hFw2U9KUSvmmkv4s6R5J/yNppVy+ch6emcePfKVW2szMytXOHdvpwIQ65SdH\nxPb5dRGApK2BfYE35Gl+IGkFSSsA3wd2A7YG9st1AU7M8xoFzAcOzuUHA/MjYgvg5FzPzMysqZaJ\nLSKuBOa1Ob+JwDkR8UxE3AfMBMbk18yIuDci/gmcA0yUJOBdwHl5+jOAvSvzOiO/Pw/YOdc3MzNr\naGmesX1S0i25qXJoLhsOzK7UmZPLGpWvAyyIiOdqyl8yrzz+8VzfzMysoYEmtqnA5sD2wEPAt3N5\nvTuqGEB5s3m9jKTJkmZImjF37txmcZuZWeEGlNgi4uGIeD4iXgB+TGpqhHTHtVGl6gjgwSbljwJr\nSRpcU/6SeeXxa9KgSTQipkXE6IgYPWzYsIGskpmZFWJAiU3SBpXB9wF9PSYvBPbNPRo3BUYB1wHX\nA6NyD8iVSB1MLoyIAC4DPpinnwRcUJnXpPz+g8Afcn0zM7OGBreqIOlsYDywrqQ5wFHAeEnbk5oG\nZwGHAETE7ZLOBe4AngMOi4jn83w+CVwCrACcFhG350V8EThH0vHATcCpufxU4KeSZpLu1PZd6rU1\nM7PitUxsEbFfneJT65T11f8a8LU65RcBF9Upv5clTZnV8qeBfVrFZ2ZmVuX/PGJmZkVxYjMzs6I4\nsZmZWVGc2MzMrChObGZmVhQnNjMzK4oTm5mZFcWJzczMiuLEZmZmRXFiMzOzojixmZlZUZzYzMys\nKE5sZmZWFCc2MzMrSsufrTEz60Yjp/ym0yEsU7NO2KPTIfQs37GZmVlRnNjMzKwoTmxmZlYUJzYz\nMyuKE5uZmRXFic3MzIrixGZmZkVxYjMzs6I4sZmZWVGc2MzMrChObGZmVhQnNjMzK4oTm5mZFcWJ\nzczMiuLEZmZmRXFiMzOzojixmZlZUZzYzMysKE5sZmZWFCc2MzMrihObmZkVxYnNzMyK4sRmZmZF\ncWIzM7OitExskk6T9Iik2ypla0uaLume/HdoLpekUyTNlHSLpB0r00zK9e+RNKlSvpOkW/M0p0hS\ns2WYmZk1084d2+nAhJqyKcClETEKuDQPA+wGjMqvycBUSEkKOAoYC4wBjqokqqm5bt90E1osw8zM\nrKGWiS0irgTm1RRPBM7I788A9q6UnxnJtcBakjYAdgWmR8S8iJgPTAcm5HFDIuKaiAjgzJp51VuG\nmZlZQwN9xrZ+RDwEkP+ul8uHA7Mr9ebksmblc+qUN1vGy0iaLGmGpBlz584d4CqZmVkJXunOI6pT\nFgMo75eImBYRoyNi9LBhw/o7uZmZFWSgie3h3IxI/vtILp8DbFSpNwJ4sEX5iDrlzZZhZmbW0EAT\n24VAX8/GScAFlfIDcu/IccDjuRnxEmAXSUNzp5FdgEvyuEWSxuXekAfUzKveMszMzBoa3KqCpLOB\n8cC6kuaQejeeAJwr6WDgfmCfXP0iYHdgJvAkcBBARMyTdBxwfa53bET0dUg5lNTzclXgt/lFk2WY\nmZk11DKxRcR+DUbtXKduAIc1mM9pwGl1ymcA29Qpf6zeMszMzJrxfx4xM7OiOLGZmVlRnNjMzKwo\nTmxmZlYUJzYzMyuKE5uZmRXFic3MzIrixGZmZkVxYjMzs6I4sZmZWVGc2MzMrChObGZmVhQnNjMz\nK4oTm5mZFcWJzczMiuLEZmZmRXFiMzOzojixmZlZUZzYzMysKE5sZmZWFCc2MzMrihObmZkVxYnN\nzMyK4sRmZmZFcWIzM7OiOLGZmVlRnNjMzKwoTmxmZlYUJzYzMyuKE5uZmRXFic3MzIrixGZmZkVx\nYjMzs6I4sZmZWVGc2MzMrChObGZmVhQnNjMzK4oTm5mZFcWJzczMirJUiU3SLEm3SrpZ0oxctrak\n6ZLuyX+H5nJJOkXSTEm3SNqxMp9Juf49kiZVynfK85+Zp9XSxGtmZuV7Je7Y3hkR20fE6Dw8Bbg0\nIkYBl+ZhgN2AUfk1GZgKKRECRwFjgTHAUX3JMNeZXJluwisQr5mZFWxZNEVOBM7I788A9q6UnxnJ\ntcBakjYAdgWmR8S8iJgPTAcm5HFDIuKaiAjgzMq8zMzM6lraxBbA7yTdIGlyLls/Ih4CyH/Xy+XD\ngdmVaefksmblc+qUv4ykyZJmSJoxd+7cpVwlMzPrZYOXcvq3RsSDktYDpku6q0ndes/HYgDlLy+M\nmAZMAxg9enTdOmZm9uqwVHdsEfFg/vsIcD7pGdnDuRmR/PeRXH0OsFFl8hHAgy3KR9QpNzMza2jA\niU3SapLW6HsP7ALcBlwI9PVsnARckN9fCByQe0eOAx7PTZWXALtIGpo7jewCXJLHLZI0LveGPKAy\nLzMzs7qWpilyfeD83AN/MPDziLhY0vXAuZIOBu4H9sn1LwJ2B2YCTwIHAUTEPEnHAdfnesdGxLz8\n/lDgdGBV4Lf5ZWZm1tCAE1tE3AtsV6f8MWDnOuUBHNZgXqcBp9UpnwFsM9AYzczs1cf/ecTMzIri\nxGZmZkVxYjMzs6I4sZmZWVGc2MzMrChObGZmVhQnNjMzK4oTm5mZFcWJzczMiuLEZmZmRXFiMzOz\nojixmZlZUZzYzMysKE5sZmZWFCc2MzMrihObmZkVxYnNzMyK4sRmZmZFcWIzM7OiOLGZmVlRnNjM\nzKwoTmxmZlYUJzYzMyuKE5uZmRXFic3MzIrixGZmZkVxYjMzs6I4sZmZWVGc2MzMrChObGZmVhQn\nNjMzK4oTm5mZFcWJzczMiuLEZmZmRXFiMzOzojixmZlZUZzYzMysKE5sZmZWFCc2MzMrStcnNkkT\nJN0taaakKZ2Ox8zMultXJzZJKwDfB3YDtgb2k7R1Z6MyM7Nu1tWJDRgDzIyIeyPin8A5wMQOx2Rm\nZl1scKcDaGE4MLsyPAcYW1tJ0mRgch5cLOnu5RBbp6wLPLq8FqYTl9eSXhX82fW20j+/TZb7EpeR\nbk9sqlMWLyuImAZMW/bhdJ6kGRExutNxWP/5s+tt/vx6R7c3Rc4BNqoMjwAe7FAsZmbWA7o9sV0P\njJK0qaSVgH2BCzsck5mZdbGuboqMiOckfRK4BFgBOC0ibu9wWJ32qmhyLZQ/u97mz69HKOJlj6zM\nzMx6Vrc3RZqZmfWLE5uZmRXFic3MzIrixGZm1oSkVSW9vtNxWPuc2MyWISUfk/TVPLyxpDGdjsva\nI2kv4Gbg4jy8vSR/5ajLuVdkF5O0iDr/aYX0H1kiIoYs55CsnyRNBV4A3hURW0kaCvwuIt7U4dCs\nDZJuAN4FXB4RO+SyWyJi285GZs109ffYXu0iYo1Ox2BLbWxE7CjpJoCImJ//2YD1huci4nGp3n/3\ns27lxNZDJK0HrNI3HBH3dzAca8+z+eeXAkDSMNIdnPWG2yR9BFhB0ijgU8DVHY7JWvAzth4g6b2S\n7gHuA64AZgG/7WhQ1q5TgPOB9SR9Dfgj8J+dDcn64d+BNwDPAD8HHgcO72hE1pKfsfUASX8htfP/\nPiJ2kPROYL+ImNxiUusCkrYEdiY9G700Iu7scEjWJkk7RMRNnY7D+sd3bL3h2Yh4DBgkaVBEXAZs\n3+mgrDVJ3wXWjojvR8T3nNR6zkmS7pJ0nKQ3dDoYa48TW29YIGl14ErgrHyyfK7DMVl7bgS+LGmm\npG9K8u959ZCIeCcwHpgLTJN0q6QvdzYqa8VNkT1A0mrAU6QLkY8CawJn5bs46wGS1gY+QPrppY0j\nYlSHQ7J+kvRG4AvAhyPCPVu7mHtFdrnco+6CiHg3qTfdGR0OyQZmC2BLYCRwR2dDsXZJ2gr4MPBB\n4DHgHOCIjgZlLTmxdbmIeF7Sk5LWjIjHOx2P9Y+kE4H3A38DzgWOi4gFnY3K+uG/gbOBXSLiwU4H\nY+1xYusNTwO3SpoOPNFXGBGf6lxI1qb7gDdHxKOdDsT6LyLGdToG6z8/Y+sBkibVKY6IOHO5B2Nt\nkbRlRNwlacd64yPixuUdk7VP0rkR8SFJt/LSf2vX9+/s/C+1upjv2HrDWhHx3WqBpE93Khhry2eB\nycC364wL0vcSrXv1HV97djQKGxDfsfUASTdGxI41ZTf1/VNW616SVomIp1uVWXeSdGJEfLFVmXUX\nf4+ti0naT9KvgE0lXVh5XUbqoWXdr97/FfT/Guwd76lTtttyj8L6xU2R3e1q4CFgXV7apLUIuKUj\nEVlbJL0WGA6sKmkH0rMZgCHAazoWmLVF0qHAJ4DNJFWPtTWAP3UmKmuXmyLNloHc4edAYDQwozJq\nEXB6RPxvJ+Ky9khaExgKfB2YUhm1KCLmdSYqa5cTWw+o+cHRlYAVgSf8Q6PdT9IHIuIXnY7Dlo5/\nMqq3uCmyB9T+4KikvYExHQrH2iDpYxHxM2CkpM/Wjo+IkzoQlvWTpL2Ak4ANgUeATYA7ST9lY13K\nnUd6UET8EncX73ar5b+rk57L1L6sNxwPjAP+GhGbkn5+yM/YupybInuApPdXBgeRntu8IyLe3KGQ\nzF4VJM2IiNH5NxF3iIgXJF0XEW4x6WJuiuwNe1XeP0f6Be2JnQnF+kPSN0hX/U8BFwPbAYfnZkrr\nfrU/GfUI/smoruc7NrNlSNLNEbG9pPcBewOfAS6LiO06HJq1If9k1NOkr2v4J6N6hO/YeoCk1wFT\ngfUjYhtJ2wLvjYjjOxyatbZi/rs7cHZEzJPUrL51kYh4ojLon4zqEe480ht+DHwJeBYgIm4h/WCl\ndb9fSboCGcM+AAACSklEQVSL9Fz0UknDSHcA1gMkLZK0sOY1W9L5kjbrdHxWn+/YesNrIuK6mit9\nt/P3gIiYkn+TbWH+bb0n8PPRXnIS8CDwc1Jz5L7Aa4G7gdOA8R2LzBpyYusNj0ranPwlbUkfJP2r\nLetyklYE9gf+JV+YXAH8sKNBWX9MiIixleFpkq6NiGMlHdmxqKwpJ7becBgwDdhS0gOkH6/8aGdD\nsjZNJT1n+0Ee3j+XfbxjEVl/vCDpQ8B5efiDlXHuedel3CuyB0hamXRAjQTWBhaSfuzw2E7GZa1J\n+kttD8h6Zdad8nO07wJvJiWya0k9Wx8AdoqIP3YwPGvAd2y94QJgAXAjqb3fesfzkjaPiL/BiyfK\n5zsck7UpIu7lpd8jrXJS61JObL1hRERM6HQQNiCfBy6TdG8eHgkc1LlwrD/8VZve5O7+veFqSW/s\ndBA2IH8CfgS8kF8/Aq7paETWH/6qTQ/yHVtveBtwoKT7gGdI3Y4jIrbtbFjWhjNJz0SPy8P7AT8F\n9ulYRNYf/qpND3Ji6w3+Kfre9fqajiKX5X+oa73BX7XpQU5sPSAi/t7pGGzAbpI0LiKuBZA0Fv/s\nSS/xV216kLv7my1Dku4EXg/0/eLyxqQfqnwBNyd3PX/Vpjf5js1s2XJv1t7mr9r0IN+xmZk1IOm2\niNim03FY/7i7v5lZY/6qTQ/yHZuZWQOS7gC2IHUa8VdteoQTm5lZA5I2qVfunsrdzYnNzMyK4mds\nZmZWFCc2MzMrihObmZkVxYnNzMyK4sRmZmZF+T+/VSabber66gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f920d16e350>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sa_results.value_counts().plot(kind='bar', title='# of tweets flagged in each sentiment class / at the 30-70% threshold')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text</th>\n",
       "      <th>link</th>\n",
       "      <th>hashtags</th>\n",
       "      <th>mentions</th>\n",
       "      <th>state</th>\n",
       "      <th>geography</th>\n",
       "      <th>NB</th>\n",
       "      <th>NB_prob+</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>@barackobama \\n@fbi\\n@lorettalynch \\nall in co...</td>\n",
       "      <td>https://t.co/5GMNZq40V3</td>\n",
       "      <td>[nojustice, trumppence]</td>\n",
       "      <td>[barackobama, fbi, lorettalynch, realdonaldtrump]</td>\n",
       "      <td>LA</td>\n",
       "      <td>{u'type': u'Polygon', u'coordinates': [[[-91.2...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.268989</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>#cnn #newday clear #trump deliberately throwin...</td>\n",
       "      <td></td>\n",
       "      <td>[cnn, newday, trump, isis]</td>\n",
       "      <td>[]</td>\n",
       "      <td>MD</td>\n",
       "      <td>{u'type': u'Polygon', u'coordinates': [[[-76.7...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.200862</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>@mike4193496 @realdonaldtrump i totally concur...</td>\n",
       "      <td></td>\n",
       "      <td>[]</td>\n",
       "      <td>[mike4193496, realdonaldtrump]</td>\n",
       "      <td>MD</td>\n",
       "      <td>{u'type': u'Polygon', u'coordinates': [[[-76.5...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.059660</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>@hillaryclinton you are the co-founder of isis...</td>\n",
       "      <td></td>\n",
       "      <td>[]</td>\n",
       "      <td>[hillaryclinton]</td>\n",
       "      <td>TX</td>\n",
       "      <td>{u'type': u'Polygon', u'coordinates': [[[-97.0...</td>\n",
       "      <td>1</td>\n",
       "      <td>0.965413</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>@realdonaldtrump, you wouldn't recognize a lie...</td>\n",
       "      <td>https://t.co/pKSQM8yikm</td>\n",
       "      <td>[nevertrump]</td>\n",
       "      <td>[realdonaldtrump]</td>\n",
       "      <td>CA</td>\n",
       "      <td>{u'type': u'Polygon', u'coordinates': [[[-116....</td>\n",
       "      <td>1</td>\n",
       "      <td>0.936727</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                text                     link  \\\n",
       "0  @barackobama \\n@fbi\\n@lorettalynch \\nall in co...  https://t.co/5GMNZq40V3   \n",
       "1  #cnn #newday clear #trump deliberately throwin...                            \n",
       "2  @mike4193496 @realdonaldtrump i totally concur...                            \n",
       "3  @hillaryclinton you are the co-founder of isis...                            \n",
       "4  @realdonaldtrump, you wouldn't recognize a lie...  https://t.co/pKSQM8yikm   \n",
       "\n",
       "                     hashtags  \\\n",
       "0     [nojustice, trumppence]   \n",
       "1  [cnn, newday, trump, isis]   \n",
       "2                          []   \n",
       "3                          []   \n",
       "4                [nevertrump]   \n",
       "\n",
       "                                            mentions state  \\\n",
       "0  [barackobama, fbi, lorettalynch, realdonaldtrump]    LA   \n",
       "1                                                 []    MD   \n",
       "2                     [mike4193496, realdonaldtrump]    MD   \n",
       "3                                   [hillaryclinton]    TX   \n",
       "4                                  [realdonaldtrump]    CA   \n",
       "\n",
       "                                           geography  NB  NB_prob+  \n",
       "0  {u'type': u'Polygon', u'coordinates': [[[-91.2...   0  0.268989  \n",
       "1  {u'type': u'Polygon', u'coordinates': [[[-76.7...   0  0.200862  \n",
       "2  {u'type': u'Polygon', u'coordinates': [[[-76.5...   0  0.059660  \n",
       "3  {u'type': u'Polygon', u'coordinates': [[[-97.0...   1  0.965413  \n",
       "4  {u'type': u'Polygon', u'coordinates': [[[-116....   1  0.936727  "
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tweets.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# attach tweet to politician"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def trump_in_text(tweet):\n",
    "        '''This function takes the text of a tweet and \n",
    "            returns true if there is a mention to  Donald Trump or false \n",
    "            on the other hand.'''\n",
    "        if ('donald' in tweet.lower()) or ('trump'  in tweet.lower()):\n",
    "            return True\n",
    "        return False\n",
    "    \n",
    "def clinton_in_text(tweet):\n",
    "        '''This function takes the text of a tweet and \n",
    "            returns true if there is a mention to  Hillary Clinton  or false \n",
    "            on the other hand.'''\n",
    "\n",
    "        if ('hillary' in tweet.lower()) or ('clinton'  in tweet.lower()):\n",
    "            return True\n",
    "        return False\n",
    "\n",
    "def categorize(tr,hil):\n",
    "        '''This function  categorizes each tweet based\n",
    "           on text '''\n",
    "        if tr==hil:\n",
    "            return 'irrelevant'\n",
    "        elif tr:\n",
    "            return 'Trump'\n",
    "        else:\n",
    "            return'Clinton'\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "tweets['Trump'] = tweets['text'].apply(lambda tweet: trump_in_text(tweet))\n",
    "tweets['Clinton'] = tweets['text'].apply(lambda tweet: clinton_in_text(tweet))\n",
    "tweets['Politician']=map(lambda tr_col, hil_col: categorize(tr_col, hil_col), tweets['Trump'],tweets['Clinton'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "tweets.drop(['Trump','Clinton','geography'],axis=1,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text</th>\n",
       "      <th>link</th>\n",
       "      <th>hashtags</th>\n",
       "      <th>mentions</th>\n",
       "      <th>state</th>\n",
       "      <th>NB</th>\n",
       "      <th>NB_prob+</th>\n",
       "      <th>Politician</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>@barackobama \\n@fbi\\n@lorettalynch \\nall in co...</td>\n",
       "      <td>https://t.co/5GMNZq40V3</td>\n",
       "      <td>[nojustice, trumppence]</td>\n",
       "      <td>[barackobama, fbi, lorettalynch, realdonaldtrump]</td>\n",
       "      <td>LA</td>\n",
       "      <td>0</td>\n",
       "      <td>0.268989</td>\n",
       "      <td>Trump</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>#cnn #newday clear #trump deliberately throwin...</td>\n",
       "      <td></td>\n",
       "      <td>[cnn, newday, trump, isis]</td>\n",
       "      <td>[]</td>\n",
       "      <td>MD</td>\n",
       "      <td>0</td>\n",
       "      <td>0.200862</td>\n",
       "      <td>Trump</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>@mike4193496 @realdonaldtrump i totally concur...</td>\n",
       "      <td></td>\n",
       "      <td>[]</td>\n",
       "      <td>[mike4193496, realdonaldtrump]</td>\n",
       "      <td>MD</td>\n",
       "      <td>0</td>\n",
       "      <td>0.059660</td>\n",
       "      <td>Trump</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>@hillaryclinton you are the co-founder of isis...</td>\n",
       "      <td></td>\n",
       "      <td>[]</td>\n",
       "      <td>[hillaryclinton]</td>\n",
       "      <td>TX</td>\n",
       "      <td>1</td>\n",
       "      <td>0.965413</td>\n",
       "      <td>Clinton</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>@realdonaldtrump, you wouldn't recognize a lie...</td>\n",
       "      <td>https://t.co/pKSQM8yikm</td>\n",
       "      <td>[nevertrump]</td>\n",
       "      <td>[realdonaldtrump]</td>\n",
       "      <td>CA</td>\n",
       "      <td>1</td>\n",
       "      <td>0.936727</td>\n",
       "      <td>Trump</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                text                     link  \\\n",
       "0  @barackobama \\n@fbi\\n@lorettalynch \\nall in co...  https://t.co/5GMNZq40V3   \n",
       "1  #cnn #newday clear #trump deliberately throwin...                            \n",
       "2  @mike4193496 @realdonaldtrump i totally concur...                            \n",
       "3  @hillaryclinton you are the co-founder of isis...                            \n",
       "4  @realdonaldtrump, you wouldn't recognize a lie...  https://t.co/pKSQM8yikm   \n",
       "\n",
       "                     hashtags  \\\n",
       "0     [nojustice, trumppence]   \n",
       "1  [cnn, newday, trump, isis]   \n",
       "2                          []   \n",
       "3                          []   \n",
       "4                [nevertrump]   \n",
       "\n",
       "                                            mentions state  NB  NB_prob+  \\\n",
       "0  [barackobama, fbi, lorettalynch, realdonaldtrump]    LA   0  0.268989   \n",
       "1                                                 []    MD   0  0.200862   \n",
       "2                     [mike4193496, realdonaldtrump]    MD   0  0.059660   \n",
       "3                                   [hillaryclinton]    TX   1  0.965413   \n",
       "4                                  [realdonaldtrump]    CA   1  0.936727   \n",
       "\n",
       "  Politician  \n",
       "0      Trump  \n",
       "1      Trump  \n",
       "2      Trump  \n",
       "3    Clinton  \n",
       "4      Trump  "
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tweets.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['Trump', 'Clinton', 'irrelevant'], dtype=object)"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tweets.Politician.unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#### give a label to each tweet (ex. pro-Trump / anti-Hillary etc)\n",
    "def label_tweet(pol,sent, upper_threshold=0.5):\n",
    "    '''Label tweet depending on politician and sentiment.\n",
    "    Return neutral if politician unknown or Naive Bayes prob close to 0.5\n",
    "    otherwise return Politician initials and +/-'''\n",
    "    if (((sent<upper_threshold) and (sent>(1-upper_threshold))) or (pol=='irrelevant')):\n",
    "        return 'N'\n",
    "    if (pol=='Trump'):\n",
    "        label='T'\n",
    "    if (pol=='Clinton'):\n",
    "        label='C'\n",
    "    if (sent>0.5):\n",
    "        return label+'+'\n",
    "    if (sent<0.5):\n",
    "        return label+'-'\n",
    "    return 'error'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "tweets['label'] = map(lambda name,sent: label_tweet(name,sent) ,tweets['Politician'],tweets['NB_prob+'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text</th>\n",
       "      <th>link</th>\n",
       "      <th>hashtags</th>\n",
       "      <th>mentions</th>\n",
       "      <th>state</th>\n",
       "      <th>NB</th>\n",
       "      <th>NB_prob+</th>\n",
       "      <th>Politician</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>@barackobama \\n@fbi\\n@lorettalynch \\nall in co...</td>\n",
       "      <td>https://t.co/5GMNZq40V3</td>\n",
       "      <td>[nojustice, trumppence]</td>\n",
       "      <td>[barackobama, fbi, lorettalynch, realdonaldtrump]</td>\n",
       "      <td>LA</td>\n",
       "      <td>0</td>\n",
       "      <td>0.268989</td>\n",
       "      <td>Trump</td>\n",
       "      <td>T-</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>#cnn #newday clear #trump deliberately throwin...</td>\n",
       "      <td></td>\n",
       "      <td>[cnn, newday, trump, isis]</td>\n",
       "      <td>[]</td>\n",
       "      <td>MD</td>\n",
       "      <td>0</td>\n",
       "      <td>0.200862</td>\n",
       "      <td>Trump</td>\n",
       "      <td>T-</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>@mike4193496 @realdonaldtrump i totally concur...</td>\n",
       "      <td></td>\n",
       "      <td>[]</td>\n",
       "      <td>[mike4193496, realdonaldtrump]</td>\n",
       "      <td>MD</td>\n",
       "      <td>0</td>\n",
       "      <td>0.059660</td>\n",
       "      <td>Trump</td>\n",
       "      <td>T-</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>@hillaryclinton you are the co-founder of isis...</td>\n",
       "      <td></td>\n",
       "      <td>[]</td>\n",
       "      <td>[hillaryclinton]</td>\n",
       "      <td>TX</td>\n",
       "      <td>1</td>\n",
       "      <td>0.965413</td>\n",
       "      <td>Clinton</td>\n",
       "      <td>C+</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>@realdonaldtrump, you wouldn't recognize a lie...</td>\n",
       "      <td>https://t.co/pKSQM8yikm</td>\n",
       "      <td>[nevertrump]</td>\n",
       "      <td>[realdonaldtrump]</td>\n",
       "      <td>CA</td>\n",
       "      <td>1</td>\n",
       "      <td>0.936727</td>\n",
       "      <td>Trump</td>\n",
       "      <td>T+</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                text                     link  \\\n",
       "0  @barackobama \\n@fbi\\n@lorettalynch \\nall in co...  https://t.co/5GMNZq40V3   \n",
       "1  #cnn #newday clear #trump deliberately throwin...                            \n",
       "2  @mike4193496 @realdonaldtrump i totally concur...                            \n",
       "3  @hillaryclinton you are the co-founder of isis...                            \n",
       "4  @realdonaldtrump, you wouldn't recognize a lie...  https://t.co/pKSQM8yikm   \n",
       "\n",
       "                     hashtags  \\\n",
       "0     [nojustice, trumppence]   \n",
       "1  [cnn, newday, trump, isis]   \n",
       "2                          []   \n",
       "3                          []   \n",
       "4                [nevertrump]   \n",
       "\n",
       "                                            mentions state  NB  NB_prob+  \\\n",
       "0  [barackobama, fbi, lorettalynch, realdonaldtrump]    LA   0  0.268989   \n",
       "1                                                 []    MD   0  0.200862   \n",
       "2                     [mike4193496, realdonaldtrump]    MD   0  0.059660   \n",
       "3                                   [hillaryclinton]    TX   1  0.965413   \n",
       "4                                  [realdonaldtrump]    CA   1  0.936727   \n",
       "\n",
       "  Politician label  \n",
       "0      Trump    T-  \n",
       "1      Trump    T-  \n",
       "2      Trump    T-  \n",
       "3    Clinton    C+  \n",
       "4      Trump    T+  "
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tweets.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# export for each state"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from data.US_states import states"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#initialize a df indexed by label values\n",
    "state_sentiment = pd.DataFrame(index=tweets.label.unique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "for state in states.keys():\n",
    "    state_sentiment[state] = tweets[tweets['state']==state]['label'].value_counts()\n",
    "state_sentiment = state_sentiment.transpose()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>T-</th>\n",
       "      <th>C+</th>\n",
       "      <th>T+</th>\n",
       "      <th>C-</th>\n",
       "      <th>N</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>51.000000</td>\n",
       "      <td>51.000000</td>\n",
       "      <td>51.000000</td>\n",
       "      <td>51.000000</td>\n",
       "      <td>51.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>1708.549020</td>\n",
       "      <td>830.196078</td>\n",
       "      <td>2367.372549</td>\n",
       "      <td>667.294118</td>\n",
       "      <td>1922.568627</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>2498.403181</td>\n",
       "      <td>1160.520418</td>\n",
       "      <td>3417.309076</td>\n",
       "      <td>931.886802</td>\n",
       "      <td>2717.523522</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>51.000000</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>74.000000</td>\n",
       "      <td>25.000000</td>\n",
       "      <td>43.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>397.000000</td>\n",
       "      <td>227.000000</td>\n",
       "      <td>521.000000</td>\n",
       "      <td>154.000000</td>\n",
       "      <td>461.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>715.000000</td>\n",
       "      <td>399.000000</td>\n",
       "      <td>1264.000000</td>\n",
       "      <td>312.000000</td>\n",
       "      <td>1036.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1938.000000</td>\n",
       "      <td>976.000000</td>\n",
       "      <td>2621.000000</td>\n",
       "      <td>842.500000</td>\n",
       "      <td>2158.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>13842.000000</td>\n",
       "      <td>6388.000000</td>\n",
       "      <td>18079.000000</td>\n",
       "      <td>5082.000000</td>\n",
       "      <td>14284.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 T-           C+            T+           C-             N\n",
       "count     51.000000    51.000000     51.000000    51.000000     51.000000\n",
       "mean    1708.549020   830.196078   2367.372549   667.294118   1922.568627\n",
       "std     2498.403181  1160.520418   3417.309076   931.886802   2717.523522\n",
       "min       51.000000    27.000000     74.000000    25.000000     43.000000\n",
       "25%      397.000000   227.000000    521.000000   154.000000    461.500000\n",
       "50%      715.000000   399.000000   1264.000000   312.000000   1036.000000\n",
       "75%     1938.000000   976.000000   2621.000000   842.500000   2158.000000\n",
       "max    13842.000000  6388.000000  18079.000000  5082.000000  14284.000000"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "state_sentiment.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>T-</th>\n",
       "      <th>C+</th>\n",
       "      <th>T+</th>\n",
       "      <th>C-</th>\n",
       "      <th>N</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>WA</th>\n",
       "      <td>2276</td>\n",
       "      <td>1027</td>\n",
       "      <td>3150</td>\n",
       "      <td>833</td>\n",
       "      <td>2300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>WI</th>\n",
       "      <td>687</td>\n",
       "      <td>331</td>\n",
       "      <td>1332</td>\n",
       "      <td>286</td>\n",
       "      <td>1281</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>WV</th>\n",
       "      <td>291</td>\n",
       "      <td>81</td>\n",
       "      <td>430</td>\n",
       "      <td>44</td>\n",
       "      <td>161</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>FL</th>\n",
       "      <td>7685</td>\n",
       "      <td>3595</td>\n",
       "      <td>10974</td>\n",
       "      <td>3100</td>\n",
       "      <td>9331</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>WY</th>\n",
       "      <td>51</td>\n",
       "      <td>56</td>\n",
       "      <td>119</td>\n",
       "      <td>49</td>\n",
       "      <td>71</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      T-    C+     T+    C-     N\n",
       "WA  2276  1027   3150   833  2300\n",
       "WI   687   331   1332   286  1281\n",
       "WV   291    81    430    44   161\n",
       "FL  7685  3595  10974  3100  9331\n",
       "WY    51    56    119    49    71"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "state_sentiment.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "pickle.dump(state_sentiment,open('results/state_sentiment_0.5.pickle','wb'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Investigate tweets using hashtags"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def get_sentiment_from_hashtag(hashtag_list,\n",
    "                               anti_trump_list=set(['nevertrump','dumptrump']), anti_hillary_list=set(['lockherup']),\n",
    "                               pro_trump_list=set(['trumptrain']), pro_hillary_list=set(['imwithher'])\n",
    "                              ):\n",
    "    '''Given a list of hashtags, classify the tweet positively (1), neutral (0) or negatively(-1)'''\n",
    "    hashtag_list = set(hashtag_list)\n",
    "    pro_trump=anti_trump=pro_hillary=anti_hillary = False\n",
    "    negative=positive = False\n",
    "    if (len(hashtag_list.intersection(anti_trump_list))>0) | (len(hashtag_list.intersection(anti_hillary_list))>0):\n",
    "        negative = True\n",
    "    \n",
    "    if (len(hashtag_list.intersection(pro_hillary_list))>0) | (len(hashtag_list.intersection(pro_trump_list))>0):\n",
    "        positive = True\n",
    "\n",
    "    if positive == negative:\n",
    "        return 0 #both negative+positive hashtags\n",
    "    if positive:\n",
    "        return 1\n",
    "    if negative:\n",
    "        return -1\n",
    "    return 0 #no 'explanatory' hashtag found"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "-1\n"
     ]
    }
   ],
   "source": [
    "#test our function\n",
    "print get_sentiment_from_hashtag(['lockherup', 'trumptrain'])\n",
    "print get_sentiment_from_hashtag(['lockherup', 'dumptrump'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-1,  0,  1])"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.unique([tweets.hashtags.apply(lambda x: get_sentiment_from_hashtag(x))])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "tweets['sentiment_hashtag'] = tweets.hashtags.apply(lambda x: get_sentiment_from_hashtag(x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text</th>\n",
       "      <th>link</th>\n",
       "      <th>hashtags</th>\n",
       "      <th>mentions</th>\n",
       "      <th>state</th>\n",
       "      <th>NB</th>\n",
       "      <th>NB_prob+</th>\n",
       "      <th>Politician</th>\n",
       "      <th>label</th>\n",
       "      <th>sentiment_hashtag</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>@barackobama \\n@fbi\\n@lorettalynch \\nall in co...</td>\n",
       "      <td>https://t.co/5GMNZq40V3</td>\n",
       "      <td>[nojustice, trumppence]</td>\n",
       "      <td>[barackobama, fbi, lorettalynch, realdonaldtrump]</td>\n",
       "      <td>LA</td>\n",
       "      <td>0</td>\n",
       "      <td>0.268989</td>\n",
       "      <td>Trump</td>\n",
       "      <td>T-</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>#cnn #newday clear #trump deliberately throwin...</td>\n",
       "      <td></td>\n",
       "      <td>[cnn, newday, trump, isis]</td>\n",
       "      <td>[]</td>\n",
       "      <td>MD</td>\n",
       "      <td>0</td>\n",
       "      <td>0.200862</td>\n",
       "      <td>Trump</td>\n",
       "      <td>T-</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>@mike4193496 @realdonaldtrump i totally concur...</td>\n",
       "      <td></td>\n",
       "      <td>[]</td>\n",
       "      <td>[mike4193496, realdonaldtrump]</td>\n",
       "      <td>MD</td>\n",
       "      <td>0</td>\n",
       "      <td>0.059660</td>\n",
       "      <td>Trump</td>\n",
       "      <td>T-</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>@hillaryclinton you are the co-founder of isis...</td>\n",
       "      <td></td>\n",
       "      <td>[]</td>\n",
       "      <td>[hillaryclinton]</td>\n",
       "      <td>TX</td>\n",
       "      <td>1</td>\n",
       "      <td>0.965413</td>\n",
       "      <td>Clinton</td>\n",
       "      <td>C+</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>@realdonaldtrump, you wouldn't recognize a lie...</td>\n",
       "      <td>https://t.co/pKSQM8yikm</td>\n",
       "      <td>[nevertrump]</td>\n",
       "      <td>[realdonaldtrump]</td>\n",
       "      <td>CA</td>\n",
       "      <td>1</td>\n",
       "      <td>0.936727</td>\n",
       "      <td>Trump</td>\n",
       "      <td>T+</td>\n",
       "      <td>-1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                text                     link  \\\n",
       "0  @barackobama \\n@fbi\\n@lorettalynch \\nall in co...  https://t.co/5GMNZq40V3   \n",
       "1  #cnn #newday clear #trump deliberately throwin...                            \n",
       "2  @mike4193496 @realdonaldtrump i totally concur...                            \n",
       "3  @hillaryclinton you are the co-founder of isis...                            \n",
       "4  @realdonaldtrump, you wouldn't recognize a lie...  https://t.co/pKSQM8yikm   \n",
       "\n",
       "                     hashtags  \\\n",
       "0     [nojustice, trumppence]   \n",
       "1  [cnn, newday, trump, isis]   \n",
       "2                          []   \n",
       "3                          []   \n",
       "4                [nevertrump]   \n",
       "\n",
       "                                            mentions state  NB  NB_prob+  \\\n",
       "0  [barackobama, fbi, lorettalynch, realdonaldtrump]    LA   0  0.268989   \n",
       "1                                                 []    MD   0  0.200862   \n",
       "2                     [mike4193496, realdonaldtrump]    MD   0  0.059660   \n",
       "3                                   [hillaryclinton]    TX   1  0.965413   \n",
       "4                                  [realdonaldtrump]    CA   1  0.936727   \n",
       "\n",
       "  Politician label  sentiment_hashtag  \n",
       "0      Trump    T-                  0  \n",
       "1      Trump    T-                  0  \n",
       "2      Trump    T-                  0  \n",
       "3    Clinton    C+                  0  \n",
       "4      Trump    T+                 -1  "
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tweets.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Benchmark my classifier on negative hashtag tweets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "negative_tweets = clf.predict(selector.transform(vect.transform(tweets.text[tweets.sentiment_hashtag==-1])))\n",
    "negative_idx = tweets.text[tweets.sentiment_hashtag==-1].index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.998751269877 @dds_officer thx 4 the follow. i look 4ward 2 learning n 2 sharing. best, nick. pls visit me at www.  #nevertrump \n",
      "\n",
      "0.714839142351 @rogerjstonejr how is the best dressed man in nyc tonight? wishing you well and praying for you tonight. #maga \n",
      "\n",
      "0.070043121167 i hope they release everything soon. i think she'll have 2 quit due to illness, but either way we need to go.   \n",
      "\n",
      "0.367458973798 all of you so called #conservatives will be putting a lib dem to appoint if you don't vote #trump.  it's more than 4 years at stake #gop \n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/antonis/anaconda2/envs/USelections/lib/python2.7/site-packages/ipykernel/__main__.py:3: DeprecationWarning: \n",
      ".ix is deprecated. Please use\n",
      ".loc for label based indexing or\n",
      ".iloc for positional indexing\n",
      "\n",
      "See the documentation here:\n",
      "http://pandas.pydata.org/pandas-docs/stable/indexing.html#ix-indexer-is-deprecated\n",
      "  app.launch_new_instance()\n"
     ]
    }
   ],
   "source": [
    "for i in range(1,5):\n",
    "    j = random.randint(0,len(negative_idx))\n",
    "    print tweets.ix[j,'NB_prob+'],tweets.text[j],'\\n'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    10429\n",
       "0     8468\n",
       "dtype: int64"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(negative_tweets).value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### We can see that there is a problem in our sentiment classifier, as we would expect it to label more tweets negatively\n",
    "We should investigate ways to improve it, such as train on a political corpus dataset, use bigrams etc. For more on that, please check the discussion section in our assignment."
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Raw Cell Format",
  "kernelspec": {
   "display_name": "Python [conda env:USelections]",
   "language": "python",
   "name": "conda-env-USelections-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}