2.2.1 NB Experiment.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview\n",
    "\n",
    "In this notebook, we use the TakeCV corpus as train and validation data to train Naive Bayes classifiers. Then, we test the models on the Survey corpus. \n",
    "- in the first section, we import the TakeCV data and Multinomial NB from `sklearn`.\n",
    "- We initiate baseline NB, which takes bags-of-unigrams as input. Baseline NB are trained on three types of labels: full labels (color and shape), color, and shape. As well, we assess the results of combining predictions from the color model and the shape model (color + shape).\n",
    "- Then, we build different NB models that take unigrams, bigrams, trigrams, and four-grams. We also report 5-fold cross-validation results, as seen below.\n",
    "- The last section, we import the Survey data and test our models on it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Import data and model\n",
    "## data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import re"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(525, 2)\n"
     ]
    }
   ],
   "source": [
    "df = pd.read_csv('data_clean.csv')\n",
    "print (df.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# split label into color and shape\n",
    "df[['color','shape']] = df['label'].str.split(' ',expand=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "wooden    153\n",
      "blue       84\n",
      "green      68\n",
      "orange     51\n",
      "yellow     48\n",
      "gray       33\n",
      "pink       30\n",
      "red        30\n",
      "purple     28\n",
      "Name: color, dtype: int64\n",
      "t    56\n",
      "f    50\n",
      "w    47\n",
      "u    47\n",
      "i    44\n",
      "v    44\n",
      "l    43\n",
      "z    42\n",
      "p    42\n",
      "y    39\n",
      "n    36\n",
      "x    35\n",
      "Name: shape, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "# check if split correctly\n",
    "print (df['color'].value_counts())\n",
    "print (df['shape'].value_counts())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## MultinomialNB"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "https://practicaldatascience.co.uk/machine-learning/how-to-create-a-naive-bayes-text-classification-model-using-scikit-learn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "from sklearn.model_selection import cross_val_score\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import classification_report, confusion_matrix, ConfusionMatrixDisplay\n",
    "from sklearn.metrics import f1_score\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Baseline NB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# default setting\n",
    "test_size = 0.2\n",
    "seed = 1\n",
    "\n",
    "# transform df into BoW and define X,y\n",
    "count_vec = CountVectorizer(stop_words='english', ngram_range=ngram_range)\n",
    "bow = count_vec.fit_transform(df['correct'])\n",
    "bow = np.array(bow.todense())\n",
    "\n",
    "# define X,y\n",
    "X = bow\n",
    "y = df['label']\n",
    "\n",
    "# shuffle and split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X,y, test_size = test_size, random_state = seed)\n",
    "\n",
    "# split labels\n",
    "y_train_color = [i.split(' ')[0] for i in y_train]\n",
    "y_train_shape = [i.split(' ')[1] for i in y_train]\n",
    "y_test_color = [i.split(' ')[0] for i in y_test]\n",
    "y_test_shape = [i.split(' ')[1] for i in y_test]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((420, 3198), (105, 3198), (420,), (105,))"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.shape, X_test.shape, y_train.shape, y_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "true: wooden z : pred: wooden y\n",
      "true: wooden p : pred: orange y\n",
      "true: orange u : pred: orange u\n",
      "true: wooden f : pred: wooden w\n",
      "true: gray f : pred: blue v\n",
      "true: purple n : pred: purple n\n",
      "true: blue v : pred: blue v\n",
      "true: wooden i : pred: wooden y\n",
      "true: pink p : pred: green p\n",
      "true: orange u : pred: yellow t\n",
      "true: wooden i : pred: wooden y\n",
      "true: blue v : pred: blue v\n",
      "true: yellow u : pred: yellow u\n",
      "true: purple n : pred: wooden n\n",
      "true: green t : pred: green t\n",
      "true: wooden p : pred: purple p\n",
      "true: wooden l : pred: wooden p\n",
      "true: blue z : pred: blue v\n",
      "true: wooden p : pred: wooden f\n",
      "true: blue v : pred: blue i\n",
      "true: wooden f : pred: wooden p\n",
      "true: red x : pred: wooden x\n",
      "true: red x : pred: red x\n",
      "true: blue v : pred: blue v\n",
      "true: wooden l : pred: wooden y\n",
      "true: pink p : pred: pink p\n",
      "true: wooden p : pred: wooden y\n",
      "true: pink p : pred: pink p\n",
      "true: gray f : pred: gray f\n",
      "true: wooden n : pred: wooden f\n",
      "true: orange l : pred: orange l\n",
      "true: wooden y : pred: wooden y\n",
      "true: gray f : pred: gray f\n",
      "true: purple n : pred: orange l\n",
      "true: wooden w : pred: wooden f\n",
      "true: wooden l : pred: wooden p\n",
      "true: green t : pred: green t\n",
      "true: wooden l : pred: wooden p\n",
      "true: wooden w : pred: wooden w\n",
      "true: orange l : pred: orange t\n",
      "true: green w : pred: green w\n",
      "true: wooden z : pred: wooden i\n",
      "true: wooden w : pred: red x\n",
      "true: orange l : pred: yellow t\n",
      "true: purple n : pred: yellow u\n",
      "true: blue i : pred: blue i\n",
      "true: green w : pred: green w\n",
      "true: pink p : pred: pink p\n",
      "true: wooden i : pred: wooden w\n",
      "true: purple n : pred: wooden i\n",
      "true: pink p : pred: pink p\n",
      "true: gray f : pred: gray f\n",
      "true: wooden f : pred: wooden y\n",
      "true: blue z : pred: orange w\n",
      "true: wooden z : pred: wooden y\n",
      "true: yellow t : pred: yellow u\n",
      "true: wooden z : pred: wooden y\n",
      "true: purple n : pred: wooden w\n",
      "true: wooden y : pred: wooden y\n",
      "true: orange u : pred: yellow u\n",
      "true: wooden y : pred: wooden l\n",
      "true: green t : pred: green t\n",
      "true: green t : pred: green w\n",
      "true: yellow u : pred: yellow u\n",
      "true: blue z : pred: blue z\n",
      "true: blue i : pred: blue i\n",
      "true: pink p : pred: blue p\n",
      "true: wooden n : pred: wooden l\n",
      "true: wooden i : pred: wooden i\n",
      "true: blue i : pred: blue i\n",
      "true: red x : pred: red x\n",
      "true: green w : pred: green w\n",
      "true: blue z : pred: blue v\n",
      "true: green w : pred: green w\n",
      "true: green w : pred: orange l\n",
      "true: pink p : pred: pink p\n",
      "true: blue v : pred: blue v\n",
      "true: red x : pred: red x\n",
      "true: purple n : pred: wooden n\n",
      "true: wooden l : pred: wooden y\n",
      "true: green w : pred: green w\n",
      "true: gray f : pred: gray f\n",
      "true: blue i : pred: blue i\n",
      "true: wooden y : pred: wooden y\n",
      "true: yellow t : pred: orange u\n",
      "true: green t : pred: green t\n",
      "true: red x : pred: red x\n",
      "true: wooden l : pred: blue z\n",
      "true: purple n : pred: purple n\n",
      "true: orange u : pred: wooden f\n",
      "true: green t : pred: green w\n",
      "true: green t : pred: green t\n",
      "true: green w : pred: wooden w\n",
      "true: red x : pred: red x\n",
      "true: yellow u : pred: yellow u\n",
      "true: wooden z : pred: blue z\n",
      "true: wooden y : pred: wooden v\n",
      "true: wooden v : pred: blue v\n",
      "true: yellow u : pred: yellow u\n",
      "true: pink p : pred: pink p\n",
      "true: wooden f : pred: wooden f\n",
      "true: wooden y : pred: wooden y\n",
      "true: wooden v : pred: wooden z\n",
      "true: yellow u : pred: yellow u\n",
      "true: wooden i : pred: wooden y\n"
     ]
    }
   ],
   "source": [
    "# train models\n",
    "label_model = MultinomialNB().fit(X_train,y_train)\n",
    "color_model = MultinomialNB().fit(X_train,y_train_color)\n",
    "shape_model = MultinomialNB().fit(X_train,y_train_shape)\n",
    "\n",
    "# get predictions\n",
    "y_pred = label_model.predict(X_test)\n",
    "color_pred = color_model.predict(X_test)\n",
    "shape_pred = shape_model.predict(X_test)\n",
    "\n",
    "cs_pred = []\n",
    "\n",
    "for x,y in zip(X_test, y_test):\n",
    "    c_pred = color_model.predict(x.reshape(1,-1))\n",
    "    s_pred = shape_model.predict(x.reshape(1,-1))\n",
    "\n",
    "    color_shape_pred = c_pred.item()+' '+s_pred.item()\n",
    "    cs_pred.append(color_shape_pred)\n",
    "    \n",
    "    print ('true:', y, ': pred:', color_shape_pred)   \n",
    "\n",
    "# cannot do cross-validation, except by manual coding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Acc on full label: 0.5428571428571428\n",
      "Acc on color: 0.7714285714285715\n",
      "Acc on shape: 0.5714285714285714\n",
      "Acc on color + shape: 0.47619047619047616\n"
     ]
    }
   ],
   "source": [
    "# print accuracy\n",
    "# this is accuracy on test data\n",
    "print('Acc on full label:', accuracy_score(np.array(y_test), y_pred))\n",
    "print('Acc on color:', accuracy_score(y_test_color, color_pred))\n",
    "print('Acc on shape:', accuracy_score(y_test_shape, shape_pred))\n",
    "print('Acc on color + shape:', accuracy_score(y_test, cs_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# NB Extension"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# default setting\n",
    "ngram_range = (1,3) # n-grams (min, max)\n",
    "cv = 5 # cross validation folds: set to 5 because the smallest class have only 5 instances\n",
    "\n",
    "def transform_data(label, ngram_range):\n",
    "    # transform df into BoW and define X,y\n",
    "    count_vec = CountVectorizer(stop_words='english', ngram_range=ngram_range)\n",
    "    bow = count_vec.fit_transform(df['text'])\n",
    "    bow = np.array(bow.todense())\n",
    "\n",
    "    # define X,y\n",
    "    X = bow\n",
    "    y = df[label]\n",
    "\n",
    "    return X, y\n",
    "\n",
    "\n",
    "def train_and_predict(X,y,test_size=test_size, seed=seed):\n",
    "    \"\"\"\n",
    "    input: X (vectorized, as np.array), y (label, as string)\n",
    "    print: results on accuracy\n",
    "    output: y_prediction on test data\n",
    "    \n",
    "    \"\"\"\n",
    "    # split data\n",
    "    X_train, X_test, y_train, y_test = train_test_split(X, y, \n",
    "                                                    test_size=test_size,\n",
    "                                                    random_state=seed, \n",
    "                                                    stratify=y) # make sure to split instances from each class of y\n",
    "    # train and predict\n",
    "    # initialize model\n",
    "    batch_model = MultinomialNB().fit(np.random.rand(X_train.shape[0],X_train.shape[1]), y_train) # feed with random data\n",
    "    \n",
    "    # pre-trained model's prediction\n",
    "    y_pred = batch_model.predict(X_test)\n",
    "    print('Pre-trained accuracy:', accuracy_score(y_test, y_pred)) \n",
    "    print('Pre-trained F1 score:', f1_score(y_test, y_pred, average=\"macro\"))\n",
    "\n",
    "    # trained model's prediction\n",
    "    batch_model.fit(X_train, y_train)\n",
    "    y_pred = batch_model.predict(X_test)\n",
    "    \n",
    "    # get scores\n",
    "    print('Trained accuracy:', accuracy_score(y_test, y_pred)) \n",
    "    print('Trained F1 score:', f1_score(y_test, y_pred, average=\"macro\"),'\\n')\n",
    "    \n",
    "    return (y_test, y_pred, batch_model)\n",
    "    \n",
    "def report_prediction(y_test,y_pred, batch_model):\n",
    "    \"\"\"\n",
    "    input: true label and prediction\n",
    "    print: classification report, confusion matrix\n",
    "    return: nothing\n",
    "    \"\"\"\n",
    "    \n",
    "    # get prediction report\n",
    "    print(classification_report(y_test,y_pred))\n",
    "\n",
    "    # confusion matrix and viualization\n",
    "    cm = confusion_matrix(y_test, y_pred, labels=batch_model.classes_)\n",
    "    disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=batch_model.classes_)\n",
    "    disp.plot()\n",
    "    plt.show()\n",
    "    \n",
    "def cross_val(X, y, batch_model, cv=cv): \n",
    "    scores = cross_val_score(batch_model, X, y, cv=cv)\n",
    "    print ('Cross validation accuracy score:', scores)\n",
    "    print ('Cross validation accuracy score (average):', np.average(scores))\n",
    "    return np.average(scores)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trained accuracy, on train data: 0.76\n"
     ]
    }
   ],
   "source": [
    "# Accuracy on full training data\n",
    "\n",
    "(X,y) = transform_data('label', (1,1))\n",
    "\n",
    "model = MultinomialNB().fit(X,y)\n",
    "y_pred = model.predict(X)\n",
    "\n",
    "print('Trained accuracy, on train data:', accuracy_score(y, y_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Label: label\n",
      "\n",
      "n-gram: (1, 1)\n",
      "Pre-trained accuracy: 0.0380952380952381\n",
      "Pre-trained F1 score: 0.01316661930472672\n",
      "Trained accuracy: 0.5619047619047619\n",
      "Trained F1 score: 0.43392660479617007 \n",
      "\n",
      "Cross validation accuracy score: [0.53333333 0.52380952 0.51428571 0.53333333 0.55238095]\n",
      "Cross validation accuracy score (average): 0.5314285714285715\n",
      "\n",
      "n-gram: (1, 2)\n",
      "Pre-trained accuracy: 0.009523809523809525\n",
      "Pre-trained F1 score: 0.021739130434782608\n",
      "Trained accuracy: 0.5714285714285714\n",
      "Trained F1 score: 0.42668020385411687 \n",
      "\n",
      "Cross validation accuracy score: [0.54285714 0.44761905 0.52380952 0.57142857 0.57142857]\n",
      "Cross validation accuracy score (average): 0.5314285714285714\n",
      "\n",
      "n-gram: (1, 3)\n",
      "Pre-trained accuracy: 0.06666666666666667\n",
      "Pre-trained F1 score: 0.03714479387447252\n",
      "Trained accuracy: 0.5904761904761905\n",
      "Trained F1 score: 0.4342464298986038 \n",
      "\n",
      "Cross validation accuracy score: [0.56190476 0.44761905 0.52380952 0.56190476 0.57142857]\n",
      "Cross validation accuracy score (average): 0.5333333333333332\n",
      "\n",
      "n-gram: (1, 4)\n",
      "Pre-trained accuracy: 0.0380952380952381\n",
      "Pre-trained F1 score: 0.01941901856089499\n",
      "Trained accuracy: 0.5904761904761905\n",
      "Trained F1 score: 0.425879917184265 \n",
      "\n",
      "Cross validation accuracy score: [0.56190476 0.43809524 0.53333333 0.57142857 0.57142857]\n",
      "Cross validation accuracy score (average): 0.5352380952380951\n",
      "\n",
      "n-gram: (2, 2)\n",
      "Pre-trained accuracy: 0.0761904761904762\n",
      "Pre-trained F1 score: 0.02474712057461005\n",
      "Trained accuracy: 0.4380952380952381\n",
      "Trained F1 score: 0.347914194606066 \n",
      "\n",
      "Cross validation accuracy score: [0.44761905 0.32380952 0.37142857 0.43809524 0.35238095]\n",
      "Cross validation accuracy score (average): 0.38666666666666666\n",
      "\n",
      "n-gram: (2, 3)\n",
      "Pre-trained accuracy: 0.05714285714285714\n",
      "Pre-trained F1 score: 0.04072557876905703\n",
      "Trained accuracy: 0.47619047619047616\n",
      "Trained F1 score: 0.38145483388450147 \n",
      "\n",
      "Cross validation accuracy score: [0.45714286 0.34285714 0.35238095 0.41904762 0.37142857]\n",
      "Cross validation accuracy score (average): 0.3885714285714286\n",
      "\n",
      "n-gram: (2, 4)\n",
      "Pre-trained accuracy: 0.08571428571428572\n",
      "Pre-trained F1 score: 0.04998086645498234\n",
      "Trained accuracy: 0.47619047619047616\n",
      "Trained F1 score: 0.38046041767269384 \n",
      "\n",
      "Cross validation accuracy score: [0.46666667 0.33333333 0.35238095 0.42857143 0.37142857]\n",
      "Cross validation accuracy score (average): 0.39047619047619053\n",
      "\n",
      "n-gram: (3, 3)\n",
      "Pre-trained accuracy: 0.0761904761904762\n",
      "Pre-trained F1 score: 0.031578573458368855\n",
      "Trained accuracy: 0.37142857142857144\n",
      "Trained F1 score: 0.28975131149044187 \n",
      "\n",
      "Cross validation accuracy score: [0.2952381  0.25714286 0.2        0.2952381  0.26666667]\n",
      "Cross validation accuracy score (average): 0.26285714285714284\n",
      "\n",
      "n-gram: (3, 4)\n",
      "Pre-trained accuracy: 0.0761904761904762\n",
      "Pre-trained F1 score: 0.04885265700483092\n",
      "Trained accuracy: 0.38095238095238093\n",
      "Trained F1 score: 0.3044138126596897 \n",
      "\n",
      "Cross validation accuracy score: [0.3047619  0.26666667 0.2        0.2952381  0.2952381 ]\n",
      "Cross validation accuracy score (average): 0.2723809523809524\n",
      "\n",
      "n-gram: (4, 4)\n",
      "Pre-trained accuracy: 0.05714285714285714\n",
      "Pre-trained F1 score: 0.036547152395679745\n",
      "Trained accuracy: 0.1523809523809524\n",
      "Trained F1 score: 0.11053367936518108 \n",
      "\n",
      "Cross validation accuracy score: [0.14285714 0.16190476 0.13333333 0.08571429 0.12380952]\n",
      "Cross validation accuracy score (average): 0.12952380952380954\n",
      "\n",
      "Label: color\n",
      "\n",
      "n-gram: (1, 1)\n",
      "Pre-trained accuracy: 0.2761904761904762\n",
      "Pre-trained F1 score: 0.04809286898839137\n",
      "Trained accuracy: 0.8761904761904762\n",
      "Trained F1 score: 0.8827212003682592 \n",
      "\n",
      "Cross validation accuracy score: [0.78095238 0.8        0.80952381 0.77142857 0.75238095]\n",
      "Cross validation accuracy score (average): 0.7828571428571429\n",
      "\n",
      "n-gram: (1, 2)\n",
      "Pre-trained accuracy: 0.26666666666666666\n",
      "Pre-trained F1 score: 0.04786324786324787\n",
      "Trained accuracy: 0.8095238095238095\n",
      "Trained F1 score: 0.7810739158799449 \n",
      "\n",
      "Cross validation accuracy score: [0.79047619 0.73333333 0.8        0.7047619  0.68571429]\n",
      "Cross validation accuracy score (average): 0.7428571428571429\n",
      "\n",
      "n-gram: (1, 3)\n",
      "Pre-trained accuracy: 0.2857142857142857\n",
      "Pre-trained F1 score: 0.050125313283208024\n",
      "Trained accuracy: 0.819047619047619\n",
      "Trained F1 score: 0.8000749753861433 \n",
      "\n",
      "Cross validation accuracy score: [0.78095238 0.71428571 0.8        0.6952381  0.67619048]\n",
      "Cross validation accuracy score (average): 0.7333333333333334\n",
      "\n",
      "n-gram: (1, 4)\n",
      "Pre-trained accuracy: 0.26666666666666666\n",
      "Pre-trained F1 score: 0.06844444444444446\n",
      "Trained accuracy: 0.8571428571428571\n",
      "Trained F1 score: 0.8339275103980986 \n",
      "\n",
      "Cross validation accuracy score: [0.78095238 0.73333333 0.81904762 0.7047619  0.6952381 ]\n",
      "Cross validation accuracy score (average): 0.7466666666666667\n",
      "\n",
      "n-gram: (2, 2)\n",
      "Pre-trained accuracy: 0.29523809523809524\n",
      "Pre-trained F1 score: 0.06209692279344021\n",
      "Trained accuracy: 0.6\n",
      "Trained F1 score: 0.5168177761551256 \n",
      "\n",
      "Cross validation accuracy score: [0.64761905 0.53333333 0.56190476 0.57142857 0.58095238]\n",
      "Cross validation accuracy score (average): 0.579047619047619\n",
      "\n",
      "n-gram: (2, 3)\n",
      "Pre-trained accuracy: 0.2857142857142857\n",
      "Pre-trained F1 score: 0.06326928432191589\n",
      "Trained accuracy: 0.6476190476190476\n",
      "Trained F1 score: 0.612679549934452 \n",
      "\n",
      "Cross validation accuracy score: [0.61904762 0.52380952 0.51428571 0.55238095 0.57142857]\n",
      "Cross validation accuracy score (average): 0.5561904761904761\n",
      "\n",
      "n-gram: (2, 4)\n",
      "Pre-trained accuracy: 0.2857142857142857\n",
      "Pre-trained F1 score: 0.06126855600539812\n",
      "Trained accuracy: 0.6190476190476191\n",
      "Trained F1 score: 0.5755582112466171 \n",
      "\n",
      "Cross validation accuracy score: [0.6        0.4952381  0.5047619  0.52380952 0.53333333]\n",
      "Cross validation accuracy score (average): 0.5314285714285714\n",
      "\n",
      "n-gram: (3, 3)\n",
      "Pre-trained accuracy: 0.2761904761904762\n",
      "Pre-trained F1 score: 0.04809286898839137\n",
      "Trained accuracy: 0.4857142857142857\n",
      "Trained F1 score: 0.4099817265388044 \n",
      "\n",
      "Cross validation accuracy score: [0.39047619 0.36190476 0.33333333 0.33333333 0.34285714]\n",
      "Cross validation accuracy score (average): 0.35238095238095235\n",
      "\n",
      "n-gram: (3, 4)\n",
      "Pre-trained accuracy: 0.26666666666666666\n",
      "Pre-trained F1 score: 0.04678362573099414\n",
      "Trained accuracy: 0.4\n",
      "Trained F1 score: 0.36810742815644776 \n",
      "\n",
      "Cross validation accuracy score: [0.31428571 0.28571429 0.27619048 0.28571429 0.33333333]\n",
      "Cross validation accuracy score (average): 0.299047619047619\n",
      "\n",
      "n-gram: (4, 4)\n",
      "Pre-trained accuracy: 0.2857142857142857\n",
      "Pre-trained F1 score: 0.04938271604938272\n",
      "Trained accuracy: 0.3238095238095238\n",
      "Trained F1 score: 0.26463435284606096 \n",
      "\n",
      "Cross validation accuracy score: [0.28571429 0.26666667 0.31428571 0.19047619 0.24761905]\n",
      "Cross validation accuracy score (average): 0.26095238095238094\n",
      "\n",
      "Label: shape\n",
      "\n",
      "n-gram: (1, 1)\n",
      "Pre-trained accuracy: 0.08571428571428572\n",
      "Pre-trained F1 score: 0.039834793424183425\n",
      "Trained accuracy: 0.7047619047619048\n",
      "Trained F1 score: 0.7001896930487589 \n",
      "\n",
      "Cross validation accuracy score: [0.7047619  0.61904762 0.63809524 0.46666667 0.24761905]\n",
      "Cross validation accuracy score (average): 0.5352380952380953\n",
      "\n",
      "n-gram: (1, 2)\n",
      "Pre-trained accuracy: 0.06666666666666667\n",
      "Pre-trained F1 score: 0.05911680911680911\n",
      "Trained accuracy: 0.7333333333333333\n",
      "Trained F1 score: 0.7318775125840343 \n",
      "\n",
      "Cross validation accuracy score: [0.6952381  0.61904762 0.6        0.47619048 0.3047619 ]\n",
      "Cross validation accuracy score (average): 0.5390476190476191\n",
      "\n",
      "n-gram: (1, 3)\n",
      "Pre-trained accuracy: 0.0761904761904762\n",
      "Pre-trained F1 score: 0.06251134618070102\n",
      "Trained accuracy: 0.7142857142857143\n",
      "Trained F1 score: 0.7123381751099144 \n",
      "\n",
      "Cross validation accuracy score: [0.71428571 0.60952381 0.59047619 0.5047619  0.32380952]\n",
      "Cross validation accuracy score (average): 0.5485714285714287\n",
      "\n",
      "n-gram: (1, 4)\n",
      "Pre-trained accuracy: 0.06666666666666667\n",
      "Pre-trained F1 score: 0.054522031026626776\n",
      "Trained accuracy: 0.7142857142857143\n",
      "Trained F1 score: 0.7123381751099144 \n",
      "\n",
      "Cross validation accuracy score: [0.72380952 0.6        0.59047619 0.4952381  0.3047619 ]\n",
      "Cross validation accuracy score (average): 0.5428571428571429\n",
      "\n",
      "n-gram: (2, 2)\n",
      "Pre-trained accuracy: 0.1619047619047619\n",
      "Pre-trained F1 score: 0.11741783024117851\n",
      "Trained accuracy: 0.5619047619047619\n",
      "Trained F1 score: 0.5480010449175534 \n",
      "\n",
      "Cross validation accuracy score: [0.51428571 0.5047619  0.45714286 0.32380952 0.27619048]\n",
      "Cross validation accuracy score (average): 0.4152380952380952\n",
      "\n",
      "n-gram: (2, 3)\n",
      "Pre-trained accuracy: 0.11428571428571428\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pre-trained F1 score: 0.0993219652533378\n",
      "Trained accuracy: 0.5428571428571428\n",
      "Trained F1 score: 0.5371685819054239 \n",
      "\n",
      "Cross validation accuracy score: [0.53333333 0.45714286 0.44761905 0.34285714 0.27619048]\n",
      "Cross validation accuracy score (average): 0.41142857142857137\n",
      "\n",
      "n-gram: (2, 4)\n",
      "Pre-trained accuracy: 0.09523809523809523\n",
      "Pre-trained F1 score: 0.0633501096915731\n",
      "Trained accuracy: 0.5428571428571428\n",
      "Trained F1 score: 0.5414253756649267 \n",
      "\n",
      "Cross validation accuracy score: [0.53333333 0.43809524 0.42857143 0.33333333 0.26666667]\n",
      "Cross validation accuracy score (average): 0.39999999999999997\n",
      "\n",
      "n-gram: (3, 3)\n",
      "Pre-trained accuracy: 0.09523809523809523\n",
      "Pre-trained F1 score: 0.06543400102722136\n",
      "Trained accuracy: 0.44761904761904764\n",
      "Trained F1 score: 0.45468663889716515 \n",
      "\n",
      "Cross validation accuracy score: [0.39047619 0.28571429 0.3047619  0.23809524 0.26666667]\n",
      "Cross validation accuracy score (average): 0.29714285714285715\n",
      "\n",
      "n-gram: (3, 4)\n",
      "Pre-trained accuracy: 0.14285714285714285\n",
      "Pre-trained F1 score: 0.09492673992673994\n",
      "Trained accuracy: 0.45714285714285713\n",
      "Trained F1 score: 0.4666188386776622 \n",
      "\n",
      "Cross validation accuracy score: [0.4        0.28571429 0.3047619  0.22857143 0.25714286]\n",
      "Cross validation accuracy score (average): 0.29523809523809524\n",
      "\n",
      "n-gram: (4, 4)\n",
      "Pre-trained accuracy: 0.1619047619047619\n",
      "Pre-trained F1 score: 0.08399959689433373\n",
      "Trained accuracy: 0.23809523809523808\n",
      "Trained F1 score: 0.23975125403696831 \n",
      "\n",
      "Cross validation accuracy score: [0.24761905 0.21904762 0.17142857 0.16190476 0.13333333]\n",
      "Cross validation accuracy score (average): 0.18666666666666665\n"
     ]
    }
   ],
   "source": [
    "# get accuracy for each type of label and n-gram\n",
    "ngram_list = [(1,1),(1,2),(1,3),(1,4),(2,2),(2,3),(2,4),(3,3),(3,4),(4,4)]\n",
    "y_list = ['label','color','shape']\n",
    "\n",
    "# store result\n",
    "result_list = []\n",
    "\n",
    "for i,y in enumerate(y_list):\n",
    "    print ('\\nLabel:', y_list[i])\n",
    "    for n in ngram_list:\n",
    "        print ('\\nn-gram:', n)\n",
    "        (X,y) = transform_data(y_list[i], n)\n",
    "        (y_test, y_pred, batch_model) = train_and_predict(X,y)\n",
    "        #report_prediction(y_test, y_pred, batch_model)\n",
    "        avg_cv_score = cross_val(X,y, batch_model)\n",
    "        result_list.append([y_list[i], n, accuracy_score(y_test,y_pred), avg_cv_score])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   Label  Accuracy           CV_Accuracy          \n",
      "              mean       std        mean       std\n",
      "0  color  0.643810  0.196374    0.558476  0.197144\n",
      "1  label  0.460952  0.136042    0.396190  0.141075\n",
      "2  shape  0.565714  0.158361    0.417143  0.126752\n"
     ]
    }
   ],
   "source": [
    "result_df = pd.DataFrame({\n",
    "    'Label': [item[0] for item in result_list],\n",
    "    'Ngram': [item[1] for item in result_list],\n",
    "    'Accuracy': [item[2] for item in result_list],\n",
    "    'CV_Accuracy': [item[3] for item in result_list]\n",
    "    })\n",
    "\n",
    "print (result_df.groupby('Label').agg({'Accuracy':['mean','std'],'CV_Accuracy':['mean','std']}).reset_index())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Ngram</th>\n",
       "      <th>Accuracy</th>\n",
       "      <th>CV_Accuracy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>(1, 1)</td>\n",
       "      <td>0.561905</td>\n",
       "      <td>0.531429</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>(1, 2)</td>\n",
       "      <td>0.571429</td>\n",
       "      <td>0.531429</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>(1, 3)</td>\n",
       "      <td>0.590476</td>\n",
       "      <td>0.533333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>(1, 4)</td>\n",
       "      <td>0.590476</td>\n",
       "      <td>0.535238</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>(2, 2)</td>\n",
       "      <td>0.438095</td>\n",
       "      <td>0.386667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>(2, 3)</td>\n",
       "      <td>0.476190</td>\n",
       "      <td>0.388571</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>(2, 4)</td>\n",
       "      <td>0.476190</td>\n",
       "      <td>0.390476</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>(3, 3)</td>\n",
       "      <td>0.371429</td>\n",
       "      <td>0.262857</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>(3, 4)</td>\n",
       "      <td>0.380952</td>\n",
       "      <td>0.272381</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>(4, 4)</td>\n",
       "      <td>0.152381</td>\n",
       "      <td>0.129524</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Ngram  Accuracy  CV_Accuracy\n",
       "0  (1, 1)  0.561905     0.531429\n",
       "1  (1, 2)  0.571429     0.531429\n",
       "2  (1, 3)  0.590476     0.533333\n",
       "3  (1, 4)  0.590476     0.535238\n",
       "4  (2, 2)  0.438095     0.386667\n",
       "5  (2, 3)  0.476190     0.388571\n",
       "6  (2, 4)  0.476190     0.390476\n",
       "7  (3, 3)  0.371429     0.262857\n",
       "8  (3, 4)  0.380952     0.272381\n",
       "9  (4, 4)  0.152381     0.129524"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "res = result_df[result_df['Label']=='label']\n",
    "res = res.drop(columns=['Label'])\n",
    "res"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmEAAAJcCAYAAACxEXM4AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAsCUlEQVR4nO3de7RdZXkv/u9jgkVQkUtqVcBED5ZbiEICih6gFRUvFRAspIoHKVCqUdGfDhn2FCkee7TaU7VeONRSi9XEigUtRagUkHpBEhGRgFS8EjkqQgUCREjy/v7YO+km5LKT7JV3753PZ4yMrnl71/NkLcO375xrzmqtBQCALetRvQsAANgaCWEAAB0IYQAAHQhhAAAdCGEAAB0IYQAAHQhhwKRTVdOrqlXV1OHlq6rq5HXse1ZV/cMox/1EVf2vTaxpk48FJichDBjXqupHVfVAVS0d8efJvesC2FxTexcAMAq/11q7vHcRAGPJTBgwIQ3PkB0+YnnUpxU3MO5nq+pnVXV3VV1dVfusscsuVfWlqrq3qr5cVU8dceyew9vuqqpbqur3N7ceYPISwgAe7otJ9kjym0muS/KpNba/Ksm7kuyS5PpV26tq+yRfSvLp4WPnJvnoWkIcQBKnI4GJ4aKqWj78+qrW2lGDeqPW2nmrXlfVWUn+s6p2aK3dPbz6X1prVw9v/5Mkd1fVbkkOTvKj1trfDe93XVV9LsmxSRYPql5g4hLCgIngqC1xTVhVTUny7iSvTDItycrhTbskWRXCblu1f2ttaVXdleTJSZ6a5KCq+tWIIacm+eSAywYmKCEMmKjuS7LdiOXfGoMx/yDJkUkOT/KjJDsk+c8kNWKf3Va9qKrHJtkpye0ZCmdfbq29YAzqALYCrgkDJqrrkxxfVdtU1ewMnfbbXI9L8uskd2Yo4P35WvZ5SVU9r6oenaFrw77RWrstycVJnlFVJwzXtE1VzamqvcagLmASEsKAiepPkzw9QzNVf5ahC+I31/lJfpzkp0luSnLNWvb5dJJ3JrkryQEZulA/rbV7k7wwyfEZmhn7WZL3JvmNMagLmISqtda7BgCArY6ZMACADoQwAIAOhDAAgA6EMACADibcfcKOOOKIdumll/YuAwBgNGpdGybcTNgvf/nL3iUAAGy2CRfCAAAmAyEMAKADIQwAoIMJd2E+ADDkoYceypIlS7Js2bLepWz1tt122+y6667ZZpttRn2MEAYAE9SSJUvyuMc9LtOnT0/VOn+Ex4C11nLnnXdmyZIlmTFjxqiPczoSACaoZcuWZeeddxbAOquq7Lzzzhs9IymEAcAEJoCND5vyOQhhAAAdCGEAwEY57LDDctlllz1s3Qc+8IG87nWvG9XxZ555Zi6//PINvseiRYsesf4Tn/hE5s2bN/pixzEhDADYKHPnzs2CBQsetm7BggWZO3fuBo9dsWJFzj777Bx++OGDKm/CEMIAgI1y7LHH5uKLL86vf/3rJMmPfvSj3H777fn0pz+d2bNnZ5999sk73/nO1ftPnz49Z599dp73vOfls5/9bE488cRccMEFSZKzzz47c+bMyb777ptTTz01rbXVx/3DP/xDDj744Oy777659tprH1HHHXfckWOOOSZz5szJnDlz8tWvfnXAnY8tIQwA2Cg777xzDjzwwFx66aVJhmbBjjvuuLz73e/OokWLcsMNN+TLX/5ybrjhhtXHbLvttvnKV76S448//mFjzZs3LwsXLsyNN96YBx54IBdffPHqbffdd1++9rWv5aMf/WhOOumkR9Txpje9KW9+85uzcOHCfO5zn8vJJ588oI4HQwgDADbayFOSq05F/uM//mP233//POtZz8rixYtz0003rd7/uOOOW+s4V155ZQ466KDMnDkzV1xxRRYvXvyw90iSQw45JPfcc09+9atfPezYyy+/PPPmzcszn/nMvPzlL88999yTe++9d4w7HZyB3qy1qo5I8sEkU5J8vLX2nrXsc1iSDyTZJskvW2uHDrImAGDzHXXUUXnLW96S6667Lg888EB23HHHvP/978/ChQuz44475sQTT3zYfbO23377R4yxbNmyvO51r8uiRYuy22675ayzznrYMWve9mHN5ZUrV+brX/96HvOYx4xxd1vGwGbCqmpKko8keXGSvZPMraq919jnCUk+muTlrbV9krxyUPUAAGPnsY99bA477LCcdNJJmTt3bu65555sv/322WGHHfLzn/88X/ziFzc4xqrAtcsuu2Tp0qWrrxNb5TOf+UyS5Ctf+Up22GGH7LDDDg/b/sIXvjAf/vCHVy9ff/31m9nVljXImbADk9zaWvtBklTVgiRHJrlpxD5/kOSfWms/SZLW2i8GWA8AMIbmzp2bV7ziFVmwYEH23HPPPOtZz8o+++yTpz3taXnuc5+7weOf8IQn5JRTTsnMmTMzffr0zJkz52Hbd9xxxxx88MG55557ct555z3i+A996EN5/etfn/322y/Lly/PIYccknPOOWfM+hu0GvkrhDEduOrYJEe01k4eXj4hyUGttXkj9vlAhk5D7pPkcUk+2Fo7fy1jnZrk1CTZfffdD/jxj388kJoBYCK5+eabs9dee/Uug2Hr+DzWeSv9QV6Yv7Y3XTPxTU1yQJKXJnlRkj+tqmc84qDWzm2tzW6tzZ42bdrYVwoAsIUN8nTkkiS7jVjeNcnta9nnl621+5LcV1VXJ5mV5D8GWBcAQHeDnAlbmGSPqppRVY9OcnySL6yxz+eT/PeqmlpV2yU5KMnNA6wJAGBcGNhMWGtteVXNS3JZhm5RcV5rbXFVnTa8/ZzW2s1VdWmSG5KszNBtLG4cVE0AAOPFQO8T1lq7JMkla6w7Z43l9yV53yDrAAAYb9wxHwCgg4HOhAEAW84Bb3vEXZ42yzff95pR7XfhhRfmFa94RW6++ebsueeeY1rDZCaE0d1Y/6Mx0mj/ARmkyd4fwPz58/O85z0vCxYsyFlnnTWQ91ixYkWmTJkykLF7cToSANhkS5cuzVe/+tX87d/+7eoHeq9YsSJvfetbM3PmzOy3337567/+6yTJwoULc/DBB2fWrFk58MADc++99+YTn/hE5s1bfR/3vOxlL8tVV12VZOjRSGeeeWYOOuigfP3rX8/ZZ5+dOXPmZN99982pp56aVTecv/XWW3P44Ydn1qxZ2X///fP9738/J5xwQj7/+c+vHvdVr3pVvvCFNW/S0JeZMABgk1100UU54ogj8oxnPCM77bRTrrvuunzjG9/ID3/4w3zrW9/K1KlTc9ddd+XBBx/Mcccdl8985jOZM2dO7rnnng0+ePu+++7Lvvvum7PPPjtJsvfee+fMM89Mkpxwwgm5+OKL83u/93t51atelTPOOCNHH310li1blpUrV+bkk0/OX/3VX+XII4/M3Xffna997Wv5+7//+4H/fWwMM2EAwCabP39+jj/++CTJ8ccfn/nz5+fyyy/PaaedlqlTh+Z6dtppp9xyyy150pOetPr5kI9//ONXb1+XKVOm5Jhjjlm9fOWVV+aggw7KzJkzc8UVV2Tx4sW5995789Of/jRHH310kmTbbbfNdtttl0MPPTS33nprfvGLX2T+/Pk55phjNvh+W9r4qgYAmDDuvPPOXHHFFbnxxhtTVVmxYkWqKgcccECqHv70wtbaI9YlydSpU7Ny5crVy8uWLVv9etttt119HdiyZcvyute9LosWLcpuu+2Ws846K8uWLcv6noF9wgkn5FOf+lQWLFiw1geA92YmDADYJBdccEFe85rX5Mc//nF+9KMf5bbbbsuMGTOy//7755xzzsny5cuTJHfddVf23HPP3H777Vm4cGGS5N57783y5cszffr0XH/99Vm5cmVuu+22XHvttWt9r1XhbJdddsnSpUtzwQUXJBmaUdt1111z0UUXJUl+/etf5/7770+SnHjiifnABz6QJNlnn30G9dewycyEAcAksaV/MT1//vycccYZD1t3zDHH5Oabb87uu++e/fbbL9tss01OOeWUzJs3L5/5zGfyhje8IQ888EAe85jH5PLLL89zn/vczJgxIzNnzsy+++6b/ffff63v9YQnPCGnnHJKZs6cmenTp68+rZkkn/zkJ/NHf/RHOfPMM7PNNtvks5/9bJ72tKfliU98Yvbaa68cddRRg/xr2GS1vmm88Wj27Nlt0aJFvctgDE32WzhM9v6Afm6++ebstddevcsYt+6///7MnDkz1113XXbYYYeBv986Po9HnoMd5nQkADDpXH755dlzzz3zhje8YYsEsE3hdCQAMOkcfvjh+clPftK7jPUyEwYA0IEQBgDQgRAGANCBEAYA0IEL8wFgkvjJ2TPHdLzdz/zOmI7Hw5kJAwA22c9+9rMcf/zxefrTn5699947L3nJS1JVueWWWx623+mnn56/+Iu/WO9Yb3rTm/KUpzzlYY8xmsyEMABgk7TWcvTRR+ewww7L97///dx000358z//8xx22GFZsGDB6v1WrlyZCy64IMcdd9w6x1q5cmUuvPDC7Lbbbrn66qsHVvOKFSsGNvbGcjpyghjUXdfdcR2ATXXllVdmm222yWmnnbZ63TOf+cx88IMfzHHHHZd3vvOdSZKrr74606dPz1Of+tT1jrXvvvvmuOOOy/z583PYYYclSX7+85/ntNNOyw9+8IMkycc+9rEcfPDBOf/88/P+978/VZX99tsvn/zkJ3PiiSfmZS97WY499tgkyWMf+9gsXbo0V111Vf7sz/4sT3rSk3L99dfnpptuylFHHZXbbrsty5Yty5ve9KaceuqpSZJLL70073jHO7JixYrssssu+dKXvpTf/u3fzte+9rVMmzYtK1euzDOe8Yxcc8012WWXXTbr708IAwA2yY033pgDDjjgEev322+/POpRj8q3v/3tzJo1KwsWLMjcuXPXO9b8+fMzd+7cHHnkkXnHO96Rhx56KNtss03e+MY35tBDD82FF16YFStWZOnSpVm8eHHe/e5356tf/Wp22WWX3HXXXRus9dprr82NN96YGTNmJEnOO++87LTTTnnggQcyZ86cHHPMMVm5cmVOOeWUXH311ZkxY0buuuuuPOpRj8qrX/3qfOpTn8rpp5+eyy+/PLNmzdrsAJY4HQkADMDcuXOzYMGCLF++PJ///Ofzyle+cp37Pvjgg7nkkkty1FFH5fGPf3wOOuig/Ou//muS5Iorrsgf//EfJ0mmTJmSHXbYIVdccUWOPfbY1UFop5122mA9Bx544OoAliQf+tCHMmvWrDz72c/Obbfdlu9973u55pprcsghh6zeb9W4J510Us4/f+iM1HnnnZfXvva1m/A38khmwgCATbLPPvvkggsuWOu2uXPn5oUvfGEOPfTQ7LfffvnN3/zNdY5z6aWX5u67787MmUO/7rz//vuz3Xbb5aUvfela92+tpeqRz8WeOnXq6ov6W2t58MEHV2/bfvvtV7++6qqrcvnll+frX/96tttuuxx22GFZtmzZOsfdbbfd8sQnPjFXXHFFvvGNb+RTn/rUOnvZGEIYAEwSW/qWEr/7u7+bd7zjHfmbv/mbnHLKKUmShQsX5v7778+hhx6anXfeOWeccUZOP/309Y4zf/78fPzjH199yvK+++7LjBkzcv/99+f5z39+Pvaxj+X000/PihUrct999+X5z39+jj766Lz5zW/OzjvvnLvuuis77bRTpk+fnm9+85v5/d///Xz+85/PQw89tNb3u/vuu7Pjjjtmu+22y3e/+91cc801SZLnPOc5ef3rX58f/vCHq09HrpoNO/nkk/PqV786J5xwQqZMmTImf39ORwIAm6SqcuGFF+ZLX/pSnv70p2efffbJWWedlSc/+clJhmbDvvvd7+boo49e5xj3339/LrvssofNem2//fZ53vOel3/+53/OBz/4wVx55ZWZOXNmDjjggCxevDj77LNP/uRP/iSHHnpoZs2albe85S1JklNOOSVf/vKXc+CBB+Yb3/jGw2a/RjriiCOyfPny7LfffvnTP/3TPPvZz06STJs2Leeee25e8YpXZNasWQ/7NefLX/7yLF26dMxORSZJtdbGbLAtYfbs2W3RokW9y9jiJvOvIwfVW6I/YHK7+eabs9dee/UuY6uwaNGivPnNb86///u/r3OfdXwejzy/OczpSACA9XjPe96Tj33sY2N2LdgqkyaETeaZIgCYDC677LK8/e1vf9i6GTNm5MILL+xU0eicccYZOeOMM8Z83EkTwgBga7SuX/SNRy960Yvyohe9qHcZA7Epl3e5MB8AJqhtt902d9555yYFAMZOay133nlntt122406zkwYAExQu+66a5YsWZI77rijdylbvW233Ta77rrrRh0jhAHABLXNNts87C7wTCxORwIAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdDC1dwHApvvJ2TMHMu7uZ35nIOMC8F+EMCY1IQWA8crpSACADsyEbcBkn0mZ7P0BwHhlJgwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgg4GGsKo6oqpuqapbq+qMtWw/rKrurqrrh/+cOch6AADGi6mDGriqpiT5SJIXJFmSZGFVfaG1dtMau/57a+1lg6oDAGA8GuRM2IFJbm2t/aC19mCSBUmOHOD7AQBMGIMMYU9JctuI5SXD69b0nKr6dlV9sar2WdtAVXVqVS2qqkV33HHHIGoFANiiBhnCai3r2hrL1yV5amttVpK/TnLR2gZqrZ3bWpvdWps9bdq0sa0SAKCDQYawJUl2G7G8a5LbR+7QWruntbZ0+PUlSbapql0GWBMAwLgwyBC2MMkeVTWjqh6d5PgkXxi5Q1X9VlXV8OsDh+u5c4A1AQCMCwP7dWRrbXlVzUtyWZIpSc5rrS2uqtOGt5+T5Ngkf1xVy5M8kOT41tqapywBACadgYWwZPUpxkvWWHfOiNcfTvLhQdYAADAeuWM+AEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB1N7FwAwnh3wtvMHMu433/eagYy7sSZ7fzCemQkDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoYGrvAoCJ7YC3nT+Qcb/5vtcMZFyA8cJMGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHAw1hVXVEVd1SVbdW1Rnr2W9OVa2oqmMHWQ8AwHgxsBBWVVOSfCTJi5PsnWRuVe29jv3em+SyQdUCADDeDHIm7MAkt7bWftBaezDJgiRHrmW/NyT5XJJfDLAWAIBxZZAh7ClJbhuxvGR43WpV9ZQkRyc5Z30DVdWpVbWoqhbdcccdY14oAMCWNsgQVmtZ19ZY/kCSt7fWVqxvoNbaua212a212dOmTRur+gAAupk6wLGXJNltxPKuSW5fY5/ZSRZUVZLskuQlVbW8tXbRAOsCAOhukCFsYZI9qmpGkp8mOT7JH4zcobU2Y9XrqvpEkosFMABgazCwENZaW15V8zL0q8cpSc5rrS2uqtOGt6/3OjAAgMlskDNhaa1dkuSSNdatNXy11k4cZC0AAOOJO+YDAHQghAEAdDDQ05EAbJ1+cvbMgYy7+5nfGci40IOZMACADsyEAXRgpggwEwYA0IGZMGBcGtRMUWK2CBgfzIQBAHQghAEAdCCEAQB0IIQBAHTgwnwAmIAOeNv5Axv7m+97zcDG5r+YCQMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6MB9wgCYlNxHi/HOTBgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHow5hVbX9IAsBANiabDCEVdXBVXVTkpuHl2dV1UcHXhkAwCQ2mpmwv0ryoiR3Jklr7dtJDhlkUQAAk92oTke21m5bY9WKAdQCALDVmDqKfW6rqoOTtKp6dJI3ZvjUJAAAm2Y0M2GnJXl9kqckWZLkmcPLAABsog3OhLXWfpnkVVugFgCArcYGQ1hV/V2Stub61tpJA6kIAGArMJprwi4e8XrbJEcnuX0w5QAAbB1GczrycyOXq2p+kssHVhEAwFZgUx5btEeS3ce6EACArclorgm7N0PXhNXw//1ZkrcPuC4AgEltNKcjH7clCgEA2JqsM4RV1f7rO7C1dt3YlwMAsHVY30zYX65nW0vyu2NcCwDAVmOdIay19jtbshAAgK3JaO4TlqraN8neGbpPWJKktXb+oIoCAJjsNniLiqp6Z5K/Hv7zO0n+IsnLRzN4VR1RVbdU1a1VdcZath9ZVTdU1fVVtaiqnreR9QMATEijuU/YsUmen+RnrbXXJpmV5Dc2dFBVTUnykSQvztAs2tyq2nuN3f4tyazW2jOTnJTk46MvHQBg4hpNCFvWWluZZHlVPT7JL5I8bRTHHZjk1tbaD1prDyZZkOTIkTu01pa21lY9l3L7rOUZlQAAk9E6Q1hVfbiqnpvk2qp6QpK/SfLNJNcluXYUYz8lyW0jlpcMr1vzfY6uqu8m+ZcMzYatrZZTh09XLrrjjjtG8dYAAOPb+i7M/16S9yd5cpKlSeYneUGSx7fWbhjF2LWWdY+Y6WqtXZjkwqo6JMm7khy+ln3OTXJuksyePdtsGQAw4a1zJqy19sHW2nOSHJLkriR/l+SLSY6qqj1GMfaSJLuNWN41ye3reb+rkzy9qnYZTeEAABPZBq8Ja639uLX23tbas5L8QZKjk3x3FGMvTLJHVc2oqkcnOT7JF0buUFX/rapq+PX+SR6d5M6N7AEAYMIZzQO8t0lyRIZC1POTfDnJn23ouNba8qqal+SyJFOSnNdaW1xVpw1vPyfJMUleU1UPJXkgyXEjLtQHAJi01vfsyBckmZvkpRm6EH9BklNba/eNdvDW2iVJLllj3TkjXr83yXs3smYAgAlvfTNh70jy6SRvba3dtYXqAQDYKnh2JABAB6O5WSsAAGNMCAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhgau8CAIDx5SdnzxzIuLuf+Z2BjDtRmQkDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOjAfcIAYCO5jxZjwUwYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABABwMNYVV1RFXdUlW3VtUZa9n+qqq6YfjP16pq1iDrAQAYLwYWwqpqSpKPJHlxkr2TzK2qvdfY7YdJDm2t7ZfkXUnOHVQ9AADjySBnwg5Mcmtr7QettQeTLEhy5MgdWmtfa6395/DiNUl2HWA9AADjxiBD2FOS3DZiecnwunX5wyRfXNuGqjq1qhZV1aI77rhjDEsEAOhjkCGs1rKurXXHqt/JUAh7+9q2t9bOba3Nbq3NnjZt2hiWCADQx9QBjr0kyW4jlndNcvuaO1XVfkk+nuTFrbU7B1gPAMC4MciZsIVJ9qiqGVX16CTHJ/nCyB2qavck/5TkhNbafwywFgCAcWVgM2GtteVVNS/JZUmmJDmvtba4qk4b3n5OkjOT7Jzko1WVJMtba7MHVRMAwHgxyNORaa1dkuSSNdadM+L1yUlOHmQNAADjkTvmAwB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdDC1dwEAAGs64G3nD2Tcb77vNQMZd1OYCQMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6GBq7wIAALaUn5w9c2Bj737mdzZqfzNhAAAdCGEAAB0IYQAAHQhhAAAdCGEAAB0IYQAAHQhhAAAdCGEAAB0IYQAAHQhhAAAdCGEAAB0IYQAAHQhhAAAdCGEAAB0IYQAAHQhhAAAdCGEAAB0IYQAAHQhhAAAdCGEAAB0IYQAAHQhhAAAdDDSEVdURVXVLVd1aVWesZfueVfX1qvp1Vb11kLUAAIwnUwc1cFVNSfKRJC9IsiTJwqr6QmvtphG73ZXkjUmOGlQdAADj0SBnwg5Mcmtr7QettQeTLEhy5MgdWmu/aK0tTPLQAOsAABh3BhnCnpLkthHLS4bXbbSqOrWqFlXVojvuuGNMigMA6GmQIazWsq5tykCttXNba7Nba7OnTZu2mWUBAPQ3yBC2JMluI5Z3TXL7AN8PAGDCGGQIW5hkj6qaUVWPTnJ8ki8M8P0AACaMgf06srW2vKrmJbksyZQk57XWFlfVacPbz6mq30qyKMnjk6ysqtOT7N1au2dQdQEAjAcDC2FJ0lq7JMkla6w7Z8Trn2XoNCUAwFbFHfMBADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoYaAirqiOq6paqurWqzljL9qqqDw1vv6Gq9h9kPQAA48XAQlhVTUnykSQvTrJ3krlVtfcau704yR7Df05N8rFB1QMAMJ4McibswCS3ttZ+0Fp7MMmCJEeusc+RSc5vQ65J8oSqetIAawIAGBeqtTaYgauOTXJEa+3k4eUTkhzUWps3Yp+Lk7yntfaV4eV/S/L21tqiNcY6NUMzZUny20luGUjRa7dLkl9uwffb0vQ3sU3m/iZzb4n+Jjr9TVxburdfttaOWNuGqQN801rLujUT32j2SWvt3CTnjkVRG6uqFrXWZvd47y1BfxPbZO5vMveW6G+i09/ENZ56G+TpyCVJdhuxvGuS2zdhHwCASWeQIWxhkj2qakZVPTrJ8Um+sMY+X0jymuFfST47yd2ttf83wJoAAMaFgZ2ObK0tr6p5SS5LMiXJea21xVV12vD2c5JckuQlSW5Ncn+S1w6qns3Q5TToFqS/iW0y9zeZe0v0N9Hpb+IaN70N7MJ8AADWzR3zAQA6EMIAADrY6kJYVT2mqr48fEf/VNWlVfWr4XuWjeb4Q6rquqpaPnwvtFXrp1XVpYOqe7TGoL+3VNVNw4+R+reqeurw+u79jUFvp1XVd6rq+qr6yqonOIyH3obr2Kz+RoxzbFW1qpo9vDzu+quqZ1bV16tq8fB37bhRHD9uv5vDdWxufxPm+7kp/Y0YZ9x9P9fo7alV9c3hz2H1dcwbOH4ifXYb3d+IccbdZzdcx8P+7Rxe9/iq+mlVfXgjxtni/W11ISzJSUn+qbW2Ynj5fUlO2Ijjf5LkxCSfHrmytXZHkv9XVc8diyI3w+b2960ks1tr+yW5IMlfJOOmv83t7dOttZmttWdmqK//k4yb3pLN7y9V9bgkb0zyjVXrxml/9yd5TWttnyRHJPlAVT1hA8eP5+9msvn9TaTv56b0N56/nyN7+39JDh7+HA5KckZVPXkDx0+kz25T+hvPn13yyH87k+RdSb482gF69bc1hrBXJfn8qoXW2r8luXe0B7fWftRauyHJyrVsvmh4/J42t78rW2v3Dy9ek6F7t61yUfr2t7m93TNicfs8/MbAF2WCf3bD3pWh/wgsW2P9RRlH/bXW/qO19r3h17cn+UWSaes7eJx/N5PN72/CfD83pb9h4/X7ObK3B1trvx5e/xsZxX8nJ9hnt9H9DRuvn12yxr+dVXVAkicm+deNGKNLf1tVCKuh+5U9rbX2owG9xaIk/31AY2/QAPr7wyRfHLHcrb+x6q2qXl9V38/Q/9jeOGLThP/squpZSXZrra3t9OW47a+qDkzy6CTf34ghx813Mxm7/ibi93O0/Y3X7+faequq3arqhiS3JXnvcNDc0DgT5rPb2P7G62eXPLK/qnpUkr9M8raNGKNbf1tVCMvQ86J+NcDxf5Fkg9O6AzRm/VXVq5PMztApsVV69jcmvbXWPtJae3qStyf5nyM2TejPbvgfnr9K8v+tY5dx2V9VPSnJJ5O8trW2ttnlRxiH381kjPqbaN/P0fY3zr+fj+ittXbb8Gnv/5bkf1TVEzc0yET67Damv3H+2SWP7O91SS5prd02moN797e1hbAHkmw7wPG3HX6PXsakv6o6PMmfJHn5iGnrpG9/Y/3ZLUhy1Ijlif7ZPS7JvkmuqqofJXl2ki+susA047C/qnp8kn9J8j9ba9eMZpBx+t1Mxqi/Ecb993Mj+xvP3891/m9veIZocTZuJmTcf3arjLK/8fzZJY/s7zlJ5g3X+v4MPZXnPes5vmt/W1UIa639Z5IpVbXB/9hV1f+uqqM38i2ekeTGTSpuDIxFf8PTsv83Q/+R+8Uam7v1N0a97TFi8aVJvjdieUJ/dq21u1tru7TWprfWpmfomqmXt9YWDe8yrvobPoVwYZLzW2ufHbnvRPtuJmPW34T5fm5sf+P5+7mW3natqscMv94xyXOT3DK8PBk+u43qbzx/dsP1Pay/1tqrWmu7D9f61gx9R89Ixmd/W1UIG/avSZ63aqGq/j3JZ5M8v6qWVNWLhjfNTPKzNQ+uqjlVtSTJK5P836paPGLz72To/zPsabP6y9Apnscm+WwN/YR55PM+e/e3ub3Nq6GfZF+f5C1J/seIbb17Sza/v/UZb/39fpJDkpw4/D27vqqeObxtIn43k83vbyJ9Pzelv/Xp3d/I3vZK8o2q+naGfl33/tbad4a3TYbPblP6W5/x1t/6jL/+Wmtb1Z8kz0ryyVHsd9kmjH11kh31pzf96U9/E6e/ydyb/sZ3f1vdTFhr7VtJrqwRN3Vbx34vWt/2NVXVtCT/pw1NjXYzmfubzL0l+huxn/5G0N/gTebeEv2N2G/c9ecB3gAAHWx1M2EAAOOBEAYA0IEQBgDQgRAGTApV1arqL0csv7WqzupYEsB6CWHAZPHrJK+oql025eAN/bIKYKwJYcBksTzJuUnevOaGqnp6VV1TVQur6uyqWjq8/rCqurKqPp3kO8PrLqqqbw7ffPPUEWMsrar3Dm+7vKoOrKqrquoHVfXyLdQjMIkIYcBk8pEkr6qqHdZY/8EkH2ytzUly+xrbDkzyJ621vYeXT2qtHZChh4S/sap2Hl6/fZKrhrfdm+R/JXlBkqOTnD32rQCTnRAGTBqttXuSnJ/kjWtsek6GHgGVJJ9eY9u1rbUfjlh+4/AjXa5JsluSVc8FfDDJpcOvv5Pky621h4ZfTx+TBoCtihAGTDYfSPKHGZq5Go37Vr2oqsOSHJ7kOa21WUm+lWTVQ9Ufav91d+uVGboGLa21lUmmbm7RwNZHCAMmldbaXUn+MUNBbJVrkhwz/Pr49Ry+Q5L/bK3dX1V7Jnn2YKoEEMKAyekvk4z8leTpSd5SVdcmeVKSu9dx3KVJplbVDUnelaHwBjAQnh0JTHpVtV2SB1prraqOTzK3tXZk77qArZvrGICtwQFJPlxVleRXSU7qWw6AmTAAgC5cEwYA0IEQBgDQgRAGANCBEAYA0IEQBgDQwf8PPihlOIIlZWEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmEAAAJcCAYAAACxEXM4AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAqGUlEQVR4nO3de7heZXkv6t9jgo2gIKd6qUATLZZTEoQEFF2AhSrWVkBsISIspEBRUdGtW7a2lJ0uu23VFq0HFm1ZlhYTKy2gbARLAa0HIAEViEjFE0SqclhyjpDkXX/MmXQmJCEJ88s758x9X1cu5zi93/PM7yP5+Y7xjVGttQAAsGk9rXcBAACbIyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDCBJVR1cVYt71wFsPoQwYMKpqjdU1cKqeqiq/rOqvlhVL+9dF8BIQhgwoVTVu5KcneTPkjwnyS5JPpnk8AG+5qRBjQ1MXEIYMGFU1TZJ5iZ5a2vtX1prD7fWHm+tfaG19p6q+pWqOruq7hr+c3ZV/cpaxtq9qq6pql9U1aKqeu2IbZ+uqk9V1WVV9XCSV2yiFoEJRAgDJpKXJpmS5KK1bH9/kpck2TvJzCT7Jfmj1Xeqqi2SfCHJl5L8apK3Jbmgqn5jxG5vSPKBJM9K8tXRKR/YnAhhwESyfZJ7WmtL17L92CRzW2s/b63dneT/TXLcGvZ7SZJnJvlga+2x1tpVSS5NMmfEPpe01r7WWlveWlsyij0AmwkhDJhI7k2yQ1VNXsv25yX58YjlHw+vW9N+d7bWlq+27/NHLN/5VAoFEMKAieQbSZYkOWIt2+9K8msjlncZXrem/Xauqqettu9PRiy3jS8TIFnb/1sEGHdaa/dX1ZlJPlFVSzN0TdfjSQ7N0MXz85L8UVUtyFCIOjPJP65hqOuSPJzk/66qjyR5WZLfTTJ78F0AmwshDJhQWmt/WVU/y9AF9xckeTDJDRm6iP7GJFsnuWl4988l+R9rGOOx4W9DfjLJ/5OhGbDjW2vfHXwHwOaiWjOjDgCwqbkmDACgAyEMAKADIQwAoAMhDACgg3H37cjDDjusXX755b3LAABYH7W2DeNuJuyee+7pXQIAwFM27kIYAMBEIIQBAHQghAEAdDDuLswHAIY8/vjjWbx4cZYsWdK7lM3elClTstNOO2WLLbZY72OEMAAYpxYvXpxnPetZmTp1aqrW+iU8Bqy1lnvvvTeLFy/OtGnT1vs4pyMBYJxasmRJtt9+ewGss6rK9ttvv8EzkkIYAIxjAtjYsDHvgxAGANCBEAYAbJCDDz44V1xxxSrrzj777LzlLW9Zr+PPPPPMXHnllU/6GgsXLnzC+k9/+tM57bTT1r/YMUwIAwA2yJw5czJ//vxV1s2fPz9z5sx50mOXLVuWuXPn5tBDDx1UeeOGEAYAbJDXv/71ufTSS/PLX/4ySfKjH/0od911Vz7zmc9k1qxZ2XPPPfMnf/InK/efOnVq5s6dm5e//OX53Oc+lxNOOCEXXnhhkmTu3LmZPXt29tprr5xyyilpra087h//8R9zwAEHZK+99sr111//hDruvvvuHHXUUZk9e3Zmz56dr33tawPufHQJYQDABtl+++2z33775fLLL08yNAt29NFH5wMf+EAWLlyYm266KV/+8pdz0003rTxmypQp+epXv5pjjjlmlbFOO+20LFiwILfcckseffTRXHrppSu3Pfzww/n617+eT37ykznxxBOfUMc73vGOvPOd78yCBQvyz//8zznppJMG1PFgCGEAwAYbeUpyxanIf/qnf8o+++yTF7/4xVm0aFG+853vrNz/6KOPXuM4V199dfbff/9Mnz49V111VRYtWrTKayTJgQcemAceeCC/+MUvVjn2yiuvzGmnnZa99947r33ta/PAAw/kwQcfHOVOB8fNWgGADXbEEUfkXe96V2688cY8+uij2XbbbfPhD384CxYsyLbbbpsTTjhhlftmbbXVVk8YY8mSJXnLW96ShQsXZuedd85ZZ521yjGr3/Zh9eXly5fnG9/4Rp7xjGeMcnebhpkwAGCDPfOZz8zBBx+cE088MXPmzMkDDzyQrbbaKttss01+9rOf5Ytf/OKTjrEicO2www556KGHVl4ntsJnP/vZJMlXv/rVbLPNNtlmm21W2f7KV74yH//4x1cuf+tb33qKXW1aZsIAgI0yZ86cvO51r8v8+fOz22675cUvfnH23HPPvOAFL8jLXvayJz3+2c9+dk4++eRMnz49U6dOzezZs1fZvu222+aAAw7IAw88kPPOO+8Jx3/sYx/LW9/61syYMSNLly7NgQcemHPOOWfU+hu0GvkthPFg1qxZbU33DQGAzc2tt96a3XffvXcZDFvL+7HWW+k7HQkA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCB+4QBwASx73vOH9XxbvjQ8eu130UXXZTXve51ufXWW7PbbruNag0T2YQJYaP9wVthfT+AALC5mjdvXl7+8pdn/vz5OeusswbyGsuWLcukSZMGMnYvTkcCABvtoYceyte+9rX83d/93coHei9btizvfve7M3369MyYMSN//dd/nSRZsGBBDjjggMycOTP77bdfHnzwwXz605/OaaedtnK83/md38k111yTZOjRSGeeeWb233//fOMb38jcuXMze/bs7LXXXjnllFOy4obzt99+ew499NDMnDkz++yzT77//e/nuOOOyyWXXLJy3GOPPTaf//znN9FvZf1MmJkwAGDTu/jii3PYYYflRS96UbbbbrvceOONue666/LDH/4w3/zmNzN58uTcd999eeyxx3L00Ufns5/9bGbPnp0HHnjgSR+8/fDDD2evvfbK3LlzkyR77LFHzjzzzCTJcccdl0svvTS/+7u/m2OPPTZnnHFGjjzyyCxZsiTLly/PSSedlL/6q7/K4Ycfnvvvvz9f//rX8/d///cD/31sCDNhAMBGmzdvXo455pgkyTHHHJN58+blyiuvzKmnnprJk4fmerbbbrvcdtttee5zn7vy+ZBbb731yu1rM2nSpBx11FErl6+++ursv//+mT59eq666qosWrQoDz74YH7yk5/kyCOPTJJMmTIlW265ZQ466KDcfvvt+fnPf5558+blqKOOetLX29TGVjUAwLhx77335qqrrsott9ySqsqyZctSVdl3331TteojE1trT1iXJJMnT87y5ctXLi9ZsmTlz1OmTFl5HdiSJUvylre8JQsXLszOO++cs846K0uWLMm6noF93HHH5YILLsj8+fPX+ADw3syEAQAb5cILL8zxxx+fH//4x/nRj36UO++8M9OmTcs+++yTc845J0uXLk2S3Hfffdltt91y1113ZcGCBUmSBx98MEuXLs3UqVPzrW99K8uXL8+dd96Z66+/fo2vtSKc7bDDDnnooYdy4YUXJhmaUdtpp51y8cUXJ0l++ctf5pFHHkmSnHDCCTn77LOTJHvuueegfg0bzUwYAEwQm/ob/fPmzcsZZ5yxyrqjjjoqt956a3bZZZfMmDEjW2yxRU4++eScdtpp+exnP5u3ve1tefTRR/OMZzwjV155ZV72spdl2rRpmT59evbaa6/ss88+a3ytZz/72Tn55JMzffr0TJ06deVpzST5h3/4h/zhH/5hzjzzzGyxxRb53Oc+lxe84AV5znOek9133z1HHHHEIH8NG63WNY03Fs2aNastXLjwCevdogKAzc2tt96a3XffvXcZY9YjjzyS6dOn58Ybb8w222wz8Ndby/vxxHOww5yOBAAmnCuvvDK77bZb3va2t22SALYxnI4EACacQw89NHfccUfvMtbJTBgAQAdmwmDABnW9YuKaRYDxzEwYAEAHQhgAQAdORwLABHHH3OmjOt4uZ948quOxKjNhAMBG++lPf5pjjjkmL3zhC7PHHnvkt3/7t1NVue2221bZ7/TTT89f/MVfrHOsd7zjHXn+85+/ymOMJjIhDADYKK21HHnkkTn44IPz/e9/P9/5znfyZ3/2Zzn44IMzf/78lfstX748F154YY4++ui1jrV8+fJcdNFF2XnnnfOVr3xlYDUvW7ZsYGNvKCEMANgoV199dbbYYouceuqpK9ftvffe+ehHP7pKCPvKV76SqVOn5td+7dfWOdZee+2VN7/5zZk3b97K9T/72c9y5JFHZubMmZk5c2a+/vWvJ0nOP//8zJgxIzNnzsxxxx2XZOhZkSueKZkkz3zmM5Mk11xzTV7xilfkDW94Q6ZPHzple8QRR2TffffNnnvumXPPPXflMZdffnn22WefzJw5M4ccckiWL1+eXXfdNXfffXeSobD467/+67nnnns2+ve2gmvCAICNcsstt2Tfffd9wvoZM2bkaU97Wr797W9n5syZmT9/fubMmbPOsebNm5c5c+bk8MMPz/ve9748/vjj2WKLLfL2t789Bx10UC666KIsW7YsDz30UBYtWpQPfOAD+drXvpYddtgh991335PWev311+eWW27JtGnTkiTnnXdetttuuzz66KOZPXt2jjrqqCxfvjwnn3xyvvKVr2TatGm577778rSnPS1vfOMbc8EFF+T000/PlVdemZkzZ2aHHXbYuF/aCGbCAIBRN2fOnMyfPz9Lly7NJZdckt/7vd9b676PPfZYLrvsshxxxBHZeuuts//+++dLX/pSkuSqq67Km9/85iTJpEmTss022+Sqq67K61//+pVBaLvttnvSevbbb7+VASxJPvaxj2XmzJl5yUtekjvvvDPf+973cu211+bAAw9cud+KcU888cScf/7QPR/PO++8vOlNb9qI38gTmQkDADbKnnvuucrpv5HmzJmTV77ylTnooIMyY8aM/Oqv/upax7n88stz//33rzxV+Mgjj2TLLbfMa17zmjXu31pL1ROfiz158uSVF/W31vLYY4+t3LbVVlut/Pmaa67JlVdemW984xvZcsstc/DBB2fJkiVrHXfnnXfOc57znFx11VW57rrrcsEFF6y1lw0hhAHABLGpbynxm7/5m3nf+96Xv/mbv8nJJ5+cJFmwYEEeeeSRHHTQQdl+++1zxhln5PTTT1/nOPPmzcvf/u3frjxl+fDDD2fatGl55JFHcsghh+RTn/pUTj/99CxbtiwPP/xwDjnkkBx55JF55zvfme233z733Xdftttuu0ydOjU33HBDfv/3fz+XXHJJHn/88TW+3v33359tt902W265Zb773e/m2muvTZK89KUvzVvf+tb88Ic/XHk6csVs2EknnZQ3vvGNOe644zJp0qRR+f05HQkAbJSqykUXXZR//dd/zQtf+MLsueeeOeuss/K85z0vydBs2He/+90ceeSRax3jkUceyRVXXLHKrNdWW22Vl7/85fnCF76Qj370o7n66qszffr07Lvvvlm0aFH23HPPvP/9789BBx2UmTNn5l3veleS5OSTT86Xv/zl7LfffrnuuutWmf0a6bDDDsvSpUszY8aM/PEf/3Fe8pKXJEl23HHHnHvuuXnd616XmTNnrvJtzte+9rV56KGHRu1UZJJUa23UBtsUZs2a1RYuXPiE9YN6Pp9n8/FUeXYkMCi33nprdt99995lbBYWLlyYd77znfn3f//3te6zlvfjiec3hzkdCQCwDh/84AfzqU99atSuBVtBCAMANokrrrgi733ve1dZN23atFx00UWdKlo/Z5xxRs4444xRH1cIA4BxbG3f6BuLXvWqV+VVr3pV7zIGYmMu73JhPgCMU1OmTMm99967UQGA0dNay7333pspU6Zs0HFmwgBgnNppp52yePHilY/UoZ8pU6Zkp5122qBjhDAAGKe22GKLVe4Cz/jidCQAQAdCGABAB0IYAEAHQhgAQAcuzKc7j/UBYHNkJgwAoAMhDACgAyEMAKADIQwAoAMX5o8Tg7p43YXrANCHmTAAgA6EMACADgYawqrqsKq6rapur6oz1rB9m6r6QlV9u6oWVdWbBlkPAMBYMbAQVlWTknwiyauT7JFkTlXtsdpub03yndbazCQHJ/lIVT19UDUBAIwVg5wJ2y/J7a21H7TWHksyP8nhq+3TkjyrqirJM5Pcl2TpAGsCABgTBhnCnp/kzhHLi4fXjfTxJLsnuSvJzUne0VpbvvpAVXVKVS2sqoV33333oOoFANhkBhnCag3r2mrLr0ryrSTPS7J3ko9X1dZPOKi1c1trs1prs3bcccfRrhMAYJMbZAhbnGTnEcs7ZWjGa6Q3JfmXNuT2JD9MstsAawIAGBMGGcIWJNm1qqYNX2x/TJLPr7bPHUkOSZKqek6S30jygwHWBAAwJgzsjvmttaVVdVqSK5JMSnJea21RVZ06vP2cJH+a5NNVdXOGTl++t7V2z6BqAgAYKwb62KLW2mVJLltt3Tkjfr4rySsHWQMAwFjkjvkAAB0IYQAAHQhhAAAdCGEAAB0M9ML8ieCOudMHMu4uZ948kHEBgPHBTBgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHk3sXAGy8O+ZOH8i4u5x580DGBeC/mAkDAOjATNhmzkwKAPRhJgwAoAMhDACgAyEMAKADIQwAoAMX5jOh+eIBAGOVmTAAgA6EMACADoQwAIAOhDAAgA6EMACADoQwAIAOhDAAgA6EMACADoQwAIAOhDAAgA6EMACADoQwAIAOhDAAgA6EMACADoQwAIAOhDAAgA6EMACADoQwAIAOhDAAgA6EMACADoQwAIAOhDAAgA6EMACADoQwAIAOhDAAgA6EMACADoQwAIAOhDAAgA6EMACADoQwAIAOhDAAgA6EMACADoQwAIAOhDAAgA6EMACADoQwAIAOhDAAgA6EMACADoQwAIAOhDAAgA6EMACADoQwAIAOhDAAgA6EMACADoQwAIAOhDAAgA6EMACADoQwAIAOhDAAgA6EMACADib3LgBgLNv3PecPZNwbPnT8QMYFxg8zYQAAHQhhAAAdCGEAAB0IYQAAHQhhAAAdCGEAAB0IYQAAHQhhAAAdCGEAAB0IYQAAHQhhAAAdCGEAAB0IYQAAHQhhAAAdCGEAAB0IYQAAHUzuXQAA/ez7nvMHMu4NHzp+IOPCRGImDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgg4GGsKo6rKpuq6rbq+qMtexzcFV9q6oWVdWXB1kPAMBYMbCbtVbVpCSfSPJbSRYnWVBVn2+tfWfEPs9O8skkh7XW7qiqXx1UPQAAY8kgZ8L2S3J7a+0HrbXHksxPcvhq+7whyb+01u5IktbazwdYDwDAmDHIxxY9P8mdI5YXJ9l/tX1elGSLqromybOSfLS19oRnaFTVKUlOSZJddtllIMUCG8djbwA2ziBnwmoN69pqy5OT7JvkNUleleSPq+pFTziotXNba7Naa7N23HHH0a8UAGATG+RM2OIkO49Y3inJXWvY557W2sNJHq6qrySZmeQ/BlgXAEB3g5wJW5Bk16qaVlVPT3JMks+vts8lSf5bVU2uqi0zdLry1gHWBAAwJgxsJqy1trSqTktyRZJJSc5rrS2qqlOHt5/TWru1qi5PclOS5Un+trV2y6BqAgAYKwZ5OjKttcuSXLbaunNWW/5Qkg8Nsg4AgLHGHfMBADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoY6H3CADbWHXOnD2zsXc68eWBjA6wvM2EAAB0IYQAAHQhhAAAdCGEAAB0IYQAAHQhhAAAdCGEAAB0IYQAAHQhhAAAdCGEAAB0IYQAAHQhhAAAdeIA3QAeDekC5h5PD+GEmDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKCDyb0LAGDiuWPu9IGMu8uZNw9kXOjBTBgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB5N7FwAAg7Dve84f2Ng3fOj4gY3N5sNMGABAB0IYAEAHQhgAQAfrHcKqaqtBFgIAsDl50hBWVQdU1XeS3Dq8PLOqPjnwygAAJrD1mQn7qySvSnJvkrTWvp3kwEEWBQAw0a3X6cjW2p2rrVo2gFoAADYb63OfsDur6oAkraqenuTtGT41CQDAxlmfmbBTk7w1yfOTLE6y9/AyAAAb6Ulnwlpr9yQ5dhPUAgCw2XjSEFZV/ytJW319a+3EgVQEALAZWJ9rwi4d8fOUJEcmuWsw5QAAbB7W53TkP49crqp5Sa4cWEUAAJuBjXls0a5JdhntQgAANifrc03Ygxm6JqyG//enSd474LoAACa09Tkd+axNUQgAwOZkrSGsqvZZ14GttRtHvxwAgM3DumbCPrKObS3Jb45yLQAAm421hrDW2is2ZSEAAJuT9blPWKpqryR7ZOg+YUmS1tr5gyoKAGCiW59vR/5JkoMzFMIuS/LqJF9NIoQBAGyk9blP2OuTHJLkp621NyWZmeRXBloVAMAEtz4hbElrbXmSpVW1dZKfJ3nBYMsCAJjY1nWLio8nmZfk+qp6dpK/SXJDkoeSXL9JqgMAmKDWdU3Y95J8OMnzMhS85iX5rSRbt9Zu2gS1AQBMWGs9Hdla+2hr7aVJDkxyX5L/leSLSY6oql03UX0AABPSk14T1lr7cWvtz1trL07yhiRHJvnuwCsDAJjAnjSEVdUWVfW7VXVBhmbC/iPJUQOvDABgAlvXhfm/lWROktdk6EL8+UlOaa09vIlqAwCYsNZ1Yf77knwmybtba/dtonoAADYLnh0JANDB+tysFQCAUSaEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQw0BBWVYdV1W1VdXtVnbGO/WZX1bKqev0g6wEAGCsGFsKqalKSTyR5dZI9ksypqj3Wst+fJ7liULUAAIw1g5wJ2y/J7a21H7TWHksyP8nha9jvbUn+OcnPB1gLAMCYMnmAYz8/yZ0jlhcn2X/kDlX1/CRHJvnNJLPXNlBVnZLklCTZZZddRr1QABhv9n3P+QMb+4YPHT+wsfkvg5wJqzWsa6stn53kva21ZesaqLV2bmttVmtt1o477jha9QEAdDPImbDFSXYesbxTkrtW22dWkvlVlSQ7JPntqlraWrt4gHUBAHQ3yBC2IMmuVTUtyU+SHJPkDSN3aK1NW/FzVX06yaUCGACwORhYCGutLa2q0zL0rcdJSc5rrS2qqlOHt58zqNcGABjrBjkTltbaZUkuW23dGsNXa+2EQdYCADCWuGM+AEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAeTexcAAOPNHXOnD2TcXc68eSDjMjaZCQMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhgcu8CAABWt+97zh/IuDd86PiBjLsxzIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdDC5dwEAwNhyx9zpAxl3lzNvHsi445WZMACADoQwAIAOhDAAgA6EMACADoQwAIAOhDAAgA6EMACADoQwAIAOhDAAgA6EMACADoQwAIAOhDAAgA6EMACADoQwAIAOhDAAgA6EMACADoQwAIAOhDAAgA6EMACADoQwAIAOhDAAgA6EMACADib3LgAAYFO5Y+70gY29y5k3b9D+ZsIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhhoCKuqw6rqtqq6varOWMP2Y6vqpuE/X6+qmYOsBwBgrBhYCKuqSUk+keTVSfZIMqeq9lhttx8mOai1NiPJnyY5d1D1AACMJYOcCdsvye2ttR+01h5LMj/J4SN3aK19vbX2v4cXr02y0wDrAQAYMwYZwp6f5M4Ry4uH163NHyT54po2VNUpVbWwqhbefffdo1giAEAfgwxhtYZ1bY07Vr0iQyHsvWva3lo7t7U2q7U2a8cddxzFEgEA+pg8wLEXJ9l5xPJOSe5afaeqmpHkb5O8urV27wDrAQAYMwY5E7Ygya5VNa2qnp7kmCSfH7lDVe2S5F+SHNda+48B1gIAMKYMbCastba0qk5LckWSSUnOa60tqqpTh7efk+TMJNsn+WRVJcnS1tqsQdUEADBWDPJ0ZFprlyW5bLV154z4+aQkJw2yBgCAscgd8wEAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhhoCKuqw6rqtqq6varOWMP2qqqPDW+/qar2GWQ9AABjxcBCWFVNSvKJJK9OskeSOVW1x2q7vTrJrsN/TknyqUHVAwAwlgxyJmy/JLe31n7QWnssyfwkh6+2z+FJzm9Drk3y7Kp67gBrAgAYE6q1NpiBq16f5LDW2knDy8cl2b+1dtqIfS5N8sHW2leHl/8tyXtbawtXG+uUDM2UJclvJLltIEWv2Q5J7tmEr7ep6W98m8j9TeTeEv2Nd/obvzZ1b/e01g5b04bJA3zRWsO61RPf+uyT1tq5Sc4djaI2VFUtbK3N6vHam4L+xreJ3N9E7i3R33inv/FrLPU2yNORi5PsPGJ5pyR3bcQ+AAATziBD2IIku1bVtKp6epJjknx+tX0+n+T44W9JviTJ/a21/xxgTQAAY8LATke21pZW1WlJrkgyKcl5rbVFVXXq8PZzklyW5LeT3J7kkSRvGlQ9T0GX06CbkP7Gt4nc30TuLdHfeKe/8WvM9DawC/MBAFg7d8wHAOhACAMA6GCzC2FV9Yyq+vLwHf1TVZdX1S+G71m2PscfWFU3VtXS4XuhrVi/Y1VdPqi619co9PeuqvrO8GOk/q2qfm14fff+RqG3U6vq5qr6VlV9dcUTHMZCb8N1PKX+Rozz+qpqVTVreHnM9VdVe1fVN6pq0fBn7ej1OH7MfjaH63iq/Y2bz+fG9DdinDH3+Vytt1+rqhuG34eV1zE/yfHj6b3b4P5GjDPm3rvhOlb5u3N43dZV9ZOq+vgGjLPJ+9vsQliSE5P8S2tt2fDyh5IctwHH35HkhCSfGbmytXZ3kv+sqpeNRpFPwVPt75tJZrXWZiS5MMlfJGOmv6fa22daa9Nba3tnqK+/TMZMb8lT7y9V9awkb09y3Yp1Y7S/R5Ic31rbM8lhSc6uqmc/yfFj+bOZPPX+xtPnc2P6G8ufz5G9/WeSA4bfh/2TnFFVz3uS48fTe7cx/Y3l9y554t+dSfKnSb68vgP06m9zDGHHJrlkxUJr7d+SPLi+B7fWftRauynJ8jVsvnh4/J6ean9Xt9YeGV68NkP3blvh4vTt76n29sCIxa2y6o2BL844f++G/WmG/hFYstr6izOG+mut/Udr7XvDP9+V5OdJdlzXwWP8s5k89f7GzedzY/obNlY/nyN7e6y19svh9b+S9fh3cpy9dxvc37Cx+t4lq/3dWVX7JnlOki9twBhd+tusQlgN3a/sBa21Hw3oJRYm+W8DGvtJDaC/P0jyxRHL3fobrd6q6q1V9f0M/cf29hGbxv17V1UvTrJza21Npy/HbH9VtV+Spyf5/gYMOWY+m8no9TceP5/r299Y/Xyuqbeq2rmqbkpyZ5I/Hw6aTzbOuHnvNrS/sfreJU/sr6qeluQjSd6zAWN062+zCmEZel7ULwY4/s+TPOm07gCNWn9V9cYkszJ0SmyFnv2NSm+ttU+01l6Y5L1J/mjEpnH93g3/xfNXSf6vtewyJvurqucm+Yckb2qtrWl2+QnG4GczGaX+xtvnc337G+Ofzyf01lq7c/i0968n+e9V9ZwnG2Q8vXcb0t8Yf++SJ/b3liSXtdbuXJ+De/e3uYWwR5NMGeD4U4Zfo5dR6a+qDk3y/iSvHTFtnfTtb7Tfu/lJjhixPN7fu2cl2SvJNVX1oyQvSfL5FReYZgz2V1VbJ/n/k/xRa+3a9RlkjH42k1Hqb4Qx//ncwP7G8udzrf/tDc8QLcqGzYSM+fduhfXsbyy/d8kT+3tpktOGa/1whp7K88F1HN+1v80qhLXW/neSSVX1pP/YVdX/V1VHbuBLvCjJLRtV3CgYjf6Gp2X/Z4b+kfv5apu79TdKve06YvE1Sb43Ynlcv3ettftbazu01qa21qZm6Jqp17bWFg7vMqb6Gz6FcFGS81trnxu573j7bCaj1t+4+XxuaH9j+fO5ht52qqpnDP+8bZKXJblteHkivHcb1N9Yfu+G61ulv9basa21XYZrfXeGPqNnJGOzv80qhA37UpKXr1ioqn9P8rkkh1TV4qp61fCm6Ul+uvrBVTW7qhYn+b0k/7OqFo3Y/IoM/T/Dnp5Sfxk6xfPMJJ+roa8wj3zeZ+/+nmpvp9XQV7K/leRdSf77iG29e0ueen/rMtb6+/0kByY5Yfhz9q2q2nt423j8bCZPvb/x9PncmP7WpXd/I3vbPcl1VfXtDH277sOttZuHt02E925j+luXsdbfuoy9/lprm9WfJC9O8g/rsd8VGzH2V5Jsqz+96U9/+hs//U3k3vQ3tvvb7GbCWmvfTHJ1jbip21r2e9W6tq+uqnZM8pdtaGq0m4nc30TuLdHfiP30N4L+Bm8i95bob8R+Y64/D/AGAOhgs5sJAwAYC4QwAIAOhDAAgA6EMGBCqKpWVR8ZsfzuqjqrY0kA6ySEARPFL5O8rqp22JiDn+ybVQCjTQgDJoqlSc5N8s7VN1TVC6vq2qpaUFVzq+qh4fUHV9XVVfWZJDcPr7u4qm4YvvnmKSPGeKiq/nx425VVtV9VXVNVP6iq126iHoEJRAgDJpJPJDm2qrZZbf1Hk3y0tTY7yV2rbdsvyftba3sML5/YWts3Qw8Jf3tVbT+8fqsk1wxvezDJ/0jyW0mOTDJ39FsBJjohDJgwWmsPJDk/ydtX2/TSDD0CKkk+s9q261trPxyx/PbhR7pcm2TnJCueC/hYksuHf745yZdba48P/zx1VBoANitCGDDRnJ3kDzI0c7U+Hl7xQ1UdnOTQJC9trc1M8s0kKx6q/nj7r7tbL8/QNWhprS1PMvmpFg1sfoQwYEJprd2X5J8yFMRWuDbJUcM/H7OOw7dJ8r9ba49U1W5JXjKYKgGEMGBi+kiSkd+SPD3Ju6rq+iTPTXL/Wo67PMnkqropyZ9mKLwBDIRnRwITXlVtmeTR1lqrqmOSzGmtHd67LmDz5joGYHOwb5KPV1Ul+UWSE/uWA2AmDACgC9eEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHTwfwDdO6e7xBT3sgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmEAAAJcCAYAAACxEXM4AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAtY0lEQVR4nO3dedhdZXkv/u9tgkVQkCG1yiBRsUxJEBJwoIB1Qm0FxBYiYpEKRY2KVi85tqX86LHVao9D1XLQciytJioWpBwESwGtA5LggESk4kikKoMyR0jy/P54d3LehMxk53nfN5/PdeVyr+nZ9529ga/PWnutaq0FAIDN6xG9CwAA2BIJYQAAHQhhAAAdCGEAAB0IYQAAHQhhAAAdCGHAhFVVJ1bVl3rXAbA6Qhgw7lXVIVX1laq6s6ruqKovV9Ws3nUBrM3k3gUAPBxVtV2Si5O8Jsmnkjwyye8k+XXPugDWxUwYMN49NUlaa3Nba0tba/e31j7fWrtu+Q5V9Z6q+mVV/bCqXjhq/auq6oaquruqflBVfzJq2+FVtaiq3l5Vt1XVj6rq+FHbf2Mw7k+q6udVdXZVPWpzNQ2Mf0IYMN79V5KlVfVPVfXCqtphle0HJ7kxyc5J/jbJP1ZVDbb9IsnvJdkuyauSvLeqDhh17G8NjtslyR8lOaeqfnuw7V0ZCYD7J3nKYJ8zNnFvwARWnh0JjHdVtXeStyV5bkaC0yVJTk7ywiR/3lp7ymC/bZLcm+TxrbWfrWacC5Nc2Vp7f1UdnuTyJNu31u4dbP9Ukm8n+Z9J7kkyvbX2/cG2ZyT5RGtt6vA6BSYSM2HAuNdau6G1dmJrbdck+yV5QpL3DTb/bNR+9w1ePjpJBjNnVw8u5v9VkhdlZOZruV8uD2ADPx6MPSXJNkmurapfDY69dLAeYL0IYcCE0lr7bpKPZSSMrVFV/UaSzyR5T5LHtdYem5EZtBq12w5Vte2o5d2T3JLktiT3J9m3tfbYwZ/tW2uP3mSNABOeEAaMa1W1V1X9aVXtOljeLcnsJFev49BHJvmNJLcmWTK4YP/5q9nv/6uqR1bV72Tk+rFPt9aWJflIRq4h+83B++5SVS/YNF0BWwIhDBjv7s7Ixfdfq6p7MxK+rk/yp2s7qLV2d5I3ZOS2Fr9M8vIkF62y288G225J8vEkpw5m2pKRa9BuSnJ1Vd2VkevHfjsA68mF+QCrMbgw/18G15kBbHJmwgAAOhDCAAA6cDoSAKADM2EAAB2Muwd4H3HEEe3SSy/tXQYAwPqoNW0YdzNht912W+8SAAAetnEXwgAAJgIhDACgAyEMAKCDcXdhPgAw4sEHH8yiRYuyePHi3qVs8bbeeuvsuuuu2Wqrrdb7GCEMAMapRYsW5TGPeUz22GOPVK3xR3gMWWstt99+exYtWpSpU6eu93FORwLAOLV48eLstNNOAlhnVZWddtppg2ckhTAAGMcEsLFhYz4HIQwAoAMhDADYIIcffnguu+yylda9733vy2tf+9r1Ov6MM87I5Zdfvs73WLBgwUPWf+xjH8ucOXPWv9gxTAgDADbI7NmzM2/evJXWzZs3L7Nnz17nsUuXLs1ZZ52V5z73ucMqb9wQwgCADfKyl70sF198cX79618nSX70ox/llltuySc+8YnMnDkz++67b/7yL/9yxf577LFHzjrrrBxyyCH59Kc/nRNPPDHnn39+kuSss87KrFmzst9+++WUU05Ja23Fcf/yL/+SZz7zmdlvv/1yzTXXPKSOW2+9Ncccc0xmzZqVWbNm5ctf/vKQO9+0hDAAYIPstNNOOeigg3LppZcmGZkFO/bYY/OOd7wjCxYsyHXXXZcvfOELue6661Ycs/XWW+dLX/pSjjvuuJXGmjNnTubPn5/rr78+999/fy6++OIV2+6999585StfyYc//OGcdNJJD6njjW98Y970pjdl/vz5+cxnPpNXv/rVQ+p4OIQwAGCDjT4lufxU5Kc+9akccMABedrTnpaFCxfmO9/5zor9jz322NWOc+WVV+bggw/OtGnTcsUVV2ThwoUrvUeSHHroobnrrrvyq1/9aqVjL7/88syZMyf7779/XvKSl+Suu+7K3XffvYk7HR43awUANthRRx2VN7/5zfn617+e+++/PzvssEPe8573ZP78+dlhhx1y4oknrnTfrG233fYhYyxevDivfe1rs2DBguy2224588wzVzpm1ds+rLq8bNmyfPWrX82jHvWoTdzd5mEmDADYYI9+9KNz+OGH56STTsrs2bNz1113Zdttt83222+fn//85/nc5z63zjGWB66dd94599xzz4rrxJb75Cc/mST50pe+lO233z7bb7/9Stuf//zn54Mf/OCK5W9+85sPs6vNy0wYALBRZs+enZe+9KWZN29e9tprrzztaU/Lvvvumyc96Ul51rOetc7jH/vYx+bkk0/OtGnTsscee2TWrFkrbd9hhx3yzGc+M3fddVfOPffchxz/gQ98IK973esyffr0LFmyJIceemjOPvvsTdbfsNXoXyGMBzNnzmyru28IAGxpbrjhhuy99969y2BgDZ/HGm+l73QkAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB+4TBgATxIFvPW+Tjnftu1+5XvtdcMEFeelLX5obbrghe+211yatYSITwsaJTf0P1nLr+w8YAKzJ3Llzc8ghh2TevHk588wzh/IeS5cuzaRJk4Yydi9ORwIAG+2ee+7Jl7/85fzjP/7jigd6L126NG95y1sybdq0TJ8+PX//93+fJJk/f36e+cxnZsaMGTnooINy991352Mf+1jmzJmzYrzf+73fy1VXXZVk5NFIZ5xxRg4++OB89atfzVlnnZVZs2Zlv/32yymnnJLlN5y/6aab8tznPjczZszIAQcckO9///s54YQT8tnPfnbFuMcff3wuuuiizfS3sn7MhAEAG+3CCy/MEUcckac+9anZcccd8/Wvfz1f+9rX8sMf/jDf+MY3Mnny5Nxxxx154IEHcuyxx+aTn/xkZs2albvuumudD96+9957s99+++Wss85Kkuyzzz4544wzkiQnnHBCLr744vz+7/9+jj/++Jx++uk5+uijs3jx4ixbtiyvfvWr8973vjdHHnlk7rzzznzlK1/JP/3TPw3972NDmAkDADba3Llzc9xxxyVJjjvuuMydOzeXX355Tj311EyePDLXs+OOO+bGG2/M4x//+BXPh9xuu+1WbF+TSZMm5ZhjjlmxfOWVV+bggw/OtGnTcsUVV2ThwoW5++6789Of/jRHH310kmTrrbfONttsk8MOOyw33XRTfvGLX2Tu3Lk55phj1vl+m9vYqgYAGDduv/32XHHFFbn++utTVVm6dGmqKgceeGCqVn5kYmvtIeuSZPLkyVm2bNmK5cWLF694vfXWW6+4Dmzx4sV57WtfmwULFmS33XbLmWeemcWLF2dtz8A+4YQT8vGPfzzz5s1b7QPAezMTBgBslPPPPz+vfOUr8+Mf/zg/+tGPcvPNN2fq1Kk54IADcvbZZ2fJkiVJkjvuuCN77bVXbrnllsyfPz9Jcvfdd2fJkiXZY4898s1vfjPLli3LzTffnGuuuWa177U8nO2888655557cv755ycZmVHbddddc+GFFyZJfv3rX+e+++5Lkpx44ol53/velyTZd999h/XXsNHMhAHABLG5f/E+d+7cnH766SutO+aYY3LDDTdk9913z/Tp07PVVlvl5JNPzpw5c/LJT34yr3/963P//ffnUY96VC6//PI861nPytSpUzNt2rTst99+OeCAA1b7Xo997GNz8sknZ9q0adljjz1WnNZMkn/+53/On/zJn+SMM87IVlttlU9/+tN50pOelMc97nHZe++9c9RRRw3zr2Gj1dqm8caimTNntgULFvQuY7NziwoAVnXDDTdk77337l3GmHXfffdl2rRp+frXv57tt99+6O+3hs/joedgB5yOBAAmnMsvvzx77bVXXv/612+WALYxnI4EACac5z73ufnJT37Su4y1MhMGANCBEAYA0IEQBgDQgRAGANCBC/Ppbli330jGxi04Jnp/wNjxk7OmbdLxdj/j25t0PFZmJgwA2Gg/+9nPctxxx+XJT35y9tlnn7zoRS9KVeXGG29cab/TTjstf/u3f7vWsd74xjdml112WekxRhOZEAYAbJTWWo4++ugcfvjh+f73v5/vfOc7+eu//uscfvjhmTdv3or9li1blvPPPz/HHnvsGsdatmxZLrjgguy222754he/OLSaly5dOrSxN5QQBgBslCuvvDJbbbVVTj311BXr9t9//7z//e9fKYR98YtfzB577JEnPvGJax1rv/32y2te85rMnTt3xfqf//znOfroozNjxozMmDEjX/nKV5Ik5513XqZPn54ZM2bkhBNOSDLyrMjlz5RMkkc/+tFJkquuuirPfvaz8/KXvzzTpo2csj3qqKNy4IEHZt99980555yz4phLL700BxxwQGbMmJHnPOc5WbZsWfbcc8/ceuutSUbC4lOe8pTcdtttG/33tpxrwgCAjXL99dfnwAMPfMj66dOn5xGPeES+9a1vZcaMGZk3b15mz5691rHmzp2b2bNn58gjj8zb3/72PPjgg9lqq63yhje8IYcddlguuOCCLF26NPfcc08WLlyYd7zjHfnyl7+cnXfeOXfcccc6a73mmmty/fXXZ+rUqUmSc889NzvuuGPuv//+zJo1K8ccc0yWLVuWk08+OV/84hczderU3HHHHXnEIx6RV7ziFfn4xz+e0047LZdffnlmzJiRnXfeeeP+0kYxEwYAbHKzZ8/OvHnzsmTJknz2s5/NH/zBH6xx3wceeCCXXHJJjjrqqGy33XY5+OCD8/nPfz5JcsUVV+Q1r3lNkmTSpEnZfvvtc8UVV+RlL3vZiiC04447rrOegw46aEUAS5IPfOADmTFjRp7+9Kfn5ptvzve+971cffXVOfTQQ1fst3zck046KeedN/Ijq3PPPTevetWrNuJv5KEmzEyYB1wDwOa17777rnT6b7TZs2fn+c9/fg477LBMnz49v/mbv7nGcS699NLceeedK04V3nfffdlmm23y4he/eLX7t9ZS9dDnYk+ePHnFRf2ttTzwwAMrtm277bYrXl911VW5/PLL89WvfjXbbLNNDj/88CxevHiN4+6222553OMelyuuuCJf+9rX8vGPf3yNvWyIoYawqjoiyfuTTEry0dbaO1fZ/tYkx4+qZe8kU1pr655XBABWsrlvKfG7v/u7efvb356PfOQjOfnkk5Mk8+fPz3333ZfDDjssO+20U04//fScdtppax1n7ty5+ehHP7rilOW9996bqVOn5r777stznvOc/MM//ENOO+20LF26NPfee2+e85zn5Oijj86b3vSm7LTTTrnjjjuy4447Zo899si1116bP/zDP8xnP/vZPPjgg6t9vzvvvDM77LBDttlmm3z3u9/N1VdfnSR5xjOekde97nX54Q9/uOJ05PLZsFe/+tV5xStekRNOOCGTJk3aJH9/QzsdWVWTknwoyQuT7JNkdlXtM3qf1tq7W2v7t9b2T/I/knxBAAOA8aGqcsEFF+Tf//3f8+QnPzn77rtvzjzzzDzhCU9IMjIb9t3vfjdHH330Gse47777ctlll60067XtttvmkEMOyb/927/l/e9/f6688spMmzYtBx54YBYuXJh99903f/Znf5bDDjssM2bMyJvf/OYkycknn5wvfOELOeigg/K1r31tpdmv0Y444ogsWbIk06dPz1/8xV/k6U9/epJkypQpOeecc/LSl740M2bMWOnXnC95yUtyzz33bLJTkUlSrbVNNthKA1c9I8mZrbUXDJb/R5K01v5mDft/IsmVrbWPrG3cmTNntgULFjxk/UQ/HTmR+5voNzOd6P0B/dxwww3Ze++9e5exRViwYEHe9KY35T//8z/XuM8aPo+Hnt8cGOaF+bskuXnU8qLBuoeoqm2SHJHkM2vYfkpVLaiqBct/IgoAsDm8853vzDHHHJO/+ZvVziNttGGGsNUlvzVNu/1+ki+v6VRka+2c1trM1trMKVOmbLICAYDN57LLLsv++++/0p+1naocK04//fT8+Mc/ziGHHLJJxx3mhfmLkuw2annXJLesYd/jksxdwzYAYA3W9Iu+segFL3hBXvCCF/QuYyg25vKuYc6EzU+yZ1VNrapHZiRoXbTqTlW1fZLDknx2iLUAwISz9dZb5/bbb9+oAMCm01rL7bffnq233nqDjhvaTFhrbUlVzUlyWUZuUXFua21hVZ062H72YNejk3y+tXbvsGoBgIlo1113zaJFi+J66f623nrr7Lrrrht0zFDvE9ZauyTJJausO3uV5Y8l+dgw6wCAiWirrbZa6S7wjC8eWwQA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0MFQQ1hVHVFVN1bVTVV1+hr2ObyqvllVC6vqC8OsBwBgrJg8rIGralKSDyV5XpJFSeZX1UWtte+M2uexST6c5IjW2k+q6jeHVQ8AwFgyzJmwg5Lc1Fr7QWvtgSTzkhy5yj4vT/KvrbWfJElr7RdDrAcAYMwYZgjbJcnNo5YXDdaN9tQkO1TVVVV1bVW9cnUDVdUpVbWgqhbceuutQyoXAGDzGWYIq9Wsa6ssT05yYJIXJ3lBkr+oqqc+5KDWzmmtzWytzZwyZcqmrxQAYDMb2jVhGZn52m3U8q5JblnNPre11u5Ncm9VfTHJjCT/NcS6AAC6G+ZM2Pwke1bV1Kp6ZJLjkly0yj6fTfI7VTW5qrZJcnCSG4ZYEwDAmDC0mbDW2pKqmpPksiSTkpzbWltYVacOtp/dWruhqi5Ncl2SZUk+2lq7flg1AQCMFcM8HZnW2iVJLlll3dmrLL87ybuHWQcAwFjjjvkAAB0IYQAAHQhhAAAdCGEAAB0IYQAAHQhhAAAdCGEAAB0IYQAAHQhhAAAdCGEAAB0IYQAAHQhhAAAdCGEAAB0IYQAAHQhhAAAdCGEAAB0IYQAAHQhhAAAdCGEAAB0IYQAAHQhhAAAdTO5dADC+HfjW84Yy7rXvfuVQxgUYK8yEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdOCO+UxoPzlr2lDG3f2Mbw9lXAC2HGbCAAA6EMIAADoQwgAAOnBNGIxjrnkbvgPfet5Qxr323a8cyrgbaqL3B2OZmTAAgA7MhG3hzKQAQB9mwgAAOhDCAAA6cDpyHZyuAwCGwUwYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0IYAEAHQhgAQAdCGABAB0MNYVV1RFXdWFU3VdXpq9l+eFXdWVXfHPw5Y5j1AACMFZOHNXBVTUryoSTPS7Ioyfyquqi19p1Vdv3P1trvDasOAICxaJgzYQcluam19oPW2gNJ5iU5cojvBwAwbgwzhO2S5OZRy4sG61b1jKr6VlV9rqr2Xd1AVXVKVS2oqgW33nrrMGoFANishhnCajXr2irLX0/yxNbajCR/n+TC1Q3UWjuntTaztTZzypQpm7ZKAIAOhhnCFiXZbdTyrkluGb1Da+2u1to9g9eXJNmqqnYeYk0AAGPCMEPY/CR7VtXUqnpkkuOSXDR6h6r6raqqweuDBvXcPsSaAADGhKH9OrK1tqSq5iS5LMmkJOe21hZW1amD7WcneVmS11TVkiT3JzmutbbqKUsAgAlnaCEsWXGK8ZJV1p096vUHk3xwmDUAAIxF7pgPANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQgRAGANCBEAYA0IEQBgDQweTeBQAAG+7At543tLGvffcrhzY2/4+ZMACADoQwAIAOhDAAgA6EMACADoQwAIAOhDAAgA7cogKACcktHBjrzIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQghAEAdCCEAQB0IIQBAHQwuXcBAKvzk7OmDW3s3c/49tDGBlhfZsIAADowEwbQwbBm+szywfhhJgwAoAMhDACgAyEMAKADIQwAoIOhhrCqOqKqbqyqm6rq9LXsN6uqllbVy4ZZDwDAWDG0EFZVk5J8KMkLk+yTZHZV7bOG/d6V5LJh1QIAMNYMcybsoCQ3tdZ+0Fp7IMm8JEeuZr/XJ/lMkl8MsRYAgDFlve8TVlXbttbu3YCxd0ly86jlRUkOXmXMXZIcneR3k8xay3ufkuSUJNl99903oAQAenAfNFi3dc6EVdUzq+o7SW4YLM+oqg+vx9i1mnVtleX3JXlba23p2gZqrZ3TWpvZWps5ZcqU9XhrAICxbX1mwt6b5AVJLkqS1tq3qurQ9ThuUZLdRi3vmuSWVfaZmWReVSXJzkleVFVLWmsXrsf4AADj1nqdjmyt3TwISsutdeZqYH6SPatqapKfJjkuyctXGXfq8tdV9bEkFwtgAMCWYH1C2M1V9cwkraoemeQNGZyaXJvW2pKqmpORXz1OSnJua21hVZ062H72w6gbAGBcW58QdmqS92fkQvtFST6f5HXrM3hr7ZIkl6yybrXhq7V24vqMCQAwEawzhLXWbkty/GaoBQBgi7HOEFZV/ycP/VVjWmsnDaUiAIAtwPqcjrx41OutM3Jfr1V/5QgAwAZYn9ORnxm9XFVzk1w+tIoAALYAG/PYoj2TuG09AMDDsD7XhN2dkWvCavC/P0vytiHXBQAwoa3P6cjHbI5CAAC2JGsMYVV1wNoObK19fdOXAwCwZVjbTNjfrWVbS/K7m7gWAIAtxhpDWGvt2ZuzEACALcl6PcC7qvZLsk9G7hOWJGmtnTesogAAJrr1+XXkXyY5PCMh7JIkL0zypSRCGADARlqf+4S9LMlzkvystfaqJDOS/MZQqwIAmODWJ4Qtbq0tS7KkqrZL8oskTxpuWQAAE9vablHxwSRzk1xTVY9N8pEk1ya5J8k1m6U6AIAJam3XhH0vyXuSPCEjwWtukucl2a61dt1mqA0AYMJa4+nI1tr7W2vPSHJokjuS/J8kn0tyVFXtuZnqAwCYkNZ5TVhr7cettXe11p6W5OVJjk7y3aFXBgAwga0zhFXVVlX1+1X18YzMhP1XkmOGXhkAwAS2tgvzn5dkdpIXZ+RC/HlJTmmt3buZagMAmLDWdmH+25N8IslbWmt3bKZ6AAC2CJ4dCQDQwfrcrBUAgE1MCAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhgqCGsqo6oqhur6qaqOn0124+squuq6ptVtaCqDhlmPQAAY8XkYQ1cVZOSfCjJ85IsSjK/qi5qrX1n1G7/keSi1lqrqulJPpVkr2HVBAAwVgxzJuygJDe11n7QWnsgybwkR47eobV2T2utDRa3TdICALAFGGYI2yXJzaOWFw3WraSqjq6q7yb5v0lOWt1AVXXK4HTlgltvvXUoxQIAbE7DDGG1mnUPmelqrV3QWtsryVFJ/mp1A7XWzmmtzWytzZwyZcqmrRIAoIOhXROWkZmv3UYt75rkljXt3Fr7YlU9uap2bq3dNsS6AOBh+clZ04Yy7u5nfHso4zI2DTOEzU+yZ1VNTfLTJMclefnoHarqKUm+P7gw/4Akj0xy+xBrAgDWQcjcPIYWwlprS6pqTpLLkkxKcm5rbWFVnTrYfnaSY5K8sqoeTHJ/kmNHXagPADBhDXMmLK21S5Jcssq6s0e9fleSdw2zBgCAscgd8wEAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6mNy7AACAVR341vOGMu61737lUMbdGGbCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoQwgAAOhDCAAA6EMIAADoYagirqiOq6saquqmqTl/N9uOr6rrBn69U1Yxh1gMAMFYMLYRV1aQkH0rywiT7JJldVfusstsPkxzWWpue5K+SnDOsegAAxpJhzoQdlOSm1toPWmsPJJmX5MjRO7TWvtJa++Vg8eokuw6xHgCAMWOYIWyXJDePWl40WLcmf5zkc6vbUFWnVNWCqlpw6623bsISAQD6GGYIq9Wsa6vdserZGQlhb1vd9tbaOa21ma21mVOmTNmEJQIA9DF5iGMvSrLbqOVdk9yy6k5VNT3JR5O8sLV2+xDrAQAYM4Y5EzY/yZ5VNbWqHpnkuCQXjd6hqnZP8q9JTmit/dcQawEAGFOGNhPWWltSVXOSXJZkUpJzW2sLq+rUwfazk5yRZKckH66qJFnSWps5rJoAAMaKYZ6OTGvtkiSXrLLu7FGvX53k1cOsAQBgLHLHfACADoQwAIAOhDAAgA6Gek0YAMBY8pOzpg1t7N3P+PYG7W8mDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoAMhDACgAyEMAKADIQwAoIOhhrCqOqKqbqyqm6rq9NVs36uqvlpVv66qtwyzFgCAsWTysAauqklJPpTkeUkWJZlfVRe11r4zarc7krwhyVHDqgMAYCwa5kzYQUluaq39oLX2QJJ5SY4cvUNr7RettflJHhxiHQAAY84wQ9guSW4etbxosG6DVdUpVbWgqhbceuutm6Q4AICehhnCajXr2sYM1Fo7p7U2s7U2c8qUKQ+zLACA/oYZwhYl2W3U8q5Jbhni+wEAjBvDDGHzk+xZVVOr6pFJjkty0RDfDwBg3BjaryNba0uqak6Sy5JMSnJua21hVZ062H52Vf1WkgVJtkuyrKpOS7JPa+2uYdUFADAWDC2EJUlr7ZIkl6yy7uxRr3+WkdOUAABbFHfMBwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhACAMA6EAIAwDoQAgDAOhgqCGsqo6oqhur6qaqOn0126uqPjDYfl1VHTDMegAAxoqhhbCqmpTkQ0lemGSfJLOrap9Vdnthkj0Hf05J8g/DqgcAYCwZ5kzYQUluaq39oLX2QJJ5SY5cZZ8jk5zXRlyd5LFV9fgh1gQAMCZUa204A1e9LMkRrbVXD5ZPSHJwa23OqH0uTvLO1tqXBsv/keRtrbUFq4x1SkZmypLkt5PcOJSiV2/nJLdtxvfb3PQ3vk3k/iZyb4n+xjv9jV+bu7fbWmtHrG7D5CG+aa1m3aqJb332SWvtnCTnbIqiNlRVLWitzezx3puD/sa3idzfRO4t0d94p7/xayz1NszTkYuS7DZqedckt2zEPgAAE84wQ9j8JHtW1dSqemSS45JctMo+FyV55eBXkk9Pcmdr7b+HWBMAwJgwtNORrbUlVTUnyWVJJiU5t7W2sKpOHWw/O8klSV6U5KYk9yV51bDqeRi6nAbdjPQ3vk3k/iZyb4n+xjv9jV9jprehXZgPAMCauWM+AEAHQhgAQAdbXAirqkdV1RcGd/RPVV1aVb8a3LNsfY4/tKq+XlVLBvdCW75+SlVdOqy619cm6O/NVfWdwWOk/qOqnjhY372/TdDbqVX17ar6ZlV9afkTHMZCb4M6HlZ/o8Z5WVW1qpo5WB5z/VXV/lX11apaOPiuHbsex4/Z7+agjofb37j5fm5Mf6PGGXPfz1V6e2JVXTv4HFZcx7yO48fTZ7fB/Y0aZ8x9doM6Vvp352DddlX106r64AaMs9n72+JCWJKTkvxra23pYPndSU7YgON/kuTEJJ8YvbK1dmuS/66qZ22KIh+Gh9vfN5LMbK1NT3J+kr9Nxkx/D7e3T7TWprXW9s9IX/8rGTO9JQ+/v1TVY5K8IcnXlq8bo/3dl+SVrbV9kxyR5H1V9dh1HD+Wv5vJw+9vPH0/N6a/sfz9HN3bfyd55uBzODjJ6VX1hHUcP54+u43pbyx/dslD/92ZJH+V5AvrO0Cv/rbEEHZ8ks8uX2it/UeSu9f34Nbaj1pr1yVZtprNFw7G7+nh9ndla+2+weLVGbl323IXpm9/D7e3u0YtbpuVbwx8Ycb5ZzfwVxn5j8DiVdZfmDHUX2vtv1pr3xu8viXJL5JMWdvBY/y7mTz8/sbN93Nj+hsYq9/P0b090Fr79WD9b2Q9/js5zj67De5vYKx+dskq/+6sqgOTPC7J5zdgjC79bVEhrEbuV/ak1tqPhvQWC5L8zpDGXqch9PfHST43arlbf5uqt6p6XVV9PyP/sL1h1KZx/9lV1dOS7NZaW93pyzHbX1UdlOSRSb6/AUOOme9msun6G4/fz/Xtb6x+P1fXW1XtVlXXJbk5ybsGQXNd44ybz25D+xurn13y0P6q6hFJ/i7JWzdgjG79bVEhLCPPi/rVEMf/RZJ1TusO0Sbrr6pekWRmRk6JLdezv03SW2vtQ621Jyd5W5I/H7VpXH92g3/xvDfJn65hlzHZX1U9Psk/J3lVa211s8sPMQa/m8km6m+8fT/Xt78x/v18SG+ttZsHp72fkuSPqupx6xpkPH12G9LfGP/skof299okl7TWbl6fg3v3t6WFsPuTbD3E8bcevEcvm6S/qnpukj9L8pJR09ZJ3/429Wc3L8lRo5bH+2f3mCT7Jbmqqn6U5OlJLlp+gWnGYH9VtV2S/5vkz1trV6/PIGP0u5lsov5GGfPfzw3sbyx/P9f4z95ghmhhNmwmZMx/dsutZ39j+bNLHtrfM5LMGdT6now8leedazm+a39bVAhrrf0yyaSqWud/7Krqb6rq6A18i6cmuX6jitsENkV/g2nZ/52R/8j9YpXN3frbRL3tOWrxxUm+N2p5XH92rbU7W2s7t9b2aK3tkZFrpl7SWlsw2GVM9Tc4hXBBkvNaa58eve94+24mm6y/cfP93ND+xvL3czW97VpVjxq83iHJs5LcOFieCJ/dBvU3lj+7QX0r9ddaO761tvug1rdk5Dt6ejI2+9uiQtjA55Mcsnyhqv4zyaeTPKeqFlXVCwabpiX52aoHV9WsqlqU5A+S/O+qWjhq87Mz8v8Me3pY/WXkFM+jk3y6Rn7CPPp5n737e7i9zamRn2R/M8mbk/zRqG29e0sefn9rM9b6+8MkhyY5cfA9+2ZV7T/YNh6/m8nD7288fT83pr+16d3f6N72TvK1qvpWRn5d957W2rcH2ybCZ7cx/a3NWOtvbcZef621LepPkqcl+ef12O+yjRj7i0l20J/e9Kc//Y2f/iZyb/ob2/1tcTNhrbVvJLmyRt3UbQ37vWBt21dVVVOS/K82MjXazUTubyL3luhv1H76G0V/wzeRe0v0N2q/MdefB3gDAHSwxc2EAQCMBUIYAEAHQhgAQAdCGDAhVFWrqr8btfyWqjqzY0kAayWEARPFr5O8tKp23piD1/XLKoBNTQgDJoolSc5J8qZVN1TVk6vq6qqaX1VnVdU9g/WHV9WVVfWJJN8erLuwqq4d3HzzlFFj3FNV7xpsu7yqDqqqq6rqB1X1ks3UIzCBCGHARPKhJMdX1farrH9/kve31mYluWWVbQcl+bPW2j6D5ZNaawdm5CHhb6iqnQbrt01y1WDb3Un+Z5LnJTk6yVmbvhVgohPCgAmjtXZXkvOSvGGVTc/IyCOgkuQTq2y7prX2w1HLbxg80uXqJLslWf5cwAeSXDp4/e0kX2itPTh4vccmaQDYoghhwETzviR/nJGZq/Vx7/IXVXV4kucmeUZrbUaSbyRZ/lD1B9v/u7v1soxcg5bW2rIkkx9u0cCWRwgDJpTW2h1JPpWRILbc1UmOGbw+bi2Hb5/kl621+6pqryRPH06VAEIYMDH9XZLRv5I8Lcmbq+qaJI9Pcucajrs0yeSqui7JX2UkvAEMhWdHAhNeVW2T5P7WWquq45LMbq0d2bsuYMvmOgZgS3Bgkg9WVSX5VZKT+pYDYCYMAKAL14QBAHQghAEAdCCEAQB0IIQBAHQghAEAdPD/AwpoFMwICrfHAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot with y = color + shape\n",
    "plot_label = result_df[result_df['Label']=='label']\n",
    "plot_label = plot_label.drop(columns=['Label'])\n",
    "\n",
    "fig, ax1 = plt.subplots(figsize=(10, 10))\n",
    "tidy = plot_label.melt(id_vars='Ngram').rename(columns=str.title)\n",
    "sns.barplot(x='Ngram', y='Value', hue='Variable', data=tidy, ax=ax1).set_title('Full label')\n",
    "sns.despine(fig)\n",
    "\n",
    "# plot with y = color\n",
    "plot_color = result_df[result_df['Label']=='color']\n",
    "plot_color = plot_color.drop(columns=['Label'])\n",
    "\n",
    "fig, ax1 = plt.subplots(figsize=(10, 10))\n",
    "tidy = plot_color.melt(id_vars='Ngram').rename(columns=str.title)\n",
    "sns.barplot(x='Ngram', y='Value', hue='Variable', data=tidy, ax=ax1).set_title('Color')\n",
    "sns.despine(fig)\n",
    "\n",
    "# plot with y = shape\n",
    "plot_shape = result_df[result_df['Label']=='shape']\n",
    "plot_shape = plot_shape.drop(columns=['Label'])\n",
    "\n",
    "fig, ax1 = plt.subplots(figsize=(10, 10))\n",
    "tidy = plot_shape.melt(id_vars='Ngram').rename(columns=str.title)\n",
    "sns.barplot(x='Ngram', y='Value', hue='Variable', data=tidy, ax=ax1).set_title('Shape')\n",
    "sns.despine(fig)\n",
    "\n",
    "# from the figure, smaller n-grams performed bettwe than large n_grams"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Testing on Survey Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(176, 3)\n"
     ]
    }
   ],
   "source": [
    "df = pd.read_excel('processed_survey.xlsx')\n",
    "print (df.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# split label into color and shape\n",
    "df[['color','shape']] = df['label'].str.split(' ',expand=True)\n",
    "#df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "bow_survey = count_vec.transform(df['text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = label_model.predict(bow_survey)\n",
    "color_pred = color_model.predict(bow_survey)\n",
    "shape_pred = shape_model.predict(bow_survey)\n",
    "\n",
    "com_pred = []\n",
    "for c,s in zip(color_pred,shape_pred):\n",
    "    com = c+' '+s\n",
    "    com_pred.append(com)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Acc on full label: 0.4034090909090909\n",
      "Acc on color: 0.7443181818181818\n",
      "Acc on shape: 0.4431818181818182\n",
      "Acc on color + shape: 0.6363636363636364\n"
     ]
    }
   ],
   "source": [
    "# print accuracy\n",
    "# this is accuracy on survey data\n",
    "print('Acc on full label:', accuracy_score(np.array(df['label']), y_pred))\n",
    "print('Acc on color:', accuracy_score(np.array(df['color']), color_pred))\n",
    "print('Acc on shape:', accuracy_score(np.array(df['shape']), shape_pred))\n",
    "print('Acc on color + shape:', accuracy_score(com_pred, y_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}