LDA_irisdataset.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dimensionality reduction via linear discriminant analysis\n",
    "Principal component analysis (PCA) and linear discriminant analysis (LDA) are two data preprocessing linear transformation techniques that are often used for dimensionality reduction in order to select relevant features that can be used in the final machine learning algorithm. PCA is an unsupervised algorithm that is used for feature extraction in high-dimensional and correlated data. PCA achieves dimensionality reduction by transforming features into orthogonal component axes of maximum variance in a dataset. An implementation of PCA using iris dataset can be found here: https://github.com/bot13956/principal_component_analysis_iris_dataset\n",
    "\n",
    "The goal of LDA is to find the feature subspace that optimizes class separability. Hence, LDA is a supervised algorithm. In this code, we illustrate the implementation of LDA using the iris dataset.\n",
    "\n",
     "An in-depth description of PCA and LDA can be found in this book: Python Machine Learning by Sebastian Raschka, Chapter 5.\n",
    "\n",
    "author: Benjamin O. Tayo\n",
    "\n",
    "date: 11-06-2018 \n",
    "\n",
    "Reference: Python Machine Learning by Sebastian Raschka, Chapter 5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Import necessary libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "np.set_printoptions(precision=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Read dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('iris.data.csv', header=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5.1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.7</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4.6</td>\n",
       "      <td>3.1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5.0</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     0    1    2    3            4\n",
       "0  5.1  3.5  1.4  0.2  Iris-setosa\n",
       "1  4.9  3.0  1.4  0.2  Iris-setosa\n",
       "2  4.7  3.2  1.3  0.2  Iris-setosa\n",
       "3  4.6  3.1  1.5  0.2  Iris-setosa\n",
       "4  5.0  3.6  1.4  0.2  Iris-setosa"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Encode categorical class labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import LabelEncoder\n",
    "class_le = LabelEncoder()\n",
    "y = class_le.fit_transform(df[4].values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Standardize features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "stdsc = StandardScaler()\n",
    "X_train_std = stdsc.fit_transform(df.iloc[:,range(0,4)].values) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Construct within-class covariant scatter matrix S_W"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "S_W = np.zeros((4,4))\n",
    "for i in range(3):\n",
    "    S_W += np.cov(X_train_std[y==i].T)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1.1672, 0.783 , 0.3461, 0.1839],\n",
       "       [0.783 , 1.8616, 0.2181, 0.3051],\n",
       "       [0.3461, 0.2181, 0.1796, 0.0954],\n",
       "       [0.1839, 0.3051, 0.0954, 0.2178]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "S_W"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Construct between-class scatter matrix S_B"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "N=np.bincount(y) # number of samples for given class\n",
    "vecs=[]\n",
    "[vecs.append(np.mean(X_train_std[y==i],axis=0)) for i in range(3)] # class means\n",
    "mean_overall = np.mean(X_train_std, axis=0) # overall mean\n",
    "S_B=np.zeros((4,4))\n",
    "for i in range(3):\n",
    "    S_B += N[i]*(((vecs[i]-mean_overall).reshape(4,1)).dot(((vecs[i]-mean_overall).reshape(1,4))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 92.8059, -54.7706, 113.8033, 113.6835],\n",
       "       [-54.7706,  58.7821, -73.7624, -68.4292],\n",
       "       [113.8033, -73.7624, 141.1978, 139.7382],\n",
       "       [113.6835, -68.4292, 139.7382, 139.3254]])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "S_B"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Calculate sorted eigenvalues and eigenvectors of  inverse(S_W)dot(S_B)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Eigenvalues in decreasing order:\n",
      "\n",
      "1581.3259321867629\n",
      "13.60077632816224\n",
      "1.5798874091613385e-13\n",
      "1.5798874091613385e-13\n"
     ]
    }
   ],
   "source": [
    "eigen_vals, eigen_vecs = np.linalg.eig(np.linalg.inv(S_W).dot(S_B))\n",
    "eigen_pairs = [(np.abs(eigen_vals[i]), eigen_vecs[:,i]) for i in range(len(eigen_vals))]\n",
    "eigen_pairs = sorted(eigen_pairs,key=lambda k: k[0], reverse=True)\n",
    "print('Eigenvalues in decreasing order:\\n')\n",
    "for eigen_val in eigen_pairs:\n",
    "    print(eigen_val[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Plot main LDA components"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XmcVOWV//HPoUEbRYgKYcAGGxTEhqYbaNoNAxgX3CAIooA6jYqjGVxjJswvLkSzmIn5ERdGBo3BhQhqXFBRI0HQEEUbBGVTETA2OIoYEaMo0Gf+qNvXoqjl0nR1dcP3/XrVq+re+9x7Tz1in7rbeczdERERAWiS6wBERKThUFIQEZGQkoKIiISUFEREJKSkICIiISUFEREJKSmIiEhISUFEREJKCiIiEmqa6wB2VevWrb2wsDDXYYiINCoLFy78xN3bZGrX6JJCYWEhlZWVuQ5DRKRRMbP3o7TT6SMREQkpKYiISEhJQUREQkoKIiISUlIQEZGQkoKIiISUFEREJKSkICIiISUFEREJKSmIiEhISUFEREJKCiIiElJSEBGRUNaSgpnda2Yfm9nSFMvNzG43s1Vm9qaZ9c5WLCIiEk02jxSmAoPSLD8V6BK8LgHuymIsIiISQdbGU3D3l8ysME2TIcD97u7Aq2b2HTNr5+4fZiumPd0fF/ydJxevy3UYIrKLitq35MYzu+c6DCC31xQOAT6Im64K5u3EzC4xs0ozq9ywYUO9BNcYPbl4Hcs//DzXYYhII5bLkdcsyTxP1tDdpwBTAMrKypK2qSs/e2oZy9c3zj+syz/8nKJ2LZnxb8fkOhQRaaRymRSqgA5x0wXA+mzucOIL72Rs88bfP2PD5q8ztis4sHldhFSnitq1ZEhp0oMtEZFIcpkUZgLjzGw6cBSwqSFcT+jfNeO41gBcfVLXLEciIlL/spYUzOwhYADQ2syqgBuBZgDuPhmYBZwGrAK+BMZkKxYREYkmm3cfjcyw3IF/z9b+RURk1+mJZhERCSkpiIhISElBRERCSgoiIhJSUhARkZCSgoiIhJQUREQkpKQgIiIhJQUREQkpKYiISEhJQUREQkoKIiISUlIQEZGQkoKIiISUFEREJKSkICIiISUFEREJKSmIiEhISUFEREJKCiIiElJSEBGRkJKCiIiElBRERCSkpCAiIiElBRERCSkpiIhISElBRERCWU0KZjbIzN42s1VmNj7J8o5m9qKZvWFmb5rZadmMR0RE0staUjCzPGAScCpQBIw0s6KEZtcBD7t7L+Bc4L+zFY+IiGSWzSOFcmCVu69292+A6cCQhDYOtAw+twLWZzEeERHJoGkWt30I8EHcdBVwVEKbCcCfzexyYH/gxCzGIyIiGWQ8UjCzZmZ2hZk9GrwuN7NmEbZtSeZ5wvRIYKq7FwCnAQ+Y2U4xmdklZlZpZpUbNmyIsGsREamNKKeP7gL6EDvf/99A72BeJlVAh7jpAnY+PXQR8DCAu78C5AOtEzfk7lPcvczdy9q0aRNh1yIiUhtRTh/1dfeSuOk5ZrYkwnqvA13MrBOwjtiF5FEJbf4OfB+YamZHEksKOhQQEcmRKEcK283ssJoJM+sMbM+0krtvA8YBzwMriN1ltMzMbjKzwUGzHwFjgyTzEFDh7omnmEREpJ5EOVL4MfCima0mdp3gUGBMlI27+yxgVsK8G+I+LweOixytiIhkVcak4O5/MbMuwBHEksJKd/8665GJiEi9S5kUzOwEd59jZmclLDrMzHD3x7Icm4iI1LN0Rwr9gTnAmUmWOaCkICKyh0mZFNz9xuDjTe6+Jn5ZcEeRiIjsYaLcffSnJPMeretAREQk99JdU+gGdAdaJVxXaEnseQIREdnDpLumcARwBvAddryusBkYm82gREQkN9JdU3gSeNLMjglKUIiIyB4uysNrb5jZvxM7lRSeNnL3C7MWlYiI5ESUC80PAP8CnALMI1bYbnM2gxIRkdyIkhQOd/frgX+6+33A6UBxdsMSEZFciJIUtgbvn5lZD2IjpBVmLSIREcmZKNcUppjZgcTGU54JtACuz2pUIiKSE2mTQjAK2ufu/g/gJaBzvUQlIiI5kfb0kbtXExsTQURE9gJRrim8YGbXmlkHMzuo5pX1yEREpN5FuaZQ8zzCv8fNc3QqSURkjxNlkB1VRBUR2UtEOX0kIiJ7CSUFEREJKSmIiEgo3XgKHYOP2919XT3FIyIiOZTuQvN9xO4y+hQYXj/hiIhILqVLChOC96/rIQ4REWkA0iWFiuD9M+DV7IciIiK5lm7ktTEAZpZXf+GIiEguRbn7aJWZ/cbMirIejYiI5FSUpNATeAe4x8xeNbNLzKxllI2b2SAze9vMVpnZ+BRtRpjZcjNbZmZ/3IXYRUSkjmVMCu6+2d3vdvdjgf8AbgQ+NLP7zOzwVOsFp50mAacCRcDIxKMNM+sC/CdwnLt3B66q/VcREZHdlTEpmFmemQ02s8eB24DfEiuG9xQwK82q5cAqd1/t7t8A04EhCW3GApOC8Rpw949r8R1ERKSORKmS+i7wIvAbd/9b3PxHzex7adY7BPggbroKOCqhTVcAM5sP5AET3P25CDGJiEgWREkKF7j7X+NnmNlx7j7f3a9Is54lmedJ9t8FGAAUAC+bWQ93/yxhf5cAlwB07NgRERHJjigXmm9PMu+OCOtVAR3ipguA9UnaPOnuW919DfA2sSSxA3ef4u5l7l7Wpk2bCLsWEZHaSFf76BjgWKCNmV0Tt6glsVM9mbwOdDGzTsA64FxgVEKbJ4CRwFQza03sdNLq6OGL7L6tW7dSVVXFli1bch2KyG7Lz8+noKCAZs2a1Wr9dKeP9gFaBG0OiJv/ORFqIbn7NjMbBzxPLInc6+7LzOwmoNLdZwbLTjaz5cB24MfuvrFW30SklqqqqjjggAMoLCzELNlZT5HGwd3ZuHEjVVVVdOpUu/HR0j3RPA+YZ2ZT3f39WgY4i4Q7lNz9hrjPDlwTvERyYsuWLUoIskcwMw4++GA2bNhQ622kO330O3e/CrjTzBIvEOPug2u9V5EGRglB9hS7+2853YXmB4L3W4k9m5D4EpFGaO3atfTo0SNjmz/+8dsCA5WVlVxxRbqbDaOZO3cuFRUVTJ06lQkTJiRt06JFCwDWr1/P8OG7X7V/5syZ3HLLLbu0zmmnncZnn32WuWESFRUVPProozvNj+/DqVOnMm7cOAAmT57M/fffH85fvz7xfpydFRYW7vBel9KdPloYvM+r872KSINWkxRGjYrdG1JWVkZZWVm9xtC+ffukf1x3xbZt2xg8eDCDB+/aiY1Zs9I9l1s7qfrw0ksvDT9PnTqVHj160L59+zrff1QpjxTM7C0zezPVqz6DFNnT3X///fTs2ZOSkhLOP/98YOdfnDW/oOfOnUv//v0ZMWIEXbt2Zfz48UybNo3y8nKKi4t577330q4fb+3atRx//PH07t2b3r1787e/xZ5PHT9+PC+//DKlpaVMnDiRuXPncsYZZ1BdXU1hYeEOv6IPP/xwPvroIzZs2MCwYcPo27cvffv2Zf78+Tvtb5999qFVq1Y0b948jGfNmjUcc8wx9O3bl+uvv36H2GqOaJYtW0Z5eTmlpaX07NmTd999N22/XXPNNQwcOJCf/OQnO/wqr6io4LLLLmPgwIF07tyZefPmceGFF3LkkUdSUVER7ruwsJBPPvmEtWvXcuSRRzJ27Fi6d+/OySefzFdffQXA3XffTd++fSkpKWHYsGF8+eWX4fqzZ8/m+OOPp2vXrjz99NPhf7czzjhjpz6ZMGECt956K48++iiVlZWMHj2a0tJSnnnmGYYOHRq2e+GFFzjrrLMAqLk1Pxu36Ke7+2jn6EX2cD97ahnL139ep9ssat+SG8/snnL5smXL+MUvfsH8+fNp3bo1n376acZtLlmyhBUrVnDQQQfRuXNnLr74Yl577TVuu+027rjjDn73u99Fiu273/0uL7zwAvn5+bz77ruMHDmSyspKbrnlFm699dYd/qABNGnShCFDhvD4448zZswYFixYQGFhIW3btmXUqFFcffXV9OvXj7///e+ccsoprFixYof9HXvssRx77LE7zLvyyiu57LLLuOCCC5g0aVLSOCdPnsyVV17J6NGj+eabb9i+fXvafnvnnXeYPXs2eXl5TJ06dYdt/eMf/2DOnDnMnDmTM888k/nz53PPPffQt29fFi9eTGlp6Q7t3333XR566CHuvvtuRowYwZ/+9CfOO+88zjrrLMaOHQvAddddx+9//3suv/xyIJbQ5s2bx3vvvcfAgQNZtWpVxv8Ww4cP58477+TWW2+lrKwMd+dHP/oRGzZsoE2bNvzhD39gzJgxALz++us7vNellEcK7v5+uledRyKyl5ozZw7Dhw+ndevWABx00EEZ1+nbty/t2rVj33335bDDDuPkk08GoLi4mLVr10be99atWxk7dizFxcWcffbZLF++POM655xzDjNmzABg+vTpnHPOOUDs1/G4ceMoLS1l8ODBfP7552zevDnj9ubPn8/IkSMBwl/7iY455hh++ctf8utf/5r333+f5s2bp+23s88+m7y85I9TnXnmmZgZxcXFtG3bluLiYpo0aUL37t2T9l2nTp3CRNGnT5+wzdKlSzn++OMpLi5m2rRpLFu2LFxnxIgRNGnShC5dutC5c2dWrlyZsR8SmRnnn38+Dz74IJ999hmvvPIKp5566i5vZ1elu/vor+7ez8w2EytPYfHv7h6pfLZIY5LuF322uHvSO0aaNm1KdXV12Oabb74Jl+27777h5yZNmoTTTZo0Ydu2bRnXrzFx4kTatm3LkiVLqK6uJj8/P2O8xxxzDKtWrWLDhg088cQTXHfddQBUV1fzyiuv0Lx586hfPZTpjplRo0Zx1FFH8cwzz3DKKadwzz33pOw3gP333z/ltuL7KrEfa/ouWXuAvLy88PRRRUUFTzzxBCUlJUydOjU8mkr2fWp7R9CYMWM488wzyc/P5+yzz6Zp0yiViXZPuiOFfsH7Ae7eMvE965GJ7CW+//3v8/DDD7NxY+y5zZrTIIWFhSxcuBCAJ598kq1bt+7SdqOsv2nTJtq1a0eTJk144IEH2L59OwAHHHBAyl/5ZsbQoUO55pprOPLIIzn44IMBOPnkk7nzzjvDdosXL44U53HHHcf06dMBmDZtWtI2q1evpnPnzlxxxRUMHjyYN998M2W/1ZfNmzfTrl07tm7dulPcjzzyCNXV1bz33nusXr2aI444ItI2E/u9ffv2tG/fnp///Oc7XPPIpii1jzCz3mZ2hZldbma9sh2UyN6ke/fu/PSnP6V///6UlJRwzTWxZznHjh3LvHnzKC8vZ8GCBWl//SYTZf0f/vCH3HfffRx99NG88847YZuePXvStGlTSkpKmDhx4k7rnXPOOTz44IPhqSOA22+/ncrKSnr27ElRURGTJ0+OFOdtt93GpEmT6Nu3L5s2bUraZsaMGfTo0YPS0lJWrlzJBRdckLLf6svNN9/MUUcdxUknnUS3bt12WHbEEUfQv39/Tj31VCZPnhzpCAxiRx+XXnoppaWl4RHJ6NGj6dChA0VF9TP4pcUeKk7TwOwG4GzgsWDWD4BH3P3nWY4tqbKyMq+srKzVuhNfeKfO4rj6pK51ti3JrRUrVnDkkUfmOgyRpMaNG0evXr246KKLIq+T7N+0mS1094z3FUc5QTUS6OXuW4IN3wIsAnKSFERE9hZ9+vRh//3357e/rb/nhaMkhbVAPlBTQnJf4L1sBSQiIjE114TqU7q7j+4gdrfR18AyM3shmD4J+Guq9UREpPFKd6RQc+J+IfB43Py5WYtGRERyKl3to/vqMxAREcm9jNcUzKwL8CugiNi1BQDcvXMW4xIRkRyI8pzCH4C7gG3AQOB+vi2rLSJ1ILEeUCbxxdWilIa+4YYbmD17dtrt1EZN4bhk8+PfE8UX67v44osjldfIZFf7ML5k9a5K12/xZbeTlQFfvHhxpCqsEyZMYOrUqVRUVOzwtHS2Rbn7qLm7/8XMLKh5NMHMXgZuzHJsIjlRl8+zQLRnWmqqk9ZGlNLQN910U623n2333HPPbq2/fft28vLydrkP40tW16Vkf/Djy4AvXryYyspKTjvttKzsf3dFOVLYYmZNgHfNbJyZDQW+m+W4RPYq8WWxBwwYwPDhw+nWrRujR4+m5gHT5557jm7dutGvXz8ee+yxcN2a0tCbNm2isLAwrHf05Zdf0qFDB7Zu3brDL/NU26kp4VyjR48eYfG3H/zgB/Tp04fu3bszZcqUjN8nsbSzuzNu3DiKioo4/fTT+fjjj8O2AwYMoLKyku3bt1NRUUGPHj0oLi4On6RetWoVJ554IiUlJfTu3Zv33nuPuXPnMnDgQEaNGkVxcfFOfRiltHj89x0wYAA/+clPKC8vp2vXrrz88stA6tLiAJ9//jlDhw6lqKiISy+9NOz3ZEdPNWXAv/nmG2644QZmzJhBaWkpM2bMoEuXLuHwmdXV1Rx++OF88skntGjRgubNm9OqVSv22WefjH1eV6IcKVwF7AdcAdwMnAD8azaDEtmbvfHGGyxbtoz27dtz3HHHMX/+fMrKyhg7dixz5szh8MMP36G8RI1WrVpRUlLCvHnzGDhwIE899RSnnHIKzZo1C9ts2bIl43aSuffeeznooIP46quv6Nu3L8OGDQtrHiWTWNr58ccf5+233+att97io48+oqioiAsvvHCHdRYvXsy6detYunQpQHgKZvTo0YwfP56hQ4eyZcsWqqur+eCDD3jttddYunRp0gHqa1NafNu2bbz22mvMmjWLn/3sZ8yePTtlaXGA1157jeXLl3PooYcyaNAgHnvssYwjxe2zzz7cdNNNVFZWhnWiVq5cybRp07jqqquYPXs2JSUltG7dmmuvvRYg8n+jupLxSMHdX3f3L9y9yt3HuPtZ7v5qfQQnsjcqLy+noKCAJk2aUFpaytq1a1m5ciWdOnWiS5cumBnnnXde0nVTlbWuEXU7iW6//XZKSko4+uij+eCDD8JBbqJ66aWXGDlyJHl5ebRv354TTjhhpzadO3dm9erVXH755Tz33HO0bNmSzZs3s27dunCwmfz8fPbbbz8g1k/JEgLUrrR4zQA28eWx05UWLy8vp3PnzuTl5TFy5Ej++tfaPb514YUXhtc27r333nDMhFzJmBTMrKuZ3W1mfzazOTWv+ghOZG+UWKq5ppxzlPLLgwcP5tlnn+XTTz9l4cKFSf/4ptpOfKltiB1VQOx0zOzZs3nllVdYsmQJvXr1CpftikzxH3jggSxZsoQBAwYwadIkLr74YtLVZotSHhtSlxZPtU58n8eXFq+srNyh/Hhdlcfu0KEDbdu2Zc6cOSxYsKBexkxIJ8o1hUeI1Tq6Dvhx3EtE6km3bt1Ys2ZNeD78oYceStquRYsWlJeXc+WVV3LGGWfsNNBMuu0UFhayaNEiABYtWsSaNWuAWHntAw88kP3224+VK1fy6qu7fqLge9/7HtOnT2f79u18+OGHvPjiizu1+eSTT6iurmbYsGHcfPPNLFq0iJYtW1JQUMATTzwBwNdff73DsJfZlqq0OMROH61Zs4bq6mpmzJhBv379Im0zWVnyiy++mPPOO48RI0akHByovkRJCtvc/S53f83dF9a8sh6ZiITy8/OZMmUKp59+Ov369ePQQw9N2TZZWeso2xk2bBiffvoppaWl3HXXXXTtGrtratCgQWzbto2ePXty/fXXc/TRR+9y/EOHDqVLly4UFxdz2WWX0b9//53arFu3jgEDBlBaWkpFRQW/+tWvAHjggQe4/fbb6dmzJ8ceeyz/+7//u8v7r61UpcUhNtjQ+PHj6dGjB506ddphPOV0Bg4cyPLly8MLzRA7wvviiy9yfuoIopXOngB8TKzUxdc18929fke0CKh0ttQ1lc6WXKusrOTqq68O73raXdkunV1zp1H8KSMH9ESziMhuuuWWW7jrrrtSjjpX3zImBXdPfnlfRER22/jx4xk/fnyuwwilK519grvPMbOzki1398eSzRcRkcYr3YXmmitBZyZ5RSqWYmaDzOxtM1tlZilToZkNNzM3s4znu0SyIdO1NZHGYnf/LacrnX1j8F6ry+FmlgdMIjYoTxXwupnNdPflCe0OIPa09ILa7Edkd+Xn57Nx40YOPvjgWt9rLtIQuDsbN24kPz8/c+MUopTO/g5wAVAY397dr8iwajmwyt1XB9uZDgwBEssh3gz8F3Bt5KhF6lBBQQFVVVVh/RmRxiw/P5+CgoJarx/l7qNZwKvAW0B1hrbxDgE+iJuuAo6Kb2BmvYAO7v60maVMCmZ2CXAJQMeOHXchBJHMmjVrlrJcgsjeJkpSyHf3a2qx7WTH4eHJrqDy6kSgItOG3H0KMAVizynUIhYREYkgyhPND5jZWDNrZ2YH1bwirFcFdIibLgDWx00fAPQA5prZWuBoYKYuNouI5E6UI4VvgN8AP+XbX/pRHl57HehiZp2AdcC5wKiahe6+CWhdM21mc4Fr3b12jyuLiMhui5IUrgEOd/edx9xLw923mdk44HkgD7jX3ZeZ2U1ApbvP3PVwRUQkm6IkhWVArcoSuvssYheq4+fdkKLtgNrsQ0RE6k6UpLAdWGxmL7JjQbxMt6SKiEgjEyUpPBG8RERkDxelIN599RGIiIjkXrqCeA+7+wgze4u45wtquHvPrEYmIiL1Lt2RwpXBe6TidyIi0vilK4j3YVDU7vfufmI9xiQiIjmS9olmd98OfGlmreopHhERyaEodx9tAd4ysxeAf9bM1C2pIiJ7nihJ4ZngJSIie7goSeFRYEtwKqlm8Jx9sxqViIjkRJQqqX8BmsdNNwdmZyccERHJpShJId/dv6iZCD7vl72QREQkV6IkhX+aWe+aCTPrA3yVvZBERCRXolxTuAp4xMxqBshpB5yTvZBERCRXotQ+et3MugFHEBtic6W7b816ZCIiUu8ynj4ys7OJXVdYCgwBZsSfThIRkT1HlGsK17v7ZjPrB5wC3Afcld2wREQkF6Ikhe3B++nAXe7+JLBP9kISEZFciZIU1pnZ/wAjgFlmtm/E9UREpJGJ8sd9BPA8MMjdPwMOAn6c1ahERCQn0g2y09LdPwfygbnBvIOIjdNcWS/RiYhIvUp3S+ofiQ2ws5DYyGsWt8yBzlmMS0REciDdIDtnBO+d6i8cERHJpXSnj9I+i+Dui+o+HBERyaV0p49+G7znA2XAEmKnkHoCC4B+2Q1NRETqW8q7j9x9oLsPBN4Hert7mbv3AXoBq+orQBERqT9Rbknt5u5v1UwE5S5KsxeSiIjkSpSksMLM7jGzAWbW38zuBlZE2biZDTKzt81slZmNT7L8GjNbbmZvmtlfzOzQXf0CIiJSd6IkhTHAMuBKYmW0lwfz0gqG7ZwEnAoUASPNrCih2RtAmbv3JDbs539FD11EROpalNLZW4CJwWtXlAOr3H01gJlNJ1ZldXnctl+Ma/8qcN4u7kNEROpQultSXyT2kNqn7j68Fts+BPggbroKOCpN+4uAZ1PEcglwCUDHjh1rEYqIiESR7kihglhS2J6mTTqWZJ4nbWh2HrHbXvsnW+7uU4ApAGVlZUm3ISIiuy9dUphL7I/4BtL/wk+lCugQN10ArE9sZGYnAj8F+rv717XYj4iI1JF0ZS52t7zF60AXM+sErAPOBUbFNzCzXsD/EKvA+vFu7k9ERHZT1sZFcPdtwDhiZbdXAA+7+zIzu8nMBgfNfgO0AB4xs8VmNjNb8YiISGYZ7z7aHe4+C5iVMO+GuM8nZnP/IiKyazSCmoiIhJQUREQkpKQgIiIhJQUREQkpKYiISEhJQUREQkoKIiISUlIQEZGQkoKIiISUFEREJKSkICIiISUFEREJKSmIiEhISUFEREJKCiIiElJSEBGRkJKCiIiElBRERCSkpCAiIiElBRERCSkpiIhISElBRERCSgoiIhJSUhARkZCSgoiIhJQUREQkpKQgIiKhrCYFMxtkZm+b2SozG59k+b5mNiNYvsDMCrMZj4iIpJe1pGBmecAk4FSgCBhpZkUJzS4C/uHuhwMTgV9nKx4REcmsaRa3XQ6scvfVAGY2HRgCLI9rMwSYEHx+FLjTzMzdPYtxSQ5MfOGdOtvW1Sd1rbNticiOsnn66BDgg7jpqmBe0jbuvg3YBBycxZhERCSNbB4pWJJ5iUcAUdpgZpcAlwB07Nix1gHpF2buqO9FGodsHilUAR3ipguA9anamFlToBXwaeKG3H2Ku5e5e1mbNm2yFK6IiGQzKbwOdDGzTma2D3AuMDOhzUzgX4PPw4E5up4gIpI7WTt95O7bzGwc8DyQB9zr7svM7Cag0t1nAr8HHjCzVcSOEM7NVjwiIpJZNq8p4O6zgFkJ826I+7wFODubMYiISHR6ollEREJKCiIiElJSEBGRkJKCiIiElBRERCSkpCAiIiElBRERCSkpiIhISElBRERCSgoiIhJSUhARkZCSgoiIhJQUREQkZI1t+AIz2wC8n+XdtAY+yfI+sqUxxw6NO/7GHDso/lyqj9gPdfeMo5Q1uqRQH8ys0t3Lch1HbTTm2KFxx9+YYwfFn0sNKXadPhIRkZCSgoiIhJQUkpuS6wB2Q2OOHRp3/I05dlD8udRgYtc1BRERCelIQUREQnttUjCze83sYzNbmmK5mdntZrbKzN40s971HWMqEWIfYGabzGxx8LqhvmNMx8w6mNmLZrbCzJaZ2ZVJ2jTI/o8Ye4PtfzPLN7PXzGxJEP/PkrTZ18xmBH2/wMwK6z/SnUWMvcLMNsT1/cW5iDUdM8szszfM7Okky3Lf9+6+V76A7wG9gaUplp8GPAsYcDSwINcx70LsA4Cncx1nmvjbAb2DzwcA7wBFjaH/I8beYPs/6M8WwedmwALg6IQ2PwQmB5/PBWbkOu5diL0CuDPXsWb4HtcAf0z2b6Qh9P1ee6Tg7i8Bn6ZpMgS432NeBb5jZu3qJ7r0IsTeoLn7h+6+KPi8GVgBHJLQrEH2f8TYG6ygP78IJpsFr8QLi0OA+4LPjwLfNzOrpxBTihh7g2ZmBcDpwD13jYwGAAAF9klEQVQpmuS87/fapBDBIcAHcdNVNKL/+YFjgsPsZ82se66DSSU4PO5F7FdfvAbf/2lihwbc/8Hpi8XAx8AL7p6y7919G7AJOLh+o0wuQuwAw4JTjo+aWYd6DjGT3wH/AVSnWJ7zvldSSC1Zdm4sv0oWEXukvQS4A3gix/EkZWYtgD8BV7n754mLk6zSYPo/Q+wNuv/dfbu7lwIFQLmZ9Uho0mD7PkLsTwGF7t4TmM23v7pzzszOAD5294XpmiWZV699r6SQWhUQ/yujAFifo1h2ibt/XnOY7e6zgGZm1jrHYe3AzJoR+6M6zd0fS9KkwfZ/ptgbQ/8DuPtnwFxgUMKisO/NrCnQigZ2ujJV7O6+0d2/DibvBvrUc2jpHAcMNrO1wHTgBDN7MKFNzvteSSG1mcAFwV0wRwOb3P3DXAcVhZn9S815SDMrJ/bfeWNuo/pWENvvgRXu/v9TNGuQ/R8l9obc/2bWxsy+E3xuDpwIrExoNhP41+DzcGCOB1c+cylK7AnXnQYTu+bTILj7f7p7gbsXEruIPMfdz0tolvO+b1qfO2tIzOwhYneJtDazKuBGYheucPfJwCxid8CsAr4ExuQm0p1FiH04cJmZbQO+As5tCP9TxzkOOB94Kzg/DPD/gI7Q4Ps/SuwNuf/bAfeZWR6xZPWwuz9tZjcBle4+k1jSe8DMVhH7lXpu7sLdQZTYrzCzwcA2YrFX5CzaiBpa3+uJZhERCen0kYiIhJQUREQkpKQgIiIhJQUREQkpKYiISEhJQXLOzL5IMu9SM7ugnuOYa2ZvByUSVprZnTX3xQfL/1YH+ygzs9t3cZ17zKxod/edsM3vmNkP63KbsmfQLamSc2b2hbu3qOd9GrF//9Vx8+YC17p7pZntA/wKKHP3/nW0z6ZBPZucC+o2Pe3uiWUiZC+nIwVpkMxsgpldG3yea2a/Dmrpv2Nmxwfz88zsN2b2evDr/t+C+S3M7C9mtsjM3jKzIcH8QouNg/DfxOoTpSyW5u7fECtc1tHMSoL1vwje25nZSxar1780Lp5BwT6XmNlf4r7HFDP7M3C/xcZaeDpu2X1m9mczW2tmZ5nZfwUxPxeU06j5/mU1MZjZL4J9vGpmbYP5Z1qs/v4bZjY7bv4Ei42/MdfMVpvZFcFXvAU4LPgOv0n1nWTvo6QgjUVTdy8HriL2BDfARcTKX/QF+gJjzawTsAUY6u69gYHAb2vKTgBHECvJ3cvd30+3Q3ffDiwBuiUsGgU8HxRmKwEWm1kbYrV2hgWF8M6Oa98HGOLuo5Ls5jBipZSHAA8CL7p7MbEnoU9P0n5/4NVgHy8BY4P5fyU2tkAvYnV1/iNunW7AKUA5cGOQbMYD77l7qbv/ONl3Stc3sufaa8tcSKNTU3huIVAYfD4Z6Glmw4PpVkAXYkXFfmlm3yNWovgQoG3Q5v1gfIaoklWtfB24N/jj+oS7LzazAcBL7r4GwN3ji5jNdPevUmz/WXffamZvAXnAc8H8t+K+Z7xvgJoRuxYCJwWfC4AZQe2ffYA1ces8ExSJ+9rMPubbvkj7nVLEK3s4HSlIY1FT+XI73/6YMeDy4Nduqbt3cvc/A6OBNkCf4JfvR0B+sM4/o+4wqLFTTEJRtWCQo+8B64jVqbkgiCXVBbp0+/w62GY1sDWuRlI1yX+0xbeJ74s7iI04Vgz8G99+33AfSdbJ9J1kL6SkII3Z88QKz9Wce+9qZvsTO2L4OPgFPhA4dFc3HGzzV8AH7v5mwrJDg+3fTayAWW/gFaB/cPoKMztoN75XbbQi9gcdvq2ymc5mYsOJAim/k+yFdPpIGoL9LFbttUaqctqJ7iF2imVRcM1gA/ADYBrwlJlVEjs3nlgaOp1pZvY1sC+xQVqGJGkzAPixmW0FvgAucPcNZnYJ8JiZNSE2MthJSdbNlgnAI2a2DngV6JSusbtvNLP5ZraU2FjYS0n4TlmOVxoo3ZIqIiIhnT4SEZGQkoKIiISUFEREJKSkICIiISUFEREJKSmIiEhISUFEREJKCiIiEvo/C+cWkZJ0EgsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xb487198>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tot = sum(eigen_vals.real)\n",
    "discr = [(i / tot) for i in sorted(eigen_vals.real, reverse=True)]\n",
    "cum_discr = np.cumsum(discr)\n",
    "plt.bar(range(1, 5), discr, width=0.2,alpha=0.5, align='center',label='individual \"discriminability\"')\n",
    "plt.step(range(1, 5), cum_discr, where='mid',label='cumulative \"discriminability\"')\n",
    "plt.ylabel('\"discriminability\" ratio')\n",
    "plt.xlabel('Linear Discriminants')\n",
    "plt.ylim([-0.1, 1.1])\n",
    "plt.legend(loc='best')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Project original features onto the new feature space"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "W=np.hstack((eigen_pairs[0][1][:, ].reshape(4,1),eigen_pairs[1][1][:, ].reshape(4,1))).real\n",
    "X_train_lda = X_train_std.dot(W)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Plot transformed features in LDA subspace"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>LD1</th>\n",
       "      <th>LD2</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.790936</td>\n",
       "      <td>0.115414</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.583202</td>\n",
       "      <td>-0.265463</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.663881</td>\n",
       "      <td>-0.083657</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.514646</td>\n",
       "      <td>-0.225901</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.807076</td>\n",
       "      <td>0.189973</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        LD1       LD2  class\n",
       "0  1.790936  0.115414      0\n",
       "1  1.583202 -0.265463      0\n",
       "2  1.663881 -0.083657      0\n",
       "3  1.514646 -0.225901      0\n",
       "4  1.807076  0.189973      0"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data=pd.DataFrame(X_train_lda)\n",
    "data['class']=y\n",
    "data.columns=[\"LD1\",\"LD2\",\"class\"]\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAFgCAYAAACFYaNMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3X10XPV97/v3d/SAFNkOtpEDkSCQwEnAkEuCwFBwk8PNg5NzlskN0ALNDU7IcsKCNDm5N6vE6Wp601OO27N6wrlASXwCNmX1QAJuT3xXKZSE0BoCLkpCwQ6QGMKDZIyFbfCTZD3M9/6xZ0tboxnNSPOw9+z5vNbSkmbPntFPy+LLT9/f9/f9mbsjIiL1l4l7ACIizUoBWEQkJgrAIiIxUQAWEYmJArCISEwUgEVEYqIALCISEwVgEZGYKACLiMSkNe4B1NOqVav8gQceiHsYIpJ+Vs5NTTUDfuONN+IegojIpKYKwCIiSaIALCISEwVgEZGYNNUinKTb2NgYAwMDjIyMxD2UWXV0dNDb20tbW1vcQ5GYxRqAzewO4D8Ce9z9zALPG/DfgU8CR4A17v6L3HNXA3+cu/U/u/ud9Rm1JNXAwAALFy7k5JNPJvjVSR53Z+/evQwMDHDKKafEPRyJWdwpiE3Aqlme/wRwWu5jLXAbgJktAb4FrADOA75lZotrOlJJvJGREZYuXZrY4AtgZixdujTxs3Spj1gDsLv/C7BvllsuAf7GA08Ax5rZCcDHgYfcfZ+77wceYvZALk0iycE31AhjlPqIewZcSg/wauTxQO5aseszmNlaM+s3s/6hoaGaDVREZK6SHoALTRV8luszL7pvcPc+d+/r7u6u6uBECnnggQd473vfy6mnnsr69evjHo4kWNKrIAaAEyOPe4Fduesfzrv+SN1GJQ1v9c2Psvfw6IzrS7va2fLli+b9vhMTE1x33XU89NBD9Pb2cu6557J69WrOOOOMSoYrFarVv3elkj4D3gJ81gLnA2+5+2vAg8DHzGxxbvHtY7lrImXZe3iUJV3tMz4K/Uc6F//6r//Kqaeeyrvf/W7a29u54oor+NGPflSlUct81erfu1Jxl6HdTTCTPc7MBggqG9oA3P27wP0EJWg7CcrQPpd7bp+Z/RnwZO6tvu3usy3midTF4OAgJ5449Udbb28v27Zti3FEkmSxBmB3v7LE8w5cV+S5O4A7ajGupNg6sJVNOzYxeGiQngU9rFm+hpW9K+Melswi+JWdTlUPUkzSUxBNa+vAVm7cdiNDw0Msal/E0PAQN267ka0DW+Memsyit7eXV1+dKtAZGBjgne98Z4wjkiRTAE6oTTs20dbSRmdrJ2ZGZ2snbS1tbNqxKe6hySzOPfdcfvOb3/Db3/6W0dFR7rnnHlavXh33sCShkl4F0bQGDw2yqH3RtGsdLR0MHhqMaUTpsrTIAszSrvaK3re1tZVbbrmFj3/840xMTPD5z3+e5cuXV/SeUrla/XtXSgE4oXoW9DA0PERna+fktZGJEXoWFNxvInNUy9KjT37yk3zyk5+s2fvL3MVZajYbpSASas3yNYxNjDE8Poy7Mzw+zNjEGGuWr4l7aCJSJQrACbWydyXrVqyju7ObA6MH6O7sZt2KdaqCEEkRpSASbGXvSgVckRTTDFhEJCYKwCIiMVEAFhGJiQKwNK/8bcMFthHP1ec//3mWLVvGmWfOOGFLZAYFYGlO/Rvh8Vumgq578Lh/Y0Vvu2bNGh544IEqDFCagQKwNB93GD0Ez9w3FYQfvyV4PHqoopnw7/7u77JkyZIqDlbSTGVoDUTd0arEDC64Pvj6mfuCD4CzLguuq3uZ1IkCcIJFA25XWxd7j+xlUceiad3R1qHNGfMSBuEw+IKCr9SdUhAJld+O8uUDL/PW6FuMT4yrO1o1hGmHqGhOWKQOFIATKr8dZdazZCzD3pG9k/eoO9o8RXO+Z10Gax8JPkdzwiJ1oACcUIOHBulo6Zh83JZpw3HGsmOT19QdbZ7MoH3B9JzvBdcHj9sXVJSGuPLKK7ngggt4/vnn6e3t5fbbb6/iwCVtlANOqPx2lMd1HsfgocEgELszMjFSsDuaFurK1Pe5YKYbBtswCFeYA7777rurMDhpFpoBJ1R+O8qWTAuLj1nMSQtPKtodTccYzVF+sNUCnNSZZsAJtbJ3JetYN202+/W+r886m43mjYHJz5t2bNIsWCSBFIATbK7tKHWMkUhjUQoiRXoW9DAyMTLtmhbqRJJLAThFdIyRSGNRAE4RHWMk0liUA04ZHWMUr1dffZXPfvaz7N69m0wmw9q1a/nKV74S97AkoRSApWnVoma6tbWVv/qrv+KDH/wgBw8e5JxzzuGjH/0oZ5xxRpVGLWkSawrCzFaZ2fNmttPMbijw/HfM7Kncx6/N7M3IcxOR57bUd+TS6GpVM33CCSfwwQ9+EICFCxdy+umnMzioKhQpLLYZsJm1ALcCHwUGgCfNbIu7/yq8x93/U+T+LwMfiLzFsLufXa/xSrrUo2b6pZde4pe//CUrVqyoyvtJ+sSZgjgP2OnuLwKY2T3AJcCvitx/JfCtOo2t6rRFOFlqXTN96NAhLr30Um666SYWLVpU+gXSlOJMQfQAr0YeD+SuzWBm7wJOAR6OXO4ws34ze8LMPlXsm5jZ2tx9/UNDQ9UY95xpi3Dy1LJmemxsjEsvvZQ/+IM/4NOf/nTF7yfpFWcALrTxvlgfwCuA+9x9InLtJHfvA64CbjKz9xR6obtvcPc+d+/r7u6ubMTzlN9aUr1841ermml355prruH000/na1/7WnUGK6kVZwAeAE6MPO4FdhW59wpgWpspd9+V+/wi8AjT88OJkt9aErRFOG61qpl+7LHHuOuuu3j44Yc5++yzOfvss7n//vurNGpJmzhzwE8Cp5nZKcAgQZC9Kv8mM3svsBh4PHJtMXDE3Y+a2XHAhcBf1mXU85DfWhK0RTgJalEzfdFFF+Fq6C5lim0G7O7jwPXAg8CzwA/dfYeZfdvMVkduvRK4x6f/Vp8O9JvZvwE/BdZHqyeSRluERaSQWDdiuPv9wP151/4k7/GfFnjdz4Czajq4KirUWrKWVRDlVFyoKkMkftZMfy719fV5f39/3MOoqbDioq2ljY6WjsmTM6L5zXLuaUTPPvss73vf+7CEN1Z3d5577jlOP/30uIcitVPWL6Ga8aRMORUXaa3K6OjoYO/evYnOwbo7e/fupaOjo/TNknrqBZEy5WwwSGvj9t7eXgYGBoir3rtcHR0d9Pb2xj0MSQAF4JQpp+IirVUZbW1tnHLKKXEPQ6RsSkGkTDkVF6rKEEkGBeCUKWeDgRq3iySDqiBEJBVW3/woew+Pzri+tKudLV++qOL756isKgjlgEUkFfYeHmVJV3vB69W4vxaUghARiYlmwA1CO9dE0kcz4AagfsIi6aQA3ADCnWsT2QlePvAyAwcHGBoe4qZf3BT30ESkAkpBNIDBQ4NkyLD7yG7MjBZrYcIneOHNF9g6sFWpCBGC6oViVQ3VuL8WFIAbQM+CHv5t6N8wMzK5P1oMoy3TVtVDJEUa2VxLx6pQalYxpSAawJrlaxjPjoMHzVyynsVxjus8ruH7N4g0MwXgBrCydyXveft7yFiGLFlaM60c/7bjaWtpa/j+DSLNTCmIBvHVc75asIfvbP0bVLomkmzaitxA8gNq3zv66H+9v2CATWvTdZEGUdZWZAXgBlUqwF7z4DUzWk4Ojw/T3dnN7R+/PcaRiyRXFftD6ESMNCt1qsXgoUE6WqafupCGpusitRT2h8j/qFV/COWAE65YHrfUqRZpbboukiaaASfYbFuQexb0MDIxMu3+aIBV03WR5FMATrDZ0gylAqyaroskn1IQCTZbmmFl70rWsW7WMrOVvSsVcEUSTAE4wUrlcRVgRaqr3v0hFIATbM3yNdy47UaAsjdflEMbNKQR1PjIoILq3R9COeAEq0UeV72FpVHUuyQsDrEGYDNbZWbPm9lOM7uhwPNrzGzIzJ7KfXwh8tzVZvab3MfV9R154ypVPywi9RNbADazFuBW4BPAGcCVZnZGgVt/4O5n5z6+n3vtEuBbwArgPOBbZra4TkOvm1rMVrVBQyQ54pwBnwfsdPcX3X0UuAe4pMzXfhx4yN33uft+4CFgVY3GGZtazFZL1Q+LSP3EGYB7gFcjjwdy1/JdamZPm9l9ZnbiHF/b0GoxW9UGDZHkiDMAF2pWkd8Z6P8DTnb39wM/Bu6cw2uDG83Wmlm/mfUPDQ3Ne7BxqMVsVRs0pFEs7Wpn3+HRGR/1PDKo1uIsQxsATow87gV2RW9w972Rh/8D+IvIaz+c99pHCn0Td98AbICgG1olA663WpWhlVs/rHI1iVMSjgyqtThnwE8Cp5nZKWbWDlwBbIneYGYnRB6uBp7Nff0g8DEzW5xbfPtY7lqqxDlbVbmaSO3FNgN293Ezu54gcLYAd7j7DjP7NtDv7luAPzSz1cA4sA9Yk3vtPjP7M4IgDvBtd99X9x+iDuLa7RZdAAQmP+sQUJHqUUP2lKo0fbBq8yoWtS/CbCrd7u4cGD3AA5c+UIshi6RJWQ3ZtRU5haKnZUTTB+soP32hfsJSiTi2ETfSeELaipxC1agfVrmaVCJp24iTNp6QZsApVOq0jHKU0+5SpFaSOmOtNgXgFKpW+iDWdpfuEMk/z3gsqRbOWAtdTxOlIFKo4dMH/Rvh8VuCoAvB58dvCa6LpIhmwCnU0OkDdxg9BM/cFzy+4Pog+D5zH5x1mWbCTWznnkOMjme5cP3D0643clpCATilZksfJHqHm1kQdCEIumEgPuuy4LqCb0OoxckS49ksmQwzUhPlpCXqfdJFuRSAm0w1StRqLgzCYfAFBd8Gk7QZadLGE1IOuMk0REP2MOcbFc0JS+oVasSTzUJrJl0hSzPgJlONErWaCoNvmPON5oBBM+EmUWjGeuH6hwtWRjQyBeAmU5MdbtUsGTOD9gXTc75hTrh9gYJvg2uW+t5yKQA3maq3uOzfGFQthMEynMG2L4C+z83vPfs+Nz2Ih0FYwbfhVVLfm9SFtEooADeZqpao1bJkLP91Cr5NL40zZAXgJlS1HW4qGZM6S1sKQwFYKqOSMamjtG1RTldNh9SfSsZE5k0zYJk/lYzJHKVxIa0SCsAyfyoZkzlqxDxtLSkAS2VUMiYJ0miLdArAUjmVjEmdlEphNNoinQJwwiW6c5lInSVxFlsJVUEkWNi5bGh4aFrnsq0DW+MemohUgWbACRbtXAZMft60Y1NZs2DNnkWSTQE4wV548wWOjB1h3Mdpy7RxXOdxdLV1ldW5rNy+v4kJ0joDTpqQAnBCbR3YyqGxQ0z4BK3WyriP89rh11jSsYR3LXpXydeXM3tOTHP2WjT0kVQqVeXQaHXGCsAJtWnHJo495lj2jewj61kMI0uW/Uf3863l3yr5+nL6/laa4qgKnQEnc1CqyqHRFukUgBNq8NAgSzqW0J5pZ+/IXsayY7Rn2ulo6SgrOJbT9zcRzdnV0EeamKogEqpnQQ8jEyMsPGYhJ7/9ZE5bfBrv6HoHpy4+tazXl3M0ffg9oipuzj4f0SAcUvCVJhBrADazVWb2vJntNLMbCjz/NTP7lZk9bWY/MbN3RZ6bMLOnch9b6jvy2isngM5mZe9K1q1YR3dnNwdGD9Dd2c26FdNzu5V+j6pRQx9pUuYx/ZKbWQvwa+CjwADwJHClu/8qcs+/B7a5+xEzuxb4sLv/fu65Q+6+YC7fs6+vz/v7+6v2M9RaPSoUYq+CmK2hj9IQkhMuvu1+a4TouZytmQynLlvAvsOjPHbDxfENcKayfmnjzAGfB+x09xcBzOwe4BJgMgC7+08j9z8BfKauI4xZ1Rqnx/w9ZqWGPlKGcPFt3+FRxrNZAMYnnLGJCZ4ZfAuA93zjfgAyGTj9+EUNsSAXZwDuAV6NPB4AVsxy/zXAP0Yed5hZPzAOrHf3/1XoRWa2FlgLcNJJJ1U0YKkRNfSRMp26bOqP3ud2H2Bswulsa5l2z3g2m9jeD/niDMCF/usqmA8xs88AfcCHIpdPcvddZvZu4GEze8bdX5jxhu4bgA0QpCAqH3bjij3dMBs19JEmFGcAHgBOjDzuBXbl32RmHwG+CXzI3Y+G1919V+7zi2b2CPABYEYAlkC1N10kOphLakRzv3sOTlXstGbSUcAVZwB+EjjNzE4BBoErgKuiN5jZB4DvAavcfU/k+mLgiLsfNbPjgAuBv6zbyBtQNTddJGYHnaRKoV1uu98aob01QyYzPeiGeeBGF1sAdvdxM7seeBBoAe5w9x1m9m2g3923AP8VWADca8GfpK+4+2rgdOB7ZpYlKKVbH62eSJNqzTSruekiETvoJHUK7XLbc3CE8WyW1kxmWtANv2z0RFWsO+Hc/X7g/rxrfxL5+iNFXvcz4Kzaji5+1ZxplrMzrlyJ2EEnTSW6+Aaw7/AoS7vaeXb3AUbHp8+GM5nk9n7Ip63ICVbNmeaa5Wu4cduNQBAsRyZG5r3poprBXGSudu45NBl0ly3smLye1GOHZqMAnGDVnGmu7F3JOtZVnM7YOrCVN0fe5JUDr9CaaWVZ5zJaW1prv4NO7SolZzybJZNhRrqiUUrPohSAE6zaM81KN11EUyIndJ3AG8NvsOvwLt5z7Hv4et/Xa5f/VbvKVJjPgZmtmQyj41n2RV6XzUJ7q6ogpMaqmTaohmhKpLO1k0XHLGJ4fJhjjzm2dsFX7SpTo1QryUK9fJd0tc8I0Beuf7jg+zQiBeAEK5U2qHctbiyLb2pX2VBmm+WW0mj522pQAE64YmmDOGpxY1t8C4NwGHxBwTeh4jwWfs/BES5c//CM60lenEtHIqUJRdMBZkZnaydtLW1s2rGpZt8ztvaValcpEUtzTXnyPyBIWeR/JHlxTjPgBhVHOqBalRRzMlu7StBMuAkVm80Wmv0mnQJwg4orHVD39pVqV5kaxQ7M3Hd4tOFSB9WiANygklYhUVNqV5kKs81c48obx00BuEHFkg6IizZhNIxaHwtfSZVFEikAN7BapAMS12ZSmzAaSq1TBrNVWdQ6+NeCArBMSlybSW3CkDloxHyxAnCDqeUMNXFtJrUJQ1JOdcANJJyhDg0PTZuhbh3YWpX3Hzw0SEdLx7RrsbeZjAbhkIJvqhSr601y6qBaNANuILWeoSayzWSxTRgKwqlRTupgtqOJ8nsFNxLNgBtIrWeose10KyZ/E8baR4LPz9ynnXBNJlx8y++CFnZKa9TZsmbADaTWM9TElbZpE4bkKXQyxmM3XBzTaCqnANxAarH5otCi3u0fv31O908G6FrU62oThqSYUhANZGXvStatWEd3ZzcHRg/Q3dnNuhXzLxGb66LerPf3b5yeFgjTB/0b5/vjTskPtgq+khKaATeYam6+mOuiXvH7N7Jy4QdUrysyR7MGYDNrAb4A9AIPuPtjkef+2N3/c43HJzU0145qxe/fBR/LpS2qVa8713TGXO7X1uaGE56AnJ1+ADKZTFAh0YibMKB0CuJ7wIeAvcD/a2b/LfLcp2s2KqmLngU9jEyMTLs226LerPdXs153rumMudxfy1SJ1MyWL1/EsoUdnPHORdM+3nf8ooZu2lMqAJ/n7le5+03ACmCBmf2dmR0DaMrQ4OZadjbr/dVqmh7dfhy+PkxnjB6a+X5zuX+u7y1SY6VywJPFde4+Dqw1sz8BHgYat/pZgLmXnRW9v+ei6jVNn8v24zB1UO792tosCVMqAPeb2Sp3fyC84O7fNrNdwG21HZrUw1wX9YreX069bhgwi30OlXMGXH6XtPOvgyf+GqwF3ra0eEDV+XKSILMGYHf/TJHr3we+X5MRSWMqVa8bBsy2Lhg7nAuYtwYBevTQ9PaSpbYf53dJO/86uOdKGN4PnYtn3h+lrc0NYfXNj85YdJtwZ9ebw3S0tTT09uOokmVoZrYUuAp4X+7Ss8Dd7r630m9uZquA/w60AN939/V5zx8D/A1wDsFC4O+7+0u5574BXANMAH/o7g9WOh6pULF63WjAXHg8HHgNXnoUDu6eevz+y6dysOWkM8IZ9tP3wuN/DSP74ZQPwRV3B4G9UPpD58s1jHBhLbr1eGR8IvhVym0/DjXqNmQoXYZ2OkG+90HglwQLb+cC68zsYnd/br7fOFfidivwUWAAeNLMtrj7ryK3XQPsd/dTzewK4C+A3zezM4ArgOXAO4Efm9m/c/eJ+Y5Haig/93pkCPb+GjoWB0Hx/ZdPD37lpDOiqYRMJnivK+4Ovi62XVlbmxtaR2sL49ksyxZ2NPT246hSM+A/A77i7j+MXjSzS4E/By6t4HufB+x09xdz73kPcAkQDcCXAH+a+/o+4BYzs9z1e9z9KPBbM9uZe7/HKxhPat321G3c9exdHBk7QntLO4uPWYyZ0dXWBQ6Hxw/Xvu9DNGB2LQvSBV3dhbcWl7P9OJpKeNtxweMnbp0eWAsFVG1tlgQpVYZ2Vn7wBXD3zcCZFX7vHuDVyOOB3LWC9+SqMN4Clpb5WgDMbK2Z9ZtZ/9DQUIVDbjy3PXUb33v6ewyPD2MYw+PD7Dq8iwNHD/DCmy/w4lsvYljVewvPEA2Yh/fkPg9NXc+vsM9XKPhGu6S9//Lp5WWzBVRtbZaEKDUDPjzP58pR6Lc+vxCz2D3lvDa46L4B2ADQ19fXdIWedz17F2ZGq7Uymh3FMBzn4NhB2lvawWHfyD5OfvvJQI1Ov4gGzIXHB4+Pf/9UDviJ7wY54TCFUOrcN6USUqXQQZu73xoh605ryrvVlArAy8zsawWuG9Bd4fceAE6MPO4FdhW5Z8DMWoG3A/vKfG3TKdSp7MjYEVot+Gf2vP9HZcjgOGPZMaCGp19EA2Z+FURbV3DP69unUgjl9JHo+1wwa46mEs6/Lgjg0lAKHbS57/Aow2MTjI5P/8sok2nsRbd8pQLw/wAWFnmu0jK0J4HTzOwUYJBgUe2qvHu2AFcT5HYvAx52dzezLcD/zG2NfidwGvCvFY6noRU7ULO9pZ2x7BittE7OfkNZgl/utkwbUHgbctXOoIvmXqMbKMzgnDVTlQvlbo4odFpyWNam05Ib3qnLFvDc7gMsW9hR+uYGVqoO+P8p9pyZfbWSb+zu42Z2PUGFRQtwh7vvMLNvA/3uvgW4Hbgrt8i2jyBIk7vvhwQLduPAdc1eAVGsU9niYxbz+pHXGWecjGUY93EAFrYt5Mj4EQxjSceSgtuQq35KcnS2Gv0cVi6UuzmiUB1wGMDPuiyYGWsm3PCyWSZnxjv3HGI8t06w+60RLlz/MBDMhhu1EQ9U1o7ya8BNlXxzd78fuD/v2p9Evh4BLi/y2j8nqMQQincqG50Y5Yvv/+JkFURna+dkFcQJC06YrILo7uyeMbutyynJ0drf8LHZ7Jsjojnfbd8N6oC7uoOFuOgGD82EG0400E6489zuAwCMTzgdbS3B12QnA3MjN+KBygKwVjqqoFp/4s92XNG1Z1/LtWdfO+f3nGu7yjnr3whHDwZfb98MZ+aqGgf7S2+OCHO+4SYMmDkTVpvJhjOezdKa++tlbGJi2tdpVMnfaU1XUVBt1TxmvhYHas61XeWchGmE7ZthoH8q+G7fDD19wePZKhrCnG9Xd7AJY2Q/3LR8+i43Bd+GED2WPpsNgvB4NtsUM7xSO+EOUjjQGtBZ4LrMQTX/xC/V2Ww+M+1anEE3KX933JsvB1+HwTO8p5BoWVuYdrhpebC5A4LHCr4NI5rDvXD9w5PphTD9kGalFuGKVUBIFVT7T/xincrmu5hW81OS59uZLFrWFqYdupblnmuZviNOGsqegyPsORj81TU+4dNSD2FuuDVFC6w6Ey5GtT5mPlTJTLuaZ9DNUElnsrAOOJrzjeaAQUG4QUQ3YoxN+FTqwaCzNVh4Gx6bmFaSFjbjafSaYAXgGNX0T/yImi+mzUc1OpNlMtoRlwLRjRi73hyevO4+Netta7HUNOCJUgCOUc3/xM+p10x7Tqq1nVjNdVKltcUmUwzj2SzvOz6YOOxr8HKzYhSAY1bTP/Fz6jXTnrNCwTN/O3E5pWRqriMNKj3ZbClqZe9K1q1YR3dnNwdGD9Dd2c26FfPczVZt+ccMPXHr9BOLf3bz9BOLdXBmqqy++VF2vzXCc7sP8NzuA4xNOCNjE4yMp7PuN59mwCmXX372zRXfTEbgzZe/vfiC6+GHV8Pgk7Di2uk75rTLLTX2Hh4lkyGSdpgAD/65s9n0LLYVowCcYsXKz1a/sZr+1/trmnees0InFh/eM70KvZwuaZIohVpNwvQeDq2ZTKTELPg3zWZh+TsXNXSfh3IoAKdYofKzI2NH+P7279OzoKc6DXaqKb8uuGtZsCNu++bgA7TLrcEUajUZXg8VOmBz3+HR1AdfUA441QYPDdLRMr2d38HRg0xkJ+hs7cTM6GztpK2ljU07NsUzyKhCdcHh9ZCCr6SIAnCKFerlMDoxSntm+owk9ppgKHzM0JmXwrbbpo4ugqkjh0RSQAE4xQo16GnJtLDomOmbMmKtCQ6DaVgXfOalU7lgCHLAJ54LX/znIDA/c19QGRENwgrIDSvaiCf6kdZFt3zKAadYoY0enzj5E2x5YQvD48Px1wTnn2pxzpoguP58U1DlcMxCOP9a+J0vT+WHB54MuqeFCp0fl79ApwW7xGqGPO9sFIBTrtBGjzOPO7Pmu+9KKlR29vgtwWJbWOWQv1EDglaV2zdP9YzIr4z4+aaZRxWpdC02S7vaJxfc9hwcmTz8OpMhNadaVEIBuAnNtvuuamfAlVKo7AxmVjlEg6/Z1Gw4+powbREG9afvDa6rQXvsirWajGr0Uy0qYd5E+bO+vj7v7+8vfWOTitYNR9MTNd015w4bPjz1eO0jpYNk+Bp3GN4H560N0hVHDwZB9wdXwe5nguczGVjxJVVPJMCF6x9m3+HRyZrfUEprfsv6ZdMinEyK1g3XpUStWDvK2SYF4WuOvBFUR2TH4Ym/hufuhydug++thAOvBcF4eH/wX7catCdGeORQ9COTad5ZsAKwTCpUN1yzErVCZWdhlUOxIBy+5ul7YdlyWLAMjv+M0X3eAAAUy0lEQVTfgude+VkQkN94Hva9CBNj0Lk4OLIo2l9CJEGUA5ZJdW1bOZ92lOFrlpwcLMZBsCDnwNgRaF8Io4eDWfHbFsNXd6hBuySaArBMqnvbynNy7xsNiqWC5DlrgvRCeIqyO4wdDj57Nsi8tbQF25ifuDVIP4AatCfA0q52dr81wjjTc8BpOmJorhSAZVK9GsQD02uAofxysbASAoKcb5gHbmkPXvuu34GDu2Hh8VMz3/wewxKLLV++qGglRFobrpeiACzT1KNBfNEa4LmWi43sh/a3QVtXkJLY1Q8n51bS2xcEM+P2BQq+CbH65kfZc3CE3W9N3x6fycDpxy8q8qp0UwCW+iu3Bng2A/25Rbbcacgnngu9fUE52jlrpjZhKO2QGHsPj04eMRTVLJ3PCtHUQOIRDcKhcoJvmKp48+WgUfvaR4LAvX3z1HbmQps4JDF27jk0eQLGc7sPsPutES5c/zCrb3407qHVXSwB2MyWmNlDZvab3OfFBe4528weN7MdZva0mf1+5LlNZvZbM3sq93F2fX+C5Ng6sJVrHryGVZtXcc2D17B1YGvcQyrPfGqAYXr1RLRHxFmXBbNfBd3Ey68FzmRgSWTLcjOJawZ8A/ATdz8N+Enucb4jwGfdfTmwCrjJzI6NPP91dz879/FU7YecPOHOtaHhoWnN1RMfhOdTAxzV97mZ25UvuF69HhJuz8GR6ee+5T7GJ5q3RjuuHPAlwIdzX98JPAL8UfQGd/915OtdZrYH6AberM8QA3XrjTAPhU68CK8nZYwFVeNIep2E3HCyWWhvzTA+MTFto24z75GJpReEmb3p7sdGHu939xlpiMjz5xEE6uXunjWzTcAFwFFyM2h3P1rktWuBtQAnnXTSOS+//HLZ44ylN8IcrNq8ikXti7BI8HF3Dowe4IFLH4hxZGVqhLaRjTDGBnHaN+8HYGzCJ+NvGH3aWoxsFo5/e7ATMwUd0sr6JanZDNjMfgwcX+Cpb87xfU4A7gKudvewgvsbwG6gHdhAMHv+dqHXu/uG3D309fXN6f82SZ9h1nXnWi3MdRZb72CY369YrS0rsmxhB0u62nlu94HJzRcjY8Hx862ZDONkWdLVzs49hyYX5qJSEJRnqFkO2N0/4u5nFvj4EfB6LrCGAXZPofcws0XAPwB/7O5PRN77NQ8cBTYC59XiZ6hrb4R5KHTiRWzN1Wutf+P0/HAYDPs31ub7RWuVw+8b5q1HDzX3380VCk9BHs9mcYJZcLgwR+7rcGEu+pHGRbq4csBbgKuB9bnPP8q/wczagb8H/sbd78177gR3f82Cv70/BWyvxSCTPsOs6861OFVr48ZcVKNWWQqKnoL83O4DAAXrg5tBXAF4PfBDM7sGeAW4HMDM+oAvufsXgN8DfhdYamZrcq9bk6t4+Fsz6ybIszwFfKkWg6x7b4R5qMXOtcQtPMYVDMPvG34/UPCtwNICs9hwYa5ZqSF7CYkLRjU268Jjz0XxBp/5NG+v9PuFM+2QZsBVVag3RLFZ8b7Dozx2w8V1G1uF4l2ES4u69EZIkMmFx6NHwA/SuWBZ7vpGVr7yVP0XoML0QhgMo+mG8Fy4QsGw0gW7/FrlaNoDFISrpNlnxQrAMs3goUEWtS8Cz50oAXR0dTP4xnPw0gu1ybkWC5ZhFUJ4ttvT98KiE+DfrZqZE46+vhrVC9WoVZaSClU1rL75UfbmjqePSuNR9QrAMs3kwmNu5svwfkaG99HjGTjryurP/IoFy7auoJtZGGTbuoLge3D3VFCGmcGwmgt2+acyh0FYwbfqTvnGPxQsLDGD3/6X/1D/AdWJArBMM23hsaubkeF9jAFrJjoKBp+KcuTRYOke9HYIg2V40jEUz8EWCobVXrDTjruqCWe2+ZZ2tRf9/2Lal6iaI9GSQrVqwrOydyXrVqyju/M4DhwYoNszrBt/Gyuz7TP6NFTciyIMlse+C7bdFiywhcEX4Bd3zt4xrVgwnG+nNampvYdHZ9T2prW+t1yaAcekkpljtFIhGvjWUZ0t0it7LgoW3F56YSrtUGABqmo7BXv7YOc/BV+H/X3DI4d+dvP0e2dbeAsV67SmIFx30Vnv7rdG2HMwaMbemslMqwduVpoBx6DSmWPNj48vtgB11mXTcq5V3SnYuThY9Hvj+eCooeWfDq5v3zy3jmmVdlqTqorOejMZJltQjmezpV/cBDQDjkGlM8fJSoWIqm+RLmMBquKdgmGw3L45aK7+zH3BGW8j+4Pvc8zCuVchqHpBGogCcAwqDaB12yJdYgGq4p2CYbAMc75m0NUdfD3YD5ffOf37lluFoOqFhtSMf5woBRGDngU9jExMP5hwLgE0KU14phbsujkweoDuzu65t+oMj6YPUw1f/Gc4/1rY//LMPC6UH0RVvZA40SY82Wyws23f4VGWdrXTc2wnZ/W8fcZHz7Gdpd+4gWkGHINKZ45JasJT8U7BQqmG8KghpQxSJbrolr+tOL/1ZLNQAI5BNQJoqrZIK2WQWoW2GofXRQE4NqkKoNWglEEqpa2BerUpByzJlr8y04wrNZJamgFLculIoKbRrKkKBWBJprk01dHBmQ2vWVMVCsCSTOU21dEsWRqYcsCSXKWa6ujgTGlwmgFLcpVqqqODM6XBaQYsyVRuUx21npQGpgAsyVRmR7ais+RK0w8qf5M6UApCkqvUDrlaHZyphT2pEwVgSbbZdsjVovVkNc+UEynBvIn+tOrr6/P+/v64hyHVVu064OjMOqSFPZmbsn5RlAOWxlftPhJa2JM6UQBuULU6lHNO0rpQVauFPZE8CsANqOLTiKuhf+P0oBQGrf6Nlb1v3EFdZ8pJHcUSgM1siZk9ZGa/yX1eXOS+CTN7KvexJXL9FDPblnv9D8ws3R078tT8UM5SarUDrVZBfS7KLX8TqYK4qiBuAH7i7uvN7Ibc4z8qcN+wu59d4PpfAN9x93vM7LvANcBttRtustTlUM7Z1GIHWpKqD9QgXuokrhTEJUDuxEXuBD5V7gvNzICLgXCJek6vT4NKz5SrimovVEVnms/cBxs+PL2+t97BTw3ipQ7iCsDvcPfXAHKflxW5r8PM+s3sCTMLg+xS4E13H889HgDqGHnil4hDOWuxUKXqA2kyNQvAZvZjM9te4OOSObzNSe7eB1wF3GRm76FwfV3R/+rNbG0uiPcPDQ3N8adIpqqcRlyJWi1UqfpAmkzNcsDu/pFiz5nZ62Z2gru/ZmYnAHuKvMeu3OcXzewR4APAZuBYM2vNzYJ7gV2zjGMDsAGCjRjz/XmSJtYz5Wq1A60W24pFEiyuRbgtwNXA+tznH+XfkKuMOOLuR83sOOBC4C/d3c3sp8BlwD3FXi81Vu2FqloEdZGEi2UrspktBX4InAS8Alzu7vvMrA/4krt/wcx+B/gekCVIldzk7rfnXv9uguC7BPgl8Bl3P1rq+2orcgPIZiGTKf64mnSUkdROWb9I6gUhyeAOP9801YUsVKsuZOp4JrVVVgBWNzSJX/9GOHow+Hr75qlFt8F+2P/y3OqAy5nVJqnmWJqaArDEKwyG2zfDmZcGH0/cBiP7oWMxnH9t+bnlcme1OspIEkIBWOKVHwxhKvh2dcPvfLn8mW85s9rwc/h9oy0nFXylztSMR6prPs10okH4cK4isas7uF5uHXChnXTbvgcLj4fzr5s+I+7fqJpjSQQFYKme+TbTcYef3RwE3+H90Lk4CKRnXjq3zR35O+l8Ag68Bk/cOr3O+OjB4Pup45nETCkIqY75LmyFgXH7Zug5F3r7guvRnHC5dcD5s9quZcEM+Ol7Z+Z5f75JNccSOwVgqY75Lmzlb8DIv37OmrkF30I76Q4PTaU0wrGo45kkgOqApbrcg/xraO0j1SsfK+XJO2Ds8FQgnZiAH1wFr2+Htx0X3KNKB6kPnQkndVbJwlal7R/7NwbBN1xwy2aD4Lt7O6z4kvK8kkhKQUh1xNlMp1D++Ylb4fUdcPyZU0E5muct9B6aFUudKQBLdcTZTKdY/nnFF4PgG/aSCO/7+abgfw7ahiwxUwCW6olzYWsuGyu0DVkSQgFYqiuuo3yK5Z/zg7C2IUuCaBFOGt9cT+jQ0UeSEArA0vjmepS8tiFLQigFIelQbv5ZRx9JgigAS/yqdTLFbPnnaBe09gXBFmdtQ56z1Tc/yt7DozOuL+1qZ8uXL4phRI1NKQiJ13wb+FTyPc5ZE3z++abgcxiEVYJW0t7Doyzpap/xUSgoS2kKwBKf6AaKMECG6YDRQ9XJyRb7Hts3T/8emvlKDJSCkPjUoyRMZWeSYJoBS7zqURKmsjNJKAVgiVc9SsJUdiYJpRSExKceJWEqO6uqpUUW3JZ2tccwmsanACzxqUcDnzibBKWQSs2qSw3ZJX7VqgOO+3uITFFDdmkQ9WjgE1eTIJFZKACLiMREOeAmtnVgK5t2bGLw0CA9C3pYs3wNK3tXxj0skaYRSw7YzJYAPwBOBl4Cfs/d9+fd8++B70QuvQ+4wt3/l5ltAj4EvJV7bo27P1Xq+yoHPGXrwFZu3HYjbS1tdLR0MDIxwsGjB1nSsYTD44cVkEUqk+gc8A3AT9z9NOAnucfTuPtP3f1sdz8buBg4AvxT5Javh8+XE3xluk07NtHW0kZnaydmxkR2gv1H9/PKwVdY1L6IoeEhbtx2I1sHtsY9VJHUiisAXwLcmfv6TuBTJe6/DPhHdz9S01E1kcFDg3S0dEw+fmP4DTKWYcInMDM6Wztpa2lj045N8Q1SJOXiCsDvcPfXAHKfl5W4/wrg7rxrf25mT5vZd8zsmGIvNLO1ZtZvZv1DQ0OVjTpFehb0MDIxMvl4LDuGYbRl2iavdbR0MHhoMI7hiTSFmgVgM/uxmW0v8HHJHN/nBOAs4MHI5W8Q5ITPBZYAf1Ts9e6+wd373L2vu7t7Hj9JOq1ZvoaxiTGGx4dxdzKWIetZlnYsnbxnZGKEngU9MY5SJN1qVgXh7h8p9pyZvW5mJ7j7a7kAu2eWt/o94O/dfSzy3q/lvjxqZhuB/7sqg24iK3tXso51k1UQ71r0LvYe2UtrSyvuzsjECGMTY6xZvibuoYqkVlxlaFuAq4H1uc8/muXeKwlmvJMiwdsI8sfbazXQNFvZu3JalYPK0kTqK64ytKXAD4GTgFeAy919n5n1AV9y9y/k7jsZeAw40d2zkdc/DHQTlHo8lXvNoVLfV2VoIlInZZWhqReEiEj1JboOWESk6SkAi4jERAFYRCQmCsAiIjFRABYRiYkCsIhITBSARURiogAsIhITBWARkZgoAIuIxEQBWEQkJgrAIiIxUQAWEYmJArCISEwUgCUd8tuqNlGbVWlcCsDS+Po3wuO3QDbXs989ePzkHfGOS6QEBWBpbO4wegi2fRfuuTIIwo/fAk/fC795MAjOIgkV15lwItVhBudfBy89Cr/9Z/jOcujqhkUnwMHdQXB2D+4TSRjNgKXxZTJwxd3QsRhG9sPeXwfB96zL4ILrFXwlsRSApfG5wxO3BjPf0OE9wcxYwVcSTAFYGlu44Pb0vUHa4bj3QudiGN4/lRMWSSgFYGlsZtDWNZXzPesy+OoOOOVD8PqOYGaskjRJKC3CSeM79/NBIB49NJXzveLuIPi2L1AaQhJLAVjSoe9z06sdMhktwEniKQUh6ZEfbBV8JeEUgEVEYqIALCISk1gCsJldbmY7zCxrZn2z3LfKzJ43s51mdkPk+ilmts3MfmNmPzCz9vqMXESkeuKaAW8HPg38S7EbzKwFuBX4BHAGcKWZnZF7+i+A77j7acB+4JraDldEpPpiCcDu/qy7P1/itvOAne7+oruPAvcAl5iZARcD9+XuuxP4VO1GKyJSG0nOAfcAr0YeD+SuLQXedPfxvOsFmdlaM+s3s/6hoaGaDVZEZK5qVgdsZj8Gji/w1Dfd/UflvEWBaz7L9YLcfQOwAaCvr09bokQkMWoWgN39IxW+xQBwYuRxL7ALeAM41sxac7Pg8LqISENJcgriSeC0XMVDO3AFsMXdHfgpcFnuvquBcmbUIiKJElcZ2v9hZgPABcA/mNmDuevvNLP7AXKz2+uBB4FngR+6+47cW/wR8DUz20mQE7693j+DiEilzJuoU5SZDQEvV+GtjiNIhTSDZvpZQT9vmtXzZ33D3VeVuqmpAnC1mFm/uxfdQJImzfSzgn7eNEviz5rkHLCISKopAIuIxEQBeH42xD2AOmqmnxX086ZZ4n5W5YBFRGKiGbCISEwUgEVEYqIAPA9m9l/N7Dkze9rM/t7Mjo17TLVUbv/mRlas93RamdkdZrbHzLbHPZZaM7MTzeynZvZs7vf4K3GPKaQAPD8PAWe6+/uBXwPfiHk8tVayf3MjK9F7Oq02ASU3CqTEOPB/ufvpwPnAdUn591UAngd3/6dIO8wnCBoCpVaZ/ZsbWcHe0zGPqabc/V+AfXGPox7c/TV3/0Xu64MErQ2KtrCtJwXgyn0e+Me4ByEVKdZ7WlLGzE4GPgBsi3ckgZq1o2x05fQzNrNvEvx587f1HFstVKF/cyObU49paUxmtgDYDHzV3Q/EPR5QAC6qVD9jM7sa+I/A/+4pKKauQv/mRlas97SkhJm1EQTfv3X3v4t7PCGlIObBzFYRtMRc7e5H4h6PVKxg7+mYxyRVkjtH8nbgWXf/b3GPJ0oBeH5uARYCD5nZU2b23bgHVEvF+jenRYne06lkZncDjwPvNbMBM0vzyeIXAv8ncHHuv9enzOyTcQ8KtBVZRCQ2mgGLiMREAVhEJCYKwCIiMVEAFhGJiQKwiEhMFIClqZnZoQLX/tTMBnPlSr8xs7+LNm8xs+tzXdPczI6r74glTRSARQr7jruf7e6nAT8AHjaz7txzjwEfAV6ObXSSCgrAIiW4+w+AfwKuyj3+pbu/FOugJBUUgEXK8wvgfXEPQtJFAVikPIU6polURAFYpDwfIOgTIVI1CsAiJZjZpcDHgLvjHoukiwKwNLu35bqBhR9fy13/T2EZGvAZ4GJ3HwIwsz/MdYfrBZ42s+/HNHZpcOqGJiISE82ARURiogAsIhITBWARkZgoAIuIxEQBWEQkJgrAIiIxUQAWEYnJ/w9bReU2O8zrNAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xb46ee10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "markers = ['s', 'x','o']\n",
    "sns.lmplot(x=\"LD1\", y=\"LD2\", data=data, markers=markers,fit_reg=False, hue='class', legend=False)\n",
    "plt.legend(loc='upper center')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# LDA implementation using scikit-learn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.discriminant_analysis import LinearDiscriminantAnalysis\n",
    "lda = LinearDiscriminantAnalysis(n_components=2)\n",
    "X_train_lda = lda.fit_transform(X_train_std, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>LD1</th>\n",
       "      <th>LD2</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-8.084953</td>\n",
       "      <td>0.328454</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-7.147163</td>\n",
       "      <td>-0.755473</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-7.511378</td>\n",
       "      <td>-0.238078</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-6.837676</td>\n",
       "      <td>-0.642885</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-8.157814</td>\n",
       "      <td>0.540639</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        LD1       LD2  class\n",
       "0 -8.084953  0.328454      0\n",
       "1 -7.147163 -0.755473      0\n",
       "2 -7.511378 -0.238078      0\n",
       "3 -6.837676 -0.642885      0\n",
       "4 -8.157814  0.540639      0"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data=pd.DataFrame(X_train_lda)\n",
    "data['class']=y\n",
    "data.columns=[\"LD1\",\"LD2\",\"class\"]\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAFgCAYAAACFYaNMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3X14XNV9J/Dvb0YjS5ZksGUZGwliU3gI2FACIjYvSlmSFJft42wx7EJfEoH7uKHAhma3T4nYdvuy9dJt07I1hMQNQd1sC0lwsrANNaF1s7EJdiISAnYNwQ4QJBASMrZerImkmbN/nLnSndG83Jn7cu7c+/08jx5p7lzNHI/GPx397u/8jiilQEREwUuYHgARUVwxABMRGcIATERkCAMwEZEhDMBERIYwABMRGcIATERkiLEALCJNIvI9EfmRiBwWkT8yNRYiIhPE1EIMEREALUqpSRFJAdgP4FNKqQNGBkREFLAGU0+sdOSfzN1M5T7K/jbYvHmz2rNnj99DIyKqlVRzstEcsIgkReQFACMAnlFKHSxyznYRGRCRgSNHjgQ/SCIinxgNwEqpjFLqEgBdAD4oIhuKnLNLKdWtlOru6OgIfpBERD4JRRWEUuoEgG8D2Gx4KEREgTFZBdEhIqfnvm4G8BEAL5saDxFR0IxdhAOwBsDfikgS+hfBV5VS/2BwPBRis7OzGBwcRDqdNj2UspqamtDV1YVUKmV6KFQHTFZBvAjgA6aen+rL4OAg2trasHbtWugKxvBRSmFsbAyDg4NYt26d6eFQHQhFDpioknQ6jfb29tAGXwAQEbS3t4d+lk7hwQBMdSPMwddSD2Ok8GAAJiIyhAGYqAp79uzB+eefj3PPPRf33Xef6eFQnTNZBREqW3bux9jUzKLj7S2NePKuqw2MiGrl188yk8ngjjvuwDPPPIOuri5cfvnl2LJlCy688EI3w6UYYwDOGZuawYqWxqLHqb749bP83ve+h3PPPRfnnHMOAODmm2/GE088wQBMNWMKgsihoaEhnHXWWfO3u7q6MDQ0ZHBEVO84AyZyqFjrVlY9uLdvcB/6D/djaHIIna2d6F3fi56uHtPDCgRnwEQOdXV14c0335y/PTg4iDPPPNPgiOrfvsF92HFwB0anR7GscRlGp0ex4+AO7BvcZ3pogWAAJnLo8ssvx6uvvorXXnsNMzMzeOyxx7BlyxbTw6pr/Yf7kUqm0NzQDBFBc0MzUskU+g/3mx5aIJiCyGlvaSx55Zzqi18/y4aGBjzwwAO47rrrkMlkcNttt2H9+vWuHjPuhiaHsKxxWd6xpmQThibjkVtnAM5hqVl0+PmzvP7663H99df79vhx09naidHpUTQ3NM8fS2fS6GztNDiq4DAFQUTG9K7vxWxmFtNz01BKYXpuGrOZWfSu7zU9tEAwABORMT1dPejb2IeO5g6Mz4yjo7kDfRv7YlMFwRQEERnV09UTm4BbiDNgIiJDGICJiAxhACYiMoQBmKKpcNlwkWXE1brtttuwatUqbNiwwfVjEQEMwBRFA48Azz2wEHSV0rcHHnH1sL29vdizZ48HAyTSGIApWpQCZiaBlx5fCMLPPaBvz0y6mgl/6EMfwooVKzwcLMUdy9AoWkSAK+7UX7/0uP4AgItu1MfZvcy4OHc/K8QATNFjBWEr+AIMviFhdT9LJVPz3c9+f//vo31pO6Zmp2IXkJmCoOix0g529pwwGVPY/WwuM4eTMyfxxvgbbEdJVPfsOd+LbgS2f1t/tueEyZihySE0JZvmb4+lx5CQBLIqG8t2lAzAFC0iQGNrfs73ijv17cZWV2mIW265BVdccQVeeeUVdHV14eGHH/Zw4PHQ2dqJdCY9f3s2OwsFhVQiNX+M7SiJ6ln3rXqmawVbKwi7zAE/+uijHgwu2ipdYOtd34sdB3cA0IE2KUnMZmexsnnl/DlsR0lU7wqDLS/A+c7J9kKF3c/Objsby5csRzKRjGU7Ss6AicgT9gtsAOY/9x/uz5sFF3Y/i3NZGgMwEXmi1u2F2I6SiMilwgtsQLzyubVgACYiT8R9e6FaMAATkSfivr1QLZgDJnLozTffxMc//nEMDw8jkUhg+/bt+NSnPmV6WKES53xuLRiAKZL8uLLe0NCAz372s7j00ksxMTGByy67DB/96Edx4YUXejRqihumIChynNSj1mLNmjW49NJLAQBtbW244IILMDQUjxVb5A8GYIqcwoYvfvQXeP311/HDH/4QGzdu9OwxKX6YgqDIqbUe1anJyUls3boV999/P5YtW1b5G8gXUVjAwRkwRY6f9aizs7PYunUrfu3Xfg033HCD68ej2viVZgoaAzBFjl/1qEopbNu2DRdccAE+/elPezNYqkkQaaYgMABT5PhVj/rss8/iy1/+Mvbu3YtLLrkEl1xyCZ566imPRk3VKOwrDNRnG0vmgCmS/KhHvfrqq6HY0D0UOls7MTo9Ot/wB6jPZc/GZsAicpaI/IuIHBGRwyLCinYiciQqy55NzoDnAPwnpdQPRKQNwPMi8oxS6l8NjomICjitNgiyKqGnqwd96Kv7KghjAVgp9TaAt3NfT4jIEQCdABiAqSilFCTkjdWjlqIotovxjoM70Ie+RT19nZznpSgsew7FRTgRWQvgAwAOFrlvu4gMiMjA6Oho0EOjkGhqasLY2FioA5xSCmNjY2hqaqp8cp1wWm0QlaqEoBm/CCcirQB2A7hbKTVeeL9SaheAXQDQ3d0d3v995Kuuri4MDg4i7L+Em5qa0NXVZXoYnnG6qMXvxS9RZTQAi0gKOvj+nVLq6ybHQuGWSqWwbt0608OIHafVBlGpSgiaySoIAfAwgCNKqb80NQ4iKs1ptUFUqhKCZjIHfBWA3wBwrYi8kPu43uB4iKiA00UtbMZeGwnzRY1C3d3damBgwPXjbNm5H2NTM4uOt7c04sm7rvbse4godqoq0zF+Ec6EsakZrGhpLHrcy+8hIionFGVoRERxFMsZMBGZF4V+vm5xBkxEgYtKP1+3GICJKHD3/+B+jE6PYnBiEG+Mv4FMNhPLlXOxTEG0tzSWrGjw8nuIaLF9g/tw7MQxJCSBpCQxp+bw9tTbWL10dexWzsUyANdSNsZSMyJv9B/uRyqRQlZlAQAJJJCVLEamR/DzHT9veHTBYgqCiAI1NDmElc0roaCQVVndYEkBc9m52K2ci+UMmIi8U201g9U3YvXS1RhLj2E2O4ukJLF22VpWQRAROVVLNYPVN6Ih2YD3LXsfutq6sLJ5Je6+7O4ARx4ODMBEVDMnfYD3De7Dtqe3YfPuzdj29DYAYN+InMinINjDgcg/lfoAl9wpY2MfHr7uYRNDDpXIz4CtHg6FH+zhQOReZ2sn0pl03jF7H2DulFFe5AMwEfmnUh/gockhNCXzt2gqnCHb0xNcCUdE5FClPsDlZshcjhyDHDAR+avc7sS963ux4+AOAHrmm86k52fI9vQEgPnP/Yf7Y3NBjjNgIvJNuRlypfREHER+BsweDkRmlZohcyPPGARgU6VmLH+jKPGjd2+59ERcMAXhE5a/UVT4dbGMG3kyABNRBX7W8vZ09aB3fS86WzsxNDmE/sP9saqCYAAmorL8vFgW91I0BmAiKqvSajc34r5SjgGYiMqqtNrNjbiXojEA+6S9pRHHp2YWfbD8jeqNnxfL/Jxd1wNRSpkeg2Pd3d1qYGDA9DCIIsP01vD2bmn2UrQ6roaQak7mDJgopsJwASzupWiRX4hBRMWFpRdDuV4SUccATBRTlZqpV8N0KqNeMQDnhHXpcFjHRfXPq14MJXe9QHxSCbViDjgnrEuHwzouqn9elZfFvZbXDc6AfcYZLIVVT1cP+tDnOnXgZSojbhiAfWbNYIsdp5hTChApfTsAXlwAY1vJ2jEFQWTCwCPAcw/ooAvoz889oI/XGT9XykUdZ8AeKZVqGJlIF50BU4wpBcxMAi89rm9fcacOvi89Dlx0o5GZsBtepTLiiAE4x+3OGaVSDcMn00XODm5cFEIiOugCOuhagfiiG/XxEAbfSmVmca7ldYMBOCesF8TCOi5yyQrCVvAFQh18WWbmD+aAfZZIgE15aDEr52tnzwmHCMvM/MMZsM9WtTXh2XuuNT0MChMr+Fo5X3sOGAjdTJhlZv5hAA4J1gvHiAjQ2Jqf87Vywo2t3gRfD0vcWGbmHwZgj/h1EY/1whHVfWt+ULSCsBfBd+ARXWVhPZ41425s1c9bJe5e7B8GYI9wlkpVKwy2Xs18PS5xY5mZf4wGYBH5EoBfBjCilNpgcixuMH1AoeFTiRvLzPxhugqiH8Bmw2NwjQ1zKFTsQdgSsgt7pBkNwEqp7wA4bnIMRJFTRyVucRf6HLCIbAewHQDOPvtsw6PxD1e8kSfqrMQt7kIfgJVSuwDsAvSmnIaH4xvmiskTQZS4kWdCH4BN4wU2qjt+lriRpxiAK3BSn8v0AYWOHyVu5DnTZWiPArgGwEoRGQTwX5VSD5scUy04EyaqXZw39DRdBXGLUmqNUiqllOqqx+BLRLWzOq2NTo/mdVrbN7jP9NACwRQEEbniZgZr77QGYP5z/+H+WMyCGYCJwiQE+8RZnARWt72ChyaHkEACr0+9jtnsLFKJFNqb2mPTaY0BuAInF9hYKUGe8LiJjhtOA6vbGWxLqgXHThxDMpFEUpKYU3N4a+ot/NzpP+fPPyxkGIArcBJA2cmMXAvZPnFOA6vrXsEKEAigAAVd5m/djgMGYKIwCNk+cU4Dq9tewVNzU1jdshrH08fnUxArmlZgam7K/T+iDphuxkNElhA10els7UQ6k7+hbLHA6nZL+s7WTqSSKaw9bS3OW34e1p62FqlkKjbN3hmAicIiRE10nAbWnq4e9G3sQ0dzB8ZnxtHR3IG+jc4363QbwOudqDrqkNTd3a0GBgZMD2ORq+7bixUtjTg6Mom5bHb+eDYLrD6tiRfjqLJyTXQMpSGCWiARsYUYVf2QmAOu0pad+3FkeBy2OIuMUhg6MQ0ASCUXXv/GhgT7ApMzIWyiE1QT9jg3e2cArpIVTBsb8rM3c9ksslng/auXFfs2osrYRCd2GICJwqQOmuhELGVgFAOwD4rlgq+6by9zwWSElwHT7co3yscA7IO5bBYNiYUUxRyyWNHSiCPD47jqvr2LzmdgJr94HTDj3rvBawzAHkokgONTM8hmddC1WME4mwVXzFGgvA6Yrle+FRHnlAbrgKtk9YCYmcvmfQDABauX4dl7rsXq05rw/tXL5j/OXdVqcsgUY0OTQ2hKNuUdcxMwnS7QcIrtKKkqhakCeyOesakZXHXfXgyfTOP41AwDLxnndqlwod71vdhxcAcAHcjTmbSrhRNxT2lwBuyS1YjH/pFIIO8iHBGAxSvaAlgEZV9pNvGzCRw7cQw/Hf8pTqRP1DTLdLvyrZDXM/R6wxmwDxoSCczMZXG8ILebKPPrji0tI85Qq8merh70oQ/3/+B+/HT8p0glUljTsgazarbmi3FeLpzweoZebxiAfXDuqlYcn5rBs/dcm3e8XJBlS8sIM9xqsqerB/2H+3H2srPzAh1g/k99r1Ma9YYBOEDlZrLFytPsOEOuYyFoNelH9YJT5aocrBl6XKsgGIAdKBf8gsIZcp2zgrAVfIFAlxmb+lPfSR1ynHtB8CKcA8UutFlNdtpbGnF8ambRR5DBmeqA4VaTpto+2qscRATNDc1IJVPoP9zv6/PWC86AXeKf/1RRuVaTQCAzYVN/6ptMfdQDBmADiqU0RibSGJlIY1VbfkkOZ9IREJJWkyb+1I97lUMlDMAGFMvnrsilMgorJygiYtpqMu5VDpUwAHsgiAqF9hKN3TlDriNW/a89CAe823HQ4l7lUAkDsAOVgl8QFQrMNUeAocUYTvnVFCfOVQ6VMAA7wOBHrhlejFEJ+/yawQBMFIQQLMYoJ8imOHFuP1mIdcAGsHY4puxB2BKC4AsE1xQn7u0nC3EG7CGnWxExpRFTpRZjhCAIB1UuFvf2k4U4A/aANaO1GrNbuC09zStcjLH92/rzS48HuiKulKBWysW9/WQhzoA9YM1or7pvb9FqCKKwLMYoJahyMS7MyMcATBQUvxdjFFZSFNyudPGrsFxs3+A+bHt6m6cBmQsz8jEFQRSkwmDrVfAdeCQ/lWGlPAYeAVD9xS+/LpZ5vaNGveMMmKjeOagxrvbil58Xy7gwY0HZACwiSQC/CaALwB6l1LO2+/6LUuq/+Ty+urFl536MTKQxfDJ/x9hEQu+WTHWowp/0rs+v9XsKOagxrrYrGbuYBaNSCuILAH4BwBiAvxaRv7Tdd4Nvo6pDY1MzeP/qZbjwzPyPVW1NLDurRxX+pHd9fq3fU0qFGuNqt5P3evt5Kq5SAP6gUupXlVL3A9gIoFVEvi4iSwCYrx4n8oP9T3orQFp/0s9MFt/duJrza/2eSmMu0/C92jIzUw3c46ZSDni+pkopNQdgu4j8AYC9AFr9HBiRMdUuG3ZyfmFqwculyQ4avldbZsYuZsGoFIAHRGSzUmqPdUAp9cci8haAh/wdGpFB1e7hZj//1BigMsCmO8p3PXPyHPbAbc2K7bdFHNcYV3vxixfL/Fc2ACulfr3E8S8C+KIvI4qIoyOTSM9moAD83Geemj9uXZRjXjjkql02bD9fZYDp94DHbgFufhQ48GDxrmeVnsPevvL5fuBnE/qcJW3AZb35QT2mDd/rXcUyNBFpB/CrAN6fO3QEwKNKqTG3Ty4imwH8TwBJAF9USt3n9jFNKewZPDOXhUJuctKwkGqfy2a5NDnsqt3DrfD8TXfo4Pva/wP+aj3Q0gFcfFP+91V6jk13LOSIrZnvwYcABWDT7cB3dwKHducHdb9qjMk3lcrQLoDO9z4N4IfQF94uB9AnItcqpV6u9YlzJW4PAvgogEEA3xeRJ5VS/1rrY5pUOKO96r69GJlIoyHBtS51p9plw8XOv/lRHXwTieKz0UrPkUjk54iV0sFXoAMvEJpWllS7SjPgPwHwKaXUV+0HRWQrgD8FsNXFc38QwFGl1E9yj/kYgI8BqMsATBFT7Z/09vOV0mmHlo6F84ulLyo9hz1HLJL/eEAgwde+fLmloQUQYGp2Cp2tneho7sB3hr6DU7OnsDS1FL9xwW/g9ktu93U8UVNpenZRYfAFAKXUbgAbXD53J4A3bbcHc8fyiMh2ERkQkYHR0VGXT0lUhWr/pLdfcHvpcZ122P7t8l3PKi3s+O7Oha+nRoGpkYX7fe6iZl+OLBD85ORPcOzEMSSQwCvHX8E3X/smTs2eQoM0YHpuGl948Qt46AVem69GpRnwVI33OVHsnbfo3aSU2gVgFwB0d3eb7dnnkZGJNK66b++i415u4kmGuOl6Zr/oBujge/Ah4Mxu4KzLF3LAG3J/eJbKSXvEvhz5nal3kJAEIMBYemx+kYZSCpIQNKABc5jDl498mbPgKlQKwKtE5NNFjguADpfPPQjgLNvtLgBvuXzM0GhvacTIRHpRj2ArJez3Jp5kUPetuhu/PZWw6Y6FH34xxfo5DA3ogHvW5bryYWMusFlVEFaw9ykNYV+OPJudRQIJCASz2VlklX5fK9ucKYkkTs2e4pZDVagUgP8GQFuJ+9yWoX0fwHkisg7AEICboastImNVW9OiY6V2WKYIKbb78YEHy+9+XGphxqbbgSvvWngc61zA9cy3UqC09+5NJVKYy85BQSGVSCGjMsiqLMT2h2wGGTQmG7m5ZxUq1QH/Uan7RORuN0+slJoTkTuhKyySAL6klDrs5jFN27Jz/3xwHT6Znp/wNCQSOHeVXjjI4Btx2Wz+THbTHaXrgAsVW5hhBV/r/sLza+RkF2R7794VTSswPDUMpRTOaDkD4zPjODlzEiICpRQyyEApheVLlnPLoSq4aUf5aQD3u3lypdRTAJ6qeGKdGJuamU8t2EvQ0rMZvDw8DkD//7TfbwVmigBr5rvpDn37xa8Bz31Opx42frLyjDVvMUcuUFvVE4CnqQYn7SYLlyOfc9o581UQ5684v2gVxBPHnuCWQ1VwE4BZfOiQAuaD8Rx0BG5IJPI28KQ6V5jD3XSHDr7p94Cm5QvLkst9v1U9sfx9QGe3Pm5fiLGkrXQKo0pO201Wuxx54J0Bz7ccinJO2c0qgUhUJJhgBd9sFtyWPiqs9IFVcnb/+oXg29Kh0xDlSsasC2obturgay222LAVGBzQt2vpklaCX+0mve6iFvVt7CuthJtA8UArAJqLHCcHrLTD8akZPHvPtYZHQ56xqh0OfE73gmheDtx9eCEHDDhbzGE9lj0X7PGqN7d7s5WalXrdRS3q29hXughXqgKCKihMMUzPZgDo31xWPpgixqp2kKQOvi2r9G0rJ+ykZMxe4eC0E1sN3ATKShfwvOyiFvWdObgnnIfsJWb2Ot+hE9NoTiUBAOm5DOYyepajgPkFGVyEUefsOdyNv5Vf/QBUrgMu9lh25Tqx1ajWQBnkrDTq29gzAHuoVAA9796n5mfDSi1cvRQsBGqWp9U5J811nKi2E5sBQc5Ko76NPQNwAFa1Nc0H2peHxxcqIlgFES1e9OR1s5Q5IEHOSqO+MwcDMJGXal0sYQ/cxZYyh2Dmawl6VhrlnTnYrNaA9FwG6dkMZjMKLw+P4+XhcQyfTGPLzv2mh0ZBKCwlK7Y78oEH83dHDknwBXKz0o196GjuwPjMODqaO9C3kUuNa8EAHID2lsb5Wt9sNr/SqCGRQEMigUSCeeBYKAy22Szw4z3AwS94sztyQHq6evDwdQ/j3o33AgD+9OCfYtvT2yJTnxsUpiACYL84t2Xnfhx+a3z+moyVBy7cOcPeV8KO1RJ1rFjHswMPAuNvA2esd787so+K1f0CYOMdlxiAA/bkXVfjqvv2Fm1HedwWcO19Jew4S65jpTqeXXyTLlP7m3+zcG7Igm+xQLu0YWmkF0kEgSkIoiDZg7DFqhm283m3i2rY635FRLenTKbw+sTrbLzjEmfAREEqXGShlN5BeWK47up+oXT5WVQXSQSBAdiAUk3Z2YwnguzlZdYeb9Z28lawPfh54IwNCx3TCpcul+shHIBSdb9rT1uLU7OnAERzkUQQGIAN4EW0CCkMjvbbhTtjALqz2fL3LV5kkWrRq+XsPYUTiYUZc7ndNHxWqu73d7t/FwAiu0giCAzAIWKvfBiZSGP4pG4XmEgsbG/EWXKIFNt6yAqWl/Uurnh47gHgxBsLm2oC+YssilVJ2JclG5oJV1qNxoBbOwbgELFXPtgrINi2MoSKNmB/YCG9oNRCKqFSeVnhijcn3+ODco3Po7wazSQG4JDYsnM/hk+mMTKR3yS7IZEoWo5GhtmD5cHPA9/5c6BhKXDFb+vA+92dwFvPA+uu0edYs9dKgbTYvnABBV/W9AaPZWghMTY1g0RiYWWc9cGGPSFmXTDLZoCZU8CpUR1ov/YJYN9fAG8cBH78tD42NQqcerdyeVmpVpQ+l6SVKjXrP9zv6/PGHWfARLWyeja0rNK3p94F/vmPc/1GBWhbA4y/BZx2pg7WbauBH31Vn1tsVmuwFWXUG5+HFWfARLWwB8uLbwJ+51/13m/I6E77LSuBqRHgxOt6qfFFNwL/4e91MH77xeKBtFQryotu9L0VpV97xFF5nAGHSLGUQzbLyodQsgdL6wIcAEiDngFnlU5NZGd1+sFa7TYxDKy9unRFg72ncGHe2McqiKg3Pg8rUSFZ7uhEd3e3GhgYMD0MX5TrD8EKiBDLZnOLKR7SM9+Nn9TH9/2Fzgs3LtVB2NojzmlFQ7kSN5/qgaO8/XuAqvoNyRlwSByfmpmv+7VrbGCWKNQSCWBJG9B5OdDVvZC3bWwFTj8LuGwbsP8v9C7JwMJqt3IM1QOz1Cx4DMAhsaKlsWKHNAqp7lv1wgtAB8YlbcCm23WwPfi5hYt0ktRpCKelaECoW1SSewzAIcAa4AiwB0VrSyFrV2QrT2zfJTmk9cAULAbgELDXANuxBriOJRLuNtcMaGv6IDC3XBoTjCFi7RVnfcxmFPeKq2fdt+YHTCsIV7qIVlgPvP3b+vNLj4eqT7AT1gq70enRvBV23LpI4ww4TBTyrqGKAveKq3e17JJcB1vTO2VfYQeEc9cMkzN0BmBD7J3Phk+mkVUKCgDUQgxWWJyWIJfKtY8ME3s9MBC6remdCvsKO9M9MBiAfVZqc82RiTTev3rZ/NeNiSTSsxkAQFMqCUDngM9d1cpKCK8YqK2dV0vgr2X2HDKlmrmHZYWd6Rk6p1c+s1pMFn7Yr69ZK+AU9Kx3LpvFXDbL2a+X7LW1QW//XrgVvfXcA4/495wh0bu+F7OZWUzPTUMphem56VCtsBuaHDK6rx1nwCFw7qpWAMDLw+MAMD8zJg+Zqq0NaZP1oFRq5m6a6Rk6A3CINCQSmJnLLko5sBeER0zU1tbJogo/L0SFeYWd6R4YDMAhYuV72fvBJ6Zqa8O8qEIp7BvaH9tm7KZn6AzAhiQSxZcZc7brEz967Tq9sFYs8H93J3DlXfk7JgcdkHMXJfsnfrhwIWpyBM2SBJYsDVWpmJ9MztAZgH1Wagv6C1Yv4+7IQfK6ttZpRUWxwP+1TwAHHtL3X3mX/mzfzDOIMjlbbnqodRbLlnUBkyO6aVDz8lCVikUZA7DP7EF23We+OX8hfOjENNbe800A+v/Xa//935oYXrx4VVtbzYW1wsAPAJ3dwOD39Rb1wML3nv4+4GcTCzNjP8vkbL+AOl/+EkbffQXNEN02s3UV0nPToSkVizIG4ACV+wsVKF0z3N7SyNmyV7yora32wlph4LdmvYd2A7uu0V9bW9Uf2r3w+C6rJSpeWMs9T+/hv8eOhlMAFJpaOpAOWalYlLHQNERK1QxzKXII2YOwpdxsunAbeisIW668S89021broLvrGv25bTWQaqkp+FbswZCbYfdkG9E3txQdKoHx8UF0NK9E30ZvL8DtG9yHbU9vw+bdm7Ht6W3sBZHDGXBIHR2ZnO+Gls3qHTMAzoZDw01FRamLcoDeP+7UqO4hPDWiz52dqnoGXHGFV0FuuueKO9Fj3V73AaDTu/eYV8t9o9hVzcgMWERuEpHDIpJXHB6PAAAQOklEQVQVkW4TYwg7ayVcQyKBRAKcDYeJm25lpb730G59f9tqfSHs3Vf052VrnO2iUaDiCq8ANwD1Ysv7qHZVMzUDPgTgBgBfMPT8RLVzU1FR6nuVAoYGgMl3gKbleoflqVG9iaeTXTQKOFrhFVDDHy8a8pju2eAXIwFYKXUEACQMhegBq6NWrlSOm+BV7HuvvAt4vh/48Z6F+1o69Iy4hhmp4xVeATT88WK5b9i7qtUq9BfhRGS7iAyIyMDo6Kjp4bjSeXozLuo8bdFH5+n6jdne0ojjUzM4PjWDbJZNeULPTfAqdu7MpM4BX3yTTk1cfJO+XUOzoJ6uHvRt7ENHcwfGZ8bR0dzh+YU1p7xoyNPZ2ol0Jn/LrjB1VauVbzNgEfknAKuL3HWvUuoJp4+jlNoFYBegt6X3aHihZL+4VmqbeoooKzVx8U2eNWEPSw8GL5b7mu7Z4BffArBS6iN+PXYclFpBx6XKERaRJuzFuP1lYLpng19YhhZSLDWLqQg0YfdLWGb0XjJVhvYrIjII4AoA3xSRp02Mg8iIwnwur8zGlqkqiG8A+IaJ5zaJaQUyui0ShQ5TEAFiWiEGyrWojPnuGLQYAzCRVyrNbutkdwwKDgtMibzgdNPPapv4UKRxBkzkBaezW1PbIlEocQZM5JVKs1s3TXwokhiAKZ78KAUrNbu1px8C6kBG9YEpCIofP0rBnG76GeHVblQ9BmCKF79KwappUcnVbpQjqo7yTt3d3WpgYMD0MKje2WerFq9KwZxuVU9RVdUPmzlgih8/S8E4u6UqMABT/FS6WBYVIeo5wU05i2MApngJYymYH4Fy4JH8f4/17x54xP1jVymq+7l5gQGY4iVspWB+BEqnq/IC4sWmnFHFKgiKn7CUgvlZkRGinhNR3c/NCwzAFE9huFjmZ6C0Htte6WGo3tiLTTmjiikIIpP8qsgI0YVGLzbljCoGYCKT/AiUIbvQGKYdmsOGKQgiU5wuX65WNavyAhLF/dy8wABMZIqfgTIsFxqpLAZgokJBLifuvhXIZvMD5aY7gIQH2cEwXGikspgDJrILegHDwCPAgQf181gfBx7Ux6O2Mo8W4QyYyFJrXW6tM2b78w1+H+js1scP7QY2bAW+uxNY0sbdkiOMAZjIUktdrpvewtbzKQUceAh49VtA03Jg0+36/kO7uVtyxDEFQWTnpC7Xnp5wu+RXBLjyLqClQwff9Hs68FrBlxfOIo0BmMiuUl2uPUdsXTBrWw0c/AKw65r8kjKnaYjnHtDntnToY1Mj+nO1wTdE3c/IGQZgIkulBQzZ7OIZ74EHgfG3AZVZeJxqg+9Lj+uc70U3As3Lgen3dBD+7k7nQTRE3c/IOeaAiSyV6nITicU5YqWAZWvyA67Tbeat59uwVd8+tBvYmMv/Dg7o207qd/1q6kO+45ZERIUqVTUopdMNSgFTo0DrqsUr2apNQzzfv3Axz1LNRqF+brNE1ajqxeYMmKhQuQUM9hyxiJ4Vt56hc8H2qoZqVrIV2y0ZqC54hqj7GTnHHDCRU8VyxJ2XAcMvetPkxs3KtRB1PyPnGICJiilWUVCYIwb04gkRnbMFdNA7tDvYnSdC1v2MnGMKgqhQpcUV9lTBlXfpz4d267wwEHzuNYTdz8gZBmAiu2orCqyFFId2LxwzkXtl97O6xABMZFftcuRSuVcTwY/dz+oOc8BEhZxuE8TcK7nEAExUyGlFQdi2uKe6wxQEkV212wQx90ouMABHyJad+zE2NbPoeHtLI56862oDI6pDpWa1hYsrCoNu4WO4EeSOHGQUA3CEjE3NYEVLY9HjVIXCWe3z/frzZb36czU9f6vlpr8w1R3mgImKsc90ZyZ1mVmtPX+d8qK/MNUVzoCJyqlll4x6eC4KBc6AiSpxWpZWb89FxhkJwCLy5yLysoi8KCLfEJHTTYyDyJEgG92wqU6smJoBPwNgg1LqYgA/BvAZQ+OIlPaWRhyfmln00V7kwhw5FORiCy7siB0jOWCl1LdsNw8AuNHEOKKGpWY+CLLRDZvqxI7xHTFE5P8C+IpS6n+XuH87gO0AcPbZZ1/2xhtvBDk8Ii3I2lzWAdezqn5QvgVgEfknAKuL3HWvUuqJ3Dn3AugGcINyMBBuSUREIReOLYmUUh8pd7+IfALALwP4sJPgS0QUNUZywCKyGcDvAfgFpdQpE2MgImf2De5D/+F+DE0OobO1E73re9HT1WN6WJFgJAcsIkcBLAEwljt0QCn1yUrfxxQEkb8Kg233Gd148tiTSCVTaEo2IZ1JYzYzi76NfQzCxYUjB+wHBmAi/+wb3IcdB3fkBdu3Jt/C6UtOR3tz+/x503PT6GjuwMPXPWxwtKFVVQDmSjgiAgD0H+5HKplCc0MzRATNDc3IZDMY/9l43nlNySYMTQ4ZGmW0MAATEQBgaHIITcmmvGONyUbMZPO76aUzaXS2dgY5tMhiACYiAEBnayfSmXTesbbGNiQTSUzPTUMphem5acxmZtG7vtfMICOGAZiIAAC963sxm5nNC7apRAq/ueE30dHcgfGZcXQ0d/ACnIfYjpKIAAA9XT3oQx9LzgLEAExE83q6ehhwA8QUBBGRIQzARESGMAATERnCAExEZAgDMBGRIQzARESGMAATERnCAExEZAgDMBGRIQzARESGMAATERnCAExEZAgDMBGRIQzARESGMAATERnCAEzklcIdxutox3EygwGYyAvf/xLw3AMLQTeb1bcHHjE7Lgo17ohB5NbAI8CrTwPjb+vbm+4AHrsFeOcQsPGTOiiLmB0jhRIDMJEbSgEzk8DEMLBsDfDi14DnPgek3wPW/YIOxgy+VAIDMJEbIsAVd+qvX3ocGPux/rppOXDzo0CCWT4qje8OIrdE9Ex3amThWEsHcOBBXoijsjgDJnIrm9U53+n3gOblQMsqoG21TkcAeobMNAQVwRkwkRtK6ZnuO4d1zvfuw8BFNy7khFMtDL5UEmfARG6IAI2twMbf0mmIRGIhJ9zYCnTfanZ8FGoMwERudd+aX2pmXZjjzJcqYAqCyAuFwZbBlxxgACYiMoQBmIjIEAZgIiJDGICJiAxhACYiMoQBmIjIEAZgIiJDGICJiAxhACYiMoQBmIjIEFF11K9UREYBvFHDt64E8K7Hw6kVx7JYWMYBcCzFhGUcQPjH8q5SarPTB6irAFwrERlQSnWbHgfAsYR5HADHEuZxANEbC1MQRESGMAATERkSlwC8y/QAbDiWxcIyDoBjKSYs4wAiNpZY5ICJiMIoLjNgIqLQYQAmIjIkMgFYRG4SkcMikhWR7oL7PiMiR0XkFRG5rsT3rxORgyLyqoh8RUQaPRrXV0TkhdzH6yLyQonzXheRl3LnDXjx3EWe4w9FZMg2nutLnLc591odFZF7fBjHn4vIyyLyooh8Q0ROL3Geb69JpX+jiCzJ/eyO5t4Xa718ftvznCUi/yIiR3Lv308VOecaETlp+7n9gU9jKft6i/bXudfkRRG51KdxnG/7t74gIuMicnfBOb69JiLyJREZEZFDtmMrROSZXHx4RkSWl/jeT+TOeVVEPlHxyZRSkfgAcAGA8wF8G0C37fiFAH4EYAmAdQCOAUgW+f6vArg59/XnAdzuwxg/C+APStz3OoCVPr9GfwjgP1c4J5l7jc4B0Jh77S70eBy/CKAh9/WfAfizIF8TJ/9GAL8N4PO5r28G8BWffiZrAFya+7oNwI+LjOUaAP/g53vDyesN4HoA/whAAGwCcDCAMSUBDAN4X1CvCYAPAbgUwCHbsf8B4J7c1/cUe88CWAHgJ7nPy3NfLy/3XJGZASuljiilXily18cAPKaU+plS6jUARwF80H6CiAiAawE8njv0twD+nZfjyz3HvwfwqJeP64MPAjiqlPqJUmoGwGPQr6FnlFLfUkrN5W4eANDl5eM74OTf+DHo9wGg3xcfzv0MPaWUelsp9YPc1xMAjgDo9Pp5PPIxAP9LaQcAnC4ia3x+zg8DOKaUqmUFbE2UUt8BcLzgsP39UCo+XAfgGaXUcaXUewCeAVB2VVxkAnAZnQDetN0exOI3eDuAE7agUOwct3oAvKOUerXE/QrAt0TkeRHZ7vFz292Z+/PxSyX+jHLyennpNuhZVTF+vSZO/o3z5+TeFyeh3ye+yaU5PgDgYJG7rxCRH4nIP4rIep+GUOn1Dvq9Aei/PkpNWoJ4TSxnKKXeBvQvTQCripxT9evT4NnwAiAi/wRgdZG77lVKPVHq24ocK6y9c3KO23HdgvKz36uUUm+JyCoAz4jIy7nfxFUpNxYADwH4E+h/259Ap0RuK3yIIt9bda2ik9dERO4FMAfg70o8jCevSbHhFTnm6XuiWiLSCmA3gLuVUuMFd/8A+k/wyVze/v8AOM+HYVR6vYN+TRoBbAHwmSJ3B/WaVKPq16euArBS6iM1fNsggLNst7sAvFVwzrvQf0415GY7xc6peVwi0gDgBgCXlXmMt3KfR0TkG9B/JlcdbJy+RiLyNwD+ochdTl4v1+PIXaD4ZQAfVrkEWpHH8OQ1KcLJv9E6ZzD38zsNi/8s9YSIpKCD798ppb5eeL89ICulnhKRz4nISqWUp01pHLzenrw3qvBLAH6glHqnyFgDeU1s3hGRNUqpt3Npl5Ei5wxC56YtXdDXpEqKQwriSQA3565qr4P+Lfk9+wm5APAvAG7MHfoEgFIz6lp8BMDLSqnBYneKSIuItFlfQ1+kOlTsXDcK8nW/UuI5vg/gPNFVIY3QfwI+6fE4NgP4PQBblFKnSpzj52vi5N/4JPT7ANDvi72lflG4kcsrPwzgiFLqL0ucs9rKP4vIB6H/3455PA4nr/eTAD6eq4bYBOCk9We5T0r+1RjEa1LA/n4oFR+eBvCLIrI8l977xdyx0vy4imjiAzqgDAL4GYB3ADxtu+9e6KverwD4JdvxpwCcmfv6HOjAfBTA1wAs8XBs/QA+WXDsTABP2Z77R7mPw9B/pvvxGn0ZwEsAXsy9odYUjiV3+3roq/HH/BhL7jV+E8ALuY/PF47D79ek2L8RwB9D/1IAgKbc++Bo7n1xjk8/k6uh/0x90fZ6XA/gk9Z7BsCdudfgR9AXLa/0YRxFX++CcQiAB3Ov2UuwVRv5MJ6l0AH1NNuxQF4T6KD/NoDZXEzZBp3//2cAr+Y+r8id2w3gi7bvvS33njkK4NZKz8WlyEREhsQhBUFEFEoMwEREhjAAExEZwgBMRGQIAzARkSEMwBRZIjJZ5Ji9I9yrIvJ1EbnQdv+duW5fSkRWBjtiihsGYIqjv1JKXaKUOg/AVwDsFZGO3H3PQi+cCaz5C8UXAzDFmlLqKwC+BeBXc7d/qJR63eigKDYYgIl0Y5f3mx4ExQ8DMFHxLlZEvmMAJtI9eI+YHgTFDwMwxZqIbIXuWhX2nUooghiAKcqWisig7ePTueO/Y5WhAfh1ANcqpUYBQET+o4gMQvdyfVFEvmho7BQD7IZGRGQIZ8BERIYwABMRGcIATERkCAMwEZEhDMBERIYwABMRGcIATERkyP8H7DcHd5FCIIAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xbc15cf8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "markers = ['s', 'x','o']\n",
    "colors = ['r', 'b','g']\n",
    "sns.lmplot(x=\"LD1\", y=\"LD2\", data=data, hue='class', markers=markers,fit_reg=False,legend=False)\n",
    "plt.legend(loc='upper center')\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}