13-Intro_to_deep_learning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Intro to Deep Learning\n",
    "We're going to continue working with CAP imagery for the second half of this week. Recall the two main guiding questions for this week:\n",
    "- _What_ is in an image (e.g. debris, buildings, etc.)?\n",
    "- _Where_ are these things located _in 3D space_ ?\n",
    "\n",
    "## Motivation\n",
    "\n",
    "We've already seen how structure from motion can begin to answer the second of those questions, at least in a coarse manner. The first question is more difficult than the second for one crucial reason: it (so far) can not be answered *solely* using image data. Recall that structure from motion was able to leverage image metadata, features and geometric constraints without having to insert any \"outside\" data. When detecting the components of an image, there is a certain amount of subjectivity that so far requires human input. For example, if you want to detect flooding in an image, there need to be some set of rules that the software can determine \"if (rules are satisfied), then (flooding = True)\".\n",
    "\n",
    "How can you start to tackle this problem? We're going to make the problem a bit simpler by turning it into a classification problem rather than a localization problem. That is, instead of finding where flooding is in an image, we'll find whether there is flooding at all. With that in mind, let's first try a naive approach and see if we can simply enumerate the rules. Let's look at some flooding images:\n",
    "\n",
    "<img src=\"notebook_images/010_1282_99809b0c-fc64-46e5-957f-7ff8e7547d8c.jpg\" width=\"500\"  />\n",
    "\n",
    "<img src=\"notebook_images/DSC_0655_0cbbe7e5-be24-4274-bf5a-664c6f2dc3e1.jpg\" width=\"500\"  />\n",
    "\n",
    "So what do we see? It looks like flooding is this murky, brown color that covers most of the image. So let's make a rule: if some percentage of the image is this brownish color (you can think of detecting this by creating some sort of index, like you did with the satellite imagery), there is flooding in the image. This might work for the two images we saw, but the real test is seeing if it works with other images. So let's look at another one:\n",
    "\n",
    "<img src=\"notebook_images/DSC_8218_7bb15c80-481b-411a-a54a-a36e9d8b6098.jpg\" width=\"500\"  />\n",
    "\n",
    "This image has a large percentage of it covered by brownish looking water. However, it's clearly not flooding, but rather just a lake. No problem, let's just make another rule: if some percentage of the image is this brownish color *and* there are also lots of buildings (let's also make an index for that), there is flooding in the image. Let's see if this works:\n",
    "\n",
    "<img src=\"notebook_images/DSC_8192_fe6812cd-c985-4e59-a62c-04a883ec42cc.jpg\" width=\"500\"  />\n",
    "\n",
    "Again, this is just some lake (the same lake as before, in fact). At this point, we might be thinking of shifting our strategy a bit. Clearly having someone sit down and enumerate all of the rules is impractical in this case\\*. There is simply too much variability in the images to come up with a set of rules that is remotely generalizable. \n",
    "\n",
    "This is one of the major motivations of machine learning. The idea is that, rather than enumerate the rules that determine flooding, we're going to just enumerate the outcomes (e.g. whether there is flooding or not) and develop algorithms for the computer to *learn* what the rules are. There is a rich literature (and a host of undergraduate and graduate courses) on how these algorithms are designed, but on these next couple of days we will instead focus on how to interpret and implement them. \n",
    "\n",
    "\\* This isn't to say that enumerating the rules isn't practical in *every* case. It really depends on the characteristics of the data. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A simple case study\n",
    "\n",
    "We're going to start with a simple dataset. We will use the  Breast Cancer Wisconsin (Diagnostic) Data Set (https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+(Diagnostic)), a dataset that shows whether a tumor is benign or malign and various other features. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting scikit-learn==0.23.1\n",
      "  Using cached scikit_learn-0.23.1-cp37-cp37m-manylinux1_x86_64.whl (6.8 MB)\n",
      "Requirement already satisfied: numpy>=1.13.3 in /opt/conda/lib/python3.7/site-packages (from scikit-learn==0.23.1) (1.18.5)\n",
      "Requirement already satisfied: joblib>=0.11 in /opt/conda/lib/python3.7/site-packages (from scikit-learn==0.23.1) (0.15.1)\n",
      "Collecting threadpoolctl>=2.0.0\n",
      "  Using cached threadpoolctl-2.1.0-py3-none-any.whl (12 kB)\n",
      "Requirement already satisfied: scipy>=0.19.1 in /opt/conda/lib/python3.7/site-packages (from scikit-learn==0.23.1) (1.4.1)\n",
      "Installing collected packages: threadpoolctl, scikit-learn\n",
      "  Attempting uninstall: scikit-learn\n",
      "    Found existing installation: scikit-learn 0.22.2.post1\n",
      "    Uninstalling scikit-learn-0.22.2.post1:\n",
      "      Successfully uninstalled scikit-learn-0.22.2.post1\n",
      "Successfully installed scikit-learn-0.23.1 threadpoolctl-2.1.0\n"
     ]
    }
   ],
   "source": [
    "# If the cell below returns an error related to \"as_frame\" not working,\n",
    "# you need to update scikit-learn. To do this, uncomment the next\n",
    "# line and restart the kernel\n",
    "!pip install scikit-learn==0.23.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from matplotlib import pyplot as plt\n",
    "from sklearn.datasets import load_breast_cancer\n",
    "\n",
    "# read the dataframe\n",
    "#df = load_breast_cancer(return_X_y=False, as_frame=True)[\"data\"]\n",
    "#df[\"target\"] = load_breast_cancer(return_X_y=False, as_frame=True)[\"target\"] # benign is 1, malign is 0\n",
    "#df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO2df7AkV3Xfv+fNvof03mIkjRZlA+w8l1GcAgGS2KIoIJhCVorIGDAunKwWRyTEi18VWwKbsoW3KghS6ygYCATKPxZQvDAvwnJwRSpsTGSBQIAjePrBSrIApWp/CFmlXUn80GoFkfad/NHduz39+v7q3zP9/VTdmpme6e7bPd3fe/rcc88VVQUhhJD+MNd2BQghhDQLhZ8QQnoGhZ8QQnoGhZ8QQnoGhZ8QQnrGprYr4MO5556ry8vLbVeDEEKmittvv/0RVd2SXT4Vwr+8vIy1tbW2q0EIIVOFiBzOW167q0dEBiJyp4h8If58jojcJCL3x69n110HQgghp2nCx38lgPtSn68CcLOqng/g5vgzIYSQhqhV+EXkuQB+BcCnUovfCGB//H4/gDfVWQdCCCGT1G3xfxTA7wFYTy07T1UfAoD49dl5K4rILhFZE5G1Y8eO1VxNQgjpD7UJv4i8HsBRVb29yPqquk9Vt6vq9i1bNnRKE0IIKUidFv8rAbxBRA4B+ByA14rIGMDDIrIVAOLXozXWgZDKWV0FlpeBubnodXW17RoREkZtwq+q71XV56rqMoB/A+DLqvpWADcCuCL+2RUAbqirDoRUzeoqsGsXcPgwoBq97tpF8SfTRRsjd68BcKmI3A/g0vgzIVPBnj3AiROTy06ciJYTMi00MoBLVW8BcEv8/lEAlzSxX0Kq5siRsOWEdBHm6iEkgG3bwpYT0kUo/IQEsHcvsLg4uWxxMVpOyLRA4SckgJ07gX37gNEIEIle9+2LlhMyLUxFkjZCusTOnRR6Mt3Q4ieEkJ5B4SeEkJ5B4SeEkJ5B4SeEkJ5B4SeEkJ5B4SeEkJ5B4SeEkJ5B4SeEkJ5B4SeEkJ5B4SeEkJ5B4SeEkJ5B4SeEkJ5B4SeEkJ5B4SeEkJ5B4SeEkJ5B4SeEkJ5B4SeEkJ5B4SeEkJ5B4SeEkJ5B4SeEkJ5B4SeEkJ5Rm/CLyBki8i0R+Y6I3Csi74+XXy0iD4rIXXG5rK46EEII2cimGrf9MwCvVdXjIjIP4Osi8sX4u/+qqh+qcd+EEEIM1Cb8qqoAjscf5+Oide2PEEKIH7X6+EVkICJ3ATgK4CZVvS3+6p0ickBErhWRsw3r7hKRNRFZO3bsWJ3VJISQXlGr8KvqSVW9EMBzAbxMRC4A8CcAfgHAhQAeAvBhw7r7VHW7qm7fsmVLndUkhJBe0UhUj6r+CMAtAF6nqg/HDcI6gE8CeFkTdSCEEBJRZ1TPFhE5K35/JoBfBvBdEdma+tmvAbinrjoQQgjZSJ1RPVsB7BeRAaIG5npV/YKIfFZELkTU0XsIwDtqrAMhhJAMtVn8qnpAVS9S1Rer6gWq+oF4+W+q6ovi5W9Q1YfqqkPvWF0FlpeBubnodXW17RoRQjoIR+7OCqurwK5dwOHDgGr0umtXL8Sf7R0hYVD4Z4U9e4ATJyaXnTgRLZ9hetzeEVIYCv+scORI2PIZoaftHSGloPDPCtu2hS2fEXra3hFSCgr/rLB3L7C4OLlscTFaPsP0tL0jpBQU/llh505g3z5gNAJEotd9+6LlM0xP2ztCSlFnHD9pmp07Z17osySHu2dP5N7Zti0S/Z6dBkKCoPCTqaeH7R0hpaCrh5ApgeMVSFXQ4idkCkjGKyShq8l4BYBPOyQcWvyETAEcr0CqhMJPyBTA8QqkSij8hEwBHK9AqoTCT8gUwPEKpEoo/IRMAT0dn0dqglE9hEwJHK9AqoIWPyGE9AwKPyE9goPACEBXDyG9gYPASAItflINNCU7DweBkQQKPylPw/Mfso0pBgeBkQQKPylPg6Yk59gtDgeBkQQKPylPg6Yk3RXF4SAwkkDhJ+Vp0JSku6I4HARGEmoTfhE5Q0S+JSLfEZF7ReT98fJzROQmEbk/fj27rjqQAMo4zhs0JemuKMfOncChQ8D6evRK0e8ndVr8PwPwWlV9CYALAbxORF4O4CoAN6vq+QBujj+TNinrOG/QlKS7gpDy1Cb8GnE8/jgfFwXwRgD74+X7AbyprjoQT6pwnDdoSp555un3wyHdFYSEUquPX0QGInIXgKMAblLV2wCcp6oPAUD8+mzDurtEZE1E1o4dO1ZnNcmUOM6TB5NHHz297Mkn26sPIdNKrcKvqidV9UIAzwXwMhG5IGDdfaq6XVW3b9mypb5KzhJF/fQ+jvMOBM8zooeQamgkqkdVfwTgFgCvA/CwiGwFgPj1aBN1mHlsfnqXaLsc5x0Jnp+SB5NO0oF2m3QJVa2lANgC4Kz4/ZkAbgXwegB/BOCqePlVAD7o2tZLX/pSJQ5GI9VIlifLcKi6uDi5bHFRdTyeXH88jrYhEr2mvzdtezRq7PA6VI2pYzz2uwTI7AFgTXM0VaLvqkdEXoyo83aA6MnielX9gIgMAVwPYBuAIwDeoqqP2ba1fft2XVtbq6WeM8PcXHRP+zIaRZ2wZbdd0/WTRzbJGBA9mLBz187ycvSQliXkEiDTiYjcrqrbs8vrjOo5oKoXqeqLVfUCVf1AvPxRVb1EVc+PX62iTzwJDWQP8Y+Yti3SqM+AA5CKQRcZyeIUfhE5T0Q+LSJfjD+/QETeXn/VSBAmP/1wmP/7kIZi795IabOoNt6z2ucBSHX23ZN+4WPx/zmALwH4p/Hn7wN4V10VIgUxmcMf+1j5EU87d5pdOjQbG6FM/zoHvZEN5Dn+0wXAt+PXO1PL7nKtV2Vh525JbB23vrBntVXKnv4qLgEyfcDQuetj8T8Rd8gqAMRpF35cTzPUI5qMr8vzj4Tu38dsZMxgbfj66U1/QZ9dZCSHvNYgXQBcDOAbiMT+G4hcPS92rVdlmTmLv+34urz9i6iurLjXy5qNybJkG1MYM1jWGm7Cmvax+Nu+rEj3gMHid4n+AMC7Ec3N+0IAFwCYt61TR5l64c8qw3DYrtvEpCIiYSqRpzRT5goqK5ZNia3PfuiNI1kKCX+0Hm5x/abuMtXC7yOOaeFtgqxlXlQlTErjc0wdcTqXFcsmxdZ1ylx/K/37/aOM8O8F8AkA/yJ2+1wM4GLXelWWqRZ+H3E0PbfXdbfa6hTS+NiUxqaAHfJJmA7B9zSUXb9KbA9yHTjVpAVMwu/TufuK2M3zAQAfjsuHSncu9AXfcMcmc+OY4vKBsOBu129NMYOGbGurV97WeN9w2Rj3LsXI5/W/i0SXUBomtiONWe1lykxa/MNhu7lxVlbKm4KmTuKkrqZt5ZjJY+zQRRxv3DKdFh+/L9kHxba9iqRdUMLV8x/zimu9KstUC38RZWjKf1CFOyl0G+Ox6mCw4dhGONiYrzzkEHwOryPdFbmww7fflBH+302VPQD+HsC1rvWqLFMt/KrhyjCro3UsHd2Ck52zTLtmzYcwA1G2pAIKC/+GFYBnAPhS6HplytQLfyhlFKfEurW3FxbfQ5sWf2h1u24t53nxfDxwbdBVG2VWqFL4zwZwf+h6ZUrvhF+1+B1RUK0asW4tUUBt+fiLVLfL/vHx2FzvrjVY0/xENS2UcfXcDeBAXO5FNGPWbtd6VZZeCn9RCqpVI9atI7R1jB06wkEVnNQRDrYuAG1a/FW3+7ZLoC2r2xb3QKqhjPCPUuU5ADa51qm6UPgDKKhWjVi3tiigDpqnbVmkRTNqqIaPzWvT6rbVte1Gf1YoI/yf9VlWZ6HwB1DwTm7Mus2alysrnX7eb8MaNmX08MmoEZqNo82nGtvTSQfa/dpo8poqI/x3ZD5vAvAPrvWqLBT+QApcWa36W9nDd4rx2CyGPoIY+rTQZj+G7Vi73I9Shqbvs2DhB/BeAI8DeBrAT+LyOIBHAVxjWq+OQuFvBupv+7gyfPgIYsj/2HbkUtv5Cpum6fNtEn7nZOsi8p9V9b3+Y4Grh5Otk75gm9ceqH6C9LYnsG97/01j+n9ForkSqqbMZOv/N7OhgYi8r7KaEeJJH+Z5seX4qWO6xLYnsG97/03TmdxOeY8B6QLgfwD4GwBbAbwIwLcBfMi1XpWldlcPfRydp0sx33VeLqbBzcMhL8tZoPM+/okfAf8awCMAjgB4pc86VZZahb+KZGWkejLqOho+3qovuMkUCLRDZptpieo5H8A3AfwZgK8B+FMAi671qiy1Cf80DXNU7aYi1FGnHLMIWLdGutR5Knzm0uni5UJIGeH/LoBfjt8LomRt97rWq7LUJvzTlLe2i76OuszfzP8yxg5jErcmTkWZicZMtJEYtY46kG5TRvh/LmfZ+R7rPQ/AVwDchyjVw5Xx8qsBPAjgrrhc5tpWbcJf1RSETVB1HFjRu97T/A3efHqFzPZMCdya+suKTjRmO9QyOfiqaHO7ZEc0TZ8avDLCfx6ATwP42/jzCwC83WO9rYinaATwTADfj9e9GsB7XOunS+MWf+ik401Q5UibMne9h/k7xuVhm3c0Ji5rv+6HNNchhwqmrQ1PC3sybUGyvEqXU9vx+23RtwavjPB/EcBvAPhO/HkTgLtd6+Vs5wYAl3ZK+MskRWmaKu/UMtvyMH9HgwfCNu9Q1rYtfpvgDgbhl4vtFM7PmxsX02CnIo3fNGYerYK+NXgm4feJ4z9XVa8HsB6Hfz4N4KTHeqcQkWUAFwG4LV70ThE5ICLXisjZhnV2iciaiKwdO3YsZHf+JEHEw+HpZeecA7zylfXsrwx5E6r6BnavrgLnnhsFSotEc/jm4TM/sMc8u4dPPif3q8OHDfH3tv2KYO/wI1hceNq128pj3NOceWb+8pMngf37Tx+Tz1gD0ykUAZ56Kv+7EyeARx9113Ma5wpuEtOl5js19syQ1xqkC4BbAAwR5+wB8HIAX3Wtl1p/M4DbAbw5/nwegAGiwWN74TGbV63hnNP07FfEOTkem83IImaPLcNmXKecmRXtp9fDDMvL7daEn9bHvZI6dK9LyfQ7n78o6LwGHldXL/sqocXv7+q5GMA3APw4fv0+gJe41ovXnQfwJQC/Y/h+GcA9ru3UKvzTeCVUkYylrGpY9h/cxnRYhXxPn21yc1M65OwpdO1jOCw2t72JLnZy1l2nDl9qtVBG+J+ByK//QgAXxGL+DI/1BMBnAHw0s3xr6v27AXzOta1ahX/anJ2hV67LJ59zh5W9+QqFP9p6NVvEJ6InqWrZS8nmw0/+4i6KdVU0JcqzfA6zlBH+O3yW5fzmVQAU0cxdp0I3AXwWp2f1ujHdEJgKLf4UoWalze+Ss06uJwcnFVjX0fBxb+9SoeiTzIpj7NCRHFbBuvEGNd3EVcXJu9xWaXEqeymNx6oLCxvX903XMB5PNh7TluZh2m7FaSBY+AH8EwAvRRSHf1Hs8rkYwGsAfNe0Xh2FPv4UvmalS30XFnKP0Rm6uPBUYLz5umZH3S4sGHz0qZ37zMFr+uuqmNvFdvrm5yNRzWtsqthv0eEVeY3G/Hx3L+Us0/bwPQ0UEf4rEA3AehzAl+P3X0EUlvlm03p1FCZpS+FrFtkU3GIKVjpYaTzW8fzbdB5PTqw/Jyc3iNTiouoYl59aYArhTO/bdIgmKz3EcrRt23Z5tHUp2f5u03F37bKnxV89ZVw9v+76Td2FE7Gk8DUrK550vZAFNhp5x+ADqqPBA6c+mAZtpfft638vYjl2yfr0EWjbucircxcfdLtYp2mnsPB3oVD4M/goQUHzqcrRoSriPeoWUBWs63j+bXFjkZ+UrYzFn1jrVZ2+JixmXzEMtfi7al137Slk2qHwTzNF4/cLmk/jscZpkE9uEG5fH3/SMxpi8Q+H0fZtlmt6lKxtSIHJAp6f3+gLD4m1r9Kf74OvyynUx9+lJxpSHxT+aaWMwpQ1n8ZjHQ936wgHVXDSO6onXee8TtqB5D8FbN7sbhzyOngTcQx1/bgs3bzTZ4veqcNith1T3rnIhoSaunNcx0BLezYoJfwAXgHgcgD/Nik+61VVei38XX0mt5Gp8xg7JhoPn5wzoSLtO9DK9jThwscNVrXF7Dqu7LnwtRNcx0Lf+mxQpnP3s4gmYvljAB+Py39zrVdl6bXw1/lMXpdD1VHnMpZ5+tDT1S+zPd921Kdxqbo9dgl09jIoMnq4qWMhzVNG+O8DIK7f1Vl6Lfx1WfxlndS2RiOu84Slj4M6Hu62HtLSws82uIVMh+5jfecVXx9/Hq4GpkorOX16h0PVuTm/y6CInUB//+xSRvj/0md0bZ2l18If8uweYr0HOnknNj98XMfzbzPXKY7f3zAAK+4YHo/z88Yt4Ge6go+faiyW8GPNi+5ZWfFLUWw6rKIPOk35xfP+ct8Gq4idYFuHUTbTTRnh/wqAH8bJ1m5Mimu9KkuvhV/VffcVsd59cvjEITR5Qj2PJ3WMHUalMEXzDIfR7k3CPcLBUx9s2ygi+lX8DW1G8gyHbhEuUseQEdBdna6C5FNG+H8pr7jWq7L0XvhdVGnmZe/y8dgotEMc3bgwVgpb/P54bHEv4OSpD+ZtmCdez8tgWVac077wtMulrlw4IVk5fPMU+dgO2e9Nl0jVE9TxqaI+GM7ZFnVc1dlt2oTbtg3PRPPmr9cnF6RGTtni90cjs9U+xDGPbZiFv4wrJ/Q0VZWHJ7vc+DQ0stfLVJ+iTyquKamrOM8crVsvZSz+lwP4NoDjAP4fotm3fuJar8oytcJfx1VtG7VkUwrTtjwsf5vwD/CUIum8Tbl+xthhFOikwzJX+JeePKUm4+Fu64CuDesO/U5fVV0htlOc3Y8pcVze8oWFja617GXj+5AXj6MrdHm4jr2KS3sao5WL0NZTTRnhXwPwfAB3Ipo5698B+EPXelWWqRV+01W9eXP128yKf0gHsNX3Ijrc/GTuVxuEQJ6YEP8hjhpvapc7I90mRY2L2cr3FR2f0bjZ0+PTFeKzH9N2TKJs8+ePx371CQ0FzTuO0HqHCnbVEUVddBu1+VRTSvjj1wOpZd90rVdlmVrht6lG0R4y1/N3SAcwEDmtV1aiYtjueLg7Nx1AriDgqVPhmyv4uDG1sq2DNj900Sz8Pje4y/I13ZyujuQ6BpPZhM8l5un6FKl7lpWVfJuiaGPie66KWPxddRu1+VRTRvi/BmAhnk3rg/GsWd9xrVdlmVrhtynAYFB6mxvi5Jd+yyz8LjWyCL+K6HjlVh3NHbEKcLYs4vhEeGa6WmVH74bcPD7dGTbLO9THHzKgzBSfb3Jb+fyNyTHbfheSpz+k4zdUzKoU6666jdocJ1FG+EcAzgDwcwDeB+AjAJ7vWq/KMrXC77r7sr/1eUaNn79zJyrB8dOuluRq8/VZDAb2OMLY6RySdA1IhWcmd198nCFZO0+XyUbHVyBcYulKaTzhdvKYFdLXGzc/b25wQpOrZUXOdcxLS+7zZqNKwa7KPdPVgWhTafFH6+JMAL/o89s6ytQKv6r5apybO/2b0LvIIsDpOPiJbfmY2B7+jrwGxyqqODk5YWy8/dAGBDD7vV3C4RJ22z4Hg/CkqHmnOunI9YnesQmDa51E5HyeOsq6QLrmT++qxT+tPv5fBfA9AAfjzxf2fgBXyBVvcqHMzZ1eL/SKtVjM6Tj4DfvLGy6bLqZjy6hI4mICTp6K6olecw5h8EDucYY2IGVCFU2n12c+3ZCb1eRSMsX7F+k49m0sfPoZirhluiT0Wbrq40/qNm1RPbcDeBaAO1PLDrjWq7K0Kvy+cXm2f3JpyX7nhT6jjsc6ksP5m8yz+JPi6qE1Hb+HQpqEfGlJo8nSBw9sGOmbbkBsImgbKOXTZtoEObSvIS9cMrk8QiNdioSK+qZp9h2m4UuXRTVN1xunpikj/LfFr/0T/qpi5l3CXuAZdbxyqy7KE5M3YtrHbyomdcobjumrHnFZwcetvntj/UajYD+676lNyItOAcwx86ZjCAmXzP6VeS6q0I5j29NL3t9na9hC4gu66kYhdsoI/6fjXPwHAJwfp2X+U9d6VZbWhD8kLs/Wg+S6awqaUxPWzfDxKPulrXFyNV5ZtQ2MSzTF7U9sPu+JpETugzIWf3r3RaJWioZt5o0dSIQ4+xfk/edlRuxmS/q3RfpJ2u44JXbKCP8igL3x6N21+P0ZrvWqLK0Jf0hcns308blbQ6J6XL8bj3UsOydDPRNL20etkrp5Hv8YO2LRd4d6GvsgCvoNbOI2GJzuULXWyTAIK+sZW1iYrGKZeQB8Eq7ZjjlZN3FX5W3H15VUpp+EFn+3Ya6eIvjG5fkOGy1zp5ue2XP2PR7rhlQHiziu47m3+ju1RyOvRiK0k9bUBzHGDh0NHtgoZsmTjOW82Sz0ECFOC+jKSk5G0vkwYQ0pRdo9l2D79gcUfWpq28dPf76bMhb/dgB/BeCO2N1zwMfHD+B5cUrn+wDcC+DKePk5AG4CcH/8erZrW53y8efF5dV5xY3H7miczB06GjwQJLrGknf8mcTwIWGZJh+/q/GYWM/Q0FUpwoBfV05gF4hXCbmcXILt2x9QNhtoG3SxIeoiZYT/ewDeAODn48FcIwAjj/W2Arg4fv9MAN8H8IJ49O9V8fKrAPwX17Y6FdVT5ZXls20fRUvu0Dh0NDjU01byfAmpSB/XQKy5uXRUz+W5Aew+jUfirsqOAvYV36oE2iaGVY1G9n2AdNXRVxzbcOOUva3oevKjjPB/3fUbnwLgBgCXxg3J1njZVgDfc63buTj+KvC9K32UIrlzHEJqtPhdzmpTwjeHaBsFLHPs8BrFezI374+P2M7NTf6ujEDbhMX3qcOnbyC0yyhd0oPOfB5Oq7CeQ4S8iv2xs9mPMsJ/CYBPAdgB4M1Jca2X2cYygCNx2ocfZb77oWGdXXFn8tq2bdvqP0NNY1KJbK+fryg7BkfZQim9FS97dy8tGdw066dyv6nqqfXGuDzy42NdR5sf0TEu1zF2eKVvMA0Q8ynZfDi+nbKhXTkhnb2uJ5AiQWJ5xTcnT9kuqBAhr8Jap8XvRxnhH8cCvB/Af4/Lta71UutvjgeBvTn+7CX86TKTFn+ZkJB0I2HoyduQwC1P9JPYfV8T2KBWpqiexcVovIEuLhobI58Q0Gi98Nw+g4F57JzPPleW/lxHw8e9xbBIG2r7TVWXjs88BWUIFeEqrHX6+P0oI/x3u35jWXce0Vy9v5NaRldPykdeSvTT2FRkaSl/IFpikvsIv6O+RvfS4AHr9+YQ0PXoySDl2y9yikJ+O9z85MaG0qAmeRayT39DXhSvj4Cl91fk0qmTUCGvylrvUmdzVykj/J8E8ALX73LWE0SpnD+aWf5Hmc7dD7q2NVPCX1UoSDafvyn6Z9Om6LuVldOKkQS4J1Tw9OHqUA6dP3eEgxNO/LxRwT7V9v2Nqm5QpLzOZNNfmM5DlxYjXx+77Te+DUpbwh8q5LTWm6OM8N+HaMrF78WhnHd7hnO+CoDG69wVl8sADAHcHIdz3gzgHNe2pkr4XXdxVXGHec/veW6bpaVoxq+8bSwtVRaK4upQNn0/xFF7f8TmzbluIgmYF8A3lXH6h7muqY3dKV5CV/bSsQ0nSf/WOJ1lza6eIkJOa70Zygj/KK+41quyTI3w+9wBVfj2kxJvd+ImGj4eDdSqah+exdWhbPx+uNvZH2FqNEJcHibxTI/GHQ93n6qHMdvoKMy14StwtkvHdslk3Uau0cZ1QSHvJoWFvwulc8IfapqlTUGXxT8/Pxk3b7PIR6N8wbAka/Pq9A0Q+/S2Jmbbytn2hn0Pd3udF2viNw+vWTo1gWk0bt5oZ5O4+1r8IUJs26bjEvC6NEk/ofBXRRHTLJvS0XQX56VYtE2JCASN0g0K8/QQ/VLb8u3lFDG7kUaTrpe80++bmiA0IsfHtWES7KyLRtVu1ZumZ8xeWmS2qKIRp/BXRRH18InfMwVc29YRCRqlGzywy1JKbSukl9PWyAR2lNraZR8PXF5WTdtN6XMqXf0GPqeSzB5VdYBT+KvCph6mfytv8pZs2bw5bH8FBLjKVA6Ft5V9+vFMWWF1E3libJdx0PjkFDr1YpoQ8S4a7JUN7iKzQVUBBBT+qnD9I3li5mvO5eWld6wb4nIxDZZq1OJPnyffAPYKTB9T1EtyrmxRPEXxDZhK59YJtfwZBjmbVJWSgsJfFUWEKHQsf9b37VjHp8N2jB26gJ9uWH0eTzbn408PGgsxaQKdnXmx9BtnnVzXIY5O1DedGrqKjlGPvy73kEPFn+6e2YMWv3ZM+FXDe13K3sm2XkLPbdri6ENFP6TBMTZsNWXZCnGZ5B57xb2lpikfs6ej6DH4VpnRPtMFffzaQeEPpeydbLoKLrlkcjSuJTFNpamayxZXR3j6uEejyeRuI/vFH9bGrtv3XxFp0U1P8G6bajHE7eOqMkfKTieM6pl24VctHrqR+P3z/BfZu3lhwRj7V2VET+li6wjP5EUI9b2HjY9bN++/xssgt6+h4Ly5PlWua6TxLDJrT0YU/qaoIvFKtuSFepruZoPwVxnDX7rYOsIzx2d0URnmrA1pW4dLTzZ6l7v+eltum+wTQ0iVbY0hOc0sPhlR+JsgJFKlrN+/QOqHKkft2hoZZ9m82a1c8fH55OpPn+a8v2B+fmN6B9889VXi+svrGoxly/Xjew5mzRLOYxafjCj8TRAaqRKSbCarCkVH/LRYchsek0nlsPhtpzlPpKoQrrLbcLXVdSVTs/Wl+4jaLFrCeczirF4U/qLY7vbsdz6ibXLy+qpaejtVJnwrUjwbLuNELWlXU1ah4/NmmuGr6Rs0RPxMl4yrra4zi6bPpWliFi3hPGbxOCn8RbDd7XnfucyqosMzTX6JKkU8lSLBqyQjkm2Nz/y8QbhTu013Ls/Pbwi4n2w01nUOTxuFv84b1FcUQi+ZphquMqI2i5ZwHrP4ZEPhL0KRvDy2iVqLuGcmJq/1rJ+l5LpbkuRwIcKf1MnxuxEOWX9iCyd1NRpN3qC+4uczsNv0oFRnw1VG1GbREjYxa30ZFP4iFM3qZbpyXK6ZZGasEJ+CbXtJ+DaxszgAAA8dSURBVIdFSBdxPJoXN/RpJPFLOBofV+esLZzU179f9w0aIta+CVrbsCyLitosWsJ9gcJfhCIWv80MsomkK02jyQHsk6zdIaSjkaNuprK0ZMqH4CXernBSn4gewXpV/3YutvYwT/xCXELTZFlOW31JBIW/CKEOW5/55vJUJJ2czXe6pfQ2jaoYm5mxGhlH74pjv7YyN2cdMWzqnM3myQltNE4J6uCBsP8zUL1MQp43dYLpL06nKCKkSSj8RQmJ6nEJSTaiJy8bZ8h0S651klFODiEtbPF7FuP4AYfLbDzcrYubNiaWO9XO4riOcbn//1jAX1GkYzOvzzvUNUILm1QBhb8O8tIo2BqJMtM22dQmb9sLCxvmGbSmPxiPN85LWKfoW+qZPi+nTjFO6hBHdYijk9vy7WEs2ENZpUfPt6r0qZOqoPBXjU9naEhET2L9uzqN0/u3NTqGBmSjEF9+ev3YV1/VCF/vNBHp/AumnARl1bBg3oIiuy0b/tinKBpSLxT+qvF1jSR3q48PfWHBbvGnE7bZ+h6KuG3i+vmKtU/jEJQYTtV9XC43mYHxWOPsnjl19chbEOp2KSvcfYmbJ/VD4a8a387QTAerswyH7nSMNp9+kQFigWLt2zh4p4IeDOznKO+4PK393LYkW9eKTemyDye0+ElVUPirJtTi942TTyzQlnLx+Ii1ryUfZPGHDB4LUEKjiKbrUIMpXaZzlj5+UhWNCz+AawEcBXBPatnVAB4EcFdcLvPZVieFP9THn6zjEvT07N4tCL+PWPta8kE+/iJPKh6CbXSbpOta0JSuM/KmzageRhTNDm0I/6sBXJwj/O8J3VYnhV/VPqTTFOit2oqg+xajWMtb/RuHzGhha1+AzXXlKlVY/AVN6Vm1ymf1uPpKK64eAMszLfyqxe6Ulqz5EPHfINapOQOtlvxgEOXb9xVuVySTqXgm1B+PVRcXnsqva0AHse9fOO1++Fk9rr7SJeE/BOBA7Ao622c7nRZ+1fCIk6JZOtssGbfKeOVWHcnh4iGfaSUp0hAG5DAeD3fnP3WUULNZjbyZ1ePqK10R/vMADADMAdgL4FrLursArAFY27ZtW71npyhpn33oUM2sI7Wou6OpkieSNleXreT1fYQ2hCFKVIOazaplPKvH1Vc6Ify+32VLJy1+H7EKmVmjqMujiVLF7ObJ70y9haGRTCFKVIOadd0XzkycRLUjwg9ga+r9uwF8zmc7nRR+X5EKuWNWVtoX+bzGK3sMaVXxsfhDs5S5zm2sRN7iVpOadTX6pezhdvW4SDhtRPVcB+AhAE8B+AGAtwP4LIC7Yx//jemGwFY6Kfy+lm6oVZmX4Wt+/nQag2xKg0suqUfw8+74bF9G6PZc+Ia7xqIfJG49UjO6a0gCB3BVTYhbIrGKfQUnVKTqEP+she5ybc3N2bfn8qcHjoswitvw8d4IvAl20JIECn/VFI3MqcNhWkd4aJJGITnWIp246eLq7/C09BO8BmbVdb47Di1+kmAS/jmQYuzcCezbB4xGgAgwHAILC+71TpwA9uypti5HjlS7PQA4eTJ6XV0Fdu06/bkuXMewvh6d85ht2/J/tg2Z7dRxvjvO3r3A4uLkssXFaDkhAGZY+FdXgeVlYG4uel1drX4fO3cChw5FovTII8C110YNgYuqhdqkggmLi1HDFMJgEL3u2ROJZ1keeyx/efI/qdrXzxxjrrjhCezFH2xct46GscNkbZLRKPqcajdJ38l7DOhaCXb1tB2T5nJbVP3MbXM7JRE1oa6pxMdfpfvIZ3Ia0zHk/HcbukKGu9v1cfSoA5lMB+iVj79uJ6frBrcJ2uKifaauothCQdMZQn3CMOfmTteprG/f5O8Pjdv3OfdtNvhtGxuE5NAv4a8zrMH3Bk8LWzqqZ2WlOoGwjRx2HbcrHDVpLHxEObutTZvc64Q8fdhSW5syoDZtdbNHlXSQfgl/nTdhyLbzRKiquoW4bhYW/OuRFXQfi7/O9BM+01faMqE2BWMoSQfpl/DX+djte4Ob6mAT2RDKhHAm7qZ4jt3SJXvcZbaVHqyWtdhtTyltu1Vo8ZMO0i/hV63vkd/3BrdZp1UIRNm8PsNhJLJlRT8dn18266jrf2q60zwE+vhJB+mf8NeF7w0eIsxFBKIrOf3T9S5TJ58nHlfD0rZbhVE9pGOYhH924/jrwjdI2hVbnzAcblzfZwzCZZcVPYJqEAFWVibrbYuXX1mxj3HwOV/JuU/GGBTZRp2kx3UcOsTAedJd8lqDrpVOWfy++Lo9su4J3ycKk3W9tHTanTQYRHl88rZXpgPWlG3Txw1WhUuEbhVCvABdPS2QfvR3ib8rciXbQIREkeS5IKrwx+ftJyTUtYxLJN7GGJfraPCACtbpXSEkA4W/akLFyzPHvLegVxFF4tswhfjTG/Rz0/AnxA6Fv0qKKI6PhT0a+Qt61apncv2Y0i13IEyREZSE2DEJPzt3i5CXuMyVBTLdKWziyBH/1IpVZ+L62Mc2ZhddWADe8Y7Opno09SX3LCcbIeHktQZdK52z+MuO0nSZqm2FBZr229EwReNpHDzQuboS0gYwWPwSfddttm/frmtra21X4zTLy8DhwxuXj0ZRGJ+LJMd9+qlhcZG5cwPJPY14AvvwW9iJ6+IFPK+kv4jI7aq6Pbucrp4ilJ3pggnTK2HDaRz8YFL0gV5OxEKICwp/EaoQ7roH+9gGgdU9SU2R7Res08RpXN82KfoJdPoTMkme/6drpXM+/q5jiiAaDqtNC+277yIRT1WmsWCYD+kpoI+/R5j6IIDoCSXvP/ftnyi6b9v2y/aZJLDvhJAJ6OPvEzbXhqmhr8odUiTGsqq4zI70nTQx3TMhZaDwzyJFkpVVleDMtB3b9ousY6LlRGnJQ8fhw1Ebe/hw9JniT7oEhX8WyYs6SiMy+bnKAVlFIp7KRkl1iCJj+whpmtqEX0SuFZGjInJPatk5InKTiNwfv55d1/57TeLyGA43fre4CPz2b9fnDinibumIi6YKOJqYTAO1de6KyKsBHAfwGVW9IF72QQCPqeo1InIVgLNV9fdd22LnbglWVyNz88iRyHWyd+9UCuq0UFU/NSFV0Hjnrqp+DcBjmcVvBLA/fr8fwJvq2v9ME9J7yMlBGmWGvFZkhmnax3+eqj4EAPHrs00/FJFdIrImImvHjh1rrIKdh72HnWaGvFZkhqk1jl9ElgF8IeXq+ZGqnpX6/oeq6vTz09WTgr4EQognXYnjf1hEtsYV2grgaMP7n37Ye0gIKUnTwn8jgCvi91cAuKHh/U8/Vca8E0J6SZ3hnNcB+HsAvygiPxCRtwO4BsClInI/gEvjzyQE9h4SQkqyqa4Nq+oOw1eX1LXPXpD0EjJEkxBSkNqEn9TIzp0UekJIYZiygRBCegaFnxBCegaFnxBCegaFnxBCegaFnxBCesZUTL0oIscAGOYSnDnOBfBI25XoMDw/dnh+7PTt/IxUdUt24VQIf58QkbW83BokgufHDs+PHZ6fCLp6CCGkZ1D4CSGkZ1D4u8e+tivQcXh+7PD82OH5AX38hBDSO2jxE0JIz6DwE0JIz6Dwt4SIXCsiR0XkntSyc0TkJhG5P351Tks5qxjOz9Ui8qCI3BWXy9qsY5uIyPNE5Csicp+I3CsiV8bLeQ3Ben54DYE+/tYQkVcDOA7gM6k5iT8I4DFVvUZErgJwtqr+fpv1bAvD+bkawHFV/VCbdesC8dSlW1X1DhF5JoDbAbwJwNvAa8h2fn4DvIZo8beFqn4NwGOZxW8EsD9+vx/RhdpLDOeHxKjqQ6p6R/z+cQD3AXgOeA0BsJ4fAgp/1zhPVR8CogsXwLNbrk8XeaeIHIhdQb10Y2QRkWUAFwG4DbyGNpA5PwCvIQo/mSr+BMAvALgQwEMAPtxuddpHRDYD+DyAd6nqT9quT9fIOT+8hkDh7xoPx77JxEd5tOX6dApVfVhVT6rqOoBPAnhZ23VqExGZRyRqq6r6V/FiXkMxeeeH11AEhb9b3Ajgivj9FQBuaLEunSMRtJhfA3CP6bezjogIgE8DuE9VP5L6itcQzOeH11AEo3paQkSuA/AaRGliHwbwPgD/C8D1ALYBOALgLarayw5Ow/l5DaJHdAVwCMA7En923xCRVwG4FcDdANbjxX+AyI/d+2vIcn52gNcQhZ8QQvoGXT2EENIzKPyEENIzKPyEENIzKPyEENIzKPyEENIzKPyEtISIHBKRc+P332y7PqQ/UPgJqRAR2VRkPVV9RdV1IcQEhZ9MPSKyLCLfFZFPicg9IrIqIr8sIt+I89K/LP7dUpyY69sicqeIvDG1/q0ickdcXhEvf42I3CIi/zPe/mo8IjS7/1tE5A9F5KsArhSRXxWR2+J9/J2InBf/bigi/zte/mcAJLWN46l9fiG1/BMi8rb4/TUi8g9xgrFepxUm5ShknRDSQZ4P4C0AdgH4NoDLAbwKwBsQjdh8E4A9AL6sqv9eRM4C8C0R+TtE+WwuVdWfisj5AK4DsD3e7kUAXgjgHwF8A8ArAXw9Z/9nqeovAUCc8fHlqqoi8h8A/B6A30U0+vjrqvoBEfmVuK5eiMg5iFIM/PN4u2f5rktIFgo/mRUOqurdACAi9wK4ORbIuwEsx7/5lwDeICLviT+fgSi1wT8C+ISIXAjgJIB/ltrut1T1B/F274q3lSf8f5F6/1wAfxHnhVkAcDBe/moAbwYAVf1rEflhwPH9BMBPAXxKRP4awBccvyfECF09ZFb4Wer9eurzOk4bOALg11X1wrhsU9X7ALwbUT6glyCy9BcM2z0Js7H0ROr9xwF8QlVfBOAdiBqYBFeOlKcxeV+eAQCq+jSiTJKfR/T08reO7RBihMJP+sSXAOxO/PQiclG8/FkAHopT9f4mgEHJ/TwLwIPx+ytSy78GYGe8738FIG8SkMMAXiAizxCRZwG4JP79ZgDPUtW/AfAuRInGCCkEXT2kT/wnAB8FcCAW/0MAXg/gjwF8XkTeAuArmLTei3A1gL8UkQcB/B8APx8vfz+A60TkDgBfRZQ9cwJVfUBErgdwAMD9AO6Mv3omgBtE5AxETy7vLllH0mOYnZMQQnoGXT2EENIzKPyEENIzKPyEENIzKPyEENIzKPyEENIzKPyEENIzKPyEENIz/j8zh1wy1UdhOgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# for now we're just going to focus on mean radius and mean texture.\n",
    "# we'll plot one against the other, and color code depending on survival\n",
    "df_b = df[df[\"target\"] == 1]\n",
    "df_m = df[df[\"target\"] == 0]\n",
    "\n",
    "# plotting\n",
    "plt.plot(df_b[\"mean radius\"], df_b[\"mean texture\"], \"ro\")\n",
    "plt.plot(df_m[\"mean radius\"], df_m[\"mean texture\"], \"bo\")\n",
    "plt.xlabel(\"mean radius\")\n",
    "plt.ylabel(\"mean texture\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So what do we see? Clearly there are factors beyond the ones plotted that explain whether a tumor is benign. However, it's not difficult to see a pattern here. The further up and to the right you are, the less likely you are to survive. So as a first attempt, we will have the software learn a *linear classifier*. This classifier will be a line that will classify you as benign if you are below the line, and malign otherwise. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[-0.7755968  -0.15309652]]\n",
      "[14.49306717]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.7/site-packages/mlxtend/plotting/decision_regions.py:249: MatplotlibDeprecationWarning: Passing unsupported keyword arguments to axis() will raise a TypeError in 3.3.\n",
      "  ax.axis(xmin=xx.min(), xmax=xx.max(), y_min=yy.min(), y_max=yy.max())\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEICAYAAABYoZ8gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOydd3iUVfqw7zMtk2TSQwgl9IAIKqCiq2JdK0EWURALYF8V1/atZfnp2nZhdbEs2LDCqkhRVwUVUMECKggqCCi9JySTOpNk+vn+mMwwk8wkkzKZlHNfF1fIO2953hCe85ynCiklCoVCoeg8aGItgEKhUChaF6X4FQqFopOhFL9CoVB0MpTiVygUik6GUvwKhULRyVCKX6FQKDoZUVf8QgitEOInIcTSmu/ThRArhRA7ar6mRVsGhUKhUBxFRDuPXwhxD3ASkCylzBNCPAmUSClnCiEeANKklPfXe5PtK1SxQQBrNm3n5+psjj35zFiLolAo2iipCXqG90oToT7TRfPBQoiewGjgH8A9NYfHAmfX/H0esBqoX/Gbf4+KfO2V07pJ3nzlfQy9R8RaFIVC0UbJTjEyvFdoh0q0XT3PAvcBnoBjXaWU+QA1X7NCXSiEuFkI8aMQ4se5H66JspjtCyEEbocNl9MRa1EUCkU7JGqKXwiRBxRKKTc05Xop5Vwp5UlSypNuHnt6C0vX/vm/y45n3ftzYy2GQqFoh0TT1XM6cKkQ4hLACCQLId4Cjgghukkp84UQ3YDCKMrQYenXPQPMG6goMZOcnhlrcToNM6ZNwmq11DluMiXx4JwFMZBIoWg8UVP8UsoHgQcBhBBnA/9PSnmNEOIpYAows+brh025vwdBpTYdt84IhIxfxBiJ1mUj0V2ChujEpm86dwBLdm0hOf2sqNxfURer1UK/G2fXOb771TtiII1C0TSiGtwNw0xgkRDiBmA/cEVTblKpTUdvSsUk3Ig2qPelBLs0UmmFJHdxVJ6RlZaEecOPcLJS/AqFInJaRfFLKVfjzd5BSlkMnNfce7p1xjar9AGEgDjc2HRGcEfnGQN6diFHbKXSUk5iUkp0HqJQKDoc7bhyV7RZpe/DK190hbzlouPZsPDpqD5DoVB0LNqx4lcA9OqaRkZclLYUCoWiQxILH3+H4rNvNnDnjFdxu93cePkFPHDT5a0uQ/9MI3s2fkXfEcrXH21MpqSQgVyTKSkG0igUTUMp/mbgdru5/YmXWfnqY/TsmsHJE+/l0nNGcuyAXq0qxx1jhnPV7E+V4m8FVMqmoiPQKRT/yGumYy6vrnM8MyWedW/9o8n3Xbd5BwN6daNfTjYAV148ig+//KHVFT/AhBO78PWaTxl8+sWt/myFQtG+6BSK31xezZBbnqlzfMvLdzfrvoeOFJOTfbR4qmd2Jj9sik1foRH9sli+zhyTZysUivaFCu42g1CdTUWMism6Z6Zg272eSkt5TJ6vUCjaD0rxN4Oe2ZkcKDhqZR8sMNM9Kz0msuh0WiaN6s+uH1fF5PkKhaL9oBR/Mzh5aC479h1mz8ECHA4n7376DZeec0rM5LngxAHYf18ds+crFIr2Qafw8UcLnU7LnOm3cOFNj+D2eLh+3B8Zktv6gd1Akox6HHYbhjhjTOVQKBRtl06h+DNT4kMGcjNT4pt970vOOolLzjqp2fdpKe4dO5wH33mKM657KNaiKBSKNkqnUPzNSdlsb3TvkkKmxkLRwT106dk31uIoFIo2iPLxd0BmTD6d31ctjrUYCoWijaIUfwdEp9VgKSkKmW6qUCgUSvF3QDQaDRNHpLNz07pYi6JQKNogSvF3UM4/sT/7vlqIx606dyoUimCU4u+gpCcnclz3eBx2W6xFUSgUbQyl+JvB9dOfI+uMaxl66bRYixKSQd3T2LX+i1iLoVAo2hhK8TeDqePO47O5j8RajLBcceZgKn//RgV5FQpFEJ1K8ZtLKxg/7TGKyypa5H5nnjSU9BRTi9wrWlx/zgA2fPhqrMVQKBRtiE6l+Oe/v5zSQzuZ997yWIvSavxhcE/clsJYi6FQKNoQnUbxm0srWLpyFS9e1pWlK1e1mNXfHjC5Sik6uCfWYigUijZCp1H8899fTl5/waCuRvL6i05l9d/3pxH8/s1HsRZDoVC0ETqF4vdZ+5NPTAZg8onJncrq794lhT4cpqTwcKxFUSgUbYBOofh91n6myduTLtOkaxGrf9L/e4o/TLqP3/ceouc51/HaeytaQtyoMKBbKrZKa6zFUCgUbYBO0Z1z9bpfOJxv553N+UHHu5t/4Z4brmjyfRf8+6/NFa3VuOTk/kx7Yy7dp/071qIoFIoYEzXFL4QwAl8DcTXPWSKl/LsQ4hHgJqCo5tS/SSk/iZYcAB+9/EQ0b98uyEhJ5Pju8eTv3ka3foNjLY5CoYgh0XT12IFzpZQnAMOAi4QQp9Z89oyUcljNn6gqfcVR7r98JLtWzo+1GAqFIsZETfFLLz6nsr7mTwuWkEraekGqV762I6RGo+Hsgcls/6HtxiIUCkX0iWpwVwihFUL8DBQCK6WUP9R8NE0IsUkI8boQIi3MtTcLIX4UQvw498M1dT7XumzYpbbNKn8pwS61aF1tq0natecMoWjn5liLoVAoYkhUg7tSSjcwTAiRCnwghBgKvAg8jtcUfhyYBVwf4tq5wFwA1s6uo94T3SVUWsGmMwIiau/QdCRal4VEd0msBQlCoxHYi/ZSZbWQYEqKtTgKhSIGtEpWj5SyTAixGrhISulPKxFCvAIsbco9NUiS3MWg2s03Co1Gw+3n57JgwzcMO+uSWIujUChiQNRcPUKILjWWPkKIeOCPwG9CiG4Bp40Dfo2WDIrQnHZcX6y/LMPlcsZaFIVCEQOiafF3A+YJIbR4F5hFUsqlQoj/CiGG4XX17AVuiaIMihBoNBr6dU3B7XKh0+ljLY5CoWhlopnVs0lKOVxKebyUcqiU8rGa49dKKY+rOX6plDK/oXspIsdcZmX8Ay9RXF5Z73njTxvAxiWzW0kqhULRlugULRs6E/OXraW04ADzltbNhApkaN+umBxFWMtLW0kyhULRVugULRs6C+YyK0u/Ws+Ll2Vy69L1TMk7nYyUxLDnP3H1KUx75wVGXTe9FaVsWWZMm4TVaqlz3GRK4sE5C2IgkULR9lGKvwMxf9la8gZoGJQVR94AG/OWruGeqy8Ie35yYjweZ9uqM2gsVquFfjfWdVntfvWOGEijULQPlKung+Cz9ieP8Fr4k0cksvSr9Q36+s/pF8eujd+0hogKhaKNoBR/B8Fn7Qe1nh6gadDXf9lpg9jz3TI8blUQoVB0FpSrp4OweuN2DhfaeWdz8Hzd7ke21+vuSUtO4LzcRArz99O1Z99oi6lQKNoASvF3ED6aNa3J16abjGzP36cUv0LRSVCKX8H4UYNZ+twHDDr57FiL0mhMpqSQgVyT6kOkUIRFyLba3jKQEE3aFC3Lh9/t4IvSrhx3waRYi6IIgUpbVTSW7BQjY07oHrKDpbL4FQCM/UMun72xIdZiKMKg0lYVLYnK6lH4kdXlVJSYYy2GQqGIMkrxK/zMvPZUfv7kzViLoVAoooxS/Ao/qUkJdPcUUHhoX6xFUSgUUUQpfkUQFw/rSWmhapiqUHRkVHBXEcTAnCwOvvsRA4edghBtcaRl50SlrSpaEqX4FUF075LCcV2gtCif9KzusRZHUYNK2VS0JMrVo6jD3X86mV/fey7WYigUiiihLH5FHRKMBlLi1UjGjoYqAlP4UIpfEZILh2by+ecLGfrHibEWRdFCqCIwhQ/l6lGEZMypuRzZ+j0upyPWoigUihZGKX5FWO65OJct3yyL+HxLWQmvTL9BzfFVKNo4ytWjCEuXFBO2LcURn7/+04Xojmxm3Sfvcu6kW6Mik/JTKxTNRyl+RVj6ds/AVPw5FaVmktMy6z3XUlbC719/wPPjenD70g8YecmVmFLSWlwm5adWKJqPUvyKepl05jHM/Xwhp15xe73nrf90IWNyYUBWPGNyK6Nq9SuahioCU/hQil9RL6cM7snb36zC6bCjN8SFPMdn7f99YgoAk0akcNXC6Fn9iqahXGEKH1EL7gohjEKIdUKIX4QQW4QQj9YcTxdCrBRC7Kj5qjRDG8FcZmX8Ay9RXF4ZdPyvY4ex7t2nw17ns/YzEr25/xmJesbkwrpP3o2qvAqFomlEM6vHDpwrpTwBGAZcJIQ4FXgA+EJKmQt8UfO9og0wf9laSgsOMG/pmqDjOV3TiCN8WueOn9awcJONUc8f9P9ZuMnGjp/WhL1GoVDEjqi5eqR3pqO15lt9zR8JjAXOrjk+D1gN3B8tORSRYS6zsvSr9bx4WSa3Ll3PlLzTyUhJ9H8+IMXDvs3f0/u4U+tce8uTb7WanMpPrVA0n6jO3BVCaIENwADgeSnl/UKIMillasA5pVLKOu4eIcTNwM0AL9838cSbx54eNTkV8PTbK+DQBu45M4Wnvy6HHidyz9UX+D93OF1c9cL3XHDbP1XXToWiHVDfzN2oFnBJKd1SymFAT2CkEGJoI66dK6U8SUp5klL60cVn7U8e4bXwJ49IZOlX64N8/RWVNvb+spat67+NlZgKhaKFaJWsHillmRBiNXARcEQI0U1KmS+E6AYUtoYMnQFzmZVbZr7F3AevDXLTNPTZ/GVryRugIdPk/XXINOnIG6Bh3tI1fqt//rK1uK3F/Lz6Y4aMHNU6L6RoMVThmyKQqCl+IUQXwFmj9OOBPwL/Aj4CpgAza75+GC0ZOhuBwdlAN01Dn63euJ3DhXbe2Ry8Bnc/sp17rr7AvyOYP6krF7z2GRUl95GcXn9Bl6JtoQrfFIFE0+LvBsyr8fNrgEVSyqVCiO+ARUKIG4D9wBVRlKHTUF9wtqHA7UezptV7b9+O4LhuRv7Y38Cyuf9g0gPPRPV9FApF9Iiaj19KuUlKOVxKebyUcqiU8rGa48VSyvOklLk1X0uiJUNnwqecB2XF+d00kXzWELX9/09eks7PX33CkQN7WvwdFApF66AqdzsAPuW8aII3pXHyiEQmLPJa9lLKsJ/V9vWHorb/X7iq6Z8q+WD2Q/y5FdM4OzvKR69oSRpU/EKIrsA/ge5SyouFEMcCf5BSvhZ16RQRUV9wFmgwcFsfgf5/j0dSVFpBkkFw4Jf1WMtLVUuGVkL56BUtSSQW/5vAG8D0mu+3AwsBpfjbCPUFZ4F6A7cNEej/D8z1P/ulQ3z57ktcesuDLfAGimijCt8UgUSi+DOllIuEEA8CSCldQgh3lOVSNIKGgrPhqC/FM9S5gS6jGRemcfUHH3HulX8OsvotZSW8+9RfmXTfv9VuoJVR7iBFpESi+CuFEBl42y1Q02+nPKpSdRIao3ijwfxlazlyaB/nT3uGlXPurleG2u6kfpkGesaX88OyBZx31W3+81pjGEtL0pGUpXIHKSIlEsV/D97c+/5CiDVAF+DyqErVSagvtz7a+Cz4Ub0NrPithBfeW8VD1+eFPT+UO6mw3EXp918y8pIrefepvzLmlumtMoylJWmusuxIC4ei81Cv4q/JwT+r5s8gQAC/SymdrSBbh6K2dd9Qbn20mb9sLWfnwOrtVTx3STzTVnzHbePPCStDKHdSYamFu/77k9/K//jFRxs1jKUjuIVay8pWPnpFS1Kv4pdSuoUQY6WUzwBbWkmmDklt6z44t97Wqla/b9E5p4eTvFwdp/TUc15OVYNWf22y0pJI1bv4/usP+M/Ybkx+Yx0XjToGiGwYS3tzC8UStXtQtCSRuHrWCCHm4M3k8XftklJujJpUHYza1n3eqGHNyq1vLj5rf9X2ShZdnoBeI7h+uIGrP6rf6g9FaeEhBpmsdDdIJg3V8fmWEgac3SNoGEsopV57Ru/ar76g2marc55ymSgULU8kiv+0mq+PBRyTwLktL07HpLZ1f/+cxc3KrW8uqzdu59ddFYwdAIVVbgqr3AgB5/d2N0oGc5mVI4f20y/JRXWlhYv665j8wRHmb3Kh1WoBMBWsCan4a8/onb3mIAPvrqvgVWAycpQ7SBEpDSp+KeU5rSFIRyVUVe3Lc/ay52A872y2B50baW59c/lo1jQuvXcO3xwx880ngZ/o6O6KXAbfgnZKjokv8zVcObIfk4vNbO82rkHffu0ZvW98fRBnVTn6hJRmvFnj6UjKUu2MFJESSeXuw6GO+3rvKOonVFXtLael1xl00pJEkiba1Nz/QHyZPv/9xcbhChfPr/eOZwxn5fv49v03cJblI/Aq+YxEPWOP0fHpxk/IPGNSs+VqDPUpy0gydtriwqEyjRQNEVEef8DfjUAesC064nQ8Gmp5HA1aK03Ut3hIKbnthc/pe9kDpHXJbvC6n7/6mBTp4LzZOzEleRVkWbETTfeN0MqKvz4iydhpi4q0veTzqwUqdkTi6pkV+L0Q4t948/oVEdASlnVjaGqa6Mhbn8dssdc5npkUx7oXb6/3WiEEN5w3iJe/X87IMVPqPddSVkJqgoHnJxzL7UsruXbmQkwpaUyfmkefa59qUE5F/cyYNolScyGb5gTvuLTGBOJjJFM42ssC1RFpSnfOBKBfSwuiaBmamiZqttgZctOsOse3vHJvRM896ZhevP7Fcjyea9Fownf7rh3U9WX9tEWXSXvEarWQfeUTxGX2Cjp++M27wNg2m/FuefVe3LYq//dOawnTp+Ypyz+KROLj30xNuwZAi7dy9/FoCqVoGvW1Z26NNNF0Uzy2qkoSwijrUEFdX66/+g9+lBnTJnFwzw6kCF5ANUJDj959W/xnFWuXi9tWRfepz/q/t5v306NPrrL8o0gkJkBgRY8LOCKldEVJHkUziGR2bjS5+08nctfb/+LMm58I+bnP2s9I1AM0mOvfWbFaLeiSMoOUIXgVovXz/0Tlecrl0rmIRPE/IaW8NvCAEOK/tY8pYk8sAsmBZKQk0i9Fkr97G936Da7z+Y6f1vBToY2Fmw4GHW8oCyhWxML95PPRSwkH3rjTf1xjSCDzwsh+RlqtFrt5f9Axp7UEU2b/FpVV0XhivbvyEYniHxL4jRBCB5wYHXEUzaG1A8mh+PukU7n+jffp1m96nc9uaWcTu2LhfvL56D0e0Gfm+I/nz7sr4ntk59QNwdkzs9qcO823sDqtJUELla/4ryPSVnZXYRV/Tf/9vwHxQogK32HAAbzSCrIpWpHMpLiQgdzMpLhG3Uej0VBVXoLH46k3yKuIDu0pSO5biKZPzaNHn9wYS9O5CKv4pZQzgBlCiBlSSjVmqYPTUMpmY7jpzByWrV3BkDMuarF7KiKjrVn1kdCeFquOQiSunp2B39S0av4/KeWj0RFJ0d45bUgvXn/xM3JPPhtDnLFF7tlWfKOtgVarxe2y4Sjc4z/mspgpXPgwPXr3bfHnxVrxdrR/v/ZAJIr/PCHEeOAGIBN4HfgqqlK1MrGehNXRSIyPI++EbDZv+4n+w/7QIvdsK77R1liAQvnoXVnd+MebS1vk/rVRirfzEUnl7lVCiInAZqAKmCSlXBN1yVqRWE7C6qhMOmcIK/7zbosp/lgSqOxLzYXoTemAtxp2yI3eoreWWoBibX0roktb+feNpIArF7gTeA8YDFwrhPhJSllV/5Xtg1hPwmos7WV3otFoSDUZoxLkLTiwG7fbDXgV8fSp3lKTaLl9Ancbh/bu8FfFHn4z8kybSFHWd8emrfz7RuLq+RiYJqX8XAgh8M7gXU+tNM/2SiwnYTWFtrI7iWQBuuHcgTy3eDanTrwz5OdNxe12+5Wv3pTuV8rtpeCoJdxFzb1HZ4qZKOoSieIfKaWsAJBSSmCWEKLBJm1CiBxgPpANeIC5UsrnhBCPADcBRTWn/k1K+Unou0SXWLc4aCwtvTtpzu4h3AJUu9lbUUkZHy79lOSUtAYVSkeYwRsJLRGvaO492krMpLVRC56XSBR/vBDiGaCHlPIiIcSxwB+AHQ1c5wLulVJuFEIkARuEECtrPntGSvnvpovdMsS6xUFjaendSVN3D/UtQLWbvdktZez85Qes275p8L71zeAN9I3W9rN3FMqLzUyfmkdZUUFQnx6N0JCSkdnplFM06KwLXm0iUfxvAm8AvlLM7Xjn775W30VSynwgv+bvFiHENqBHkyWNArFucdAYWnp30pzdQ30LUH5RKYWz7/OfK6UHh6UUr5cwPLVn8NYe0h6o8KZPzQv5n7c18LhcVBXsBsBeUcSGmRMB0Gnqf79IcDlsWG0uPGjQJh59d6HR0e/G2Z1OOSmiRySKP1NKuaimkhcppUsI4W7MQ4QQfYDhwA/A6cA0IcRk4Ee8u4LSENfcDNwM8PJ9E7l57OmNeWREBLY4aOtB05bYnQS+Y5PbNzewAHmElj7XPRN0TcmPyyj+9t167xuuXXOsmTFtEuXFZr+C93jcCKEF6SYhq3fLZvVotXSf+izVhfuD2jUcfj32rTgUHYuIJnAJITKoac0shDgVKI/0AUIIE96MoLuklBVCiBfxtnWWNV9nAdfXvk5KOReYC8Da2bL25y1NWwmahqMldie+d3xhySpW/fBLk3YPTVmAkgaOxLxqXth71teuOZSvvzVT4qxWC8PvP7rb8GX1HH7zLr/SVyjaG5Eo/nvwTtzqL4RYg7cf/xWR3FwIocer9N+WUr4PIKU8EvD5K0B0qlIaQXtI6azdgC3Qeo+EwHec+M53jB+a0KTdQ1MWIH1yF2+XpzA0tl1ze/dzh1u4RCPMm+Yufm0ln7w2KvjaOkSi+LcAZwGD8P73/R1oMDG7JvXzNWCblPLpgOPdavz/AOOAXxsrdEvT3lI6ofE7lMB3PL+3mzd+rODD351B50Sye2ioA6hGI6guOlj3uFbL/m0b6DW4bmPX9tauubmEU2C35Y1s9j1a6/poEe3ga1td8FqbSBT/d1LKEXgXAACEEBuBEQ1cdzpwLbBZCPFzzbG/AZOEEMPwunr2Arc0VuiWpL2ldELjdijmMitTH38TS3kZ712ZDMD953dnQ6mFxU/d1eA7Njb20S09iSF9u9Y5LrpmcvDrhSEVf1PbNYezDi0lRSSld6lzvCl58qXmQg7tPZrA5nY567mqeWiEBrt5P9LjDOrT46kqZ/erd9SrnGZMm8ShfXvwSE/QcSE99Oyb22YVfWujfg5e6mvLnI03CydeCDGco5v1ZLxzd+tFSvktoTf4McnZD0d7S+mExu1Q5i9by67d+7jiuPhGv+PIW5/nwOFCsvUWhk5+kvgkrw++vgHsge2d80ssuF1u9K5K3Dojn3z+DR+sHEV6ds8W+Q8YzjrcMHNii+XJb5pza9D82qqC3djN+3FaS4Lu1xIWY4/efUNO2OrVr2HFbbVayJr4WMhZu6EWR0Xnpj6L/0JgKtATbwDWp8Qr8FruHYL2lNIJjduh+M7tkazhjR8tLN0l0ASkHTb0jkfKqkhKSmTm6C7csayKvlc/giEhqd4B7IELQr9rnib5mNPQ/bYM1zGjyRk1ns1fvI9198amvn6rozUmBLVmcFpLSMvMIqdP/xa3HpU1qmgt6uvHPw+YJ4QYL6V8rxVlalXawtSqxtCYHYrv3HvO7M3TX5dDjxMbtZg5qqxMGGGiX6aRsbnVrNywgpxR4yO+3u12U/Xb18we3ZU7ln2N48QLkFUVeNztZ2Rz7cyd3a/eEbUumaGIVrBTBVE7N5F05+ywSr89EukOpbmxC3OZlTh3JZef0A2Ay09I4sMlXuUdKaEWjr4jz2PjW+tDnt+elFFryRqtYGdbrWBVwdfWIZLgrqINEekOpbmxi/nL1jL2GC3pid7r0xN1jM0VrNywIqLnh1s4skacD65qKi3lJNbEDHwEKqMtr96L2+ZtAFtq3tWkDpyB9wCvm2b61LwWUc6trTij+S5tiY70Lm0ZpfjbCY3Nrmlu7GL1xu2s3+rkg30FQcdF8mYiGYUdduHYuJJqm4PHb52AVqcPuqas6Oiz3LYquk99FgC7eb9/JmvtgGooRavTCHa/egc2cyHZVz7hP67VasnO6ReRcg517/JiM9LjYvrUvKBsH999o0ngzwOO/kx8MppMSRxa+DAe6UG63UhfKMftxm0wNrhI+BYW34Lio6MtLAovESl+IcRpQJ/A86WU86MkkyIEjc3bb27s4qNZ0+p02gTAboloAPvqjdvZsM3Foi1bg4675PtoDEYS+w2n+wXBmby+tgiREkohzZg2KeS5jVXOoe4d2CMoMNvHbt4f8X2jRSS9jOpb8HwLS+Ai29A1ivZLJINY/gv0B34GfD16JN6Wy4pWIJqVxfXtJJozgL2+haffNU+j1elwVJgxJGc26f7hfOzlxWZ/i4W2ppwVirZCJBb/ScCxNb34FTEgmpXF0epRFHK3AP7dgrvvqez87HVSRo7zf+bxuNny6r0R9cCpL4e/I+FzOTmtJUGLl1YbicOt4fv68N2/ufdVtA8iUfy/4h2mkt/QiYqWJ9LsnKZ0F629k8gbNYwHX3g/4nvU98zaffl9+GoAPEKPLj4pqOBIl5iGzXzQr+iqCnbhKx/x+dNLzYX85dJTcHkkm+YEt3Nozd78gfn9TmsJ9swsoGWyT8LtZso+f6nFGsPVdmVNn5oX5OJRdGwiassMbBVCrAP8JpyU8tKoSaXwE2l2TlMs99o7ifvnLKa88HCDU7V8CIeV3gn2sM9c88rfcdiqg445rRVopJuuQELPY6j8fQ2Jg7wtt7VaHSmZWfzjzaXMmDaJA0se9Q9c8WHM7InNfJDsK58IWaUaSDSVc6ACbunc/lC7mYIDuzn09oN1fO7RTnMsKyoICvYGPlcFfdsvkSj+R6IthCI8kWTnNCUGUHsnMXqQkRe+3cX7U7pzxVsrGXPmMHJzvIoylPVuLTrELy/dydvjc/jbF6Gf6bBVk1OrN3910UGOLHqI/Hf/D11iKo4KM2VrFyGEqLHYHYDXIg0XpIzUndOSyrl2X34fvl440SY7px9VNYuiTx6r1YLVaqmThVPbjeOb6KURmrAZO2E7htYMgamNCvq2byIp4PqqNQRRhCZckNRcZmX8Ay81eahK7Z3Esm1WrhqqI0FWktcf7pu9mA+eDB/c3fPZa+Sma1i7p5q8Afpze2AAACAASURBVLpG7TS6pXsXmyE3PUX+1nXYTD0x5QwGwiuUX1++E1m8B48pG4/HzZH//Quh1aExJNDj6hn+84T0tGgBUGCzNr0p3Z/KqjUmMOTGWex+9Y4Wr6AtNRdScGB3vVlI9dUR1F7gIsnyCfcOoaz9lqQ9Fe11JCLJ6jkVmA0MBgyAFqiUUiZHWbYOQbQmezV3qErgTsLl9lBhsTL/T0ZsNjt3n2bk7Dd3seNAod/qD8RadAgKtvLiFUb+3+cVvHRFd/5ca6fhdruhohB3VQXahPC/Kvr4RCrKCqBG8YelooCBXQwc6T8Kep2EIb0HEsiff48/6Om0lpBTTyfKpigZn4L1DWDxUdut1BBh/fZFBYx4cHHQsU1zbvX+/NowvvnAtWmswm6rFcQdnUhcPXOAK4HFeDN8JgMqChQh85et5cihfZxz6yx6d8vgzYenNnsBaImhKoE7iaffXgGHNjAgW4CjklQjTBqqC2n1r3nl77gKd3H7MMGgTC3n93Xx8teHGZwIQyc/SU73LNa9eDs6dzX9TXaOfL8Y43EX+q/XaQUeoKiknBUzbgLAZatkz7KXvJ+HmF3rrCwnFQuPXtyXv3zyJbaswegM3uwgnU7vD0raM7PqVTqxVDItlYUUuAMJDG77diDhzg1sLd0SBWce6VEKux0TUQGXlHKnEEIrpXQDbwgh1kZZrg6BT0GP6m1g4U8lFElLi6RNNmeoiq8/v0D4F6HVG7dzsMDGM6srSDeCt0xDUGLfQ3F5ZdD19iorGXGSC3Pj2Fkm+EMvA5M/qKbKKUHv4khZFeYyK71Ttdx1QiL/77sN9P3jOAwJR10tW4Au6Sn+uIF5z1aKCgrIPHUcP/1rkt+S9PnUda4qrhsq6JOq4dL+bhbsWoPd5G33ENgeuTmBzvrqAlobrTGBgnf/zx+M9mEyJfkXkP07tyI0Ryufjyz8PzbNuRWX5ai84VpLd+SaBuU6ioxIFH+VEMIA/CyEeBJvWmfbnFDSxpi/bC1n58CKbZV0NQmmn6HjqVXrmlWAVTsoGzhURUrZoFvJ158/xQjn3/EMK2ffzUezpvH02ytwH1jPpX1s9E7VsK/Mw0d745m3dA2ZSXFseukuLKVmpEsy7g/ppGUnI3QGMoXgspNLeH2tmYFpGg5ZrP6FKTdTz8XZxSx66W5/L3/w5vIHZgn9/vlCyg7tYu+qd5EeNyVFNYFs6caY0YOuWgvjB5eijzNy5Yk6li35hdSsqegTUrAHBDybQyzqAra8ei8ejztkWmpamPc66l4RQQPZtaY0uk99ln1zJgNeBeiz9F0uJ06HveaqxhF+TKQnxNmxR7mOIiMSxX8t3lGL04C7gRwg8t68nRSfgj6nh5Mu8XBGLx0ndBWM6ta8Aqz60jsBf0rn5NGn1VkEzGVW3v/iBzLiJQ+crueuz4p5YckqHrohj9Ubt/PbHguvrHWQbBRU2CTonBxTtJ11L3oXho9XrOanA5W8v83Bol+LEBpvqFNKid5j5x+jc7jxv3v4cNU63rsymUxTCg9luNhcYWHxU7eQkZLoj3kcKTvqw3bYqkk89mxST70CkZCCRu914+TPuwsqChg70kRqvHfaZ1pNz5+PNn5C5hmh2zM0hi2v3ovN7F1o6nOdaLXaIEvZt9Nozi7Dbaui64THScjuh8thx1chmT//Hmw2S8j+OmVFBV7L3uX0//wBXBYzdvN+NML7c7JaLehN6cRl9sJTXcGhl2/0nyulhwKNNqRbrTaxCvoqokskWT37hBDxQDcp5aOtIFOHwGftL99mRYubKSckkKgXXNTHzfRmWP3h0ju7HNqGvdrqT+mstDvq5PXPX7aWLnobZ/TWcXyWYMIQHe98tpbbLj+H1x+ayoT7nmPRhCQyTTrMVhcTFll44+Hr/IvY4+clMHlhJQnHnoW133nEd+kJgOWHxUyyvke/TCMD0iWjujvCxhx8QWlHVfCvXtKwi7D8/CnJp10ZdFzjqmbRrzreWmNFo/VP/8QuF1Dx29pm57G7bVX+Rm6hgrfhsoRachCL3bwft8uF0Hh/Jp5qC/FdetKvJmsoECk0dJ/6LE6HHaEz+I8ffH4KPfrkYs+o2wJDa0qn2xRvgzfpcuCpOBLU4C2WqDbMsSGSrJ4xwL/xZvT0rZmX+1hnL+BqKFtn9cbt/Lqrgj4mNxf005Fo8FpXvVM0nNXd0WSrP1x6py9AOygrjrNzqnhvxXe8NTGD8fO9OflpSQm8/8UPaO12Jg6NI90IY3J1LNth44Ulq0g0GurdSVzcD7oY7Fw3zMBLv6xE0/0UAFxVFeh3rmLcuV4l5HYTdtrX5NGn+YPSF849hKPK4vf9C60ejdGEo3APxh7H+K9zJfWg5+0vsvvVO3g8hOtjxrRJEWeXhFIyTmsJWq02bBZNapfsFnElhXu2IS6O7Jx+QVlDhqT0FqvQbSlq+87LigrYMHMiGqEhJWCxaazCVn732BBpAddIYDWAlPJnIUSfqEnUTmioUvajWdO49N45rN+6h9+LXcxef3TqlFarYVhly412rO33Fx4n5+U4Sdfb/Tn5o07I9Vv76UYw6gR9UgVjBmp557O19OnelaKSujuJjIPbcNqszD4fUvSCqcMNvLvFyt73HqZQm4DWVc11x3lINcRRUbCPZy5OYP4mJ6/9XIVT510QNRpBiXRw3JSnuHKgg0FZaYwbrOfdGt+/01qB9Lgx9joOR8lhCFD8DdEYn264jps+xRtN6nt2Y/ANZA/cIQAg6rptfJXLbmsJDp+byuP9PTy0d0ejAteNqRtQtH0iUfwuKWW5CPGL1RkIZdlHWinbWmMdA/3+ZquLVTsr+c+FBkorrP6c/IpKO1v32/h+r4t/rwWfMa4VkG6SnD9ycMiFyBf01bltZKRo0Wu0TB3uZu4vgmP6dqGy2s6SbYdYc0TDkVJrTYsFI1qthZw73gS81bq9TS52z7+PPx3j9d9Pv7Abv5R7ff8n3/4yHp0WQ2oGBZtW4hkwEqHVId0unDZLs33pPuorlmqKDz/U/Xw9+1O7ZAcdb6mskpSMTHr0yaXgwO6gXYpA1pF3yI2z2PLqvWAppnjZ0wBIt8tf+OZpowHa5qBcR5ERUZM2IcRVgFYIkQv8Beg06ZyhLPtodstsCoF+/xJLNWMHgESSYhD0TNZx1VAdm11xbF7wKMOvfoQu8RKNBjweKKryoDEksnLdNtZs3l3HdVU76OvD6fBgPryP9OwcuiV6yLdIymQy3UY/AoDtf09SbXcRH+f9Fdvz2WtcNlCSGucNYWaadFzcz5tZlBqfzLZFD+HxSNxOO1Xb1yK0OoT01FuQ1VhCWa2+YqnaDcoiyRYKdb9De3dQvPTpOpPEAqeIQd3WCr7KYIis2VztnUJteX33tpkLybriEXRp3niMECB0Bm/gvIaOlALZ3uSNFZEo/juA6XgbtC0AlgOPR1OotkIoy15K2axZti0hU+0dSODO4tJ75/BVfhFLtlbQJUGDRmPzKvjqPbz43mruPiuDe848mlrpG8IOsHTlV3UWMZ/L6nDhUbeAy+1BI6zcfaqe+1fsZs7FBu5YBVZ9sj/gK7Q6qMlTcdus2A7+xooSD+9uttAt8wgajaCi0ka1zc7dV4/0P9PhdHH9nFWcdcfRaVPRpL6c+ZagvgEngYo6WPk6wtYmRGrR+hSgb7Fxlx8Jlstaigavxa9SIDsfkWT1VOFV/NOjL07bIpRlDzRrlm1LyNRQbMEX6K2t4Od/uQGdRob144dzXdV2Wfnu/3uZnbGDtJyWoyOvVzVzN9nZ8/o9ALgqS/3nO3esYeofsrj5tDTmLN9B7+GnM3n0aUy47zlezOsa9EyDXsdZg7uyY91Kckee36ifzc8vTMPjsAHgrirjtryRgLca+D8f/RDyGl+/ndb0Uxcc2E2pubBOUDoSCzvw88DFIrBZW+37aHX6Op1MDUnpmIxq8mpnJZKsnpOAv1F39OLx0RMr9oTrg28wmjCXNn2WbXPkmfr4m1jKy5jbQGwhXMpnr+zMkHGHwIyghlxX/l3Q6AQmzi/irXFxxOkE1w/X87/dAiY8ijY+mQPPTwG8WT9xB3/g8kleN8b4wQYe+Wo9lTaHf1E9IaWQoZOfJD4phfwSCx6PxGV/G63BSGqXbDxuFx5LIQ/N+wJTSlqQPIEWsLOsgOwrvQ3bhEaDMdO7+/AVNQUSOLzcUV7kXySE9Pj989FydbjdbvSm9DpWdmMt7Na01Nua77wjuadiQSRL/tvAX4HNQMTRICFEDt7xjNk1182VUj4nhEgHFuJdSPYCE6SUpeHuEyvCFUrRI3QQtCk0poGbr+L2iuPiGZSVHlZBm8us6PU6ls/5a8TDVHyKfPwbB5gxOqtOw7XacuQN0LDo53LO66shN10DAvqnafhTfwfvfPtf4oZfinQ5OPjSDRg8dq47TpJQZcdWBV1qfPuvf7aWb27xKthxx+hYlp9I36sfwXOkivguPSlY8Qq2g1vpd+NszN8uwPTzfNZ98i7nTrq1jkxNIXB4eVXBbhKyvT7zw2/e5VemLaFAHdZSqgv3Iz0uf+aQ2+XC7XY1cGXLUTtwDd7gtSmzf8T3aGvKVLmnmkckir9ISvlRE+7tAu6VUm4UQiQBG4QQK4GpwBdSyplCiAeAB4D7m3D/qBJJH/zmEunwlMCK20v6unG5PWFjC7PeXsHuHb8z6+0V/PO2cWHvGShD3gANy7ZZKa10sHSrlbwBOp5fvIpfdh4MGew9XGhnX0EZyXGSlbu8CkwIsDhA6jaSfNFtlKd1JcvoZmhXAz8UmvnhU98dJBWVFhK0Lv+imhavYWyuYOWGFdDzDADST87jwPbvcVaWY//+bf51QRx3L5zNyk8+RKP1XhfYuwag7Nkb/W0MnOYDdd41aHdQa5RhUwhlBfuyenzHpceFLrUrAvyN5aoL99OcSaaNadRmMiVhXf5M7Vu0aAFaU1AWe2yJRPH/XQjxKvAFwRO43q/vIillPjXjGqWUFiHENqAHMBY4u+a0eXjrA9qc4o92KmZjhqf4Km5H9dHTJ02LucxKdkZyndiCuczK+59/x9PnGbhn5VruvfqCBq3+1Ru3sz+/mgqLldkXx3HHp6UkJ5lAbCBdX7fQzPdz8QV8DxaWk2AQmAwaEB6k9FBddBCntZTMLl34aFbdnv6+a096vpCDRRVUO9wYEj3YHR+SMOZ4nJ6DCAHS7aD4+/e4dKCG3H45XH5yFR9pz/S3aojEujv09oN4PO46/nQhPf5ga1Nz+CNRULePPomid4J/vR2WEnTJXZr0TAjfKnrvnClsmnMrTmtJ2IEroWgJN05jFbmy2GNLJIr/OuAYQM9RV48E6lX8gdQUfA0HfgC61iwKSCnzhRB1G757r7kZuBng5fsmcvPY0yN9XLsgXEpobfePb4GIc7p451cXb21yUFRloUtaNRqNCNqBzHp7BRf1lZzTV89FfW0RWf2+YLB193pe/8XKdadkILNPYPW6X3gxL7nOouST742Hr0NKyUnXPMKCGwaQoJeUVLqY/J4Ft83qPddiZ+Stz7PuxduD3itwUe029u/0SbSSn9yX/o695B/ehOHUSTjMB9Dq43D+9gWXXeitCr78hGQ+XLIS54hL0Cek1H2ZEHgc3n44vQYMDsp9d7/zYJDrJVqkdskOmUKaMfrOln+YlGEziOqjJSxspcjbF5Eo/hOklMc19QFCCBPwHnCXlLIi0kIwKeVcYC4Aa2c3fV/cBgkMHJutLlZtL6fstx+Yknd6HfePb4G458ze/ut9KZiBlrjP2l8yXk+CXnDjCD2Xv9ew1R/YTK600gFuJwuWr+X6E00hg72B8gGMG6wnoSofh8tNklbHmB425n08k8TM7gy56VH/cPVQbi1zmZU4dyV3n2nkkVVbeHRiH+745GvcJ4wGQOu20zse0uK9zchqN2grLzbjkR5+/OflILRIj5sDs685+nIeN2i1/o6UbrfbbyELofX/vapgd9DPxFlZzuH//QtPlPzwLZVCWtt3L6Ubu3k/Wq22nqsUisgU//dCiGOllFsbe3MhhB6v0n87wDV0RAjRrcba7wYUhr9Dx2T+srVc3A/Kykr5YDtU2pxkJtp4fvEqr6Ud4P7x9cp/7utiMlIS/f1vascafNb+oAzvf/pBGVou6uts0Or3NZNbtb2SF/MSufljK1q3h0v6exeLwFiCr4bhnxelMX7+CtBo6Zpi5JsCwSFzVU0Bkon07K4cP/kR/zN8i8t/xh7tHZSbk8X8ZWsZe4yWX4+4mTREQ0+OkNfNyRtvTsPhliS4K9l0wMOFb4LOJNDqapqYJW+EMybhkR6yr3yCwo9n0W3Ks0iXAynBVXoIgKIlXhn0cXF1X1wIfyM2R3kRokZZCulhz9w/00tfxhFN15A/s+b6pxubQlr7eT7ffm2ffoFGW6cQraPS1rKM2huRKP4zgClCiD14ffwCkA2lcwqvaf8asE1K+XTARx8BU4CZNV8/bIrg7RlfNewzq6ox6ATPj47njk+r2fH5eq4fYQyytH2umKUrvyLv/NPDBoHfXbEOrdvFmv1HB6eU2STuXT/4FX+oLCJfM7mxA8AtPQzrCpsLJQacOF0eyspKubifPqiGYe3eanKS3ByscHPNJedxz9UX0O+ap+sMZHdUVlBmPsKL76/2ZkgZHP7eQa/8bQpLv1rPXSfoeHKtk7cuS8Bil1w7qhcrCyuxuwX7FjxGYamF7uOfIOvCB6j8dh7xZ0xBazRxaO8OpMeDVqtFul01Sl8idHrQaDGkZvsrYUM1YDOY0jh+2otAcDGVpayEtx68kkdOdDNtaREPXnuRP5gMdQPKgYRTRM1VULWf5/PtN3b8Y0dCBYCbRySK/6Im3vt0vL38Nwshfq459je8Cn+REOIGYD9wRRPv327xtUA+5dg4HPZqLjyhB7dWlPP6hkomj8gAjlraeaOG+YPAN/zvB1Zt3BFyfKMEqt2C6qCBWQJ9wK4/cE7vzzu8GTs+WaZflESKUcNl1kN8vN3NFR94qLIdpLraRny8kf75NW2fRyfw5yWHeeESIxMWV7NoxXdMyQsdfzmycSU9Ehy8t+I7Pr0ui/JSs7930Kx3VpA3QMO3+5zkDdSSnaRFK9xoqGJsruCtjd44QVZaEng8VHw8k/7GCnZ9PBNPvFehS7eL7Jx+FOr06A1x3lbFQoNA+DNoGsv6TxcyJhd6JQsuPzk7KJgMjfdZR0NB+Vw8gdPHoO0ORwmFsthjS0T9+JtyYynlt4Qf+HNeU+7ZHogkN9/nXlm9vYrZF8dhLrNySX94+0c7vhiIr27g/jmL/UHgLnozBYf3BfnJR976PGaLHU1SFoGqLjMpjnUvHs2oqT2nN0kv61QiFxRXkG3ScM2IJPS9R7Lqh19qKmurOO2E/iSaN7Nsm5WL+wuOy9Jy1XF6Pt5p9d/Hh6Oygi1LniHeXsRZPXSkGJ1IZxUpRkG2SctVQ3XMX76OzGQje/Lt6DXw7BpvMZWkHI1Wg9Ae7TWfkpSIQa9hxuUDuGNZJalXzUSfkBLRdKxAf3p9/XBmTJtERXkpCZb93DwuAbfLw+ge5Sz+39uNCibXvmekLqH6zi0vNrN/51Zq/3eSHk/QvWZMmxQzZdpYRa4s9tiiarZbmIZy8wODqXm5OgZm6thdUoleo+WcPhpOnn2Q9KR4ADweSVFZMa/c3Quz1UWJ1c7DZxqYGTDIxWyx13GxAP6gqm8hGjawJ3kDNPRL13NejhNDXLw3WyjeRFGJnbd+OUJRaQXp8RpKqj2k7vyR64bF+d1O87/cCB4XFRUW3rg0jh0lHi7sr+PdX6v54IsfSI1PYcsr9+J2u6koyic1zsPFg/RsKfSwt8zNgs1HyDZp/c3h3C5PxEVmBo2kq76aPhlxjM2tatT0rUB/en39cKxWC6lDzuRS9+dk90rHabeRpTNw6cAdLG/itK9D+/aQNfGxuscXPlznWH3uI4/0IDT6oFGLAJqElKDFIpbKVCny9oVS/E2kqe2afYHdFVsslNrgnV8dNWMODSQnJjK0/9G2Cr5WCgAXvLyfSUN1DO+mCxrk4na72fT2Pzhm3F+CBpq73W7GP/ASw3J7cuTQPt7be4DPb+xKcbmV64cbuOPTKs4ZZCCxr7cSObC/z2MrS3lvcyVndJO43ElMHpHI0p0Wzjl1BM696xiUBToheGiVlWtPMLKxyMa4M0/hnqsv4LFXl/LGB4fINmm46jgDvbqY2JVfwe2f2lh5ax8yEr2+p6e/Lo+ov5Ev82fyyK58u9MSlNLpm47lspjZN2cyHo8bIbQgBAZTWh2rvj7lNH1qHlV7NrKowsaiXw/jqul37650ouvhDSY3Fo/01OmR4zteH4HtJJzWEjweN9Ljwll8EH1Gz0bLoVDURin+JtLUds2+wG51tZv4eCPJiUaS46B7Vt0+Or4q2ee+LUXrdnJKdz0l1R5GD9Dw4FfehcVRZaWHtoIjG1aQM+roKGRHlZWS/FLe3rGPs3u6idM6MWo9HLJUkp6goazKRVW1nVVfHY0jBA5yOaenkxSdCCoWm//lRkrKKnltvQeb041WwOq9Tgx6Pe6N3glbC5av5fiuGv7QU0vvVA3F5VayTIKze2s56bkDZKQcVcaRVEH7Mn8GZBnZXuQISumsPR2rrrvE0ahe/r2ufcr/98AA6vEBxyH6/unAdhJ2836Klj2HeeksPNUVaGtcVQAavbFFnqdoe0S7slkp/ibQlHbNvh3Cv++cwJ+feIX/XJzF+Pn5LPjnbeTmhKxh46NZ0zCXWRl10z+ZMkTQNUlH/5yu6LQa8grKeWHJKuLclfxjdB/uWPY1jhMvwJCQhKOygjh3JY+fl8pt/ytma6HkUIVk4a97SDZqcXk8pMbB6+vLmXZmlj+O4GuhsHZvNZvyHXyxx0WZvcpfLNarayY/v/Uwv+87wnl/nkn/NMERexwD+vTgjYev4/nFq4jz2LA5JQu3uHh7s5WSag9dTTqsDklKcjI//reum6M+Vm/czoZtLhb+uhuLTfLkisNohMAuF5DWZ2jQuU35D2EpK+Hdp/5aJ2c/XAA1Fi0Fuoy+E48HzMtmkXPdc/7jdvN+7J//p1VlUbQO0S6IU4q/CTSlXbNvh3D/nMWcnQMPfJzPeb2kP7UxXEB4/rK1xOFgyVbJ6xudoDtIcqLX0nPJjYw9Rku/TCNjc6tZWWP1H9m4kksHack02Jk4RAcShmdruPqDasxVLkwGwcNnGblnuY1nvyokKcHCgXyjvy9RiQUmDDVw5fHx3PpxNePOPZmHbjjaAuDuZ98lNU7y/CUJXPV+Fdt37uWFJatYsHwtU4bquOsUPRINO0tcLNvpwWA0MfmkFCYsslBcXhn0jg0FwwN3Qct+2MFnpT04/o+NSwSrz3o65fQz0R3ZjKdKE/SZb9BJJANZoknRsufwOL2tpj3VFRx4/S8ACEM86X+8hVJzITOmTVI+dkWjUIq/kTSlXbNvyPh/xmbwpzd2kTvUhMXmoku2jhW/7uLpd1aGDAj7nvX5Lb38YxUnLLKw+Km7kFIy4b7nmDjUiK3oAGN6elj8wRLMm1ajqzJz+Z/iSNJLrhum5+r3qym3axiYoWF7seSKk7vRP7OK608UzP7BxtABPXn6zgncMWsBW3bn0ysjkekXpTL/x3JSDJW889labrv8HDJSEvl+8x42bNnNPX8w0D9dMHGIjnm/uFj42Vp0HjsLNrtZvMWJyyOpdHjTTF0eB1NOTg05tyDSRnUAo0/J5ZO5X9HYDOBw1tPOl2/l968/4PlxPZj02g52vnxrUM4+NM99I6QnZK59qLTLcA3kpMtO98lP45ESV8kh/2zdIwsfonDJo8Rn9gy5qCkU9aEUfyNpSrvmp99e4S9gunKIjsW/VvDehHj+8qmdC/ppeG/lWj66rnudgHC4ZwXuME475mh16V+Ky/nGnMSoTEl2sp3iKjtaASd01fDfX5w8eb6R/1tlZ0zvalKNgqkjjCzYZGP7zr3c/cxCCg/uRe/xkCI8QCpLt1Tw0phEJi6p5IUlq3johjxumjGPLgmCycfrEXgXlsVbXKTobewsEyTqNCQbBeVVLspsEq1GQ+9U/NlKgX79xjSq86EXkiqrhYQwCrm+ubq1xxV6qsoYMzyZAVnxTDmjO9u7jW2xts8APfvmht5pZNatrg202KdPzfNX4BZrdf6aBJcQ/vbRcSldggrQ6kN1wlTURin+RtLYds2BO4TS0hLO76flg21QYYcTsjUs2OxkwhA9/TP0dSzi+p4FcLjQzvyfC8gvqaRbeiI6rQZzxQEO5Bt56XsbHqedJIOk0iHpatKwu9TDVUP1pOsdpMUb0WngqqF6Xv/JyebfdjLnEiNPfG3n50M2cmfuZuIQHW7p4dQegtmLV3H+KUM5dKSEyScYKKqUFFZKdBo4p4+WlXs9xOskS2/MYWBWHI8vL2TJL+V4tEZmnq/nH+sMvD/r7iDFHkkwvLYr6OGJJ3PXwmcZdd1DIf996purG4izshyTu4JJI7wpkpNGpHDVwg8YecmVdYa9NJWmKtXw7aOb1rJKNVBT1EYp/kbS2HbNgVZ7ijEThzOfa4/X8/1BD38+KZ6Pt7u4+SRvEdfoQUbOm7uSIf178ueZb7Fi9l1hA78+jrZzOCNIaV5853P8/NseLHZJnBbm5hl57Gs7e8s8LNziJMno7bAtpAe3lIwdpOOk7louG6znrU1OdBrolm7i2L5Z9NheSI6pglv+OY/uSRq+3e9m6XYXTrckPUEDCHomaxnZXbB0q5XJCVpW7azkrD46BE56JWmCUlAhvMusttV/ys1Pk+wp80/oAigpK+fLtRv4+yv/a9S/RSDlP33C2GN0ZCTqAchI1DMmlxYd9lIbn+VdVlSAFEdjChqhISUj02+Bh7P+m9o+WtH+iHbmmFL8USbQaq+otOF22DAZfFCTngAAIABJREFUIDNBQ2GlhwnH6pBSYqmysWyXkxyTmz/PnE+6zs59sxfzwZN1+9n7qM9Vcv7IwdjLC9ld6mZcrpu0BMHTFyVw7/Iq9pR6KKhwo9FqSYnzWpFXHKsnI16QN1DLKxscPHuRkZlrK5g0PIVVOyt59Nw4bv2omAEZWv5xXjwOl2TK/6oosEr0eg3Fdg+bj3jwUMac761c0Fuy4bCb5y8xYjIILunr5v6AwrP63FiBi4O7qoKZV/fjjmVV9L36EQwJSdit5Xz/8gPN+nep2rORxYedrCw4GHTcVLAmaorfZ3lvmnOrP10T8LdRjqYFHlgb4MNpLVGB4Vq0FbdYtJ+lFH+UCdwh+AaQAJQD8zZb0EgPb/xcRZc0LdbKCv5zURy3La3imbxErv9oFzsOFIa1+n3FYFpHhb+Rmq+nv6+L5uhXD/LloWQWba3E43aRZhRUO8HlkVw1VANo0eBhYKYGvRY+2+nmquP15GZoGN5VMOr5/Yw7Rsv9K6rJzdDiQfB9YTz3nJ3BnWWh20NPuO85eqQ76ZdWxaAMLfsrPPRM1jIitYIX3lvFrZedzXOLVpGWqOedzfagdwp0mfly+GtnLWkNcXicDjweDxpNcDZOKMK2Qe49jHvbuNILtPx8bagBf/Ga75yGCKwN8GE378caYTpoW1GI0aazuMWU4m9Fwg06X7ryK1KyshmVWcExXR1ce4KTHw+5uWqoLqzV71Pus88Hl9PBaVkO/lZjUfus6bV7qxmYoSE7pyuX/7E3Fdu/5fphOuaut7Fsh5MlW53YXJJ4nWDJVhduKTFoBfP/FE+1EyYN1fPxdhtf7/egQbK7VPKfi43ctbyE+Zuc6LSaOrEN37O/3G7jQImDhb86qbBLbE6JB6j+5Ute+HA92Xo7Fa54dr0TOq/f936PnOIbwpLEh0uO1irEx+nZ//uv9Blcb5NYoPFtkEPRkorP7XbhdBxd8Nwu7zze8mJznXPbilLtLAqxs6AUfwz5fd8RXl6yktcuz2Lqwl08c3MPLGUV3H6ygWvetzE7L4F3FoS2+n3Wvs5tI9Eg8HjcjOpm44Ulq7zN1UYncOuSw7wyNpEx7+xk556DvHCBQIuHG0fo2ZDv5txMDZ/uctPF5G2jUFLlZkyuDo2A+z+3MXdMPGMHaVm63cWAdC3HddVwUk8D14yII3XQGUEumVtmvsWM2y5j9uJVpCUasFYDLg3xOm+MocoJb41L4OoPbGhclTx8gYm7P62ok9cf+H55AzSkxmuoLs5HxqWjrzZz6LuP6XveVSQmxLNr5Rv0GRw8TzZavtGWVHze9tFHm9AJjY64zF4NtnJoKiZTEqXmXXVmDKuBLZ0XpfgboKECo0i6cYY7/4Hnl5DXH774vYyrhuqospSRHi/onqQlb6CO5dudYa1+X+uHuWvs2FweUo2CkmpI3/kj1w2PY9k2K3m5OoZm6RiUIRiY7iQzwYDbIxmYoWHsIB17yzxIKXFoEjDotVjKylm81ck7mx30TNFw5huVuDzQN1Wwq9TN9FEGiivdjB6g87eM8PnrfcVpfVM15J1/ek1sw8whcwVODFx1rCTdpGNgppbjc0z0ToWxx2iDWkSHGur+5BcVaDUShyynR6Kbvd8vpccfxqDRaOiVZqj94425hVzfziBWPDhnQVCQOJDdIc5XdHyU4m+AhgqMGlOAFHj+U28tZ/Nvu5g9NZGp71dSbBO8/GNFUONdpweMeg16w4E69/H10T+nh5FEYee2kQbuW17JysOS+T97h6f/d5yRrUVOqhySRVucvLfVgd0NPZIFeo0gM0EwcaiBnSKbD568nUvvncPugwVYLRZeGxPP5P9VM3aQjquO07PgVxfr8iWXDtIyOCeLvAIL85au8Ren/fOiNK55ZxcfXteTOz5dz+Kn7iIjJZHek57C4HFyxRkZVAB2907GD3CB1DJ+sIG/rDzaIrr2UPfhNz2HTlPO4+caeWRVNQ+dGc9fV1Tz1VO3IIwm5n38DWL5H3n8rc+b8k8bFerbGfh2I25LMQefn3L0w5qmcu2pn76ifaMUfz00VGDU2AKkwPMvfOU7rhmqo2eyjrcuN4E+gXe2yDrB0nAEjkxcdHkCZdUerhyiY/neKsbnnUVi8WZOPzYFp8vDK2OPcMVCCwcrJL1SNBwo96DXwsEK+f/bO+8wqYqsD7/VabonJ2bIOSkoKIgZYccAOAKKsIKKuiroimv8dNVv1dX9Vl3FCBLUVVBAwoABBEFhQMk4IAgoCJJhcp6ejvX90YHumZ7I5K73efqZ5vbte+tWX07VPXXO77DzjBWTyTWwfDV1CsmPvceFBgs94zXc0E2HXguRRsH1XbXc/WUpM7db0RpsRIYZvfkEnvWECX11xBmsftE51pIixl1oIiZUx9wt6YzqpadLjJasYgfRJi1JHSwYQkws/b58kZkzGTnccVlbDpbYGN/XTo84HfcMCGXaTkH/v07FEBpB6mv34nS4IpSaOp6nkefuTm5wf7kqfFI9gqWflOGvhKoSjKqTgBToeF1j9Zg0Nhb+Ilh1qACnEzJLCmkVE0n7MpmtFbmRfEsmphc7cDocnJ+gZVgXO4vW7CBEjzeEFLsVnQbiQrXMvTWCCUuKoe9IuiRNYO8HT3D4s8e959t38A+evMnA7zlO7uyn56EVpQzt7CTapGHUeXq+/F1D7y7tvQJy4556x7ueMG+MifzCYib0j2fCku1c0a8nhQW5zN/lZPE+O6WFeRg0Dt7eYkY6JUIIIkMEvRMstNLb/YrMeOSYr+3RipdXZPLeDXqiQ5zc1d/A57sLOLohhR7D7iYqPJTtX3zApWMeqPPf39dt46lzC5SrddscOFcXWLAYxMZ2FTYUyvBXQFUJRtVNQAp0vOz8Ipb8OZyHV1pYfHdH4sK0vLnBPzQyK6+I6x9+mzBKAg4oX02dwsgnpvFDehYrlriMe6RRUFAqCY3UeFUwPSGkR87kMeECPUjJnzprWbj3R7okTfA75oyUVG4+T8dFbbQcyXO5HQa01TBmkZmOidFoNAZ6dYpBr9eRnV/sHcg86wltwrUIHGAzk9xdw4OvzaVnrIZjprYwcBz2VVMxDn8CQ1Qip+Y/SwejmU0PuzJnx3x8zK/IjCeUc/3BAkZ012LUuXIfEE4m9NUxbedqrIPHYDAYsBbm1OEvfxZft82Z44e92b9nPv/fGoVSlqU5GtFgMYjBgjL8FVBVglF1EpAqOt7B3FKEgH6J+FXc8g2NnJGSijDncFXvCJavDzyg+M66F42L8BNyy84vRkqJXq9j2tN3M/Z/3mbC1Z0whYdw+2A7q5YUYC3xX4RMWZdGUb6NtYetFFgkUoLVAVoNXoVOT/jpnOUbvQuwJ7MK0eBk6maXiqRWW0xkmImioiIW32ri1sW/UvjNK3SPsHNs5WsY41vTIVrHQwNjiQ/X8WZqNqN66f2KzKSmHWD7PhtWZw567Hy4QxJucK2A6DSCKJ3k5BZXeGaCyCX9+GESy2jxlMUjwTz+qTdqLMvgq/NTlWJnVclSyogqGhtl+CugKk2emmr2ePb319bxr7jlISuviJQ1m5k2wsTz61yVsqozoEB5IbfcM8d58LW53HqejghKgBBi3cVM1vy02u+csaFaZl0fQajGjsMpeXhlKUM6a9l43EHK2p94cMwQln6/lTDhYNn3W/nirccrXNO4+anpXKA7Sv82eoZ10/LtIRuf3BzBLYtKmPfMLYx7dhZzd9n9FqJ9i8wsfv1RLnloFgPvn8ruuS9SdPJXSuwhhFFKmEGDzekk75cfCDWZeOCGPrycuoLE2yr3kW9fuRBd+p56lWUID48g98ghWt/2L7/tWq2Wom/fquBbCkXDogx/BVSlyeP53NcPL6Vk8qufBYxN9y2nGEhbx5cZKakkdbBxaftQkns4KXbYKpz1VzQAxZ3Yj620iHdHxTH0/aOsKhAs3H8abZgZ4daJEZF78CyJegaQi7smcOhEOl3CnYzuraPEJnBIiAjR8Ma81ZzMyKFzlIb48FJvVI8nhv+Z95cy+5k7ycorYs+vh3jrDiM2h4Pj+ZLxffV0j4Gx52m48fH3uLCNkeTrrnKd/ORPXHn+2WLmyWdcZRnjI0LY+8ETaIECp5E4u5nZo41Eh2optsLEZblEx0bQvX0r2tn3UlyQR1hkdMA+LczL8UowP7S8bsXYfFGhk4rmgDL854hvOCdQrULrZSt3+S7gemb780Ya0GsEE/sZGLekmKG9ogPq9ev1uoBFyz01dOMNVh68JASrQ2B2wKKDDkwR4QA4SvIoKcwhO7/YO4DM3JLvVvUErYC2ERpG9TKw4YSZOSs2EqaRvHiNnn+st7Bw9WY++WYLUZpSnp62mPyMU8xZvpEffj7IhL6uRLCRC4o5kieZeZOreMzkQaHM35PPk4MjeMOn2LvvwOV0SnI3rWPHnP/1XpfnepL6nR0g/padD+16A3BXUh/+vXQaV939vwF/p+0rF3JTD+ieYCKpYwHvPTqWh99eXC/GX6Fo6ijDfw74GvJJX27DKSUfVlFoPVDlLt/BYu6KTVzXyYEQOvZl2pAS+sRLPtqWz4W55eURAg00nozg7ye1JT8/hwcHGbk9xczMsW3ZU1DC4tcnExcV5uev9zyRjHxiGrt++4OMEtfi7uliJz+dKaXULjEIyX2XGUgMF9zQRfDZ7lxCdNA5UceGXw7y/k0RTF2/nUNnCthmtTF9ayl6LdzT3+BamJWSBJPkjgv0TP8hm+TzYgPWMfBtl69E9bHTZl5bm+WVoIazrrVeHVoRq/VXr/RE5TgddkILjzHp5lBO/pFHaZ6ZaLONH5Z+zPB7Hq/w922Oi7AKRXVQhv8c8DXk17TLZU+6g14JcQFDO8tGAd3Yy8ifZq2mQ6sI5ow5O1i4Zt5G1p1yfa+guBSz2U5cXKyf+6myHAJPRrC5KB+Hw0GrUAPJPXQs31dEcnedX+JV2e9X5OLqe9sLUJrPhAt0gOCKDlpmp1l5brCRf6yzMKyHnvYR0lWU5pokisxWFi7/joJSyce7bMzZ7RrEpHQ9SWgEfDQ+lAlL/Nte0XV9NXVKlW6ytpF6ju3bQcfzBwJno3KyflzASMd3tO4YS26xnbST+3l7RBiTVn/O1bfcU+GsXy3CKloqyvDXEl9DbrM7GdHFwfpDFrKLHQFDO8suwq7YX0SHCAeWkiJ6JbTyDhZljfu4p95hRnIiDy4v8UbqTH71M/r3bB8whyArr4gde//gsFEyb3cuWg04ZQkajcCJlXbxEX6JV9XJQcjKK8LhsHFXPwNtwrU4pSTfIhjfV89v2U5G99KxP9uBkE4m9HcZ88xCG0lddNw30MSHP1lY8KsGS6mZ8X10vDXMRIFN4w379F0r6N+jPUM6wLMrMhjaMbyc4mhlyXJP3jKQie8v9Rp+DyV/pLGooJRFv5zCai7mth4aYo2C4Z2s9bLQq54UFE2dejP8Qoj/AslAhpSyr3vbi8D9QKZ7t2ellN/UVxvqE19Dfia7gM4xWkb10jNnex6PD4mrtJqW3eGkoKCQt28w8OTqs4PFmAXbWJt2gDnP3+OnWV/WNZR+8igpR47z3X2usou+A83cFZt47Jo4Jl4cxrUzjzJtZAzP/yi8EgqAXwho2e971hl81x3mrthEiLSyYI+dxXttICV57nDPLtEaPhpp5NpPS8g1O0mwmRnSAZbutXFrXxOmhE7cPthOyu/HaRctWHnYyeaPitFpNH5JawA5p48x/9AxJvQ1kFtsBZ9F7eokywkhGHF+FGlbVtPzsrOfdbzzdcBVdStvwWPc3LeQxC49mZxgq/OqW9D8nxSCRYI5mKnPGf8nwDRgbpntb0kp36jH8zYIvoY8I7cQh8PlE3diZf5eV6KPb2in70z+zXmryfvtRwZ1gAcu0XgHi2vaWVm85yjTF69j+/6j5OXmsGyCS3bhqjalPP7dFoQQDGijY82BQhzOs6X48nLzeOOzb9n6834WjYtg7o58RvfS8fJ3OZwfr+f9lHX84y/JwFllz7y8XKJNcX4hoBNvvILr//Y2UeJs4lhq2gFKnDqwO4k0CnJLHGiEoFO04L+jTMSHCsaer2f05yVEhNrQ6XSM6SVoFW6gNPM4FrOTaE0xrySF8K8frLwxLJLLeyd6k9Ym3ngF4556h5eTQnnoixxW/2blw5FhPLjCtaj9+mffsuy7zax/sD1QebLcn/p1JDXVv7iKh9ytKSS3ySHG5BJ3a4iqW80RJcHc8qk3wy+l3CCE6Fxfx29salqC0YPHZfHKVZBjdnJpe8Gdy3L5ZJeV3IIierUysODbTURpbaQX2REimuz8IqJ0di5PsPJ7nob1h0oJ0cLFbx+jTVwEmfkllJZambVsA3+7Op57Pj9FXomd67tqsNglGqeDlNWb+euYocRFhXmVPd9dX4rJZCYyzBVx0zb9AEVmKwW52bw7No7n3bNtT5awp4hMbk4uBq2GvZlORsw3o9VqcUrQajXsnPcS97z0MetOZbHulAQEOYVWkrvoaBupZVB7Hbd/UUpsRIb3nOByOyWaLAzprGFPhgOHdNIvET7eUYBTbmNMD8BmBvSVJsvFRoZhPraFksKCcn1fuDeVlGIzC3ebiWl1dnCoz6pbCkVTpDF8/FOEEBOBHcATUsrcQDsJISYBkwBmPfVnJo26sgGbWHOqK8/scVlc2rO1d9uUrHx+yIrk6vgCr4vm2SsNPLrKykVvHUM67UQbBVnFDs5LDCE2VMO7w0MZ/Xkx0/9+N3e/OJuOYXYOZNuZtTmH1mGSMINk1e+Cfw0N4falZob1gDc+W82B4+m88cg47v/nbJ4bZuDFTTri4uL45Pm7kVJy9f3/5r4BBloZLAzvavIaV9+BbtCD0zl+KoNxPUq572Kjd/t/d1orXKd4/kZXZvFzbez8vKjQ63rydTtFGSMYf4GNXd9ZSExI5Llhkq2f5rH/RDarDmlZtC+dVjFmNBpXBm+gZDm9TsuEK7uw8udNfr52p8NOqCWbd2+K4NFvLUx+50sVyqkIWhra8M8AXgak++9U4C+BdpRSzgZmA7DpPRlon8bG19hXV545UMKV0ynJzMvmg8c6MndHPrf10dElBiYNNDIzzcm9V8QTpnOSV1CIVqcnQuekb4KOCX113P9/nxDiLGX6iBDuXOYkvdjBK9dHcs/SQsb10bLydzsdIgVf/2ol/PhWusRoeXraYq5pZ6VThIMoUUrWqSLmLN9IkdlKiLRw30Xh2KVkRDfJw2vKu1S2zXBJOG9Mz2LjSt+rC6FtWuCKXNWRvgi0VhKlKaJbrIYJA2JAo6uWeukNA7uz8J3lfv7otQtm0PP0Mq68OJ67irOUe0cR1DSo4ZdSpnveCyE+AGpfC68J4DH20xevI3Xbz954ft8F2rJUVH6Rkz8BsHxvAe8NMxBtlEzsp+ezn4soMVvYnm7lSI6d/FIL88eEsS/TxvXddMzZlcvEfgbOi9cyvq+eGdutRJgMJIZruLarjtc3WpiRbGLMohIctlLevqk9t8w5xAPDjYSFaMkrsfH8EBP/990WMgrM3H+hnvgwDTanJD+n1G/WX9V1ePAdEGsifVF2reSTXVbyCyzMSjbxn80FzBzblgeqIX8thCAuIhS73YZOp/dm7b7wZ1fy1/iLo+plUbepoBZnFVXRoIZfCNFGSnna/c+bgV8a8vx1iW944ehPN/GXAeHeeP7Fe45WOfP3xWP8pm3KY1R3cDqdRIVo0Aq440I9c3YX8uPDXZmzPY+8gkK6tY6kdVwkpzLzuaW3FQ0SrQbuuFDH57/YeGpFFkjBmt9tjOiho39r16Dw9W92QmUxf+6jZcsJO3szHNzcW8eFCRouTzCzOLOUhXs1LNxrA6CgVILORu/MA+VyEipza/k+/VRX+iIQLr0fC8N6mtiXVerNQxh0/1SkIbzc/vERIWyb4apU9sAN5/OveW9w5V3PeLN248L0QMtf1D3XxVkVjtryqc9wzgXAECBeCHECeAEYIoToj8vVcwSYXF/nr288Loq4UC0h0sKIbmHeeP61v0uWfb+1ypmpB9+s2RWHT7DigIVIo8BmdwISpEvF02yxo8HJJ7szSYgp5XRmPiFaJ52jNezNdICEoZ11zP3ZSqdowaJ9NhbeGsr+TAfXd9WxYI+V+5fl8dilBh5eWYpWA5/eHEp2iYOr2kk+3OEg1BRKqNFlICNDoG1CfDmffdmoH19+O5rOm/PXsGRiIs9+X/XsvCKy8orYd+AIT47Usy/T5l0Ej4wIp6CglMuemVXuO3s/eML7vnv7VkQ69pKbeYaDOzeyM6OUhbv9o33Uom5g1FNBy6c+o3rGB9j8UX2dryHxTd6auyOf2y/Qo3OUkpmnoXOMllt66/n+qJnrprzFmmmPVdvwlY2e0Ye4tuuBHgnlVTw98gq/5Ti5bq5LBtgpJaEGwfPXhPDcWgtGnSAuTEdcqJMLE7Wkl8DOdMnFbfVckKjhys5GzhQ5+HyfhZ5xGqwhIV4t/0C8vySVgtxs3h4Ty/1L1nDT4P5+heD/Pn0JHcIdbPrD7M0Sru6Tjy9zV2xi8hWxfuJtU7Jc4Z/TVlb+oOh5Inn3idt45utPmPyfz2p8foWiJaMyd2uB74Jk6u/FnCqw8cFPFoqshbQO16LRgMUOBk2pX/x8dahJmGig6JlLE6xYLWZ6xOu5tquD8UvN2J1gtTsxaAXvDjfyyKpSCiySzccFH+204XBIDFrJ+zeaeOibHA4ez/Az5h5D+spfb2HBt5u4b4CBWH0pyd3wFoLPyivi9hc+4sChI3w9IZy/ray+Tz4Qla0NVIXH1bRw9TbMhaE1Oq9CEQwow18L/I2SEUKMYC8lIcRO2hNdXcVQ5hznpaEGpvjEz9cnnhq8qQdKeG94CCAY19fA1wedxEaHU1BiYVR3J33amHhkSLRfdIxHOz+5t4ltp5xeY+577Nwzx3n0rQWESAt39Q8jp9jOw5cauPbTQxw8nsHXG3aRffoYveMEfRN05bSBqjvr9wwyH1ewOA7Q9Y43K/2+r7TDrTdey6HdW+l24aXV70yFooXTYg1/dePqa0OgWbnHRTNwegY5hWZGdYdYk+C6TvZauzuqi8fYDW1nI7mHjp7xOg7nOLi0V1umXF1IUewFpG77meeGuWLpJ8bYvZmvB49lkPbLQd67z7Vw9+AgI1f/95B31u859ruj4rh+1h/87bIQhJTEmQTRRsH4vjoefXMhRQX5vHC1nn//aGHTMYufT75jBcVpfNtf07DYiigr7ZCVmckfR5bSpc/AZlGQvS5Qi7OKqhBSNskQeX9qEcd/Vsnxmno1umWpqBSir1ZOXeMJB009kM+pApcKZp7ZgUYfQlS4CbvTpev/+OCz/nKPXMKnq7YwNKGAJ680UWJz0jlGz0upJeyxd2LZfx7yHvu28yDpg3QsTii1SRLDXbLITgmni2HSJWFMGahh/m4bGEJ5fEhcuTrClbV/+Zr1DLnqcldYbHIoDy4vqbDPBj04naxCS7ntUSYtMaKoXN8PHnw1UcP/3iJDNxWKimgdZeSmfm1FoM80Dd2YhsD3cX/5+u1k5xc32LmrKoXoad+Yv8+ss3alph1g/h4LpyxGCImg0KHH6oCwsDB2fPoiHVvHMX+PhYHTM7yv+XssrNm2n5Nncvj2kJ2LZxZwzcdFdJiax0c7bez87bi3HydeHEZuYQkfjjSSZ3by5z565txsYt6tYcweHUHbCA3DukivBMW0jbn0f/cMc3eZeWfROu91Brpu399qwbebGN5VuGfr/n3myzev3MVF7UPZPn0yhz973Pu6c8h5Afv+98NHOfVbWp30dVUU5uXwwXP3UpQfMCFdoWgStEhXT3WUHM+FytxI1anFe67ujLL4loG85+U56DKziIqCEiHIzi+ucMH4zXmrua5dSYVKnm/OW01ydw1RRg3ZGugUq0ergdWH7SzaZycxLoq84lJGddeQGKn3Lgh7om8Av4Iqga47UFgsVC7EVlH/Vdj3CfmEpH0Dg5LOua+roiHq+ioU50qLM/xlC55UZkBqS2WGO1DhdM8g4du+ynTlz6VdWaeOUmxxcqLERr92/lr3vgNVoJBUX32eiTdewTuL1hETpmfmlnywW7E6nEzoq+efQ428/ION6PMuYfveI/yQnsUPXwGcNbitTu7HYi7yXmfy1f0Dlp0MFBZrdzgrFGLztPvfw2K49VP/cNLKIqJW/3SYr76ZS78RE+ukrwPRUHV9FYpzpcW5eqrjajkXaupGKluT1/9ppG7b9eXabTwySEt2sY1usRoOZZpZ9v1WZixN9WuDbzuyix3M2pTDnf0MRBkFI7q5jPGMlFS6xWgYkzSI9vERfH1vB6KNGiYNMGDUCf7S30DK6s18/Pw97Pj0xXKv6wad53edT09bXO66y4bFLtxr46b5xZw/9YTXHZWa5h++6fnOpiNmOoQ7eOq9xdXqn+sHdMWSfaxO+roifOv6ejKDFYqmSIsz/B5/d1l/dlkDUltqYrjLDhIHjmV4febgehqpqzWIuSs2cU07K2mn7MSHCqaPMGLUgdFZQsrqzeUGKk8/DZl5nBHdtWQW2ckxOzFgY3hXmL9qEzNuiSdlzWaGdoRFu/JJ6qKhZ7wWrYCuMYKkDjbeT1lX4XV7rnN8v1D2/HqI5N5Gv+tevXW/97fyrE9ERkXSu0t77wASqNzkjb2MLN9bwAejwtjzqysCqToYLHnkZpw8164OiGe2P/7is3pAv21Ypnz9iiZJi3P11FYnvzrU1I1Udq3hsbcWMiCmiGiTyzhUpitfm3Y9PcDJw8ut3Huxga4xGsaer+O9bSXc2tdYbr3jq6lTyMor4qLbX2TtUcmSX820iokEJOk5WcSGaumVEMJ1nRx8vKOAwmILkSGS7w7b3WcVmO2C0NNp5RLUyj51CbuZCX1dcf2PJ4R4rztQofXK8BxTQ3KJAAALC0lEQVR3xf4iknvovAqlZfMOKuIfYwfw91VLufLPdV9QJNj0gBTNmxZn+OuTqiSGfQk0SLz6+mF2ayRfHjzhLX4CgXXlq4Nn/aB/D1f93a1n9ITqNdx7kQG9RnDPxSEs+MVGhCnE2wbfgcpTpvHxwVHe0MvMvCJWrd3A6AtdImhPX9eWn3ILCTGFk5njP3vVA20TYsu1q+wiq0d100metzpZba47Ne0Ax06bKSgs4tObjezLtHFDdx0LvzpCdn5xlWslreMiaWP7iez0U8Qltq32eauD0gNSNCeU4a8B1YnY8VB2kABoG6EJWAM3ENVJQPOsH6QcT0enkRw5k8eEvjpOFjo56VblTeqqY+WvxbxyE+VKLJYdmEbP30pGTj6zhxt5fn0hU65ylWUc0gG+OFDImmn/UyPRubrmq6lTvHkFvho+k7Pyq/3UdFGXBH4pzK9zw6/0gBTNCWX4a0BNDFrZQcKVzash0WQnuXsI0xev4+ffT1RL2jiQQfOPDnIlO93z0sdsyshi02rXPmdn2hoGTj87WPmWO/R9eonXmbmoC7QK19A/UTDwnePERYWSU2gmVl//GcjVoSaDbyAG9+3Asnlz6ND91fpqokLR5GmxmbtNiUDZvFfPOkPrULjp+iEB3UTjnnqn0gxWz8zX101TE6PsqwIKYHc4ycsv4LObTVzV2cjeDBu3fyVZ9J9HeOBfH1SZTduceGn+RvRXTSa+bafGbopCUW8EXeZuUyOQ2ydEWng5KTRgVE9VkUNlo2ZqEx301dQpfuGXY5IGMeZ8A5d2dK0H9IzXkdTBxuNvL6yX8NPG5Jlxl/Hr1zMbuxkKRaOhDH8DUDbEdNC0kwzt7HH7lJdzqMqo10euQsq6NL7+zc7lHxYycHYBl39YyBf7bew78Ee9hJ82JnqdFr0WmsXTrkJRDygffwMQSDc/kFKmJ9Kmqsihc/VzB6JjYiynhNNvm7SaGdNLVCuKqblxx+XtWbRqHhcNv6Oxm6JQNDjKx9/A+Prmvdt8fPRlfe8eypZAbAiaUltqQnUluce+voprHniNEJMq1qJoeVTm41eGv4Fprsa0OVFdSe6Nuw+zIKsb/QaPaMDWKRQNQ2WGX7l6GphgMO71WQSnOueurgjeped3YuY7K7BdmoQ+JKRB26lQNCZqcTdIqeuaAL6UFaZryDbVREtJp9PSp2MsNmv5oi4KRUtGGf4WTkXGtLbGuTrnq20RnHNtU23CXJP6dWT3Nx/X6nwKRXNFGf4WTiBjWp8VymorO10XbapNmOslPdsSWnSC0pLmHaKqUNQEZfhbMBUZ0/qsCVDbxLK6aFNtJbmfHzeAHYvfrfH5FIrmilrcbcEEKkEZSJytriqU1US91Je6qppW24XzVjERCKfy8yuCBzXjb6FUNPuekZJabxXKajvjru+qadVhQILgj583N9j5FIrGpN5m/EKI/wLJQIaUsq97WyywEOgMHAHGSSlViaJ6oCJjOnftT+g0sk6zfj3UdsZdH5nINWViUh/u/3g5nS+8DCEChj4rFC2GekvgEkIMBoqAuT6G/z9AjpTyVSHE34EYKeXTVR6sBSVwNRQqUazmfLgyjZM9bqNt5x6N3RSF4pxplAQuKeUGIUTnMptHAUPc7+cAqUDVhl9RjqqSpJRxrzmx4UYO5GaCMvyKFk5D+/gTpZSnAdx/EyraUQgxSQixQwixY/aXzV8KuK6przj8YGbUFb05tWF+YzdDoah3muzirpRytpRyoJRy4KRRVzZ2c5oU9RmHH8xotRqSL+nMvtQvGrspCkW90tCGP10I0QbA/Tejiv0VAaivOHwF3Db4PCzHdzV2MxSKeqWhDf9XwF3u93cBXzbw+Zs9dVF9S1E5dnMhJYUFjd0MhaLeqDfDL4RYAGwGegkhTggh7gVeBa4TQhwErnP/W1EDmkLMe0vntTsuY8cXsxq7GQpFvVGfUT3jK/goqb7OGQw0hZj3lk5MZCix1tNknT5OfJsOjd0chaLOUZINzQwVptkw3HZlVxafOKwMv6JF0mSjehSKxqRDQjSnd65VBdkVLZLmMeMPqzDcX6GoF9p3SWBI7zzCdXZMoQ1bRUyhqAuiQ/UVftY8au4GCUKISVLK2Y3djqaM6qPKUf1TOap/XChXT9NiUmM3oBmg+qhyVP9UjuoflOFXKBSKoEMZfoVCoQgylOFvWgS977EaqD6qHNU/laP6B7W4q1AoFEGHmvErFApFkKEMv0KhUAQZyvA3EkKI/wohMoQQv/hsixVCrBFCHHT/jWnMNjYmFfTPi0KIk0KIXe7XiMZsY2MihOgghFgnhNgvhNgrhHjEvV3dQ24q6aOgv4+Uj7+RqNOaxC2QCvrnRaBISvlGY7atKeCuZ9FGSpkmhIgAfgJGA3ej7iGg0j4aR5DfR2rG30hIKTcAOWU2j8JVixj339EN2qgmRAX9o3AjpTwtpUxzvy8E9gPtUPeQl0r6KOhRhr9pUe2axEHMFCHEbrcrKGjdGL4IIToDFwFbUfdQQMr0EQT5faQMv6I5MQPoBvQHTgNTG7c5jY8QIhxIAR6VUqqyYQEI0EdBfx8pw9+0UDWJK0FKmS6ldEgpncAHwKDGblNjIoTQ4zJo86SUS92b1T3kQ6A+UveRMvxNDVWTuBI8Bs3NzcAvFe3b0hFCCOAjYL+U8k2fj9Q95KaiPlL3kYrqaTTcNYmHAPFAOvAC8AWwCOgIHAPGSimDcoGzgv4ZguvxXAJHgMkef3awIYS4CvgB2AM43ZufxeXDVvcQlfbReIL8PlKGX6FQKIIM5epRKBSKIEMZfoVCoQgylOFXKBSKIEMZfoVCoQgylOFXKBSKIEMZfoWikRBCHBFCxLvfb2rs9iiCB2X4FYo6RAihq833pJRX1HVbFIqKUIZf0ewRQnQWQvwqhPhQCPGLEGKeEOJaIcRGty79IPd+YW5Rru1CiJ1CiFE+3/9BCJHmfl3h3j5ECJEqhFjiPv48dzZo2fOnCiH+LYRYDzwihLhJCLHVfY7vhBCJ7v3ihBCr3dtnAcLnGEU+51zus32aEOJu9/tXhRD73OJiQSsprDh3ajU7USiaIN2BscAkYDswAbgKGIkrW3M08BywVkr5FyFENLBNCPEdLj2b66SUpUKIHsACYKD7uBcBfYBTwEbgSuDHAOePllJeA+BWe7xMSimFEPcBTwFP4Mo+/lFK+ZIQ4kZ3W6uFECIWl7xAb/dxo6v7XYWiLMrwK1oKf0gp9wAIIfYC37sN5B6gs3uf64GRQogn3f824pI2OAVME0L0BxxAT5/jbpNSnnAfd5f7WIEM/0Kf9+2BhW5NGAPwh3v7YOAWACnlCiFEbg2urwAoBT4UQqwAllexv0JRIcrVo2gpWHzeO33+7eTsBEcAY6SU/d2vjlLK/cBjuPSA+uGa6RsqOK6DiidLxT7v3wOmSSkvACbjGmA8VKWRYsf//6URQEppx6UimYLr6WVVFcdRKCpEGX5FMPEt8LDHTy+EuMi9PQo47ZbpvRPQnuN5ooCT7vd3+WzfANzuPvdwIFABkKPA+UKIECFEFJDk3j8ciJJSfgM8iktkTKGoFcrVowgmXgbeBna7jf8RIBl4H0gRQowF1uE/e68NLwKLhRAngS1AF/f2fwILhBBpwHpc6pl+SCmPCyEWAbuBg8BO90cRwJdCCCOuJ5fHzrGNiiBGqXMqFApFkKFcPQqFQhFkKMOvUCgUQYYy/AqFQhFkKMOvUCgUQYYy/AqFQhFkKMOvUCgUQYYy/AqFQhFk/D+d+v8G2MMhIwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn import svm\n",
    "from sklearn.model_selection import train_test_split\n",
    "try:\n",
    "    from mlxtend.plotting import plot_decision_regions\n",
    "except:\n",
    "    !pip install mlxtend\n",
    "    from mlxtend.plotting import plot_decision_regions\n",
    "\n",
    "# setting the features and the targets\n",
    "X = df[[\"mean radius\", \"mean texture\"]]\n",
    "y = df[\"target\"]\n",
    "\n",
    "# splitting training and testing set 80/20\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=1)\n",
    "\n",
    "# Initialize SVM classifier\n",
    "classifier = svm.SVC(kernel='linear')\n",
    "\n",
    "# Fit data\n",
    "classifier = classifier.fit(X_train, y_train)\n",
    "\n",
    "# Printing parameters\n",
    "coef = classifier.coef_\n",
    "print(coef)\n",
    "print(classifier.intercept_)\n",
    "\n",
    "# Plot decision boundary\n",
    "plot_decision_regions(X_train.values, y_train.values.astype(np.integer), clf=classifier, legend=2)\n",
    "plt.xlabel(\"mean radius\")\n",
    "plt.ylabel(\"mean texture\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is a lot going on here, so let's try to break it down. Recall that we are trying to determine whether a patient survives using just the (normalized) age and detected nodes. These are called the explanatory variables or *features*, and we refer to it as $X$. The variable we are trying to predict is the prediction variable or *target*, and we refer to it as $y$. \n",
    "\n",
    "The underlying notion behind machine learning is that patterns that hold in a subset of data will hold in the population as a whole. So in a sense, we care about the performance of the classifier on data that the algorithm has *not* seen by learning patterns in that data that we have seen. To this end, one of the major paradigms in machine learning is to separate our data into a *training set* and a *testing set*. The training set is a subset of the given data that we will actually provide to the algorithm to learn, while the testing set will be used solely to report the performance on data that the algorithm has not seen. In our case, we set 80% of the data as training data and 20% as testing data.\n",
    "\n",
    "There are numerous algorithms that can train on data. One of the more popular ones (especially before neural networks were introduced) are called *support vector machines* (SVM). We will not discuss how it actually does the learning, but suffice to say it takes in the training data and finds the line that minimizes the classification error. This line is called the *decision boundary* and it is defined by the following equation:\n",
    "\n",
    "\\begin{equation}\n",
    "C_1(mean\\ radius) + C_2(mean\\ texture) + C_3 = 0\n",
    "\\end{equation}\n",
    "\n",
    "Where $C_1$ and $C_2$ are the coefficients of the linear classifier. So really, the function of the SVM is to find these two coefficients. The algorithm then does classification as follows:\n",
    "\n",
    "\\begin{equation}\n",
    "y_{pred} =\n",
    "\\begin{cases}\n",
    "    1               & \\text{if } C_1(mean\\ radius) + C_2(mean\\ texture) + C_3\\geq 0\\\\\n",
    "    0,              & \\text{otherwise}\n",
    "\\end{cases}\n",
    "\\end{equation}\n",
    "\n",
    "### Exercise\n",
    "- Look at the first 5 samples in the testing data set. Calculate the predicted target of each of the 5 samples?\n",
    "- Look at the corresponding targets. Are the predictions accurate? What proportion of the predicted values are correctly predicted?\n",
    "- Which of the two coefficients is most important? Why?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How good is our classifier? We can look at the performance of our classifier on the testing set to find out. Of course, an important metric to look out for is the accuracy:\n",
    "\n",
    "\\begin{equation}\n",
    "\\text{Accuracy} = \\frac{\\#\\ (y_{true} == y_{pred})}{\\# \\ \\text{samples}}\n",
    "\\end{equation}\n",
    "\n",
    "However, for most applications it is also worth understanding what your *false positive* and *false negative* rate is. A false positive is a sample that is classified as a 1 but is in reality a 0, and a false negative is the opposite\\*. The false positive rate is defined as $\\frac{\\# \\text{False positives}}{\\# \\ \\text{False positives + True negatives}}$, while the false negative rate is defined as $\\frac{\\# \\text{False negatives}}{\\# \\ \\text{False negatives + True positives}}$. **Why would this be a metric we care about, especially in this case?**\n",
    "\n",
    "<details>\n",
    "    <summary>ANSWER</summary>\n",
    "    Imagine your tumor is predicted as malignant but is in fact benign. There is certainly now a cost you incur in further tests or surgeries, but ultimately it is a manageable cost. Now imagine if you predicted a tumor as benign even thoguh it is malignant. Here, the mistake of the classifier could certainly cost you your life.\n",
    "</details>\n",
    "\n",
    "It turns out there is a useful tool called a *confusion matrix* that handily lays it out for you.\n",
    "\n",
    "\\* It's worth putting some effort into what we call positives and negatives. Here, we use the term positive to mean 1 (benign) and negative to mean 0 (malign). However, anyone that has had a test of any sort knows that positive usually means that you have the disease (in our case, malign tumor). A lot of data science is communicating results, so something worth thinking about."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.8508771929824561\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATUAAAEWCAYAAAAHJwCcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3dd5xcZb3H8c93d9MgDbJphBJAWqgGCEWBAFcp4qVLiXiliKLAVbGAFwUBBUGuoIBcRC4iTarApUsLVSAhlIQOEgKkEgIJoST7u3+cs8vsZjNzTjKzOzv7ffOaF3PmPPOcZ+Zkfvs8pzw/RQRmZrWirrMbYGZWTg5qZlZTHNTMrKY4qJlZTXFQM7Oa4qBmZjXFQQ2Q1EfSLZLmSbp2OeoZJ+mucrats0jaTtKLy/je9SQ9JekDSceWu20dRdJYSdMqWP+Fkn5esHyUpBmS5ksalP5/rUptv1apK12nJulg4IfA+sAHwCTgVxHx0HLWewhwDLBtRCxa7oZWOUkBrBMRr1So/j8D70fEDypRf0eRNBa4PCJW7YBt9QDeB7aOiKcrvb1a1mV6apJ+CJwD/BoYCqwOXADsWYbq1wBe6g4BLQtJDctZxRrA5E7adqn66ytZ/3IYCvRmGb+3QpX+DqteRFT9AxgAzAf2L1KmF0nQezt9nAP0SteNBaYBxwEzgXeAQ9N1vwQ+AT5Nt3E4cDLJX+jmukcCATSky98EXiPpLb4OjCt4/aGC920LPAHMS/+/bcG6+4FTgYfTeu4CGpfy2Zrb/5OC9u8F7A68BLwL/Kyg/BjgUeC9tOx5QM903fj0syxIP+8BBfX/FJgO/LX5tfQ9a6fbGJ0urwLMBsa209Z7gcXAR2n966b77zJgFvAGcCJQV/CdPQz8Lt3GaTn3bavvPH0tgM+lzy8F/gjcln7mf2un/pWB/03rngv8vfB7Lyh3PPBqur+mAHsXrPsc8EC6r2cDf0tfV/rZZqbrngE2Kmjbael3tCBt93zg3nY+Ry/gt8BUYAZwIdCnzb+Plv3X2b/ZTo0Xnd2ATI2EXYFFpEFlKWVOAR4DhgCDgUeAUwt2+qK0TA+SYPAhsFK6/mRaB7G2yyPTf2ANwIokw4T10nXDgQ3T5y0/sPSHMhc4JH3fQenyoHT9/ekPZF2gT7p8xlI+W3P7f5G2/1skAeJKoB+wIUkQWSstvzmwdbrdkcDzwPcL6mv5sbSp/zfpj6cPS/6gv5XWswJwJ/DbIvvifuCIguXLgJvSto4kCcSHF3xni0iG/w3NP9Qc+7blO2/v85EEjnnAF0hGJr3bqf9W4G/ASun3u0PB91L4HexPEtDrSP4YLACGp+uuAv6reRvAF9PXdwEmAANJAtwGBe+5lDSI0+YPZzuf4xzgZpJ/V/2AW4DTl7b/Ovs326nxorMbkKmRMA6YXqLMq8DuBcu7AP8q2OkL2/yDmUly/ALyB7X3gH3b/uOhdVA7BHi8zfpHgW+mz+8HTixY913gjqV8tub216fL/dL2bFVQZgKw11Le/33gxoLl9oLaJxT84Nv+oNPXbgaeJelt9CqyL+4nDWpAPfAxMKpg/beB+wu+s6nLsW9bvvP2Ph9J4LisSN3DgSbSP3DtfO/Tirx3ErBn+vwy4CJg1TZldiIJ4luT9k4L1l1KhqBGEgwXAGsXrNsGeH1p+687P7rKMbU5QGOJYwWrkAxtmr2RvtZSR7Q+ZvYh0DdvQyJiAclf6e8A70i6VdL6GdrT3KYRBcvTc7RnTkQsTp8vTP8/o2D9wub3S1pX0v9Jmi7pfZLjkI1F6gaYFREflSjzJ2Aj4A8R8XGJss0agZ4suW8Kv4c3S9RRat+WUqz+1YB3I2JuqUokfUPSJEnvSXqP5Lto/l5/QhJ8Hpc0WdJhABFxL8nw/3xghqSLJPXP0XZIeqcrABMKtn1H+nqzLPuvW+gqQe1RkuHVXkXKvE1ygLrZ6ulry2IByT+iZsMKV0bEnRHxJZK/8i+Q/NhLtae5TW8tY5vy+CNJu9aJiP7Az0h+cMUUPQ0uqS/JEOjPwMmSVs7Yltkkxyvb7pvC76Hotim+b1vtK0mt9lWG+t8EVpY0sFgDJK1Bsp+PJjmEMBB4jvR7jYjpEfGtiFiFpCd6gaTPpet+HxGbkxwmWBf4cbFttWM2yR+tDSNiYPoYEBGFfwRLfYfdRpcIahExj+R40vmS9pK0gqQeknaTdGZa7CrgREmDJTWm5S9fxk1OAraXtLqkAcAJzSskDZX075JWJBlWzSc5MN7WbcC6kg6W1CDpAGAU8H/L2KY8+pEc95uf9iKParN+BpD3+qdzgQkRcQTJMagLs7wp7V1eA/xKUr80OPyQfPum2L59GthQ0maSepMcOsgsIt4BbicJQiul/662b6foiiSBYxaApENJemqky/tLar70Y25adrGkLSVtlV6ysYDkj3N7/16KtbGJJKD+TtKQdHsjJO2Sp57uoksENYCI+G+SH8OJJP+w3iT5q/n3tMhpwJMkx3ueBSamry3Ltu4mOXD8DMmxqsJAVEdyFvVtkrN1O5AcD2tbxxxgj7TsHJLhyR4RMXtZ2pTTj4CDSc7S/YnksxQ6GfhLOpT5WqnKJO1JcrLmO+lLPwRGSxqXsT3HkPygXwMeIjnBcUnG90KRfRsRL5GcSPgH8HJaf16HkPQmXyA51vr9tgUiYgpwNsmoYQawMclZ22ZbAv+UNJ/k2ON/RsTrQH+SfTCXZNg8h+QsZl4/BV4BHksPKfwDWG8Z6ql5XeriWzOzUrpMT83MLAsHNTOrKQ5qZlZTHNTMrKZU1Y2v9X0GREP/IZ3dDMthzSG5r1+2TjT9ram89+6cUtcsFlXff42IRQtLFwRi4aw7I2LX5dleXlUV1Br6D2HEuHM7uxmWwyVHbdPZTbAcDttnp+WuIxYtpNd6Ja8EAuCjSeeXupOl7KoqqJlZVyBQ9R65clAzs3wE1FXrtHQOama2LLRch+Uqqnr7kGZWpdLhZ5ZHqZqkXSW9KOkVSce3s34lSTdKekbS45I2aq+eQg5qZpaflO1RtArVk0zJtBvJZA8HSRrVptjPgEkRsQnwDZKJFYpyUDOzfES5empjgFci4rWI+AS4miVzjowC7gGIiBeAkZKGFqvUQc3McsrYS0t6ao2Snix4HFlQ0QhaT+A5jdaTh0IytdQ+AJLGkMyrVzS7l08UmFl+2c9+zo6ILZayrr3xadtpg84AzpU0iWTaqadI8jEslYOameVUtuvUppFMp95sVdrMVh0R7wOHAkgSSfa214tV6uGnmeUjynKigCRt5DqS1pTUEziQZILNzzYlDUzXARwBjE8D3VK5p2Zm+ZWhpxYRiyQdTZJysR64JCImS/pOuv5CkpSCl0laTJJr9fBS9TqomVlO5btNKiJuI8nnUfjahQXPHwXWyVOng5qZ5SOg3rdJmVktqeLbpBzUzCwnz9JhZrXGPTUzqynuqZlZzch2DVqncVAzs/w8SaSZ1Q6fKDCzWuPhp5nVjOb51KqUg5qZ5eThp5nVGp8oMLOa4mNqZlYz5OGnmdUa99TMrJaoioNa9fYhzawqJbN5K9OjZF2lkxkPkHSLpKclTZZ0aKk63VMzs3wkVLf8PbWCZMZfIknC8oSkmyNiSkGx7wFTIuKrkgYDL0q6Is0T2i731MwstzL11LIkMw6gX5pJqi/wLk6RZ2blluOYWqOkJwuWL4qIi9Ln7SUz3qrN+88jyTD1NtAPOCAimopt0EHNzHLLEdSWN5nxLsAkYCdgbeBuSQ8WS5Pn4aeZ5aMcj+JKJjMmSWR8QyReIUlkvH6xSh3UzCwXke14WobeXMlkxsBUYGcASUOB9YDXilXq4aeZ5VZX12HJjE8FLpX0LEnf76cRMbtYvQ5qZpZbuS6+zZDM+G3gy3nqdFAzs3yyHS/rNA5qZpZbNd8m5aBmZrk0nyioVg5qZpZbOW6TqhQHNTPLRx5+mlmNcVAzs5rioGZmNcMnCsys9lRvTHNQM7OcVJ7bpCrFQc3McvPw08xqS/XGNAe15fHFdRs5Yc9R1Etc9/ibXHx/6xlRDtthTfb4/AgA6uvEWkP68sVf/oN5Cz/ltP03ZocNhvDu/E/Y878f7Izmd0uPT3qZ8//3Vpqagt133pyD9tq+1fp/PPg0V9+U7I8+vXvy/SO+ytojhwNw/W2Pcts9TxIRfGXnLdj3K9t2ePurRbftqUnaFTiXZFqRiyPijEpuryPVCU7ce0OO+NPjzJj3EX875gvcN2Umr86c31Lmkgde55IHXgdg7AZD+MZ2I5m38FMAbnxyGlc88gZnHLBpp7S/O1rc1MTv/3wLZ574TQYP6s93T7iQbbZYn5GrDmkpM3zISvzu5MPp17cP/3zqJf77ops5/9ff5vWpM7jtnic5/9ffpkdDPcf/+jK2Gr0eqw4f1ImfqHNkzRTVWSp2tK8gU8xuwCjgIEmjKrW9jrbxagOZOvtDpr27kE8XB7c//Q47bTh0qeV332wVbpv0TsvyhNfnMu/DTzuiqZZ64ZVpjBg2iFWGrkyPhgZ23HZjHnni+VZlNlxvdfr17QPAqHVWY9aceQBMfWsWG6yzGr179aS+vp5NNhjJQ49PWWIb3UW5UuRVQiVPYWTJFNNlDR3Qm+nzPmpZnj5vIUP692q3bO8edWy3XiN3Pzu9o5pn7Zj97vsMHjSgZXnwoAHMfveDpZa//d4JjPn8ugCMXG0Izzz/L+Z98CEfffwJ/3zq5ZaA1x2pTpkenaGSw88smWKQdCRwJEB9v8EVbE555dldY0cNZeK/5rYMPa2TtE3pASytM/HUc69x+30TOOeUbwGwxqpDOHDP7fjJaZfSp3dP1l5jGPVVfFlDpZWrF1bqEJWkHwPj0sUGYANgcES8u7Q6KxnUsmSKIU2XdRFAr6HrtPPPrjpNn/cRwwb0blkeNqAPM9//uN2yu286vNXQ0zpH46D+rXpXs+bMY9BK/ZYo9+ob0zn7f/7O6Sd8gwH9Vmh5ffedNmf3nTYH4OIr72bwoP6Vb3Q1KtMN7VmSGUfEWcBZafmvAj8oFtCgssPPLJliuqznps1jjcYVGbFSH3rUi902Hc59U2YsUa5v7wa2XGtl7p285DrrWOuvPYK33pnDOzPn8umiRdz3yLNsu0XrxEQzZr/Hyb+9ihOO3o/VVmlstW7uvPktZR56fAo7fWGTDmt7NRFJDzfLo4S8h6gOAq4qVWkle2otmWKAt0gyxRxcwe11qMVNwa9umsyfjhhDXR3c+MQ0XpkxnwO2Xh2Avz02FYB/23AoD780m4WfLm71/rMO3owxa63MwBV7cu/PduS8u1/mhiemdfjn6E7q6+s55rA9+Omv/kJTUxO77TiakasN5Za7Hgfgq18ew1+vu5/353/IuRffkr6njj+ecRQAJ599Ne9/8CENDXUce/geLScUup+ynQTIdIgKQNIKwK7A0SVbF1G5EZ+k3YFz+CxTzK+Kle81dJ0YMe7cirXHyu/yo7bp7CZYDoftsxMvPPvUckWk3sPWjTX+4w+Zyr505q5vAIXZn1oytEvaH9glIo5Ilw8BxkTEMW3rkXQA8PWI+GqpbVb0OrX2MsWYWReXbWjZrFiG9jyHqA4kw9ATfEeBmeUkoK48l2tkOkQlaQCwA/D1LJU6qJlZbuU4pJYxmTHA3sBdEbEgS70OamaWW0clM06XLwUuzVqng5qZ5ZPvmFqHc1Azs1yEPEmkmdUW99TMrKZU89RDDmpmlo+PqZlZLUnu/azeqOagZma5VXFMc1Azs/zKdEdBRTiomVk+ZZpPrVIc1Mwsl+b51KqVg5qZ5VTd2aQc1MwstyqOaQ5qZpaTfKLAzGqIr1Mzs5rjoGZmNaWKY5qDmpnlV809teqdFMnMqlPGnJ9Z4p6kXSW9KOkVSccvpcxYSZMkTZb0QKk63VMzs1ySSSI7JkO7pIHABcCuETFV0pBS9bqnZma51UmZHiVkydB+MHBDREwFiIiZJdu2DJ/HzLq5HMPPRklPFjyOLKimvQztI9psal1gJUn3S5og6Rul2ubhp5nlonw3tBdLZtxeJdFmuQHYHNgZ6AM8KumxiHhpaRt0UDOz3Mp0Q0GWDO3TSALjAmCBpPHApkD+oCbpDywZNVtExLEZGm1mNagDM7TfBJwnqQHoCWwF/K5YpcV6ak8ue1vNrFaJ5Azo8sqSoT0inpd0B/AM0ARcHBHPFat3qUEtIv5SuCxpxaxp382stpXrfvaMGdrPAs7K3LZSBSRtI2kK8Hy6vKmkC7JuwMxqjJL51LI8OkOWSzrOAXYB5gBExNPA9pVslJlVt3LdUVAJmc5+RsSbbaLu4so0x8yqnSDLhbWdJktQe1PStkBI6gkcSzoUNbPuqZonicwy/PwO8D2SK33fAjZLl82sG8o69Kza4WdEzAbGdUBbzKyLqObhZ5azn2tJukXSLEkzJd0kaa2OaJyZVSdlfHSGLMPPK4FrgOHAKsC1wFWVbJSZVbeufkmHIuKvEbEofVxOkdunzKy2JWc/sz06Q7F7P1dOn96Xzkh5NUkwOwC4tQPaZmbVSOWZJLJSip0omEASxJpb/+2CdQGcWqlGmVl1q+YcBcXu/VyzIxtiZl1D8/CzWmW6o0DSRsAooHfzaxFxWaUaZWbVrUv21JpJOgkYSxLUbgN2Ax4CHNTMuqnqDWnZzn7uRzKV7vSIOJRk1sleFW2VmVUtCerrlOnRGbIMPxdGRJOkRZL6AzMBX3xr1o1V8/AzS0/tyTT33p9IzohOBB6vaKvMrKp1VDLjNJHxvDSZ8SRJvyhVZ5Z7P7+bPr0wnVa3f0Q8U7q5ZlaLRKacnqXryZDMOPVgROyRtd5iF9+OLrYuIiZm3YiZ1ZDyzcDRkswYQFJzMuO2QS2XYj21s4usC2Cn5dlwezZadQAPn7l7uau1Clppy6M7uwmWw8evvlWWenIcU2uUVJjE6aKIuCh93l4y463aqWMbSU+TpM/7UURMLrbBYhff7pitzWbWnQio77hkxhOBNSJivqTdgb8D6xTbYJYTBWZmrZTphvaSyYwj4v2ImJ8+vw3oIamxaNtyfxoz6/bKFNRakhmnqQIOBG4uLCBpmNKxrqQxJDFrTrFKM90mZWbWLLlco2OSGZNc/H+UpEXAQuDAiCg69VmW26REMp33WhFxiqTVgWER4WvVzLqpjkpmHBHnAeflaluGMhcA2wAHpcsfkFxbYmbdVJdOvAJsFRGjJT0FEBFz0/GvmXVDAhqq+DapLEHt0/TK3wCQNBhoqmirzKyqVXFMyxTUfg/cCAyR9CuSA3cnVrRVZla1pPLcJlUpWe79vELSBJLphwTsFRHO0G7WjVVxTMt09nN14EPglsLXImJqJRtmZtWrq0/nfSufJWDpDawJvAhsWMF2mVmVEnTaBJBZZBl+bly4nM7e8e2lFDezWteJOT2zyH1HQURMlLRlJRpjZl2DqjhLQZZjaj8sWKwDRgOzKtYiM6tqtZAir1/B80Ukx9iur0xzzKwr6LJBLb3otm9E/LiD2mNmXUA1J14pNp13Q3oX/VKn9Taz7idJkdfZrVi6Yj21x0mOn02SdDNwLbCgeWVE3FDhtplZlerSdxQAK5NMyrYTn12vFoCDmlk31JVPFAxJz3w+x2fBrFnRSdrMrLZVcUet6Hxq9UDf9NGv4Hnzw8y6JVGX8VGyphLJjAvKbSlpsaT9StVZrKf2TkScUrJVZtatiPL01LImM07L/YZk2u+SivXUqriDaWadRtBQp0yPElqSGUfEJ0BzMuO2jiG5NnZmluYVC2o7Z6nAzLqX5p5axum8GyU9WfA4sqCq9pIZj2i1LWkEsDdwIRkVS2b8btZKzKx7yXFJx/ImMz4H+GlELM56wa9T5JlZbmU6+1kymTGwBXB1GtAagd0lLYqIvy+tUgc1M8tFlC0LeksyY+AtkmTGBxcWiIg1W7YrXQr8X7GABg5qZpaXynNHQcZkxrk5qJlZLskdBeUZf5ZKZtzm9W9mqdNBzcxyq+brvRzUzCy3ar5NykHNzHJS15xPzcysPWU8+1kRDmpmlltXn0/NzOwz6qLTeZuZtcfDTzOrOe6pmVlNqd6Q5qBmZjkJqHdPzcxqSRXHNAc1M8tLqIoHoA5qZpabe2pmVjOSSzqqN6o5qJlZPnJPzcxqTDXfJlXNFwabWRVKJonM9ihZV4lkxpL2lPSMpElpNqovlqrTPTUzy60cZz8zJjO+B7g5IkLSJsA1wPrF6nVPzcxyy5H3s5iSyYwjYn5ENKfNW5ElU+gtwT21nP7xyBROOPs6Fjc1ccie2/KDb3651fqI4Pizr+PuhyfTp3dPLjjpEDZdP8kCNu+DDzn2tCt5/tV3kOAPPx/HmE3W4tmXpnHcGVcz/8OPWX34IC469T/o37dPZ3y8mrfzNhtw+nH7UV9Xx19veoRz/nJ3q/UD+vXhvJ9/nTVXbeSjTz7lmFOv4PlX36FXzwZuvej79OrRQH1DPTff8xRnXHTbUrZS+8p0nVp7yYy3WmJb0t7A6cAQ4CulKq1YT03SJZJmSnquUtvoaIsXN/HjM6/h2nO/y2PXnMj1d03ghdfeaVXm7kem8OrUWUy44STO+dlBHHfG1S3rjj/7OnbeZhSPX/dzHrzyBNZbcxgA/3nalZz0vT155Or/Yo8dN+UPf72nQz9Xd1FXJ876ydfY/z8vYOuvnca+X968ZR80O+7QXXj2pWl88eDTOeqkv3L6cfsB8PEni9jzqN+z3bgz2P7g09l5m1FssdHITvgUnS/nMbViGdqzJDMmIm6MiPWBvYBTS7WvksPPS4FdK1h/h5sw+V+stVojI1dtpGePBvb50mhue+CZVmVue+AZDvzKGCSx5cZrMu+DhUyfPY/35y/kkade5ZA9twGgZ48GBvRbAYBXps5k29GfA2DsmPW55b5JHfvBuonNNxzJa2/O5o235vDposXccPdEdt9hk1Zl1ltzGOOfeBGAl9+YwerDV2bwyv0AWLDwEwB6NNTTo6Gez0ZF3YxEXcYHaYb2gsdFBTVlSWbcIiLGA2tLaizWvIoFtbQB71aq/s7wzqx5jBi6UsvyKkNX4p1Z89qUea91mSEDeWfme7zx1hwaB/ble7+8nO3HncGxp13BgoUfA7D+WsO5ffyzANx0z0TemjG3Az5N9zN88IBW3+3bM+YyfPCAVmWee/kt9thxMwBGj1qD1YatzCpDBgJJT2/8Fcfz0l1ncP8/X2DC5Dc6rvFVRhkfJbQkM5bUkySZ8c2ttiN9Tuk8R5JGAz2BOcUq7fQTBZKObO6azpo9q7ObU1R7f5nbHgxt74+3JBYtXszTL77JYfttx/grjmeF3r0459LkeM55vxjHxdeOZ+whv2H+hx/To0d9JZrf7bU3B1jb/XXOX+5mYP8VGH/F8Rx5wA4889I0Fi9uAqCpKdh+3Bls+JUTGb3hGmyw9vCOaHbVac77mbGntlQRsQhoTmb8PHBNczLj5oTGwL7Ac5ImkZwpPSBKdJE7/URB2h29CGDzzbeo6v78KkMGLvGXfljjgOJlZr7HsMEDEGKVIQNbjsP8+86btRykXnfkMG4472gAXnljBnc9NLnCn6R7envme0v0tKfPbt3T/mDBRxx9yuUty0/f9EveeLt1x+D9+Qt5aMLL7LzNKJ5/tfUx1e6iXJfelkpmHBG/AX6Tp85O76l1JaNHrcGrU2fxxluz+eTTRdxw90R22771MZndtt+Yq299nIjgiWdfp3/fPgxrHMDQxv6MGLoSL/9rBgDjn3ix5SD1rHc/AKCpqYnfXnInh+5b8vpCWwYTp7zB2qsPZvVVBtGjoZ59vjSa28e3Pibav28fejQkPeVv7LUtjzz1Ch8s+IhBA/u2nJHu3asHY8es17Ivu6UyjT8rodN7al1JQ0M9Z/7ka+x77PksXhyM+/et2WDt4Vxy/YMAHLbvdnz5Cxty98OTGb33L+nTuwfn/+LrLe8/80f7c+QvLuWTTxczckRjy7rr73ySi68bD8AeYzdj3Fe37vgP1w0sXtzET868hut//z3q68UVNz/GC69N59B9kj8i/3vDQ6y35jD+ePIhLG5q4sXXp3PMqVcAMKyxPxecfAj1dXXU1Ykb/zGROx+qmRP7uVXzbVKq1BkcSVcBY4FGYAZwUkT8udh7Nt98i3j4n09WpD1WGStteXRnN8Fy+PjFa2j6cOZyRaQNNv58XHbT/ZnKjll74ISI2GJ5tpdXxXpqEXFQpeo2s05WvR01Dz/NLJ/kcFn1RjUHNTPLx/OpmVmtqeKY5qBmZnnJyYzNrLZUcUxzUDOzfDrxutpMHNTMLL8qjmoOamaWmy/pMLOa4mNqZlY7fJ2amdUaDz/NrGYI99TMrMZUcUzzJJFmtgzKNElkhgzt49IM7c9IekTSpqXqdE/NzHIrxySRGTO0vw7sEBFzJe1GMvX/ErlBW7VtuVtmZt1OmTpqWTK0PxIRzUk/HiNJo1eUg5qZ5Zc9qhVLZtxehvYRRbZ6OHB7qaZ5+GlmueScJHJ2kem8M2VoB5C0I0lQK5mVyEHNzPIp38W3mTK0S9oEuBjYLSKKJjIGDz/NbBl0YIb21YEbgEMi4qUsbXNPzcxyKs8kkRGxSFJzhvZ64JLmDO3p+guBXwCDgAvSbS4qlZ3KQc3McivXHQUZMrQfARyRp04HNTPLxZNEmlntqeKo5qBmZrl5lg4zqymepcPMaoegzkHNzGpL9UY1BzUzy8WTRJpZzanimOagZmb5uadmZjWlHLdJVYqDmpnlVr0hzUHNzHKS836aWa3xHQVmVluqN6Y5qJlZflUc0xzUzCwvlSVFXqV4Om8zy6X5joIsj5J1lU5mvL6kRyV9LOlHWdrnnpqZdYqMyYzfBY4F9spar3tqZpZbmXpqWZIZz4yIJ4BPs7bNQc3MclPG/yhvMuNMPPw0s3zyXXxblmTGeTiomVkuZZx6KFMy47w8/DSz3HIMP4spmcx4WbinZma5laOnliWZsaRhwJNAf6BJ0veBURHx/tLqdVAzs9zKdelthmTG00mGpZk5qJlZftV7Q4GDmpnlI6jq26QUsdxnUMtG0izgjc5uRwU0ArM7uxGWS63usyMS3EYAAAQLSURBVDUiYvDyVCDpDpLvJ4vZEbHr8mwvr6oKarVK0pNFrtWxKuR91nX5kg4zqykOamZWUxzUOsZFnd0Ay837rIvyMTUzqynuqZlZTXFQM7Oa4qBWQaWmKrbqI+kSSTMlPdfZbbFl46BWIQVTFe8GjAIOkjSqc1tlGVwKdOjFolZeDmqVU3KqYqs+ETGeZF5866Ic1CqnIlMVm1lxDmqVU5Gpis2sOAe1yqnIVMVmVpyDWuVUZKpiMyvOQa1CImIR0DxV8fPANRExuXNbZaVIugp4FFhP0jRJh3d2mywf3yZlZjXFPTUzqykOamZWUxzUzKymOKiZWU1xUDOzmuKg1oVIWixpkqTnJF0raYXlqOtSSfulzy8udrO9pLGStl2GbfxL0hJZh5b2epsy83Nu62RJP8rbRqs9Dmpdy8KI2CwiNgI+Ab5TuDKdGSS3iDgiIqYUKTIWyB3UzDqDg1rX9SDwubQXdZ+kK4FnJdVLOkvSE5KekfRtACXOkzRF0q3AkOaKJN0vaYv0+a6SJkp6WtI9kkaSBM8fpL3E7SQNlnR9uo0nJH0hfe8gSXdJekrS/5Ahj7ekv0uaIGmypCPbrDs7bcs9kganr60t6Y70PQ9KWr8cX6bVDmdo74IkNZDM03ZH+tIYYKOIeD0NDPMiYktJvYCHJd0FfB5YD9gYGApMAS5pU+9g4E/A9mldK0fEu5IuBOZHxG/TclcCv4uIhyStTnLXxAbAScBDEXGKpK8ArYLUUhyWbqMP8ISk6yNiDrAiMDEijpP0i7Tuo0kSonwnIl6WtBVwAbDTMnyNVqMc1LqWPpImpc8fBP5MMix8PCJeT1//MrBJ8/EyYACwDrA9cFVELAbelnRvO/VvDYxvrisiljav2L8Bo6SWjlh/Sf3SbeyTvvdWSXMzfKZjJe2dPl8tbescoAn4W/r65cANkvqmn/fagm33yrAN60Yc1LqWhRGxWeEL6Y97QeFLwDERcWebcrtTeuojZSgDyWGLbSJiYTttyXzfnaSxJAFym4j4UNL9QO+lFI90u++1/Q7MCvmYWu25EzhKUg8ASetKWhEYDxyYHnMbDuzYznsfBXaQtGb63pXT1z8A+hWUu4tkKEharjnIjAfGpa/tBqxUoq0DgLlpQFufpKfYrA5o7m0eTDKsfR94XdL+6TYkadMS27BuxkGt9lxMcrxsYpo85H9IeuQ3Ai8DzwJ/BB5o+8aImEVyHOwGSU/z2fDvFmDv5hMFwLHAFumJiCl8dhb2l8D2kiaSDIOnlmjrHUCDpGeAU4HHCtYtADaUNIHkmNkp6evjgMPT9k3GU6RbG56lw8xqintqZlZTHNTMrKY4qJlZTXFQM7Oa4qBmZjXFQc3MaoqDmpnVlP8H0ln39jHwoqkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.metrics import plot_confusion_matrix\n",
    "\n",
    "predictions = classifier.predict(X_test)\n",
    "accuracy = np.sum(predictions == y_test)/np.size(predictions)\n",
    "print(accuracy)\n",
    "\n",
    "matrix = plot_confusion_matrix(classifier, X_test, y_test,\n",
    "                                 cmap=plt.cm.Blues,\n",
    "                                 normalize='true')\n",
    "plt.title('Confusion matrix for our classifier')\n",
    "plt.show(matrix)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a final note, we used two variables (radius and texture) because it made it possible to visualize. However, there is nothing preventing us from using the rest of the data set columns as part of our features. While visualization in more than 2D is much more difficult, the underlying principles are exactly the same.\n",
    "\n",
    "### Exercise\n",
    "Explore this link which contains a number of classification datasets: https://archive.ics.uci.edu/ml/datasets.php?format=&task=cla&att=&area=&numAtt=&numIns=&type=&sort=nameUp&view=table. Using an SVM, attempt to predict the class of each sample. Consider the following:\n",
    "- While multiclass classification is possible, limit yourself thus far to datasets that have one class. Make sure that y is set up with two distinct numbers for the classes (e.g. 0 and 1)\n",
    "- Make sure that your dataset does not have missing values. If it does, handle them appropriately (e.g. by substituting it with 0 or with the mean of the column).\n",
    "\n",
    "How does your classifier perform? Report the accuracy and confusion matrix. What could be done to improve the performance?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>school</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>address</th>\n",
       "      <th>famsize</th>\n",
       "      <th>Pstatus</th>\n",
       "      <th>Medu</th>\n",
       "      <th>Fedu</th>\n",
       "      <th>Mjob</th>\n",
       "      <th>Fjob</th>\n",
       "      <th>...</th>\n",
       "      <th>freetime</th>\n",
       "      <th>goout</th>\n",
       "      <th>Dalc</th>\n",
       "      <th>Walc</th>\n",
       "      <th>health</th>\n",
       "      <th>absences</th>\n",
       "      <th>G1</th>\n",
       "      <th>G2</th>\n",
       "      <th>G3</th>\n",
       "      <th>Grade</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>GP</td>\n",
       "      <td>F</td>\n",
       "      <td>18</td>\n",
       "      <td>U</td>\n",
       "      <td>GT3</td>\n",
       "      <td>A</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>at_home</td>\n",
       "      <td>teacher</td>\n",
       "      <td>...</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>6</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>GP</td>\n",
       "      <td>F</td>\n",
       "      <td>17</td>\n",
       "      <td>U</td>\n",
       "      <td>GT3</td>\n",
       "      <td>T</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>at_home</td>\n",
       "      <td>other</td>\n",
       "      <td>...</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>GP</td>\n",
       "      <td>F</td>\n",
       "      <td>15</td>\n",
       "      <td>U</td>\n",
       "      <td>LE3</td>\n",
       "      <td>T</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>at_home</td>\n",
       "      <td>other</td>\n",
       "      <td>...</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>10</td>\n",
       "      <td>7</td>\n",
       "      <td>8</td>\n",
       "      <td>10</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GP</td>\n",
       "      <td>F</td>\n",
       "      <td>15</td>\n",
       "      <td>U</td>\n",
       "      <td>GT3</td>\n",
       "      <td>T</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>health</td>\n",
       "      <td>services</td>\n",
       "      <td>...</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>2</td>\n",
       "      <td>15</td>\n",
       "      <td>14</td>\n",
       "      <td>15</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>GP</td>\n",
       "      <td>F</td>\n",
       "      <td>16</td>\n",
       "      <td>U</td>\n",
       "      <td>GT3</td>\n",
       "      <td>T</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>other</td>\n",
       "      <td>other</td>\n",
       "      <td>...</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>6</td>\n",
       "      <td>10</td>\n",
       "      <td>10</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>390</th>\n",
       "      <td>MS</td>\n",
       "      <td>M</td>\n",
       "      <td>20</td>\n",
       "      <td>U</td>\n",
       "      <td>LE3</td>\n",
       "      <td>A</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>services</td>\n",
       "      <td>services</td>\n",
       "      <td>...</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>11</td>\n",
       "      <td>9</td>\n",
       "      <td>9</td>\n",
       "      <td>9</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>391</th>\n",
       "      <td>MS</td>\n",
       "      <td>M</td>\n",
       "      <td>17</td>\n",
       "      <td>U</td>\n",
       "      <td>LE3</td>\n",
       "      <td>T</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>services</td>\n",
       "      <td>services</td>\n",
       "      <td>...</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>14</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>392</th>\n",
       "      <td>MS</td>\n",
       "      <td>M</td>\n",
       "      <td>21</td>\n",
       "      <td>R</td>\n",
       "      <td>GT3</td>\n",
       "      <td>T</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>other</td>\n",
       "      <td>other</td>\n",
       "      <td>...</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>10</td>\n",
       "      <td>8</td>\n",
       "      <td>7</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>393</th>\n",
       "      <td>MS</td>\n",
       "      <td>M</td>\n",
       "      <td>18</td>\n",
       "      <td>R</td>\n",
       "      <td>LE3</td>\n",
       "      <td>T</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>services</td>\n",
       "      <td>other</td>\n",
       "      <td>...</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "      <td>10</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>394</th>\n",
       "      <td>MS</td>\n",
       "      <td>M</td>\n",
       "      <td>19</td>\n",
       "      <td>U</td>\n",
       "      <td>LE3</td>\n",
       "      <td>T</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>other</td>\n",
       "      <td>at_home</td>\n",
       "      <td>...</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>8</td>\n",
       "      <td>9</td>\n",
       "      <td>9</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>395 rows × 34 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    school sex  age address famsize Pstatus  Medu  Fedu      Mjob      Fjob  \\\n",
       "0       GP   F   18       U     GT3       A     4     4   at_home   teacher   \n",
       "1       GP   F   17       U     GT3       T     1     1   at_home     other   \n",
       "2       GP   F   15       U     LE3       T     1     1   at_home     other   \n",
       "3       GP   F   15       U     GT3       T     4     2    health  services   \n",
       "4       GP   F   16       U     GT3       T     3     3     other     other   \n",
       "..     ...  ..  ...     ...     ...     ...   ...   ...       ...       ...   \n",
       "390     MS   M   20       U     LE3       A     2     2  services  services   \n",
       "391     MS   M   17       U     LE3       T     3     1  services  services   \n",
       "392     MS   M   21       R     GT3       T     1     1     other     other   \n",
       "393     MS   M   18       R     LE3       T     3     2  services     other   \n",
       "394     MS   M   19       U     LE3       T     1     1     other   at_home   \n",
       "\n",
       "     ... freetime goout  Dalc  Walc  health absences  G1  G2  G3  Grade  \n",
       "0    ...        3     4     1     1       3        6   5   6   6  False  \n",
       "1    ...        3     3     1     1       3        4   5   5   6  False  \n",
       "2    ...        3     2     2     3       3       10   7   8  10  False  \n",
       "3    ...        2     2     1     1       5        2  15  14  15   True  \n",
       "4    ...        3     2     1     2       5        4   6  10  10  False  \n",
       "..   ...      ...   ...   ...   ...     ...      ...  ..  ..  ..    ...  \n",
       "390  ...        5     4     4     5       4       11   9   9   9  False  \n",
       "391  ...        4     5     3     4       2        3  14  16  16   True  \n",
       "392  ...        5     3     3     3       3        3  10   8   7  False  \n",
       "393  ...        4     1     3     4       5        0  11  12  10  False  \n",
       "394  ...        2     3     3     3       5        5   8   9   9  False  \n",
       "\n",
       "[395 rows x 34 columns]"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filepath = \"student-mat.csv\"\n",
    "df = pd.read_csv(filepath, delimiter = ';')\n",
    "df['Grade'] = df['G3'] > 10\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[-1.99902609  0.        ]]\n",
      "[0.99983768]\n"
     ]
    }
   ],
   "source": [
    "# setting the features and the targets\n",
    "X = df[[\"failures\", \"absences\"]]\n",
    "y = df[\"Grade\"]\n",
    "\n",
    "# splitting training and testing set 80/20\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n",
    "\n",
    "# Initialize SVM classifier\n",
    "classifier = svm.SVC(kernel='linear')\n",
    "\n",
    "# Fit data\n",
    "classifier = classifier.fit(X_train, y_train)\n",
    "\n",
    "# Printing parameters\n",
    "coef = classifier.coef_\n",
    "print(coef)\n",
    "print(classifier.intercept_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.5949367088607594\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUgAAAEWCAYAAAAEkA60AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3debxVdb3/8debAwoCigyigiIqDjgGgjPOhWUOmdeByrGyIn9evZaVldpwMzW7NzHCNM3KqRwwySHKMVSGcABTCVEQGQ7iAOIVDp/fH2sd3Gz23mcf2PusDfv95LEf7LXWd33Xd+91zud8h7W+SxGBmZmtqV3WBTAzq1UOkGZmRThAmpkV4QBpZlaEA6SZWREOkGZmRThAVpikTpLuk/SOpDvXIZ8Rkh6qZNmyIulgSS+t5b47S/qnpPcknVfpsrUVSYdKmlPF/EdL+m7O8lckzZe0RFKP9P/tq3X8DZXq9TpISacBFwC7AO8BU4EfRcQT65jv54GvAwdExIp1LmiNkxTAgIiYUaX8bwDejYj/rEb+bUXSocDvIqJvGxyrA/AusF9EPFvt423I6rIGKekC4OfAj4HewLbAdcBxFci+H/ByPQTHckhqv45Z9AOmZXTslvJvqGb+66A30JG1/N5yVfs7rHkRUVcvYDNgCXBSiTQbkwTQuenr58DG6bZDgTnAhcAC4E3gzHTbZcCHwPL0GGcDl5LUHJrz3g4IoH26fAYwk6QW+yowImf9Ezn7HQBMBN5J/z8gZ9sjwA+AJ9N8HgJ6FvlszeX/Rk75jwc+CbwMvAV8Oyf9UGAC8Haa9lpgo3TbY+lnWZp+3pNz8v8mMA+4pXldus8O6TEGpctbA43AoQXK+jegCfggzX+n9Pz9FlgIvAZcArTL+c6eBK5Jj/HDVp7b1b7zdF0AO6bvbwJ+CYxLP/ORBfLvDvwmzXsxcE/u956T7mLg3+n5mg6ckLNtR+DR9Fw3Aren65V+tgXptueA3XPK9sP0O1qalnsJ8LcCn2Nj4CrgdWA+MBrolPfzser8Zf07m2m8yLoAbf6BYTiwgjRAFUlzOfAUsAXQC/gH8IN026Hp/pcDHUgCy/vA5un2S1k9IOYvb5f+sLYHOpM0hXZOt20F7Ja+X/XLmv7SLQY+n+53arrcI93+SPrLthPQKV3+SZHP1lz+76Xl/yJJsPkD0BXYjSQgbZ+mHwzslx53O+BF4Pyc/Fb94uXlf0X6i9iJNYPDF9N8NgEeBK4qcS4eAc7JWf4tcG9a1u1IgvrZOd/ZCpIujvbNv/StOLervvNCn48kCL0DHEjS+upYIP/7gduBzdPv95Cc7yX3OziJ5I9DO5I/LEuBrdJttwLfaT4GcFC6/hPAZKAbSbDcNWefm0j/IJD3R7jA5/g5MJbk56orcB/w38XOX9a/s5nGi6wL0OYfGEYA81pI82/gkznLnwBmpe8PBZbl/fAtIOnvgdYHyLeBE/N/EFk9QH4eeCZv+wTgjPT9I8AlOdu+CjxQ5LM1l78hXe6almffnDSTgeOL7H8+cHfOcqEA+SE5wSM/OKTrxgLPk9SCNi5xLh4hDZBAA/B/wMCc7V8GHsn5zl5fh3O76jsv9PlIgtBvS+S9FbCS9I9lge99Tol9pwLHpe9/C4wB+ualOZzkD8J+pLXmnG03UUaAJAmsS4EdcrbtD7xa7PzV86se+yAXAT1b6FvZmqT51uy1dN2qPGL1Psb3gS6tLUhELCWpPZwLvCnpfkm7lFGe5jL1yVme14ryLIqIpvT9svT/+TnblzXvL2knSX+WNE/SuyT9tj1L5A2wMCI+aCHN9cDuwC8i4v9aSNusJ7ARa56b3O9hdgt5tHRuW1Iq/22AtyJicUuZSPqCpKmS3pb0Nsl30fy9foMkkD0jaZqkswAi4m8kXRyjgPmSxkjatBVlh6TWvAkwOefYD6Trm5Vz/upCPQbICSRNyONLpJlLMjjQbNt03dpYSvID2WzL3I0R8WBEHEVS+/gXSeBoqTzNZXpjLcvUGr8kKdeAiNgU+DbJL28pJS+NkNSFpJl3A3CppO5llqWRpH83/9zkfg8lj03pc7vauZK02rkqI//ZQHdJ3UoVQFI/kvM8kqSbpBvwAun3GhHzIuKLEbE1SQ35Okk7ptv+NyIGk3SF7ARcVOpYBTSS/AHcLSK6pa/NIiL3D2pL32HdqLsAGRHvkPS/jZJ0vKRNJHWQdLSkn6bJbgUukdRLUs80/e/W8pBTgWGStpW0GfCt5g2Seks6VlJnkqbjEpJBiXzjgJ0knSapvaSTgYHAn9eyTK3RlaSfdElau/1K3vb5QGuvr/sfYHJEnEPSZze6nJ3SWu8dwI8kdU0DzQW07tyUOrfPArtJ2ltSR5LukbJFxJvAX0gC2ubpz9WwAkk7kwShhQCSziSpQZIunySp+XKgxWnaJklDJO2bXsazlOQPfaGfl1JlXEkSnK+RtEV6vD6SPtGafOpF3QVIgIj4Gckv1iUkP6SzSf6a35Mm+SEwiaR/7HlgSrpubY71MEmn/XMkfXu5Qa0dyWj4XJJR10NI+g/z81gEHJOmXUTSBDsmIhrXpkyt9F/AaSSjrdeTfJZclwI3p821/2gpM0nHkQyUnZuuugAYJGlEmeX5OklwmAk8QTK4dGOZ+0KJcxsRL5MM4vwVeCXNv7U+T1LL/RdJ3/T5+QkiYjpwNUlrZj6wB8noe7MhwNOSlpD01f6/iHgV2JTkHCwm6RpYRDIa3VrfBGYAT6XdJn8Fdl6LfDZ4dXuhuJlZS+qyBmlmVg4HSDOzIhwgzcyKcIA0Mytig7wRvVv3HrF1322zLoa1wox572VdBGul5Qv/3RgRvVpOWVzDpv0iVixrOSEQyxY+GBHD1+V4rbVBBsit+27LLWMfzboY1grHXjE+6yJYK80d/Zn8u7taLVYsY+OdW7w6DIAPpo5q6Q6uitsgA6SZrS8Eqt2ePgdIM8uOgHa1Oq2mA6SZZU0t3dqfHQdIM8uQm9hmZsW5BmlmVoBwDdLMrDC5BmlmVpRHsc3MCvEgjZlZYcJNbDOzolyDNDMrxE1sM7PCBDTU7iBN7YZuM6sPUnmvFrPRcEkvSZoh6eIC2zeTdJ+kZ9PnjZ/ZUp4OkGaWobSJXc6rVC5SAzAKOJrkkcinShqYl+xrwPSI2As4FLha0kal8nWANLNsVaYGORSYEREzI+JD4DbguLw0AXSVJKALyaOWV5TK1H2QZpat8gdpekqalLM8JiLGpO/7kDzfvtkcYN+8/a8lec74XKArcHJErCx1QAdIM8tOmf2LqcaI2KdYTgXWRd7yJ4CpwOHADsDDkh6PiHeLHdBNbDPLVruG8l6lzQG2yVnuS1JTzHUmcFckZgCvAruULForP4qZWQVVZpAGmAgMkNQ/HXg5haQ5net14AgASb2BnYGZpTJ1E9vMslWBWw0jYoWkkcCDQANwY0RMk3Ruun008APgJknPkzTJvxkRjaXydYA0s+xUcD7IiBgHjMtbNzrn/Vzg463J0wHSzDLkWw3NzIrzfJBmZkV4ujMzswLkJraZWXGuQZqZFSYHSDOzNSVPXHCANDNbk4TaOUCamRXkGqSZWREOkGZmRThAmpkVIgrP5FgjHCDNLDNCrkGamRXTrp3vpDEzK8g1SDOzQtwHaWZWnGuQZmYFeJDGzKwE32poZlaI3MQ2MyuqlgNk7V6AZGZ1QVJZrzLyGS7pJUkzJF1cYPtFkqamrxckNUnqXipPB0gzy0zzIM26BkhJDcAo4GhgIHCqpIG5aSLiyojYOyL2Br4FPBoRb5XK1wHSzLKlMl+lDQVmRMTMiPgQuA04rkT6U4FbW8rUfZBmlh216lbDnpIm5SyPiYgx6fs+wOycbXOAfQseUtoEGA6MbOmADpBmlqlWDNI0RsQ+xbIpsC6KpP008GRLzWtwgDSzrFVmEHsOsE3Ocl9gbpG0p1BG8xocIGvSU1Ne5n9u+DMrV67kmCOH8PkTD1lt++NPT+fXt/4VSTQ0tOO8sz7FXgO3y6awBsCwXbfguyfuSUM7cfuE1/jVwy+vtv2LRwzg2H36AtC+XTt22LIrQ751P++8vzyL4taUCl3mMxEYIKk/8AZJEDytwLE2Aw4BPldOplULkJKagOdzVh0fEbOKpF0SEV2qVZb1SVPTSn42ZizXXHoWW/TYlHO+cR0HDd2F/tv0XpVm8J47cNDQXZHEjFlv8r2rbuUP116QYanrWzvBpSftxemjnmTe28u4+6LDGP/8m8yY996qNNePf4Xrx78CwOG7b8lZh+3o4AhlX8LTkohYIWkk8CDQANwYEdMknZtuH50mPQF4KCKWlpNvNWuQy9LhdGuFF1+ZQ9+tetBny+TyrCMP2pMnnnlxtQC5SaeNV73/4IPlqJanQ6kDe/XrzmuNS5m96H0A/jx5DkfusdVqATLXpwf35b7Jc9qyiDWtUheKR8Q4YFzeutF5yzcBN5WbZ5td5iOpi6TxkqZIel7SGkPwkraS9FjOhZwHp+s/LmlCuu+dkjbY2ubCt95hi56brVru1WMzFi56d410jz41jdNG/oyLfnQz3xp5YlsW0fL07taRNxcvW7U87+1l9O7WsWDajh0aGLZrbx6Y+kZbFa/mqZ3KemWhmgGyU85V63cDHwAnRMQg4DDgaq35p+M04MG05rkXMFVST+AS4Mh030nAGu1JSV+SNEnSpMWLFlXxY1VXFBh3K/QH9pD9duMP117Af1/8Oa6/9eHqF8yKKvirW2T89Ig9tmTyzEVuXueo1J001dBmTWxJHYAfSxoGrCS5bqk3MC9nn4nAjWnaeyJiqqRDSK6MfzL9kjYCJuQfLL0eagzAwD0/Vmx4v+Zt0WMzFjS+s2p54aJ36Nl906Lp996tP3PnvcXb7y6l26ad26KIlmfe2x+w1eadVi1v2a0T89/5oGDaYwa5eb2aGp+soi3vpBkB9AIGp4FzPrBaOyQiHgOGkYxC3SLpCyR/oB9uvkUoIgZGxNltWO42tcuAPsx+s5G5899i+fIV/PWJ5zhwyK6rpZnz5iIirWq+9O83WL6iic26bpJFcQ147vXFbNerC317bEKHBnHM4L6Mf/7NNdJ16dieoTv25K8FttUrkbSQynlloS0v89kMWBARyyUdBvTLTyCpH/BGRFwvqTMwCPgRMErSjhExI70Kvm9EvJy//4agfUMDF3zxWC647DesXBl86ojBbL9tb+554GkAjh++L49MeIEHHvkn7Rsa2Hij9lx24Sk1/Vd4Q9e0Mrjszme56asH0k7wx6de45V573HqgdsBcOuTswD4xF5b88S/FrDsw6bsCltzanvCXEWhTq9KZJx36U7al3gf0AGYChwIHB0Rs5rTSjoduAhYDiwBvhARr0o6HLgCaB6+vSQixhY79sA9Pxa3jH20Kp/LquPYK8ZnXQRrpbmjPzO5xJ0tZem45U7R7/RflJX25Z8OX+fjtVbVapD51zVGRCOwf6m0EXEzcHOB7X8DhlShmGaWpQybz+XwnTRmlhkB7fzIBTOzwlyDNDMropYHaRwgzSw77oM0MytMqDUT5rY5B0gzy5RrkGZmRbgP0sysEPdBmpkVltyLXbsR0gHSzDJVw/HRAdLMsuU7aczMCqnx+SAdIM0sM83zQdYqB0gzy1BtzwdZu5ewm1ldqNSM4pKGS3pJ0gxJFxdJc2j6nKxpklqcNNY1SDPLjiozSCOpARgFHAXMASZKGhsR03PSdAOuA4ZHxOuStmgpX9cgzSwzzddBVuCphkOBGRExMyI+BG4D8h8tfRpwV0S8DhARC1rK1AHSzDJVoQDZB5idszwnXZdrJ2BzSY9Impw+FLAkN7HNLFOtGKPpKWlSzvKY9HHPUPjx5PkP3GoPDAaOADoBEyQ9VeoBgA6QZpapVoxiN5Z4aNccYJuc5b7A3AJpGiNiKbBU0mPAXkDRAOkmtpllp8wR7DJi6ERggKT+kjYCTgHyn3x6L3CwpPbp46P3BV4slalrkGaWmWTC3HUfxY6IFZJGAg8CDcCNETFN0rnp9tER8aKkB4DngJXAryPihVL5OkCaWabaVehC8YgYB4zLWzc6b/lK4Mpy83SANLNM1fCNNA6QZpYdebIKM7Piani2s+IBUtIvWPM6olUi4ryqlMjM6sr6Oh/kpBLbzMzWmUhGsmtV0QAZETfnLkvqnF5gaWZWMTVcgWz5QnFJ+0uaTnpBpaS9JF1X9ZKZ2YavzPuwsxrIKedOmp8DnwAWAUTEs8CwahbKzOpHpeaDrIayRrEjYnZeBG+qTnHMrJ6Iyl0oXg3lBMjZkg4AIr3H8TxauH/RzKxctTyKXU4T+1zgayRzq70B7J0um5mtk3Kb1zXbxI6IRmBEG5TFzOpQLTexyxnF3l7SfZIWSlog6V5J27dF4cxsw6cyX1kop4n9B+AOYCtga+BO4NZqFsrM6sf6fpmPIuKWiFiRvn5HiVsQzczKlYxil/fKQql7sbunb/+ePmP2NpLAeDJwfxuUzcw2dKrMhLnVUmqQZjJJQGwu/ZdztgXwg2oVyszqx3o53VlE9G/LgphZ/WluYteqsu6kkbQ7MBDo2LwuIn5brUKZWf1YL2uQzSR9HziUJECOA44GngAcIM1sndVueCxvFPuzJA/anhcRZ5I8R3bjqpbKzOqCBA3tVNYrC+U0sZdFxEpJKyRtCiwAfKG4mVVELTexy6lBTpLUDbieZGR7CvBMVUtlZnWjUvdiSxou6SVJM9JLE/O3HyrpHUlT09f3WsqznHuxv5q+HZ0+dHvTiHiu5eKamZUmVJF7sSU1AKOAo4A5wERJYyNiel7SxyPimHLzLXWh+KBS2yJiSrkHMTMrqHIz9QwFZkTETABJtwHHAfkBslVK1SCvLrEtgMPX5cDV1KlDA7v13TTrYlgrvPXU+KyLYBlpRR9kT0m5DxMcExFj0vd9gNk52+YA+xbIY39JzwJzgf+KiGmlDljqQvHDyiuzmdnaEdBQfoBsjIh9SmSVL3/OiClAv4hYIumTwD3AgFIHLGeQxsysaio0WcUcYJuc5b4ktcRVIuLdiFiSvh8HdJDUs2TZWv1pzMwqqEIBciIwQFL/9NEwpwBjcxNI2lJpe17SUJL4t6hUpmXdamhmVg3JJTzrPkoTESskjQQeBBqAGyNimqRz0+2jSW56+YqkFcAy4JSIKDl1Yzm3GorkkQvbR8TlkrYFtowIXwtpZuusUjfJpM3mcXnrRue8vxa4tlVlKyPNdcD+wKnp8nsk1xuZma2z9fqhXcC+ETFI0j8BImJx2sY3M1snAtrX8K2G5QTI5elV6gEgqRewsqqlMrO6UcPxsawA+b/A3cAWkn5E0tF5SVVLZWZ1QarMrYbVUs692L+XNJlkyjMBx0fEi1UvmZnVhRqOj2WNYm8LvA/cl7suIl6vZsHMrD6s749cuJ+PHt7VEegPvATsVsVymVkdEGQ2GW45ymli75G7nM7y8+Uiyc3MypfhM6/L0eo7aSJiiqQh1SiMmdUf1fBTacrpg7wgZ7EdMAhYWLUSmVnd2BAe+9o15/0Kkj7JP1WnOGZWb9bbAJleIN4lIi5qo/KYWZ2p5Yd2lXrkQvt0hoyij14wM1sXyWNfsy5FcaVqkM+Q9DdOlTQWuBNY2rwxIu6qctnMrA6s13fSAN1JJpU8nI+uhwzAAdLM1sn6PEizRTqC/QIfBcZmJSeZNDMrVw1XIEsGyAagC+U9DMfMbC2IduvpdZBvRsTlbVYSM6s7Yv2tQdZwsc1sgyBoX8OdkKUC5BFtVgozq0vrbQ0yIt5qy4KYWX2q5ct8avgSTTOrB5V6aJek4ZJekjRD0sUl0g2R1CTpsy3l6QBpZpkRSRAq51Uyn+S26FHA0cBA4FRJA4uku4Lk+dktcoA0s+woaWKX82rBUGBGRMyMiA+B24DjCqT7OslkOwvKKZ4DpJllJrmTpiIBsg8wO2d5Trruo2NJfYATgNHllq/VE+aamVVSK4ZoekqalLM8JiLGlMgm/4aWnwPfjIimcmcQcoA0s0y1YhC7MSL2KbJtDrBNznJfYG5emn2A29Lg2BP4pKQVEXFPsQM6QJpZhlSp+SAnAgMk9QfeAE4BTstNEBH9Vx1Vugn4c6ngCA6QZpah5lHsdZXOXTuSZHS6AbgxIqZJOjfdXna/Yy4HSDPLVKUuFI+IccC4vHUFA2NEnFFOng6QZpYdraePXDAzq7ZKNbGrxQHSzDLlGqSZWRG1Gx4dIM0sQwIaXIM0MyushuOjA6SZZUmohhvZDpBmlinXIM3MCkgu86ndCOkAaWbZKXO28Kw4QJpZpmr5mTQOkGaWmWTC3KxLUZwDpJllyqPYZmZF1HALu6bvE68rf/3HdIaceDmDTriUa256aI3tL8+ax8fPuoreB5zPL2756xrbm5pWMmzETzj5P3/ZFsU14Ij9d+WZP36XyXd9n/NPP2qN7Zt27sitP/syj//+Yv5x+3c47dP7fbStSydu+snZPH3nJTx1xyUM2aP/GvvXC5X5LwttUoOU1AMYny5uCTQBC9PloelTyOpWU9NKLvrpHdx97Ui27t2Nw0+/kqOH7cEu22+1Ks3mm3bmJxeexP2PPlswj9G3/Z2d+vfmvaUftFWx61q7duLKb/wHJ4y8lrnz3+ZvN1/EXx57npdenbcqzTknDeOlmfM49YJf0aNbFyb+8bvc+ZeJLF/RxE8u/CzjJ0znjItvoEP7Bjp13CjDT5OdWu+DbJMaZEQsioi9I2JvkieKXdO8HBEfSqrrpv7kabPYfpuebNe3Jxt1aM9njhrEuEefWy1Nr+5dGbRbPzq0b1hj/zfmL+ahJ6bxheMOaKsi173Bu23HzNmNvPbGIpavaOKuh6fwyUP2XC1NAF06bwxA5002ZvG777OiaSVdO3fkgI/twC33TgBg+Yom3l2yrK0/Qm0o84mGWY10ZxaY0mdCvAV8DJgi6T1gSURclW5/ATgmImZJ+hxwHrAR8DTw1Yhoyqbklffmwnfo03vzVctb996cyS/MKnv/b//sT1x23vEsed+1x7ayVa/NeGP+4lXLc+cvZvDu262W5vo7HuUPV3+ZF//yI7ps0pGzv30jEUG/Pj1ofHsJo77/OXYf0IepL87mW1f/kfc/qM+GVA1XIDPvg9wJODIiLiyWQNKuwMnAgWkNtAkYUSDdlyRNkjRpYePC/M01LSL/6ZTld1w/8Pjz9Ny8K3vvum2FS2WlFJrDMP80Hr7frjz/8hx2Pfo7DBvx3/z0opPo2rkj7Rsa2Gvnbbjxj49zyOeu4P0P/o/zz1izD7MeVPC52FWRdYC8s4ya4BHAYGCipKnp8vb5iSJiTETsExH79OrZqwpFrZ6tt+i2Rm1ky56blbXv08/O5IHHn2fPY7/H2d/+DY9PfJkvfffmahXVUnMXvL1GrX9e4zurpRnx6f3489+TPuNX5zTy2txFDOjXm7kLFjN3wdtMnvYaAGPHT2WvnbehXqnMVxayDpBLc96vYPXydEz/F3BzTp/lzhFxaVsVsC0MGtiPf7++kNfeaOTD5Su46+EpHD1sz5Z3BL4/8jim3f9Dnht7OTf8+EwOHrITY35wepVLbFOmv8YO2/Zi26170KF9A585ahB/eWz1fuM58xYzbMjOQNKHvGO/3sx6o5EFi97jjfmL2bHfFgAMG7LzaoM7daeGI2QtDY7MAo4BkDQIaL7uYTxwr6RrImKBpO5A14h4LZtiVl779g389Bv/wYnnjaKpKRhx7H7susNW3PinxwE468SDmd/4Loef/lPeW/oBkhh92yNMuP07bNqlU8alr09NTSv5xk/v4E//+zUaGsTvxz7Fv2bO48zPHATAb+56gitveIBR3/8cT976bSS47Np7eeudpE7wjavuZMzlZ7BRhwZmvdHI1y7/XZYfJ1O1fKuhCvV/VfWA0qXAEmB3kgd3/zFd3wm4F9iC5CHgBwFHp4M0JwPfIqlhLge+FhFPFTvG4MH7xJNPT6rq57DK2nzIyKyLYK30wdRRkyNin3XJY9c9Pha/vfeRstIO3aFbyeNJGg78D8lzsX8dET/J234c8ANgJUmL9fyIeKLUMdu8BlmseRwRy4CPF9l2O3B7FYtlZlmpQAVSUgMwCjgKmEMyZjE2IqbnJBsPjI2IkLQncAewS6l8s+6DNLM6lnQvVuROmqHAjIiYmd54chtwXG6CiFgSHzWZO5NcqlqSA6SZZSedD7KcF9Cz+VK+9PWlnJz6ALNzluek61Y/nHSCpH8B9wNntVS8WhqkMbM61IoWdmOJPshC2axRQ4yIu4G7JQ0j6Y88stQBHSDNLEMqeNH9WpgD5F5M2heYWyxxRDwmaQdJPSOisVg6N7HNLFOtaGKXMhEYIKm/pI2AU4Cxqx9HOyqNxumlhBsBi0pl6hqkmWWmUteAR8QKSSOBB0ku87kxIqZJOjfdPho4EfiCpOXAMuDkaOE6RwdIM8tWha4Tj4hxwLi8daNz3l8BXNGaPB0gzSxTfuSCmVkRNXynoQOkmWXIz8U2MyvOTWwzswKEa5BmZkXVcHx0gDSzjNVwhHSANLNM1fKEuQ6QZpap2g2PDpBmlrUajpAOkGaWmeYJc2uVA6SZZccXipuZFVfD8dEB0syyVLEJc6vCAdLMMlXD8dEB0syyU6kJc6vFAdLMslXDEdIB0swy5ct8zMyKcB+kmVkhgnYOkGZmxdRuhPRzsc0sM80T5lbgudhIGi7pJUkzJF1cYPsISc+lr39I2qulPF2DNLNMVaL+KKkBGAUcBcwBJkoaGxHTc5K9ChwSEYslHQ2MAfYtla8DpJllqkKDNEOBGRExM8lTtwHHAasCZET8Iyf9U0DfljJ1gDSzTLXiVsOekiblLI+JiDHp+z7A7JxtcyhdOzwb+EtLB3SANLNMtaIC2RgR+7QimyiYUDqMJEAe1NIBHSDNLDPlDsCUYQ6wTc5yX2DumsfTnsCvgaMjYlFLmXoU28wypTL/tWAiMEBSf0kbAacAY1c7jrQtcBfw+Yh4uZyyuQZpZtmqQA0yIlZIGgk8CDQAN0bENEnnpttHA98DegDXpf2eK0o02QEHSDPLWKUuE4+IccC4vHWjc96fA5zTmjwdIJbJEW0AAAXXSURBVM0sQ/JjX83MCmm+k6ZWeZDGzKwI1yDNLFO1XIN0gDSzTHnCXDOzQvxcbDOzwmp9kMYB0swy5Sa2mVkRrkGamRVRw/HRAdLMMlbDEdIB0swyI6jpWw0VUXBOyfWapIXAa1mXo0p6Ao1ZF8JaZUM9Z/0iote6ZCDpAZLvpxyNETF8XY7XWhtkgNyQSZrU0hRNVlt8ztZfvhfbzKwIB0gzsyIcINc/Y1pOYjXG52w95T5IM7MiXIM0MyvCAdLMrAhfKJ4xSU3A8zmrjo+IWUXSLomILm1SMCtJUg9gfLq4JdAELEyXh0bEh5kUzCrKfZAZa03Qc4CsTZIuBZZExFU569pHxIrsSmWV4CZ2jZHURdJ4SVMkPS/puAJptpL0mKSpkl6QdHC6/uOSJqT73inJwbQNSbpJ0s8k/R24QtKlkv4rZ/sLkrZL339O0jPpOfyVpIaMim0lOEBmr1P6SzJV0t3AB8AJETEIOAy4WlrjZtXTgAcjYm9gL2CqpJ7AJcCR6b6TgAva7mNYaieSc3BhsQSSdgVOBg5Mz2ETMKKNymet4D7I7C1Lf0kAkNQB+LGkYcBKoA/QG5iXs89E4MY07T0RMVXSIcBA4Mk0nm4ETGijz2AfuTMimlpIcwQwGJiYnqtOwIJqF8xazwGy9owAegGDI2K5pFlAx9wEEfFYGkA/Bdwi6UpgMfBwRJza1gW21SzNeb+C1VtpzedRwM0R8a02K5WtFTexa89mwII0OB4G9MtPIKlfmuZ64AZgEPAUcKCkHdM0m0jaqQ3LbWuaRXJukDQI6J+uHw98VtIW6bbu6Tm1GuMaZO35PXCfpEnAVOBfBdIcClwkaTmwBPhCRCyUdAZwq6SN03SXAC9Xv8hWxJ+AL0iaStIt8jJAREyXdAnwkKR2wHLga2y4U/Stt3yZj5lZEW5im5kV4QBpZlaEA6SZWREOkGZmRThAmpkV4QBZpyQ15dzLfaekTdYhr5skfTZ9/2tJA0ukPVTSAWtxjFnp7ZRlrc9Ls6SVx1rtHmqrXw6Q9WtZROwdEbsDHwLn5m5c28kTIuKciJheIsmhQKsDpFkWHCAN4HFgx7R293dJfwCel9Qg6UpJEyU9J+nLAEpcK2m6pPuBLZozkvSIpH3S98PTmYWeTWco2o4kEP9nWns9WFIvSX9KjzFR0oHpvj0kPSTpn5J+RXJ7XkmS7pE0WdI0SV/K23Z1Wpbxknql63aQ9EC6z+OSdqnEl2kbDt9JU+cktQeOBh5IVw0Fdo+IV9Mg805EDEnvznlS0kPAx4CdgT1IJtKYDtyYl28v4HpgWJpX94h4S9JocuZOTIPxNRHxhKRtgQeBXYHvA09ExOWSPgWsFvCKOCs9RieSiSD+FBGLgM7AlIi4UNL30rxHkjxM69yIeEXSvsB1wOFr8TXaBsoBsn51Sm+Bg6QGeQNJ0/eZiHg1Xf9xYM/m/kWS+8QHAMOAW9NZa+ZK+luB/PcDHmvOKyLeKlKOI4GBOTO6bSqpa3qMz6T73i9pcRmf6TxJJ6Tvt0nLuohkVqTb0/W/A+5SMlfmAcCdOcfeGLMcDpD1a7Vp1gDSQJE7G42Ar0fEg3npPgm0dI+qykgDSTfP/hGxrEBZyr4PVtKhJMF2/4h4X9Ij5M2ClCPS476d/x2Y5XIfpJXyIPCVdN5JJO0kqTPwGHBK2ke5FcnEvvkmAIdI6p/u2z1d/x7QNSfdQyTNXdJ0zQHrMdJJZCUdDWzeQlk3AxanwXEXkhpss3ZAcy34NJKm+7vAq5JOSo8hSXu1cAyrMw6QVsqvSfoXp0h6AfgVSavjbuAVkoeN/RJ4NH/HiFhI0m94l6Rn+aiJex9wQvMgDXAesE86CDSdj0bTLwOGSZpC0tR/vYWyPgC0l/Qc8AOS6d+aLQV2kzSZpI/x8nT9CODstHzTgDUeb2H1zbP5mJkV4RqkmVkRDpBmZkU4QJqZFeEAaWZWhAOkmVkRDpBmZkU4QJqZFfH/AS+HpBawzvWcAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "predictions = classifier.predict(X_test)\n",
    "accuracy = np.sum(predictions == y_test)/np.size(predictions)\n",
    "print(accuracy)\n",
    "\n",
    "matrix = plot_confusion_matrix(classifier, X_test, y_test,\n",
    "                                 cmap=plt.cm.Blues,\n",
    "                                 normalize='true')\n",
    "plt.title('Confusion matrix for our classifier')\n",
    "plt.show(matrix)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deep learning\n",
    "So far, we've worked exclusively with linear decision boundaries. However, usually data is much more complicated (as you should have seen from the exercise). Therefore, we might want to look for something that has higher predictive power. One of the most popular tools in machine learning has been the *neural network*. A neural network is (loosely) inspired by the way neurons are connected and operate. Let's first look at the simplest possible neural network and work our way up:\n",
    "\n",
    "<img src=\"notebook_images/nn_nohidden_new.png\" width=\"500\"  />\n",
    "\n",
    "Neural networks are made up of *layers*. In the image above, the neural network has two layers. The first layer is called the *input layer*, and the last layer is called the *output layer*. Each layer is composed of *neurons*. In the input layer, each neuron is one of the features from the training data. For example, the first neuron might be the mean radius, and the second neuron might be the mean texture. We're going to refer to each neuron by $x_{l, n}$, where $l$ is the number of the layer and $n$ is the number of the neuron within the layer. Using this notation, mean radius = $x_{1, 1}$.\n",
    "\n",
    "Each neuron in the input layer is connected to *every* neuron in the following layer by a *weight*. We will denote each weight by $w_{l, i, o}$, where $l$ is the layer at the tail of the arrow, $i$ is the number of the neuron at the tail of the arrow and $o$ is the number of the neuron at the head of the arrow. In the figure above, the color of the arrow corresponds to the value of the weight (blue means negative, red means positive). Every neuron in subsequent layers is equal to the sum of the products of the neurons and the weight that connects them. In this case, the neuron in the output layer is defined as:\n",
    "\n",
    "\\begin{equation}\n",
    "x_{2, 1} = x_{1, 1}w_{1, 1, 1} + x_{1, 2}w_{1, 2, 1} + w_0\n",
    "\\end{equation}\n",
    "\n",
    "Once you have the value of the output layer, you can obtain the predicted classification by applying some function to $x_{2, 1}$. An example of such a function would be similar to what we did previously:\n",
    "\n",
    "\\begin{equation}\n",
    "y_{pred} =\n",
    "\\begin{cases}\n",
    "    1               & \\text{if } x_{2, 1}\\geq 0\\\\\n",
    "    0,              & \\text{otherwise}\n",
    "\\end{cases}\n",
    "\\end{equation}\n",
    "\n",
    "So training a neural network is simply finding the weights $w$ that minimize the error. If you are keen-eyed though, you might have realized that the example above is identical to what we were doing before. Therefore, we still have not arrived at a more powerful classifier. Let's try a more complicated neural network.\n",
    "\n",
    "<img src=\"notebook_images/nn_hidden_new.png\" width=\"500\"  />\n",
    "\n",
    "This network is similar to the previous one, but now we have an additional layer in between the input and output layers. All layers that are between input and output are called *hidden layers*, though functionally they behave in exactly the same way. As a quick concept question, **what is the value of $x_{2, 3}$? What about $x_{3, 1}$?**\n",
    "\n",
    "<details>\n",
    "    <summary>ANSWER</summary>\n",
    "    $x_{2, 3} = x_{1, 1}w_{1, 1, 3} + x_{1, 2}w_{1, 2, 3}$ ,  \n",
    "    $x_{3, 1} = x_{2, 1}w_{2, 1, 1} + x_{2, 2}w_{2, 2, 1} + x_{2, 3}w_{2, 3, 1} + x_{2, 4}w_{2, 4, 1} + x_{2, 5}w_{2, 5, 1}$\n",
    "</details>\n",
    "\n",
    "Why are we introducing a hidden layer? Notice that each of the neurons in the hidden layer are essentially acting as linear classifiers. By having a weighted sum of these neurons in the output layer, we are arriving at a function that, by virtue of being a sum of lines is not itself linear. One of the biggest results in machine learning is that, as you increase the number of neurons in the hidden layer, you can approximate *any* function, meaning that you can always make a more powerful classifier by just adding more neurons. However, usually this is not computationally efficient. Rather, most of the time it makes more sense to introduce more hidden layers as opposed to more neurons, as shown in the image below:\n",
    "\n",
    "<img src=\"notebook_images/nn_deep.png\" width=\"500\"  />\n",
    "\n",
    "The added \"depth\" as a result of more layers is where the \"deep\" in \"deep learning\" comes. As computational resources have increased, scientists have increasingly resorted to adding more and more layers to their neural networks. \n",
    "\n",
    "### Exercise\n",
    "Let's start thinking about the LADI dataset. We want to use deep learning to detect whether there is flooding in the image.\n",
    "- Using the paradigm above, what would be the features of the dataset? What are the targets?\n",
    "- Propose a way that you would input the features into a neural network?\n",
    "- What problems might you run into if you directly put an image into the neural network above?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Convolution\n",
    "Convolution is a fairly old technique in image processing. Convolution works by multiplying a matrix (called a *kernel*) with portions of the image and adding up the result. An animation of how this works is shown below. The result of applying different kernels is also shown:\n",
    "\n",
    "<img src=\"notebook_images/convolution_anim.gif\" width=\"500\"  />\n",
    "\n",
    "<img src=\"notebook_images/convolution.png\" width=\"500\"  />\n",
    "\n",
    "As you can see, different kernels can pick up different features depending on the specific coefficients of the kernel. In the late 1980s, scientists first thought to use neural networks to learn the kernel values themselves, which created the foundation for the *convolutional* neural network (CNN). These complement the hidden layers from the previous architecture with new convolutional layers, which perform convolution on the previous layer. In so doing, these networks conserve the relationship between nearby pixels and perform very well in image classification.\n",
    "\n",
    "<img src=\"notebook_images/cnn.png\" width=\"500\"  />\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The LADI dataset\n",
    "As we already saw, the LADI dataset contains CAP images as well as crowdsourced labels for a number of different categories. Today we're going to start the multi-day exercise of applying deep learning to the LADI dataset in order to do classification of flooding. This section borrows heavily from the following tutorial: https://github.com/LADI-Dataset/ladi-tutorial\n",
    "\n",
    "### Data reading and cleaning\n",
    "We first have to go through some work to make sure that we only have the components of the dataset that we want. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>url</th>\n",
       "      <th>WorkerId</th>\n",
       "      <th>Answer</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>https://ladi.s3-us-west-2.amazonaws.com/Images...</td>\n",
       "      <td>0</td>\n",
       "      <td>['damage:flood/water']</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>https://ladi.s3-us-west-2.amazonaws.com/Images...</td>\n",
       "      <td>1</td>\n",
       "      <td>['damage:flood/water']</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>https://ladi.s3-us-west-2.amazonaws.com/Images...</td>\n",
       "      <td>2</td>\n",
       "      <td>['damage:flood/water']</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>https://ladi.s3-us-west-2.amazonaws.com/Images...</td>\n",
       "      <td>20</td>\n",
       "      <td>['damage:flood/water']</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>https://ladi.s3-us-west-2.amazonaws.com/Images...</td>\n",
       "      <td>0</td>\n",
       "      <td>['damage:flood/water']</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>289835</th>\n",
       "      <td>https://ladi.s3-us-west-2.amazonaws.com/Images...</td>\n",
       "      <td>488</td>\n",
       "      <td>['damage:none']</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>289836</th>\n",
       "      <td>https://ladi.s3-us-west-2.amazonaws.com/Images...</td>\n",
       "      <td>496</td>\n",
       "      <td>['damage:none']</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>289837</th>\n",
       "      <td>https://ladi.s3-us-west-2.amazonaws.com/Images...</td>\n",
       "      <td>516</td>\n",
       "      <td>['damage:misc']</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>289838</th>\n",
       "      <td>https://ladi.s3-us-west-2.amazonaws.com/Images...</td>\n",
       "      <td>539</td>\n",
       "      <td>['damage:none']</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>289839</th>\n",
       "      <td>https://ladi.s3-us-west-2.amazonaws.com/Images...</td>\n",
       "      <td>539</td>\n",
       "      <td>['damage:none']</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>289840 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                      url  WorkerId  \\\n",
       "0       https://ladi.s3-us-west-2.amazonaws.com/Images...         0   \n",
       "1       https://ladi.s3-us-west-2.amazonaws.com/Images...         1   \n",
       "2       https://ladi.s3-us-west-2.amazonaws.com/Images...         2   \n",
       "3       https://ladi.s3-us-west-2.amazonaws.com/Images...        20   \n",
       "4       https://ladi.s3-us-west-2.amazonaws.com/Images...         0   \n",
       "...                                                   ...       ...   \n",
       "289835  https://ladi.s3-us-west-2.amazonaws.com/Images...       488   \n",
       "289836  https://ladi.s3-us-west-2.amazonaws.com/Images...       496   \n",
       "289837  https://ladi.s3-us-west-2.amazonaws.com/Images...       516   \n",
       "289838  https://ladi.s3-us-west-2.amazonaws.com/Images...       539   \n",
       "289839  https://ladi.s3-us-west-2.amazonaws.com/Images...       539   \n",
       "\n",
       "                        Answer  \n",
       "0       ['damage:flood/water']  \n",
       "1       ['damage:flood/water']  \n",
       "2       ['damage:flood/water']  \n",
       "3       ['damage:flood/water']  \n",
       "4       ['damage:flood/water']  \n",
       "...                        ...  \n",
       "289835         ['damage:none']  \n",
       "289836         ['damage:none']  \n",
       "289837         ['damage:misc']  \n",
       "289838         ['damage:none']  \n",
       "289839         ['damage:none']  \n",
       "\n",
       "[289840 rows x 3 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read the tsv file with the labels\n",
    "file = pd.read_csv(\"http://ladi.s3-us-west-2.amazonaws.com/Labels/ladi_aggregated_responses_url.tsv\",delimiter='\\t',header='infer')\n",
    "file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Strip off bracket and comma from the Answer catagory\n",
    "file[\"Answer\"] = file[\"Answer\"].str.strip('[|]')\n",
    "file[\"Answer\"] = file[\"Answer\"].str.split(\",\",expand = True)\n",
    "\n",
    "# Extract labels with damage and infrastructure categories\n",
    "label_damage_infra = file[file['Answer'].str.contains('damage|smoke/fire',na=False,case=False)]\n",
    "\n",
    "#Filter out infrastructure label with label 'none'\n",
    "label_clean = label_damage_infra[~label_damage_infra['Answer'].str.contains('none',na=False,case=False)]\n",
    "\n",
    "# Extract data with label does contain 'flood'\n",
    "label_flood = label_clean[label_clean['Answer'].str.contains('flood',na=False,case=False)]\n",
    "\n",
    "# Extract url data with the label does contain 'flood'\n",
    "im_flood_lst = label_flood['url'].unique().tolist()\n",
    "\n",
    "# Extract url data with the label does not contain 'flood'\n",
    "label_notflood = label_damage_infra[~label_damage_infra['url'].isin(im_flood_lst)]\n",
    "im_not_flood_lst = label_notflood['url'].unique().tolist()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have the images we're interested in, we're going to generate the true/false labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "0\n"
     ]
    }
   ],
   "source": [
    "# Load ladi_images_metadata.csv\n",
    "metadata = pd.read_csv('http://ladi.s3-us-west-2.amazonaws.com/Labels/ladi_images_metadata.csv')\n",
    "\n",
    "# Generate flood and non-flood metadata\n",
    "flood_metadata = metadata[metadata['url'].isin(im_flood_lst)]\n",
    "not_flood_metadata = metadata[metadata['url'].isin(im_not_flood_lst)]\n",
    "\n",
    "# Generate url and s3_path features into list\n",
    "flood_meta_lst = flood_metadata['url'].tolist()\n",
    "flood_meta_s3_lst = flood_metadata['s3_path'].tolist()\n",
    "\n",
    "not_flood_meta_lst = not_flood_metadata['url'].tolist()\n",
    "not_flood_meta_s3_lst = not_flood_metadata['s3_path'].tolist()\n",
    "\n",
    "# Check how many images do not have metadata but have human labels\n",
    "human_label_only = list(set(im_flood_lst) - set(flood_meta_lst))\n",
    "print(len(human_label_only))\n",
    "human_label_non_flood = list(set(im_not_flood_lst) - set(not_flood_meta_lst))\n",
    "print(len(human_label_non_flood))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the purposes of this exercise, we're going to take just a sample of the overall imagery. We're going to take 100 images that are labeled as flood and 100 that are not:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "from random import sample\n",
    "# sampling\n",
    "flood_tiny_lst = sample(flood_meta_s3_lst, 100)\n",
    "not_flood_tiny_lst = sample(not_flood_meta_s3_lst, 100)\n",
    "flood_tiny_metadata = metadata[metadata['s3_path'].isin(flood_tiny_lst+not_flood_tiny_lst)]\n",
    "\n",
    "# creating the new datasets\n",
    "flood_data = []\n",
    "for path in flood_tiny_lst:\n",
    "    data_lst = []\n",
    "    data_lst.append(path)\n",
    "    data_lst.append(True)\n",
    "    flood_data.append(data_lst)\n",
    "\n",
    "not_flood_data = []\n",
    "for path in not_flood_tiny_lst:\n",
    "    data_lst = []\n",
    "    data_lst.append(path)\n",
    "    data_lst.append(False)\n",
    "    not_flood_data.append(data_lst)\n",
    "\n",
    "label_data = flood_data+not_flood_data\n",
    "label_df = pd.DataFrame(label_data, columns = ['s3_path', 'label']) \n",
    "\n",
    "flood_tiny_metadata.to_csv('flood_tiny_metadata.csv')\n",
    "label_df.to_csv('flood_tiny_label.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise\n",
    "Take a look at the LADI csv files. Find some classification task that you find meaningful (e.g. detecting a bridge in an image). Create a csv file like the ones above for the metadata and the label. You might have to do some additional steps in data cleaning, depending on your choice"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'damage:none'                            100266\n",
       "'damage:flood/water'                      28921\n",
       "'damage:misc'                             24262\n",
       "'vehicle:none'                            16275\n",
       "'water:none'                              14521\n",
       "'infrastructure:building'                 13377\n",
       "'infrastructure:none'                     12489\n",
       "'vehicle:car'                             12462\n",
       "'environment:dirt'                        12257\n",
       "'environment:grass'                        8866\n",
       "'environment:none'                         7281\n",
       "'water:lake'                               5240\n",
       "'damage:rubble'                            5032\n",
       "'water:river'                              4362\n",
       "'water:flooding'                           4254\n",
       "'infrastructure:bridge'                    3297\n",
       "'vehicle:boat'                             2611\n",
       "'water:ocean'                              2422\n",
       "'infrastructure:road'                      2278\n",
       "'environment:trees'                        2267\n",
       "'damage:smoke/fire'                        1351\n",
       "'water:puddle'                             1259\n",
       "'damage:landslide'                          870\n",
       "'environment:shrubs'                        831\n",
       "'damage:washout'                            702\n",
       "'vehicle:truck'                             521\n",
       "'environment:sand'                          282\n",
       "'infrastructure:utility-line'               244\n",
       "'vehicle:aircraft'                          207\n",
       "'environment:rock'                          181\n",
       "'infrastructure:dam-levee'                  160\n",
       "'infrastructure:railway'                    135\n",
       "'infrastructure:pipe'                       108\n",
       "'infrastructure:water-tower'                 86\n",
       "'infrastructure:communications-tower'        70\n",
       "'environment:snow/ice'                       62\n",
       "'environment:lava'                           31\n",
       "Name: Answer, dtype: int64"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "file[\"Answer\"].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Strip off bracket and comma from the Answer catagory\n",
    "file[\"Answer\"] = file[\"Answer\"].str.strip('[|]')\n",
    "file[\"Answer\"] = file[\"Answer\"].str.split(\",\",expand = True)\n",
    "\n",
    "# Extract labels with damage and infrastructure categories\n",
    "label_damage_infra = file[file['Answer'].str.contains('damage|infrastructure',na=False,case=False)]\n",
    "\n",
    "#Filter out infrastructure label with label 'none'\n",
    "label_clean = label_damage_infra[~label_damage_infra['Answer'].str.contains('none',na=False,case=False)]\n",
    "\n",
    "# Extract data with label does contain 'flood'\n",
    "label_fire = label_clean[label_clean['Answer'].str.contains('smoke/fire',na=False,case=False)]\n",
    "\n",
    "# Extract url data with the label does contain 'flood'\n",
    "im_fire_lst = label_fire['url'].unique().tolist()\n",
    "\n",
    "# Extract url data with the label does not contain 'flood'\n",
    "label_notfire = label_damage_infra[~label_damage_infra['url'].isin(im_fire_lst)]\n",
    "im_not_fire_lst = label_notfire['url'].unique().tolist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "0\n"
     ]
    }
   ],
   "source": [
    "# Load ladi_images_metadata.csv\n",
    "metadata = pd.read_csv('http://ladi.s3-us-west-2.amazonaws.com/Labels/ladi_images_metadata.csv')\n",
    "\n",
    "# Generate flood and non-flood metadata\n",
    "fire_metadata = metadata[metadata['url'].isin(im_fire_lst)]\n",
    "not_fire_metadata = metadata[metadata['url'].isin(im_not_fire_lst)]\n",
    "\n",
    "# Generate url and s3_path features into list\n",
    "fire_meta_lst = fire_metadata['url'].tolist()\n",
    "fire_meta_s3_lst = fire_metadata['s3_path'].tolist()\n",
    "\n",
    "not_fire_meta_lst = not_fire_metadata['url'].tolist()\n",
    "not_fire_meta_s3_lst = not_fire_metadata['s3_path'].tolist()\n",
    "\n",
    "# Check how many images do not have metadata but have human labels\n",
    "human_label_only = list(set(im_fire_lst) - set(fire_meta_lst))\n",
    "print(len(human_label_only))\n",
    "human_label_non_fire = list(set(im_not_fire_lst) - set(not_fire_meta_lst))\n",
    "print(len(human_label_non_fire))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1248\n"
     ]
    }
   ],
   "source": [
    "from random import sample\n",
    "# sampling\n",
    "print(len(fire_meta_s3_lst))\n",
    "fire_tiny_lst = sample(fire_meta_s3_lst, len(fire_meta_s3_lst))\n",
    "not_fire_tiny_lst = sample(not_fire_meta_s3_lst, len(fire_tiny_lst))\n",
    "fire_tiny_metadata = metadata[metadata['s3_path'].isin(fire_tiny_lst+not_fire_tiny_lst)]\n",
    "\n",
    "# creating the new datasets\n",
    "fire_data = []\n",
    "for path in fire_tiny_lst:\n",
    "    data_lst = []\n",
    "    data_lst.append(path)\n",
    "    data_lst.append(True)\n",
    "    fire_data.append(data_lst)\n",
    "\n",
    "not_fire_data = []\n",
    "for path in not_fire_tiny_lst:\n",
    "    data_lst = []\n",
    "    data_lst.append(path)\n",
    "    data_lst.append(False)\n",
    "    not_fire_data.append(data_lst)\n",
    "\n",
    "label_data = fire_data+not_fire_data\n",
    "label_df = pd.DataFrame(label_data, columns = ['s3_path', 'label']) \n",
    "\n",
    "fire_tiny_metadata.to_csv('fire_tiny_metadata.csv')\n",
    "label_df.to_csv('fire_tiny_label.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}