Traffic_Sign_Classifier.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Self-Driving Car Engineer Nanodegree\n",
    "\n",
    "## Deep Learning\n",
    "\n",
    "## Project: Build a Traffic Sign Recognition Classifier\n",
    "\n",
    "In this notebook, a template is provided for you to implement your functionality in stages, which is required to successfully complete this project. If additional code is required that cannot be included in the notebook, be sure that the Python code is successfully imported and included in your submission if necessary. \n",
    "\n",
    "> **Note**: Once you have completed all of the code implementations, you need to finalize your work by exporting the iPython Notebook as an HTML document. Before exporting the notebook to html, all of the code cells need to have been run so that reviewers can see the final implementation and output. You can then export the notebook by using the menu above and navigating to  \\n\",\n",
    "    \"**File -> Download as -> HTML (.html)**. Include the finished document along with this notebook as your submission. \n",
    "\n",
    "In addition to implementing code, there is a writeup to complete. The writeup should be completed in a separate file, which can be either a markdown file or a pdf document. There is a [write up template](https://github.com/udacity/CarND-Traffic-Sign-Classifier-Project/blob/master/writeup_template.md) that can be used to guide the writing process. Completing the code template and writeup template will cover all of the [rubric points](https://review.udacity.com/#!/rubrics/481/view) for this project.\n",
    "\n",
    "The [rubric](https://review.udacity.com/#!/rubrics/481/view) contains \"Stand Out Suggestions\" for enhancing the project beyond the minimum requirements. The stand out suggestions are optional. If you decide to pursue the \"stand out suggestions\", you can include the code in this Ipython notebook and also discuss the results in the writeup file.\n",
    "\n",
    "\n",
    ">**Note:** Code and Markdown cells can be executed using the **Shift + Enter** keyboard shortcut. In addition, Markdown cells can be edited by typically double-clicking the cell to enter edit mode."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "---\n",
    "## Step 0: Load The Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Load pickled data\n",
    "import pickle\n",
    "\n",
    "training_file = 'traffic-signs-data/train.p'\n",
    "validation_file = 'traffic-signs-data/valid.p'\n",
    "testing_file = 'traffic-signs-data/test.p'\n",
    "\n",
    "with open(training_file, mode='rb') as f:\n",
    "    train = pickle.load(f)\n",
    "with open(validation_file, mode='rb') as f:\n",
    "    valid = pickle.load(f)\n",
    "with open(testing_file, mode='rb') as f:\n",
    "    test = pickle.load(f)\n",
    "\n",
    "X_train, y_train = train['features'], train['labels']\n",
    "X_valid, y_valid = valid['features'], valid['labels']\n",
    "X_test, y_test = test['features'], test['labels']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "---\n",
    "\n",
    "## Step 1: Dataset Summary & Exploration\n",
    "\n",
    "The pickled data is a dictionary with 4 key/value pairs:\n",
    "\n",
    "- `'features'` is a 4D array containing raw pixel data of the traffic sign images, (num examples, width, height, channels).\n",
    "- `'labels'` is a 1D array containing the label/class id of the traffic sign. The file `signnames.csv` contains id -> name mappings for each id.\n",
    "- `'sizes'` is a list containing tuples, (width, height) representing the the original width and height the image.\n",
    "- `'coords'` is a list containing tuples, (x1, y1, x2, y2) representing coordinates of a bounding box around the sign in the image. **THESE COORDINATES ASSUME THE ORIGINAL IMAGE. THE PICKLED DATA CONTAINS RESIZED VERSIONS (32 by 32) OF THESE IMAGES**\n",
    "\n",
    "Complete the basic data summary below. Use python, numpy and/or pandas methods to calculate the data summary rather than hard coding the results. For example, the [pandas shape method](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.shape.html) might be useful for calculating some of the summary results. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Provide a Basic Summary of the Data Set Using Python, Numpy and/or Pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of training examples = 34799\n",
      "Number of validation examples = 4410\n",
      "Number of testing examples = 12630\n",
      "Image data shape = (32, 32)\n",
      "Number of classes = 43\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "n_train = np.size(y_train)\n",
    "n_valid = np.size(y_valid)\n",
    "n_test = np.size(y_test)\n",
    "image_shape = (np.shape(X_train)[1], np.shape(X_train)[2])\n",
    "n_classes = np.size(np.unique(y_train))\n",
    "\n",
    "print(\"Number of training examples =\", n_train)\n",
    "print(\"Number of validation examples =\", n_valid)\n",
    "print(\"Number of testing examples =\", n_test)\n",
    "print(\"Image data shape =\", image_shape)\n",
    "print(\"Number of classes =\", n_classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Include an exploratory visualization of the dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Visualize the German Traffic Signs Dataset using the pickled file(s). This is open ended, suggestions include: plotting traffic sign images, plotting the count of each sign, etc.\n",
    "\n",
    "The [Matplotlib](http://matplotlib.org/) [examples](http://matplotlib.org/examples/index.html) and [gallery](http://matplotlib.org/gallery.html) pages are a great resource for doing visualizations in Python.\n",
    "\n",
    "**NOTE:** It's recommended you start with something simple first. If you wish to do more, come back to it after you've completed the rest of the sections."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "38\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADUFJREFUeJztnE2oLVl1x39r76pz7n3dzy/yQaMSRRwkA2lBTCATIQRC\nJiYDgw7EQaAdRFBwEMkowwySTAMtCg4EERKIAyEESQYZJHSnEY1pEkVEWxs/BkH7vXdO1a69Mthr\n7apz3r3vnr73ufvRr/5wOOdU7dp716q11/cuUVVWtEF4rSfwOGEldkOsxG6IldgNsRK7IVZiN8RK\n7Ia4EbFF5A9E5H9E5Dsi8pmHNanXK+S6To2IROB/gd8HXgKeAz6iqv/98Kb3+kJ3g2vfD3xHVb8L\nICJfAj4IXErsbd/r+dkWFEQua3XpiQuhKKj/vrqHC1nLGE6hTkxkXvTZzoudcwYVa7/b7RjG4cqJ\n34TYbwV+sPj/EvDbx41E5BngGYDz7YYPPP0e0Ezwmwp+U3YjF5JK6nkxcqk1m3Jmyrkcq0RQpPbj\n/WttnyvFnXjl+kkViYUkcXNml0d2w1h+dj0Ayf5vohBi4N9feO6COd+PmxD7Iqrcxziq+izwLMCb\nb9/WLkYORVe50SKVjGDGVd5ONdfB/LsSTIRoD8zPaZ4I1sfkDe3hhiB0odz2ME52zsbLCezBRWs/\nThPRzueUAOiiPUDJKOmi274QNyH2S8DbF//fBvzowZcoOmVUpE7PieKsWgisi98FMyGNi/1IDOSp\nECEs2mt9iEao5dKvYuFwdgIEGztI+e6CEENpmO3B+fMLsSNEqeLlKtzEGnkOeLeIvFNENsCHga/c\noL/XPa7N2aqaROQTwD8BEfi8qn7rwVcJGiJZp8qZas/bRQE5c2whiQg5Tz5u+XaZXxoAReYCpg9c\nFHkfdeZgMrqKpos4s4p8Qeocy8HILJJ0oU+uwk3ECKr6VeCrN+njccKNiP1qoRTuU6RyduUJ49yl\nsgkh2iFFXba7iHe5a/0BtY2KLLqR2s7/uvURq0VjXYvUlYAp7Ekhm54IPlvvSgMZOVU/ru56SzTl\n7OKCFD4MRw5CNuuBhY2cl9aFHv5YiuBqU4tfl6uc9ZZdcBmeoY5V4CtIp5lJh32xpaXf1lVRdYmt\nwqxKJtDC9LsWsipBoZNyw9nolA7MMSeaX6XVmZntbXtYOeM36+0VIU2H9rJfH0Ko5mPWfHAdLByc\ntC/tme3++sStDTmCpgUnPBirGGmItgpSZwV57Lj4wpYFkzjnxSAlBsLspFTPMATGVJZ8mpzDQ+Xs\nztonMQUcFWI8OOceIikzTSY+psHmlwnmwoejVQKgWU/Vjytnt0RbmS2QRZhUmI6cDefojM6K8UCW\nehflx2ju/X5Ic4zDEGQpj8s5d3h2CXRTeGwzlXNPdoUMEmO9DuPwyDRH++p9+P8Mj6qCVIWUFdVZ\njIRqSMjcqMIUJULWQ49wMDGxGzOazR43pdt3wuSKLlvcpD8v/xOkwaOEbmgXwvYb6CyyN5qY2o0J\nSXM0cTEtQghk5T6P9zKsYqQhmnK2iBTOkW55sHyZHZzGgaAe6zDOFmE/FVNsMo5OFulDQ1V4NeAf\nhIy3L4pue/YkADFLNevc7LxnjJmGxLl7nKYU95nqYoq6uKrqHOXQH3gQVs5uiMacHei3Z3RxQ2ec\nI8blYsJ7SgmhcOqdoXDv3WHHkO4BkFLhrs5WxBtvnTOYk7I3E7Bwv8WjO/MgLRAypkW6a8GhAOMk\nqHmOm77M67w/YzSzcTRF7MmDcZzKalpjI48eGkf9lEkz0/4uaqmpGIr2r5kYQI3bnWF2aWA0Ge1x\niq3FM54IwtlmA8DPd7P7neMWgN76iov8pEcJj7NEGdibTRooXHy76zm/VfKRd0e3oMo4m15BIiGc\nxrONTT8ljTvSfldvykVGqPEQqUpzb2HO3TigppzkKFFwb79na4S9fVa+76bEbiy3lq3/0fRpznmR\nJZeDPnURrp1sfmka6TZzLhRm71UUm9eqIB85tPUgNaP7HTKlyjlUc8rjDoIa5yRPBky5OjNuru2r\np5dQK9l4cls4e+ojr4xF0Q1D6T/a9UXHGWd6IncRQnXHxWd1NyV6c4KGoZiRuZqCWkRWPgzZXoaV\nsxuieTy7xJMj0YIhucYZnNO1cpy4O61zuqq62M7hOdOZQO49BHC+5XxTFO8dM9f2KdW+Ok/cdmbC\nmTmZZa6u8qDiflJwee9RRU/h5UWs+wQ09yBD16EacC9MaxFN+SoFOUWp5bRcnkcZbF/60yxSvOZj\nG4UnLMahZ8VS+T/PIyZlY12dbUubu9bXfpoW+aKCSecaFV2kLqBk10OID6qlO8AqRhqisRgRYujQ\nkKsXpqZsJldamudStEUAXw4s8WVpmmBBOXZm/4ZxorflvbWA4K2N2+5TTcl51dPGRM5+mGtWljUl\nLj68tiV7HYkIIcjxmrsUK2c3RFuZjScJAmIyVcRlt3mICMHjJZKOrr4fWedon3uZu/1YYye9KcE3\ndoXFEwI2ljspBOd6qR5qTYFl9aAiwVZctjjLlBPTlB9eWkxE3i4i/yIiL4rIt0Tkk3b8LSLyzyLy\nbft+84ljPrY4RYwk4NOq+pvA7wB/JiK/BXwG+Jqqvhv4mv1/IJRifYQY8eqNrNRsR3GXc2F/KSag\nBCGEgJgX75V1IlI/3l5VUBWGpNzZDdzZDeScyTlzFoWzKJz3kb4L5dN39H1HypmUMzkrEgISAgEl\noIgqXYz26ehiR4w9MfYoUhyaE+PZV4oRVX0ZeNl+/0JEXqQUwn8Q+IA1+wLwr8CfP6gvEaHfnpGy\nzEWQwZRPLcONNVYRbOnLOKEeS6mFON4p1HCqm2Ch454pV9mXREH3RG/9w8bEwTDPzK6TOSeKF1Eq\nk+UqoynS4HNFaoHnKXhVClJE3gG8F/gP4NftQfgD+bVLrnlGRJ4XkecHc6EfV5ysIEXkSeDvgU+p\n6s9PLQBf7jx40xveoIqWOgwvave4gim5EGJVWF3nFf6pKlLJXuQ+Z7ylKkNTap3QU+Iknsry4vgY\nQ1V0g6ffskcU56x8b0NvQsDyHNVZDMto4RykvBIncbaI9BRCf1FV/8EO/1hEnrLzTwE/OW3IxxdX\ncrYUtvkc8KKq/u3i1FeAjwF/Zd//eMqAQlGGtYShOjOzEBZLDMQ4F0WmWmji85r7rI6IdbrphLO+\ncLbX+rkDEzQTKo95ddVc+1fnY2zcRcErlyX63pp5LkWxn3Lnp4mR3wU+CnxTRL5ux/6CQuQvi8if\nAt8HPnTakI8vTrFG/o3L9zH83qsdsGRKtHKTWPGNV0R1UWrM2rdVbPoO9TLd49CxzrV2Hr2LAmOy\n2HP0tFu5cNSM2rlpewug1gWWMQ93nkkIhOrVuA5xPbBseTWah1hVFYLUHOS8+2uueHIR03l0ro9A\nid7dM1NOF0rNl/FchZa5a5l290JrmlDnRMQ03Ck/ZI4WdkZCI68pQZcj85a862CNjTRE47SYFg+y\nDzWSVvfK1E2noTKO7xaQrkdtqqNx9Gg1JTopOTvPuKJbRp4PswFzRcliz6NvNAV6cWU7lx+7aEku\n+nxeEkBCk32QK14lmsvswt2ZeUev23IWUcuhCljnythtiFo4edsXhRe1yORRM2Pd2WVmnmrlWt//\nMqtRqcXvVTfYeAFFxePkkx3ranvvwx0fEbG4zmm3/poQmzyVgkioG438RkIQauGiESNrrkt56xv5\n+6IwuwA6+t2ash33VYHOG0w9ex/ozLrozW6Ofh2zzV0t8cXeeN9OXWmrHhY7DasYaYjmm06zKjol\ngpWdVWutViXlRVnYzJWeGPZSL2+zOdsQLGS4t1LgKYeFOKD2ATClXDna4eH/nNO87c6nFea3SMjx\n2yIWm2dPwcrZDdG+1i8lgs5xDPfOfFsGhMqOlcOzVi4fhl3pK9sOsXGodRxzTLy8kKCMafGP5IWZ\nif29V8q5Wl/oK0jxQGNNSC9q+Y5NvJLQ0JPF9srZDdHcqZmmCQmRZJwWfHui+zZoNf3ycuutlwwP\nRS5vN4Vzt9ueCZezC8fILhvtFUPJ4iE7JvZ7y+LUbExB1vLmnDKf+7ei1KyM758MAcLp9kh7BTkl\ntAsMg9d0lameW4117G/VbXs1hjGORBcVYxEj52elZjpkIfq7m5Z1117HfV76tWdUSpPNHh92Oztm\nkIAe1VoLWsO5+UhRonpybfbBOCt++bj2e/2uNZjIT4E7wM+aDXp9/Aqnz/M3VPVXr2rUlNgAIvK8\nqr6v6aDXwC9jnqsYaYiV2A3xWhD72ddgzOvgoc+zucx+nLGKkYZoRuxH+V3bD6jU/UsR+aGIfN0+\nf3ijcVqIkUf9XdtW0fWUqr4gIreB/wT+CPgT4BVV/euHMU4rzq7v2lbVAfB3bT8SUNWXVfUF+/0L\nwCt1HypaEfuid20/9Jt5GDiq1AX4hIh8Q0Q+f9OC/1bEvigw9siZQceVusDfAe8CnqbUqP/NTfpv\nRexrvGu7LS6q1FXVH6vqpGXzzWcp4vDaaEXsR/pd25dV6npJtOGPgf+6yThN4tnXe9d2U1xWqfsR\nEXmaIvK+B3z8JoOsHmRDrB5kQ6zEboiV2A2xErshVmI3xErshliJ3RArsRvi/wHmq1WAr7PFcQAA\nAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x116e77b38>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAm4AAAF3CAYAAAAcmcfdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu0ZGV55/HvTxoQUAS1MR0uaVRQ0TGoHTRjNEaMcoki\nBiMso0RNUAMTTWKSJs6o0cEQrxlzwUHBSzQoSkQSUECMMcmo0CA0zU0abaWhAygGUBIM8Mwfex+7\n+nTdTtt16uw+389atU7tt55666l6q+o8tfd+905VIUmSpIXvAdNOQJIkSeOxcJMkSeoICzdJkqSO\nsHCTJEnqCAs3SZKkjrBwkyRJ6ggLN0mSpI6wcJMkSeoICzdJkqSOsHCTJEnqiCXTTmBSHv7wh9fy\n5cunnYYkSdJIl1566XeraumouG22cFu+fDmrVq2adhqSJEkjJfn2OHFuKpUkSeoICzdJkqSOsHCT\nJEnqCAs3SZKkjrBwkyRJ6ggLN0mSpI6wcJMkSeoICzdJkqSOsHCTJEnqCAs3SZKkjrBwkyRJ6oiJ\nFW5J9k7yj0muSXJVkte17Q9NcmGS69u/u7ftSfK+JGuTrE7y5J6+jm3jr09y7KRyliRJWsgmucbt\nXuD3q+pxwNOA45McAKwELqqq/YCL2mWAQ4H92stxwCnQFHrAm4GnAgcBb54p9iRJkhaTJZPquKo2\nABva63cluQbYEzgCeFYb9hHgS8Afte0fraoCvppktyTL2tgLq+p2gCQXAocAZ0wq94Vi+cpzx45d\nd/LhE8xEkiQtBPOyj1uS5cCTgK8Bj2iLupnibo82bE/gxp67rW/bBrVLkiQtKhMv3JI8CDgLeH1V\n3TkstE9bDWnv91jHJVmVZNVtt90292QlSZIWsIkWbkm2pynaPl5Vf9c239JuAqX9e2vbvh7Yu+fu\newE3D2nfTFWdWlUrqmrF0qVLt94TkSRJWgAmOas0wGnANVX1np6bzgFmZoYeC3y2p/3l7ezSpwF3\ntJtSzweem2T3dlLCc9s2SZKkRWVikxOApwMvA65Mcnnb9sfAycCZSV4FfAd4cXvbecBhwFrgbuAV\nAFV1e5K3AZe0cW+dmajQNXOZbCBJkjTbJGeV/gv9908DOLhPfAHHD+jrdOD0rZedJElS93jmBEmS\npI6wcJMkSeoICzdJkqSOsHCTJEnqCAs3SZKkjrBwkyRJ6ggLN0mSpI6Y5AF4Jf2E5nrQ5nUnHz6h\nTCRJC4Fr3CRJkjrCwk2SJKkjLNwkSZI6wsJNkiSpI5ycIM2zuU44kCRphmvcJEmSOsLCTZIkqSMs\n3CRJkjrCfdwkSdKceHDw6XGNmyRJUkdYuEmSJHWEhZskSVJHWLhJkiR1hIWbJElSR1i4SZIkdYSF\nmyRJUkdYuEmSJHWEhZskSVJHeOYEaRaPCC5JWqhc4yZJktQRFm6SJEkdMbHCLcnpSW5Nsqan7ZNJ\nLm8v65Jc3rYvT/IfPbe9v+c+T0lyZZK1Sd6XJJPKWZIkaSGb5D5uHwb+EvjoTENVvWTmepJ3A3f0\nxN9QVQf26ecU4Djgq8B5wCHA5yaQ76LiflySJHXPxNa4VdWXgdv73dauNfs14IxhfSRZBuxaVV+p\nqqIpAl+4tXOVJEnqgmnt4/YM4Jaqur6nbd8kX0/yT0me0bbtCazviVnftkmSJC060zocyDFsurZt\nA7BPVX0vyVOAs5M8Hui3P1sN6jTJcTSbVdlnn322YrqSJEnTN+9r3JIsAV4EfHKmraruqarvtdcv\nBW4A9qdZw7ZXz933Am4e1HdVnVpVK6pqxdKlSyeRviRJ0tRMY1Ppc4Brq+rHm0CTLE2yXXv9kcB+\nwDeragNwV5KntfvFvRz47BRyliRJmrpJHg7kDOArwGOSrE/yqvamo9l8UsIzgdVJrgA+DbymqmYm\nNrwW+CCwlmZNnDNKJUnSojSxfdyq6pgB7b/Rp+0s4KwB8auAJ2zV5CRJkjrIMydIkiR1hIWbJElS\nR1i4SZIkdYSFmyRJUkdYuEmSJHWEhZskSVJHWLhJkiR1hIWbJElSR1i4SZIkdYSFmyRJUkdYuEmS\nJHXExM5Vqm3L8pXnzil+3cmHTygTSZIWL9e4SZIkdYSFmyRJUkdYuEmSJHWEhZskSVJHWLhJkiR1\nhIWbJElSR1i4SZIkdYSFmyRJUkdYuEmSJHWEhZskSVJHWLhJkiR1hIWbJElSR1i4SZIkdYSFmyRJ\nUkdYuEmSJHWEhZskSVJHWLhJkiR1hIWbJElSR0yscEtyepJbk6zpaXtLkpuSXN5eDuu57cQka5Nc\nl+R5Pe2HtG1rk6ycVL6SJEkL3STXuH0YOKRP+3ur6sD2ch5AkgOAo4HHt/f56yTbJdkO+CvgUOAA\n4Jg2VpIkadFZMqmOq+rLSZaPGX4E8Imqugf4VpK1wEHtbWur6psAST7Rxl69ldOVJEla8Kaxj9sJ\nSVa3m1J3b9v2BG7siVnftg1qlyRJWnTmu3A7BXgUcCCwAXh3254+sTWkva8kxyVZlWTVbbfd9pPm\nKkmStKDMa+FWVbdU1X1VdT/wATZuDl0P7N0Tuhdw85D2Qf2fWlUrqmrF0qVLt27ykiRJUzavhVuS\nZT2LRwIzM07PAY5OsmOSfYH9gIuBS4D9kuybZAeaCQznzGfOkiRJC8XEJickOQN4FvDwJOuBNwPP\nSnIgzebOdcCrAarqqiRn0kw6uBc4vqrua/s5ATgf2A44vaqumlTOkiRJC9kkZ5Ue06f5tCHxJwEn\n9Wk/DzhvK6YmSZLUSZ45QZIkqSMs3CRJkjrCwk2SJKkjLNwkSZI6wsJNkiSpIyzcJEmSOsLCTZIk\nqSMs3CRJkjrCwk2SJKkjLNwkSZI6wsJNkiSpIyzcJEmSOsLCTZIkqSMs3CRJkjrCwk2SJKkjLNwk\nSZI6wsJNkiSpIyzcJEmSOmLJtBOQJG1dy1eeO6f4dScfPqFMJG1trnGTJEnqCAs3SZKkjrBwkyRJ\n6ggLN0mSpI5wcoK2eXPdUVuSpIXKNW6SJEkdYeEmSZLUERZukiRJHeE+bloQ5rIfmgcLlSQtVq5x\nkyRJ6ggLN0mSpI6YWOGW5PQktyZZ09P2ziTXJlmd5DNJdmvblyf5jySXt5f399znKUmuTLI2yfuS\nZFI5S5IkLWSTXOP2YeCQWW0XAk+oqicC3wBO7Lnthqo6sL28pqf9FOA4YL/2MrtPSZKkRWFikxOq\n6stJls9qu6Bn8avAUcP6SLIM2LWqvtIufxR4IfC5rZrsNmAxHWR2MT1XSZJ6TXMft1eyaQG2b5Kv\nJ/mnJM9o2/YE1vfErG/bJEmSFp05r3FLsjuwd1Wt3tIHTfJG4F7g423TBmCfqvpekqcAZyd5PNBv\nf7Ya0u9xNJtV2WeffbY0PUmSpAVprDVuSb6UZNckDwUuAz6Q5D1b8oBJjgV+BXhpVRVAVd1TVd9r\nr18K3ADsT7OGba+eu+8F3Dyo76o6tapWVNWKpUuXbkl6kiRJC9a4m0ofUlV3Ai8CPlpVTwWeM9cH\nS3II8EfAC6rq7p72pUm2a68/kmYSwjeragNwV5KntbNJXw58dq6PK0mStC0Yt3Bb0k4U+DXgH8a5\nQ5IzgK8Aj0myPsmrgL8EHgxcOOuwH88EVie5Avg08Jqqur297bXAB4G1NGvinJggSZIWpXH3cXsr\ncD7wr1V1SbtW7Pphd6iqY/o0nzYg9izgrAG3rQKeMGaekiRJ26yxCreq+hTwqZ7lbwK/OqmkJEmS\ntLlxJyfsn+SimbMgJHlikv852dQkSZLUa9x93D5Ac5aD/wJoDwVy9KSSkiRJ0ubGLdx2rqqLZ7Xd\nu7WTkSRJ0mDjFm7fTfIo2oPfJjmK5qC5kiRJmifjzio9HjgVeGySm4BvAb8+sawkSZK0mXFnlX4T\neE6SXYAHVNVdk01LkiRJs41VuCV5O/COqvr3dnl34PerypmlWvSWrzx32ilIkhaJcfdxO3SmaAOo\nqu8Dh00mJUmSJPUzbuG2XZIdZxaS7ATsOCRekiRJW9m4kxM+BlyU5EM0M0tfCXxkYllJkiRpM+NO\nTnhHkiuBg4EAb6uq8yeamSRJkjYx7ho3qupzwOcmmIskSZKGGPdcpS9Kcn2SO5LcmeSuJHdOOjlJ\nkiRtNO4at3cAz6+qayaZjCRJkgYbd1bpLRZtkiRJ0zXuGrdVST4JnA3cM9NYVX83kawkSZK0mXEL\nt12Bu4Hn9rQVYOEmSZI0T8Y9HMgrJp2IJEmShht3Vun+SS5KsqZdfmISz1MqSZI0j8adnPAB4ETg\nvwCqajVw9KSSkiRJ0ubG3cdt56q6OElv270TyEfSIrV85blzil938uETykRamPyMCMZf4/bdJI+i\nmZBAkqOADRPLSpIkSZsZd43b8cCpwGOT3AR8C3jpxLKSJEnSZkYWbkkeAKyoquck2QV4QFXdNfnU\nJEmS1GvkptKquh/4w/b6Dy3aJEmSpmPcfdy+kOQNSfZO8tCZy0QzkyRJ0ibG3cftJe3f43vaCnjk\n1k1HkiRJg4y7j9uvV9W/zkM+kiRJGmDcfdz+cks6T3J6kltnzrjQtj00yYVJrm//7t62J8n7kqxN\nsjrJk3vuc2wbf32SY7ckF0mSpK4bdx+3i5L8amYdgXcMHwYOmdW2ErioqvYDLmqXAQ4F9msvxwGn\nQFPoAW8GngocBLx5ptiTJElaTMYt3F4NfAq4J8mdSe5KcueoO1XVl4HbZzUfAXykvf4R4IU97R+t\nxleB3ZIsA54HXFhVt1fV94EL2bwYlCRJ2uaNNTmhqh68FR/zEVW1oe13Q5I92vY9gRt74ta3bYPa\nJUmSFpWxCrckz+zX3q5R21r6bYatIe2bd5AcR7OZlX322WfrZSZJkrQAjHs4kD/ouf5Amn3NLgWe\nvQWPeUuSZe3atmXArW37emDvnri9gJvb9mfNav9Sv46r6lSaU3OxYsWKvsWdJElSV427qfT5vctJ\n9gb+fAsf8xzgWODk9u9ne9pPSPIJmokId7TF3fnA23smJDwXOHELH1vSPFm+8txppyBJ25xx17jN\nth543KigJGfQrC17eJL1NLNDTwbOTPIq4DvAi9vw84DDgLXA3cArAKrq9iRvAy5p495aVbMnPEiS\nJG3zxt3H7S/YuF/ZA4ADgctG3a+qjhlw08F9YotNz8zQe9vpwOnj5CpJkrStGneN26qe6/cCZ3gm\nBUmSpPk1buH2aeA/q+o+gCTbJdm5qu6eXGqSJEnqNfaZE4CdepZ3Ar6w9dORJEnSIOMWbg+sqh/M\nLLTXd55MSpIkSepn3MLth7NO+v4U4D8mk5IkSZL6GXcft9cDn0pyc7u8DHjJZFKSJElSP+MegPeS\nJI8FHkNzCqprq+q/JpqZJEmSNjHWptIkxwO7VNWaqroSeFCS355sapIkSeo17j5uv1VV/z6zUFXf\nB35rMilJkiSpn3ELtwckycxCku2AHSaTkiRJkvoZd3LCBTTnF30/zamvXgt8fmJZSZIkaTPjFm7/\ni2bT6GtoJidcAJw2qaQkSZK0uaGFW5IlwNuBVwA30hRtewPfotnMet+kE5QkSVJj1D5u7wQeCjyy\nqp5cVU8C9gUeArxr0slJkiRpo1GF26/QzCi9a6ahvf5a4LBJJiZJkqRNjSrcqqqqT+N9NJMUJEmS\nNE9GFW5XJ3n57MYkvw5cO5mUJEmS1M+oWaXHA3+X5JXApTRr2X4O2Ak4csK5SZIkqcfQwq2qbgKe\nmuTZwONpZpV+rqoumo/kJEmStNG4J5n/IvDFCeciSZKkIcY95ZUkSZKmzMJNkiSpIyzcJEmSOsLC\nTZIkqSMs3CRJkjrCwk2SJKkjLNwkSZI6wsJNkiSpIyzcJEmSOsLCTZIkqSPmvXBL8pgkl/dc7kzy\n+iRvSXJTT/thPfc5McnaJNcled585yxJkrQQjHWu0q2pqq4DDgRIsh1wE/AZ4BXAe6vqXb3xSQ4A\njqY5yf1PA19Isn9V3TeviUuSJE3ZtDeVHgzcUFXfHhJzBPCJqrqnqr4FrAUOmpfsJEmSFpBpF25H\nA2f0LJ+QZHWS05Ps3rbtCdzYE7O+bZMkSVpUpla4JdkBeAHwqbbpFOBRNJtRNwDvngntc/ca0Odx\nSVYlWXXbbbdt5YwlSZKma5pr3A4FLquqWwCq6paquq+q7gc+wMbNoeuBvXvutxdwc78Oq+rUqlpR\nVSuWLl06wdQlSZLm3zQLt2Po2UyaZFnPbUcCa9rr5wBHJ9kxyb7AfsDF85alJEnSAjHvs0oBkuwM\n/DLw6p7mdyQ5kGYz6LqZ26rqqiRnAlcD9wLHO6NUkiQtRlMp3KrqbuBhs9peNiT+JOCkSeclSZK0\nkE17VqkkSZLGZOEmSZLUERZukiRJHWHhJkmS1BFTmZwgaTKWrzx3TvHrTj58QplIC9NcPiN+PrQQ\nucZNkiSpIyzcJEmSOsLCTZIkqSMs3CRJkjrCwk2SJKkjLNwkSZI6wsJNkiSpIyzcJEmSOsLCTZIk\nqSMs3CRJkjrCwk2SJKkjLNwkSZI6wsJNkiSpI5ZMOwFJkrYFy1eeO6f4dScfPqFMtC1zjZskSVJH\nWLhJkiR1hIWbJElSR1i4SZIkdYSFmyRJUkdYuEmSJHWEhZskSVJHWLhJkiR1hAfglRYxDxgqmNv7\nwPeANF2ucZMkSeoICzdJkqSOmFrhlmRdkiuTXJ5kVdv20CQXJrm+/bt7254k70uyNsnqJE+eVt6S\nJEnTMu01br9UVQdW1Yp2eSVwUVXtB1zULgMcCuzXXo4DTpn3TCVJkqZs2oXbbEcAH2mvfwR4YU/7\nR6vxVWC3JMumkaAkSdK0TLNwK+CCJJcmOa5te0RVbQBo/+7Rtu8J3Nhz3/VtmyRJ0qIxzcOBPL2q\nbk6yB3BhkmuHxKZPW20W1BSAxwHss88+WydLSZKkBWJqa9yq6ub2763AZ4CDgFtmNoG2f29tw9cD\ne/fcfS/g5j59nlpVK6pqxdKlSyeZviRJ0rybSuGWZJckD565DjwXWAOcAxzbhh0LfLa9fg7w8nZ2\n6dOAO2Y2qUqSJC0W09pU+gjgM0lmcvjbqvp8kkuAM5O8CvgO8OI2/jzgMGAtcDfwivlPWZI0V56d\nQ9q6plK4VdU3gZ/t0/494OA+7QUcPw+pSZIkLVgL7XAgkiRJGsDCTZIkqSOmeTgQSVqU3O9L0pZy\njZskSVJHWLhJkiR1hIWbJElSR1i4SZIkdYSFmyRJUkdYuEmSJHWEhZskSVJHWLhJkiR1hIWbJElS\nR3jmBElSZ831LBRS17nGTZIkqSMs3CRJkjrCwk2SJKkj3MdNE+F+J9smx1Xaevw8aUu4xk2SJKkj\nLNwkSZI6wsJNkiSpIyzcJEmSOsLJCZL0E3Incy1Ec3lfrjv58Almoq3JNW6SJEkdYeEmSZLUERZu\nkiRJHeE+bpIWhbnuh+Y+P9L0+HkdzDVukiRJHWHhJkmS1BEWbpIkSR0x74Vbkr2T/GOSa5JcleR1\nbftbktyU5PL2cljPfU5MsjbJdUmeN985S5IkLQTTmJxwL/D7VXVZkgcDlya5sL3tvVX1rt7gJAcA\nRwOPB34a+EKS/avqvnnNWtKi4kF1pa3Hz9PWM+9r3KpqQ1Vd1l6/C7gG2HPIXY4APlFV91TVt4C1\nwEGTz1SSJGlhmeo+bkmWA08CvtY2nZBkdZLTk+zetu0J3Nhzt/UML/QkSZK2SVMr3JI8CDgLeH1V\n3QmcAjwKOBDYALx7JrTP3WtAn8clWZVk1W233TaBrCVJkqZnKoVbku1piraPV9XfAVTVLVV1X1Xd\nD3yAjZtD1wN799x9L+Dmfv1W1alVtaKqVixdunRyT0CSJGkK5n1yQpIApwHXVNV7etqXVdWGdvFI\nYE17/Rzgb5O8h2Zywn7AxfOYsiRN1WLasXsxPVdpS0xjVunTgZcBVya5vG37Y+CYJAfSbAZdB7wa\noKquSnImcDXNjNTjnVEqSZIWo3kv3KrqX+i/39p5Q+5zEnDSxJKSJEnqAM+cIEmS1BHT2FQqSeqo\nxbQP2mJ6rl03ybFad/LhE+t7S7jGTZIkqSMs3CRJkjrCwk2SJKkjLNwkSZI6wskJPwF3XJWmx8+f\ntPX4eeoO17hJkiR1hIWbJElSR1i4SZIkdYSFmyRJUkdYuEmSJHWEhZskSVJHWLhJkiR1hIWbJElS\nR1i4SZIkdYSFmyRJUkdYuEmSJHWEhZskSVJHWLhJkiR1hIWbJElSR1i4SZIkdYSFmyRJUkdYuEmS\nJHWEhZskSVJHWLhJkiR1hIWbJElSR1i4SZIkdYSFmyRJUkdYuEmSJHVEZwq3JIckuS7J2iQrp52P\nJEnSfOtE4ZZkO+CvgEOBA4Bjkhww3awkSZLmVycKN+AgYG1VfbOqfgR8AjhiyjlJkiTNq64UbnsC\nN/Ysr2/bJEmSFo0l005gTOnTVpsFJccBx7WLP0hy3USzgocD310g8Qspl0nHL6RcJh2/kHKZa/xC\nymXS8Qspl0nHL6RcJh2/kHKZdPxCymXS8XPqO38251y21M+MFVVVC/4C/Dxwfs/yicCJCyCvVQsl\nfiHl4nP1uS60XHyuPlefa3dy6fpznfSlK5tKLwH2S7Jvkh2Ao4FzppyTJEnSvOrEptKqujfJCcD5\nwHbA6VV11ZTTkiRJmledKNwAquo84Lxp5zHLqQsofiHlMun4hZTLpOMXUi5zjV9IuUw6fiHlMun4\nhZTLpOMXUi6Tjl9IuUw6ftK5TFTa7beSJEla4Lqyj5skSZKmPTuiixfgEOA6YC2wcoz404FbgTVj\nxO4N/CNwDXAV8LoR8Q8ELgauaOP/ZIzH2A74OvAPYz7fdcCVwOWMmF0D7AZ8Gri2fQ4/PyT2MW2f\nM5c7gdeP6P932+e5BjgDeOCI+Ne1sVf167vf2AAPBS4Erm//7j4k9sVt3/cDK8bo+53ta7Ma+Ayw\n24j4t7WxlwMXAD89zvsKeAPNIXMePqL/twA39YzBYcP6Bv5H+96/CnjHiL4/2dPvOuDyEfEHAl+d\neZ8BB42I/1ngK+178++BXYd9hoaM66D4vmM7JL7v2A6J7zu2g+L7je2QvgeN68C++43tkP77ju2Q\n+M3GdkjsoHHt+10H7At8rR3XTwI7jIg/gea7e/bnY1D8x9vXZQ3N+3D7EfGntW2rab4LHzTOdzXw\nF8APRvT9YeBbPa/9gSPiA5wEfKN9nX9nRPw/9/R9M3D2iPiDgcva+H8BHj0k9tlt7BrgI8CSYf+X\nBo3rkPi+4zokvu+4DojtO6bTukztgbt6aQf0BuCRwA7tYB4w4j7PBJ7MeIXbMuDJ7fUHtx+4gf23\nH8yZL4bt2zf600Y8xu8Bf8vcCrfNPggDYj8C/GZ7fQd6CpMxXtd/A35mSMyeNF9aO7XLZwK/MST+\nCe2Hcmea/Tm/AOw3amyAd9AW5MBK4M+GxD6OpgD9EpsXbv3inzvzhQX82UzfQ+J37bn+O8D7R72v\naP4hng98m03/MfXr/y3AG8Z5zwK/1L6GO7bLe4z7HgfeDbxpRP8XAIe21w8DvjQi/hLgF9vrrwTe\nNuwzNGRcB8X3Hdsh8X3Hdkh837EdFN9vbIf0PWhcB8X3HdthufQb2yH9bza2Q2IHjWvf7zqa74Gj\n2/b3A68dEf8kYDmzvteGxB/W3haaH4uj+u8d1/ew8T038LsaWAH8DRsLt0F9fxg4qs+4Dop/BfBR\n4AGzxnXk/w3gLODlI/r/BvC4tv232/z6xf53moPo79+2vxV41azH2+T/0qBxHRLfd1yHxPcd1wGx\nfcd0Whc3lc7dnE+/VVVfBm4fp/Oq2lBVl7XX76L5lTTwLBHV+EG7uH17qUHxSfYCDgc+OE4+c5Fk\nV5p/sKe1uf2oqv59zLsfDNxQVd8eEbcE2CnJEpqC7OYhsY8DvlpVd1fVvcA/AUf2BgwYmyNoClDa\nvy8cFFtV11RV3wM9D4i/oM0FmjUQe42Iv7NncRd6xnbI++q9wB8y630wx/dhv9jXAidX1T1tzK3j\n9J0kwK/RfDkOiy9g1/b6Q+gZ2wHxjwG+3F6/EPjVNnbQZ2jQuPaNHzS2Q+L7ju2Q+L5jO+I7YJOx\n3YLvi0Hxfcd2VP+zx3ZI/GZjOyR20LgO+q57Ns1aENh0XPvGV9XXq2pdn9dmUPx57W1FsyZprxHx\nd/a8Njuxcaz6xrfn4n4nzbgOzWV2zmPEvxZ4a1Xd38bdOiKeNvcHt6/r2SPi+41rv9j7gHuq6htt\n+4/HtX28Tf4vta9d33HtF9/m2Hdch8T3HdcBsX3HdFos3OZu3k6/lWQ5za+Ir42I2y7J5TSbki6s\nqmHxf07zBXH/HFIp4IIkl7ZnpxjkkcBtwIeSfD3JB5PsMuZjHE3PP/a+SVTdBLwL+A6wAbijqi4Y\ncpc1wDOTPCzJzjS/sPYeI5dHVNWG9jE3AHuMcZ8t8Urgc6OCkpyU5EbgpcCbRsS+ALipqq6YQx4n\nJFmd5PQkuw+J2x94RpKvJfmnJD83Zv/PAG6pqutHxL0eeGf7XN9Fc6DtYdYAL2ivv5g+YzvrMzRy\nXMf9zI0R33dsZ8ePGtve+FFj2yeXoeM6K37k2A54rgPHdlb80LGdFTtwXGd/19Fs/fj3noJ5k+/j\nOX43Do1Psj3wMuDzo+KTfIhmC8JjaTaBDos/AThn5r05Ri4nteP63iQ7joh/FPCSJKuSfC7JfmO+\nNkcCF/X+uBgQ/5vAeUnWt6/Nyf1iaQqj7ZOsaLs7ik0/r7P/Lz2MIePaJ36UgfF9xrVv7KAxnQYL\nt7kb6/RbP/GDJA+iWVX9+lm/zDd/8Kr7qupAml8MByV5woA+fwW4taounWM6T6+qJwOHAscneeaA\nuCU0m7NOqaonAT+k2SQ1VHtQ5RcAnxoRtzvNWpN9gZ8Gdkny64Piq+oamk1WF9J8KK8A7h0UP5+S\nvJEml4+Piq2qN1bV3m3sCUP63Bl4IyOKu1lOoflyP5CmGH73kNglwO40m0j+ADiz/QU6yjGMKMpb\nrwV+t32uv0u75naIV9K8Hy+l2dT2o94b5/IZ2prxg8a2X/ywse2Nb/sbOLZ9+h46rn3ih47tkNem\n79j2iR+X7gnFAAAHw0lEQVQ4tn1iB47r7O86mrXqs9Wg+EHfjWPG/zXw5ar651HxVfUKmu+oa4CX\nDIl/Jk1xulkhMKDvE2kKh5+j2Wfzj0bE7wj8Z1WtAD5Asy/XOM91s3EdEP+7NPtP7gV8iGYzYr9x\nejzNj/P3JrkYuIv2u3jA/6WB/2fn+n9sjPgfj+uw2EFjOhU1xe20Xbywhaffotn2PnIftzZ2e5r9\nWH5vC/J7M332bWlv+1OaXy7raH453A18bI79v2VI/z8FrOtZfgZw7hh9HgFcMEbci4HTepZfDvz1\nHHJ/O/Dbo8aGZofVZe31ZcB1o8aRPvu4DYoHjqXZ+XrnubxPaM5jN7uvH8cD/43mF+669nIvzdrJ\nnxqz/9mvw+zlzwPP6lm+AVg64rkuAW4B9hrj8e5g4yGKAtw5h9dmf+DinuXNPkMjxnXgZ67f2A6K\nHzS2w/rvN7az44eN7Rh9z36d+702A8d2yHPtO7YD+u87tmPkvsm4zrrtzTRF5nfZuG/hJt/PfeLf\n0LO8jiH77vbGt9fPpt1XbJz+27ZfZMC+xG38m2m+i2fG9X6aXXHG6ftZI/p+A81kmeU9r/sdYzzX\nhwHfY8jEr57X/oaetn2Aq8fM/bnAme31fv+XPj5oXAfEf6yn703GdVj87HEd1feoMZ2vi2vc5m6i\np99qf+WeBlxTVe8ZI35pkt3a6zsBz6H5sG6mqk6sqr2qanmb9xerauAaq7bPXdLs70C72fO5NJsy\n+vX/b8CNSR7TNh0MXD3qOTD+GpnvAE9LsnP7Oh1M8+tnWP57tH/3AV405uOcQ/MPmPbvZ8e4z1iS\nHELzK/kFVXX3GPH79Sy+gAFjC1BVV1bVHlW1vB3j9TQ7f//bkP6X9SweyYCxbZ1Ns98JSfanmXwy\n6sTLzwGurar1I+Kg2aftF9vrz6aZTTZQz9g+APifNDswD/sM9R3XLfjM9Y0fNLZD4vuObb/4QWNL\nUzj167vvuA55rn3HdsRrs9nYDonfbGyHvC6DxrXfd901NDNTj2rv3juuY383DotP8pvA84Bjqt1X\nbEj8dUke3fNaPJ+N49ov/tKq+qmecb27qh49JJdlPX2/kI3jOui5/nhc29f/G2O8Ni+mKUz+c8Rz\nvQZ4SPt+Afhl4Johuc+M6440n5P3w8D/Sy8dNK5z/T82KL7fuPaLBV42aEynZppVY1cvNPtKfYPm\nV+kbx4g/g2ZzxX/RfOG+akjsL9CsEp45TMCPp/IPiH8izbTl1TQf4jeN+RyexRi/Gmj2W7uCjVO7\nhz5fmk0zq9p8zqY95MKQ+J1pft09ZMy8/4TmQ7OGZhbWjiPi/5mmeLwCOHicsaH5xXkRTeFwEfDQ\nIbFHttfvofknev6IvtfS7CM5M7bvHxF/VvtcV9McGmHPcd9XbP7Ls1//f0Nz2IXVNIXNsiGxOwAf\na/O5DHj2qFxoZpm9ZszX/ReAS9ux+hrwlBHxr6P5HH6DZt+amTU6fT9DQ8Z1UHzfsR0S33dsh8T3\nHdtB8f3Gdkjfg8Z1UHzfsR2WS7+xHdL/ZmM7JHbQuPb9rqP5jrq4ff0/xcaZsYPif6cd13tpCsoP\njoi/l+a7fibHNw2Kp9n96F/b134NzZqjXYf1P+v1+8GIXL7Y0/fH2Dh7c1D8bsC57X2+AvzsqFxo\n1jAfMs7/GZrPyJXtuH6pHYtBse+kKfauY8Bhn+j5vzRoXIfE9x3XIfF9x3V27LAxndbFMydIkiR1\nhJtKJUmSOsLCTZIkqSMs3CRJkjrCwk2SJKkjLNwkSZI6wsJNUqelOaXZ5e3l35Lc1LO8w5h9PCvJ\nVWlO1bZDkve0yycnOT7JS3/CHD/Uc3xDSdpiHg5E0jYjyVtojoX1rlntofm+63tuwyQfBL5QVZ9o\nY79Pc5y3uZzTV5ImzjVukrZJSR6d5OokH6c5ePSyJKemOeH2VUne1Ma9huasGn+a5KM0Byx9MHBZ\nkqOS/O8kr29j90/yxSRXJLkszcnRex/zwWlO5n1FkjVJjmrb/yXJge31Vyf5RpoTun8wyZ+37R9L\n8n+S/L8k30xy5Ly8UJI6Zcm0E5CkCXoscGxVXQKQZGVV3Z5kCfCPST5dVe9P8gvAp6vq7Pa271Zz\nkmxmCq7WGcBbqurvkzyQzX/8HkZzvt5D2/s+pPfGJHsDK2lOV/VDmqPNX9wTsgfwdJpzk54JfOYn\nfwkkbUtc4yZpW3bDTNHWOibJZTSndXoccMC4HSXZneYUYn8PUFX/WZufb3Y1cEi7b9zTq+qOWbc/\nleZcid+vqh8Bn551+9nVWA3sOW5ukhYPCzdJ27IfzlxJc1L319Gch/OJwOeBB86xv6E7BVfVNcAK\nmk2zJyf541khGdH/PXOIlbQIWbhJWix2Be4C7kyyDHjeXO5cVd8Hvpvk+QBJHphk596YJHvSTI74\nG+DdNJtEe30N+KUkuyXZnmbfOkkam/u4SVosLgOuBq4Fvg386xb08VLg/yY5CfgR8KttXzN+lmZN\n2/3t7a/pvXNVfSfJO4FLgNuB64DZm1MlaSAPByJJ8yjJg6rqB+0at88Cp8zsNydJo7ipVJLm19uS\nfJ1mIsN1wD9MOR9JHeIaN0mSpI5wjZskSVJHWLhJkiR1hIWbJElSR1i4SZIkdYSFmyRJUkdYuEmS\nJHXE/wcUFYLkpMRr2AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10dfd7240>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "### Data exploration visualization code goes here.\n",
    "### Feel free to use as many code cells as needed.\n",
    "import matplotlib.pyplot as plt\n",
    "import random\n",
    "# Visualizations will be shown in the notebook.\n",
    "%matplotlib inline\n",
    "\n",
    "# Show random image\n",
    "index = random.randint(0, len(X_train))\n",
    "image = X_train[index].squeeze()\n",
    "\n",
    "plt.figure(figsize=(1, 1))\n",
    "plt.imshow(image, cmap=\"gray\")\n",
    "print(y_train[index])\n",
    "\n",
    "# Save histogram\n",
    "plt.figure(figsize=(10, 6))\n",
    "plt.hist(y_train, bins=np.arange(0,44))\n",
    "plt.xlabel('Traffic sign')\n",
    "plt.ylabel('Occurrences')\n",
    "plt.xticks(np.arange(0,44))\n",
    "\n",
    "plt.savefig('writeup/histogram.png')\n",
    "\n",
    "# Save a small picture serie to the filesystem\n",
    "from PIL import Image\n",
    "for i in np.arange(23205,23210):\n",
    "    im = Image.fromarray(X_train[i])\n",
    "    im.save(\"writeup/yield/\" + str(i) + \".png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "----\n",
    "\n",
    "## Step 2: Design and Test a Model Architecture\n",
    "\n",
    "Design and implement a deep learning model that learns to recognize traffic signs. Train and test your model on the [German Traffic Sign Dataset](http://benchmark.ini.rub.de/?section=gtsrb&subsection=dataset).\n",
    "\n",
    "There are various aspects to consider when thinking about this problem:\n",
    "\n",
    "- Neural network architecture\n",
    "- Play around preprocessing techniques (normalization, rgb to grayscale, etc)\n",
    "- Number of examples per label (some have more than others).\n",
    "- Generate fake data.\n",
    "\n",
    "Here is an example of a [published baseline model on this problem](http://yann.lecun.com/exdb/publis/pdf/sermanet-ijcnn-11.pdf). It's not required to be familiar with the approach used in the paper but, it's good practice to try to read papers like these.\n",
    "\n",
    "**NOTE:** The LeNet-5 implementation shown in the [classroom](https://classroom.udacity.com/nanodegrees/nd013/parts/fbf77062-5703-404e-b60c-95b78b2f3f9e/modules/6df7ae49-c61c-4bb2-a23e-6527e69209ec/lessons/601ae704-1035-4287-8b11-e2c2716217ad/concepts/d4aca031-508f-4e0b-b493-e7b706120f81) at the end of the CNN lesson is a solid starting point. You'll have to change the number of classes and possibly the preprocessing, but aside from that it's plug and play!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Pre-process the Data Set (normalization, grayscale, etc.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Use the code cell (or multiple code cells, if necessary) to implement the first step of your project."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Definitions of the preprocessing of the data\n",
    "from sklearn.utils import shuffle\n",
    "import numpy as np\n",
    "\n",
    "def normalize(channel, min, max):\n",
    "    return (channel - min) / (max - min)\n",
    "\n",
    "\n",
    "def preprocess_image(image):\n",
    "    image = cv2.cvtColor(image, cv2.COLOR_RGB2YUV)\n",
    "    Y = image[:, :, 0]\n",
    "    Y = normalize(Y, np.min(Y), np.max(Y))\n",
    "    U = normalize(image[:, :, 1], 0, 255)\n",
    "    V = normalize(image[:, :, 2], 0, 255)\n",
    "    image = np.stack((Y, U, V), axis=2)\n",
    "    return image\n",
    "\n",
    "\n",
    "def preprocess(X_train, y_train):\n",
    "    X = []\n",
    "    for image in X_train:\n",
    "        image = preprocess_image(image)\n",
    "        X.append(image)\n",
    "    return X, y_train\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x1648de46898>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAD41JREFUeJztnGuMXddVx3/rnPuaO3feL9sTv2s7ocROqFWoaGggrVLx\ngVIkHgWhIpCCEJVA4kMrPvGxHwCJT0ipiAQSCKG2EhGqVKqKqATS1okbQhLHsRM7tscz4/GMPXPv\nzH2es/mw1j73emxn7syYM6P4Lsk+d85jP9b5n7XX+u+1tzjn6Ek6Eux0Ax4m6Sk7RekpO0XpKTtF\n6Sk7RekpO0XpKTtF2ZayReTzInJeRC6KyNceVKM+qiJbDWpEJATeBT4HXAPOAF9yzr394Jr30ZLM\nNp79JHDROfc+gIj8M/AF4L7KLg2Pu7E9h+4459b9EGn/fpCx7b3Kkm6e66IRS/OXWV2+uWFx21H2\nNHC14+9rwM+uv0lEngOeAxidOsBXv3EGcZJ0Praj71Qo7T+cXYwcHQZP++SSvxziX5S/5lxSrhdn\nqogdBCIdJWkZvjFuncpaMQTceb+z++MYnAh/88en13f7nrIdZd/rTd6FA+fc88DzAAdOnHZxLAQd\nI4VEdhR/P8SJQq1TCLFpL7BzEug9sYPAfwn2kmLXLrDdIHd3o2N7qb5uHGJ/RMmXJsmL8y/1jq9w\nE7IdZV8D9nf8/QhwfaOHHBBHLul1omQ7EcVtRDuvMAfiFWnl+I7HEdSb+sYazaYe44hA9I0GoXYx\nlw0ByIYBYbCubtfx4u5CcccL8l9ch5kTuQtf95XteCNngGMiclhEcsBvAS9uo7yPvGwZ2c65loh8\nBfguEAIvOOfe2ui52Dm1rfGdn3Viw+M2ljyC4pgE0s16DYDFhVkAZq+c59bcJQCqyzf0/kYdCbMA\nhMURAPpH9gAwtO9jjO89BMDYyDAAhayqIaDDpEjbhAVJQ/w1rwP7t1GnTbZjRnDOfQf4znbKeJhk\nW8rerDggjs1nWAdpSQzh3b5fVG8xd+UcABfPfh+AGxdeAaC5OI+rrgAQt9bsgQgnaqMlyAGwlC8C\nMDM4SX7fowBMnvg0APsfU29ianIPfVm1rJ12OkGu3PlD7jXofoj0wvUUJVVkA+BtXHyne+c6fGQP\npUZlGYC3f/xvvPffOvbWZ88D0FrRa3GzivcL5S5Hr21fxeNq6RqNuXe1rEuv6akLnwTgkdO/wpHH\nngRgsFQADI3roNv2oDYnqSvb4ZLgQ+XOnkgM9eUlAH7y0gsAXHrlRZrz1wCIq6tWUHz30z5YCdo1\nuMSPtGOrBqt1ABpmflor8/r34hWqK78JwIlP/BIAYyMDiauY9KEziErJ9evJJiVdZDsNIERc22zY\nJQ+QVr3CWy//AwDvv/xNAJo3ZqDesDI0gPEBTyZfIDcwAEC2VNJz2RyxwS9q6nOtakWPlTKtmiI7\nbum1yFzGtYs/5FpTB1kx0/TYzz3L2FCfnvMDo2u7hZuxJT1kpyjpIlsgEEV3Qu36kDxqAXD1je9y\n4ZVvAdC4oXZa6s32/VkNVoqTewEoTU4T9CmilcUCF7fdtVziP2r5cXWF2uIcANWbNwFo1RrJteoH\nbwAwE/4TAPn+YXKfeErrKmR8N5KmB+uZqw+R9L0R0caub2JtaQaACy9/k9p1IxPr+rk7B5LTT3nw\nwFEAChMaEUoml0R7SYgnkBBPiWb0JUkpT19hEIBsvx4rs1pfs7yKq5UBEqVfP/NtRvce0DoPH9Hn\nwg5zEt6jM/eRnhlJUVJFtmAUqbQ94iDWAe/6OxoZLlx6G1et6kUfSGZyDB44DkBhaq896P3yOEFW\nHGlZrVgIw7vPgVGnzniT4X0A9BszuHr1PRpl5V7i1dsArLz3Y2befAmAiT1ad77UZ23QL6cXQe5C\nSd1mBwJxR2DTXF0A4Mqb/wlAY2Upcbu8e1gY20NhbAyA5bJGjjXjruMoImpakGLuoeT7mJqcACBs\n6VcyP6eBy3KlgoTKl5RG9J7x4XEAilN1ovr7AEQNHVBbt2ZZOv9f+uypzwAwMqDjRhD0XL9dKzsQ\nrhtvbJBo3dLJnVszyle4RoMELtk8AMWJaeJIETp36U0AymuK5kwohHZ/1FI0UhhmdEg9jVs31NNY\nmFf+e7A0SBjpF7Aya/yKU6ROj0yQG1wEoLaolIFrVqlcUz5m/rIyj3sPHQagLwiSsaMb2QHXT4ic\nQywSrC++A0BU1ghPSSWLDotDAISlIpiyM/ZcX79Gjfunp8mg565duaxlBhnEFFq+rcrrG5oC4MCR\nYwSRRomzly8AsLxk3MjocQqjk/p7WV9E1IxpGndSnTdA1J/RY7bY9eAIPTOSquwM6+fA2fR1dcXY\nvFqHu2eRSFjsByAIA5w1tTSsaC/l1UyURiZprN0CoG48SH5sKgk8/JSWFDTKDHJ5Mqhf2Nev5S8t\nKdKjGDL99jXldBCNm1VcQ93BqKyRZ2RtlYHivSO0+0gP2SlK6sgOMbbMQuyWMW+Y3TU86v8ZRRci\nSEbJ/Mn9J/RUYFFLINQrxuhZtD42MEyYCa0M7WLDBs/IQcaezWat/FjtcyOKoWCTBjkdnN3qGmK8\nyuqiUgprNqkxMDG6KZudvhkRCMJkcoWc11lHLkdCI3VMt4i/bh6KJ6Zc1KJcNs8h1GulYpHQzEjW\nlF4zXzyKXUJYZUzZgU0sRK0miCpbMp50kmS2oGUTFx4g4tQZ6UWQu1B2gGI1pNgg2G8uXMaoU9fx\nf2QDU+xiAo+LuPMOiJp1ViuKuFxR80By+RyBKFqzGS039jRqHCXzkaHV6f30ZrOJw3iPjshwfWZi\nMqvu7L4uo8geslOU9IMaMGZE33N2YBqAQtGmnoIgQW9UU5csbjQJC/l7llWrVqjVlScZHFN3MJMJ\nkvndrKGXirprURThnHY7tAHYjxetZhNnLKGfTgNwPm8wp/Y8ky1YP7jjuJFsiGwR2S8i/yEi50Tk\nLRH5Ezs/KiLfE5ELdhzpss6HVrpBdgv4M+fcWREZAF4Tke8Bvwd83zn3dVvi8TXgqxsV5nA2uJuT\nN6AzLsNjam/nr80SNwxdNbXFzZUKmfx6ZCt0a+UVIgtSimb/A9opxdk+fU6W1F0rlysUzI7X1hS9\njUjLKgXgrM7Ic+qA+EzYQWUJ+/qLdkGUz+7SHdlQ2c65WWDWfpdF5ByaCP8F4Gm77e+Bl+hC2WDU\npEl+5CAAjxzRyYGr712ifEs7LC1112qL8+QG1EQENg8Yt9R0rK4sE9qUWdEizmTuDegfVGp2aEk5\nkqWZi5QX9P5GVafAQjMLo6VBGktKXMXNVlJWUND7h/d9DIBsv815biJnBDY5QIrIIeBJ4EfAlL0I\n/0Im7/PMcyLyqoi8Wrm9sKnGfdSk6wFSRErAt4A/dc6tSJffTufKg4OPnnZBIEjcgYiiEvcTxzS5\nceKN11gt6yfsmvp5N1cWWF1QZPfb1FTsUd+s0j+oZeQt4d0BgbMBuE/N0579NlF8c47VNR14S8Na\n5uiE5vTnXY3bi8rweRQ6CSiMqqmbPPBxLcOSNEORu5aFfJh0hWwRyaKK/kfn3Lft9LyI7LXre4Eb\n3Vf7cMqGyBaF8N8B55xzf91x6UXgy8DX7fiv3VQolu2XLDAK1F72HfgUAI8c+wE3F2yy1QY1mnVq\n85f1efPTCiNqNx85dIxMzgKjJImyvYBJxLuYiv7x/iHGPRNoHImr65e0cuk8sWVL+eckV2DsqCJ6\nz+Gf1rIMogEQbQLZ3ZiRnwd+F/hfEXndzv05quR/EZE/AK4Av959tQ+ndOONvMz9uZZnNl+l2jnL\nLEgWDGWGNQFm+olnuTGr7Fq1fhGAZmUN6uqhrF3XCdm4ocFQcWKS0Bi6xAeLXeIpOG8pvecQZAl9\nSrJNHpdnrJ7bK+3lfZa62rf3APtOaTeHJid9o+3okmV+3Uj6Kw+c+dme2DM3UPLmXp14huML2vmK\nUaezV67TWlWeBEt8rM3rOppWZYmczZJ7l0xCISyoGyhG3bYsGm1Wa8iaKrlq02GxrdPpbE9uVF3G\nyZOf5vDJp7X8zJ3qimPZ1PK8HjeSouwINxIIuNj4ZT0Q9hmP3D/FvtO/DUC1okFHK/4Bi9fV2Wmu\n+nUzGtS0Vm7SquiAinEdYS5PYFFiwiBa3mDUrBLYsz7117uxLgjJjSuiR37qCQA+/qlfY2BwFIC6\nMYcuYQ0zZMLuod1DdoqSfq5fIEpn1y2aXFYnLShq2O4kIBjTAOToU7+v9wQZ3jmrWUk35xTh1RW1\n567RRGJFKg1Lbm+sJsuhvXSu3UmyXs29C/P6VeXH9jBwTGmDk5/RuqcPP4mz+bZGwyZ6fVJ80E8U\nZtJZB7kV0dluIRCruqGKChNvQYjN/81M6RK6Rz/7RwyPa5R37sy/A3D1qs0HrlSS2W7nlR1FybRZ\nsmDU+81hmHgvmZKlDg+ZD37gFI9/7ncAmD6iC5nCMIOz5M/A6NrIm5EwICOuNy22GyV1ZPuVNLWG\nZRxlFHl9YRL+JQgQy/uV8YMc+oUvAzB+9GcA+OB1TTG++PZZ5uc1n2PN06KNJs7zL34ZnX0tEmbI\nGRU7skd9+yOPP63HU89SGtPMKc9MioDPP/bsYsbvEBEGBD1k707ZgQFSJ30b5sE1Shp8jPr53I77\nPe3tHLicBix541COTp4EYPrJD7g1o5v3zF38IQAL7/6ImTldLzO3rGhfjbWrx09/kad+8fMAjB18\nDIDioKLZBWGSMnFHu71r6D++sH2te1z3kJ2qpJ7K4EIhjlsEtxTDeUtEj8xXizuA0vIWPtKVv9Cx\nzZEYH5KZYuKkzqBM7FOue6mvjMT/A8Bty46tROpJDB1/isnHdfVuwdN3SXDTuTuSh7G7ewMX7zht\nckF1utyIg7gJrrXGmlPXbSijivKzUK5jkYrryBHxWxP5TzpuaXS5WL7KnvFTAORKGukVSgP09elg\nljM3L7DtkarVVWxewSdGtc2DSDsnpOM9tDdjsIRP//y6HXk2kp4ZSVG2vK/flioTWQBWgZupVbp1\nGaf7dh50zk1sdFOqygYQkVedc93tzbaD8v/Rzp4ZSVF6yk5RdkLZz+9AnVuRB97O1G32wyw9M5Ki\npKbs3bzX9odk6v6FiMyIyOv275e3VU8aZmS377VtGV17OzN1gV8FfgOoOOf+8kHUkxayk722nXMN\nwO+1vSvEOTfrnDtrv8uAz9R9oJKWsu+11/YD78yDkHWZugBfEZE3ROSF7Sb8p6Xse1E1u84NWp+p\nC/wtcBR4As1R/6vtlJ+Wsre013aacq9MXefcvHMucroT4zdQc7hlSUvZu3qv7ftl6vqUaJMvAm9u\np55U+Oyt7rWdotwvU/dLIvIEavIuA3+4nUp6EWSK0osgU5SeslOUnrJTlJ6yU5SeslOUnrJTlJ6y\nU5SeslOU/wNUMf5727sVuAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1648da184a8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADBBJREFUeJztnF9sVFUexz+/mWlttcW2toVSKtaK8W8DSljCxoSIKMEE\ndxN3gyZkSda4Lya7yT6s2ad99GF3Xzdhs8Z92LhZYSMYTYiSxUhIEEsMRRQKiBVsgdbRtsBM58/Z\nhzu/M6d3Zuj8qXcae79JM9N7z5xz7+9+z+/f+d0jxhhCBINIvS9gKSEUdoAIhR0gQmEHiFDYASIU\ndoAIhR0gahK2iGwTkTMick5EXl2oi/qxQqoNakQkCpwFtgKXgOPAC8aY0wt3eT8uxGr47QbgnDHm\nAoCI/Bt4Digp7M7OTrN69eqyB1gM0W051zA6Osrk5KTM164WYfcCXzv/XwJ+4m8kIi8DLwP09fVx\n9OjROef9NyMi9lg6nbbHI5HSGi+bzc5po/8vBG7Vl17n5s2by+qrFp1d7EkW0MAYs8cYs94Ys76r\nq6tAuNlsds4NlWKStvO393/3n68ExfqKRCL2z73GamZdLcy+BPQ5/68Cvinnh5lMxn73M9Zls4tS\nAsxkMiSTSQBmZ2dtH9FoFIBYzLvFhoYG+7+OWWy2lMNk93+RebWHRS3MPg6sEZF+EWkEdgIHaujv\nR4+qmW2MSYvIK8BBIAq8boz5bJ7fkE6niUQilkF+JunUdZHJZCyrbt68CcDXX3vm4vTp05w/fx6A\nq1evApBIJCyTly1bBsDKlSsBGBgY4L777gOgu7sbgNtuu82O5WdqNpu1YxdTHZWok1rUCMaY94D3\nauljKaEmYVeL+QyYnlfWJBIJTp/2PMqDBw8C8PHHHwMwNjbGzMwMgNXdmUzGMlR1dnNzMwCdnZ2s\nWbMGgE2bNgGwceNGwPOWlOX6+2w2a7/7WV+pkQzD9QBRF2ZDaXa7btV3330HwIEDB3j77bcBGBkZ\nASAejwMe62+lS/2sHBsb48KFCwCcPHkSyM+SZ5991rL8zjvvrOX2iiJQYRtjihpAf5uJiQkA3njj\nDQD279/P6OgoANevX7ftSkHdPihUSbOzs6RSKQCrfq5duwbApUuX7Pdt27YB0NXVVZF7dyuEaiRA\nLDoDOTMzw5tvvgnA3r17AY9xiURizm+Vbc3NzbS1tQH5qd/Y2GiZrEZTWTw1NWVnhzJcZ9LQ0JB1\nLXWcHTt20N7ePmfManM2IbMDRKDMFhEb0PgNmLLs0KFD7Nu3D8gHLslk0rZvbGwEYNWqVYDnrrW2\ntgJ5Xe2mAxSaBpienmZsbAzAfiqbp6enrYv51ltvAdDW1sbTTz8N5N1H934CC2qqhRtBKvTG9+7d\ny5dffglgVQdAU1MTAPfffz+QF7ZGii5cg6bG2M2VtLS0ANiH9NVXXwHw/fffW3WjQn/nnXe49957\nAXjooYdsH+Cpk2g0WrYBDdVIgKibn62M0yl/5MgRwPN91YApGhoaLKPvvvtuIM9UN/OmqsIYY/vX\nYzqOO+1XrFgB5Jk6MjLC1NQU4LEc4MSJExw+fBgAXfhQQ+y6mGXdc0WtQ9SEujFboZHghx9+CMDk\n5GSB0enp6aGnpweAb7/9Fsjr83Q6bY2rGrrm5mb6+rxUu7p+GhTF43Gr55XZXV1dgMfcM2fOAPnc\n+Pj4OMeOHQNgy5YtANYVrBQhswNE3Zk9Pj4OwBdffAF4jFJmqweyatUqyzTNZ6jX4OpNZXhrayud\nnZ1A3tPQz/b2dtvu3LlzQF6P33XXXZa1bm5cr214eBiABx98EJi76lMOAhe2un3q+qkQ1DC56VF1\nzVpbW62AVDBqpAYGBmzfqgJisZg1jCo0XSgYHBy0D+7UqVNA3u3s7u62qkXVVSqVYnJyEsAmsFSF\n6cJE2fdeUesQNSFwZvuXw1SN3LhxA5jryimzY7HYnKkO2MBkxYoVNhWrjOvt7S3IY9xxxx2Ap5p0\ngUBnh2b6MpmMzbOoCkulUrZfzaGoa6ptw6BmEaJuOluh+tPNlShT1EUTEZsTcY0TeAbS1ffgMU7Z\nq32442j/yl7V76lUys4YPTc9PW3PX758Gci7qz09PYvbQIInKBWMehPzTUW9KRWCWzWlBkwF29bW\nZvv1CzuTydhj+gD14afT6YK1S/e8ekBqrCtFqEYCRF1SrPod8kbKrd3w14i4S2n+9GkikbAGUvtq\namoqUEUaSabT6TnVUf6+KsF8S3x+hMwOEHWPIDXYcBcAVEeqO5hMJgtYqJiZmbGMdHPcatR0xkxP\nTwNzawlV/ys7U6mUPa8zwT2v7d1ZWAnmZbaI9InI/0TkcxH5TER+mzveISLvi8hI7rO67MwSQjnM\nTgO/N8acEJFWYEhE3gd2A4eMMa/lXvF4FfhDOYO6uk5zGMuXLwfg/Pnz1nPQ4CEejxcsSaleVzcM\n8rMD8vpYgxkN2+PxuGWmehfK5mg0ameTnnP70mvVMD0SiVRU5jCvsI0xY8BY7vu0iHyOVwj/HLA5\n1+yfwGHKFLZrVLTgcXBwEPDyFRqpqYs1Pj5uE0RuZAdeDuP2228H8hEd5A2wpk+vXLkCeAsE6i+r\nf659tre32zyJ9i8itn8tyHRzIpWsQVZkIEXkHmAdcAxYnnsQ+kC6S/zmZRH5REQ+0bB4qaJsAyki\nLcA+4HfGmKlyp48xZg+wB+Dxxx83/ghS2bh+/XoAPvroI+vK6fSenJzkm2+8OntdFlMDduPGDZsv\nKbb4q6rlgQceALwaFDWW7qIBeG6lMtu5b5sJfOSRRwAs03+QZTERacAT9L+MMf/NHb4iIj258z3A\n1YpGXoKYl9niUfgfwOfGmL86pw4AvwJey33uL3dQd1YoO1Rnr1u3zrJY1U4ymeTixYtz2nd0dAAe\n25RpLvx1Kdp+2bJlBefUIA8PD1sDqeM0Njbaa9NP1+YsqIEEfgrsAoZF5NPcsT/iCfk/IvJrYBT4\nRdmjLlGU440cofibYQBbar0AZYl6JU899ZQt0hkaGgI8r0FDd13K0gBm5cqV1ptwyxsUfua5wZG6\njVqGrAkt97r6+/vtQm9vb++cc276oRwEHkH6X2vTi1VVsGnTJrtUpoby7Nmz1u/VaFHVSjweL4hC\nY7FYgWpx/Wc1kLpwoQ8S8g9M/f4nnniCJ598EiiMHI0xRUvdSiHMjQSIuuRGRMQGDereadK+o6OD\nHTt2APlpnk6nbd2HP8cxMTFhZ4C6fk1NTQVuoLI3kUjMqZzS6wGP1croDRs2APD8889b11IjWp2N\nsVisqLtZCiGzA0TgzFYWKUP1U5kdjUZtNdOLL75oj33wwQdAvoxYWZ9MJi1T9dPVwYpiRtMtqAdv\nmevRRx8FYNeuXQCsXbvW9qt6X9Ha2loyG1kMdUux+mup9cYjkYg9p7mIl156yUZ57777LuAZTfBy\nI66KAE/oxTYgAG/qq/ei0av64IODg+zevRuAxx57zF6nRrz6O/cegnqdOkSFqBuz1djcauFXF2T7\n+vrYuXMn4E1r8N5QADh69GiB8UwmkwXv3rjF8Jq16+/vB7Cu3TPPPGMLOF3/WVWFf4G4EhUCIbMD\nRV3qRkTE6lnXMJZCNpu1QYoyW/X59u3bbeGjvjx6/PhxGxhpVOi+/aWz5OGHHwby2T//viIKv16u\nNNunCJkdIOr2tpjqV9WR6l4Vs+7uFhj6qfqypaWFrVu3Ankd7C74qouozN64caPdZsivg0tl80pt\nEFDp5i6Bv06dTqe5efOmFbZGYK6w3R0R/NBz6vOOj4/bqE+Xztrb2+cUUrrtp6amCmpXXOOsgtRj\n7jX4f1dqt4ZSCNVIgKh6X7+qBhO5BlwHJgIbtHp0Uv51rjbGdM3XKFBhA4jIJ8aY9YEOWgV+iOsM\n1UiACIUdIOoh7D11GLMaLPh1Bq6zlzJCNRIgAhP2Yt5r+xaVun8Skcsi8mnub3tN4wShRhb7Xtu5\niq4et1IX+BnwS2DGGPPnhRgnKGbbvbaNMbOA7rW9KGCMGTPGnMh9nwa0UndBEZSwi+21veA3sxDw\nVeoCvCIiJ0Xk9VoL/oMSdll7bdcb/kpd4G/AALAWr0b9L7X0H5Swq95rOygUq9Q1xlwxxmSMMVng\n73jqsGoEJexFvdd2qUpdLYnO4efAqVrGCSSfXc1e2wGjVKXuCyKyFk/lXQR+U8sgYQQZIMIIMkCE\nwg4QobADRCjsABEKO0CEwg4QobADRCjsAPF/42AFEAgUsFcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1648dc9be48>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAACpVJREFUeJztnF9oVNkdxz+/mEzQRDGmZhQT26QUI5iwdaRFCqZQheKL\nFmzpIqWFQvqy0EIfuvSpjyu0vigULF3pQ6EUWtJ9EMqmtEohyOqyJG6N7hpLMq1EK5E6iWbGePow\n8ztz5sydySQznknN/UK4M3PPuefkd7739+/87hVjDDHCoKXZE9hMiIUdELGwAyIWdkDEwg6IWNgB\nEQs7IOoStoh8XUTuiMinIvJ2oyb1ukLWG9SIyBbgLnACSAMfAG8aY/7RuOm9Xmito++XgE+NMTMA\nIvI74BRQUdgtLS2mpaUFEcFfZP0uIohI2blqpPDbR0H7G2PK2lfr//LlS3tej+5cdG7GmFUnUY+w\n9wFzzvc08GW/kYiMAqMALS0t7Nixg7a2NnK5XEm7bDYLQCKRoK2treRcLpez56OQSCRWnaz2z2az\nZe2r9c9kMva8Ht25ZLNZXrx4ser4UJ+wo1ayjH7GmEvAJYDW1lYDlAjaFyzA4uIiUPrPuYtRCyot\njtu/2rV8gVZCIpFgZWWlpjnVI+w00Od87wX+Xa2DMaaMWZUY7n6OYmOlPv53vb4uatR13Pa1LuZ6\nUI838gHwBRHpF5EE8G3gvcZM6/XEupltjHkhIm8Bfwa2AO8aYz6upe9qTHXb+dB+ytSenh727NkD\nQDKZBKCjo8P2nZ+fB+Dhw4cAzMzMsLCwUHJ9XydXGtPX2Xqs1aOrR41gjLkCXKnnGpsJdQm7Hvis\njWKVy6ienh4Ajhw5AkAqlQJgcHDQnuvs7AQo8XZ0HGXzzMwMExMTAFy9ehWAO3fu2LbVPBWf0Xqu\nVgMZh+sBse4Icj3YsmWL2bp1a+S5KI+gq6sLgKGhIU6ePAnA0aNHAejryztC27Zts/3a29vLrru8\nvFzyfWlpqYTlAGNjY/aour2a9+Lj2bNnrKysvNKg5pUgkUhYQ3f8+HEATp8+zdDQEAA7d+4EogWr\n8AXstm9vb7eLqOqnt7cXyC/qxYsXgaJq0Tk1ArEaCYigzI4Kanxj09XVZVXGqVOnABgeHrZsVCh7\nc7mcVQt6jIpQ1Xj29PTYz3rs7++3Y+s8Lly4AMCtW7cqGs1qkWUUYmYHRFBmi8iqgcPIyIhl9vDw\nMEAJqzOZDFA0bul02gYu7l1SKWBJJpNWR+tRGZ5MJq2dUJw7d86O1dHRUTKOXvf58+c1/f9NN5Aq\nhAMHDgB51aHG0BWyqojJyUkgL2QoCt9FpfyKttfF0WsODg7a8dRojoyM2HHOnz9fcg13AbPZbM0R\nZKxGAqIpasRlnt6a6j8PDAxY906xsLBgGa23dDXj5BpI/9Z3DapeU1O6qVTK3k379u0D8u7njRs3\nABgfHwdKDeRa1EjM7IBous5WHXn48GEgb6T8gGVubs7qaJehCj/aU6ZCUe9HbUQow6enp23bQ4cO\nAcUgaHBwkBMnTgBw8+bNkrHXGuzEzA6IpjDbZcTBgwcB2L9/P1DUsVBkXjqdtr9rKK9eSCKRiAw6\ntK8fDLnjP378GIC7d+8CeYbrnebmXtQ70t9U16vLWCuaEkFC8R/WVKn6vO6epApsfn7eJpxU2Coo\n91oudHH03NxccW9aF0AX+MmTJ3YcVVcq9Pb2ditknatrpNeiSmI1EhBNUyPKYGWq3pKucVRmA7Zc\nQI2ffl9aWrJ5ku7ubttv9+7dQJHhei2XicpwHXtxcbEsv+JmCQcGBkr6zc/Px0HNRkXTcyOtrfkp\nuLpameqG2pXKFbLZrNWpbhtlvm8jstmsZbuyV+fgMlvvoM7OTmsvNNDxXc3/m9xItU2AKPi5EHfx\nnj59WrGfCiiTyZQt3Pbt220bP5cS1c6/bi3lbxCrkaBoOrOjoGx3fW6Fn7h39yDVhYtq7x7XUuvn\nzkehd0lU6Vw1xMwOiKYzWwMENVbLy8uWSe72VVTe2oebL/F1rt4BCwsLFes/3M/ub2qw1Sa44+Ry\nuca5fiLSJyJ/FZHbIvKxiPyw8PsuEXlfRD4pHMvj4hglqIXZL4AfG2M+FJHtwE0ReR/4HvAXY8w7\nhUc83gZ+stYJ+PlpVwdr8NDV1WWZ7bOxu7u7jPWuLlUXTj0JN8T22d/Z2WnHdHPqyuRHjx6VfPcr\ncFfDqsI2xjwAHhQ+PxWR2+QL4U8BXy00+w3wN9YhbE1vqn/rGkUVfG9vrz0fJVgVqArPdeG0vUaq\n/f39ZcXrqh46OjpsjiYqklViuJHtWly/NelsEfkc8EXgOpAsLATGmAci0lOhT8mTB5sZNQtbRDqB\nPwA/Msb8t9bVjHryoK2tzd7KuvmqiXllFhTZ1dfXZ8vCfHXjPgLi3hX+RoKyUcuL3XN67Ovrs9Go\nqqJcLmczgffu3Sv537SAs6G5ERFpIy/o3xpj/lj4eV5E9hbO7wUe1jTiJsaqzJY8hX8N3DbGnHdO\nvQd8F3incPxTrYPmcrmykt4rV/Jl3qlUqoTdkDeQmsBXaD7bfcDIhW/89A6qlvseGhqyBlLn5242\na07cvYPWYiRrUSNfAb4DTInIR4XffkpeyL8Xke8Ds8A3ax51k6IWb+TvRD8ZBvC19Q7sM0/ZMz4+\nbmv83G0n9SYU6hnMzs5WfRwkKluo11UWa5GO6+6pFzM5OWlLGKLSAa/MG3kVUGGoARwbG7P/vFZJ\nuVCBqFpJJpP29nZdMv2sgnW3yXQTQBfQ3afUaFEXc3x83BrvqLknEok467cR0TRm+7ve6n7NzMxw\n+fJlAEZHR4G8S6Z3gDLbzZvo5qwb3PiGy2W2ju26d5Bn9f379wG4du0aABMTE5EbEIp4W2yDomnM\nVvdOWaZ6MZvN2qe5lElnz5617ZVV7pNhfnF7NbiPgCij3TJkZbS6oul0OjInrv3WEtQE34P09+92\n7dpV8h2KAlWhZzIZzpw5AxSLelwDqOojarPBT/AvLS2V1IlA0ThPT0/bh5nUQLpz89WOjllL+hdi\nNRIUTdtdd29FH74hmpqaskzWqiQtMfb970rXcIst9Td1GaempoC8KlO2u9eptAmsW2yx67cBEfSh\n00QiYZLJZElgofnsKLfK1eO+vnSjP72Wsj6VSlkmK1NnZ2cBuH79elmpsJsrj9oMXu2psEJl1Kr0\njpkdEE2rYlU2qstXKWBQ+IxTNs7NzdnHMFT/Hzt2rOzxDmX22NhY2a7QWl4c40L7bejciOuiqXqo\n9vabqDSmmzTyNwgWFxcj3UCFjuVHsW5NSdRC+5Gkzi82kBsQQQ2kiDwCFoH/BBt0/fgMtc/zs8aY\n3as1CipsABG5YYw5EnTQdeBVzDNWIwERCzsgmiHsS00Ycz1o+DyD6+zNjFiNBEQwYW/kd21XqdT9\nmYj8S0Q+KvydrGucEGpko79ru1DRtdet1AVOA98CMsaYnzdinFDMtu/aNsZkAX3X9oaAMeaBMebD\nwuengFbqNhShhB31ru2G/zONgFepC/CWiEyKyLv1FvyHEnZN79puNvxKXeCXwOeBN8jXqP+inuuH\nEvaa37UdGlGVusaYeWPMijHmJfAr8upw3Qgl7A39ru1KlbpaEl3AN4Bb9YwTJJ9dz7u2A6FSpe6b\nIvIGeZX3T+AH9QwSR5ABEUeQARELOyBiYQdELOyAiIUdELGwAyIWdkDEwg6I/wEZz3l3CP30qAAA\nAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1648dd39cc0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAACshJREFUeJztnF1oXMcVx39H2i27xhu8onKJVNut5eIHm9gmm9ZQQxSK\noPFD0tq0NA9NC7XdPBha0YeGPvUxkLavNS4N8UPBKbhQQRyKKSpYAZfYidPYNXYdUX9UJlIjQbeN\nRFf29OHumZ079+6HtOqssO8fxK7unDsze+5/zpw5c+aKMYYMYdDX6w48SsiUHRCZsgMiU3ZAZMoO\niEzZAZEpOyC6UraIfFVErovITRF5ea069bBCVruoEZF+4AYwBtwF3gFeMMb8de2693Ah18W9XwRu\nGmOmAUTkNPA80FTZ5XLZDA8PIyL2mvsdwH34fX19VubBgwdN5f1rrepNk21VlnbNL7916xYff/xx\na0G6U/YwcMf5/y7wpZTOHAOOAQwNDXHmzBny+XyjA7l4F5aXl+33QqEAQD6fZ3Fxsam8f03vcVGr\n1VKvtytLq9svf/rpp1vKWNmOpNKR9iQTNskYcxI4CfDEE0+YYrEYU6j+GL2W9uNyuRylUikmpwrK\n5XIJJbVTkI/VKHk16KbGu8AW5//PAjMdNer8kKWlpdi1YrHY8odqmStTLBbbyrsPSb+3esCdIJfL\nsby8TKfzXjfeyDvAF0Tk8yLyKeBbwEQX9T30WDWzjTHLInIc+APQD7xmjLna6h4RSbBI/9eh3MyM\nKJSNasNnZmaYmYkG1NzcHBCxV+sbHBwEYPPmzQCMjIxQLpdjdalJcuGaurR+uNfaTaJWtiOpJjDG\nnAXOdlPHo4S1nwVaoK+vz9pXZWba5OQjn89z8+ZNAC5fvgzA22+/DcD8/DwLCwsAVKtVIO6h6MS6\nceNGAIaHh9m1axcABw4cAGDHjh1A5P1oHQq1y1qvXlsNsuV6QARltotOPAj1VCYmJpicnATgypUr\nQINl1Wo11fbqiJmdnY39Pz09bUfH1NQU0GD44cOHrY33Gb4W6JmymyGXy1klv/766wCcO3cuYSrS\nJjVVaK1Ws+X+RFer1WwdWqdOsNPT07z44osA7N69O9aeC9echHL9MqwQ647Z1WqVU6dOAXD2bOTo\nqEsHSUbn8/nEQqdQKMRYrvVCZJpUznUfAc6fP2/lXnrpJQD27NljR0C3yJgdEOuO2W+99RYTE9FC\ntBWjBgYGACiXy3ayTQso+ba7Wq3aSdO35wsLC3by1PlifHycrVu32nvd+9TFDLKoWUvcvn0bgDff\nfDN1ElR/eWhoCIBNmzYB6X56oVBImBu9v1wu2xWkmg99qK7yL168CMAbb7zBkSNHgIYH5cotLS3Z\n8G87ZGYkIIIy2xiTGnMArOm4d+9eQqZYLLJt2zagEeN2keYG3r9/P1bm3qcmSK/duROF5ZXp0GD7\n1NSUdQOfeeaZWBvNfkszZMwOiODM9iexW7duAfDee+8BEaN8mXK5TH9/f9N6lcWujLJW3Ttlai6X\ns2UbNmwAYPv27UA0CtTNVJmZmRm7et2zZ0+sXytFxuyA6Lk3ol7I9PQ0ELe/uvgYGhqyjFNPJS3G\nrRgcHLTeh++uuVE8rWNkZASIYt6+fLVataNPbbrK66jpdLneM2VrR8+fP99URl009wHofboZ4F5z\nw7aqNL2mLmOhUEg8OP0sl8t28lRfHBqEeP/99wHsZL1SZGYkIHrCbHeCuX79OtAIp7plrVIelO2D\ng4OWvS5806Jmxf3us79UKiVcy1qtZss//PDDWNvaRhb1W4foyaImn88ntprcJBxdFrss020ttb3+\nBgMQu69V8N9nb1rbbgqE9lHtuJt+sZJ4dk/MyOLiYmy2h3RvQZHP5+3Q9Xd43M0Ad9JMyxeByLf2\n/Xg/sOSWuVAlf/LJJwA89thjK/nZmRkJiZ4w22Vuq73ItNiDv5JcXFxM7KRDMnNKWblhw4ZEFpaL\nNEb7o0NXrGmjsBUyZgdET5idy+Usq3bu3Ak0ds3dWLROXLq4SUO1WrVsVNYvLS0lGJqWHZtWpkhb\nyWqd7ghaCdoyW0S2iMikiFwTkasi8oP69QEROScif6t/NtdIBqAzZi8DPzLGvCsiJeCSiJwDvgv8\n0RjzSv2Ix8vAj1tVpK6fays14mYbc8pdt87fwFW7CfE8bi1L2w6DyGPRaJ+/uHE9GxdujMbtQ1ps\nvRXaKtsYcw+4V/9eFZFrRInwzwOjdbFTwJ9oo2wXqgQ1I25amq+EhYWFmFsHjQexvLycGNb9/f1W\nIVqmPnKxWEyEX92Hn6ZsfYi6F+m7jJ1iRROkiHwO2Af8GfhM/UHoA9nc5J5jInJRRC7Oz8+vqHMP\nGzqeIEVkI3AG+KEx5l+d7ii7Jw92795t3BMD0Iig7d27F4DJycmEOzg7O2tXkMoqZbY7Elwoe/0R\n4Ubz9D6VmZubwydEPp+3KWkaWtW6XTPXCTpitojkiRT9G2PM7+qXPxKRx+vljwOzze7PEKEtsyWi\n8K+Ba8aYXzhFE8B3gFfqn79vV5dui7lZTMqaQ4cOAVEKgW8Ll5aW7Kbsli3RyRJlV6lUsgx1J091\nA/VT2Zu28FE2uxu+ikKhYCdGnczd/tVqtTWNjXwZ+DbwgYhcrl/7CZGSfysi3wNuA9/oqMVHGJ14\nI1OknwwD+MpKGhORppulTz75JADPPvusTWtwc/J8W69M3bp1a8tUBmV2WiBLvRE3l9B3MQcGBhgd\nHY2VKbM1ILWuo36QdJvU9z18+DBXr0ZHczRY7yrITx0ulUp2hekqyjctilqtZuvQT1fGfyiVSoWn\nnnoqtS7tsx6ObYcsNhIQPWF2rVZrGhPeuXMn4+PjALz66qtAlFvi59m58WpluRv4b5aUWa1WU7fR\nIJ5+rJPi2NhY4sBr2jnOTpAxOyB6ZrNv3LgBNJbAmkIAjcyj48ePA3DixAmbTpC28esnvLc6zeVu\nyfnRvHK5bFmsyfC62HLrV/lCobB+U4bT/Gz1l9WsuFAPZXx8nNOnTwNw6dIlIH5ItdlLBCA59Gu1\nWmKPUx90pVKxyZN6fM89fq1wTdji4mKWMrwe0bMJ0l+ttcoy2r59O0ePHgVg3759QCOTanZ2NrEj\nDkl/2R0Jflxm//79AIyOjlqWu5vAPrPdbbKVrCAzZgdEcGbryk5dK/88TDNoDOW5554DopUmxA+R\nXrhwAYjiK2qPtR2d+A4dOhRbfboynW7g+odbO50gM2YHRM+SdNRe+rsyaejv77c217fFlUqFSqUS\nk5+cnEy8iEDZPDY2Zu2yv7hJW6C4rqXCP422LmMjIkKhUGB+ft6exjp48CAQD322emeTQoNH8/Pz\n9syLKtSdKNMyo/z9yTQT1sqsrXTTQJGZkYBY9Xv9VtWYyBzwH+CfwRpdPT5N5/3cZowZbCcUVNkA\nInLRGFNpL9lb/D/6mZmRgMiUHRC9UPbJHrS5Gqx5P4Pb7EcZmRkJiGDKXs/v2m6RqftTEfmHiFyu\n/x3sqp0QZmS9v2u7ntH1uJupC3wN+Cbwb2PMz9ainVDMtu/aNsb8F9B3ba8LGGPuGWPerX+vApqp\nu6YIpey0d22v+Y9ZC3iZugDHReQvIvJatwn/oZTd0bu2ew0/Uxf4JTAC7CXKUf95N/WHUvaq37Ud\nCmmZusaYj4wx940xD4BfEZnDVSOUstf1u7abZepqSnQdXweudNNOkHj2at61HRjNMnVfEJG9RCbv\n78D3u2kkW0EGRLaCDIhM2QGRKTsgMmUHRKbsgMiUHRCZsgMiU3ZA/A+7zYPGc3K19AAAAABJRU5E\nrkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1648dd860f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAD7pJREFUeJztnFtsHNd5x3/fzF54X15EkRRFiZFsRbZsRUoVq7DjS506\nLdwgrpvWrYEaDdDUfWiA9PLQoA9FH1Og7UuBJnXTBH0oYARNEydOWsdw4toBXFeyYltyJFkirRtF\niSKXy+Uul7s7O6cP3zm7S+q2vGQoWPsHBrM7l3POfPOd7z5HjDE0EQ28jR7A7YQmsSNEk9gRokns\nCNEkdoRoEjtCNIkdIdZEbBH5dRE5KSKnReTL6zWoDytktU6NiPjA+8BjwAXgEPC0Mebn6ze8Dxdi\na7j3PuC0MWYcQESeB54ArkvsVN8mMzgyes1z7pVL/bFfgHNrlvVx0+tvMgYDTF04QzY9fdNm10Ls\nYeB83f8LwMHlF4nIs8CzAJu3buOrrxzGMxC6wdofoR2qLyD2AStluzd1I13+8AISur7sJabWroPx\nam35y8jiZOnyewBKBjyz9Lrq2O1z/PlnDlx94zWwFpl9rTd5FR8YY54zxhwwxhzo7uvHVGqDBaCi\nm2cfyoQQoFsodgOCim5hqFu1/bC2hRXdKoHeU78Zo1v9A0uoW/X+unaDUDfPA3zdlrcF2l6jM2Ut\nnH0BGKn7vxW4eKMbDJaDg9oxbxnHlitXc1go6Euh7kU57q9AcVF/l0p2H4DvL20/Gdd9Im4JCMS8\npW2GSzq4xvjN1f9lBTJpLZx9CLhTRD4iIgng94DvraG9Dz1WzdnGmEBEvgi8hE60bxhj3rvxTTrF\npV4uB7VzoJzqOM/J8UpQu75Y0P3Eeb1x/MTPmPhAdXJ2ekKvLxTw4wkAEl2bAOgZ3A7AwOg9bNu5\nDYDBzdpWa4vuPQPGyX23D+t0wjL9EtpxN6rH1yJGMMb8EPjhWtq4nbAmYq8UBquMALHcW1U0dezh\nFJU7FxRg/OQMAIde/g8Axn72IwDyk+cJcrMAVEoLel+ljBEV2p6fBMBv6QCgpW+Ynp37Adh58HEA\n7r7vIQC2D0Nrq/ZZFcWV2h9n0bihujE3Krab7nqEiJSz61EJlv4PHPdQk40LGd2/9l8vcujFbwIw\nO/Y2AMWZtN5XzmGsCeHVSU+zbC+Wr/KXx8mefReA9HuvAXD2rUcB2Pvpz7Pv4B0A9HRj27waVbu8\ngeesR7TENioivBvMJwkhd0V/v/T8PwHw1g++SfbsGADlQlavswQW6qex/hLxqsdCU1lyvQkLBPNq\nK85a8ZOfuQBA7uIp5mb+BID7H1PRMrAZvJW4nDdAU4xEiMjFSCgscRycGeWUTTEPr33n6wD83wtf\nAyB3fpzQeS7Wuwkt7yaTbbT19ADQmtK5H29JEla0wbK9bzE/B0Apm6GYU0UamqKem1WT8fLbr1Aq\n5uzAVM499BuP0t+rh5wD48zQlYqRJmdHiGg5WyAmENbFFpxwdUGnd197lTe+/88AzJ9TOW1KRYzl\nIy+mzkr31lEA+rbuIN6V0qas/22CsDp5nNIMbQfl+Qxzl84CMDeh0QXH/eVCmpnjbwJw9IV/BKCj\nq5/7H7sXgM42bbM69LogWCPYEGvE82q2tJuS2Uu6f/O7X2N27BQAYalQuyfeDsDA7nsA6BlWj1Di\nybqWnYjxEEsFsdrN9+x1qVb62lXctHSq+EmfOQ1AYW6OoKBK88qJ/wXg2EtfZ8uOvwVg913qaibi\ntbFLnIYN7aYYiRCRcragb9d4NXNKrL198tB3ARg/epjyQq56PYDxkwzs+hgAPTb5IDasZ4xBrHYN\nAm2sEnp4Med+6rGyO2cMoKyZ3Kxt9cb0//T7x1icz+v189MAXHznx/z8dY2vjWx7CoAWa4MTU8Xe\n9CBvQUQus8WzXqJlh4JaZBx9/UUA8ulLVQfE8UxqaBtdW4YAmE6r51goqtkWBAFBWWV7cUEVnd/W\nzsiIhtqlpLPk3LlzAMykM/hWzvcODgMwtEnb7tu+yOUTx6rtAixcPsu5t/4bgKlHngCgvzdZG11E\n8ewmVoiNsUaoZWMWptQ3P39aYx5hqYgzrry4huB6t+wkLKsjMn5MrYSsdUx838ezM6Fszbt4Wx+b\n+zSOPXNeLY2Jcx8A0N3dhxdoSufS+FEAKkYtnO2bhmjtnQQgN3UZgKCc59JJHdvYe4cB2Hn3AwC0\nxleWqYmc2EaW5vByE+8AUJ5VeWLCCi6ykexQ1y3R3UEYWOJa8649pWbbnTt2ErNm3sn3NYkQSBwq\nKlKmryjxOjdtBWDPvfsgUCV4+pgScXZS89ZbNn+crgG9bmFGQ7qmUqFgPcw5G8AKCkpsk1hZpr4p\nRiJE9JxtbEbbJnAz0+MAlJaYe5azO7sAFRXOXEv19QEQ61DO7h3YSt6GCRcXlZs7twyTcM5MqFMo\n3q5eZqylDd+21ZnSY+krOqsqQUhnt7afaFEHppDPES7qTFhM6wwo6188GzNpOjW3ICLnbF9sMtVy\nQ1BSE05wcWcwlgeciWY8wfNUWe64S1NaYmsUJOZRyChnlq2D1JPaRMzWLvhx3QdFVYoVDDHUIUrY\nTG8Y2JkRVEjZAEisxebH8jkIVPHOXlQlOzerHfVtieH50dSNrBp+rBYbidsRyJI5trxAw8OzdSDi\nKxFcjWJYLjOTVsvBxUlSqRS+LXuKJ7SDfNkGmyohibh2lrDXe6G+6EpQwqCBLs96lQAurLWYy9o2\nivZMbEWioSlGIkS0sRGxKTGp2aepLlV0yWQteieWs8sLNsgfhnh+LXwKtaRDsFggY8VIa5cqt5aW\nFnxPuTVpQ7LZReXGsBzg2ekUd5xt+y0Wi0DnVeOu5jZtiNLZ9RKClUgNocnZEWJjsuuGamqsrW8U\ngPYujVeL+FX3slRQc7CyWCRmvcnlyC3MsVhQrt28RW2xeNLDBMqFbsaEWZW3QRAQWrkct+dcArpU\nKhFaLVuupuHAsa9TmnEbW4cblgZehZtytoiMiMhPROS4iLwnIl+yx3tF5GUROWX3PSvo97ZEI5wd\nAH9hjDkiIp3AWyLyMvB54BVjzFfsJx5fBv7ypq2JlvU6bmrp1ehc/8AAAGdPnSEo2VhHXrlxYTZD\nvG1pqZJY7s+nZ6lYzuvq1ECzZ6jK+GSHmnLelLrf6bkMrZajs/MaLSxZPZD0PILCvB6bz1WH7CyT\njl6NEnakrP6oi8s3gpsS2xgzCUza3/MichwthH8CeMRe9m/AqzRCbJbWjbTbgsfde9V+PnH0GOkr\nWp3jBTqV5ybPVTPosVYlVGCDSZn0NPE2LS3r7E7ZVqVaKtZjw6fdUxojuXjqKOkLai8vzGsKzImH\n/p4e8pc071mxQS1BiLWp2BjcqQGrFtcNtbReQ8/d+KUgIqPAfuBNYMC+CPdCNl/nnmdF5LCIHM7M\nXFlJdx86NKwgRaQD+Dbwp8aYrDQYWzTGPAc8B/DRfQeM51MtbAdosZbWjv2PADDy0x+TndWpHAZ6\nYSE9SXpCld+m0VEAKjYZnCvkSPUq97bWOSJeqHzU2q737di1F4C2iTPMz2v7nf0ahh0a0ZKzZFhg\nykYAayVmHl2DWmK8/a5PANBus+zx2DorSAARiaOE/ndjzH/aw5dFZMieHwKmVtDvbYmbcrYoC/8r\ncNwY8w91p74H/AHwFbt/odFOjdSY27npfbt/FYCP7v8+Fy/oe5uZ0qSrCQpkzp4AXAQQOvpVGe7e\ns4+EFaJ+1ekw1fY920Fbjyrg0a6eqqvv2bJiV2o8+e5hAutIuXMSb2X73vsAuOMe1SvxuiouswKn\nphEx8gDwDHBURN62x/4KJfK3ROQPgXPA7zTe7e2JRqyRn3L9wNanVttxtWLJcnbbgFoU9zz8u5z/\nQC2C/CF9twtzWSipnJ0es9mYxY8A0D08Uo3eebYxY0zV5Q/d1HFmg58gbn/mZ9UcvDym6bGFmZlq\nlii0pmPfjl3c9dDnAOgfXjpm8fQjqEZTYxuSPDD1X37agXtW6Wz7xKf55QtK0ExGTcDxE2Ms5mzE\nvqxEz5w9DkAhM0XH5i0AtNhkgO97xNq6bPNqwrnkRCGfJ8xqhj5zSZVhpaiiQ7PlOqD2QVW6uz75\nGfY9rPXbyZalz1Ix4JVpuMKyGRuJEBsSGxEPxIaEy7ag0pmA8S7Y85gWpGczyoHlyneY+EAL1otZ\n5WwTqlOzmJlkMaOK1MWzEy1t1aSBQ7lgvcViDs/eK3XfJQCIxGgfHARg9OCDADz02T/C+lPkc0su\nJx6HWKLx0uEmZ0eI6CuiXImwLVAtqY6izYWRY9A6rFx58Les9+/FeOMnPwDg4hmVs/NpleflYhEP\n5VRT1n3RyvV6OO4LufqzkHhSFUbn0DYG96t59+hv64oeu/YMu7p4lgQCAWkHWQEFIye2B2DAZqZc\neq/68b7nQWhHlRrVmMTDv//XDG29E4DXf/Q8AO+f0OKb7Mws5YIqz8Bm14MgqNVzLxMVnh8jbmMh\nLd3qXbb3qejYvvsBPvXMnwGw62NqesT8WiWAqzq2CXticX2OZmHlLYjoS4YtVxQW7ZT3lbX9mEse\n1HGA5aTEMBz43NMAbNuriuud//k2AEfeeJVz5/VLgvmsarBisWjXzaj7UNTXR43FEnR0qok4OHo3\nAB9/6EkAfulXPkuPMnk1weyZ2jhi1vSz+WFi8ZVxa5OzI0Tk39R4vsrnwryyR2DTYXHLSWGdAHSz\nIAxBLFf17dFavE9u+xIA9z76BSZOa8Hj6SMvAzB25BXGzqoinbii3J4LtYP7f+0LPPnUMwCM3K3x\n6ZStbDL+tb/RrHqIbgGaOqqte9SvifVB9E6NZ8XpjMrq9gG1DGztOZVrqPZKpfZVVnWZIydTk+3s\nevBhAEZ2aLxka1seKqoTZmx1bK6ojzp84HHueFA5ui1hO3Ds6dd9/F/HzctjH64CV8LlxUU3RrSr\nMhg19SolmC+rR9gVVyXlVsExXk2p1a+e4w65h7NVa1xKX2HnUD8A7SlNFnV099DeruKpJakv07PZ\n9vn5bLXWo7pWlKvKql9vyh4Lw7rpb3/4bk/tvkbQFCMRYtXr+q2qM5ErQB6YjqzT1WMTjY9zuzGm\n/2YXRUpsABE5bIxpbG22DcQvYpxNMRIhmsSOEBtB7Oc2oM/VYN3HGbnMvp3RFCMRIjJi38prbd+g\nUvdvRGRCRN622+Nr6icKMXKrr7VtK7qG6it1gd8EngJyxpi/W49+ouLs6lrbxpgS4NbaviVgjJk0\nxhyxv+cBV6m7roiK2Ndaa3vdH2Y9sKxSF+CLIvKuiHxjrQX/URG7obW2NxrLK3WBrwI7gX1ojfrf\nr6X9qIi94rW2o8a1KnWNMZeNMRVjTAj8CyoOV42oiH1Lr7V9vUpdVxJt8SRwbC39RBLPXtVa29Hi\nepW6T4vIPlTknQH+eC2dND3ICNH0ICNEk9gRoknsCNEkdoRoEjtCNIkdIZrEjhBNYkeI/wfL6sxe\ne04dHQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1648ddd6e48>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import cv2\n",
    "\n",
    "# Output sample data\n",
    "image=X_train[6837]\n",
    "Image.fromarray(image).save(\"writeup/before.png\")\n",
    "yuv=(255*preprocess_image(image)).astype('uint8')\n",
    "Image.fromarray(yuv[:,:,0]).save(\"writeup/y.png\")\n",
    "Image.fromarray(yuv[:,:,1]).save(\"writeup/u.png\")\n",
    "Image.fromarray(yuv[:,:,2]).save(\"writeup/v.png\")\n",
    "rgb=cv2.cvtColor(yuv, cv2.COLOR_YUV2RGB)\n",
    "Image.fromarray(rgb).save(\"writeup/after.png\")\n",
    "\n",
    "# Plot the images\n",
    "plt.figure(figsize=(1, 1))\n",
    "plt.imshow(image)\n",
    "plt.figure(figsize=(1, 1))\n",
    "plt.imshow(yuv[:,:,0], cmap='gray')\n",
    "plt.figure(figsize=(1, 1))\n",
    "plt.imshow(yuv[:,:,1], cmap='gray')\n",
    "plt.figure(figsize=(1, 1))\n",
    "plt.imshow(yuv[:,:,2], cmap='gray')\n",
    "plt.figure(figsize=(1, 1))\n",
    "plt.imshow(rgb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Shuffle training set\n",
    "from sklearn.utils import shuffle\n",
    "shuffle(X_train, y_train)\n",
    "\n",
    "# Perform preprocessing\n",
    "X_train, y_train = preprocess(X_train, y_train)\n",
    "X_valid, y_valid = preprocess(X_valid, y_valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Model Architecture"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "from tensorflow.contrib.layers import flatten\n",
    "import tensorflow as tf\n",
    "\n",
    "def LeNet(x, keep_prob):\n",
    "    # Arguments used for tf.truncated_normal, randomly defines variables for the weights and biases for each layer\n",
    "    mu = 0\n",
    "    sigma = 0.1\n",
    "\n",
    "    # Layer 1: Convolutional. Input = 32x32x3. Output = 28x28x6.\n",
    "    conv1_W = tf.Variable(tf.truncated_normal(shape=(5, 5, 3, 6), mean=mu, stddev=sigma))\n",
    "    conv1_b = tf.Variable(tf.zeros(6))\n",
    "    conv1 = tf.nn.conv2d(x, conv1_W, strides=[1, 1, 1, 1], padding='VALID') + conv1_b\n",
    "\n",
    "    # Activation.\n",
    "    conv1 = tf.nn.relu(conv1)\n",
    "\n",
    "    # Pooling. Input = 28x28x6. Output = 14x14x6.\n",
    "    conv1 = tf.nn.max_pool(conv1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='VALID')\n",
    "\n",
    "    # Layer 2: Convolutional. Output = 10x10x16.\n",
    "    conv2_W = tf.Variable(tf.truncated_normal(shape=(5, 5, 6, 16), mean=mu, stddev=sigma))\n",
    "    conv2_b = tf.Variable(tf.zeros(16))\n",
    "    conv2 = tf.nn.conv2d(conv1, conv2_W, strides=[1, 1, 1, 1], padding='VALID') + conv2_b\n",
    "\n",
    "    # Activation.\n",
    "    conv2 = tf.nn.relu(conv2)\n",
    "\n",
    "    # Pooling. Input = 10x10x16. Output = 5x5x16.\n",
    "    conv2 = tf.nn.max_pool(conv2, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='VALID')\n",
    "\n",
    "    # Flatten. Input = 5x5x16. Output = 400.\n",
    "    fc0 = flatten(conv2)\n",
    "\n",
    "    # Layer 3: Fully Connected. Input = 400. Output = 120.\n",
    "    fc1_W = tf.Variable(tf.truncated_normal(shape=(400, 120), mean=mu, stddev=sigma))\n",
    "    fc1_b = tf.Variable(tf.zeros(120))\n",
    "    fc1 = tf.matmul(fc0, fc1_W) + fc1_b\n",
    "\n",
    "    # Activation.\n",
    "    fc1 = tf.nn.relu(fc1)\n",
    "\n",
    "    # Dropout\n",
    "    fc1 = tf.nn.dropout(fc1, keep_prob)\n",
    "\n",
    "    # Layer 4: Fully Connected. Input = 120. Output = 84.\n",
    "    fc2_W = tf.Variable(tf.truncated_normal(shape=(120, 84), mean=mu, stddev=sigma))\n",
    "    fc2_b = tf.Variable(tf.zeros(84))\n",
    "    fc2 = tf.matmul(fc1, fc2_W) + fc2_b\n",
    "\n",
    "    # Activation.\n",
    "    fc2 = tf.nn.relu(fc2)\n",
    "\n",
    "    # Dropout\n",
    "    fc2 = tf.nn.dropout(fc2, keep_prob)\n",
    "\n",
    "    # Layer 5: Fully Connected. Input = 84. Output = 43.\n",
    "    fc3_W = tf.Variable(tf.truncated_normal(shape=(84, 43), mean=mu, stddev=sigma))\n",
    "    fc3_b = tf.Variable(tf.zeros(43))\n",
    "    logits = tf.matmul(fc2, fc3_W) + fc3_b\n",
    "\n",
    "    return logits"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Train, Validate and Test the Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "A validation set can be used to assess how well the model is performing. A low accuracy on the training and validation\n",
    "sets imply underfitting. A high accuracy on the test set but low accuracy on the validation set implies overfitting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Definitions needed for training\n",
    "\n",
    "EPOCHS = 25\n",
    "BATCH_SIZE = 4096\n",
    "LEARNING_RATE = 0.006\n",
    "\n",
    "x = tf.placeholder(tf.float32, (None, 32, 32, 3))\n",
    "y = tf.placeholder(tf.int32, (None))\n",
    "keep_prob = tf.placeholder(tf.float32)\n",
    "one_hot_y = tf.one_hot(y, 43)\n",
    "\n",
    "# Training Pipeline\n",
    "logits = LeNet(x, keep_prob)\n",
    "cross_entropy = tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=one_hot_y)\n",
    "loss_operation = tf.reduce_mean(cross_entropy)\n",
    "optimizer = tf.train.AdamOptimizer(learning_rate = LEARNING_RATE)\n",
    "training_operation = optimizer.minimize(loss_operation)\n",
    "\n",
    "# Evaluation pipeline\n",
    "correct_prediction = tf.equal(tf.argmax(logits, 1), tf.argmax(one_hot_y, 1))\n",
    "accuracy_operation = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))\n",
    "saver = tf.train.Saver()\n",
    "\n",
    "def evaluate(X_data, y_data):\n",
    "    num_examples = len(X_data)\n",
    "    total_accuracy = 0\n",
    "    sess = tf.get_default_session()\n",
    "    for offset in range(0, num_examples, BATCH_SIZE):\n",
    "        batch_x, batch_y = X_data[offset:offset+BATCH_SIZE], y_data[offset:offset+BATCH_SIZE]\n",
    "        accuracy = sess.run(accuracy_operation, feed_dict={x: batch_x, y: batch_y, keep_prob: 1.})\n",
    "        total_accuracy += (accuracy * len(batch_x))\n",
    "    return total_accuracy / num_examples\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training...\n",
      "\n",
      "EPOCH 1 ...\n",
      "Training Accuracy = 0.072\n",
      "Validation Accuracy = 0.064\n",
      "\n",
      "EPOCH 2 ...\n",
      "Training Accuracy = 0.231\n",
      "Validation Accuracy = 0.237\n",
      "\n",
      "EPOCH 3 ...\n",
      "Training Accuracy = 0.392\n",
      "Validation Accuracy = 0.373\n",
      "\n",
      "EPOCH 4 ...\n",
      "Training Accuracy = 0.540\n",
      "Validation Accuracy = 0.495\n",
      "\n",
      "EPOCH 5 ...\n",
      "Training Accuracy = 0.696\n",
      "Validation Accuracy = 0.641\n",
      "\n",
      "EPOCH 6 ...\n",
      "Training Accuracy = 0.765\n",
      "Validation Accuracy = 0.723\n",
      "\n",
      "EPOCH 7 ...\n",
      "Training Accuracy = 0.832\n",
      "Validation Accuracy = 0.771\n",
      "\n",
      "EPOCH 8 ...\n",
      "Training Accuracy = 0.861\n",
      "Validation Accuracy = 0.782\n",
      "\n",
      "EPOCH 9 ...\n",
      "Training Accuracy = 0.889\n",
      "Validation Accuracy = 0.824\n",
      "\n",
      "EPOCH 10 ...\n",
      "Training Accuracy = 0.912\n",
      "Validation Accuracy = 0.835\n",
      "\n",
      "EPOCH 11 ...\n",
      "Training Accuracy = 0.918\n",
      "Validation Accuracy = 0.861\n",
      "\n",
      "EPOCH 12 ...\n",
      "Training Accuracy = 0.937\n",
      "Validation Accuracy = 0.859\n",
      "\n",
      "EPOCH 13 ...\n",
      "Training Accuracy = 0.945\n",
      "Validation Accuracy = 0.866\n",
      "\n",
      "EPOCH 14 ...\n",
      "Training Accuracy = 0.950\n",
      "Validation Accuracy = 0.880\n",
      "\n",
      "EPOCH 15 ...\n",
      "Training Accuracy = 0.944\n",
      "Validation Accuracy = 0.870\n",
      "\n",
      "EPOCH 16 ...\n",
      "Training Accuracy = 0.954\n",
      "Validation Accuracy = 0.879\n",
      "\n",
      "EPOCH 17 ...\n",
      "Training Accuracy = 0.964\n",
      "Validation Accuracy = 0.896\n",
      "\n",
      "EPOCH 18 ...\n",
      "Training Accuracy = 0.969\n",
      "Validation Accuracy = 0.897\n",
      "\n",
      "EPOCH 19 ...\n",
      "Training Accuracy = 0.974\n",
      "Validation Accuracy = 0.905\n",
      "\n",
      "EPOCH 20 ...\n",
      "Training Accuracy = 0.977\n",
      "Validation Accuracy = 0.907\n",
      "\n",
      "EPOCH 21 ...\n",
      "Training Accuracy = 0.976\n",
      "Validation Accuracy = 0.910\n",
      "\n",
      "EPOCH 22 ...\n",
      "Training Accuracy = 0.978\n",
      "Validation Accuracy = 0.913\n",
      "\n",
      "EPOCH 23 ...\n",
      "Training Accuracy = 0.977\n",
      "Validation Accuracy = 0.908\n",
      "\n",
      "EPOCH 24 ...\n",
      "Training Accuracy = 0.983\n",
      "Validation Accuracy = 0.918\n",
      "\n",
      "EPOCH 25 ...\n",
      "Training Accuracy = 0.986\n",
      "Validation Accuracy = 0.918\n",
      "\n",
      "Model saved\n",
      "Elapsed time: 754.6435992717743\n"
     ]
    }
   ],
   "source": [
    "# Do the actual training\n",
    "\n",
    "import time\n",
    "\n",
    "start = time.time()\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "    num_examples = len(X_train)\n",
    "\n",
    "    print(\"Training...\")\n",
    "    print()\n",
    "    for i in range(EPOCHS):\n",
    "        X_train, y_train = shuffle(X_train, y_train)\n",
    "        for offset in range(0, num_examples, BATCH_SIZE):\n",
    "            end = offset + BATCH_SIZE\n",
    "            batch_x, batch_y = X_train[offset:end], y_train[offset:end]\n",
    "            sess.run(training_operation, feed_dict={x: batch_x, y: batch_y, keep_prob: .8})\n",
    "\n",
    "        validation_accuracy = evaluate(X_valid, y_valid)\n",
    "        training_accuracy = evaluate(X_train, y_train)\n",
    "        print(\"EPOCH {} ...\".format(i + 1))\n",
    "        print(\"Training Accuracy = {:.3f}\".format(training_accuracy))\n",
    "        print(\"Validation Accuracy = {:.3f}\".format(validation_accuracy))\n",
    "        print()\n",
    "\n",
    "    saver.save(sess, './lenet')\n",
    "    print(\"Model saved\")\n",
    "\n",
    "print('Elapsed time:', time.time() - start)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test Accuracy = 0.903\n"
     ]
    }
   ],
   "source": [
    "# Test accuracy\n",
    "X_test, y_test = preprocess(X_test, y_test)\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    saver.restore(sess, './lenet')\n",
    "\n",
    "    test_accuracy = evaluate(X_test, y_test)\n",
    "    print(\"Test Accuracy = {:.3f}\".format(test_accuracy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "---\n",
    "\n",
    "## Step 3: Test a Model on New Images\n",
    "\n",
    "To give yourself more insight into how your model is working, download at least five pictures of German traffic signs from the web and use your model to predict the traffic sign type.\n",
    "\n",
    "You may find `signnames.csv` useful as it contains mappings from the class id (integer) to the actual sign name."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Load and Output the Images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New image labels are: [13, 4, 5, 34, 38]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFJ9JREFUeJztnHmUXFWdxz+39u7qraq39Jbeku5sJCGmEyAhQdkioKgo\ngiKiKCNHVJBzXBj06DiDqOOI+wwjCs4wKkgMCQkkQCAhZN9J0kmn0/Sa7k66u6q69vXOH78XD0eB\nFOn4yIH6ntOnql+9d++7v/d9v+3+7lVaa3IwB5a3+wbeTcgJ20TkhG0icsI2ETlhm4icsE1ETtgm\nYkLCVkotU0odUUp1KqW+cbZu6p0KdaZBjVLKCnQAlwP9wA7gRq31obN3e+8s2CZw7QKgU2vdBaCU\n+iNwLfCGwvZ6vbqmtharzU5gPABAIDAGgNUmDz2VSqEsVgDyCwrkWDpDOpUGwOnMQ/qzGdfZSCQS\nAAT90maZx4sySGR1OgGIGddHwhF0Us63YhBNy29pnSHP5QIgmUwafSex2JR8R86zWuX+XFYnKgW+\nQIBwJKpOJ7CJCLsG6HvN//3Awr89SSl1G3AbQHV1DctXrqa0soqn1j4FwKpnHgWgxCMDH/OdxO4q\nBuD8iy6WY6EQvhER5JSGmQDYXaUAFJVW0vNqNwAbV68F4JaP3oDNEJa3aQoAHSd8AOzetov4ieMA\neFQcgFRS2g7HwkyfLu2fGB4EYMQ3gLPCDoAv7Zd7LfIC0FLciNMPP33of7IS2ESE/XpP8u90ktb6\nQeBBgFmzz9OaJDo2Tn4qBIDFYHbLTBnkoMtBLCHNvPzUM3KTykKxQx7AwQPDALgKywEYC4e47roP\nAHDRzR+V655ZQ9sl1wBwXt1kAOa0SPs3X76YW278JADOPHkgifg4AAWeYrbv2CJ9OuSNcOY7sVnl\nu1tl5J4TDgD27D9KbCRGMBI9rbBgYsLuB+pe838tcPzNLlCA1ZKmpEAx0tcBwJTqKgCOvfIqABEU\n4WQKgKbJjQC0NjRiTYstjwblQQRC8kq3zZ2HKy7nW+KiHuonVVDZ0CwDtIsq6n91CIAnHv8DbfNm\nA+DzdwPgdpcBsGfPIcIx6Uclhc3evDxGB+RhRCNCjMm10ubcWe/hmWdfIp2l3ZuIN7IDmKqUalRK\nOYAbgJUTaO8djzNmttY6pZS6A1gLWIHfaq0Pvtk10XCIAzu20HzdtcQNozQ4Iuokv0AYXmB3UVct\nKqKr/zAAwZ6jdPcPADC5XtTBhfPfC8DlS5bx3JNrAHhx3z4AaurqiHsnAXAsIqoiv1r+v+t7d9Hd\nKS/gN7/xBQCWLL0AgKJJCcb7hcUtU6WfWDRAz2ExTa11cl9FadGgvuM9xMKj6Ew6K5lNRI2gtV4D\nrJlIG+8mTEjYbxXJcJSh7QcIti0goaXr/LIaAFoazgOgs+MYbS3y3eMQXTzoH6CqTDyAtgXnA1Dq\nLQEgHBhn4fxFALhconsHfAEa6wsBOC5qlqA0xcr1B1j3lz8B0NQyF4A1a8Uozpx1Phc1SVuHDh0B\nwFvo5oorrgBg14srAMjDA8Cw3095iYVOa3bjz4XrJuKMI8gzQanFrt/vKCFdX46/PB+A62//PADX\nXLoMgFBXD40FEljEhkRXugry6O0Vv3fywksA2PLCywB4PR5al74PgNTQCAD+RIbeYTEf1VOnA+As\nqQRg67Y9rHtafPx7vv5VAH74ox8BYM8vIpIS/i27WlzHBx64j6YaNwAep+h/W8YIohJxtu7Zx/79\ng4RC8dMGNaYKe05phV73/o/z8I4tjM2oBqCkTVy0trnyae/oYnjVRgCmIA+koqAcmxHEWAorAHB4\nxe9OuSJEtBi18tpaAPLyveh88X2Vo0g6r2mSz0gSPKKCoidPyKGgGOmeE0N0jcgxV6m0P3TiOD6f\nuI3HerqMexDhd/YOUlxayfPrtjI2FjitsHNqxESYaiDtmQyVsRBTLFYOK7Eqta3C8EdX/AaAhz51\nO91bJeCpTwoDQ9iJu+X7WEws3fiRbmnTHWU8KSqmp1OMmjWdhy8j4bkrI9FeQZ5cn3LlEzMC3Xyb\nBC7FdhFDpbeQKq+cnwwJm69sqCbaIFGo7QaJPG//wQ8B8JY1EhwJkU7944OaHN4iTGV2JhYheHgv\nobEw0ZQwetwirIiIGmRf91HsFsnsdcbkt9ZPfhAuE5eseCwMgFPLOUT8pILC7HAgCIBvNEFxSpJG\n1mAMgPR4RM7XNkIhaUOF5bewT84Nn+zB5RBdbw9KcqrdF6ErKoZx/t13AnD1B64HYOXal7hqyRJe\nOXQ4q/GbKmzsdqiuYnj4GD0jMkDnmHwWV8hrvvHoXr5y820APPkT8Yf3rVvB9R9ZAMAxl9xyNC5q\nyJMop7peokOrkSMptxZSY5gre54kkYiKYElkIGw43Ua6lpDhjJ/sgvGTAESeWA5A30An8xZIMtPm\nlQjy4V88AEBT8wyOdXUSj8ezGn5OjZgIU5kdQvFy2sI2fwi7RRjnCstr6wiLCugIBDgQ7AGgYW4L\nAKPHjpHZtA2AsrniN3c75dYDGYUKyCvvMEYTSAXwZMR1C4ckJWs1fnSnrWjDQMYTwnZrgXCupnYO\n7NgLwIE+MbDVc+dTe92HAbj9t78E4NYvfxGA9gPtzL1sOqs2rctq/DlmmwhTmT0YDHHfhk1c+JEb\nGbdKFOYOGa5ch2T1EnlW/mwk8L8640MAJNfvIvL4CwBUzJS8SSApgUgi7aDYKpzJpMQI5rk1tqgx\nbeaU9pM20asRFNZCiVBDdnH9XE75JK35ywO/AqDIIsfKly3lpxsk4uwxdHtoWCLVeGScNTueIxAe\nz2r8OWabCFOZ7fKWMP3KD1AweyoDByX3HDE8g6m1cwDw5ztYv+cYAF9aIjnuivpp7NqwE4Clw+K9\n5HvEV7TaC8mkTk3ICpKZFHHkzbHYxWvJJI2cs9VGQssbYEnINFeTRyacdt33nxQExYZcdM2lABxO\n+Tnk7wXg7q+Il3TyoMwqTS7Ko7Oni1TmVM9vDnNTrDrDQCrK0d2bqa6QFGhN61QA4jExaNOb6mia\nJ37ti8clIlx6yQLSUclZvPzYYwAs+vbXATg8EMKH5D9cLmkzo6PgFuFGoyJYm11+S1s0tpQItLlU\n8iy+5ZKSP/rkKtrOEzWVt0Q+d21bwYWLZwBgT4r6eH7N/wIwefZ5tE4uZ4MjOzHm1IiJMNf1C0fZ\nums/V1+9FIsWY7OvaxcA0aRhfHq2YDWCjZ6wsPF97/8MQcmo0rn5JQAWdX4EAK/bi99lRKFxMZrF\nZEik5ZgrYwQ1GTGK/kSA+iIjE+gThm984NcATKutoPkmUR/f/YPkP7r0KLpTzpvzuVsBSJSJy7in\naxupV+1EQoGsxp9jtokwldkKC1bySIz6aKgT18pilzyFtgsr58ysYai3HwD/mGTent22kssung/A\noVWiu33rngag4qZPMJYSXeo0MomliTSjWnLhLru8JX4jlHcWuPEY+ewVn5EJ31qHZPrmLlvMS4Ov\nALA5IUaxZVYT0R4x2L9Z/l8A2KYaxnk4giOQxnLaTLbAVGFbVIo85cc/2E9Rq8z/9QVEoIGQCLEi\npYiMik+8b0hez42dq7nin8X/LXtOvJiOP0mkt/BjXyTuktc8lhFhV6Wc5Cu5drBQ1EjAyIIudHs5\ncc9PAajuEM/Ge+liOffCRr7zL58B4IILxCiGe3qZ1dQGwNMbN8t19TLrs/A9zRw8sB+LfTC78Wd1\nVg5nBeZm/ZSCPCvjiRgWI2qrdBu+dJUYsFQwRPdxUSNuh1Qe+QJjbN4uhnFOs7hrg37JzvW8tJaq\na6SGJGiMZiwWIOkUlif8knM5v9Io3jrcwYb1qwForpGZ/aZlcv1N996BJU/UTr5bZtD9Q0G2bpe8\nTHe3TIudHJW3sbaiDBd2LK9biff3yDHbRJjK7HQ6RWhsFHt5Mfn5YpT8RkQYSQlTqyq9XGBUKOUV\nywTD6uWreOTPvwfgT9+SmfCDXRLw7H5mLR++8nIA4jHR0z63JmaToU1zijF0DAvDV9/7bUo8wvp5\nt14LwP0//z4AR/sHqWurB2DzZrEJx3t6qa6WY7U18hYWGQZ20wsvMaOhGTLZjf+0zFZK1SmlXlBK\ntSulDiqlvmIc9yqlnlVKHTU+Pdl1+e5FNsxOAXdrrXcrpQqBXUqpZ4FbgOe11vcbSzy+AXz9zRrK\nt9uZV11NY3k51rCE0YtnS04kr0To8cenHsftFb3cUCKfTU31ZLziIu4LSC3JpFYpTejb0AHrpEak\n9oJpAOwuSpFKikfjzhe933HPv0g/IyMs+qzUqBwZ7wbgkRdEh1fPmI7DKJ8oqGgAYGxknFKjbtBd\nLIx2l0ib0eJykr4wOpPdhO9pha21HgQGje9BpVQ7Ugh/LXCJcdojwIucRtiV3jLu/ORnCSSCNLWI\nivjdwzLFZHFLVDYYHaGxRAb35PJVABQXlTBr1iwA/rD5eQC+u/RmGcDWEboelhRoU5v44jrhZ2Gl\ntB941Mh7bBG3bcp5dThnSYH8zZ//BABVMyU/09jQzJFOWTgxf7GQYNaM+WAUUobHJWfT0SHuZ6HT\nxeE9BzDKTk6Lt2QglVINwPnANqDSeBCnHkjFG1xzm1Jqp1JqZyAcfivdveOQtYFUShUATwB3aq3H\nlcrO3XntyoOKkhK96rkNWJyKo/2iDnbvF3fqQzdcBYDHmeblHZIvCZ4UtmfCEVYNCKOdSl7Zr33w\nswBEXCmcCTF+GMs3Zs4phVc6AVj/+0cAqG0Wprd+4dPc9q8yS17ULAxPJcQNDfrHKXSIGsnEpZ9t\nW/dS5pGJ3lDQSM2KbSeYCtE8fTqDJ17NShZZMVspZUcE/ajWerlxeFgpVWX8XgWcyKrHdzFOy2wl\nFH4IaNda/8drfloJfBq43/h88nRtabuN1CQvxzqO0rpQShNCSGbvqY37AfBFx8k38hmNZRIyp5Mx\nnIWSqauaKqHyvb/7CQA/v+0OVtz3IACBR34GwLxvf5lNd30LgCKXsHbOx6Xs94lNa1ixXQxqzSyp\nL3QY8wr9viFcTuHfgV2SIylw5qMQV3FSpdQSnvSJmxoYD9AyrRqbvf90QweyUyOLgE8Bryil9hrH\n7kGE/JhS6lagF/hYVj2+i5GNN7KJ118ZBnDpW+kskoyzf+gotfWVnAhIqUDcWFPoj4vrp20FOG3C\n9pShiieVVxOw+Y1W5HxLrejR4SonnnkSdPiPCGPj9/0CTsr5tZfNAyBUK+mAb977E85fJEmwcb94\nFwUFRmF9MESz4VK2H9oNQIYkQeRGwkbpw4I2KdrZuXM7h9qPEo1lV6RjcgSZZCwwxMzGevxBeRXr\n6iU/MWiU5dqxUm5EaL2HxXhObq4lGZTJhb72owA0TWkAYNX2rVyz6CIAfH0yA+88PEppiwit9UaJ\nEr/28L8BsPDKxVQUiGu56+UDAAz0iWF1Fhcw7JeZ8kRKHn4wEqCgQlSYxSEqpv+4GMRAIMD8BW30\nHslNHpxzMHfyQGtsyQzBYJC6VqlsCuyUYMNtE/VQ6HKiMpKfthbK7e169QD5JeJvlZfIuhlluGa/\n+vVDXPczyZuM54nbtr9/iNlfkqql7//yfgCCRcZq3niEE4bbmYqJK+cwqqtimRjt3TJR4HFJf2X5\nZVxwsRjznfv2GAMR9bNoyUJ8Ph+ZLFeL5ZhtIkxltk5lSI5FyHcW0jskbnkoJvUdhcaKq1QkRsEk\nyT1UTxWmDgf9hIzVu3fdeQcAP77vBwDMnP0eHnj8/wD42JWy1r32SgfPDUrYvXqfzBTPWCpGsaKs\nnKgWlmuLMNJhN+y/Q2EzqqWiQTGK77v0Mnp6JUBavFDKlg+0i65PpYO4PQpLlqvFTBW202pjSlEZ\npU4n7QPyKqfTMmB3iRhFYnH2HxYj6CiR5XgpnSE4Jt7L3Xd8E4CFC2Sq6lh3N0+vlzU4N37pcwD8\nce0avveDnwNw+QfFc+jukTnFtvkXY2wAQVKJEB3G7LxWUcrKJXk5qsVAPvbYk2Ds4jDQK0Wangoh\ng7KniSf9ZMipkXMOpjJ7ktfL1266HmtlOUcekYixslhUhc0u7CosqyQ0KG4hxsLUgowdi1WMUrFX\nfPCRcVEF/SMBCislqrzu1lsA8HrzmX2xbAawv11WBcycJQb5iZXrcCipvhr3iar41FVSwDkWGmK3\nsdjUPy5G2lNWgT0paiYRkRl6T6m4jnn5Cv/IEFpnN3uQY7aJMLciKhRky4YXGc4k6OuSFWFOrzA7\nYmQRXUUWrA5jcWdKIjZ7Jk2eVX4fDwkbT8SMtTX5Rdi0ME4l5VhtWS1xo8zy1E463QNSbpBI2UgY\n+5kkEzL8Z9etB6CiroRIRNzBadNko4CTfWN/LcoMR2LGOOT/hI5T6q7AZnnTnT/+ihyzTYSpzC6b\nVM2t93yHue9dQs0s2biltFpC4ZFhIwT2RXBZ5bacdmFnbV0dA4PCWneJnJ826gGH+0awRUVnphPG\nJi8ZCwN94jkMGwuk2ha3AtAwuZxt68V1K3KJtxMxlpqcGA3hdstKYpsyghpPGd2BbgCsBdJnwtho\n5vDODuyZCOHgP34nnbeMcDLF5r6TOBumoD0y0NqpDQAk41JCNrm2mJYpkvrs65XX02JJU2fM/7Ub\n6kAlRejRyBgzJ8vaG6dDUqBNLY0EDGOmDLdu734RsCvjwVigwLRmuS6kRwHoGenEXSTqKjgq7mdd\nZQ1WhzjS0ZS4gMe6ZM2PTljxuCuxqdxqsXMOpm4UoJQ6CYSBEdM6PXOUkf191muty093kqnCBlBK\n7dRazze10zPAP+I+c2rEROSEbSLeDmE/+Db0eSY46/dpus5+NyOnRkyEacI+l/fafpNK3e8opQaU\nUnuNv6sm1I8ZauRc32vbqOiqem2lLvAh4HogpLX+97PRj1nM/ute21rrBHBqr+1zAlrrQa31buN7\nEDhVqXtWYZawX2+v7bM+mLOBv6nUBbhDKbVfKfXbiRb8myXsrPbafrvxt5W6wK+BZmAuUqP+44m0\nb5aw3/Je22bj9Sp1tdbDWuu0lnmv/0bU4RnDLGGf03ttv1Gl7qmSaAMfBg5MpB9T8tlnste2yXij\nSt0blVJzEZXXDfzTRDrJRZAmIhdBmoicsE1ETtgmIidsE5ETtonICdtE5IRtInLCNhH/D8F7s5WL\nD0jkAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1648b0c1be0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEeBJREFUeJztnFmwHNV5x3+nt9nu3EW70IKEwEKAsTACmwgDAhNjk2DH\nxCRUObHLuPCLq5JUHuLKUx79kOQ1VaS8xNkMKVMEF66Y1ayWkUASIF0EAgkhJLTcde5Mz/R28vB9\nPVeAhAZduVGh+apu9dzu06dPf/0/336OsdbSp2LI+bgHcC5Rn9kFUp/ZBVKf2QVSn9kFUp/ZBVKf\n2QXSnJhtjLnFGLPbGLPHGPODMzWoTyqZ03VqjDEu8BpwM3AA2ALcaa3ddeaG98kibw73Xg3ssda+\nCWCM+TnwVeCkzPZLJVuu1jAWHCPnDPoD+ejWzl50HZl4WZpi8+uG95IFo3hxFDgmgyQ/aeQGx3Fn\n78+Ouxmwmd5nDNbKxW4TI+eltbbLu8ZgDLRaIVHUef/IPkBzYfYy4O3j/j8AfO79jYwxdwN3A5Qq\nVTbc9Ic4SUZFXyAwygRSAOIswQvk3EC1AkCz0SDOYmnny7VEX5wkpZTIz2ok59x2wrgv/VlXXrFa\nqwOQOS5prNdSOaYdOXqeQxRFAHRI9HkG15c+rJV2Rj9OGR/fcXny6d/0wq85MftEX/IDMslaew9w\nD8DgyDxr0oxaqYyfyMCTThsA13N0QBZX0WVypNqMerUGQFPbpwqvNMvoZNLeM9KHYzLKA8LcTizM\nS7sozmi1Q2mXyiv4nrDBZIZSqSTt9MMlNsPRVzWOry8pz0vSDFJLr5J4Lsw+AKw47v/lwMEPu8EA\nAZCGIRV9wcB9L2oSY/E9QW8nbAHgOuAqs/IP4WbSHgcCZVDUFsY6nsF3pF/Pkftmpma0vUsQlAHI\n1DzIFKnWMTjaf+DkHyAljfWjB8J0zxem+76Hi8FxerMz5mKNbAEuMsasNsYEwJ8DD86hv088nTay\nrbWJMeb7wK8BF/ixtXbnKW7CxCm1SpWyTs0wVMR5qpg8SycWURG2WzpIh7TTAWaVZjLTBODo+Bil\nksj2VJFdr9YpxdKuVBIUO6HI4NgmUJeZoKKeNFV94HrYRH6XA2FNxfGIddZlmSJbla1rDOYDGvvk\nNBcxgrX2V8Cv5tLHuURzYvZHJccYyn7A5PgYtiYKL6iI/GulgtzUyego0o6OHwGg5pUYKVXlekuu\noVbDqvp8zlsmqiPuCFaTJCGcknZH9h0GwCoCFy5ZQlu7aMY6W2qC9NR1cXWs1uRmpyVTBYwim1xv\nuA5lv4zTozTuu+sFUqHItlacFr8U0MkEhY1QEBojx3YnYrrZAGCgJmj2U8MFK84HYGl9PgBrV10o\n1/BoN0XG19SWdl2fqCHwbagVsmPHDgC2j+6ioRaKv2SeHHPLw/MYUJPRqo7otFsY9QUcRXsS6tQg\nwSu52KzrAn0oFcpsYwye55HZWaXTyY1UdRxKrkdn/BgAFb1UKQ9w08Yb5J+G2MjNY/JBJmdCdmzf\nqf0LMxYvXsxlqz8NwNql8pHWr1oHwFXrdvPoC78DYMfBvQBMTE3Ks4drJAMi3pzcqyQjCAK5Xs5N\nRlWoUYJjT+xwnIj6YqRAKhjZGn8wLomKEUcdmEy9suljxyhF8vvWTTcB8IUrrmLq4FEAHn/oYQCa\nDTEL540swqiTUa8PAXDoyDivvfxLAC4+X8TNknkLAFixYgXf/dNvAnD/U78G4KHfPgnAwsVLsak6\nTxoy6CQdwraIqSjNjUUhJwVrY9IeXcg+sgukQpGNNTjWwSZpV9DlJpmj3nc01uLaT18JwK2fvx6A\nZCrkp//1PwAsO09k8KYvfgWAS9ZfgVXMlDR+0mlH7Bt9E4C3du0BYPere7rPW6NO0Fc23gjAc5t/\nC0DjnSPUFw0Ds3LYuAavUtJ/5Gwci8x2fY/YGKzpTWoXa41kGXGrjef5hGonJxrNmzwsNvWwW+HO\nW74GwNReOff66Kt88fqbAdh0qzD5cEMUZFqr0dK+ZtQ+dwKfeetEfKxZfzkArz6zBYAnH/w/WpFY\nGjd+Sfq887avA/CzB+6lKRIJb0g+iD9YI1OvNVZRl6o8CNshtaBMrzmBvhgpkIpHdtjGKQfEnUjP\nanhUvbPrP38tVcTE+sl//xSAJUuW8O3b7wBgvCWmX6hRwz1797BwyVIAqoHGPJIMMyzIHFebePXV\n6wFYvnIl9/5I+n34kUcA2HjjFwB4YXQbm3e/BMCa1ecBMBk3u6HeTD3HPDaSmVgyFaaP7LOOCkW2\n6zrUa1WOzUx1v7IfiJAsD40AsO6itWx7YRsA09Mil2+85ctMa7LhZ/ffD8BYKFG/8kCdZlPQ/rmr\nPg/AQKXKE6PbAWiNicPyN9/5HgC1hYNc+jlRwNuefQ6A64IbAFi1ejXb9r8KwMSM3NfxMmpVGWOe\n0YlCMQXLbgmnd2D3kV0kFYrsNMuYbrdwfIfhgQEAZhS9C4YF2fXyAO9Mipk2vGgRAFdu3MiRWNC7\net3FAHyqLvdHScb2rTITdr70spxrR8wMiA4YWSjxjx/f+x8A/OUf3866DSK/H39EHKS3334LgM+s\nv5yHnn8MoBtTH1ywkImxMQDmD8oY8/RY1I57dtWhaDvbdfDqJbIkodEUMTA9IdN1ZU1exEYJNpF5\nuXzlKgDCNMVURGleec01ALQ1+LNr1y7WXXIJAJetk+P+vW/RHhZled55ougeuFfs9MiFeXUJcJXL\nEvM4uF/y1mvXX8IFy1cCsHf6XR1PzIIR+WBGE8V5pjUIPMnW98jxvhgpkApFdmYzpuOQIAjIOoLM\nbnQtEtTUgzK+moFx7pk5DuH7Av3tSEy6zVueZ9N1mwBYuExMwPseuJ9b/+x26W9wEACvJCje8+Yb\njCy7AIBlC0RMJQ2ZZfHUTDfSuHhEQrmT7SZ+RdNumjJzXREjmYFOmnywpOAk1Ed2gVSszDZgSi5T\nMzN4iSB73pDEInKHpDXVIGkLig9rnDkIAgY1nnGoMQXAFk0GJDZj1YVrAHjhZTkXG8uQInqmJdHB\niUnpa8X5y7uZ3kzd+7FDkjqrrL+UkhX8dTQcUK1XuhVTFUV4qInlxJG/XpFdKLO9zDC/5eLEDjNG\nmN0Zkum9NxEmsqiGDeTaeQOiyIIj4wwNyIuOjY8DML7rNQDmW5/hVDy6hep5Dnccolj6qGRaXRUK\nSxbbKkNqGLc0Q3/RSglu+ZUqB9+W0pelFy4H4KgNaWtiMvHlQ6TKNjfJKHfSbtnbqagvRgqkYmMj\nyLT3Ah+TyFxOtd5u374DAEw1Z5i/RBTXsw8/DsDo669y4Wc/A8DEpNi8eaTt2uuvJVSbeFpt8dBJ\n2bPvDXloq33c02GwXiPTvOR0R5A9GcuxU4LyAhE/xxoidtKK1614ak7J7HNdgXqaWRxzgmLPk1Af\n2QVSochOspSJ1gxDi+ZhpgRxmcY8ShWR3bv3vc6GNZKcjcsyvINTx7i4JL/PXyGyNNGo3/DgAPUR\nyYhfVBZF+dTzz/Lbzc8AMHFAnJONF18m/48dZvOjTwDQ1Jlw9c3XAfD4lmfZ35Bkc+oKXAeDIWpl\nSUqkWlWSqnLPrMUE/nFlzx9Op0S2MWaFMeYJY8yoMWanMeav9Pw8Y8wjxpjX9TjS0xPPYeoF2Qnw\nt9baF40xdeAFY8wjwLeBx6y1P9QlHj8A/u7DOnIch1KtzNTUVDdSlleldjTbMrr/TTZdJ+mwSzZK\ndO6R557CDIilccMmSWUFZTEVvWqVhsYuKhov+e63vsmY1hDWXZkxA+pEvf3SLtKOmIPXXL8RgBkt\nTdg78S6HwgkA1q2T2dVuNJk4KmivVsU66paG24w0icl6NP5OyWxr7SHgkP5uGGNGkUL4rwI3aLN/\nA37DKZhtrSWOY2ya4qt36Gp2vaL1Gq8deIuf/PI+AL7+5dsAuGDdp7jvJ/8OQDgtTLz8clGYy5av\nJNB6kYaGU4frNfKK4mRcPsTOl0YBePfNvaxdu1bu/dRqAH7xpASktu9+hRVr5FzHygcIo5AFg5K1\nT2NR6pEW8FSG6sx0Qnot0P5ICtIYswq4AvgdsFg/RP5BFp3knruNMVuNMVujODpRk3OGelaQxpgB\n4BfAX1trp02PGeX3rDwYHLLGGEp+QKxiI9MYx8CgKLnYWDa/IoF/tGz3ji/dxqVXCJKfeExSWVuf\nehaAWq3GFZ8VcTM4X9RGKwzZsesVAKaPiRM0paLg6is3sPJ8KcR88GEpwH16qyQRwrhFlMq4fFWK\nfqVEmGiRvZahVbUo1CIpsl550ROyjTE+wuj/tNber6cPG2OW6vWlwJGenngO0ymRbeSz/QgYtdb+\n83GXHgS+BfxQj/97yr4ch1KlStLudNe/BL4osNgTdHSyNmjs+umtUn4QzbS567ZvAHD7l/8IgGfU\nfJs6NsnRIxLbePqpJ/Mxc+nFouCW1wXt19/1HRmE7/Loc08D8MRmmR3TkeiBZauWE2scJAzFLLR2\n1vGyWnbhqdlJ5OD7vZt+vYiRjcBfAC8bY3R+8/cIk+8zxtwF7Ae+0dMTz2E67UWnp0ODQ8P2qo03\n4CYZvppLvi5g6mhgqklMI1L3e0zMsFKU8AeXXQHANZdJSmvBoEQLBweGmJiQdloJzPDgEJ7qgpmO\nIHRGUfnci1vY9pos1UxUiA6orE+xtDUSmMe/O1HULRkOPIlje4pkGyWUXJ9nNj/F5NTk73Ud5Eem\nDCkRrgUeVqNyrVDMqLzayPowGIg9O3+FxCmOvnOQ5158AYCdo5L9bmnodMOGDSxdKkkDNG0VtUJa\n74rJN9kSEbF/QjzJd5tTDCwQ5pa1/nsiaul9ETVVfpNTEmLNHENZ7etUlbqrxZc1v0Tciboh2FNR\nPzZSIBWbPEAiZFGSdZMHrlb9VzRWnBmYUeXUbAu6Fi1cTLumK8dUOTUmBE3b336DdddsAGD8iJQV\nv3X4AO+8uVv6nScOSVwTpbtwyQhtLcQPtBCzrXHtaj3AVVERqJhLsLPLqbUSatZhdLA4/bTY2UjF\nFsNj8I2D57mgiqitKTAbqn/tuQRa3F4ezGv3UryyZGryNez5Sq9Vyy9kYnoamF3MtX3HDuZXRd5H\nuumAr2m1DIeKVmElGnHMnRUsNLWOhe46G4dU9UsW5zH4vGAefNfpuZSh4PrsjCyMSF23aznkYsHR\ndSsWSDU4FavCS9K0uxLAMdK+VhIRUDIuVT03+rooz6GgypCWs3V0tYCjr9pohgyNqKepStZXZkft\nDgNVXcCkDIySBEfzksbXde2e7Y61FXXI+mmxs4+KXXSKoeL6pDbrVu+jZpOvosPx3K4Cyreo8EpB\nN7Pd1Ox6zZOZECQW2xKRMhhIm1q5wuw+ArrFhq7TqZiAdEbae3mBU3dzAJdYiybzVQaZzbCuYDJP\nh+VpMs9xwbNnLnnQpzNHBSd8LWkW4zhOF8mxQjCX01liZc8LwM0VGRmJtsuL6HPHgthS1pqTUJdv\nTIVNRkqSSDB5ctZqsYgxtHX1V47QHJee53Xrf/MNXTzHdJdTd9QbzYvjvXKFwHHoMejXR3aRVLjp\n57ouCbYbN84jap6W4bqO03UijCI8jWJiR5BZ1lW2jWlxx3GhragdSwR5015GSU3EfKVzvt7SGvBK\n8qz8OT6ze1Hl1NZsjHVsF+W5zPY1YllyXUyS9VwMX7wYsQmZobtQM1/13Z3m6ezGWmpxYc3sVHbV\nbs7Fj18KsMo8p65VU0nIAk085JtodZSRxrgkeTmZiqks35bIOt0l2blfaAKnu5w6QsSPye3sjJ7j\nItJnnwqjQkOsxpijQBM4VthDT58W0Ps4z7fWLjxVo0KZDWCM2Wqt3VDoQ0+Dfh/j7IuRAqnP7ALp\n42D2PR/DM0+Hzvg4C5fZ5zL1xUiBVBizz+a9tj+kUvcfjDHvGGO2699X5vScIsTI2b7XtlZ0LT2+\nUhf4GnAHMGOt/ccz8ZyikN3da9taGwH5XttnBVlrD1lrX9TfDSCv1D2jVBSzT7TX9hl/mTNB76vU\nBfi+MeYlY8yP51rwXxSze9pr++Om91fqAv8CrAHWIzXq/zSX/oti9kfea7toOlGlrrX2sLU2tZI3\n+1dEHJ42FcXss3qv7ZNV6uYl0Up/Arwyl+cUEs8+rb22i6WTVereaYxZj4i8fcD35vKQvgdZIPU9\nyAKpz+wCqc/sAqnP7AKpz+wCqc/sAqnP7AKpz+wC6f8BLsFD8+PN7yEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1648a3e1128>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEe9JREFUeJztnHuUXXV1xz/7PO69czOTmUxmEsKYBIQEiJEgsnhZDJRG\nMCj4qAprtYotK6VdonRZgVpbbVcXC5UWV31RwdcqKVZFLVi1DQZa6LKRoAhCgEQMIWGYZDKTeWTu\nvef16x97n5uACbnJxAOL3L3WrHPmnt85v3P2b//26/fdP3HO0aZiyHupX+BIojazC6Q2swukNrML\npDazC6Q2swukNrMLpGkxW0QuFJEnRGSTiFx3uF7qlUpyqEGNiPjAk8AKYCvwAHCZc+6xw/d6rywK\npnHv6cAm59xTACLyDeASYL/MrnZ2uu7e2fT0dJM0YgB8p5NLrI0TIRP9L8PtuZb/lgtH8wbwbII2\nBccTnEvtVOya3SbCC+VLrA3OkV/KZK/rdvTzZ7hm1zhg+3PbGBsb2euOfdN0mD0APLPX/1uBM17Y\nSERWAasAZs7q5fJrr+Xiiy5m+NeDAHTHJQBKmb5KFobsDpV5U2ID4nmIp7/VXKLt7NP8GKp+BwCN\nSK/RUSHJxgAoe/bcOAMgDMs00ix/u+e1SdOU2BRr3dc2TqCs40ZnbO2tm9SDWBwfXPX2A/EKmB6z\n9zWSv6GTnHNfAr4EMG/+AkcjpRpW8EPtOnYmsYneGpMQ+zoATnlNqRQyOTkJQKWzCkDqlBllzyOb\nqgNQr9cAmNlVoRTr850NgLPni+8oeb7+lj3/Q3Rm6H9+c3btMWxO9IbMz2eC4CP7ZMS+aDrM3grM\n3+v/VwHPvugdDrzYQQqJfajn2UeVlQEN55pfL54yqDFVo1rRAUgSHYG4rgzuqc4kqJYBWHj8MQBs\nHHyGmb6yqGHj7wWmrrKUzFRMlnfktO8sy/DsPi+1+yRrqp1cvaX2TEdGKuCkNbs3HW/kAWCRiBwr\nIiXgUuDOaTzvFU+HLNnOuUREPgD8J+ADX3HOPfqinQUB/bPnkMQZSW5kvOdP28QJWapTX0ykAh/E\npKc2tRuAakklvbOjhC8qmRse/wUA5Tmz6OufC8DOnTsBmNw9BUDqIsQP7Rue/w6B55Nlptvzlxbw\nXK6/tV0U5DxIcbIP3bm/72+x3T7JOfcD4AfTecaRRNNi9kF35gf09PQy1YjIvOe7a7k0h2FIwh4X\nDiCKI6Ipvd7X3QXA7tFRAMYaEY9v0Am1+tvfBiCtBKz60ysB6JrZA0Cl2mGP9Emsr1yL5oYyyzLK\nvrIkNJ0tmTQlOrP3STy9IUWAbI8veABqh+sFUqGSDSop9XoDz1w/P7Pxrqm0BYGQoucJeWDimGHe\nhDeq/vPCskpqb0cnpyz/XQBO6OkFYN2D6/mdYxYDUDfPY8fYLu0nCIlNUmvm2UTmgQRhmTTT9oEF\nW0GWYd4pkei1/H/nZZC5YnT2wZInHpWgTCAe5tURmDqR3IeNavoRQFjSa/0zuuhP9Av7Snrj5LND\nAIwNb2bLiBrBgc5OAP5o2Zn87A41JaedfSYAC3pUnaTlElsnlPHDiQ7qmBnYVGJiex8PfYcg2xOF\n7jnmekcQl7WqRdpqpEgqVLJFhI5SGT+D+qS6cJQtXM+DjiShp6JByuwuNYYDQYUNP1oLwAM/+TkA\nY9u26XF0J2GokhmaVNKIiVI9H3nwEQCq/bMBOPeilSxbfCwA22KNOB/bPaLPygRnobsz1ZJlXlPV\n+eYWulxGJcNL98zSA1Fbsgukwg1kIg7EUe2oAGDqspnXmFWucHRV8x+zLTey7pvf4YHv/QcAM1ML\nmRML1/t7qGURAJ3mFkb1OsG46uOtv96kD9nyFAA7tjzN2SvOB+C1K9SwBkcdDcDPR4aYTPVZiL4f\nnoDkLqK5g2Yhfd/HJQn7ThP9JhXK7AzHlItJBfJcTq2h6iTXAL2dFWZbePnTb6jfvP77/0Vvhw7A\njNkzADj+1DcA0LdoPiedcYre3KnqZ3LXLqa2qNH8n7X3AvDEQ6pOBnc8yze/9lUAxi25deZ7NGt3\nYncfm3ap/143TyVBaPrjplowlVevNygFYTP9eyBqq5ECqVDJdiK4coCIkJjaCMy967Ac9pzOTn55\nx48A2LDmXgC6HQwcvwCAFe97FwClgT4AahVYt+3XAEwNqzSWuiosWKy5kQuXvh+A5VufA+CfP34D\n5bEGAGu/913tu0tny6Lzl7Pd8t55jBlLgjNJbhrGLG/j8H1p2c9uS3aBVGxQ4wnVSgdpo07Jfosj\nNUjHvmohADs3PsVDa+8BwK+pETxx2Wt509VXAJDMUSl8ckyDmo5yN5WjVIondg0D0DlnLiM1iwQr\nqk97FwwA8LHP3cTqj38SgC2PPAnA/92lAdCy00+nx9TvbosW676Q2rlnhjFN9f8wDEmylFZjyLZk\nF0jFSjZCCZ+OoETdQmVboCHcrVL84D33MbZFF3zmdaord/rK86Bfzz/2WZXKXzy1EYCu7plc/l7V\nywNzjgKgVPe59davAPCrx3T9efE8de+u/8CHueBtKwG4bdPTAIw/p/r88XU/ZcH55wCwbUIDHSkH\nNGx9rpKve9r3BDgSlxWyLHbQJAgl8XFOEF9f2bf8ZrWmHzT86Eb6O1RVnLBsCQA9SxfxhX9Vd23d\nww8B8Nd/+3cA3H7b7fzb6m8BcOP1NwDw2Zs+w8P3PwjAlVeuAuCrt9wMwIf+5q/4/D99AYDeb6mB\nrD2n7t7/rrmb11z0JgC8UTWingS4HAGQr3sGurQQRRFB2DoL22qkQCpUsn3Po7uzi0YWNyEJvbZa\nPvaMTmWGRmmMjgPQPzAPgDSERy0SPP+83wNg6SKV+muu+xgf+shfAPDYM1sA+O8HHuCqVX8GwHkX\naLQYmQ74zurVbB1U9bH8HW8BYPWNnwNgVqWDZESlvBooa3YlCb7NQjI9ijmGzqU4L1cqB6a2ZBdI\nxeZGPA+/VGZsdBdZWbvu6NBFgIpTaQ7rKUf39QPQP08NXiQZS09+LQBfuPkWAN541nIA/v6mm5iz\n+DiA5vG5xhSVmTMBmLTFgM556h5OVX1GRN3NuXN1sSGzoCVu1Bl6VrOJMxdq+23RJH6gNoRMJTq2\noMYPfeIsadn1K3YN0kFPChN41HdbBNmpTInM364Fjp2inknXnG67M4ZM2+Nru+/fqXmT2uAQ8088\nGYDqlCX304BaczlFjZlXU4akExGeqJc/GmtuJOvWARcvJIusb+NMNXGMpdq3H2pyytV1AMuR0OkH\nBO0U68uPis36iaPuO6IkpsuyeEGOjDJvdXx8nGqPSvTw9h0AdM/vZe2auwG46qqrALjkre8E4PEL\nn+DLX18NQNpQqSSKmxCzRl3xIpGtqFcqlWb0Ws0/f1wlN/VpvteOSf0t8V3TQOYAzExyqJwjSuMm\nQupA1JbsAqlQyU5xTBDhVUNqoxMATOzS1fJXdaoRmjVvLpOGYhraqkjXua8/iakRNaADA5rjqCeq\nu3vn9bFzVGfAmOVGfJfglyzBb9nE4RFtg0tZ0K+Gd/BhdQErUzq9ZvZW6TlK3c0tDV0UTryM1DAk\n5NgWw0tlgW9Yv8OUzxaR+SJyj4hsEJFHReRD9nuviKwRkY12nNVSj0cwtSLZCfBh59zPRKQLeFBE\n1gCXAz92zt1gJR7XAdce6GFp4IjSqIkuqhlsoTQwB4CO+XOpTagUP7xeQ/NTz30DH3yfZv0++clP\nAXDuJRqQ3L3mHl538mkAiEnzey79fT7yib8E4PpPXQ/AHd/TkP6UY4+jYsHJfT9cA0DFvJPu2X1g\nQdZErMFNVqrg2dpdyVOJzheAI5cRHUQ++4DMds4NAoN2PiEiG1Ag/CXAudbs68C9HIDZIhCIR5am\nipYEdjTUgG0PdYoef+5Z3LPpVwA8vWkzAJObh7jgnAsAGDHX7LFRTbG++S0rWbFC8xlJovmMc5af\nyZSv7W76zKcBOO/1pwPw0fdfyUN3/RiAYYtaY2PX2RdfxIglncYNWRkHPthaaJDmmBI9pmmKC39L\nEaSIHAO8DlgHzLWByAdkzn7uWSUi60Vk/ejIyMF094qjlg2kiHQCdwBXO+fGpUWjsHflwZKlJ7sw\nhYoXMJaoa1Xu0mk7bAav7zUnEszRJa+K1Vzcfutt/EFZ1chlyy8CYGuiBnYqhKHtalB3Tepvx7z6\nWC5/h7qGV1x4MQALy+pObr3vQdZ9V5fdXKwSfZRFnrOWHM+Guj6jZq5jmjmCZE/NDUCuOHwRcK1X\nHrQk2SISooxe7Zz7jv08JCLz7Po8YHuLfR6xdEDJFhXhLwMbnHP/uNelO4H3ATfY8d9b6dDLHGUv\naEJyaaiBbIQaAlf7+njr5e8F4Ief/TIAOwZ38i+f1/MTTtUcyfLLVHLjOGNGpmF0NFuxfmM7RlnU\npxUoyYjOoKfWqZ6+62u3MyPIMSEqa+dfqovIU10dDA2pi+g6FBbhp0KHtffSvGJNj+JSXJT9RvXZ\n/qgVNfIG4A+BR0TkIfvtoyiTvykifwxsAd7VWpdHLrXijdzP/iE/5x9MZ1nmqEcNRoZH6TCMn7Ma\nu6lR9UpGpcLAkpMAuPCKywG4+2vfaAY442vuB+DZDeqxHHfySSx741l6zTyQIBTW36mu3uZHHgdg\n19PqeXRGQtShs+mK664BwF+sMInNjUlSQz9V0jwTmCElbV+PDTJs+tzzQwJpFQ9VOLASxPfxKqW9\noLn2Ip5O1aHRMXyreRk441QA3txR5d7blHnjGxUjsm1Qp/vQjp2sXaugS6yiLPZS+tQLpDamBi/w\n9fkLlizmnEtVBZWXHAPAZsvqDdamQAw3bioj9AJyjZeVrKrMBiRLkz348haonRspkAqWbI8wLEOp\nRN3A5p5JuG9IpMwJg6Oal9hlWLxFS0/g3df9OQDbfv4wAI/c9xMAtj61iRkd6ta5WN3HSpYxGWgk\nMmvBCQCsuEhX1BeesoztngZQ63eZarIAazKNqVrOWpK8blLIJK+hUTKhJxBBcIfX9WvT4aFisX7O\n4ZKUBMfuTCUvDPPadR33clgms6WsmqFZnxgeotuquCpLXg3AyjN0dSbbXWN4m5bQd5rhKmdCZbHi\nRMTC6VHDpdwztJnU4MqTgeVnTCkHM2YQWelxDg/204Q0d/maVdRW4YZPQkF1kIdCLs3wHYRmZDzL\nO3j2kV5Cs4g0L9/bOTFBZAzCyqh3W+5iZGSQ2791mz1LreI1V13NBGb0tqkh9QMb1JkzyHKjFqs6\nKZUNB5I5ImNd3iTMmnBKQoPn5KDLTDIyoQ0/ezlS4WokziJIYqomvWKlc3lGjSgBQxkljT1LWQ1L\n3OfFqXmFwImnvZ53Gobt5pu/CMDGbJJSptIaW31OZpsBpI2EsKH6oBrqtTjVa/UkgpJth2HJvACH\nZ2Lu5ZbQZmUsjuwg9sZpS3aBVKxk40hIyZKUgD21hAChYTcS32vmLPKNAjzPI4oMZ1dV3T1u1Wbb\ntw/TP1ezu/OP0yowmVHFoCH4BmUQ07cl36cU5pAHPcb5s0shtn9Ls9ZRHGCzLy+7zmVUxGu1wmOv\nu9pUCBUb1HiClENdhLU8Q45mbVgxYRw6guYmTHvA515JJbSeqvuSb9YSlEISgyLMKumicTCVUCnr\neWLopZodcQ7P0gFeDkmwZ4oEeLalkdesDHNN9Gqa7/qTz0rPx3Oty2uxuBHnqKcRMVlz/xDf8g2R\nqYxEILWPD82ouTTFM/UxadUI5XwfqXqNqkF4K6aSqjFg1QI5X7BBTTy/yfgwLzAN8q17ILMqsTws\n9J1r4kXyRQMxA+k73Qel1VXIthopkA55X79D6kxkB7AbGC6s00OnPlp/z4XOuf4DNSqU2QAist45\nd1qhnR4C/Tbes61GCqQ2swukl4LZX3oJ+jwUOuzvWbjOPpKprUYKpMKY/XLea/tFkLqfEJFtIvKQ\n/a2cVj9FqJGX+17bhuiatzdSF3gb8G5g0jl34+HopyjJbu617ZyLgHyv7ZcFOecGnXM/s/MJIEfq\nHlYqitn72mv7sH/M4aAXIHUBPiAiD4vIV6YL+C+K2S3ttf1S0wuRusAXgeOAU1CM+j9M5/lFMfvg\n99oumPaF1HXODTnnUudcBtyCqsNDpqKY/bLea3t/SN0cEm30duCX0+mnkHz2oey1XTDtD6l7mYic\ngqq8zcCfTKeTdgRZILUjyAKpzewCqc3sAqnN7AKpzewCqc3sAqnN7AKpzewC6f8BC9/tI4eaYzUA\nAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x164d41a8240>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEQpJREFUeJztnFuMXdV5x3/fvp3LnLn7wsXEYFpiFUKNGkBKIJeSqFGE\nklKpVfNQtVIl8hKplfrQqE99zEPbh75UomrUVk1bVUqrJmqkBkUhJEAplCAgULABD7YxHo/ncmbO\nZV/WXn34vn3GJjZz8LgbC59PGu0ze6+9Lt/+r++2vrXEe8+E6qHgg+7AtUQTZtdIE2bXSBNm10gT\nZtdIE2bXSBNm10i7YraIfEFEXhWRYyLy9SvVqQ8ryeU6NSISAq8BnwdOAs8AX/Hev3zluvfhomgX\n794DHPPevwEgIv8MfBm4JLOTZtu3p+cIgoCytI8sAkD10fUbGl2AA3/BRar3zisUBDpRy7IcPS/L\n0p7J6D1BfxfOXfCec44wDN/VH9nuwejedh98WdLfXCMd9rYLXoJ2w+wbgRPn/X8SuPfdhUTkYeBh\ngFZnlk//xldptVr0hykAQZgAMEh14K3mFGADzrcH7IpCK7SPlCT6XlFkIHqv2WoBkKYpkigD+/0+\nAO1mUwccRcShDntjYwOARqLvbW1tMT07C0Ce5wDEcTxibpoNtO0otGch6XDIY9/+y3H4tStmX+xL\n/pxM8t4/AjwCMLf3Ri8ipHkBfhuFAO12G4BAQipVkqZDAJK4ibP6KuSVZVHVjxhqsyyzTjiEWAco\nei2rCkQo7WfF5CiKrO5o1J+qnX46xHu912poXZWmy7KMMIq2ob4D7YbZJ4Gbzvv/APD2Ti85D6H3\nBIYOAr1W07UoipEokUhHFcUBYl2N7L0s1ZnhymLEjIpRjUaD4ZYx3pgsJgLKQIjs40y1pgAI44oN\nQppn1k5kdSX4qpJKTIm248SDzy8QZe9Fu7FGngF+UURuEZEE+G3gO7uo70NPl41s730hIl8D/hMV\nst/03v/sPV8SkCBCgpDCZG8caBdSEwEQUJYqL5NIp23h0tFMdYbe0gRLGAd4rw8rRRdFEd5pvaWo\nuKnktHhwztsY8mosWrdzhNZQNYN8WdIfat/EdEMUm0LF4/1FZOclaDdiBO/994Dv7aaOa4l2xez3\nS0EQ0m5Pkef5SIZi8rnZNORJuK3oDP2lCHGkiMvTCmX6ehSFFGaplKUpyiInMZmbZZU61DLOQRQn\n1h9FaBDqe40kZJgVdk/fytIhzukM6MzOaDs2qwa93ki2jzX+sUtOaNdUK7LxHu89rvCEJhMrVLbb\nHf3flyO0VAjPswFhQ+3kSm5WpplIMJK5sSGWQGjYyApDpbNZUhQFYaRmZiV788JmTZ6PLJvU7P4w\nFDrTVt6so2GmdSIhpZd6ZPb7pdJ70sGQ0gtx0ABg0FdHoSzV+SjZnt4VdTodQpvqUpriK5QZee6o\nJmhl+sVBgzJQGz1IKvGh5X3pyEttMw7b9p6ztouRSMmdmpZxHFOaZd7tdq1cpUSjkbIdhyZipEaq\nV0GK0Ipj1T6m4YYmFoZDRWKz2Rw5OFFDPby8yOn1FI2BTfNQFCet1hTz89NWvw4nSRIk6l7QdjX1\nB4MB6VB/R0lo91Rclb4Y2XGJOToSBOSFla8UapRYeSEv07Ftvwmya6RakS2+JChT4rBBajK0NNRM\nNdV1LkVGyO71egA04pBGQ2W8WYBMTyuah/2Ut98+OfoNcOrUaYLm4IK2r7vhAAAHDx5kanoe2FbO\nPQuKhVEymmGVw9NsxCTmEHlTkLn1Pc8KoiC6IDL4XlQrs8NAmG0JWTHAWSAqDkxxOZ3Kjcb0aMpX\nDB6mffZ05gCYm1Umv/zSCwBsbfTIzWqpxEMcRJToVK8U4xuva4DyzTdPs2f/dQDceeedACwuqrg6\neeIEpUUaI/NeB4PhyCrqzGjbVeSxEcUQjs/siRipkWpFdrMRcfjQPl47dpyyspctVDq7sADAeq9P\no6Emmbfo3MzsHqZaivInnvgxAN31NQAWZ+fZs6Ax6EaitniWppxa1udhoPfaLcWVDyNOn1oFYHn5\nJwB89jOfAqDTWaDIVYyE1r+SnMCM9jBQtDunZqoIZGlvZDruRBNk10i1IrsRh9x64wwRe3nh5TcA\naJkX14hU3oZS4EtbNDClGUUBP37ycQA2zq0AMD2lcvbgzdfzpQcfBGB9VVde/ukfvkUcKqJzc36i\n2DzQIGJ6xkw3c04e/YHW/an772PGPNnuhqIfcTRtlSfNDPW2khQlCRJEBDIeZifIrpHqNf3E0Qi6\n3Hn4Bnr9dQCOLimCSrcJgHeOQmw9MlJkH3v9VTa65wBwXi2DT973aQA+ce89rK8tA/Af3/2utZPS\njBT5tqiCM1ncdx6JVP432zZzTNY/9qPH+bXP/SoAcaJlitwjZvqlWzr7JFTZnQ8zIGDcBIV6A1Fl\nQTlcJmzDHYfV7i0tlnlypYqRDGha0MlZDGOtu8xtt90KwIEb9gHwwOfuB+D0yZMMNtRbPHLkMACD\nXp+tLRURa+tbALz46lEAmtMLFPYFekN9VpjZmZcFSyfeAmD/PlXYZS4MzfQrjV0t+ziDYUoQj78G\nOREjNVKtyHZFxvrqW+RFj9acIvv2wwcByF99B4DUx6RmYp3tmvkWwtmV0wDcc/cdAGSWVuDyAfuv\nU4/wwA2LAMRhiHhFX7dvynZKh/r9Hz7B/P4btU0b/tS0KsV2u0FvoGhvTWn/8nRIXuhMKMzEG5jH\nSRgRSDLKQ9mJJsiukWpFdlFknFs+gReBWNEUTKkD89HbDgHQ9yfZUDBy9M3jAORFSp4rmp566kkA\nFudUue3ZM8/S68cAePnFlwDwhWNxfg8A93ziPgAeeugLAJzb7LLWU0W30beF5cppCUO6va711Zba\nYGT6OUN45vQ62+mQFeM5NFAzsyMJ2dOYo1jrMz2nTXfXdNrOzmnsY1+7Q2Axy35XA1Fx2GRgAfxT\n55QZ//b9HwHwpQe/yL7bVLQsrWpd//300/gX3wSg3dkPwJEjGte4/+57eezJpwDIbRVnM1f7vNdz\nI0vjTKp1RXGENwFQBaJi82xLn4HPGTfGOhEjNVK9pl8QEiTTzHTmaEyradXMdIoeP6W28n89/Ty3\n3nk3AJ0ZVXxp6SmqzClb9T79jnqSf/v3/8gDn1Wb+/AdRwA49sYSEqqYet5Ey0d/6WMAFGUJVUZU\nR8tsWs5fu92mN1BxVUUct1bXCWL93bJcwswUpHMO51wtGVETep9Ubzw7Tpi+7hCzC/twgSq4pbfO\nAPDTl5cAcGELMc9xbU2ja33vyMzsarZ1JsQtWzzIhjz+1HMAiH9G7w2HHFrUmbNZyX1DcWchI26p\nUt4abC/FAaxsbI2yq1pNRXMPEItfh5XMjlWua+pzycVzTH+edkS2iNwkIj8UkVdE5Gci8gd2f0FE\nHhWRo3adH6vFa5jGQXYB/JH3/jkRmQb+R0QeBX4P+IH3/hu2xePrwB+/Z01BjJ/aR5Ys8tIrGvV7\n4VV1j32iMelyUNBPLUV3Wu8lYchGb8t6rPjInJpmaRkQVZlQlv3Uas9y1lIkGi1FLTYjXDdikFfu\n94UJ+Y1GTGYZUW0LGTSiiNRmAK684Jq5AgJh3N0bOzLbe38aOG2/N0XkFTQR/svAZ6zY3wGPsQOz\nHQFdmeKV15Z44X/VNJOoEgdWKGlwelWDTgduvgWApVNv0+xoua1UmWirVzRn5uhb4rqzXOw8K0cR\nqJYtZa1saaBr6D3rliA/sPcq0y4KQ+b3zdlv/QBxGFH4of02QWDXwVZKuzU12tWwE70vBSkiNwN3\nAU8D++1DVB9k3yXeeVhEnhWRZ7sb3YsVuWZobAUpIh3g28Afeu+74y5ynr/z4MAtt/g3V7d46egS\nPlEllZe2Ym2KSRoNnE3veVNy76yucG5FTcOwqYF/b+aYEyirnQNNtQsDidi0mMiMhUeXN3s2YMEZ\nMkMbfmaeoStybrpB4yYrZ1VxF9mQRlxtKbH0Y1OQcSMhTdPt/UE70FjIFpEYZfS3vPf/arfPiMj1\n9vx6YHmsFq9h2hHZohD+G+AV7/1fnPfoO8DvAt+w67/vVFc/Tfnpa68TtqbpbaljECWKktCQmuUO\nX6gAn29p9w4eOsDGUB2P5rSahT5WFOceNnoqgzc2VUw12x3EZkBqsjfs6Ezqrq3jbFImZvKtL6uD\ntLi4yMK8KuX1sxpxTKKYhs2OgTk8RZVTGMdsdLujZMydaBwx8kngd4AXReR5u/cnKJP/RUR+H3gL\n+M2xWryGaRxr5Cdc2mp/4H21JoKLm/SGQxqG0MLMPF8qml0QUDqVt0tvaTTvrl/5ODlqOZw4rXuk\nVg3FUbNBnGj3kpahOe+PskvF8gU3bcNTP8tHTk1hifV796tuv+uXj3DujOkGXwWbCoZDldVV4k6V\nEeW8J242kDEXfGv1ID1CGYT4ICbLtcOxxSDy1FJ782K0vyYwRXT8+DF+4ZCGYKOW3hse1Q+xlWbk\nFhFMEksTAxpTKiJ6tr3vyaeeBqAwxQkwY5HGj91+OwBnzyyT2keJnCVuxgmx1Zum+sFLs6u9CGVZ\nTmIjVyPVi2zvKVyJ9zIy77yheMuU3GAwGG1InZnXCMDKyspoYXhmQdF4yJC+2t1kdUNX6pdXdaV+\nfn6OQsxcy3XGFMH2ZtL9B3T75kcs2fKcmXm9Xm+0EbU0L8s5h1TJ+dXVFKKEAe3mFEG1AWcHmiC7\nRqo3no3uuHXObafmFtVeF0MLIWUFIJONnZlZhoa05dfVzV/cpyswt958iAVTlnv36vWd5RWmpjTK\nd71lrCa2euXTlJY5KSsravIVmT4UCbe3axtagyAisfJhU/VFais8W8OBmg7j+XcfgBjJhrjMkVsA\n3kxWIsvmj5JktHrdN2thKohYW1VREVro89RJtUqCs2eZW9D1xtgOHTj0kYM4Y1psDTirq9/dpF+Z\nxa7afq2iIwxiIkvOqXZQeyej1fWwKaNyAD4bjr15CSZipFaqOW+kYOPcOcIwHOVahLaVYLT53mWY\niUvYUKSubXXJK4VqNm7DUscK5+iuKeoDS9JMB328DU0MvaFBMCKwtLHtkyCqjazeC5mZoNXGVHCk\nVZ5IFd2zbYVhGNrxGxPT76qjmhMrIY5KvPejr+ylOihATb/CBwRmfrUtU8kh+MgUqEUbq4XfgO1T\ncsIqxaD0I2UwbejtWc5fkiS0bImsOvglaWqZYZaSV8tvls/nvFCS2XNVjEGlaALdcDqu3J4gu0aq\nfQPTVLvBMEtJbfkptyWqzGIZQdzElD257dhyPhzJSx9UW6D1/VDOO65iZELkZJbem9mS2axlXqWD\njMiOPKqOJvJiMhkhtmdiMe9iWGwvn5kpWlhf4yiG886c2olqTj8rWFs/q/kW1kFnXag2cnZmOwSx\nBYqMeUEYEoUXxicSO0IoiiJK+2CxrU8WCFNN27BqsZHUGFQ6T2HiYKYza88s1cxvf8TAvNg0y2hV\n2wJtkWKYV8cjqegpx1yDnIiRGumyz/W7rMZEzqKpGCu1NXr5tIfx+3nQe793p0K1MhtARJ713n+8\n1kYvg/4/+jkRIzXShNk10gfB7Ec+gDYvh654P2uX2dcyTcRIjVQbs6/ms7bfI1P3T0XklIg8b39f\n3FU7dYiRq/2sbcvouv78TF3g14HfAra89392JdqpC9mjs7a99xlQnbV9VZD3/rT3/jn7vQlUmbpX\nlOpi9sXO2r7ig7kS9K5MXYCvicgLIvLN3Sb818Xssc7a/qDp3Zm6wF8BtwJH0Bz1P99N/XUx+7LO\n2q6TLpap670/4713XjMn/xoVh5dNdTH7qj5r+1KZulVKtNFDwEu7aaeWePZlnbVdL10qU/crInIE\nFXnHga/uppGJB1kjTTzIGmnC7BppwuwaacLsGmnC7BppwuwaacLsGmnC7Brp/wBtaTlWsiNH9AAA\nAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x164d41853c8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAE4xJREFUeJztnHmwHVWdxz+nu2/f5d17374lLy8JSUhIQgIYwCwOaMVR\nFgGnBkTFYXQspywdEXTUmVFHR52ylFHHscRBoVhLGARHUEqBCIqYAkIgSAiBQJaXvLx9ufu9vZz5\n43ceUgyQS15sUub+ql71fd2n+5z+9ff89nOU1poGRUPWGz2AY4kazI6QGsyOkBrMjpAazI6QGsyO\nkBrMjpBmxWyl1DuVUjuVUruUUp87UoP6cyV1uE6NUsoGngXeDuwHHgXeq7V++sgN78+LnFncexqw\nS2v9AoBS6hbgfOBVmd3e1qb7+voolat4XghAreoBEI/LULLNaYZHhgFwE0kAYo5LKp6QAds2AL5f\nA0BZCqVmehDg6FCjzUmF9KNmMKUVKMu0ljahuQ+leNmjsNB/fO7MOScGQDXwGBw+SDlfoFapvHjr\nq9FsmD0XGHjJ//uB01/eSCn1EeAjAH1z53LP3Xex5fHdDA6VAdi3dz8Ai4/rAmDjO9fyn9//LgDz\nFy8HoLt9HquPXwFAZzYNwPiY3BdL2diO4YIO5FDzqBiGOEEFADf0ZTxeDGVlAagaKVq15YNr28HW\njmknz0pYPpa51zPdJDtkrLumhvm3b3+N399+Z10Mmw2zX+lL/j+ZpLW+Grga4LgF/fq+e+9kcDRg\nwfGnAdDW3QrAc089CsAjvwsYHRgCYOWSEwFY2NvMlod/CcCpa04BIJHOABBoi3KxCkDazISap1E6\nBUCo4wBUA5kJiXgcL5RhhqGgPma78r+yCEJhcsyV1/NqPnFX2BTUpJ9Ywqi6SkBnogfHih2KV8Ds\nmL0fmPeS//uAwde6obk5wzlnvY0SLWQ658pJ83mOX9gEwBO/38z8jh4AjuucA8DUvr0s7u+QZ7SI\nOJksCvNcJ4sXCIN8TxjkOC7anAuUMCKMCdNzfpl43HRaE2bjmaFYAVob8WQY7IcBAYLsICbHUAvT\n/aJHdShAe/XpvdlYI48CS5RSC5VSLnAxUN98OkbpsJGttfaVUh8HfgXYwLVa6+2vdU8YaCq5CmG6\nxM5dTwKgzLTtNbL41NWrKIzkANiw4S1yX7FMrjYNQBAzU94RdOVqISkjPgrlSQASMR/HEZSXfFGo\noSVixQurWL5AOWnJNa8iaLbtGLaRCJVSUc45Cm2mX8XIfytuZpBts3fXPmrVWl08m40YQWt9N3D3\nbJ5xLNGsmP16ybIcUqkuqjHoaZeuXUtQ0maLwiNXY+3atQAUpwXN+DbTeWlXzAsqa0lBeLqjhSZX\nEJq02wHIJCFfkFszKfNYMX6oFAu4vlwsjYsiVkZRtrZ0Uguln3RKLBbPq2HLxMENRV94gVGe2ieZ\njaHsQ1p98v51tWrQEaFIkR2GinzRIhlLoKuC2piRwXZcrIWWtjlkO8RScRJyrVSo0NzRCUAmLfgo\nSXMGcz4/uvmnAIwNiJzV5RjVmLSvhoLiTKug8gMX/yULO0Q/tDeL1aNCGUsxP0k6JbNDeSK8O7IJ\nxqdGAUi40mk8Jqiv+FVK1gQhQV3vHymzx8bHuebGm4nZDqW8KMGYLcrHtY0tqxVV45yUA8M8rVl1\nkoiWnvlie9/+m4cA+N1zz7BgyZsA6J+3AYDx3SFb9gqTO3rEOm0PhXmXfuYnLO3vBuBDl4gCXntK\nMwD//s3vkkVEUqomx0zMJZaQ3zv2PAdAxbiSbiZBMmVh1SkfGmIkQjrsQNTh0Eknn6I33f8Qrh8j\nbtxilCg8HzHlvDAk5opWCyzTJg6/vl88zN9sfgqAkze+C4A5azp4eKc0u/FaaZOotTFuiznW0ira\nLZkUETA0PEUhL301xaXv8zcuBuDDF7SSKQo/umyZXapcwXGM+WgUoW8gevH734cbj/PAfb9icmLi\nkFqygewIKVKZrSywEtCkbHTFINo4NVZc0OMSEo/JsMpm0l194xbecsapAHz+7XK89RfPA/DzO2vc\nvdVECTJtAJx/wSIsY66ljOk3cFCOd9/vEaZEZudCUcA33SPP6mhOc+lGke22kvFUaiXiySbzBmIi\nZhKibBf0dDE6nEOF9Zl+kTIbQClNoTSM5ZsQayCTy1LGKnEDTDiDzY/LdP/hHVvZUxA7vL9NLIdU\nSsTEb3/xGF7yZACa5ggTH9w6wcIusaEdYySPiT5mOn+A5jkSnCxVpKOKNx+A7133S1p8iS5+6Kzj\nAIgnUhTKoqgtEyOJuzL2ttYMpZyPVaeGbIiRCClSZAe+z/TYBFk3JG6C+66ZksXQxCtSafKm/ddu\nuBWA2JI1PDowDsBln1gHQLuZuaetW8dFn7gWgGnj2WmvjekDYvqtWNEPQKZJZk7cGsc3Nn5oSSSx\nta8FAD9YyE0/fQSA5b0S+n3b6laqeUF2Km1kk8HodDFPvjpJoP263r+B7AgpWgWJImE5OCFUSoLf\nUMsx0SWORQWH6zY9BsCT04IFpztNX1auT5rYc3tMtOeS7pBvfeFCAL74X/cAEEy7xJaIXN4zJu2P\nWyjHefNW8/zQiIwnJYgNHFGAmf5lTD4vwn3zkxMArF/WSiYuiPYrEmBxjMKcLhQIU9QN2QayI6RI\nkW0rRSZuU8pVycaN4xIKggo5ySm66blseU4QFLQdL/e1dvPMvl0AfPYrvwPg+n8V1zxBwLpVgvrP\nX3YWAJ/54o0MpMVNb20ReTz4hMjzqeFB5vdLDnFgQkzG6ZIg1WmfQ3aumJY/f/A+AC6/eBG2Epns\nvJj9ErMwV6pS9EsEJmp4KIo4xArJuMLJpijlZX6nkmJnt6YlALS3Cps2i90b9krswq9o4jGxobft\n3A3AV6+S48cuXUhbVkTKupUSYLrySxfy/h/9FoAJT1JrLnL/Cf1dvOtMMSP/+/pfA5DXogxzsXZs\nWzzNsUlh4JM7K6xfIlwuTYuIcRMmd5lIUhkZQ9fJ7IYYiZCidWq0hiBgLDdIImk8xrZWMxKZ2rkJ\nKFTlWsyX6dubTJJwBV3jpsTgxw+IWLHSST79QQmVJkw361e1cMW7JQt/1TUPAJBpXglAm6c5900i\nWtYvPR+Af7jyZwBMJ30CT5DtGaX59AsjrF8m5qNKina2TbKiVi7hBi+pSTkENZAdIUWLbKVQMZue\n/g52DzwLQDgsSE1mRaE9tb3wouvebNDvTQxSKYqZVjMxC9XVB8ANd22jPCqBj69cIW6760zw0bcu\nkD5H3g3AT362FYDWPkiVpc/OtIQDumOiP3ynRD4wZmCrJB+2PrEbfa4gu+IJu2wTIzk4uI+UpdD6\nKFSQtUqVvTt2sX3bQ+x4Wl4+bBEF1veWiwBoOv5UXC3e4l90iRg59/SlaF+U03ZjI9/wgCjR1Okb\nuO3ZZwBI/q9YF5f/9RzaeAGAK94jMY6Bx/YAEBvN0xKKSNGIRaQ98SRLozWa5TvjGj+gpTaOKknf\nleITALS2S/uz13bS5MGunfWxsSFGIqRIkT0+OcbNt1/DxjM2cOactwOwbY+gsXnGnFLQE5dpfs4Z\nSwHoSf4xo73uOAn0X3+PZAyc0CWTkml+263bAMjUmvjC+wTRFZMe/N43PgjAD759K0FZxIfTJCo1\nNOVjtUChTG6zaqKSS5cuZWpcYimTpp7FK5cA+KdPfwrXrnHLA8/V9f4NZEdI0SpIKySMl3nwkd/T\nFBNFNFNoq/YIKk+cdwLdjiAnbVA5OTGOb0k1ko6Z+hITsJ8cGaKvW5RrIrsegFtu3cLZc6UC9tR1\n4iwFJqZyyfnvIG2SzDOxOi+QX1UVUjJ9xk08pGfufAqT+wAojIgcV1Vh24GJKTqbyoRepb7XP1QD\npdQ8pdT9SqkdSqntSqnLzPk2pdS9SqnnzLG1rh6PYaoH2T7wKa31VqVUBnhMKXUv8LfAJq31180S\nj88Bn32tB/X29vL5f/ki+DEwpQVhIHGQA8YVjmWgu0kE59SUIKZzfjueSW9t3i6orGm5P6mnKY9J\njd/EsFggTbECl11+DQCfuPyjAMyfKzNiaV+KeFxiKTO+SNzoC0WNmoFf1ZfZtWhlGmUih4sXiW5w\n7OpMI2y/9AqF0q9Mh2S21vogcND8ziuldiCF8OcDZ5pm1wMPcAhmh4GinLNIxuN4FWFkzCip7lbz\nlqkUGzZIguAbP7gJgOTCNXQslXqRZ/ZJ/X3NMOyE/iyXXypMaAlXyZjL42RrFwAwXRDlljVx/50D\nT9PZdxKAyedD0Sg85ZSp1kRhrzpR7OyyDzZyPahJ0KxWngLA9ks0Oy6h/hOUnymlFgAnAw8D3eZD\nzHyQrle55yNKqS1KqS1jYxOvp7s/O6pbQSql0sDtwCe11jml6vuaL115sHL5CXrP7nH657mMjsrc\nzE+LCNj8sDg5i1dv4JwLJNp39SZxXIZUnF2DImZCV+IamV6B6vMv7Oa2G0VxffmTkqyN00ZgFka0\nI5FAox9pmtdGISX3lsw5Oy6OlVOtUMr/AYCzzz0TANcuMjUtTlbCKOmuHkksx3SAXyhDnSsP6kK2\nUiqGMPpmrfUd5vSwUqrXXO8FRurq8RimQyJbCYSvAXZorb/1kkt3ApcCXzfHnx3qWQP7B7niM1+m\nKR1jfExWhK1eLrJ48cITADg4MELLkODwbz54HgCfuvoujCVGIiGmnI0pbgy62fSQJB4sLTNhzZo2\nnIQo2UynaNaSUWKZVJKsgfmYUXz5oqA/zE3wno1iMr51hdzfVpiiu0cKPeNNYmJWzJzYNzDA/J65\nKPvIralZD3wA+INS6glz7p8RJv+PUurvgH3AhXX1eAxTpLV+HR1d+pzzLmR6ehxMrNrLSTRvxTJB\nVLHs0btaqlJPe8+lANz3wijf+bGsFuuYI3Fqqyox7Da7nbFBsyTDNSUKsXGaTKnwhEnSFsyiI9vy\nSRhXPOsIorUnYznj9OVc9gFBb/i8zBZndBedWbGYKiY9VjMF/KlUip6Obta97Swee3zbn3Qd5Oum\naq3K3r3Ps3jhYlqzYvfGlbhs2598HIDWlm6eflTKga2MyI6/ev9F5AclTvLzTaLAbFsY5rW00NQn\n3mgxIccwNQe3LIzpNIH+ZuMvJgB7StRLZXAHAKcslSjeh8+dx4O3bAbg2d/cBUA2nAAlH87JyvOn\nima9pe/gFcrs23OgrvdvxEYipGgTvgTErWmaUyFzOgUlppKX5UvfAUAu73FgVMSCN7IHgNu+83U+\n9rF/BOCUnkUAXPl9ESvTxVGaT1gGgJuWZx4oTFPLidJasqwXgF079wKQ9X2sUXGMznuzFJNccq5E\nEu+47ib0hDgs3a2SID5x6QqSaZkVRbMq7MB+EVd7n93PyOBBAq9REXXUUaQKcsXyxfqWG75JMpZl\nZEjsrrJxlfsXSCWpm8xSMUApVESpHRweJJ6SONei5VIvMuGL0rr2zkd4fEg80/1G8U37sLxPHJxa\nWfpxTKZnYTbJJRvfDMBKAT3euFx79MF70SXpM2vW87Q0x9l7QGIuT22XjFDexGyUr2iKudzwq9sZ\nGh89pIKMlNknrjxe33nH9xk7OIJjAkmWidbbrlgGlcCjvUcUVm7SxCCURWCWLAfGW3PaxItzWudj\ntcoztglP8GyYqYE0mzdg6iTJqpCJ3RLs37V1i5xz5P5KbpqDeyVrXzVWTDqdoqtHvkpXr0Qkukxp\nsuumSMSTnHfxxfxh+/bGyoOjiSJemmeRK7rMmbcMy3hxcVsQPWVWhdpUmBoXsdCUElGRVGmchAAn\nZyJufkHMrcArQl6QdlJavMp7N23i5rvFdHvTqjUA9J8tCnhocg8JV+C+Ya0o1rakINYrVrECWZXm\nl42oUIqKCbfmKzKu8WmxwWtWnHTLHLywvqV5DWRHSJEi2/NDhkbKDOwZwSsIkmvGQUhnJJqXak5g\nmU0Ahg+I81GdruAgsjpm1t7ka6LUnEQCxzgzCVeU2luXteOMSmx7alxmQocp7YjHmmhqNXuVGEQO\nHJS0Vy0XoM0OP8rsT2JbITXPlDebpIGvzXogy2VqqkwQNGr9jjqKFNlahwR+haZUhlRG5Gw6aawQ\nT5BU8SsvZj46jOPjdkJ3m8RCqjVjlZh9nqp+GWWZOEtVZGtLMsbokDgena0yY8oFkzobG2V8wshY\nk/gtl6RvSydJJcwOPDOJaOWDLX25jomlIzPI1za+D1admZpo67MtSCcVaTeJMotOa2a61nwRK8qx\niDnywuWKMMjTVXbtFuUUhqIElTJxEDsgCEWkKDO9O5q78E3SIFeW5zdlzJqanIVnVo0qs11Rd4f0\nZ9sxasYb9My+USpmEQtNRr8oYw5LIsrsoEbc9eoWDw0xEiFF6tQopUaBIjAWWaeHTx3UP875WuvO\nQzWKlNkASqktWus1kXZ6GPSnGGdDjERIDWZHSG8Es69+A/o8HDri44xcZh/L1BAjEVJkzD6a99p+\njUrdLymlDiilnjB/Z8+qnyjEyNG+17ap6Op9aaUucAFwEVDQWl95JPqJCtkv7rWtZWfZmb22jwrS\nWh/UWm81v/PATKXuEaWomP1Ke20f8Zc5EvSySl2AjyulnlRKXTvbgv+omF3XXttvNL28Uhe4ClgE\nnITUqP/HbJ4fFbNf917bUdMrVepqrYe11oGWVaU/RMThYVNUzD6q99p+tUrdmZJoQ+8GnppNP5HE\nsw9nr+2I6dUqdd+rlDoJEXl7gL+fTScNDzJCaniQEVKD2RFSg9kRUoPZEVKD2RFSg9kRUoPZEVKD\n2RHS/wH6TD7fOX702gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x164d460cc18>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "### Load the images and plot them here.\n",
    "X_new = []\n",
    "y_new=[13, 4, 5, 34, 38]\n",
    "print(\"New image labels are:\", y_new)\n",
    "for i in range(0,5):\n",
    "    image = Image.open('test_images/test'+str(i+1)+'.jpg')\n",
    "    plt.figure(figsize=(1, 1))\n",
    "    plt.imshow(image, cmap=\"gray\")\n",
    "    X_new.append(np.array(image))\n",
    "\n",
    "X_new, y_new = preprocess(X_new, y_new)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Predict the Sign Type for Each Image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New image predictions: [13  2  5 34 38]\n"
     ]
    }
   ],
   "source": [
    "### Run the predictions here and use the model to output the prediction for each image.\n",
    "with tf.Session() as sess:\n",
    "    saver.restore(sess, './lenet')\n",
    "\n",
    "    prediction = sess.run(tf.argmax(logits, 1), feed_dict={x: X_new, y: y_new, keep_prob: 1.})\n",
    "    print(\"New image predictions:\", prediction)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Analyze Performance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New image accuracy = 0.800\n"
     ]
    }
   ],
   "source": [
    "### Calculate the accuracy for these 5 new images. \n",
    "with tf.Session() as sess:\n",
    "    saver.restore(sess, './lenet')\n",
    "\n",
    "    test_accuracy = evaluate(X_new, y_new)\n",
    "    print(\"New image accuracy = {:.3f}\".format(test_accuracy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Output Top 5 Softmax Probabilities For Each Image Found on the Web"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "For each of the new images, print out the model's softmax probabilities to show the **certainty** of the model's predictions (limit the output to the top 5 probabilities for each image). [`tf.nn.top_k`](https://www.tensorflow.org/versions/r0.12/api_docs/python/nn.html#top_k) could prove helpful here. \n",
    "\n",
    "The example below demonstrates how tf.nn.top_k can be used to find the top k predictions for each image.\n",
    "\n",
    "`tf.nn.top_k` will return the values and indices (class ids) of the top k predictions. So if k=3, for each sign, it'll return the 3 largest probabilities (out of a possible 43) and the correspoding class ids.\n",
    "\n",
    "Take this numpy array as an example. The values in the array represent predictions. The array contains softmax probabilities for five candidate images with six possible classes. `tk.nn.top_k` is used to choose the three classes with the highest probability:\n",
    "\n",
    "```\n",
    "# (5, 6) array\n",
    "a = np.array([[ 0.24879643,  0.07032244,  0.12641572,  0.34763842,  0.07893497,\n",
    "         0.12789202],\n",
    "       [ 0.28086119,  0.27569815,  0.08594638,  0.0178669 ,  0.18063401,\n",
    "         0.15899337],\n",
    "       [ 0.26076848,  0.23664738,  0.08020603,  0.07001922,  0.1134371 ,\n",
    "         0.23892179],\n",
    "       [ 0.11943333,  0.29198961,  0.02605103,  0.26234032,  0.1351348 ,\n",
    "         0.16505091],\n",
    "       [ 0.09561176,  0.34396535,  0.0643941 ,  0.16240774,  0.24206137,\n",
    "         0.09155967]])\n",
    "```\n",
    "\n",
    "Running it through `sess.run(tf.nn.top_k(tf.constant(a), k=3))` produces:\n",
    "\n",
    "```\n",
    "TopKV2(values=array([[ 0.34763842,  0.24879643,  0.12789202],\n",
    "       [ 0.28086119,  0.27569815,  0.18063401],\n",
    "       [ 0.26076848,  0.23892179,  0.23664738],\n",
    "       [ 0.29198961,  0.26234032,  0.16505091],\n",
    "       [ 0.34396535,  0.24206137,  0.16240774]]), indices=array([[3, 0, 5],\n",
    "       [0, 1, 4],\n",
    "       [0, 5, 1],\n",
    "       [1, 3, 5],\n",
    "       [1, 4, 3]], dtype=int32))\n",
    "```\n",
    "\n",
    "Looking just at the first row we get `[ 0.34763842,  0.24879643,  0.12789202]`, you can confirm these are the 3 largest probabilities in `a`. You'll also notice `[3, 0, 5]` are the corresponding indices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TopKV2(values=array([[  1.00000000e+00,   4.50099469e-09,   1.28149702e-09,\n",
      "          6.71458306e-11,   1.26368464e-11],\n",
      "       [  5.04667401e-01,   2.12314308e-01,   2.06644922e-01,\n",
      "          2.66185421e-02,   1.96877941e-02],\n",
      "       [  9.60256279e-01,   2.84128953e-02,   7.43444078e-03,\n",
      "          1.30447885e-03,   1.20086526e-03],\n",
      "       [  9.97596920e-01,   2.38738535e-03,   7.40509768e-06,\n",
      "          6.82809195e-06,   7.63188382e-07],\n",
      "       [  9.99999762e-01,   1.53853108e-07,   1.34919119e-07,\n",
      "          5.43775081e-10,   5.98767563e-11]], dtype=float32), indices=array([[13, 35, 25, 15,  3],\n",
      "       [ 2,  1,  4,  8,  7],\n",
      "       [ 5,  3,  7,  1, 31],\n",
      "       [34, 38, 36, 35, 32],\n",
      "       [38, 36, 34, 20, 35]]))\n"
     ]
    }
   ],
   "source": [
    "### Print out the top five softmax probabilities for the predictions on the German traffic sign images found on the web. \n",
    "with tf.Session() as sess:\n",
    "    saver.restore(sess, './lenet')\n",
    "\n",
    "    softmax=tf.nn.softmax(logits)\n",
    "    top_k=sess.run(tf.nn.top_k(softmax, k=5), feed_dict={x: X_new, y: y_new, keep_prob: 1.})\n",
    "\n",
    "print(top_k)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "> **Note**: Once you have completed all of the code implementations, you need to finalize your work by exporting the IPython Notebook as an HTML document. Before exporting the notebook to html, all of the code cells need to have been run. You can then export the notebook by using the menu above and navigating to  \\n\",\n",
    "    \"**File -> Download as -> HTML (.html)**. Include the finished document along with this notebook as your submission. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Project Writeup\n",
    "\n",
    "Once you have completed the code implementation, document your results in a project writeup using this [template](https://github.com/udacity/CarND-Traffic-Sign-Classifier-Project/blob/master/writeup_template.md) as a guide. The writeup can be in a markdown or pdf file. "
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  },
  "widgets": {
   "state": {},
   "version": "1.1.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}