Exercise_12_1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jNemS0qoqll3"
   },
   "source": [
    "# Exercise 12.1\n",
    "## Weights and activations of a convolutional network\n",
    "This task consists of two parts. First, set up and evaluate a\n",
    "convolutional neural network.\n",
    " 1. Set up and train a convolutional network on the classification of images (data set CIFAR-10). Train your network to at least 70% test accuracy.\n",
    "  - Plot the confusion matrix. What do you observe?\n",
    "  - Plot several falsely classified images along with the predicted class scores. What kinds of misclassification do you observe, why do they occur?\n",
    " 2. Plot the filter weights in the first layer of your network and see if you can make any sense of it.\n",
    " 3. Visualize the activations in the first two layers of your network for two input images of choice and describe what you see.\n",
    "  - Does it meet your expectations?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "OjF2j1SEqll6",
    "outputId": "8fa23495-ae15-4621-f5d2-15f4eb2b7cfa"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "keras 2.4.0\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "layers = keras.layers\n",
    "print(\"keras\", keras.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Jfs9BQ2fqll7"
   },
   "source": [
    "### Download CIFAR-10 data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "DsfigSRCqll7"
   },
   "outputs": [],
   "source": [
    "(x_train, y_train), (x_test, y_test) = keras.datasets.cifar10.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "1R-D0tllqll8"
   },
   "outputs": [],
   "source": [
    "x_train = (x_train - 128.) / 128.\n",
    "x_test = (x_test - 128.) / 128."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "laWs8d_mqll8"
   },
   "outputs": [],
   "source": [
    "y_train_one_hot = tf.keras.utils.to_categorical(y_train)\n",
    "y_test_one_hot = tf.keras.utils.to_categorical(y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LZ4InyZeqll8"
   },
   "source": [
    "### Design, train, and evaluate a simple CNN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "l-5KGW5hqll9",
    "outputId": "638bf343-8629-4276-e60d-110fd7aabcb2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv2d (Conv2D)              (None, 32, 32, 16)        1216      \n",
      "_________________________________________________________________\n",
      "conv2d_1 (Conv2D)            (None, 32, 32, 16)        2320      \n",
      "_________________________________________________________________\n",
      "conv2d_2 (Conv2D)            (None, 16, 16, 32)        4640      \n",
      "_________________________________________________________________\n",
      "conv2d_3 (Conv2D)            (None, 16, 16, 32)        9248      \n",
      "_________________________________________________________________\n",
      "conv2d_4 (Conv2D)            (None, 8, 8, 64)          18496     \n",
      "_________________________________________________________________\n",
      "conv2d_5 (Conv2D)            (None, 8, 8, 64)          36928     \n",
      "_________________________________________________________________\n",
      "global_max_pooling2d (Global (None, 64)                0         \n",
      "_________________________________________________________________\n",
      "dropout (Dropout)            (None, 64)                0         \n",
      "_________________________________________________________________\n",
      "dense (Dense)                (None, 10)                650       \n",
      "=================================================================\n",
      "Total params: 73,498\n",
      "Trainable params: 73,498\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "model = keras.models.Sequential([\n",
    "    layers.Convolution2D(16, kernel_size=(5, 5), padding=\"same\", activation='elu', input_shape=(32, 32, 3)),\n",
    "    layers.Convolution2D(16, kernel_size=(3, 3), padding=\"same\", activation='elu'),\n",
    "    layers.Convolution2D(32, kernel_size=(3, 3), padding=\"same\", strides=(2, 2), activation='elu'),\n",
    "    layers.Convolution2D(32, kernel_size=(3, 3), padding=\"same\", activation='elu'),\n",
    "    layers.Convolution2D(64, kernel_size=(3, 3), padding=\"same\", strides=(2, 2), activation='elu'),\n",
    "    layers.Convolution2D(64, kernel_size=(3, 3), padding=\"same\", activation='elu'),\n",
    "    layers.GlobalMaxPooling2D(),\n",
    "    layers.Dropout(0.5),\n",
    "    layers.Dense(10, activation='softmax')\n",
    "    ])\n",
    "\n",
    "print(model.summary())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "sRLF6anHqlmA",
    "outputId": "66edb5b1-f4f7-44ca-e653-2d8d56b55719",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/30\n",
      "1407/1407 [==============================] - 12s 6ms/step - loss: 1.8360 - accuracy: 0.3233 - val_loss: 1.5533 - val_accuracy: 0.4298\n",
      "Epoch 2/30\n",
      "1407/1407 [==============================] - 9s 6ms/step - loss: 1.6050 - accuracy: 0.4152 - val_loss: 1.4613 - val_accuracy: 0.4790\n",
      "Epoch 3/30\n",
      "1407/1407 [==============================] - 8s 6ms/step - loss: 1.5268 - accuracy: 0.4452 - val_loss: 1.3447 - val_accuracy: 0.5208\n",
      "Epoch 4/30\n",
      "1407/1407 [==============================] - 9s 6ms/step - loss: 1.4726 - accuracy: 0.4673 - val_loss: 1.2960 - val_accuracy: 0.5354\n",
      "Epoch 5/30\n",
      "1407/1407 [==============================] - 8s 6ms/step - loss: 1.4288 - accuracy: 0.4818 - val_loss: 1.2554 - val_accuracy: 0.5398\n",
      "Epoch 6/30\n",
      "1407/1407 [==============================] - 8s 6ms/step - loss: 1.3708 - accuracy: 0.5032 - val_loss: 1.2058 - val_accuracy: 0.5622\n",
      "Epoch 7/30\n",
      "1407/1407 [==============================] - 8s 6ms/step - loss: 1.3235 - accuracy: 0.5223 - val_loss: 1.1836 - val_accuracy: 0.5664\n",
      "Epoch 8/30\n",
      "1407/1407 [==============================] - 9s 6ms/step - loss: 1.2728 - accuracy: 0.5407 - val_loss: 1.1328 - val_accuracy: 0.6022\n",
      "Epoch 9/30\n",
      "1407/1407 [==============================] - 9s 6ms/step - loss: 1.2249 - accuracy: 0.5609 - val_loss: 1.0718 - val_accuracy: 0.6208\n",
      "Epoch 10/30\n",
      "1407/1407 [==============================] - 9s 6ms/step - loss: 1.1826 - accuracy: 0.5768 - val_loss: 1.0285 - val_accuracy: 0.6420\n",
      "Epoch 11/30\n",
      "1407/1407 [==============================] - 9s 6ms/step - loss: 1.1417 - accuracy: 0.5897 - val_loss: 1.0126 - val_accuracy: 0.6446\n",
      "Epoch 12/30\n",
      "1407/1407 [==============================] - 9s 6ms/step - loss: 1.1136 - accuracy: 0.6022 - val_loss: 0.9534 - val_accuracy: 0.6706\n",
      "Epoch 13/30\n",
      "1407/1407 [==============================] - 9s 6ms/step - loss: 1.0864 - accuracy: 0.6125 - val_loss: 0.9470 - val_accuracy: 0.6700\n",
      "Epoch 14/30\n",
      "1407/1407 [==============================] - 9s 6ms/step - loss: 1.0531 - accuracy: 0.6255 - val_loss: 0.9240 - val_accuracy: 0.6852\n",
      "Epoch 15/30\n",
      "1407/1407 [==============================] - 9s 6ms/step - loss: 1.0365 - accuracy: 0.6314 - val_loss: 0.9002 - val_accuracy: 0.6838\n",
      "Epoch 16/30\n",
      "1407/1407 [==============================] - 9s 6ms/step - loss: 1.0116 - accuracy: 0.6396 - val_loss: 0.9108 - val_accuracy: 0.6872\n",
      "Epoch 17/30\n",
      "1407/1407 [==============================] - 9s 6ms/step - loss: 0.9872 - accuracy: 0.6477 - val_loss: 0.8913 - val_accuracy: 0.6906\n",
      "Epoch 18/30\n",
      "1407/1407 [==============================] - 9s 6ms/step - loss: 0.9750 - accuracy: 0.6532 - val_loss: 0.9053 - val_accuracy: 0.6940\n",
      "Epoch 19/30\n",
      "1407/1407 [==============================] - 9s 6ms/step - loss: 0.9594 - accuracy: 0.6614 - val_loss: 0.8879 - val_accuracy: 0.6870\n",
      "Epoch 20/30\n",
      "1407/1407 [==============================] - 9s 6ms/step - loss: 0.9404 - accuracy: 0.6662 - val_loss: 0.8753 - val_accuracy: 0.6926\n",
      "Epoch 21/30\n",
      "1407/1407 [==============================] - 9s 6ms/step - loss: 0.9293 - accuracy: 0.6694 - val_loss: 0.8855 - val_accuracy: 0.6928\n",
      "Epoch 22/30\n",
      "1407/1407 [==============================] - 9s 6ms/step - loss: 0.9138 - accuracy: 0.6742 - val_loss: 0.8447 - val_accuracy: 0.7040\n",
      "Epoch 23/30\n",
      "1407/1407 [==============================] - 9s 6ms/step - loss: 0.9012 - accuracy: 0.6796 - val_loss: 0.8478 - val_accuracy: 0.7104\n",
      "Epoch 24/30\n",
      "1407/1407 [==============================] - 8s 6ms/step - loss: 0.8835 - accuracy: 0.6830 - val_loss: 0.8616 - val_accuracy: 0.7012\n",
      "Epoch 25/30\n",
      "1407/1407 [==============================] - 8s 6ms/step - loss: 0.8798 - accuracy: 0.6858 - val_loss: 0.8647 - val_accuracy: 0.6962\n",
      "\n",
      "Epoch 00025: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.\n",
      "Epoch 26/30\n",
      "1407/1407 [==============================] - 9s 6ms/step - loss: 0.7754 - accuracy: 0.7240 - val_loss: 0.7802 - val_accuracy: 0.7330\n",
      "Epoch 27/30\n",
      "1407/1407 [==============================] - 8s 6ms/step - loss: 0.7374 - accuracy: 0.7356 - val_loss: 0.7827 - val_accuracy: 0.7248\n",
      "Epoch 28/30\n",
      "1407/1407 [==============================] - 9s 6ms/step - loss: 0.7168 - accuracy: 0.7403 - val_loss: 0.7784 - val_accuracy: 0.7374\n",
      "Epoch 29/30\n",
      "1407/1407 [==============================] - 9s 6ms/step - loss: 0.6996 - accuracy: 0.7496 - val_loss: 0.7995 - val_accuracy: 0.7272\n",
      "Epoch 30/30\n",
      "1407/1407 [==============================] - 8s 6ms/step - loss: 0.6914 - accuracy: 0.7500 - val_loss: 0.7921 - val_accuracy: 0.7320\n"
     ]
    }
   ],
   "source": [
    "model.compile(\n",
    "    loss='categorical_crossentropy',\n",
    "    optimizer=keras.optimizers.Adam(1e-3),\n",
    "    metrics=['accuracy'])\n",
    "\n",
    "results = model.fit(x_train, y_train_one_hot,\n",
    "                    batch_size=32,\n",
    "                    epochs=30,\n",
    "                    verbose=1,\n",
    "                    validation_split=0.1,\n",
    "                    callbacks = [keras.callbacks.ReduceLROnPlateau(factor=0.5, patience=3, verbose=1, min_lr=1e-5)],\n",
    "                    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 297
    },
    "id": "b4kIVpDCqlmA",
    "outputId": "1ca1597e-facd-4c98-8640-b77c4d86e3c9"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1cAAAEYCAYAAACwUwxgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3xUVfrH8c9J7z2hpFJCDzV0IiBIr9KkqViwYEVxdVcFd3XVn64FC9WKCiKoFBEE6Z1QQgmB0EJCSw+QQtr5/XEjItLJ5E6S5/163VeSmZu53zi7zDxzznmO0lojhBBCCCGEEOL22JgdQAghhBBCCCEqAimuhBBCCCGEEKIUSHElhBBCCCGEEKVAiishhBBCCCGEKAVSXAkhhBBCCCFEKbAzO8DN8vPz02FhYWbHEEIIYQHbt29P1Vr7m53jdslrlRBCVFzXeq0qd8VVWFgY0dHRZscQQghhAUqpBLMzlAZ5rRJCiIrrWq9VMi1QCCGEEEIIIUqBFFdCCCGEEEIIUQqkuBJCCCGEEEKIUlDu1lwJIURFVlBQQFJSEnl5eWZHsSgnJyeCgoKwt7c3O0qZkedWCCEqPimuhBDCiiQlJeHu7k5YWBhKKbPjWITWmrS0NJKSkqhRo4bZccqMPLdCCFHxybRAIYSwInl5efj6+lbYN98ASil8fX0r/AjO5eS5FUKIik+KKyGEsDIV+c33HyrD33glleHvrgx/oxBCXI0UV0IIIYQQQghRCirdmqs1B1P4ZnMCU0Y2x85WakshhLhUZmYm3333HY8//vhN/V6vXr347rvv8PLyslAycbvkuRVCVCSZOfks3XuajJwCCouKKSzWFBVrCoqLKSrSFBZrCouLjduKjPsKizVPdK5N3aruFstV6Yqrs7kFLI89w+4TWTQP8TY7jhBCWJXMzEw+/fTTv70BLywsxM7u6i8ZS5YssXQ0cZvkuRVClHdaa7YcTWfO1uMs2Xua/MLiv9xva6OwKzlsbRT2tjZ/3mZrg52N4vyFQotmrHTFVfvafigF6w6mSnElhBCXefHFFzl8+DBNmzbF3t4eJycnvL29iYuL4+DBgwwYMIDExETy8vJ4+umnGTt2LABhYWFER0dz/vx5evbsSYcOHdi4cSOBgYEsWLAAZ2dnk/8yIc+tEKK8Sj1/gfnbk/h+WyJHUrNxd7LjnpbBDGsZTC1/t4vFlDWs+ax0xZWPqwMRgZ6si0/h6a7hZscRQoirem3RPmJPni3Vx2xQ3YOJfRte9f633nqLvXv3smvXLlavXk3v3r3Zu3fvxbban3/+OT4+PuTm5tKyZUsGDRqEr6/vXx4jPj6e2bNnM2PGDIYOHcr8+fMZNWpUqf4d5Z08t0IIcW3FxZoNh1OZszWR32JPU1CkaRnmzbjOtekVUQ1nB1uzI15RpSuuAO4I92fKmsOczSvAw0k2ORRCiKtp1arVX/Yrmjx5Mj/99BMAiYmJxMfH/+0NeI0aNWjatCkALVq04NixY2WWV9w4eW6FENbozNk8fohO5PvoRBLTc/F2sefetmHc0zKY8CqWWytVWixWXCmlPgf6AMla60ZXuN8T+AYIKcnxrtb6C0vluVRUuB8frzrEpsNpdG9YtSwuKYQQN+1aoxBlxdXV9eL3q1evZsWKFWzatAkXFxc6dep0xf2MHB0dL35va2tLbm5umWQtT+S5FUIIYw3Vyaw89iRlsedEJjGJWWw6kkZRsaZdLV8mdK9H94ZVcLSzzlGqK7HkyNWXwMfA11e5fxwQq7Xuq5TyBw4opb7VWudbMBMAzUK8cXWwZV18ihRXQghxCXd3d86dO3fF+7KysvD29sbFxYW4uDg2b95cxunE7ZDnVghhJq01p7Ly2J2Uxd4TWewpOdKzjbf+tjaKrn7pPN+qOj07tCbMz/U6j2idLFZcaa3XKqXCrnUK4K6MlWduQDpg2fYdJRzsbGhby5d18allcTkhhCg3fH19ad++PY0aNcLZ2ZkqVapcvK9Hjx5MnTqV+vXrU7duXdq0aWNiUnGz5LkVQpS1XYmZ/L7/jFFIJWWRdkkhFR7gRtf6AUQEehLpkkydfe9je3AJ7FKQ2wvaPAphUWAFTSpuhtJaW+7BjeJq8VWmBboDC4F6gDswTGv9y1UeZywwFiAkJKRFQkLCbWf7auMxJi7cx5oJnQj1LZ+VsRCi4tm/fz/169c3O0aZuNLfqpTarrWONClSqYmMjNTR0dF/ua2yP7dCiApAayjIAYdrv3fediydyb/Hsy4+9WIhFRHoSUSQJxGBntSv5oGTvS1kJcGqNyHmO3Bwg3ZPQuEF2P4F5KRBQENo/Qg0Hgr21tOZ9FqvVWY2tOgO7ALuBGoBy5VS67TWf2ufpLWeDkwH4wWrNC4eFe4HwLr4VCmuhBBCCCGEuJ7lr8KmT6BmR2g0GOr3ASdPwJj2t/mIUVRtOpKGn5sD/+xVjxGtQ3FzvKzkyEmHlf+DrTMADW0ehw7jwbWkic4dz8OeebBlKix6ClZMghb3Q8uHwDPw5jIX5MLpvXByp3F0+gd4h93mf4irM7O4GgO8pY2hs0NKqaMYo1hby+LiNfxcCfRyZl18CqPahJbFJYUQQgghhCifkrbDpo8hqBWkH4EFj8PiZ9HhdxHn143X40PYkJCDv7sjr/RpwIhWIX9vl56fDZunwIYPIf88NBkOnV4Cr+C/nmfvDM1HQ7NRcGy9UWRt+MD4vQb9oc1jENTy71MGC/Mhed+fhdSJnZAcC7rIuN81AJqOqLDF1XGgC7BOKVUFqAscKauLK6W4o44fi2NOUVhUjJ2tTVldWgghhBBCiPKjqAAWPgluVWHkXHD0QCdFk7RuFq4HF1I/bjHTceJ0jS4E33EvDnWCwNb2r7+/4ytY839w/gzU7QVdXoWA60wfVgpqRBlHxjFjpGvHLNj3I1Rvboxk6WI4ucMops7sg6KS3njOPlC9GdTpbnwNbA7u1Sy+hsuSrdhnA50AP6VUEjARsAfQWk8F/gN8qZTaAyjgH1rrMu0wERXuz+yticQkZdIi1KcsLy2EEEIIIUT5sOFDY0TontloRw9W7E/mo5UX2J3UnWDPvkxskkHn/LXUilsE3/9iFDYN+kPEEKOYWvkfY7QrpC0M/RpCbqFpjncYdH/DGOmKmQ1bphmjZwCOHlC9qTGiVb2ZUXh5hZjSDMOS3QKHX+f+k0A3S13/RrSr5YuNgrUHU6W4EkIIIYQQ4nKph4wRpwYDiPXowPOT1xN76izBPs68PSiCgc2CcLCzAYZC4Xtw+HdjvdTu743GFAABDWDEXAjvdvsFj6MbtHoYIh80RqycvMCnJthYxyw0M6cFms7LxYHGQV6si0/h2bvqmB1HCCGEEEII61FcDIueBnsn8ru9xTNf7CQ9u4B3hzShf9Pq2F++rMbOAer2NI78bDi4FJQt1O8LNqW8EbCNDQRZX3NZ6yjxTHRHuB+7EjPJyi0wO4oQQpQ7bm5uZkcQFiLPrRCCnV9Dwnro9jqf7crh4JnzvD0ogsEtgv5eWF3OwRUaDYKGA0q/sLJilb64iqrjT7GGTYdlQ2EhhBBCCCEAOHsKfnsVwqJIDB3Eh78fpHvDKnSpX+X6v1uJVeppgQBNg71wc7RjbXwqPRpVMzuOEEKY6sUXXyQ4OJhx48YBMGnSJOzs7Fi1ahUZGRkUFBTw+uuv079/f5OTipslz60Q4qb8OgGKLqD7fMCkRbHYKMXEvg3NTmX1Kn1xZW9rQ9tavqw9mILWGmVCVxEhhLiiX1+E03tK9zGrRkDPt65697Bhw3jmmWcuvgGfO3cuy5Yt46mnnsLDw4PU1FTatGlDv3795N/L2yHPrRDCmsUuhP2LoOskfjvjxu9xB/hXr/pU93I2O5nVq/TFFRjrrpbHniEhLYcwP1ez4wghhGmaNWtGcnIyJ0+eJCUlBW9vb6pWrcqzzz7L2rVrsbGx4cSJE5w5c4aqVauaHVfcBHluhRA3JDcTlkyAqhFkN3+USR9upF5Vd+5vH2Z2snJBiiuM/a4A1sanSHElhLAe1xiFsKQhQ4Ywb948Tp8+zbBhw/j2229JSUlh+/bt2NvbExYWRl5eninZKgx5boUQ16K1sSHu4ZWQsAG8a0Dz0VClDKblrZgI2ckwYg4frDrKqaw8Ph7R/PoNLAQgxRUAob4uBPs4s/ZgKve2DTM7jhBCmGrYsGE8/PDDpKamsmbNGubOnUtAQAD29vasWrWKhIQEsyOKWyTPrRBW7NxpOLwKjqwyvmYnG7f71DKKrC1TIDASmt8Lje4GR/fSz3BsPWz/Eto+QSy1+HzDeoa3CqFFqHfpX6uCkuIKUEpxR7g/P+88QUFRsVTmQohKrWHDhpw7d47AwECqVavGyJEj6du3LxEREURGRlKvXj2zI4pbJM+tEFYkPwcSNpYUUyshOda43cUPanWGmp2Nrx7VITvN2JR3x1ew6ClY+hI0GgjN74Oglre/MS9AQZ6xp5VXKMUdX+Jfn8fg5WzPP3rUvf3HrkSkuCoRFe7Pt1uOs/N4Jq1q+JgdRwghTLVnz5/NFvz8/Ni0adMVzzt//nxZRbIIpVQP4EPAFpiptX7rsvvfBzqX/OgCBGitvco2ZemqLM+tEFapIBd2fQuxC+D4ZijKB1tHCG0LjYdBrTuhSiNjg9xLufpC28ehzWOQtA12fA17f4Sd34B/PWM0q/E9xnm3au07kHYIRv/EnJh0dh7P5L2hTfBycbi9v7mSkeKqRNtavtjaKNbFp0hxJYQQlYBSyhb4BLgLSAK2KaUWaq1j/zhHa/3sJec/CTQr86BCiPIvLwu2fQabP4XsFAhoAK3GGsVUSFtwcLmxx1EKglsZR483jQJrx9ew7J+wYhLU6w3NRkNYB7BzvPF8p/fChg+gyQhSq7TnrVmraVPTh4HNAm/pz63MpLgq4elsT9NgL9bGp/JcNxn+FEKISqAVcEhrfQRAKTUH6A/EXuX84cDEMsomhKgIslNh8xTYOgMuZEGtLhD1HIS2u/2pfI7u0OI+4zizD3bMgt1zYN9PYOtgbM8Q2MJYpxXYAnxrXfmaxUWw8Elw8oLub/DfRfvJLSji9QERsi3DLZDi6hJR4X58+Hs8mTn5MgQqhDBNZdhzT2ttdgSAQCDxkp+TgNZXOlEpFQrUAFZe7cGUUmOBsQAhISFXPEeeWyEqiawk2PgRbP8KCvOgQT/oMB6qN7XM9ao0NLqQdp0Eh5ZD4lY4sQN2fgtbpxvnOHlBYPM/i63AFuDmD1umwckdMOgzNp4q5sedJ3iic21qB7hZJmsFJ8XVJaLC/flgRTwbDqXRu3E1s+MIISohJycn0tLS8PX1rbBvwrXWpKWl4eTkZHaUm3EPME9rXXS1E7TW04HpAJGRkX+rMOS5FaISSI2H9R8YI0hgrKNq/wz41ymb69s7Qf2+xgFQVAipB+DEdkiKNgqude+CLjbu9wqB8ykQ3p0L9Qbw8uT1hPi48MSdtcsmbwUkxdUlmgR54u5kx7r4FCmuhBCmCAoKIikpiZSUFLOjWJSTkxNBQUFmxzgBBF/yc1DJbVdyDzDudi4mz60Qt0Fro1V5+mFIOwyZx8GvDoTfBS5WsFb+VAyse89oVGHnCJEPQrsnwSv4+r9rSbZ2xqhWlYZG0wuA/Gwj74ntxnH2JPT+HzPWHeVISjZfjmmJk72tubnLMSmuLmFna0P7Wn6si0+tFFM3hBDWx97enho1apgdo7LYBoQrpWpgFFX3ACMuP0kpVQ/wBq7cVu8GyXMrxHVobTR7SDv8ZxGVfhjSjkD6ESjI/vvvKBsIagV1e0CdHkbnvFt5/1ZUCGf2wvFNRnv05FgoLjRGeLQ2vhYXlfx86VFyX/45cPSAqPHQ+jFjup21cnA11nyFtrt4U0JaNh+tXEvviGp0qhtgYrjyT4qry0TV8WPpvtMcSc2mlr/MNRVCiIpKa12olHoCWIbRiv1zrfU+pdS/gWit9cKSU+8B5mhZTCTE7dMactIvK55KvqYfhQtn/zxX2YJ3qLGJblh746tvTeOrZxCc2g0HlxrHiknG4RViFFl1ukNoB2Oa3JUU5BqjNgmb4PhGSNxmFEgAniHG2ih7Z6N4UzZGwXbxexsj26U/ewZCs1Hg5Gnp/4KlTmvNqwv2YW9rwyt9Gpgdp9yT4uoyd4QbnzSsO5gixZUQQlRwWuslwJLLbnv1sp8nlWUmISqE3IySEacrFFF5WX+ep2yMgsinFgS3LimgaoFPTeN2W/urXyOohXHc+S9jatvBZcaxY5bRxMHe1diEt053CG1v7OGUsNEYnTq509hjCoy26I2HGiM5IW2Mwq0SWbLnNGsOpjCxbwOqesp6ydslxdVlgn1cCPN1YV18Kve3l+kbQgghhKigzp0GGztjOpvdLXRJzssqKZhKpu1dWkTlpl9yogLPYGPUqdHgkuKppIjyCr21a1/OozpEjjGOglw4tt4Y0TqwFOIW/3mejR1UbwatHzWKqeDW1rFmqwwVF2tSzl8gKSOXk5m5/GdxLI0CPRjdJtTsaBWCFFdXEBXuz/wdSeQXFuNgZ3P9XxBCCCGEKC/OJ8OvLxj7If3BzsmY0uboAU4el331LJnupkoKqZICKif1r4/rEWiMODXod8kIVC3wDrv69DxLsHc2Gl2E3wW93jXWTx3fbDTACGxx4xv2llO5+UWcyDQKp0u/nsjI5WRWLqez8igo+nOWs4uDLTMGRGJnK+95S4MUV1cQFe7HrM0J7DieQZuavmbHEUIIIYS4fVpDzGxY+hIU5Bj7LrlXhbyzxga3eWeNNU95Jd+fPfHnbQU5xmO4VzMKpnq9/l5AWWPRotSf3fIqkLyCIo6lZXMsNafkazZHU7M5lpbNmbMX/nKurY2iqocT1b2caB7iTXUvZ6p7ORPo5USglwtB3s64OkpJUFrkv+QVtK3li62NYl18ihRXQgghhCj/MhJg8TNweCUEt4F+H93c3ktFBUb3PHtny2UUf5OZk8/O45nEJ5/jaGoOx0oKqFNZeX85z8/NgTBfV6LC/QnzdSHYx+ViEVXF3VFGpcqQFFdX4O5kT/MQL9bFpzKhu9lphBBCCCFuUXERbJkGK/9jNI/o9a6xB5PNTb7ZtrW/dnMJcdu01hxJzWZ7Qgbbj2Ww/XgGh5LPX7zfx9WBMF8X2tbyJczXlTA/V2r4uhLm54K7kzw31kKKq6uICvfn/RUHSc/Ox8e1FBZaCiGEEEKUpTOxsPBJOBEN4d2g93vmb2orLsorKCImMZPtxzPYkZDB9oQMMnIKAPBysadFiDcDmwXSItSb+lU98HSRAqo8kOLqKqLC/Xhv+UE2HEqlb5PqZscRQgghhLgxhRdg3Xuw7n9GU4q7Z0LE4FvbXFeUiuJizdG0bHYnZbI7KYsdxzPZdyKLwmKjsURNf1fualCFFqHetAj1oaafKzY28nyVR1JcXUXjIC88ne1ZF58ixZUQQgghyofEbbDwCUiJg4ih0ONNcPUzO1WlorUmKSOXPSeyiEnKZHdiFntPZHHuQiEATvY2RAR68lBUTSJDvWke6i2zpCoQKa6uwtZG0aG2H+viU9Fao+TTHiGEEEKUlYI8WPQUJO8HO0ewdfjzsPvje0djHdQf9+ekQcwcoyX6iB+gTjez/4oKr7ComJOZeRw8c47dJ7LYnZTJnqQs0rKNDYrtbRX1q3nQr2l1mgR50TjYk9r+btJgogKT4uoaosL9+GXPKQ6nnKd2gLvZcYQQQghRGRQXwfwHIe4XqN3F+LmoAPLPQ1E+FOYbX/84Ci8Y9+tiaPkQdJ0IjvK+pbTkFRRxPD2HhLQcEtKyja/pORxPyyYpI/fi1D4bBXWquNOlfgARQV40CfKkblV3HO1sTf4LRFmyWHGllPoc6AMka60bXeWcTsAHgD2QqrXuaKk8t6JDuDGMvvZgqhRXQgghhLA8reGX8RC3GHr+H7R+xOxElUZOfiGxJ8+y50QW+0+d5VhJMXX5vlEeTnaE+rrSMNCT3o2rEerjSk1/VxpU98DFQcYtKjtL/i/gS+Bj4Osr3amU8gI+BXporY8rpQIsmOWWBHm7UNPflbXxKTzQoYbZcYQQQghR0a1+E7Z/CVHPSWFlQXkFRew/ZRRSu5Oy2JOURXzyOUoGofBzc6CGnysdahv7RoX4uhDq60qYrwteLrI+SlydxYorrfVapVTYNU4ZAfyotT5ecn6ypbLcjm4NqjJt7WFWxp3hznpVzI4jhBBCiIpq6wxY8zY0GwV3vmJ2mgpDa82+k2eJKVkPtTspi4Nnzl2czufr6kDjIE+6N6pK40BPIoI8qeLhZHJqUV6ZOXZZB7BXSq0G3IEPtdZXG+UaC4wFCAkJKbOAAE93CWddfApPz97FT+PaUzvArUyvL4QQQohKYN/PsGQC1OkJfT6Utum3SWvN/lPnWBhzkkUxJzmRmQsY+0dFBHrySL2aRAR60TjIk2qeTtK4TJQaM4srO6AF0AVwBjYppTZrrQ9efqLWejowHSAyMlKXZUhnB1um3xtJv4/WM3ZWND+Pa4+H7IIthBBCiNJydC38+DAEt4bBn4OtrNu5VUdSzrMo5hQLY05wOCX7YvfnZ7qG06amL0HezlJICYsy8/+9SUCa1jobyFZKrQWaAH8rrswW6OXMpyObM3LmFp6ds4sZ90bKxm5CCCGEuH2nYmD2CPCpBcNng4OL2YnKnZOZuSzefZKFMSfZe+IsSkGrMB/GtK9Br4hqsoeUKFNmFlcLgI+VUnaAA9AaeN/EPNfUuqYvE/s24JUF+3hv+UGe717X7EhCCCGEKM/Sj8I3g8HJE0bNBxcfsxOVGynnLvDr3lMsijnJtmMZADQJ8uTl3vXp3bga1TydTU4oKitLtmKfDXQC/JRSScBEjJbraK2naq33K6WWAruBYmCm1nqvpfKUhlFtQtl74iwfrzpEg+oe9IqoZnYkIYQQQpRH55Nh1kAoLoD7F4NnoNmJrJbWmqSMXLYeTSc6IZ1txzI4lHwegPAAN567qw59m1QnzM/V5KRCWLZb4PAbOOcd4B1LZShtSin+PaAh8cnneG5uDDX8XKlfzcPsWEIIIYQoTy6cg28Hw7nTcN9C8JfZMJcqKtbEnT5L9LEMth1LZ9ux9It7Tbk72REZ6s3AZoF0qR9AvaryPkxYF1kxeZMc7WyZOqoFfUoaXCwc1wFvmcsrhBBCiBtReAHmjITTe401VsGtzE5kOq01uxIz2Xg4ja1H09mRkMG5C4UAVPVwolUNX1qFeRMZ5kOdKu7Yyrp3YcUqZ3FVXAQ2trf86wEeTkwb3YJh0zbzxOwdfDWmFXa2NqUYUAghhBAVTnEx/PQoHF0DA6ZAne5mJzKN1prYU2dZFHOKxbtPkpRhtEqvU8WNvk2r0zLMm5ZhPgR6SXc/Ub5UvuJq38+w+i14cJmxgPQWNQvx5vWBjXhh3m7e/DWOV/o0KMWQQgghhCiXtIa8TDh7suQ4AVknjO9T4uBENHR9DZqOMDupKQ4ln2NhSUF1JCUbOxtFh3A/nu1ahzvrBchsIFHuVb7iyiMQUg/A0pdgwKe39VBDI4OJPXmWz9YfpWF1D+5uHlRKIYUQQghh9VIPwe45JcXTiT8LqoLsy05U4F4VPKobhVX7p02Ja5bjaTks2m1s5ht3+hxKQZsavjzUoSY9GlWVVumiQql8xVVwS+gwHta9C/V6G8dt+Ffv+sSdPsuLP+6hdoAbjYO8SimoEEIIIaxWwkaYfY/RnMK9mlE4VWkA4d2M7z2qGx/oegaCWxWwtTc7cZk6nZXH4t0nWbT7FDGJmQC0CPVmUt8G9IqoRoCHk8kJhbCMyldcAXT8B8Qvg4VPQVArcPO/5Yeyt7XhkxHN6ffxBh6ZtZ2FT3TA392xFMMKIYQQwqrsXwzzHgCvEHh0vfFVkHb+Akv2ni7ZeyodraFRoAcv9axH78bVCPKWDZJFxVc5iys7Bxg4DaZ3gsXPwLBv4DYWS/q6OTL93hYMmrKRx77ZzncPt8HBThpcCCGEEBXOts9gyfNQvTmMmAuuvmYnMtXZvAJ+23eGhTEn2XAolaJiTe0AN57tWoc+jatR09/N7IhClKnKWVwBVGkId74My1+FmDnQ9Lrbcl1Tw+qevDO4CU/O3skL82J4Z0gT7KWDoBBCCFExaA2r34Q1bxtT/4Z8CQ6Vc9Pa3Pwifo87w6KYk6w6kEJ+YTHBPs48ckdN+japTr2q7tLhT1Ralbe4Amj7BBz4FX59AWpEgeftNaTo26Q6x9NzeGfZAZLPXWDKqBZ4OleuOdZCCCFEhVNUCL+Mhx1fQdNR0PeDSreGKq+giPXxqSzafZLlsWfIyS8iwN2RUa1D6dukGk2DvaSgEoLKXlzZ2Br7TExpDz8/DqN/BpvbG20a17k2VTyceOnH3QyaspEv7m9JsI/MMRZCCCHKpYJcY33VgSUQ9bwx66USFBF5BUXsOJ7B5sNpbD6Szq7ETPKLivF2sWdAs0D6Nq5Oqxo+sqGvEJep3MUVgE8N6P6GsfZq2wxo/chtP+TgFkEEejnz6DfbGfDJBmbcF0nzEO9SCCuEEEKIMpOTbnQETNwKPd+B1mPNTmQxF4upI+lsPpLGruNGMWWjICLQkzHtw2hX2492tXxl2YMQ1yDFFUCL+yHuF2P9Va07wS/8th+ybS1ffny8HWO+2Mbw6Zt5b2hTejeudvtZhRBCCGF5WUkw627IOApDvoCGA81OVKq01mw9ms7Gw2lsPpLGzsRM8guNYqpRSTHVpqYvkWHeuDtVrimQQtwOKa7AGN7v/zF82gZ+egQe+A1sb/8/TS1/N356vB1jZ21n3Hc7OJZWl8c71ZI5yUIIIYQ1S95vFFb552HUj8a67Apk69F03liyn5jETGyU0ZTrvrahtK3lS2SYDx5STAlxy6S4+oN7Vej9HswbA+vfh44TSuVhfd0c+fah1rwwbzfvLDtAQlo2bwyMkCF1IYQQwhr9sTmwnTOMWQJVI8xOVGqOpJznrV/j+C32DFU9nHh7UAQ9GlWT5ltClCIpri7V6G5jeuCatyD8LqjetFQe1snelg/vaUqYrwuTVx4iKSNXOgkKIYQQ1mbPPKPBlVcIjCjFAe0AACAASURBVJoP3qFmJyoVaecvMPn3eL7dchxHOxue71aHBzvUxNnB1uxoQlQ4Ulxdrtc7cGy9MT1w7BqwdyqVh1VKMb5bXUJ8XaWToBBCCGFNigqMddebP4WQtjDs2wqxOXBeQRFfbDjGp6sOkVNQxPBWwTzdpQ7+7o5mRxOiwpK5aZdz8YH+n0BKHKx6vdQffnCLIL5+oDUp5y4w4JMN7DieUerXEEIIIcQNOp8MXw8wCqvWj8J9i8p9YVVcrPlpZxJ3vruat5fG0bqmD8ueieL1ARFSWAlhYVJcXUl4V4h8ADZ+DMc2lPrD/9FJ0NXRjuHTN/NDdCJa61K/jhBCCCGuISkapnWEE9th4HTo+Xa53xx44+FU+n2ynme/j8HXzZHZD7dh5n0tqR3gbnY0ISoFKa6u5q7/gHcY/PwoXDhX6g9fy9+Nn8e1p2mwFxPm7eaxb3aQnp1f6tcRQgghxGW0hugv4IueRjH14G/QZJjZqW7ZH23VH/hyGyNmbCEju4APhjVlwbj2tK1VvkfhhChvpLi6Gkc3GDjV2Odi2b8scgkfVwe+e7gNL/Wsx8q4ZLq9v5aVcWcsci0hhBBCAAV5sPBJWPwMhEXB2NVQrbHZqW7JhcIi5m9Pos9H6xk6bRM7jmfwQo+6/P5cRwY0C8TGRrZ+EaKsSUOLawlpA23HwcaPjGmCpdQ98FK2NopHOtbijjr+PPv9Lh74MpoRrUP4V6/6uDrK0yOEEEKUmsxEmDsaTu6EqOeh8z/Bpvx1zEs5d4FvtyTwzebjpJ6/QHiAG/8dGMHAZoHSAVAIk8m79+u5YwLs/BZWTIR7F1jsMvWrebDgifa899tBpq87wsZDqbw3rCnNQ7wtdk0hhBCi0ji6Fn64Hwrz4Z7voF5vsxPdtL0nsvhiwzEWxZwkv6iYO+sFMKZ9GB1q+6GUjFIJYQ1kWuD1OHkaBdaR1XB4pUUv5Whny0u96jPn4TYUFGkGT9nI/347QEFRsUWvK4QQlZVSqodS6oBS6pBS6sWrnDNUKRWrlNqnlPqurDOK26S1MQPl6/7g4gdjV5WrwqqoWLN07ymGTttEn4/W8+veUwxvFczK5zry+f0tiQr3l8JKCCsiI1c3ouWDsGUKLJ8INTqBjWVr0tY1fVn6TBSvLYrlo5WHWHUgmQ+GNZVOP0IIUYqUUrbAJ8BdQBKwTSm1UGsde8k54cBLQHutdYZSKsCctOKWFOYb+1bu+xHq94MBn4Jj+XgtzSso4rstx/l8w1GSMnIJ9HLm5d71GRIZjKdz+e5oKERFJiNXN8LOEe58BU7vhr3zyuSS7k72vDukCVNHteBkZh69J6/niw1HKS6Wlu1CCFFKWgGHtNZHtNb5wByg/2XnPAx8orXOANBaJ5dxRnE71v3PKKy6vApDvy4XhVVhUTHfbztO53dX8+/FsVT3dGbqqOasmdCJh6JqSmElhJWTkasb1WiwMa1g5X+gQX+j4CoDPRpVpXmoFy/N38Nri2JZsf8MHwxrJpsACiHE7QsEEi/5OQlofdk5dQCUUhsAW2CS1nrplR5MKTUWGAsQEhJS6mHFTTq9F9a9CxFDIeo5s9Ncl9aapXtP885vBziSkk3TYC/+N7QJ7Wr5mR1NCHETZOTqRtnYwF2vQeZx2PZZmV46wN2JmfdF8ubdEexIyGTI1I0kpueUaQYhhKik7IBwoBMwHJihlPK60ola6+la60itdaS/v38ZRhR/U1QICx4HZ29jY2Artz4+lf6fbOCxb3dgoxTTRrfgp8fbSWElRDkkxdXNqHUn1OwMa9+BvKwyvbRSiuGtQvj24dZk5hYwaMpG4k6fLdMMQghRwZwAgi/5OajktkslAQu11gVa66PAQYxiS1izjR/CqRjo/T9w8TE7zVXFJGYycuZmRn22hbTz+bwzuDHLnrmD7g2rSpMKIcopixVXSqnPlVLJSqm91zmvpVKqUCk12FJZSlXXSZCbDus/MOXyzUO8+eGRttgoxdCpm9h2LN2UHEIIUQFsA8KVUjWUUg7APcDCy875GWPUCqWUH8Y0wSNlGVLcpJQDsPotYwp/g8uX0FmHQ8nneHTWdvp/soH9p87xap8GrHy+I0Mig7GVjX+FKNcsOXL1JdDjWieUdGp6G/jNgjlKV/WmEDEENk+BsydNiRBexZ35j7fDz92RUTO38Pv+M6bkEEKI8kxrXQg8ASwD9gNztdb7lFL/Vkr1KzltGZCmlIoFVgETtNZp5iQW11VcBAvGgYMb9HrX7DR/czIzlxfmxdDt/bWsi0/h2a51WPtCZx7oUANHO9n8V4iKwGINLbTWa5VSYdc57UlgPtDSUjks4s6XIXYBrH4T+n1kSoRAL2d+eKQtY77cxthZ2/m/QY0Z1CLIlCxCCFFeaa2XAEsuu+3VS77XwPiSQ1i7zVMgaRvcPRPcrKdrflZuAZ+uPsQXG46BhjHta/B4p1r4uklzKiEqGtO6BSqlAoGBQGfKW3HlHQYtH4ItU6HNOAioZ0oMXzdHvnu4DY/Miua5H2LIyMnnoaiapmQRQgghTJV22OjoW6cnRFjHSoP8wmK+2ZzARyvjycwtYGDTQMZ3q0OQt4vZ0YQQFmJmQ4sPgH9orYuvd6JSaqxSKlopFZ2SklIG0W5A1PPGtIPfXzM1hpujHZ/f35LeEdV4/Zf9vPVrHMYHrUIIIUQlUVwMC58EW0fo8z6Y3AxCa83i3Sfp+t4a/r04lgbVPVj0RAfeG9ZUCishKjgz97mKBOaUdMPxA3oppQq11j9ffqLWejowHSAyMtI6KgdXX2j/tPEpWcImCG1rWhRHO1smD2+Gl4s9U9ccJiM7nzcGNsLOVppBCiGEqASiP4OEDdD/E/CoZmqUrUfTeWPJfmISM6lX1Z2vHmjFHeF+0v1PiErCtOJKa13jj++VUl8Ci69UWFm1No/Dtpmw/BV4cLmpn5TZ2iheH9AIXzdHJv8eT0ZOPpOHN8PJXhbICiGEqMAyjsHyiVCrCzQdaVqMQ8nneevXOFbsP0NVDyfeGdyYu5sHSfc/ISoZixVXSqnZGO1r/ZRSScBEwB5Aaz3VUtctUw4u0OklWPQUxC2G+n1NjaOUYvxddfB1dWDSon3c9/lWZtwXiYeTvam5hBBCCIvQGhY+BcoG+n5oyoecyefy+HBFPHO2JeJsb8uE7nV5oH0NnB3kw00hKiNLdgscfhPn3m+pHBbXdCRs+gRWvGYsorU1c6al4b52YXi52PPc3BjumbaZ6fe2kDneQgghKp4dX8HRNcY6K6/g659fioqKNZ+tP8IHK+LJLyxmdJtQnryztnQAFKKSk0U5t8vWzthYOC0edn5tdpqL+jcN5LP7W3I8PYfek9ezPFb2whJCVFxKqR+VUr2VUvK6VllkJcGylyEsCprfX6aXPpqazdBpm/jvkjja1fJl+fiOTOrXUAorIYQUV6Wibk8IbmPsCJ+fbXaaizrW8Wfxkx0I9nHm4a+jeX1xLPmF123OKIQQ5dGnwAggXin1llKqrtmBhAVpDYufBV1k7DdpUzZvZ4qLNV9uOErPD9cSf+Yc7w9rwox7I6nh51om1xdCWD8prkqDUnDXv+H8Gdj0qdlp/iLMz5X5j7XjvrahzFx/lCHTNpGYnmN2LCGEKFVa6xVa65FAc+AYsEIptVEpNUYpJQtPK5qYORD/G3SZCD41rn9+KUhMz2HkzC1MWhRLm5q+/PZsRwY2C5IugEKIv5DiqrSEtIZ6fWDDh5Cdanaav3C0s+W1/o34dGRzjiSfp/fkdfy277TZsYQQolQppXyB+4GHgJ3AhxjF1nITY4nSdu40LP2HMWOk1ViLX05rzXdbjtPjg7XsOZHF24Mi+OL+llT1dLL4tYUQ5Y8UV6Wp6yQoyIElz0PhBbPT/E2viGosfqoDob6ujJ21nX8vkmmCQoiKQSn1E7AOcAH6aq37aa2/11o/CbiZm07ctqICSN4Pe+bB/IeM19j+n1h8OuCprFzu+2Ib//xpD02CvVj6TBTDWobIaJUQ4qrMb21XkfiFQ+eXYOXrkHkchnxV5t2LrifU15V5j7XlzSVxfL7hKNsT0vl4RHOCfaSboBCiXJustV51pTu01pFlHUbcIq2NKfZn9sKZWDizzzhSD0BRvnGOjT30fBv8alswhubHHSeYtGgfhUWaf/dvyKjWodjInlVCiOuQ4qq03TEB/OrCz4/DtDtg0Eyo3cXsVH/haGfLpH4NaVPThwnzdtNr8jreGdyEHo2qmh1NCCFuVQOl1E6tdSaAUsobGK61tq6FsOLvjq6DuF+Mgio5FnLS/rzPvTpUaQi174QqjYzvfcPBzsFicZLP5fHPH/eyYv8ZWoZ5887gJoRJwwohxA1SWmuzM9yUyMhIHR0dbXaM60s9BHNHG9MYOr1kFF1l1M3oZiSm5/DEdzuIScpiTPswXupZHwc768sphKgclFLbb2WkSSm1S2vd9LLbdmqtm5VeuhtXbl6rzLbrO1gwDuycIKC+UTxVaQQBDYzvXXzKNM6agyk8M2cn2flFvNC9LmPa18BWRquEEJe51muVjFxZil9teGgFLB4Pq/8LSVvh7hll/kJxPcE+LvzwaDve+tWYJrgjIYMpo1pQ3cvZ7GhCCHEzbJVSSpd8YqiUsgUsN7whbt/WGcYa5RodYfhscDB3dOibzQlMXLiP8AA3Ph7RjNoB7qbmEUKUTzJEYUkOrjBwqrFz/NG1xjTBE9vNTvU3DnY2vNq3AVNHteBwSjb9Pl7PliNp1/9FIYSwHkuB75VSXZRSXYDZJbcJa7T+A6OwqtsLRsw1tbAqKta88UssL/+8lzvC/Zj3WDsprIQQt0yKK0tTCiIfgAeWAQo+7wHbPjMW7VqZHo2q8vO49ng42zNy5ha+3nSM8jZtVAhRaf0DWAU8VnL8DrxgaiLxd1obTZ9WTIRGg2Do12BvXkvznPxCHvtmOzPWHeW+tqHMuDcSN0eZ1COEuHVSXJWVwObwyBqocQf8Mh5+egTys81O9Te1A9z4eVx7Otbx59UF+/jH/N3kFRSZHUsIIa5Ja12stZ6itR5cckzTWss/XtZEa1j6Eqx9B5qNNqbK25q3v3Py2TyGTdvM8v1nmNi3Aa/1b4SdrbwtEkLcHvlXpCy5+MCIH6Dzv2D3XJjZ1Wh8YWU8nOyZcW8kT3UJZ250EsOmb+Z0Vp7ZsYQQ4qqUUuFKqXlKqVil1JE/DrNziRLFRbDwSdgyBVo/Bv0+Ahtb0+LEnT7LgE82cDjlPDNGRzKmfQ3TsgghKhYprsqajQ10fAFGzTd2mZ/eCWLmWN00QRsbxfi76jBtdAsOnTlHn4/Ws+1YutmxhBDiar4ApgCFQGfga+AbUxMJQ1EB/Pgw7JxldM7t8aYxZd4kqw8kM3jKJoq0Zu4jbenaoIppWYQQFc8NFVdKqaeVUh7K8JlSaodSqpulw1VotbvAI2uhSgNjiuC3g42Nh61M94bGOix3JzuGT9/MN5sTZB2WEMIaOWutf8fYYiRBaz0J6G1yJlGQB3Pvhb3zoeskuPNlUwurWZsTePCraEJ8XPh5XHsaBXqalkUIUTHd6MjVA1rrs0A3wBsYDbxlsVSVhVcwjPkVev4fJGyCT9rA5inG9AkrEl7FnZ/HtScq3I+Xf97LSz/u4UKhdWUUQlR6F5RSNkC8UuoJpdRAwM3sUJVafjZ8NxQOLIFe70KHZ02LUlSseX1xLK/8vJeOdfz54dG2VPOULUeEEKXvRourPz5m6gXM0lrvu+Q2cTtsbKH1IzBuC4S2g6Uvwmd3wZl9Zif7C09ne2be15InOtdmzrZE7pm+mTNnZR2WEMJqPA24AE8BLYBRwH2mJqrM8rJg1t1wbB0MmAKtHjYtSk5+IY9+s52Z649yf7swZtwbiat0BBRCWMiNFlfblVK/YRRXy5RS7kCx5WJVQl7BMPIHGPQZZBwz9sRa+boxpcJK2Noonu9elykjm3PgtLEOa3uCrMMSQpirZMPgYVrr81rrJK31GK31IK31ZrOzVTpaQ/oR+KovnIiGwZ9D0xGmxTmZmcuwaZv5ff8ZJvVtwKR+DbG1kc+GhRCWc6Mf3TwINAWOaK1zlFI+wBjLxaqklIKIwVCzMyz7p9GuNnYB9J0MoW3NTndRz4hq1PR3Y+ysaIZN28xLverzQPswlInz6IUQlZfWukgp1cHsHJVOYT6kHoTTe0qO3cbXvEywdYR7voM63U2Lt+FQKk/O3kl+YTEz7o2kS31pXCGEsLwbLa7aAru01tlKqVFAc+BDy8Wq5Fx94e5p0HgILHoWvugBkQ8ai4GdPMxOB0Ddqu4sfKIDE36I4T+LY9l6NI3/G9wET2fz9iwRQlRqO5VSC4EfgIubCGqtfzQvUgWSmwln9v61kEqOg+IC4347J6jSEBoOgKoRUKMT+NU2JarWmqlrjvDOsjhq+bsxdXQLavnL8jshRNlQN9L5TSm1G2gCNAa+BGYCQ7XWHS2a7goiIyN1dHR0WV/WPBfOw6o3jEYX7tWg9/+gXi+zU12kteaz9Ud569c4qns58+nI5tJ9SQhxy5RS27XWkbfwe19c4WattX6gFGLdtArzWnU+GZb9C/bM/fM2V3+o2tgoov44fGqBrfnrmM7mFfD83Bh+iz1Dn8bVeHtQY1lfJYQoddd6rbrRf3EKtdZaKdUf+Fhr/ZlS6sHSiyiuytHN2BOk0SBjA8Y5w6HFGOj5Ntg5mp0OpRQPRdWkWYg3T3y3g7s/3cirfRswsnWITBMUQpQZrbVMVS9NxcWw40tYMQnyc6DtE8aU9aoR4G6d0+sOnD7Ho99sJzE9h1f7NGCMTFcXQpjgRourc0qplzBasEeVtLuV+V9lKSgSxq6BVa/Dhg+NaRlDvwbPQLOTAdAi1Jtfnopi/NxdvPzzXrYeTee/d0fgJp8YCiHKQMnI1d+mYpg1clWund4Li5+BpG0QFgW93wP/OmanuqYFu07w4vw9uDnZ8d3DbWhVw8fsSEKISupGuwUOAy5g7Hd1GggC3rFYKnFldg5w17+NoiolzugoeHSt2aku8nF14PP7WjKhe10W7z5Jv4/XE3f6rNmxhBCVw2Lgl5Ljd8ADOG9qovImPxt+e9l4bUk/AgOnwX2LrLqwyi8sZtLCfTw9ZxcRgZ788mQHKayEEKa6oTVXAEqpKkDLkh+3aq2TLZbqGirMPPbblXIAvh8FaYeNRhftnjR11/vLbTqcxlNzdnIur4D/9G/EkMhgsyMJIcqBW11zdYXHsQHWa63blUKsm1buXqvilsCvL0BWIjS/F7q+Bi7WXaScOZvHuG93EJ2QwYMdavBiz3rY297oZ8ZCCHHrrvVadUP/CimlhgJbgSHAUGCLUmpw6UUUN82/Ljy8Eur1huWvwA/3w4VzZqe6qG0tX5Y8FUXzEG8mzNvNhB9iyM0vMjuWEKLyCAcCzA5h9bKSYM5IYz2vozs8sAz6fWT1hdXmI2n0nrye2FNn+XhEM17p00AKKyGEVbjRBTH/Alr+MVqllPIHVgDzLBVM3ABHd2OK4IYP4ffXjKmCw74Bv3CzkwHg7+7IrAdb8+GKg3y06hC7k7L4ZGQzage4mx1NCFHBKKXO8dc1V6eBf5gUx/oVFcKWqbDqv6CLjRkQbZ8AW+teTv1Hh9o3f40j1NeF2Q+3JryKvKYIIazHjX7MY3PZNMC0m/hdYUlKQYdnYPRPkJ0C0zvD/sVmp7rI1kYxvltdvhzTipTzF+g1eT3T1x6mqPjGpqMKIcSN0Fq7a609LjnqaK3nm53LKp1PgRmd4bd/QVgHGLcFOjxr9YVVXkERz36/i9d/2c9d9auwYFx7KayEEFbnRgukpUqpZUqp+5VS92MsGF5iuVjiptXsZHQT9KsN34+EFa9BsfVMw+tYx5+lz0TRsY4//10Sx5CpGzmSImvNhRClQyk1UCnlecnPXkqpAWZmslq/vwbJ+2HIVzDie/AONTvRdZ3KymXI1E0siDnJhO51mTKqOe5O1l0MCiEqpxsqrrTWE4DpGJsINwama62vOd1CKfW5UipZKbX3KvePVErtVkrtUUptVEo1udnw4jJewTBmqbEYef178M0gyE4zO9VFAe5OTB/dgveHNeFwSjY9P1zHzHVHZBRLCFEaJmqts/74QWudCUw0MY91Or0Hdn4DrR+BhgOsqhHS1WxPSKfvRxs4mprNjNGRjOtcW/avEkJYrRue2qe1nq+1Hl9y/HQDv/Il0OMa9x8FOmqtI4D/YBRv4nbZOxmLkftOhoQNMC0K1r4D6UfNTgYYmw4PbBbE8mfvICrcj9d/2c+waZs4mpptdjQhRPl2pdcz2WjvUlrDsn+CszfcMcHsNDfk+23HuWf6Ztwcbfnp8XZ0bWCdGxgLIcQfrllcKaXOKaXOXuE4p5S65gZGWuu1QPo17t+otc4o+XEzxt5ZorS0uA8eWApeobDydZjcFGZ2hS3T4LwpXfT/IsDDiRn3RvK/IU04eOYcPT9cy+frj1Iso1hCiFsTrZR6TylVq+R4D9hudiircnCpsTdip5fA2cvsNNdUUFTMxAV7+cf8PbSp6cuCcR1kfZUQoly44X2ubunBlQoDFmutG13nvOeBelrrh65y/1hgLEBISEiLhISEUk5awWUmwt75sGcenNkDysZYoxUxBOr1AScPU+OdzsrjpR93s+pACq1q+PDO4MaE+rqamkkIYY5b3edKKeUKvAJ0xegauBx4Q2ttyrC41e1zVVQAn7Yx/v1/bKNVN69Iz87n8W+3s/lIOmPvqMkL3etiJ23WhRBW5FqvVaYXV0qpzsCnQAet9XUXCFndC1Z5k7zfKLL2/ACZCWDrCHW6G4VWeDdjWqEJtNbM257EvxfFUlisebFnPUa3CcXGRubVC1GZlNYmwmazuteqLdOMTYJHzDX+zbdSsSfPMnZWNMnnLvD2oAgGNpNJLUII63PbmwhbilKqMTAT6H8jhZUoBQH1ocsr8HQMPLgCWtwPxzfB3NHwbjj88jzkl/0HvUophkQG89v4O2hVw4eJC/cxYuZmjqfllHkWIUT5o5RarpTyuuRnb6XUMjMzWY3cDFj9pjFjIbyb2Wmu6tc9pxg0ZSOFRZofHmkrhZUQolwyrbhSSoUAPwKjtdYHzcpRaSkFwS2h1//B+DgY9SPU7QXbZsJn3SDjmCmxqnk68+WYlrw9KIK9J87S7YM10lFQCHEj/Eo6BAJQsqY34Hq/pJTqoZQ6oJQ6pJR68Qr336+USlFK7So5rjh93aqteQdyM6HbG1bZHbC4WPPebwd47Nsd1K/mzsIn2tMk2LrXhAkhxNVYrLhSSs0GNgF1lVJJSqkHlVKPKqUeLTnlVcAX+LTkBcuK5k9UMrZ2ULsL3D0NRs6DrESY3gmOrDYljlKKYS1DWD7+DtrXMjoK3j1lIwdOnzMljxCiXCgu+dAOuDgt/ZqfyiilbIFPgJ5AA2C4UqrBFU79XmvdtOSYWXqRy0DaYdg6HZqPhqrXXP5siryCIh77djuTVx5iaGQQs8e2IcDDnOnpQghRGizWplZrPfw69z8ElL9PACu68K7w8CqYMxJmDYRur0Obx035tLOapzMz74tk0e5TTFq4jz4freOxTrUZ17kWjna2ZZ5HCGHV/gWsV0qtARQQRUkjpGtoBRzSWh8BUErNAfoDsZYMWqaWvwp2jtD5ZbOT/E1WTgEPfb2N6IQMXu3TgDHtw2T/KiFEuSftd8Tf+daCh5Yb0wSX/RN+egQKck2JopSiX5PqrBjfkT6NqzP593j6TF7PjuMZ1/9lIUSlobVeCkQCB4DZwHPA9f7hCgQSL/k5qeS2yw0q2fR+nlIq+GoPppQaq5SKVkpFp6Sk3NwfYAlH10HcYujwLLhb1/5Qp7PyGDptEzGJWXw0vBkPdKghhZUQokKQ4kpcmaM7DJ1lfNq5+3v4vLvR0t0kPq4OvD+sKV+MaUn2hUIGTdnIa4v2kX2h0LRMQgjrUbIW6neMoup5YBYwqRQeehEQprVujNHe/aurnai1nq61jtRaR/r7+5fCpW9DcbHx4ZhHELQdZ26WyxxKPs+gKRs5kZnLl2Na0qdxdbMjCSFEqZHiSlydjQ10nADD50D6UWMd1rH1pkbqXDeA38Z3ZHSbUL7YcIzuH6xlXbwVfEIshDDb00BLIEFr3RloBmRe+1c4AVw6EhVUcttFWus0rfWFkh9nAi1KJ66F7Z4Dp3dD10lg72x2mot2Hs9gyP+3d9/xUVXpH8c/Z9I7aQTSSEIJvXfpuCo2cFXEtta1rK7+1C1u1dXtu6Jb7Lu2FUVFcVlFLKgg0ntvoSb0BBJCCGnn98cdBBGQMpObzHzfr9d9zcydO3eemwv35Mk95znPzORQTR3jb+tL/1YpbockIuJTSq7k2+WPgFunQlQivDIS5jwHfpwf7dvERoTyyMiOvHVHP8JDPVz/77n86K0l7Kuoci0mEXFdpbW2EsAYE2GtXQ3kf8tn5gGtjTG5xphwYAww6egNjDHNj3p5KbDKhzH7R9UBmPoIZPSAjpe7Hc1XPluzi2uen0N8VBhv39mPjhkJbockIuJzSq7k1KS2ge9PhVbnwgc/hkl3Q3WlqyH1ykli8j0DuWtoSyYuKuLcsdN4ZloBpRXVrsYlIq4o9M5z9S7wsTHmv8Dmk33AWlsD3A18iJM0vWmtXWGMecQYc6l3s3uMMSuMMUuAe4Ab/XYEvvLl32H/djj/D04PhAbg7QWFfP/l+bRsGsOEO/rTIjnG7ZBERPzCWBfvQJyJBjfrfbCpq4Npf4Rpf3L+KnrVqxDvfn/5FdtK+d37q5hZUEx0eAije2Zx0zk5asBFGpmTzXp/GvsYDCQAU6y1rtzSdq2tKtsGf+8O+RfAlS/V//cfx3PTC/j95NWc0yqZZ67rQVxkmNshiYiclZO1VX4rxS4ByuOBoT+HZp1g4h3wr3Phe5Mgu1z01wAAIABJREFUpZWrYXVIT+C17/dlxbZS/j1jI+PmbOblWZs4r30atw7Mo2eLRFWiEgkS1tppbsfgmqmPgK11xlq5rK7O8ocPVvH8Fxu5qHNzxo7uomk0RCTgNYz+AtL4tLsEbp4CNYfgpQth12q3IwKcJGvs6K7M+OkwfjCkJXM2lnDlM7MY9eSX/HdxEdW1dW6HKCLiH0ULYcnr0PdOSMxxNZTq2joeeGsJz3+xkRv75/CPMd2UWIlIUFByJWeuWSe4aTJgnARr+1K3I/pKWnwkPz6/LTMfHMajozqyv7KGe8cvZtCfP+PZaQWUHtS4LBEJINbCR7+E6BQY+ICroRw4VMMtL89n4qIifnx+Pg9d0h6PRz0HRCQ4KLmSs5Oa7yRYoVHw8sVQuMDtiL4mOjyU6/u24JP7B/PvG3qSkxzDHz5YTb8/TOXhSSvYtOeA2yGKiJy91e/B5i+dbtuR7lXhm7FuDxf/YwYz1u3mT5d34q6hrdQlW0SCipIrOXvJLZ0E63Cp9s2z3I7oGzwew/B2abx+W1/e++EALujQjHFzNjP0sc+5+aV5TF+7m8ZW3EVEBIDaavjoV5DaFrrf4EoIu/ZXcu/4RVz37zkAvHprH67qle1KLCIiblJyJb6R2AJu+gDimsGr34UNn7sd0Ql1zEhg7FVd+fKnw/jhsNYsLdzH916Yy3cen85/Zm/mwKEat0MUETl1S9+EvRudIhYh9VunqrbO8p/Zmxn+2DQ+WLaDe4e35oN7B9K/pSYHFpHgpFLs4lvlu+CVUVC83inT3uY8tyP6Vodqanl/6XZe/HITy4pKiYsMZUyvLL7XL4espGi3wxMJKr4oxd4Q1FtbVVcLT/aGsCi4/Quoxy54y4tK+cW7y1mydR/ntErm0ZEdyUuNrbfvFxFxi0qxS/2JbQo3vgf/GQXjr4ErX3QqCzZgEaEhfLd7Jpd1y2Dhlr28+OUmXvhyE/+asZFz26Vx0zk59MtL1rgBEWl4Vk1y/ph15Uv1lliVH6ph7EdreWnmRpJiwvnbmK5c2iVd10gREZRciT9EJzlzX427At68Ab77HHS6wu2ovpUxhh4tkujRIontpQcZN3sLr83dwscrd5KfFsctA3P5brcMQkPUm1ZEGgBrYfpjkNwa2l1aD19nmbJ8B7/530p27q/kmt7Z/OT8tiREa1JgEZHD9Fui+EdUE7h+ImT3hbdvhUXj3I7otDRPiOJH5+cz88Fh/PmKzng8hp9MWMp5j0/nvaXbqKtrXN1pRSQArfsIdi6DAfeBx79zSG0tqeDml+Zx57iFJMaE8/ad/fndZZ2UWImIHEN3rsR/IuLg2glO98D//gBqDkKvW92O6rREhoUwumcWV/bI5OOVO/nrR2u4+7VFdEgv4Efn5zOkTaq6wohI/bMWpv8VErKg82i/ftW7i4p48J2leIzhlxe148b+ObqDLyJyAro6in+FR8PV46HNBfD+A/DZ76G28VXjM8ZwXodmfHDvIB6/qgtlldXc9OI8rnp2NvM2lbgdnogEm00zoHAunHMvhPjv7tErszbxf28spmtWEz65fzC3DsxTYiUichK6Qor/hUXC6P9Al6th2p/ghfNgzzq3ozojIR7DZd0ymXr/EB4d1ZGNxQe48plZ3PTiXFZsK3U7PBEJFl88BjFNodt1ftm9tZYnP1vPr/+7gnPbpfHSTb1JbxLll+8SEQkkSq6kfoSGw2XPOBWtSjbAMwNg9jNQV+d2ZGckPNTD9X1bMP3HQ3lwRFsWbtnHRX+fwd2vLWTD7nK3wxORQFa0ADZ8Bv3uckqw+5i1lj9+sJq/fLiGUV3Tefq67kSG+XdMl4hIoFByJfWrw2Xwg9mQOxim/BT+MxL2bXU7qjMWFR7CHYNbMv0nQ/nhsFZ8unoX33l8Og++vZR1O/fT2OaRE5FG4IuxEJkAvW7x+a5r6yw/n7icZ6dv4Pq+LRg7uith6gYoInLKVNBC6l9cM7jmDVj4Cnz4c3i6P4z4k9NtsJEWh0iICuOB8/L5Xr8cnvp8PeNmb2H8vK2kxUdwTssUzmnlLM0SIt0OVUQas50rYfV7MPinTtEgH6qureP+N5fwvyXbuGtoS350Xr4K9oiInCYlV+IOY6DHDZA3GCbeCe/eCaveg0v+BrGpbkd3xlLjInjokg7cPqgln63ZxYz1e/h87W7eWVQEQF5qDANapdC/ZQr98pJVxlhETs+MxyEsBvrc4dPdVlbX8oNxC/l09S4eHNGWOwa39On+RUSChZIrcVdiDtz4Hsx+CqY+Ak/1hUuegHaXuB3ZWWmWEMnVvbO5unc2dXWWVTvKmLm+mBnr9/DW/EJembUZj4FOGQn0b5XCgFYp9M5NUvcbETmxkg2wfIIz1io6yWe73V9ZzS0vz2fephJ+d1lHru3Twmf7FhEJNkquxH2eEOj/Q2h1LrxzG7xxndNFcMSfnHEFjZzHY+iQnkCH9AS+PyiPqpo6Fm/dx4z1e5i5fg/PTd/A058XkBIbzuU9MhnTK5vclBi3wxaRhmbGE+AJg353+2yXJQequPHFuazcVsYTV3VlZNcMn+1bRCQYKbmShqNpO7h1Kkz/i1NmeOMX0PNGyBkI6d2dioMBIDzUQ+/cJHrnJnH/d9pQfqiGL9fvYcKCQv71xUaenbaBvnlJXN07m/M7NFOVLhGBsm2w+DXo/j1n3KoP7Cit5Pp/z2FzSQXPXt+D4e3SfLJfEZFgpuRKGpbQcBj2C2fS4ck/gk9/610fBdl9nEQrZyCkdwuYZCs2IpTzOzTj/A7N2FlWyYQFhYyft4V7xy+mSXQYl3XL4Ore2bRJ8+3gdRFpRGb+A2wdnHOPT3a3pbiCa/89m5LyKl6+qTf9Wib7ZL8iIsHONLZS0T179rTz5893OwypLweKYfOXsGmGs+xa4awPi4asPpAzwEm2MrpDSOAUh6irs8wsKOb1eVv4aMUOqmst3bObMKZ3Nhd3bk50uP4uIoHJGLPAWtvT7TjOlk/bqgN74IlO0H6kM1/gWdpSXMEVz8ykqraOl2/qTZesJj4IUkQkeJysrfLbb2jGmBeAi4Fd1tqOx3nfAH8DLgQqgButtQv9FY80UjHJ0P5SZ4Gjkq0vnGTr00ed9WHRTqI14s+QlOtevD7i8RgGtE5hQOsUissP8c7CIl6ft4WfTFjKo/9bychu6VzVM5uOGfEqlSwS6GY/DdUHYcB9Z72rg1W13P7qAiqra3nrjv7kN9MdcRERX/Lnn79fAv4JvHKC90cArb1LH+Bp76PIiX0j2dpz5M7W0jfhxRFw/bvQtK27cfpQcmwE3x+Ux60Dc5m3aS/j527hrfmFvDp7C+2axzO6ZyajumaQGBMY3SRF5CiVpTD3eaeCamr+We3KWsvP3lnK6h1lvHBjLyVWIiJ+4Le6z9ba6UDJSTYZCbxiHbOBJsaY5v6KRwJUTIrTVebCv8BNH4C1ToJVFHg3QY0x9M5NYuxVXZn7i3N5dFRHQj2G3/xvJX1+P5W7X1vIF+t2U1fXuLr6ishJzPsXHCqFgQ+c9a5enrmJdxdv475z2zA0v6kPghMRkWO5OXAjA9h61OtC77rtx25ojLkNuA0gOzu7XoKTRiitPdz8AbwyCl6+FK4Z73QVDEAJUWFc37cF1/dtwcptZbw5fysTFxXx3tLtZDSJ4ooemVzRI5OspGi3QxWRM1VVAbOecqapSO96Vruat6mE376/inPbNeXuoa18FKCIiByrUcxYaq19zlrb01rbMzU11e1wpCFLyoObp0BCBrx6OayZ4nZEftc+PZ6HL+3AnJ8P5x9XdyMvNYa/f7qOQX/5jOv+NYdJS7ZRWV3rdpgicroWvgIVe2Dgj85qNzvLKvnBuIVkJUUz9qqueDwapyki4i9u3rkqArKOep3pXSdyduLT4cbJMO5yeONauOxZ6HSF21H5XWRYCJd0SeeSLukU7q1gwoJC3ppfyD2vLyImPIQOGQl0TE+gQ3o8HTLiaZUaS2hIo/j7ikjwqamCmX+HFudAi35nvJuqmjp+MG4h5ZU1vHpLH+IjA6eqqohIQ+RmcjUJuNsYMx6nkEWptfYbXQJFzkhMMnxvErx+Nbx9qzMovNctbkdVbzITo/m/c9twz7DWzCwoZsqK7azYVsZrczdTWV0HQESoh7bN4mjvTbg6ZiTQtlmcJi0WaQiWjoeyIrj072e1m9+9v5IFm/fyj6u7qYCFiEg98Gcp9teBIUCKMaYQeAgIA7DWPgNMxinDvh6nFPtN/opFglRkPFw3Ad66Ed6/Hw6V+aSUcWNydEl3gNo6y4bd5azYVsaKbaUsLyrj/aXbeH3uFgBCPIaWqTF0zmxC79wk+uYmk5UUpXLvIvWptgZmPA7Nu0LL4We8m7cXFPLyrM18f2Aul3RJ92GAIiJyIn5Lrqy1V3/L+xa4y1/fLwJAWBRc9SpMvAM+edi5gzX8IQjSZCHEY2idFkfrtDhGdcsAnPLMhXsPsmJbGSu3lbJ8Wxmfrt7FhAWFAKQnRNInL5k+uUn0yUsmJzlayZaIPxkDQ34GMalnfK1aXlTKzycuo29eEj+9IHCmphARaejc7BYoUj9CwuC7zzl3smY87iRYFz4GHo03AqfEe1ZSNFlJ0VzQsRngJFzrdpUzZ0MxszeW8MW63Uxc5AyJTIuPoE9uMn3ykuibl0xeSoySLRFf8oRA59Fn/PF9FVXc8eoCEqPD+ec13TW2UkSkHim5kuDgCYGLxkJEPHz5BBzaD6OedhIv+QZjDG3S4miTFsf1/XKw1lKw+wBzNhYzZ0MJszcUM2nJNgBSYiM4p1Uyw9ulMbhNKglR+pmKuKW2znLP+MXsKjvEG7f3JSU2wu2QRESCipIrCR7GwHd+A5EJMPU3UFrkzIMV1wzimh95jEmFEP3XOJoxhlZNY2nVNJZr+7TAWsum4grnztaGYr5Yt4f/Lt5GqMeZ6Hh4uzTObdeUFskxbocuElQe/3gt09fu5veXdaJbdqLb4YiIBB39BinBZ+D9EJUI0/4MW2eDrfv6+8YDMU2/mXSl5kOb851xXEHOGENuSgy5KTGM6Z1NbZ1l8da9fLJqF1NX7eTR91by6HsradU0luHtmvKddml0y04kRPPriPjNRyt28M/P1nNVzyyu7p317R8QERGfM05dicajZ8+edv78+W6HIYGitgYO7Ib922H/Dijf4Twefn348cBuZ/uIeGh/KXQe48w/o3Fbx7WluIJPVu1k6uqdzNlQQk2dJSkmnCH5qZzbLo1eOUmkxqm7knyTMWaBtban23Gcrfpuqwp2lzPyn1+SlxrDm7f305QKIiJ+dLK2SneuJLiFhEJ8c2c5mZoq2DITlrwBK96FRa9CQhZ0uhK6jHHuaslXspOjuXlALjcPyKWssprpa3czddUupq7axTsLncIYyTHh5DeLI79ZHG2bxZHfLJ42abFEh+uyJHI6amrruPPVBYSHenj6uh5KrEREXKTfYkRORWg45A1xloseg9XvO5N8fvkEzBjrzEfTZQx0vAJiU92NtYGJjwzj4s7pXNw5nZraOhZt3ceywlLW7NjP6p37GT93KwerawFnWFx2UjT5aUcSrrbN48hNjsGjLoUix7WksJS1O8sZO7oLGU3UbVlExE1KrkROV3g0dL7SWfbvhOUTYMl4mPIgfPgLaDUcOl8FbS/S+KxjhIZ46JWTRK+cpK/W1dVZtpRUsHrHftbs2M+anWWs3rGfT1btpM7ba7lJdBi9cpKcubZyk2mfHq/xWyJeszcUAzAkv6nLkYiIiJIrkbMRlwb97nKWXatg6Ruw9E14+xZnfFbH70LXayGzV9BOXPxtPB5DTkoMOSkxX82zBVBZXcv6XeWs3F7G/E0lzNlYwscrdwIQFxFKj5xE+uQm0zs3ic6ZCYRpLh8JUjML9tC2WRxJMeFuhyIiEvSUXIn4StN2cO7DMOzXsHkGLH7NSbQWvATJraHrNU7Xwfh0lwNtHCLDQuiYkUDHjARG93Qqn+0orWTOxmLmbnSSrc/XrAYgKiyE7i2a0Cc3mV45SXTKTCA2Qpc3CXyHamqZv2kv1/TJdjsUERFByZWI73k8kDvIWS78i1MAY/Frztxanz4KLYc5iVb+RRAWeWr7rK2G4gLYvQp2rYbSrZDRHfIvDKpkrVlCJCO7ZjCyawYAe8oPMc+baM3ZWMLjn6zFWucmYV5KDJ0yEuiU2YROGQl0SI8nRgmXHIcx5gLgb0AI8C9r7R9PsN3lwASgl7W2QZStXbRlH4dq6ujfMsXtUEREBCVXIv4VEQfdr3eW4gJY8josfh0m3OxMZtzxCqfbYEZ3JyP4Kola7Sy7VjmPxeuhrsa7UwPRybB4HLz/AKR3cxK1/BGQ1uHMux9WlEDhfCicC5Vl3gRxoBNnA5USG8GITs0Z0cmp9lhaUc2CLSUsKyxjWVEpszYU8+7ibYDzY2mZGksn792wzpkJtG+uhCvYGWNCgCeB7wCFwDxjzCRr7cpjtosD7gXm1H+UJzazoBiPgd65Sd++sYiI+J3muRKpb3V1sGk6LBoHqyZBTSWktAET4k2iqr0bGkjMcbobprb1PuY724ZGwp61TtXCNZOhcJ7zkSYtnLtZbS+E7P5OqfnjxlDrJG6Fc2HrPOexeL33a0MgNAKqK5znWb2du20th0N6V/A0rjLPu/ZXsryolKWFpSwvKmVZUSk7yw4BTsKVnxbHgFYpDM5PpVdOkspYu6y+57kyxvQDHrbWnu99/TMAa+0fjtnuCeBj4MfAj77tzlV9tVWjn5lFZU0tk+4e4PfvEhERh+a5EmlIPJ4jZd0r/+p0G1zxDoRGQf4FkNoOmrZ1xmmFR594P6n5zjLwfqdq4doPYPVkmP8CzHkaIptAm/OdZCuzJ+xcAVvnOolU0UKoKnf2E53iJFBdr4HM3s5dNE+Ys13Bp7B+Knz2O2eJSnTibjncSbgSMvz90zprTeMiGdY2kmFt075at6uskmXeRGvephJembWZf83YSGSYh755yQxuk8rgNqnkpsRgVIgk0GUAW496XQj0OXoDY0x3IMta+74x5sf1GdzJHKyqZdHWvdw8INftUERExEvJlYibIhOgxw3Ocjbi0qDHjc5yqNxJitZ8AGunOBUMDzMhTtfBLmOcRCqrFyTmHr8rYc4AZxn+aziwBzZ87iRaBZ/CionONqltnSQrZ4DTPTGueaOoitg0PpLh8ZEMb+ckXBVVNczZUMK0tbuZtnY3v/mf0yMsMzGKwW1SGdQmlf4tk4mLDHMzbHGBMcYDjAVuPIVtbwNuA8jO9n+BifmbS6iutfTLS/b7d4mIyKlRt0CRQFZbA1vnwI5l0KyjkwCFx5zdPq2FXSuP3NXaPBNqnW52xDR1ug4273rkMT791BIua6G0EPasgd1rjzyWFkLn0TDox6deAOQsbSmuYNq63Uxbs5uZBXuoqKol1GPo0SKRLllNiI8MJS4yjLivPYYS730eGxFKqErDn5GG1i3QGJMAFADeW700A0qAS0/WNbA+2qo/TVnN89M3sOSh8zR2UESkHp2srVJyJSJnp/ogbF8K2xfDtsXO4+7VYOuc92NSv55sNe/ijDPbvcabQHmXPeug+sCR/UYlQkq+kwwWTHW6SV76D2jRr14Pr6qmjgWb9351V6tgdzlVNXXf+rno8BDiIkO5bVBLblG3rVPmQnIVCqwFhgNFwDzgGmvtihNs/zkNZMzVyCe/JMxjmHBnf79+j4iIfJ3GXImI/4RFQXYfZzmsqgJ2Lj+SbG1b7NzpsrXf/HxcujN2rPv1TrGO1HwnqYpJOXLHa/0n8L/74MULoNetMPwhiIyvl8MLD/XQr2Uy/Vom8+CItoAzt9D+yhrvUv3VY9kx6xZv3ccfJq9iYOsU2qTF1Uu8cnqstTXGmLuBD3FKsb9grV1hjHkEmG+tneRuhMdXVlnNssJ93DW0lduhiIjIUZRciYjvhUc7RTKyeh9ZV1XhFNXYsQTCop0EKqX1qSVJrc6FH8yCT38Lc55xxpNd/LhTsMMFEaEhRMSGkBIbceKNrKWkrJxhf5vFL99dzhu39VVxjAbKWjsZmHzMul+fYNsh9RHTt5m3sYQ6C/1aaryViEhDouRKROpHeLRTQCOr15l9PiIWRvwROl4Ok34Ir4125gkb8SfnLpc/WQt7NznzgO3d5FRarCp3iodUlcOh/VB14Kh1zuukuhompl/E0A3X8O7iIi7rlunfOCVozCooJjzUQ/fsRLdDERGRoyi5EpHGJasX3D4dZoyF6X91uhte8Een6IWv7gxVlsG2hd5Jlec784hV7Dnyfkg4hMc6CV94nDMuLDLeKd4REXfkvf07yV38Kr9IzuB37zvl4BOiVHFQzt7MgmJ6ZCdqXjYRkQZGyZWIND6h4TDkQWg/0rmLNfE2WPYWXDwWmpxmCey6OqewRuE87zLfmWAZb7GflDZO98OMHpDZyxkTFnqS7oBHsxYqirll/Yu8V5nF4x835+FLO5xefCLH2HugipXby3jgO23cDkVERI6h5EpEGq+m7eDmD2Hu8zD1EXiyLwz+sTPf1ldd9A54l/1Hnh/uzld1APbvcN4DZ+LlzF7QfhRk9nASqqiz6HZlDIx6Cs+zg3gp9CmGzGrGFT0y6ZiR4Jvjl6A0Z2MxoPFWIiINkZIrEWncPCHQ9w7IHwHv3QefPHzMBuaoLnwx3iUOYtOc5y2HQnp3J6lKbun7SZCjk+CKF2jy4ggej3ieX72bztt3noPHo+IWcmZmFhQTHR5C58wmbociIiLHUHIlIoEhsQVc9zYUF4DH4yRU4bFOqXi3q/Rl9cYMf4hhH/+Kadve5K0F2VzV6zS7L4p4zSoopldOEuGhmqhaRKSh0ZVZRAKHMZDSCpLyILapU6HQ7cTqsH53Y9uczy/DxjFp8vvsPVDldkTSCO3aX8m6XeXqEigi0kApuRIRqQ8eD2bUMxDblN/XjeVvkxe4HZE0QrMKnPFW/ZVciYg0SEquRETqS3QSYaNfIssU03vpr1m8Za/bEUkjM3tDMXGRoXRIV1EUEZGGSMmViEh9yu5D9ZBfcGHIXGa98Udq66zbEUkjMrOgmD65yYSoIIqISIPk1+TKGHOBMWaNMWa9MebB47yfbYz5zBizyBiz1BhzoT/jERFpCCIG3cfOtEHcXP4vpnw8xe1wpJEo2neQzcUV6hIoItKA+S25MsaEAE8CI4D2wNXGmPbHbPZL4E1rbTdgDPCUv+IREWkwPB6afu9FykOa0GnWvRQX73E7ImkEDo+3UjELEZGGy5+l2HsD6621GwCMMeOBkcDKo7axQLz3eQKwzY/xiIg0GCYmhYMjnyf9nctZ/vKtJN83sWFUNqw6AGs/hBUTYe9GiM9wloRMSMiCBO/zuOYQEuZ2tEFlZsEekmLCyU+LczsUERE5AX8mVxnA1qNeFwJ9jtnmYeAjY8wPgRjg3OPtyBhzG3AbQHa25oYRkcCQ2WUYn8+/gyFbn2LTh/8g54J73Ank6IRq3cdQcxBimkLzLlBaBFtmQ+W+r3/GeJwE66vEKxPSu0LHy905hgBnrWV2QTF985I0AbWISAPm9iTCVwMvWWsfM8b0A/5jjOlora07eiNr7XPAcwA9e/bU6G8RCRi9r/sNs/40ix6zH6Gm4yBCM7vWzxefKKHqdi10uAyy+4En5Mj2h8qhrAhKt0JpoZN0lRY6r7cvhtXvQd5QJVd+srm4gm2lldzZMsXtUERE5CT8mVwVAVlHvc70rjvaLcAFANbaWcaYSCAF2OXHuEREGozoiHAqLnqKkkkXEjXuOhIuewyqDzpLjfexuuLIuq+eV0BNFUTEQlQiRDZxHqOaHP91WNSRhGrlu7D2o68nVO1HQYv+X0+ojhYRC6n5znI8dXVOTOIXMzW/lYhIo+DP5Goe0NoYk4uTVI0Brjlmmy3AcOAlY0w7IBLY7ceYREQanGHd2/GHub/iJzt+BK+NPv5GJgTCY5wkKTQSwqKdMU/F5XBwn9Nt7+s3/b8uJAKwUFsFMamnllCdDo/HScDEL2ZtKKZpXAR5KTFuhyIiIifht+TKWltjjLkb+BAIAV6w1q4wxjwCzLfWTgIeAJ43xtyHU9ziRmutuv2JSFAxxnDt6DGc/0QozUPKGXNOG0Z0a0lIRLSTTB1OpE6mrg6q9sPBvU6ydXCvk3Ad/RoLrc/3XUIl9cJay6yCYga0SsY0hKInIiJyQn4dc2WtnQxMPmbdr496vhI4x58xiIg0Bi2SY3j67u/y6/8u5+6pJXRes41HR3akS1bCqe3A44HIBGdJ9G+sUr/W7ypnT/kh+mu8lYhIg+fXSYRFROTUtUmL4/Xv9+VvY7qyo7SSUU99yc/eWcbeA1VuhyYumqn5rUREGg0lVyIiDYgxhpFdM5j6wGBuPieXN+dvZehjn/P63C3U1anXdDCaWbCHzMQospKi3Q5FRES+hZIrEZEGKC4yjF9d3J737xlAm6Zx/OydZVz29EyWFZa6HZrUo7o6y+wNJfTL010rEZHGQMmViEgD1rZZPG/c3pfHr+pC0d6DXPrkDH757jL2VairYDBYub2M0oPV9G+l5EpEpDFQciUi0sAZY7isWyZTHxjMDf1yeG3OFoY9No035m2hquYk5del0Zu9wTveKk/FLEREGgMlVyIijURCVBgPX9qB9344kNyUGH769jJ6/PZj7ntjMR+t2EFlda3bIYqPzSwoJi8lhmYJkW6HIiIip8CvpdhFRMT32qfH89bt/Zi2bjeTl27n41U7mbioiOjwEIa2bcqFHZszJD+VmAhd4huzmto65m4sYWTXdLdDERGRU6SWV0SkEfJ4DEPzmzI0vynVtXXM3lDMB8t38NGKHby/dDsRoR4Gt0llRKdmDG+XRnzkt0xCLA3OsqJSyg/VqAS7iEgjouRKRKSRCwvxMLB1KgNbp/LoyI5lD6qGAAAJZElEQVTM21TClOU7+GD5dj5auZOwEMOAVilc0LEZg9qk0jwhyu2Q5RQcnt+qryoFiog0GkquREQCSIjH0Dcvmb55yfz64vYs2rqPKcu3M3nZDj5bswyA1k1jnWSsTQp9cpOIDldT0BDN3lBM22ZxpMRGuB2KiIicIrWoIiIByuMx9GiRSI8Wifz8wnas3rGfGev2MH3dbsbN2cwLX24kPMRDjxaJDGyTwqDWqbRvHo/HY9wOPegdqqll3qYSxvTKdjsUERE5DUquRESCgDGGds3jadc8nu8PyqOy2vnl3Um29vDnKWv485Q1JMWEM6BVCgNap9A7J4mMxCjCQlRYtr4t3rKPyuo6+mu8lYhIo6LkSkQkCEWGhXw1TutnwO79h/hyvXNX64t1e5i0ZBsAHgPNE6LITIwiKymarMRospKOPG8aF6E7XX4wa0MxHgN9NN5KRKRRUXIlIiKkxkUwqlsGo7plYK1lzc79LC0spbCkgq17D7K1pIIv1u1mZ9mhr30uPNRDZpMoMpOi6ZuXxA+GtHLpCALLzIJiOqQnkBClKo8iIo2JkisREfkaYwxtm8XTtln8N96rrK6laJ+TbG3de9CbfFWwteQgG3cfcCHawGOtxQADW6e4HYqIiJwmJVciInLKIsNCaJkaS8vUWLdDCVjGGN64vR/WWrdDERGR06RRyiIiIg2QMRrLJiLS2Ci5EhERERER8QElVyIiIiIiIj6g5EpERERERMQHlFyJiIiIiIj4gJIrERERERERH1ByJSIiIiIi4gNKrkRERERERHxAyZWIiIiIiIgPmMY2A7wxZjew+Sx3kwLs8UE4jYGONXAF0/HqWAPT8Y61hbU21Y1gfElt1WnTsQYmHWtg0rGepK1qdMmVLxhj5ltre7odR33QsQauYDpeHWtgCqZjPRPB9PPRsQYmHWtg0rGenLoFioiIiIiI+ICSKxERERERER8I1uTqObcDqEc61sAVTMerYw1MwXSsZyKYfj461sCkYw1MOtaTCMoxVyIiIiIiIr4WrHeuREREREREfErJlYiIiIiIiA8EXXJljLnAGLPGGLPeGPOg2/H4kzFmkzFmmTFmsTFmvtvx+JIx5gVjzC5jzPKj1iUZYz42xqzzPia6GaOvnOBYHzbGFHnP7WJjzIVuxugrxpgsY8xnxpiVxpgVxph7vesD7tye5FgD7twaYyKNMXONMUu8x/ob7/pcY8wc7/X4DWNMuNuxNgTB1E6B2qpAuJ6B2irv+oA7t2qrTr+tCqoxV8aYEGAt8B2gEJgHXG2tXelqYH5ijNkE9LTWBtxEb8aYQUA58Iq1tqN33Z+BEmvtH72/kCRaa3/qZpy+cIJjfRgot9b+1c3YfM0Y0xxobq1daIyJAxYAo4AbCbBze5JjHU2AnVtjjAFirLXlxpgwYAZwL3A/8I61drwx5hlgibX2aTdjdVuwtVOgtooAuJ6B2irUVjV6vmqrgu3OVW9gvbV2g7W2ChgPjHQ5JjkD1trpQMkxq0cCL3ufv4zzn7/RO8GxBiRr7XZr7ULv8/3AKiCDADy3JznWgGMd5d6XYd7FAsOACd71AXFefUDtVABRWxWY1FaprTrZfoItucoAth71upAA/QfiZYGPjDELjDG3uR1MPUiz1m73Pt8BpLkZTD242xiz1NsVo9F3PTiWMSYH6AbMIcDP7THHCgF4bo0xIcaYxcAu4GOgANhnra3xbhLo1+NTFWztFKitCqjr2XEE3PXsaGqrAuvc+qKtCrbkKtgMsNZ2B0YAd3lv2QcF6/R3DeQ+r08DLYGuwHbgMXfD8S1jTCzwNvB/1tqyo98LtHN7nGMNyHNrra211nYFMnHuzrR1OSRpONRWBa6AvJ4dprYq8M6tL9qqYEuuioCso15netcFJGttkfdxFzAR5x9JINvp7Rt8uI/wLpfj8Rtr7U7vBaAOeJ4AOrfefs5vA+Oste94VwfkuT3esQbyuQWw1u4DPgP6AU2MMaHetwL6enwagqqdArVVBMj17HgC+Xqmtipwzy2cXVsVbMnVPKC1t+pHODAGmORyTH5hjInxDjzEGBMDnAcsP/mnGr1JwA3e5zcA/3UxFr86fPH2uowAObfewaT/BlZZa8ce9VbAndsTHWsgnltjTKoxpon3eRROsYZVOA3XFd7NAuK8+kDQtFOgtooA/3cfiNczUFvlXR9w59ZXbVVQVQsE8JaKfAIIAV6w1v7O5ZD8whiTh/MXQIBQ4LVAOlZjzOvAECAF2Ak8BLwLvAlkA5uB0dbaRj+49gTHOgTnVrwFNgG3H9XPu9EyxgwAvgCWAXXe1T/H6d8dUOf2JMd6NQF2bo0xnXEGAYfg/FHvTWvtI97r1HggCVgEXGetPeRepA1DsLRToLaKALmegdoq1FY1+nPrq7Yq6JIrERERERERfwi2boEiIiIiIiJ+oeRKRERERETEB5RciYiIiIiI+ICSKxERERERER9QciUiIiIiIuIDSq5EGjljzBBjzHtuxyEiInIiaqskWCi5EhERERER8QElVyL1xBhznTFmrjFmsTHmWWNMiDGm3BjzuDFmhTFmqjEm1bttV2PMbGPMUmPMRGNMond9K2PMJ8aYJcaYhcaYlt7dxxpjJhhjVhtjxnlnVBcRETktaqtEzo6SK5F6YIxpB1wFnGOt7QrUAtcCMcB8a20HYBrOjPYArwA/tdZ2xpkV/fD6ccCT1touQH/g8Gzo3YD/A9oDecA5fj8oEREJKGqrRM5eqNsBiASJ4UAPYJ73D3VRwC6gDnjDu82rwDvGmASgibV2mnf9y8Bbxpg4IMNaOxHAWlsJ4N3fXGttoff1YiAHmOH/wxIRkQCitkrkLCm5EqkfBnjZWvuzr6005lfHbGfPcP+Hjnpei/5vi4jI6VNbJXKW1C1QpH5MBa4wxjQFMMYkGWNa4PwfvMK7zTXADGttKbDXGDPQu/56YJq1dj9QaIwZ5d1HhDEmul6PQkREApnaKpGzpL8YiNQDa+1KY8wvgY+MMR6gGrgLOAD09r63C6evO8ANwDPeBmkDcJN3/fXAs8aYR7z7uLIeD0NERAKY2iqRs2esPdM7uyJytowx5dbaWLfjEBERORG1VSKnTt0CRUREREREfEB3rkRERERERHxAd65ERERERER8QMmViIiIiIiIDyi5EhERERER8QElVyIiIiIiIj6g5EpERERERMQH/h/WJrNf70qJUwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(1, (12, 4))\n",
    "plt.subplot(1, 2, 1)\n",
    "plt.plot(results.history['loss'])\n",
    "plt.plot(results.history['val_loss'])\n",
    "plt.ylabel('loss')\n",
    "plt.xlabel('epoch')\n",
    "plt.legend(['train', 'val'], loc='upper right')\n",
    "\n",
    "plt.subplot(1, 2, 2)\n",
    "plt.plot(results.history['accuracy'])\n",
    "plt.plot(results.history['val_accuracy'])\n",
    "plt.ylabel('accuracy')\n",
    "plt.xlabel('epoch')\n",
    "plt.legend(['train', 'val'], loc='upper left')\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6Cy3f9RuqlmB"
   },
   "source": [
    "### Plot predictions\n",
    "#### Task\n",
    "Investigate the predictions of the trained model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "EppuDY3LqlmB"
   },
   "outputs": [],
   "source": [
    "def plot_prediction(X, Y, Y_predict, fname=False):\n",
    "    \"\"\"\n",
    "    Plot image X along with predicted probabilities Y_predict.\n",
    "    X: CIFAR image, shape = (32, 32, 3)\n",
    "    Y: CIFAR label, one-hot encoded, shape = (10)\n",
    "    Y_predict: predicted probabilities, shape = (10)\n",
    "    \"\"\"\n",
    "    X = 128 * X + 128\n",
    "    labels = np.array(['airplane', 'automobile', 'bird', 'cat', 'deer',\n",
    "                       'dog', 'frog', 'horse', 'ship', 'truck'])\n",
    "\n",
    "    fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(8, 4))\n",
    "\n",
    "    # plot image\n",
    "    ax1.imshow(X.astype('uint8'), origin='upper')\n",
    "    ax1.set(xticks=[], yticks=[])\n",
    "\n",
    "    # plot probabilities\n",
    "    ax2.barh(np.arange(10), Y_predict, align='center')\n",
    "    ax2.set(xlim=(0, 1), xlabel='Score', yticks=[])\n",
    "\n",
    "    for i in range(10):\n",
    "        c = 'red' if (i == np.argmax(Y)) else 'black'\n",
    "        ax2.text(0.05, i, labels[i].capitalize(), ha='left', va='center', color=c)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "H5YkKktKqlmD"
   },
   "source": [
    "### Plot confusion matrix\n",
    "#### Task\n",
    "Plot the confusion matrix and comment on your findings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "yNqTA9DgqlmE"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4GgHFIlSqlmF"
   },
   "source": [
    "### Plot filter weights of the first layer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "u4f42N94qlmF"
   },
   "source": [
    "### Plot activations in convolutional layers\n",
    "#### Task\n",
    "Visualize the filters in the first convolutional layer of your CNN."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "id": "5asAr6KNqlmG"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EW8RJMngqlmG"
   },
   "source": [
    "#### Task\n",
    "Visualize the activations in the first two layers of your network for two input images of choice and describe what you see."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "uwoWZ4GTqlmG",
    "outputId": "78fa070c-c482-4ef0-c4b9-8bd32e10b8d1",
    "scrolled": false
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "name": "Exercise_12_1_solution.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}