01_neural_network_regression_in_tensorflow.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "01_neural_network_regression_in_tensorflow.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyPJ5Dpbkicrqvik8JYM2rWb",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/mrdbourke/tensorflow-deep-learning/blob/main/01_neural_network_regression_in_tensorflow.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UPgo18-N1gSi"
      },
      "source": [
        "# 01. Neural Network Regression with TensorFlow\n",
        "\n",
        "There are many definitions for a [regression problem](https://en.wikipedia.org/wiki/Regression_analysis) but in our case, we're going to simplify it to be: predicting a number.\n",
        "\n",
        "For example, you might want to:\n",
        "- Predict the selling price of houses given information about them (such as number of rooms, size, number of bathrooms).\n",
        "- Predict the coordinates of a bounding box of an item in an image.\n",
        "- Predict the cost of medical insurance for an individual given their demographics (age, sex, gender, race).\n",
        "\n",
        "In this notebook, we're going to set the foundations for how you can take a sample of inputs (this is your data), build a neural network to discover patterns in those inputs and then make a prediction (in the form of a number) based on those inputs.\n",
        "\n",
        "## What we're going to cover\n",
        "\n",
        "Specifically, we're going to go through doing the following with TensorFlow:\n",
        "- Architecture of a regression model\n",
        "- Input shapes and output shapes\n",
        "  - `X`: features/data (inputs)\n",
        "  - `y`: labels (outputs)\n",
        "- Creating custom data to view and fit\n",
        "- Steps in modelling\n",
        "  - Creating a model\n",
        "  - Compiling a model\n",
        "    - Defining a loss function\n",
        "    - Setting up an optimizer\n",
        "    - Creating evaluation metrics\n",
        "  - Fitting a model (getting it to find patterns in our data)\n",
        "- Evaluating a model\n",
        "  - Visualizng the model (\"visualize, visualize, visualize\")\n",
        "  - Looking at training curves\n",
        "  - Compare predictions to ground truth (using our evaluation metrics)\n",
        "- Saving a model (so we can use it later)\n",
        "- Loading a model\n",
        "\n",
        "Don't worry if none of these make sense now, we're going to go through each.\n",
        "\n",
        "## How you can use this notebook\n",
        "\n",
        "You can read through the descriptions and the code (it should all run), but there's a better option.\n",
        "\n",
        "Write all of the code yourself.\n",
        "\n",
        "Yes. I'm serious. Create a new notebook, and rewrite each line by yourself. Investigate it, see if you can break it, why does it break?\n",
        "\n",
        "You don't have to write the text descriptions but writing the code yourself is a great way to get hands-on experience.\n",
        "\n",
        "Don't worry if you make mistakes, we all do. The way to get better and make less mistakes is to **write more code**."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "etAu7oCZ8r_G"
      },
      "source": [
        "## Typical architecture of a regresison neural network\n",
        "\n",
        "The word *typical* is on purpose.\n",
        "\n",
        "Why?\n",
        "\n",
        "Because there are many different ways (actually, there's almost an infinite number of ways) to write neural networks.\n",
        "\n",
        "But the following is a generic setup for ingesting a collection of numbers, finding patterns in them and then outputing some kind of target number.\n",
        "\n",
        "Yes, the previous sentence is vague but we'll see this in action shortly.\n",
        "\n",
        "| **Hyperparameter** | **Typical value** |\n",
        "| --- | --- |\n",
        "| Input layer shape | Same shape as number of features (e.g. 3 for # bedrooms, # bathrooms, # car spaces in housing price prediction) |\n",
        "| Hidden layer(s) | Problem specific, minimum = 1, maximum = unlimited |\n",
        "| Neurons per hidden layer | Problem specific, generally 10 to 100 |\n",
        "| Output layer shape | Same shape as desired prediction shape (e.g. 1 for house price) |\n",
        "| Hidden activation | Usually [ReLU](https://www.kaggle.com/dansbecker/rectified-linear-units-relu-in-deep-learning) (rectified linear unit) |\n",
        "| Output activation | None, ReLU, logistic/tanh |\n",
        "| Loss function | [MSE](https://en.wikipedia.org/wiki/Mean_squared_error) (mean square error) or [MAE](https://en.wikipedia.org/wiki/Mean_absolute_error) (mean absolute error)/Huber (combination of MAE/MSE) if outliers |\n",
        "| Optimizer | [SGD](https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/SGD) (stochastic gradient descent), [Adam](https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/Adam) |\n",
        "\n",
        "***Table 1:*** *Typical architecture of a regression network.* ***Source:*** *Adapted from page 293 of [Hands-On Machine Learning with Scikit-Learn, Keras & TensorFlow Book by Aurélien Géron](https://www.oreilly.com/library/view/hands-on-machine-learning/9781492032632/)*\n",
        "\n",
        "Again, if you're new to neural networks and deep learning in general, much of the above table won't make sense. But don't worry, we'll be getting hands-on with all of it soon.\n",
        "\n",
        "> 🔑 **Note:** A **hyperparameter** in machine learning is something a data analyst or developer can set themselves, where as a **parameter** usually describes something a model learns on its own (a value not explicitly set by an analyst).\n",
        "\n",
        "Okay, enough talk, let's get started writing code.\n",
        "\n",
        "To use TensorFlow, we'll import it as the common alias `tf` (short for TensorFlow)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FMqsqKpk7TrH",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "319b5f13-6fff-47a3-fd8b-f7e85e7a67d0"
      },
      "source": [
        "import tensorflow as tf\n",
        "print(tf.__version__) # check the version (should be 2.x+)"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "2.7.0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8clMYxrF6Mzv"
      },
      "source": [
        "## Creating data to view and fit\n",
        "\n",
        "Since we're working on a **regression problem** (predicting a number) let's create some linear data (a straight line) to model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "G43tWFof6i7T",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 269
        },
        "outputId": "04bc42be-84b7-43e6-f10e-41e0a06ca1b0"
      },
      "source": [
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "# Create features\n",
        "X = np.array([-7.0, -4.0, -1.0, 2.0, 5.0, 8.0, 11.0, 14.0])\n",
        "\n",
        "# Create labels\n",
        "y = np.array([3.0, 6.0, 9.0, 12.0, 15.0, 18.0, 21.0, 24.0])\n",
        "\n",
        "# Visualize it\n",
        "plt.scatter(X, y);"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAOP0lEQVR4nO3df2jc933H8ddrigZHGlCCVWNpMR4lHIRBrU6EQctIadfL8o+Vf8LyR/FYwPmjgY6Vg6j/NDAGYdcf/2wUHBriQZtRqKKEUXrNTJkpjDG5MpXT7EgpNsvJsR26oxl8YYr63h/6npFcS/dDd/refe75AKG7z33le/NFeeb8/X7P54gQACAdv1f0AACAwSLsAJAYwg4AiSHsAJAYwg4AiSHsAJCYjmG3/bDtn9j+he23bX85X3/RdtP2lfzryeGPCwDoxJ2uY7d9QtKJiPiZ7QckXZa0JOlpSf8bEV8f/pgAgG7d12mDiLgh6UZ++0Pb70iaH/ZgAID+dHzFvmdj+5SkS5L+SNLfSPpLSb+RtCbpKxHxPwf9/LFjx+LUqVP9TQoAE+ry5csfRMRst9t3HXbbH5P0b5L+LiJWbB+X9IGkkPS32jlc81f3+Llzks5J0smTJ//4+vXr3c4GAJBk+3JELHa7fVdXxdielvQDSd+NiBVJioibEbEdEb+V9LKkx+71sxFxPiIWI2Jxdrbr/+EAAPrUzVUxlvQdSe9ExDd3rZ/YtdlTkq4OfjwAQK86njyV9GlJX5S0YftKvvZVSc/YPq2dQzHXJD03lAkBAD3p5qqYn0ryPR764eDHAQAcFu88BYDEdHMoBgDQp9X1pmr1hjZbmeZmSqpWylpaGO5bgQg7AAzJ6npTyysbyra2JUnNVqbllQ1JGmrcORQDAENSqzfuRL0t29pWrd4Y6vMSdgAYks1W1tP6oBB2ABiSuZlST+uDQtgBYEiqlbJK01N71krTU6pWykN9Xk6eAsCQtE+QclUMACRkaWF+6CG/G4diACAxhB0AEkPYASAxhB0AEkPYASAxhB0AEkPYASAxhB0AEkPYASAxhB0AEkPYASAxhB0AEkPYASAxhB0AEkPYASAxhB0AEkPYASAxhB0AEkPYASAxhB0AEsOHWQMYK6vrTdXqDW22Ms3NlFStlI/8w6JHHWEHMDZW15taXtlQtrUtSWq2Mi2vbEgScd+FQzEAxkat3rgT9bZsa1u1eqOgiUYTYQcwNjZbWU/rk4qwAxgbczOlntYnFWEHMDaqlbJK01N71krTU6pWygVNNJo4eQpgbLRPkHJVzMEIO4CxsrQwT8g74FAMACSmY9htP2z7J7Z/Yftt21/O1x+y/Zbtd/PvDw5/XABAJ928Yv9I0lci4lFJfyLpS7YflfSCpIsR8Yiki/l9AEDBOoY9Im5ExM/y2x9KekfSvKQzki7km12QtDSsIQEA3evpGLvtU5IWJP2HpOMRcSN/6H1Jxwc6GQCgL12H3fbHJP1A0l9HxG92PxYRISn2+blzttdsr92+fftQwwIAOusq7LantRP170bESr580/aJ/PETkm7d62cj4nxELEbE4uzs7CBmBgAcoJurYizpO5LeiYhv7nroTUln89tnJb0x+PEAAL3q5g1Kn5b0RUkbtq/ka1+V9JKk79t+VtJ1SU8PZ0QAQC86hj0ifirJ+zz8ucGOAwA4LN55CgCJIewAkBjCDgCJIewAkBjCDgCJIewAkBjCDgCJIewAkBjCDgCJIewAkBjCDgCJ6eYfAQOQuNX1pmr1hjZbmeZmSqpWylpamC96LPSJsAMTbnW9qeWVDWVb25KkZivT8sqGJBH3McWhGGDC1eqNO1Fvy7a2Vas3CpoIh0XYgQm32cp6WsfoI+zAhJubKfW0jtFH2IEJV62UVZqe2rNWmp5StVIuaCIcFidPgQnXPkHKVTHpIOwAtLQwT8gTwqEYAEgMYQeAxBB2AEgMYQeAxBB2AEgMYQeAxBB2AEgMYQeAxBB2AEgMYQeAxBB2AEgMYQeAxBB2AEgMYQeAxBB2AEgMYQeAxBB2AEgMYQeAxBB2AEhMx7DbfsX2LdtXd629aLtp+0r+9eRwxwQAdKubD7N+VdI/SPqnu9a/FRFfH/hEQAJW15uq1RvabGWamympWinzYdE4Mh3DHhGXbJ8a/ihAGlbXm1pe2VC2tS1JarYyLa9sSBJxx5E4zDH2523/PD9U8+DAJgLGXK3euBP1tmxrW7V6o6CJMGn6Dfu3JX1C0mlJNyR9Y78NbZ+zvWZ77fbt230+HTA+NltZT+vAoPUV9oi4GRHbEfFbSS9LeuyAbc9HxGJELM7OzvY7JzA25mZKPa0Dg9ZX2G2f2HX3KUlX99sWmDTVSlml6ak9a6XpKVUr5YImwqTpePLU9muSHpd0zPZ7kr4m6XHbpyWFpGuSnhvijMBYaZ8g5aoYFMURcWRPtri4GGtra0f2fACQAtuXI2Kx2+155ykAJIawA0BiCDsAJIawA0BiCDsAJIawA0BiCDsAJIawA0BiCDsAJIawA0BiCDsAJIawA0BiCDsAJIawA0BiCDsAJIawA0BiCDsAJIawA0BiCDsAJIawA0Bi7it6AKBbq+tN1eoNbbYyzc2UVK2UtbQwX/RYwMgh7BgLq+tNLa9sKNvaliQ1W5mWVzYkibgDd+FQDMZCrd64E/W2bGtbtXqjoImA0UXYMRY2W1lP68AkI+wYC3MzpZ7WgUlG2DEWqpWyStNTe9ZK01OqVsoFTQSMLk6eYiy0T5ByVQzQGWHH2FhamCfkQBc4FAMAiSHsAJAYwg4AiSHsAJAYwg4AiSHsAJAYwg4AiSHsAJAYwg4AiSHsAJCYjmG3/YrtW7av7lp7yPZbtt/Nvz843DEBAN3q5hX7q5KeuGvtBUkXI+IRSRfz+wCAEdAx7BFxSdKv71o+I+lCfvuCpKUBzwUA6FO/x9iPR8SN/Pb7ko4PaB4AwCEd+uRpRISk2O9x2+dsr9leu3379mGfDgDQQb9hv2n7hCTl32/tt2FEnI+IxYhYnJ2d7fPpAADd6jfsb0o6m98+K+mNwYwDADisbi53fE3Sv0sq237P9rOSXpL0Z7bflfT5/D4AYAR0/Gi8iHhmn4c+N+BZAAADwDtPASAxfJj1BFtdb6pWb2izlWlupqRqpcyHRQMJIOwTanW9qeWVDWVb25KkZivT8sqGJBF3YMxxKGZC1eqNO1Fvy7a2Vas3CpoIwKAQ9gm12cp6WgcwPgj7hJqbKfW0DmB8EPYJVa2UVZqe2rNWmp5StVIuaCIAg8LJ0wnVPkHKVTFAegj7BFtamCfkQII4FAMAiSHsAJAYwg4AiSHsAJAYwg4AiSHsAJAYwg4AiSHsAJAYwg4AiSHsAJAYwg4AiSHsAJAYwg4AiSHsAJAYwg4AiSHsAJAYwg4AiSHsAJAYwg4AiSHsAJAYwg4Aibmv6AFSs7reVK3e0GYr09xMSdVKWUsL80WPBWCCEPYBWl1vanllQ9nWtiSp2cq0vLIhScQdwJHhUMwA1eqNO1Fvy7a2Vas3CpoIwCQi7AO02cp6WgeAYSDsAzQ3U+ppHQCGgbAPULVSVml6as9aaXpK1Uq5oIkATCJOng5Q+wQpV8UAKBJhH7ClhXlCDqBQhwq77WuSPpS0LemjiFgcxFAAgP4N4hX7ZyPigwH8OQCAAeDkKQAk5rBhD0k/tn3Z9rlBDAQAOJzDHor5TEQ0bX9c0lu2/ysiLu3eIA/+OUk6efLkIZ8OANDJoV6xR0Qz/35L0uuSHrvHNucjYjEiFmdnZw/zdACALvQddtv3236gfVvSFyRdHdRgAID+HOZQzHFJr9tu/znfi4gfDWQqAEDf+g57RPxK0icHOAsAYAC43BEAEkPYASAxhB0AEkPYASAxhB0AEkPYASAxhB0AEkPYASAxhB0AEkPYASAxhB0AEjPyH2a9ut5Urd7QZivT3ExJ1UqZD4sGgAOMdNhX15taXtlQtrUtSWq2Mi2vbEgScQeAfYz0oZhavXEn6m3Z1rZq9UZBEwHA6BvpsG+2sp7WAQAjHva5mVJP6wCAEQ97tVJWaXpqz1ppekrVSrmgiQBg9I30ydP2CVKuigGA7o102KWduBNyAOjeSB+KAQD0jrADQGIIOwAkhrADQGIIOwAkxhFxdE9m35Z0/cie8PCOSfqg6CFGHPvoYOyfzthHBzsm6f6ImO32B4407OPG9lpELBY9xyhjHx2M/dMZ++hg/ewfDsUAQGIIOwAkhrAf7HzRA4wB9tHB2D+dsY8O1vP+4Rg7ACSGV+wAkBjC3oHtF203bV/Jv54seqZRYPsJ2w3bv7T9QtHzjCLb12xv5L83a0XPUzTbr9i+ZfvqrrWHbL9l+938+4NFzli0ffZRzw0i7N35VkSczr9+WPQwRbM9JekfJf25pEclPWP70WKnGlmfzX9vuJxPelXSE3etvSDpYkQ8Iulifn+Svarf3UdSjw0i7OjHY5J+GRG/ioj/k/TPks4UPBNGXERckvTru5bPSLqQ374gaelIhxox++yjnhH27jxv++f5X5Mm+q+KuXlJ/73r/nv5GvYKST+2fdn2uaKHGVHHI+JGfvt9SceLHGaE9dQgwi7J9r/avnqPrzOSvi3pE5JOS7oh6RuFDotx8pmI+JR2Dll9yfafFj3QKIudS/S4TO939dygkf8EpaMQEZ/vZjvbL0v6lyGPMw6akh7edf8P8jXsEhHN/Pst269r5xDWpWKnGjk3bZ+IiBu2T0i6VfRAoyYibrZvd9sgXrF3kP+ytT0l6ep+206Q/5T0iO0/tP37kv5C0psFzzRSbN9v+4H2bUlfEL879/KmpLP57bOS3ihwlpHUT4N4xd7Z39s+rZ2/Il6T9Fyx4xQvIj6y/bykuqQpSa9ExNsFjzVqjkt63ba089/Z9yLiR8WOVCzbr0l6XNIx2+9J+pqklyR93/az2vmXX58ubsLi7bOPHu+1QbzzFAASw6EYAEgMYQeAxBB2AEgMYQeAxBB2AEgMYQeAxBB2AEgMYQeAxPw/YhrWmPXy7VoAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9ONZF8un7_xy"
      },
      "source": [
        "Before we do any modelling, can you calculate the pattern between `X` and `y`?\n",
        "\n",
        "For example, say I asked you, based on this data what the `y` value would be if `X` was 17.0?\n",
        "\n",
        "Or how about if `X` was -10.0?\n",
        "\n",
        "This kind of pattern discovery is the essence of what we'll be building neural networks to do for us."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zNCXxHnF6jjZ"
      },
      "source": [
        "## Regression input shapes and output shapes\n",
        "\n",
        "One of the most important concepts when working with neural networks are the input and output shapes.\n",
        "\n",
        "The **input shape** is the shape of your data that goes into the model.\n",
        "\n",
        "The **output shape** is the shape of your data you want to come out of your model.\n",
        "\n",
        "These will differ depending on the problem you're working on.\n",
        "\n",
        "Neural networks accept numbers and output numbers. These numbers are typically represented as tensors (or arrays).\n",
        "\n",
        "Before, we created data using NumPy arrays, but we could do the same with tensors."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XrXQ3m0prWXa",
        "outputId": "c8fd3447-9379-44cf-b671-ee1c0f82813f"
      },
      "source": [
        "# Example input and output shapes of a regresson model\n",
        "house_info = tf.constant([\"bedroom\", \"bathroom\", \"garage\"])\n",
        "house_price = tf.constant([939700])\n",
        "house_info, house_price"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3,), dtype=string, numpy=array([b'bedroom', b'bathroom', b'garage'], dtype=object)>,\n",
              " <tf.Tensor: shape=(1,), dtype=int32, numpy=array([939700], dtype=int32)>)"
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Yi3VWKH6sRrZ",
        "outputId": "4d32f5e9-7f93-46e9-a1c1-8106d21cca39"
      },
      "source": [
        "house_info.shape"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TensorShape([3])"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kOxyr9sR6m9X",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 269
        },
        "outputId": "02b5e416-6b5b-4429-b73b-0ca19f5c34e7"
      },
      "source": [
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "# Create features (using tensors)\n",
        "X = tf.constant([-7.0, -4.0, -1.0, 2.0, 5.0, 8.0, 11.0, 14.0])\n",
        "\n",
        "# Create labels (using tensors)\n",
        "y = tf.constant([3.0, 6.0, 9.0, 12.0, 15.0, 18.0, 21.0, 24.0])\n",
        "\n",
        "# Visualize it\n",
        "plt.scatter(X, y);"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAOP0lEQVR4nO3df2jc933H8ddrigZHGlCCVWNpMR4lHIRBrU6EQctIadfL8o+Vf8LyR/FYwPmjgY6Vg6j/NDAGYdcf/2wUHBriQZtRqKKEUXrNTJkpjDG5MpXT7EgpNsvJsR26oxl8YYr63h/6npFcS/dDd/refe75AKG7z33le/NFeeb8/X7P54gQACAdv1f0AACAwSLsAJAYwg4AiSHsAJAYwg4AiSHsAJCYjmG3/bDtn9j+he23bX85X3/RdtP2lfzryeGPCwDoxJ2uY7d9QtKJiPiZ7QckXZa0JOlpSf8bEV8f/pgAgG7d12mDiLgh6UZ++0Pb70iaH/ZgAID+dHzFvmdj+5SkS5L+SNLfSPpLSb+RtCbpKxHxPwf9/LFjx+LUqVP9TQoAE+ry5csfRMRst9t3HXbbH5P0b5L+LiJWbB+X9IGkkPS32jlc81f3+Llzks5J0smTJ//4+vXr3c4GAJBk+3JELHa7fVdXxdielvQDSd+NiBVJioibEbEdEb+V9LKkx+71sxFxPiIWI2Jxdrbr/+EAAPrUzVUxlvQdSe9ExDd3rZ/YtdlTkq4OfjwAQK86njyV9GlJX5S0YftKvvZVSc/YPq2dQzHXJD03lAkBAD3p5qqYn0ryPR764eDHAQAcFu88BYDEdHMoBgDQp9X1pmr1hjZbmeZmSqpWylpaGO5bgQg7AAzJ6npTyysbyra2JUnNVqbllQ1JGmrcORQDAENSqzfuRL0t29pWrd4Y6vMSdgAYks1W1tP6oBB2ABiSuZlST+uDQtgBYEiqlbJK01N71krTU6pWykN9Xk6eAsCQtE+QclUMACRkaWF+6CG/G4diACAxhB0AEkPYASAxhB0AEkPYASAxhB0AEkPYASAxhB0AEkPYASAxhB0AEkPYASAxhB0AEkPYASAxhB0AEkPYASAxhB0AEkPYASAxhB0AEkPYASAxhB0AEsOHWQMYK6vrTdXqDW22Ms3NlFStlI/8w6JHHWEHMDZW15taXtlQtrUtSWq2Mi2vbEgScd+FQzEAxkat3rgT9bZsa1u1eqOgiUYTYQcwNjZbWU/rk4qwAxgbczOlntYnFWEHMDaqlbJK01N71krTU6pWygVNNJo4eQpgbLRPkHJVzMEIO4CxsrQwT8g74FAMACSmY9htP2z7J7Z/Yftt21/O1x+y/Zbtd/PvDw5/XABAJ928Yv9I0lci4lFJfyLpS7YflfSCpIsR8Yiki/l9AEDBOoY9Im5ExM/y2x9KekfSvKQzki7km12QtDSsIQEA3evpGLvtU5IWJP2HpOMRcSN/6H1Jxwc6GQCgL12H3fbHJP1A0l9HxG92PxYRISn2+blzttdsr92+fftQwwIAOusq7LantRP170bESr580/aJ/PETkm7d62cj4nxELEbE4uzs7CBmBgAcoJurYizpO5LeiYhv7nroTUln89tnJb0x+PEAAL3q5g1Kn5b0RUkbtq/ka1+V9JKk79t+VtJ1SU8PZ0QAQC86hj0ifirJ+zz8ucGOAwA4LN55CgCJIewAkBjCDgCJIewAkBjCDgCJIewAkBjCDgCJIewAkBjCDgCJIewAkBjCDgCJ6eYfAQOQuNX1pmr1hjZbmeZmSqpWylpamC96LPSJsAMTbnW9qeWVDWVb25KkZivT8sqGJBH3McWhGGDC1eqNO1Fvy7a2Vas3CpoIh0XYgQm32cp6WsfoI+zAhJubKfW0jtFH2IEJV62UVZqe2rNWmp5StVIuaCIcFidPgQnXPkHKVTHpIOwAtLQwT8gTwqEYAEgMYQeAxBB2AEgMYQeAxBB2AEgMYQeAxBB2AEgMYQeAxBB2AEgMYQeAxBB2AEgMYQeAxBB2AEgMYQeAxBB2AEgMYQeAxBB2AEgMYQeAxBB2AEhMx7DbfsX2LdtXd629aLtp+0r+9eRwxwQAdKubD7N+VdI/SPqnu9a/FRFfH/hEQAJW15uq1RvabGWamympWinzYdE4Mh3DHhGXbJ8a/ihAGlbXm1pe2VC2tS1JarYyLa9sSBJxx5E4zDH2523/PD9U8+DAJgLGXK3euBP1tmxrW7V6o6CJMGn6Dfu3JX1C0mlJNyR9Y78NbZ+zvWZ77fbt230+HTA+NltZT+vAoPUV9oi4GRHbEfFbSS9LeuyAbc9HxGJELM7OzvY7JzA25mZKPa0Dg9ZX2G2f2HX3KUlX99sWmDTVSlml6ak9a6XpKVUr5YImwqTpePLU9muSHpd0zPZ7kr4m6XHbpyWFpGuSnhvijMBYaZ8g5aoYFMURcWRPtri4GGtra0f2fACQAtuXI2Kx2+155ykAJIawA0BiCDsAJIawA0BiCDsAJIawA0BiCDsAJIawA0BiCDsAJIawA0BiCDsAJIawA0BiCDsAJIawA0BiCDsAJIawA0BiCDsAJIawA0BiCDsAJIawA0Bi7it6AKBbq+tN1eoNbbYyzc2UVK2UtbQwX/RYwMgh7BgLq+tNLa9sKNvaliQ1W5mWVzYkibgDd+FQDMZCrd64E/W2bGtbtXqjoImA0UXYMRY2W1lP68AkI+wYC3MzpZ7WgUlG2DEWqpWyStNTe9ZK01OqVsoFTQSMLk6eYiy0T5ByVQzQGWHH2FhamCfkQBc4FAMAiSHsAJAYwg4AiSHsAJAYwg4AiSHsAJAYwg4AiSHsAJAYwg4AiSHsAJCYjmG3/YrtW7av7lp7yPZbtt/Nvz843DEBAN3q5hX7q5KeuGvtBUkXI+IRSRfz+wCAEdAx7BFxSdKv71o+I+lCfvuCpKUBzwUA6FO/x9iPR8SN/Pb7ko4PaB4AwCEd+uRpRISk2O9x2+dsr9leu3379mGfDgDQQb9hv2n7hCTl32/tt2FEnI+IxYhYnJ2d7fPpAADd6jfsb0o6m98+K+mNwYwDADisbi53fE3Sv0sq237P9rOSXpL0Z7bflfT5/D4AYAR0/Gi8iHhmn4c+N+BZAAADwDtPASAxfJj1BFtdb6pWb2izlWlupqRqpcyHRQMJIOwTanW9qeWVDWVb25KkZivT8sqGJBF3YMxxKGZC1eqNO1Fvy7a2Vas3CpoIwKAQ9gm12cp6WgcwPgj7hJqbKfW0DmB8EPYJVa2UVZqe2rNWmp5StVIuaCIAg8LJ0wnVPkHKVTFAegj7BFtamCfkQII4FAMAiSHsAJAYwg4AiSHsAJAYwg4AiSHsAJAYwg4AiSHsAJAYwg4AiSHsAJAYwg4AiSHsAJAYwg4AiSHsAJAYwg4AiSHsAJAYwg4AiSHsAJAYwg4AiSHsAJAYwg4Aibmv6AFSs7reVK3e0GYr09xMSdVKWUsL80WPBWCCEPYBWl1vanllQ9nWtiSp2cq0vLIhScQdwJHhUMwA1eqNO1Fvy7a2Vas3CpoIwCQi7AO02cp6WgeAYSDsAzQ3U+ppHQCGgbAPULVSVml6as9aaXpK1Uq5oIkATCJOng5Q+wQpV8UAKBJhH7ClhXlCDqBQhwq77WuSPpS0LemjiFgcxFAAgP4N4hX7ZyPigwH8OQCAAeDkKQAk5rBhD0k/tn3Z9rlBDAQAOJzDHor5TEQ0bX9c0lu2/ysiLu3eIA/+OUk6efLkIZ8OANDJoV6xR0Qz/35L0uuSHrvHNucjYjEiFmdnZw/zdACALvQddtv3236gfVvSFyRdHdRgAID+HOZQzHFJr9tu/znfi4gfDWQqAEDf+g57RPxK0icHOAsAYAC43BEAEkPYASAxhB0AEkPYASAxhB0AEkPYASAxhB0AEkPYASAxhB0AEkPYASAxhB0AEjPyH2a9ut5Urd7QZivT3ExJ1UqZD4sGgAOMdNhX15taXtlQtrUtSWq2Mi2vbEgScQeAfYz0oZhavXEn6m3Z1rZq9UZBEwHA6BvpsG+2sp7WAQAjHva5mVJP6wCAEQ97tVJWaXpqz1ppekrVSrmgiQBg9I30ydP2CVKuigGA7o102KWduBNyAOjeSB+KAQD0jrADQGIIOwAkhrADQGIIOwAkxhFxdE9m35Z0/cie8PCOSfqg6CFGHPvoYOyfzthHBzsm6f6ImO32B4407OPG9lpELBY9xyhjHx2M/dMZ++hg/ewfDsUAQGIIOwAkhrAf7HzRA4wB9tHB2D+dsY8O1vP+4Rg7ACSGV+wAkBjC3oHtF203bV/Jv54seqZRYPsJ2w3bv7T9QtHzjCLb12xv5L83a0XPUzTbr9i+ZfvqrrWHbL9l+938+4NFzli0ffZRzw0i7N35VkSczr9+WPQwRbM9JekfJf25pEclPWP70WKnGlmfzX9vuJxPelXSE3etvSDpYkQ8Iulifn+Svarf3UdSjw0i7OjHY5J+GRG/ioj/k/TPks4UPBNGXERckvTru5bPSLqQ374gaelIhxox++yjnhH27jxv++f5X5Mm+q+KuXlJ/73r/nv5GvYKST+2fdn2uaKHGVHHI+JGfvt9SceLHGaE9dQgwi7J9r/avnqPrzOSvi3pE5JOS7oh6RuFDotx8pmI+JR2Dll9yfafFj3QKIudS/S4TO939dygkf8EpaMQEZ/vZjvbL0v6lyGPMw6akh7edf8P8jXsEhHN/Pst269r5xDWpWKnGjk3bZ+IiBu2T0i6VfRAoyYibrZvd9sgXrF3kP+ytT0l6ep+206Q/5T0iO0/tP37kv5C0psFzzRSbN9v+4H2bUlfEL879/KmpLP57bOS3ihwlpHUT4N4xd7Z39s+rZ2/Il6T9Fyx4xQvIj6y/bykuqQpSa9ExNsFjzVqjkt63ba089/Z9yLiR8WOVCzbr0l6XNIx2+9J+pqklyR93/az2vmXX58ubsLi7bOPHu+1QbzzFAASw6EYAEgMYQeAxBB2AEgMYQeAxBB2AEgMYQeAxBB2AEgMYQeAxPw/YhrWmPXy7VoAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zaPxco6E9i1_"
      },
      "source": [
        "Our goal here will be to use `X` to predict `y`.\n",
        "\n",
        "So our **input** will be `X` and our **output** will be `y`.\n",
        "\n",
        "Knowing this, what do you think our input and output shapes will be?\n",
        "\n",
        "Let's take a look."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "j1oT1gmB9iX-",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c4aa34e4-c04c-4032-d55b-91676f199387"
      },
      "source": [
        "# Take a single example of X\n",
        "input_shape = X[0].shape \n",
        "\n",
        "# Take a single example of y\n",
        "output_shape = y[0].shape\n",
        "\n",
        "input_shape, output_shape # these are both scalars (no shape)"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(TensorShape([]), TensorShape([]))"
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m4txxFTL_EEX"
      },
      "source": [
        "Huh?\n",
        "\n",
        "From this it seems our inputs and outputs have no shape?\n",
        "\n",
        "How could that be?\n",
        "\n",
        "It's because no matter what kind of data we pass to our model, it's always going to take as input and return as ouput some kind of tensor.\n",
        "\n",
        "But in our case because of our dataset (only 2 small lists of numbers), we're looking at a special kind of tensor, more specifically a rank 0 tensor or a scalar."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Dag5y4MPaTmc",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4e041946-1095-479a-8db1-b19a390cab3f"
      },
      "source": [
        "# Let's take a look at the single examples invidually\n",
        "X[0], y[0]"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(), dtype=float32, numpy=-7.0>,\n",
              " <tf.Tensor: shape=(), dtype=float32, numpy=3.0>)"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fKtihU57_cOY"
      },
      "source": [
        "In our case, we're trying to build a model to predict the pattern between `X[0]` equalling `-7.0` and `y[0]` equalling `3.0`.\n",
        "\n",
        "So now we get our answer, we're trying to use 1 `X` value to predict 1 `y` value.\n",
        "\n",
        "You might be thinking, \"this seems pretty complicated for just predicting a straight line...\".\n",
        "\n",
        "And you'd be right.\n",
        "\n",
        "But the concepts we're covering here, the concepts of input and output shapes to a model are fundamental. \n",
        "\n",
        "In fact, they're probably two of the things you'll spend the most time on when you work with neural networks: **making sure your input and outputs are in the correct shape**.\n",
        "\n",
        "If it doesn't make sense now, we'll see plenty more examples later on (soon you'll notice the input and output shapes can be almost anything you can imagine).\n",
        "\n",
        "![example of input and output shapes for a housing price prediction problem](https://raw.githubusercontent.com/mrdbourke/tensorflow-deep-learning/main/images/01-input-and-output-shapes-housing-prices.png)\n",
        "*If you were working on building a machine learning algorithm for predicting housing prices, your inputs may be number of bedrooms, number of bathrooms and number of garages, giving you an input shape of 3 (3 different features). And since you're trying to predict the price of the house, your output shape would be 1.*"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PhAIqjrn6olF"
      },
      "source": [
        "## Steps in modelling with TensorFlow\n",
        "\n",
        "Now we know what data we have as well as the input and output shapes, let's see how we'd build a neural network to model it.\n",
        "\n",
        "In TensorFlow, there are typically 3 fundamental steps to creating and training a model.\n",
        "\n",
        "1. **Creating a model** - piece together the layers of a neural network yourself (using the [Functional](https://www.tensorflow.org/guide/keras/functional) or [Sequential API](https://www.tensorflow.org/api_docs/python/tf/keras/Sequential)) or import a previously built model (known as transfer learning).\n",
        "2. **Compiling a model** - defining how a models performance should be measured (loss/metrics) as well as defining how it should improve (optimizer). \n",
        "3. **Fitting a model** - letting the model try to find patterns in the data (how does `X` get to `y`). \n",
        "\n",
        "Let's see these in action using the [Keras Sequential API](https://www.tensorflow.org/api_docs/python/tf/keras/Sequential) to build a model for our regression data. And then we'll step through each.\n",
        "\n",
        "> **Note:** If you're using [TensorFlow 2.7.0](https://github.com/tensorflow/tensorflow/releases/tag/v2.7.0)+, the `fit()` function no longer upscales input data to go from `(batch_size, )` to `(batch_size, 1)`. To fix this, you'll need to expand the dimension of input data using `tf.expand_dims(input_data, axis=-1)`.\n",
        ">\n",
        "> In our case, this means instead of using `model.fit(X, y, epochs=5)`, use `model.fit(tf.expand_dims(X, axis=-1), y, epochs=5)`. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "P9jj-OE16yCn",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "63cf816f-16af-4867-f71a-242682033473"
      },
      "source": [
        "# Set random seed\n",
        "tf.random.set_seed(42)\n",
        "\n",
        "# Create a model using the Sequential API\n",
        "model = tf.keras.Sequential([\n",
        "  tf.keras.layers.Dense(1)\n",
        "])\n",
        "\n",
        "# Compile the model\n",
        "model.compile(loss=tf.keras.losses.mae, # mae is short for mean absolute error\n",
        "              optimizer=tf.keras.optimizers.SGD(), # SGD is short for stochastic gradient descent\n",
        "              metrics=[\"mae\"])\n",
        "\n",
        "# Fit the model\n",
        "# model.fit(X, y, epochs=5) # this will break with TensorFlow 2.7.0+\n",
        "model.fit(tf.expand_dims(X, axis=-1), y, epochs=5)"
      ],
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/5\n",
            "1/1 [==============================] - 0s 313ms/step - loss: 11.5048 - mae: 11.5048\n",
            "Epoch 2/5\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 11.3723 - mae: 11.3723\n",
            "Epoch 3/5\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 11.2398 - mae: 11.2398\n",
            "Epoch 4/5\n",
            "1/1 [==============================] - 0s 8ms/step - loss: 11.1073 - mae: 11.1073\n",
            "Epoch 5/5\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 10.9748 - mae: 10.9748\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f8df6701950>"
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bbjCwkEtetB9"
      },
      "source": [
        "Boom!\n",
        "\n",
        "We've just trained a model to figure out the patterns between `X` and `y`.\n",
        "\n",
        "How do you think it went?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VWLpG2U3erWo",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "16076fb8-e754-4edb-b18b-3e5c30d6e2be"
      },
      "source": [
        "# Check out X and y\n",
        "X, y"
      ],
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(8,), dtype=float32, numpy=array([-7., -4., -1.,  2.,  5.,  8., 11., 14.], dtype=float32)>,\n",
              " <tf.Tensor: shape=(8,), dtype=float32, numpy=array([ 3.,  6.,  9., 12., 15., 18., 21., 24.], dtype=float32)>)"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0ZpkaI_Oe6no"
      },
      "source": [
        "What do you think the outcome should be if we passed our model an `X` value of 17.0?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "X86cD66Qeo-8",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "29b569e6-90ce-46c7-8842-4ca69c5144a3"
      },
      "source": [
        "# Make a prediction with the model\n",
        "model.predict([17.0])"
      ],
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[12.716021]], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2YoKIqhffq33"
      },
      "source": [
        "It doesn't go very well... it should've output something close to 27.0.\n",
        "\n",
        "> 🤔 **Question:** What's Keras? I thought we were working with TensorFlow but every time we write TensorFlow code, `keras` comes after `tf` (e.g. `tf.keras.layers.Dense()`)?\n",
        "\n",
        "Before TensorFlow 2.0+, [Keras](https://keras.io/) was an API designed to be able to build deep learning models with ease. Since TensorFlow 2.0+, its functionality has been tightly integrated within the TensorFlow library."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qAPk1T3xgOm4"
      },
      "source": [
        "## Improving a model\n",
        "\n",
        "How do you think you'd improve upon our current model?\n",
        "\n",
        "If you guessed by tweaking some of the things we did above, you'd be correct.\n",
        "\n",
        "To improve our model, we alter almost every part of the 3 steps we went through before.\n",
        "\n",
        "1. **Creating a model** - here you might want to add more layers, increase the number of hidden units (also called neurons) within each layer, change the activation functions of each layer.\n",
        "2. **Compiling a model** - you might want to choose optimization function or perhaps change the **learning rate** of the optimization function.\n",
        "3. **Fitting a model** - perhaps you could fit a model for more **epochs** (leave it training for longer) or on more data (give the model more examples to learn from).\n",
        "\n",
        "![various options you can use to improve a neural network model](https://raw.githubusercontent.com/mrdbourke/tensorflow-deep-learning/main/images/02-improving-a-model-from-model-perspective.png)\n",
        "*There are many different ways to potentially improve a neural network. Some of the most common include: increasing the number of layers (making the network deeper), increasing the number of hidden units (making the network wider) and changing the learning rate. Because these values are all human-changeable, they're referred to as [hyperparameters](https://en.wikipedia.org/wiki/Hyperparameter_(machine_learning)) and the practice of trying to find the best hyperparameters is referred to as [hyperparameter tuning](https://en.wikipedia.org/wiki/Hyperparameter_optimization).*\n",
        "\n",
        "Woah. We just introduced a bunch of possible steps. The important thing to remember is how you alter each of these will depend on the problem you're working on.\n",
        "\n",
        "And the good thing is, over the next few problems, we'll get hands-on with all of them.\n",
        "\n",
        "For now, let's keep it simple, all we'll do is train our model for longer (everything else will stay the same)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MI0LammMgWcN",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0c01661d-6495-476a-b71e-bc63aa7e9ffe"
      },
      "source": [
        "# Set random seed\n",
        "tf.random.set_seed(42)\n",
        "\n",
        "# Create a model (same as above)\n",
        "model = tf.keras.Sequential([\n",
        "  tf.keras.layers.Dense(1)\n",
        "])\n",
        "\n",
        "# Compile model (same as above)\n",
        "model.compile(loss=tf.keras.losses.mae,\n",
        "              optimizer=tf.keras.optimizers.SGD(),\n",
        "              metrics=[\"mae\"])\n",
        "\n",
        "# Fit model (this time we'll train for longer)\n",
        "model.fit(tf.expand_dims(X, axis=-1), y, epochs=100) # train for 100 epochs not 10"
      ],
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/100\n",
            "1/1 [==============================] - 0s 319ms/step - loss: 11.5048 - mae: 11.5048\n",
            "Epoch 2/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 11.3723 - mae: 11.3723\n",
            "Epoch 3/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 11.2398 - mae: 11.2398\n",
            "Epoch 4/100\n",
            "1/1 [==============================] - 0s 4ms/step - loss: 11.1073 - mae: 11.1073\n",
            "Epoch 5/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 10.9748 - mae: 10.9748\n",
            "Epoch 6/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 10.8423 - mae: 10.8423\n",
            "Epoch 7/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 10.7098 - mae: 10.7098\n",
            "Epoch 8/100\n",
            "1/1 [==============================] - 0s 9ms/step - loss: 10.5773 - mae: 10.5773\n",
            "Epoch 9/100\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 10.4448 - mae: 10.4448\n",
            "Epoch 10/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 10.3123 - mae: 10.3123\n",
            "Epoch 11/100\n",
            "1/1 [==============================] - 0s 14ms/step - loss: 10.1798 - mae: 10.1798\n",
            "Epoch 12/100\n",
            "1/1 [==============================] - 0s 9ms/step - loss: 10.0473 - mae: 10.0473\n",
            "Epoch 13/100\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 9.9148 - mae: 9.9148\n",
            "Epoch 14/100\n",
            "1/1 [==============================] - 0s 8ms/step - loss: 9.7823 - mae: 9.7823\n",
            "Epoch 15/100\n",
            "1/1 [==============================] - 0s 9ms/step - loss: 9.6498 - mae: 9.6498\n",
            "Epoch 16/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 9.5173 - mae: 9.5173\n",
            "Epoch 17/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 9.3848 - mae: 9.3848\n",
            "Epoch 18/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 9.2523 - mae: 9.2523\n",
            "Epoch 19/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 9.1198 - mae: 9.1198\n",
            "Epoch 20/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 8.9873 - mae: 8.9873\n",
            "Epoch 21/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 8.8548 - mae: 8.8548\n",
            "Epoch 22/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 8.7223 - mae: 8.7223\n",
            "Epoch 23/100\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 8.5898 - mae: 8.5898\n",
            "Epoch 24/100\n",
            "1/1 [==============================] - 0s 4ms/step - loss: 8.4573 - mae: 8.4573\n",
            "Epoch 25/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 8.3248 - mae: 8.3248\n",
            "Epoch 26/100\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 8.1923 - mae: 8.1923\n",
            "Epoch 27/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 8.0598 - mae: 8.0598\n",
            "Epoch 28/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 7.9273 - mae: 7.9273\n",
            "Epoch 29/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 7.7948 - mae: 7.7948\n",
            "Epoch 30/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 7.6623 - mae: 7.6623\n",
            "Epoch 31/100\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 7.5298 - mae: 7.5298\n",
            "Epoch 32/100\n",
            "1/1 [==============================] - 0s 4ms/step - loss: 7.3973 - mae: 7.3973\n",
            "Epoch 33/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 7.2648 - mae: 7.2648\n",
            "Epoch 34/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 7.2525 - mae: 7.2525\n",
            "Epoch 35/100\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 7.2469 - mae: 7.2469\n",
            "Epoch 36/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 7.2413 - mae: 7.2413\n",
            "Epoch 37/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 7.2356 - mae: 7.2356\n",
            "Epoch 38/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 7.2300 - mae: 7.2300\n",
            "Epoch 39/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 7.2244 - mae: 7.2244\n",
            "Epoch 40/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 7.2188 - mae: 7.2188\n",
            "Epoch 41/100\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 7.2131 - mae: 7.2131\n",
            "Epoch 42/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 7.2075 - mae: 7.2075\n",
            "Epoch 43/100\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 7.2019 - mae: 7.2019\n",
            "Epoch 44/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 7.1963 - mae: 7.1963\n",
            "Epoch 45/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 7.1906 - mae: 7.1906\n",
            "Epoch 46/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 7.1850 - mae: 7.1850\n",
            "Epoch 47/100\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 7.1794 - mae: 7.1794\n",
            "Epoch 48/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 7.1738 - mae: 7.1738\n",
            "Epoch 49/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 7.1681 - mae: 7.1681\n",
            "Epoch 50/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 7.1625 - mae: 7.1625\n",
            "Epoch 51/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 7.1569 - mae: 7.1569\n",
            "Epoch 52/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 7.1512 - mae: 7.1512\n",
            "Epoch 53/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 7.1456 - mae: 7.1456\n",
            "Epoch 54/100\n",
            "1/1 [==============================] - 0s 31ms/step - loss: 7.1400 - mae: 7.1400\n",
            "Epoch 55/100\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 7.1344 - mae: 7.1344\n",
            "Epoch 56/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 7.1287 - mae: 7.1287\n",
            "Epoch 57/100\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 7.1231 - mae: 7.1231\n",
            "Epoch 58/100\n",
            "1/1 [==============================] - 0s 8ms/step - loss: 7.1175 - mae: 7.1175\n",
            "Epoch 59/100\n",
            "1/1 [==============================] - 0s 11ms/step - loss: 7.1119 - mae: 7.1119\n",
            "Epoch 60/100\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 7.1063 - mae: 7.1063\n",
            "Epoch 61/100\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 7.1006 - mae: 7.1006\n",
            "Epoch 62/100\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 7.0950 - mae: 7.0950\n",
            "Epoch 63/100\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 7.0894 - mae: 7.0894\n",
            "Epoch 64/100\n",
            "1/1 [==============================] - 0s 8ms/step - loss: 7.0838 - mae: 7.0838\n",
            "Epoch 65/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 7.0781 - mae: 7.0781\n",
            "Epoch 66/100\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 7.0725 - mae: 7.0725\n",
            "Epoch 67/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 7.0669 - mae: 7.0669\n",
            "Epoch 68/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 7.0613 - mae: 7.0613\n",
            "Epoch 69/100\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 7.0556 - mae: 7.0556\n",
            "Epoch 70/100\n",
            "1/1 [==============================] - 0s 26ms/step - loss: 7.0500 - mae: 7.0500\n",
            "Epoch 71/100\n",
            "1/1 [==============================] - 0s 8ms/step - loss: 7.0444 - mae: 7.0444\n",
            "Epoch 72/100\n",
            "1/1 [==============================] - 0s 8ms/step - loss: 7.0388 - mae: 7.0388\n",
            "Epoch 73/100\n",
            "1/1 [==============================] - 0s 8ms/step - loss: 7.0331 - mae: 7.0331\n",
            "Epoch 74/100\n",
            "1/1 [==============================] - 0s 8ms/step - loss: 7.0275 - mae: 7.0275\n",
            "Epoch 75/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 7.0219 - mae: 7.0219\n",
            "Epoch 76/100\n",
            "1/1 [==============================] - 0s 8ms/step - loss: 7.0163 - mae: 7.0163\n",
            "Epoch 77/100\n",
            "1/1 [==============================] - 0s 8ms/step - loss: 7.0106 - mae: 7.0106\n",
            "Epoch 78/100\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 7.0050 - mae: 7.0050\n",
            "Epoch 79/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 6.9994 - mae: 6.9994\n",
            "Epoch 80/100\n",
            "1/1 [==============================] - 0s 10ms/step - loss: 6.9938 - mae: 6.9938\n",
            "Epoch 81/100\n",
            "1/1 [==============================] - 0s 10ms/step - loss: 6.9881 - mae: 6.9881\n",
            "Epoch 82/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 6.9825 - mae: 6.9825\n",
            "Epoch 83/100\n",
            "1/1 [==============================] - 0s 8ms/step - loss: 6.9769 - mae: 6.9769\n",
            "Epoch 84/100\n",
            "1/1 [==============================] - 0s 8ms/step - loss: 6.9713 - mae: 6.9713\n",
            "Epoch 85/100\n",
            "1/1 [==============================] - 0s 12ms/step - loss: 6.9656 - mae: 6.9656\n",
            "Epoch 86/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 6.9600 - mae: 6.9600\n",
            "Epoch 87/100\n",
            "1/1 [==============================] - 0s 16ms/step - loss: 6.9544 - mae: 6.9544\n",
            "Epoch 88/100\n",
            "1/1 [==============================] - 0s 9ms/step - loss: 6.9488 - mae: 6.9488\n",
            "Epoch 89/100\n",
            "1/1 [==============================] - 0s 11ms/step - loss: 6.9431 - mae: 6.9431\n",
            "Epoch 90/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 6.9375 - mae: 6.9375\n",
            "Epoch 91/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 6.9319 - mae: 6.9319\n",
            "Epoch 92/100\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 6.9263 - mae: 6.9263\n",
            "Epoch 93/100\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 6.9206 - mae: 6.9206\n",
            "Epoch 94/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 6.9150 - mae: 6.9150\n",
            "Epoch 95/100\n",
            "1/1 [==============================] - 0s 8ms/step - loss: 6.9094 - mae: 6.9094\n",
            "Epoch 96/100\n",
            "1/1 [==============================] - 0s 6ms/step - loss: 6.9038 - mae: 6.9038\n",
            "Epoch 97/100\n",
            "1/1 [==============================] - 0s 7ms/step - loss: 6.8981 - mae: 6.8981\n",
            "Epoch 98/100\n",
            "1/1 [==============================] - 0s 10ms/step - loss: 6.8925 - mae: 6.8925\n",
            "Epoch 99/100\n",
            "1/1 [==============================] - 0s 5ms/step - loss: 6.8869 - mae: 6.8869\n",
            "Epoch 100/100\n",
            "1/1 [==============================] - 0s 13ms/step - loss: 6.8813 - mae: 6.8813\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f8df664db90>"
            ]
          },
          "metadata": {},
          "execution_count": 28
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1CIKSm7filgj"
      },
      "source": [
        "You might've noticed the loss value decrease from before (and keep decreasing as the number of epochs gets higher). \n",
        "\n",
        "What do you think this means for when we make a prediction with our model?\n",
        "\n",
        "How about we try predict on 17.0 again?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_YcacZsfi4zZ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "011ee264-7e6e-47f6-cd8e-26627befb2b5"
      },
      "source": [
        "# Remind ourselves of what X and y are\n",
        "X, y"
      ],
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(8,), dtype=float32, numpy=array([-7., -4., -1.,  2.,  5.,  8., 11., 14.], dtype=float32)>,\n",
              " <tf.Tensor: shape=(8,), dtype=float32, numpy=array([ 3.,  6.,  9., 12., 15., 18., 21., 24.], dtype=float32)>)"
            ]
          },
          "metadata": {},
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "e6k5V08ZivNO",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "9f703f97-1b6d-41af-b408-18a99aa6c8dc"
      },
      "source": [
        "# Try and predict what y would be if X was 17.0\n",
        "model.predict([17.0]) # the right answer is 27.0 (y = X + 10)"
      ],
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[30.158512]], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 30
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sYi5V8ZYi_dy"
      },
      "source": [
        "Much better! \n",
        "\n",
        "We got closer this time. But we could still be better.\n",
        "\n",
        "Now we've trained a model, how could we evaluate it?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HvC98q_h6zvG"
      },
      "source": [
        "## Evaluating a model \n",
        "\n",
        "A typical workflow you'll go through when building neural networks is:\n",
        "\n",
        "```\n",
        "Build a model -> evaluate it -> build (tweak) a model -> evaulate it -> build (tweak) a model -> evaluate it...\n",
        "```\n",
        "\n",
        "The tweaking comes from maybe not building a model from scratch but adjusting an existing one.\n",
        "\n",
        "### Visualize, visualize, visualize\n",
        "\n",
        "When it comes to evaluation, you'll want to remember the words: \"visualize, visualize, visualize.\" \n",
        "\n",
        "This is because you're probably better looking at something (doing) than you are thinking about something.\n",
        "\n",
        "It's a good idea to visualize:\n",
        "* **The data** - what data are you working with? What does it look like?\n",
        "* **The model itself** - what does the architecture look like? What are the different shapes?\n",
        "* **The training of a model** - how does a model perform while it learns?\n",
        "* **The predictions of a model** - how do the predictions of a model line up against the ground truth (the original labels)?\n",
        "\n",
        "Let's start by visualizing the model.\n",
        "\n",
        "But first, we'll create a little bit of a bigger dataset and a new model we can use (it'll be the same as before, but the more practice the better).\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "srxuqbeYopns",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "85d3bffc-9c4c-40e7-8223-25ff0209e395"
      },
      "source": [
        "# Make a bigger dataset\n",
        "X = np.arange(-100, 100, 4)\n",
        "X"
      ],
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([-100,  -96,  -92,  -88,  -84,  -80,  -76,  -72,  -68,  -64,  -60,\n",
              "        -56,  -52,  -48,  -44,  -40,  -36,  -32,  -28,  -24,  -20,  -16,\n",
              "        -12,   -8,   -4,    0,    4,    8,   12,   16,   20,   24,   28,\n",
              "         32,   36,   40,   44,   48,   52,   56,   60,   64,   68,   72,\n",
              "         76,   80,   84,   88,   92,   96])"
            ]
          },
          "metadata": {},
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pQcC0nSko3kJ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4fc242dc-0da9-499f-8f42-de2a0c837202"
      },
      "source": [
        "# Make labels for the dataset (adhering to the same pattern as before)\n",
        "y = np.arange(-90, 110, 4)\n",
        "y"
      ],
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([-90, -86, -82, -78, -74, -70, -66, -62, -58, -54, -50, -46, -42,\n",
              "       -38, -34, -30, -26, -22, -18, -14, -10,  -6,  -2,   2,   6,  10,\n",
              "        14,  18,  22,  26,  30,  34,  38,  42,  46,  50,  54,  58,  62,\n",
              "        66,  70,  74,  78,  82,  86,  90,  94,  98, 102, 106])"
            ]
          },
          "metadata": {},
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KNIw5tgGpKjb"
      },
      "source": [
        "Since $y=X+10$, we could make the labels like so:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ACgbmrAOpJwW",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "de9805da-e70b-4dba-ace3-b18ea0e277db"
      },
      "source": [
        "# Same result as above\n",
        "y = X + 10\n",
        "y"
      ],
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([-90, -86, -82, -78, -74, -70, -66, -62, -58, -54, -50, -46, -42,\n",
              "       -38, -34, -30, -26, -22, -18, -14, -10,  -6,  -2,   2,   6,  10,\n",
              "        14,  18,  22,  26,  30,  34,  38,  42,  46,  50,  54,  58,  62,\n",
              "        66,  70,  74,  78,  82,  86,  90,  94,  98, 102, 106])"
            ]
          },
          "metadata": {},
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ax3MnQDupeBp"
      },
      "source": [
        "## Split data into training/test set\n",
        "\n",
        "One of the other most common and important steps in a machine learning project is creating a training and test set (and when required, a validation set).\n",
        "\n",
        "Each set serves a specific purpose:\n",
        "* **Training set** - the model learns from this data, which is typically 70-80% of the total data available (like the course materials you study during the semester).\n",
        "* **Validation set** - the model gets tuned on this data, which is typically 10-15% of the total data available (like the practice exam you take before the final exam).\n",
        "* **Test set** - the model gets evaluated on this data to test what it has learned, it's typically 10-15% of the total data available (like the final exam you take at the end of the semester).\n",
        "\n",
        "For now, we'll just use a training and test set, this means we'll have a dataset for our model to learn on as well as be evaluated on.\n",
        "\n",
        "We can create them by splitting our `X` and `y` arrays.\n",
        "\n",
        "> 🔑 **Note:** When dealing with real-world data, this step is typically done right at the start of a project (the test set should always be kept separate from all other data). We want our model to learn on training data and then evaluate it on test data to get an indication of how well it **generalizes** to unseen examples."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5G0RDMnZrgvK",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d7bffb88-c10b-47fd-cd68-b1d678f8647f"
      },
      "source": [
        "# Check how many samples we have\n",
        "len(X)"
      ],
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "50"
            ]
          },
          "metadata": {},
          "execution_count": 34
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4Q9ptcQkrGfO",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "402152b7-1abe-4eae-fc85-b7aa5224e966"
      },
      "source": [
        "# Split data into train and test sets\n",
        "X_train = X[:40] # first 40 examples (80% of data)\n",
        "y_train = y[:40]\n",
        "\n",
        "X_test = X[40:] # last 10 examples (20% of data)\n",
        "y_test = y[40:]\n",
        "\n",
        "len(X_train), len(X_test)"
      ],
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(40, 10)"
            ]
          },
          "metadata": {},
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Rz2cIdECsLH5"
      },
      "source": [
        "## Visualizing the data\n",
        "\n",
        "Now we've got our training and test data, it's a good idea to visualize it.\n",
        "\n",
        "Let's plot it with some nice colours to differentiate what's what."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "os30CXBHsOAH",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 428
        },
        "outputId": "44a785b8-7535-4d44-8075-2e507e26e11a"
      },
      "source": [
        "plt.figure(figsize=(10, 7))\n",
        "# Plot training data in blue\n",
        "plt.scatter(X_train, y_train, c='b', label='Training data')\n",
        "# Plot test data in green\n",
        "plt.scatter(X_test, y_test, c='g', label='Testing data')\n",
        "# Show the legend\n",
        "plt.legend();"
      ],
      "execution_count": 36,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAGbCAYAAAAY8u5bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3CV9b3v8c+Xi1CEjYpRKQjBFuWimECKW921ZNCqtdbLVIsNrR73FLFaqnscrWZrbc8wY7tt6/H0qCfOdrQz0eIpetSWui1UKy3tpkFzINyOoonGUkxxGuVElMv3/LGeFRZhJVmL9azL8zzv10wma/3W5fmtW/jwXD7L3F0AAAAIz5ByTwAAACBuCFgAAAAhI2ABAACEjIAFAAAQMgIWAABAyIaVewKZjj32WK+uri73NAAAAAa1bt26v7l7VbbLKipgVVdXq6WlpdzTAAAAGJSZdfR3GZsIAQAAQkbAAgAACBkBCwAAIGQVtQ9WNnv27FFnZ6d2795d7qkgMHLkSE2cOFHDhw8v91QAAKhIFR+wOjs7NWbMGFVXV8vMyj2dxHN37dy5U52dnZoyZUq5pwMAQEWq+E2Eu3fv1rhx4whXFcLMNG7cONYoAgAwgIoPWJIIVxWG1wMAgIFFImABAABECQFrEDt37lRNTY1qamp0wgknaMKECb3nP/744wFv29LSoiVLlgy6jLPOOius6R5k3rx5gxa33nffferp6SnK8gEASKqK38m93MaNG6fW1lZJ0t13363Ro0frlltu6b187969GjYs+9NYV1enurq6QZexZs2acCZ7GO677z4tXLhQo0aNKtscAACIm9itwWpulqqrpSFDUr+bm8NfxjXXXKPFixfrjDPO0K233qq1a9fqzDPPVG1trc466yxt3bpVkvTSSy/pi1/8oqRUOLv22ms1b948nXTSSbr//vt772/06NG91583b56+/OUva9q0aWpoaJC7S5JWrFihadOmac6cOVqyZEnv/Wb68MMPtWDBAk2fPl2XXXaZPvzww97Lrr/+etXV1WnmzJn67ne/K0m6//779Ze//EX19fWqr6/v93oAACA/sVqD1dwsLVokpbd4dXSkzktSQ0O4y+rs7NSaNWs0dOhQvf/++1q9erWGDRumlStX6o477tDy5csPuc2WLVv04osv6oMPPtApp5yi66+//pAuqVdffVUbN27UJz/5SZ199tn6wx/+oLq6Ol133XV6+eWXNWXKFF111VVZ5/Tggw9q1KhR2rx5s9avX6/Zs2f3XrZ06VIdc8wx2rdvn+bPn6/169dryZIl+vGPf6wXX3xRxx57bL/XmzVrVojPHAAA8RerNViNjQfCVVpPT2o8bFdccYWGDh0qSeru7tYVV1yhU089VTfffLM2btyY9TYXXXSRRowYoWOPPVbHHXecduzYcch15s6dq4kTJ2rIkCGqqalRe3u7tmzZopNOOqm3d6q/gPXyyy9r4cKFkqRZs2YdFIyefPJJzZ49W7W1tdq4caM2bdqU9T5yvR4AAOhfrALWW2/lN16II488svf0nXfeqfr6erW1tem5557rtyNqxIgRvaeHDh2qvXv3HtZ18vXmm2/q3nvv1apVq7R+/XpddNFFWeeY6/UAAKhUzRuaVX1ftYZ8b4iq76tW84Yi7CuUg1gFrEmT8hsPS3d3tyZMmCBJevTRR0O//1NOOUVvvPGG2tvbJUnLli3Ler1zzjlHjz/+uCSpra1N69evlyS9//77OvLIIzV27Fjt2LFDv/71r3tvM2bMGH3wwQeDXg8AgErXvKFZi55bpI7uDrlcHd0dWvTcorKErFgFrKVLpb4Hw40alRovpltvvVW33367amtrQ1nj1NcnPvEJPfDAA7rgggs0Z84cjRkzRmPHjj3ketdff7127dql6dOn66677tKcOXMkSaeffrpqa2s1bdo0ffWrX9XZZ5/de5tFixbpggsuUH19/YDXAwCg0jWualTPnoP3FerZ06PGVUXYV2gQlj5KrRLU1dV5396mzZs3a/r06TnfR3Nzap+rt95KrblaujT8HdzLYdeuXRo9erTcXTfccIOmTp2qm2++uWzzyfd1AQCg2IZ8b4hch+Yak2n/d/eHvjwzW+fuWfuYYrUGS0qFqfZ2af/+1O84hCtJevjhh1VTU6OZM2equ7tb1113XbmnBABARZk0Nvs+Qf2NF1PsAlZc3XzzzWptbdWmTZvU3NxMMSgAAH0snb9Uo4Yf/O/jqOGjtHR+kfcVyoKABQAAYqHhtAY1XdykyWMny2SaPHaymi5uUsNppd+cFauiUQAAEE/NG5rVuKpRb3W/pUljJ2np/KVZg1PDaQ1lCVR9EbAAAEBFS9cvpI8QTNcvSKqIMJUNmwgBAEBFq6T6hVzlFbDM7BEze9fM2jLGjjGz35jZa8Hvo4NxM7P7zex1M1tvZrP7v+fKtXPnTtXU1KimpkYnnHCCJkyY0Hv+448/HvT2L730ktasWdN7/qGHHtLPfvaz0OeZ+cXS/WltbdWKFStCXzYAAMX0Vnf2r2Tpb7wS5LsG61FJF/QZ+46kVe4+VdKq4LwkXShpavCzSNKDhz/N8hk3bpxaW1vV2tqqxYsX9x7N19raqiOOOGLQ2/cNWIsXL9bXv/71Yk65XwQsAEAUVVL9Qq7yClju/rKk9/oMXyLpseD0Y5IuzRj/maf8SdJRZja+kMnmohTfQbRu3Tp97nOf05w5c3T++edr+/btkqT7779fM2bM0KxZs7RgwQK1t7froYce0k9+8hPV1NRo9erVuvvuu3XvvfdKkubNm6fbbrtNc+fO1cknn6zVq1dLknp6enTllVdqxowZuuyyy3TGGWeobwGrJD3//POaNm2aZs+eraeeeqp3fO3atTrzzDNVW1urs846S1u3btXHH3+su+66S8uWLVNNTY2WLVuW9XoAAFSaSqpfyFUYO7kf7+7bg9N/lXR8cHqCpLczrtcZjG3PGJOZLVJqDZcmFfilgaXYCc7d9a1vfUvPPPOMqqqqtGzZMjU2NuqRRx7RPffcozfffFMjRozQ3//+dx111FFavHixRo8erVtuuUWStGrVqoPub+/evVq7dq1WrFih733ve1q5cqUeeOABHX300dq0aZPa2tpUU1NzyDx2796tb3zjG/rtb3+rT3/60/rKV77Se9m0adO0evVqDRs2TCtXrtQdd9yh5cuX6/vf/75aWlr005/+VFLquwezXQ8AgEqS/jc8l6MIK0WoRxG6u5tZXt+94+5Nkpqk1FflFLL8gXaCC+tF+Oijj9TW1qbzzjtPkrRv3z6NH59aMTdr1iw1NDTo0ksv1aWXXjrQ3fS6/PLLJUlz5szp/TLn3//+9/r2t78tSTr11FM1a9asQ263ZcsWTZkyRVOnTpUkLVy4UE1NTZJSXz599dVX67XXXpOZac+ePVmXnev1AAAohlyrF6TKqV/IVRhHEe5Ib/oLfr8bjL8j6cSM600MxoqmFDvBubtmzpzZux/Whg0b9MILL0iSfvWrX+mGG27QK6+8os985jM5ffHziBEjJElDhw4N7Yui77zzTtXX16utrU3PPfecdu/eXdD1AAAIW3qrU0d3h1zeu9WpGLv2lEMYAetZSVcHp6+W9EzG+NeDown/UVJ3xqbEoijFTnAjRoxQV1eX/vjHP0qS9uzZo40bN2r//v16++23VV9frx/84Afq7u7Wrl27NGbMGH3wwQd5LePss8/Wk08+KUnatGmTNmzYcMh1pk2bpvb2dm3btk2S9MQTT/Re1t3drQkTJkiSHn300d7xvnPp73oAABRbFKsX8pFvTcMTkv4o6RQz6zSzf5Z0j6TzzOw1SecG5yVphaQ3JL0u6WFJ3wxt1v0oxU5wQ4YM0S9+8QvddtttOv3001VTU6M1a9Zo3759WrhwoU477TTV1tZqyZIlOuqoo3TxxRfr6aef7t3JPRff/OY31dXVpRkzZuhf//VfNXPmTI0dO/ag64wcOVJNTU266KKLNHv2bB133HG9l9166626/fbbVVtbe9Basfr6em3atKl3J/f+rgcAQLFFsXohH+Ze0G5Poaqrq/O+R8tt3rxZ06dPz/k+8tmeW6n27dunPXv2aOTIkdq2bZvOPfdcbd26NadaiFLJ93UBACBT9X3V6ujuOGR88tjJar+pvfQTOgxmts7d67JdFruvyonaTnDZ9PT0qL6+Xnv27JG764EHHqiocAUAQKGWzl960JH/UuVXL+QjdgErDsaMGZO19woAgLiIYvVCPiIRsNxdZlbuaSBQSZuVAQCVJ9fddeKw1ak/Ff9lzyNHjtTOnTv5R71CuLt27typkSNHlnsqAIAKFPf6hVxV/E7ue/bsUWdnJx1NFWTkyJGaOHGihg8fXu6pAAAqTBx2Xs9VpHdyHz58uKZMmVLuaQAAgBzEvX4hVxW/iRAAAERHKUq/o4CABQAAQlOK0u8oIGABAIDQNJzWoKaLmzR57GSZTJPHTlbTxU2xPVqwPxW/kzsAAKgMcfi2lDBFeid3AABQfun6hXTzerp+QVKiQ1Z/2EQIAAAG1biq8aCvtZGknj09alzVWKYZVTYCFgAAGBT1C/khYAEAgEFRv5AfAhYAABgU9Qv5IWABAIBBUb+QH2oaAABIMKoXDh81DQAA4BBULxQPmwgBAEgoqheKh4AFAEBCUb1QPAQsAAASiuqF4iFgAQCQUFQvFA8BCwCAhKJ6oXioaQAAIIaoXyg+ahoAAEgQ6hfKj02EAADEDPUL5UfAAgAgZqhfKD8CFgAAMUP9QvkRsAAAiBnqF8qPgAUAQMxQv1B+1DQAABARVC9UFmoaAACIOKoXooVNhAAARADVC9FCwAIAIAKoXogWAhYAABFA9UK0FBywzOwUM2vN+HnfzG4ys7vN7J2M8S+EMWEAAJKI6oVoKThguftWd69x9xpJcyT1SHo6uPgn6cvcfUWhywIAIKmoXoiWsI8inC9pm7t3mFnIdw0AQDzlWr/QcFoDgSoiwt4Ha4GkJzLO32hm683sETM7OtsNzGyRmbWYWUtXV1fI0wEAoLKl6xc6ujvk8t76heYNzeWeGgoQWtGomR0h6S+SZrr7DjM7XtLfJLmk/yppvLtfO9B9UDQKAEia6vuq1dHdccj45LGT1X5Te+knhJwNVDQa5hqsCyW94u47JMndd7j7PnffL+lhSXNDXBYAALFA/UI8hRmwrlLG5kEzG59x2WWS2kJcFgAAsUD9QjyFErDM7EhJ50l6KmP4h2a2wczWS6qXdHMYywIAIE6oX4inUI4idPf/J2lcn7GvhXHfAADEWfqoQL7EOV5C28k9DOzkDgCIk1zrFxBNA+3kHnYPFgAA0IH6hfQXNKfrFyQRshKA7yIEAKAIGlc19oartJ49PWpc1VimGaGUCFgAABQB9QvJRsACAKAIqF9INgIWAABFQP1CshGwAAAogobTGtR0cZMmj50sk2ny2MlquriJHdwTgpoGAADy0NwsNTZKb70lTZokLV0qNZCZEomaBgAAQtDcLC1aJPUEBwd2dKTOS4QsHIxNhAAA5Kix8UC4SuvpSY0DmQhYAADk6K1+Ghb6G0dyEbAAAMjRpH4aFvobR3IRsAAAyNHSpdKog5sXNGpUahzIRMACACBHDQ1SU5M0ebJklvrd1MQO7jgUAQsAAKWOEKyuloYMSf1ubs5+vYYGqb1d2r8/9ZtwhWyoaQAAJB71Cwgba7AAAIlH/QLCRsACACQe9QsIGwELAJB41C8gbAQsAEDiUb+AsBGwAACJR/0CwkbAAgDEGvULKAdqGgAAsUX9AsqFNVgAgNiifgHlQsACAMQW9QsoFwIWACC2qF9AuRCwAACxRf0CyoWABQCILeoXUC4ELABA5ORavSBRv4DyoKYBABApVC8gCliDBQCIFKoXEAUELABApFC9gCggYAEAIoXqBUQBAQsAEClULyAKCFgAgEihegFREFrAMrN2M9tgZq1m1hKMHWNmvzGz14LfR4e1PABA/ORav0D1Aipd2Guw6t29xt3rgvPfkbTK3adKWhWcBwDgEOn6hY4Oyf1A/cJAHVdApSr2JsJLJD0WnH5M0qVFXh4AIKKoX0CchBmwXNILZrbOzILKNx3v7tuD03+VdHzfG5nZIjNrMbOWrq6uEKcDAIgS6hcQJ2EGrH9y99mSLpR0g5mdk3mhu7tSIUx9xpvcvc7d66qqqkKcDgAgSqhfQJyEFrDc/Z3g97uSnpY0V9IOMxsvScHvd8NaHgAgXqhfQJyEErDM7EgzG5M+LenzktokPSvp6uBqV0t6JozlAQDih/oFxElYa7COl/R7M/s/ktZK+pW7Py/pHknnmdlrks4NzgMAEob6BSTNsDDuxN3fkHR6lvGdkuaHsQwAQDSl6xfSRwim6xckAhTiiyZ3AEBRUb+AJCJgAQCKivoFJBEBCwBQVNQvIIkIWACAoqJ+AUlEwAIAFBX1C0iiUI4iBABgIA0NBCokC2uwAACHJdduKyCJWIMFAMgb3VbAwFiDBQDIG91WwMAIWACAvNFtBQyMgAUAyBvdVsDACFgAgLzRbQUMjIAFAMgb3VbAwAhYAICD5Fq/0NAgtbdL+/enfhOugAOoaQAA9KJ+AQgHa7AAAL2oXwDCQcACAPSifgEIBwELANCL+gUgHAQsAEAv6heAcBCwAAC9qF8AwkHAAoCEoH4BKB1qGgAgAahfAEqLNVgAkADULwClRcACgASgfgEoLQIWACQA9QtAaRGwACABqF8ASouABQAJQP0CUFoELACIsFyrFyTqF4BSoqYBACKK6gWgcrEGCwAiiuoFoHIRsAAgoqheACoXAQsAIorqBaByEbAAIKKoXgAqFwELACKK6gWgchGwAKAC5Vq/QPUCUJkKDlhmdqKZvWhmm8xso5l9Oxi/28zeMbPW4OcLhU8XAOIvXb/Q0SG5H6hfGKjjCkBlMXcv7A7Mxksa7+6vmNkYSeskXSrpSkm73P3eXO+rrq7OW1paCpoPAERddXUqVPU1eXJqLRWAymBm69y9LttlBReNuvt2SduD0x+Y2WZJEwq9XwBIKuoXgOgLdR8sM6uWVCvpP4OhG81svZk9YmZHh7ksAIgr6heA6AstYJnZaEnLJd3k7u9LelDSpyTVKLWG60f93G6RmbWYWUtXV1dY0wGAyKJ+AYi+UAKWmQ1XKlw1u/tTkuTuO9x9n7vvl/SwpLnZbuvuTe5e5+51VVVVYUwHACKN+gUg+sI4itAk/bukze7+44zx8RlXu0xSW6HLAoCoo34BSIaCd3KXdLakr0naYGatwdgdkq4ysxpJLqld0nUhLAsAIitdv5D+guZ0/YJEgALipuCahjBR0wAgzqhfAOJloJoGmtwBoESoXwCSg4AFACVC/QKQHAQsACgR6heA5CBgAUCJUL8AJAcBCwAKlGv1gkT9ApAUYdQ0AEBiUb0AIBvWYAFAARobD4SrtJ6e1DiA5CJgAUABqF4AkA0BCwAKQPUCgGwIWABQAKoXAGRDwAKAAlC9ACAbAhYA9CPX+gWqFwD0RU0DAGRB/QKAQrAGCwCyoH4BQCEIWACQBfULAApBwAKALKhfAFAIAhYAZEH9AoBCELAAIAvqFwAUgoAFIHGoXwBQbNQ0AEgU6hcAlAJrsAAkCvULAEqBgAUgUahfAFAKBCwAiUL9AoBSIGABSBTqFwCUAgELQKJQvwCgFAhYAGIh1+oFifoFAMVHTQOAyKN6AUClYQ0WgMijegFApSFgAYg8qhcAVBoCFoDIo3oBQKUhYAGIPKoXAFQaAhaAyKN6AUClIWABqGi51i9QvQCgklDTAKBiUb8AIKpYgwWgYlG/ACCqCFgAKhb1CwCiqugBy8wuMLOtZva6mX2n2MsDEB/ULwCIqqIGLDMbKul/SLpQ0gxJV5nZjGIuE0B8UL8AIKqKvQZrrqTX3f0Nd/9Y0s8lXVLkZQKICeoXAERVsQPWBElvZ5zvDMZ6mdkiM2sxs5aurq4iTwdAJci1ekGifgFANJV9J3d3b3L3Onevq6qqKvd0ABRZunqho0NyP1C9MFDIAoCoKXbAekfSiRnnJwZjABKK6gUASVDsgPVnSVPNbIqZHSFpgaRni7xMABWM6gUASVDUgOXueyXdKOk/JG2W9KS7byzmMgFUNqoXACRB0ffBcvcV7n6yu3/K3Tm4Gkg4qhcAJEHZd3IHkCxULwBIAgIWgNDkWr9A9QKAuBtW7gkAiId0/UL6CMF0/YJEgAKQPKzBAhAK6hcA4AACFoBQUL8AAAcQsACEgvoFADiAgAUgFNQvAMABBCwAoaB+AQAOIGABGBT1CwCQH2oaAAyI+gUAyB9rsAAMiPoFAMgfAQvAgKhfAID8EbAADIj6BQDIHwELwICoXwCA/BGwAAyI+gUAyB8BC0ioXKsXJOoXACBf1DQACUT1AgAUF2uwgASiegEAiouABSQQ1QsAUFwELCCBqF4AgOIiYAEJRPUCABQXAQtIIKoXAKC4CFhAzORav0D1AgAUDzUNQIxQvwAAlYE1WECMUL8AAJWBgAXECPULAFAZCFhAjFC/AACVgYAFxAj1CwBQGQhYQIxQvwAAlYGABUQE9QsAEB3UNAARQP0CAEQLa7CACKB+AQCihYAFRAD1CwAQLQQsIAKoXwCAaCFgARFA/QIAREtBAcvM/s3MtpjZejN72syOCsarzexDM2sNfh4KZ7pAMlG/AADRYu5++Dc2+7yk37r7XjP7gSS5+21mVi3pl+5+aj73V1dX5y0tLYc9HwAAgFIxs3XuXpftsoLWYLn7C+6+Nzj7J0kTC7k/IGly7bYCAERLmPtgXSvp1xnnp5jZq2b2OzP7bH83MrNFZtZiZi1dXV0hTgeobOluq44Oyf1AtxUhCwCib9BNhGa2UtIJWS5qdPdngus0SqqTdLm7u5mNkDTa3Xea2RxJ/1vSTHd/f6BlsYkQSVJdnQpVfU2enGpgBwBUtoE2EQ7a5O7u5w5y59dI+qKk+R6kNXf/SNJHwel1ZrZN0smSSE9AgG4rAIivQo8ivEDSrZK+5O49GeNVZjY0OH2SpKmS3ihkWUDc0G0FAPFV6D5YP5U0RtJv+tQxnCNpvZm1SvqFpMXu/l6BywJihW4rAIivgr7s2d0/3c/4cknLC7lvIO7SHVaNjanNgpMmpcIV3VYAEH00uQNFkGv9QkNDaof2/ftTvwlXABAPBa3BAnCodP1CT7BXYrp+QSJAAUBSsAYLCFlj44FwldbTkxoHACQDAQsIGfULAAACFhAy6hcAAAQsIGTULwAACFhAyBoapKam1FfemKV+NzWxgzsAJAkBC8gD9QsAgFxQ0wDkiPoFAECuWIMF5Ij6BQBArghYQI6oXwAA5IqABeSI+gUAQK4IWECOqF8AAOSKgAXkiPoFAECuCFhIvFyrFyTqFwAAuaGmAYlG9QIAoBhYg4VEo3oBAFAMBCwkGtULAIBiIGAh0aheAAAUAwELiUb1AgCgGAhYSDSqFwAAxUDAQmzlWr9A9QIAIGzUNCCWqF8AAJQTa7AQS9QvAADKiYCFWKJ+AQBQTgQsxBL1CwCAciJgIZaoXwAAlBMBC7FE/QIAoJwIWIgc6hcAAJWOmgZECvULAIAoYA0WIoX6BQBAFBCwECnULwAAooCAhUihfgEAEAUELEQK9QsAgCggYCFSqF8AAERBQQHLzO42s3fMrDX4+ULGZbeb2etmttXMzi98qoizXKsXJOoXAACVL4yahp+4+72ZA2Y2Q9ICSTMlfVLSSjM72d33hbA8xAzVCwCAuCnWJsJLJP3c3T9y9zclvS5pbpGWhYijegEAEDdhBKwbzWy9mT1iZkcHYxMkvZ1xnc5g7BBmtsjMWsyspaurK4TpIGqoXgAAxM2gAcvMVppZW5afSyQ9KOlTkmokbZf0o3wn4O5N7l7n7nVVVVV5PwBEH9ULAIC4GXQfLHc/N5c7MrOHJf0yOPuOpBMzLp4YjAGHWLr04H2wJKoXAADRVuhRhOMzzl4mqS04/aykBWY2wsymSJoqaW0hy0J8Ub0AAIibQvfB+qGZbTCz9ZLqJd0sSe6+UdKTkjZJel7SDRxBmEy51i9QvQAAiJOCahrc/WsDXLZUEht5Eoz6BQBAUtHkjqKhfgEAkFQELBQN9QsAgKQiYKFoqF8AACQVAQtFs3Rpqm4hE/ULAIAkIGChaKhfAAAkFQELh4X6BQAA+ldQTQOSifoFAAAGxhos5I36BQAABkbAQt6oXwAAYGAELOSN+gUAAAZGwELeqF8AAGBgBCzkjfoFAAAGRsBCr1yrFyTqFwAAGAg1DZBE9QIAAGFiDRYkUb0AAECYCFiQRPUCAABhImBBEtULAACEiYAFSVQvAAAQJgIWJFG9AABAmAhYCZBr/QLVCwAAhIOahpijfgEAgNJjDVbMUb8AAEDpEbBijvoFAABKj4AVc9QvAABQegSsmKN+AQCA0iNgxRz1CwAAlB4BK6JyrV6QqF8AAKDUqGmIIKoXAACobKzBiiCqFwAAqGwErAiiegEAgMpGwIogqhcAAKhsBKwIonoBAIDKRsCKIKoXAACobASsCpNr/QLVCwAAVC5qGioI9QsAAMRDQWuwzGyZmbUGP+1m1hqMV5vZhxmXPRTOdOON+gUAAOKhoDVY7v6V9Gkz+5Gk7oyLt7l7TSH3nzTULwAAEA+h7INlZibpSklPhHF/SUX9AgAA8RDWTu6flbTD3V/LGJtiZq+a2e/M7LP93dDMFplZi5m1dHV1hTSdaKJ+AQCAeBg0YJnZSjNry/JzScbVrtLBa6+2S5rk7rWS/kXS42b2D9nu392b3L3O3euqqqoKeSyRR/0CAADxMGjAcvdz3f3ULD/PSJKZDZN0uaRlGbf5yN13BqfXSdom6eTiPIRooH4BAIDkCKOm4VxJW9y9Mz1gZlWS3nP3fWZ2kqSpkt4IYVmRRP0CAADJEsY+WAt06M7t50haH9Q2/ELSYnd/L4RlRRL1CwAAJEvBa7Dc/ZosY8slLS/0vuOC+gUAAJKFr8opAeoXAABIFgJWCVC/AABAshCwSoD6BQAAkoWAVYBcqxck6hcAAEiSMGoaEonqBQAA0IR2a6cAAAcJSURBVB/WYB0mqhcAAEB/CFiHieoFAADQHwLWYaJ6AQAA9IeAdZioXgAAAP0hYB0mqhcAAEB/CFhZ5Fq/QPUCAADIhpqGPqhfAAAAhWINVh/ULwAAgEIRsPqgfgEAABSKgNUH9QsAAKBQBKw+qF8AAACFImD1Qf0CAAAoFEcRZtHQQKACAACHL1FrsHLttwIAAChEYtZg0W8FAABKJTFrsOi3AgAApZKYgEW/FQAAKJXEBCz6rQAAQKkkJmDRbwUAAEolMQGLfisAAFAqiTmKUKLfCgAAlEZi1mABAACUCgELAAAgZAQsAACAkBGwAAAAQkbAAgAACBkBCwAAIGQELAAAgJARsAAAAEJGwAIAAAgZAQsAACBkBCwAAICQEbAAAABCZu5e7jn0MrMuSR0lWNSxkv5WguVUqqQ/fonnQOI5kHgOkv74JZ4DieegkMc/2d2rsl1QUQGrVMysxd3ryj2Pckn645d4DiSeA4nnIOmPX+I5kHgOivX42UQIAAAQMgIWAABAyJIasJrKPYEyS/rjl3gOJJ4Diecg6Y9f4jmQeA6K8vgTuQ8WAABAMSV1DRYAAEDRELAAAABCFuuAZWZXmNlGM9tvZnV9LrvdzF43s61mdn7G+AXB2Otm9p3Sz7p4zGyZmbUGP+1m1hqMV5vZhxmXPVTuuRaLmd1tZu9kPNYvZFyW9T0RJ2b2b2a2xczWm9nTZnZUMJ6Y94AU7895f8zsRDN70cw2BX8Xvx2M9/uZiJvg796G4HG2BGPHmNlvzOy14PfR5Z5nsZjZKRmvc6uZvW9mN8X9PWBmj5jZu2bWljGW9XW3lPuDvw3rzWz2YS83zvtgmdl0Sfsl/U9Jt7h7+gM1Q9ITkuZK+qSklZJODm72fyWdJ6lT0p8lXeXum0o89aIzsx9J6nb375tZtaRfuvup5Z1V8ZnZ3ZJ2ufu9fcazvifcfV/JJ1lEZvZ5Sb91971m9gNJcvfbEvYeGKqEfM4zmdl4SePd/RUzGyNpnaRLJV2pLJ+JODKzdkl17v63jLEfSnrP3e8JwvbR7n5bueZYKsHn4B1JZ0j6L4rxe8DMzpG0S9LP0n/j+nvdg3D5LUlfUOq5+W/ufsbhLDfWa7DcfbO7b81y0SWSfu7uH7n7m5JeV+of1rmSXnf3N9z9Y0k/D64bK2ZmSv1RfaLcc6kg/b0nYsXdX3D3vcHZP0maWM75lEkiPud9uft2d38lOP2BpM2SJpR3VhXhEkmPBacfUyp0JsF8SdvcvRTfnlJW7v6ypPf6DPf3ul+iVBBzd/+TpKOC/5zkLdYBawATJL2dcb4zGOtvPG4+K2mHu7+WMTbFzF41s9+Z2WfLNbESuTFY9ftIxuaApLz2ma6V9OuM80l5DyTxtT5IsMayVtJ/BkPZPhNx5JJeMLN1ZrYoGDve3bcHp/8q6fjyTK3kFujg/2Qn5T2Q1t/rHtrfh8gHLDNbaWZtWX5i/z/SbHJ8Pq7SwR+s7ZImuXutpH+R9LiZ/UMp5x2mQZ6DByV9SlKNUo/7R2WdbBHk8h4ws0ZJeyU1B0Oxeg+gf2Y2WtJySTe5+/tKwGciwz+5+2xJF0q6Idh01MtT+8zEd7+ZgJkdIelLkv5XMJSk98AhivW6Dwv7DkvN3c89jJu9I+nEjPMTgzENMB4Jgz0fZjZM0uWS5mTc5iNJHwWn15nZNqX2SWsp4lSLJtf3hJk9LOmXwdmB3hORksN74BpJX5Q0P/jDErv3wCBi81rny8yGKxWumt39KUly9x0Zl2d+JmLH3d8Jfr9rZk8rtbl4h5mNd/ftwaagd8s6ydK4UNIr6dc+Se+BDP297qH9fYj8GqzD9KykBWY2wsymSJoqaa1SO7tONbMpQcJfEFw3Ts6VtMXdO9MDZlYV7PAoMztJqefjjTLNr6j6bEu/TFL6qJL+3hOxYmYXSLpV0pfcvSdjPDHvASXjc36IYN/Lf5e02d1/nDHe32ciVszsyGDnfpnZkZI+r9RjfVbS1cHVrpb0THlmWFIHbcVIynugj/5e92clfT04mvAflToYbHu2OxhM5NdgDcTMLpP03yVVSfqVmbW6+/nuvtHMnpS0SanNJDekjxYzsxsl/YekoZIecfeNZZp+sfTd7i5J50j6vpntUeqoy8Xu3neHwLj4oZnVKLU6uF3SdZI00HsiZn4qaYSk36T+vdWf3H2xEvQeCI6gjPvnPJuzJX1N0gYLKlok3SHpqmyfiRg6XtLTwft+mKTH3f15M/uzpCfN7J8ldSh1AFBsBeHyPB38Omf9uxgXZvaEpHmSjjWzTknflXSPsr/uK5Q6gvB1ST1KHWF5eMuNc00DAABAOSR1EyEAAEDRELAAAABCRsACAAAIGQELAAAgZAQsAACAkBGwAAAAQkbAAgAACNn/B1LFXfK+Me4bAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 720x504 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cxRcHZFgtS_B"
      },
      "source": [
        "Beautiful! Any time you can visualize your data, your model, your anything, it's a good idea. \n",
        "\n",
        "With this graph in mind, what we'll be trying to do is build a model which learns the pattern in the blue dots (`X_train`) to draw the green dots (`X_test`).\n",
        "\n",
        "Time to build a model. We'll make the exact same one from before (the one we trained for longer)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4qpe0eSStSm-"
      },
      "source": [
        "# Set random seed\n",
        "tf.random.set_seed(42)\n",
        "\n",
        "# Create a model (same as above)\n",
        "model = tf.keras.Sequential([\n",
        "  tf.keras.layers.Dense(1)\n",
        "])\n",
        "\n",
        "# Compile model (same as above)\n",
        "model.compile(loss=tf.keras.losses.mae,\n",
        "              optimizer=tf.keras.optimizers.SGD(),\n",
        "              metrics=[\"mae\"])\n",
        "\n",
        "# Fit model (same as above)\n",
        "#model.fit(X_train, y_train, epochs=100) # commented out on purpose (not fitting it just yet)"
      ],
      "execution_count": 37,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hc2RHCCfqlAc"
      },
      "source": [
        "## Visualizing the model\n",
        "\n",
        "After you've built a model, you might want to take a look at it (especially if you haven't built many before).\n",
        "\n",
        "You can take a look at the layers and shapes of your model by calling [`summary()`](https://www.tensorflow.org/api_docs/python/tf/keras/Model#summary) on it.\n",
        "\n",
        "> 🔑 **Note:** Visualizing a model is particularly helpful when you run into input and output shape mismatches."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OlgJj0cFwChH",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 334
        },
        "outputId": "44547880-60d0-4070-8a8e-af3ebbbcff74"
      },
      "source": [
        "# Doesn't work (model not fit/built)\n",
        "model.summary()"
      ],
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "error",
          "ename": "ValueError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-38-7d09d31d4e66>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# Doesn't work (model not fit/built)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msummary\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/keras/engine/training.py\u001b[0m in \u001b[0;36msummary\u001b[0;34m(self, line_length, positions, print_fn, expand_nested)\u001b[0m\n\u001b[1;32m   2578\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbuilt\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2579\u001b[0m       raise ValueError(\n\u001b[0;32m-> 2580\u001b[0;31m           \u001b[0;34m'This model has not yet been built. '\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2581\u001b[0m           \u001b[0;34m'Build the model first by calling `build()` or by calling '\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2582\u001b[0m           'the model on a batch of data.')\n",
            "\u001b[0;31mValueError\u001b[0m: This model has not yet been built. Build the model first by calling `build()` or by calling the model on a batch of data."
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SJj6KLe8xsyF"
      },
      "source": [
        "Ahh, the cell above errors because we haven't fit our built our model.\n",
        "\n",
        "We also haven't told it what input shape it should be expecting.\n",
        "\n",
        "Remember above, how we discussed the input shape was just one number?\n",
        "\n",
        "We can let our model know the input shape of our data using the `input_shape` parameter to the first layer (usually if `input_shape` isn't defined, Keras tries to figure it out automatically)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zMXKFtFBuWgJ"
      },
      "source": [
        "# Set random seed\n",
        "tf.random.set_seed(42)\n",
        "\n",
        "# Create a model (same as above)\n",
        "model = tf.keras.Sequential([\n",
        "  tf.keras.layers.Dense(1, input_shape=[1]) # define the input_shape to our model\n",
        "])\n",
        "\n",
        "# Compile model (same as above)\n",
        "model.compile(loss=tf.keras.losses.mae,\n",
        "              optimizer=tf.keras.optimizers.SGD(),\n",
        "              metrics=[\"mae\"])"
      ],
      "execution_count": 39,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "24Q325x2yCoi",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "316b4790-abb8-4d16-c2a8-11087e8ca8ca"
      },
      "source": [
        "# This will work after specifying the input shape\n",
        "model.summary()"
      ],
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential_8\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " dense_8 (Dense)             (None, 1)                 2         \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 2\n",
            "Trainable params: 2\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jdurWKlryl6b"
      },
      "source": [
        "Calling `summary()` on our model shows us the layers it contains, the output shape and the number of parameters.\n",
        "* **Total params** - total number of parameters in the model.\n",
        "* **Trainable parameters** - these are the parameters (patterns) the model can update as it trains.\n",
        "* **Non-trainable parameters** - these parameters aren't updated during training (this is typical when you bring in the already learned patterns from other models during transfer learning).\n",
        "\n",
        "> 📖 **Resource:** For a more in-depth overview of the trainable parameters within a layer, check out [MIT's introduction to deep learning video](https://youtu.be/njKP3FqW3Sk).\n",
        "\n",
        "> 🛠 **Exercise:** Try playing around with the number of hidden units in the `Dense` layer (e.g. `Dense(2)`, `Dense(3)`). How does this change the Total/Trainable params? Investigate what's causing the change.\n",
        "\n",
        "For now, all you need to think about these parameters is that their learnable patterns in the data.\n",
        "\n",
        "Let's fit our model to the training data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-Kywg4q9u051",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "32d2564b-0383-4688-ab1c-0711573b4e2a"
      },
      "source": [
        "# Fit the model to the training data\n",
        "model.fit(X_train, y_train, epochs=100, verbose=0) # verbose controls how much gets output"
      ],
      "execution_count": 41,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f8df64e8250>"
            ]
          },
          "metadata": {},
          "execution_count": 41
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-jWuOwj961ri",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5f1717a8-c183-492d-e8ef-45ad5368f9de"
      },
      "source": [
        "# Check the model summary\n",
        "model.summary()"
      ],
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential_8\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " dense_8 (Dense)             (None, 1)                 2         \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 2\n",
            "Trainable params: 2\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RrrkHEkMYh5A"
      },
      "source": [
        "Alongside summary, you can also view a 2D plot of the model using [`plot_model()`](https://www.tensorflow.org/api_docs/python/tf/keras/utils/plot_model)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "liIg5WqDfdK4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 201
        },
        "outputId": "05654e49-d1cc-4c5c-c80a-f9b27b552f8b"
      },
      "source": [
        "from tensorflow.keras.utils import plot_model\n",
        "\n",
        "plot_model(model, show_shapes=True)"
      ],
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbUAAAC4CAIAAADrIeXgAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nO3de1gTZ7oA8G9CgCSQQABBQFECggoItdoFhHpctraVBUGkUsU96OqD2MpVj1y8IOCF6gJPLBzXy9I9xa0IWLAKto+eRQ6nymNXQYurDSgKIgKCJEC4z/ljns6ZhRBCLkzA9/dX883ky/t9Da8zk2/ewXAcRwAAAMZg0B0AAABoKciPAAAgG+RHAACQDfIjAADIxqS+uHXrVkZGBl2hAAAAvTw8PGJjY8mX/3L82NjYWFhYOOUhAXlu3759+/ZtuqOYxgoLC5uamuiOQuPge6K627dv37p1i9rCHLtTQUHBVMUDJhYcHIzgf4oKMAyLiYn55JNP6A5Es+B7ojpiDqng+iMAAMgG+REAAGSD/AgAALJBfgQAANkgPwIAgGyQHwGQobS01MjI6LvvvqM7EDXbsWMH9qvQ0FDqpuvXryckJBQVFQkEAmKHzZs3U3dYvXo1l8vV0dFxcnK6e/fu1Ab+L0ZGRjIzMz09PamNly9fTk9PHx4eJluKi4vJwZqZmSnxQZAfAZBhBte1MjExKSsre/z48blz58jGgwcPCoXCxMTEoKCgJ0+e2NnZmZqa5uXlXb16ldznhx9+KCgo8PPzq62tXbp0KR2xI4SQSCR6//33Y2Nje3t7qe3+/v4sFsvHx+fNmzdEy9q1a5uamioqKtasWaPcZ0F+BEAGX1/frq4uPz8/TX+QVCoddRykaWw2+6OPPnJwcNDX1ydajh07duHChYsXL3K5XHI3oVDIYDDCw8O7urqmMjz5ampq4uPjIyIi3Nzcxm6NiopydXVds2bN0NAQQgjDMGtra29v7wULFij3cZAfAaDTuXPnWltbaQygrq5u//79hw4dYrFY1HZPT8/o6OgXL17s3r2brtjGcnV1LSoq2rRpE5ncR0lOTq6urs7KylLLx0F+BGC0yspKGxsbDMO+/PJLhFBOTo6BgQGHwykpKfn44495PN6cOXO++eYbYmehUMhisczNzXfs2GFpaclisTw9PauqqoitkZGRenp6s2fPJl5+9tlnBgYGGIa1t7cjhKKjo+Pi4urr6zEMs7e3Rwhdu3aNx+MdPnx4ygYrFApxHPf39x+7KS0tzcHB4ezZs9evX5f5XhzHMzIyFi1apK+vz+fzAwICHj16RGySP2kIoeHh4QMHDtjY2LDZ7CVLluTn56tlOHw+f+XKlVlZWWq5QgL5EYDRvLy8fvzxR/Llzp07Y2JipFIpl8vNz8+vr68XCATbt28fHBxECEVGRoaFhfX29kZFRTU0NNy9e3doaOiDDz5obGxECAmFQuqtjdnZ2YcOHSJfZmVl+fn52dnZ4TheV1eHECJ+XhgZGZmywV69etXR0ZHD4YzdxGazv/rqKwaDsX379p6enrE7JCcnJyQkJCUltba2VlRUNDY2ent7v3r1Ck00aQih+Pj4L774IjMz8+XLl35+fhs3bvzpp5/UMqJ33nnnxYsXNTU1qncF+REARXl6evJ4vFmzZoWEhPT09Dx//pzcxGQyicOoxYsX5+TkSCSS3NxcJT7C19dXLBbv379ffVHL09PT8/TpUzs7u/F28PDwiImJaWhoiI+PH7VJKpVmZGSsW7cuNDTUyMjIxcXl1KlT7e3tp0+fpu4mc9L6+vpycnICAwODgoKMjY337dunq6ur3IyNRVxtfPDggepdQX4EYNL09PQQQuSh0CjLli3jcDjkmaY2a21txXFc5sEjKS0tzdHRMTs7u7KyktpeW1vb3d29bNkysmX58uV6enrktYVRqJP2+PHj3t5eZ2dnYhObzZ49e7a6ZowYDnEYqyLIjwCon76+fltbG91RTKyvrw8hNN5vHQQWi5Wbm4th2NatW6VSKdlOLKMxNDSk7mxsbCyRSCb8XOJsfd++feT6xGfPno1ar6M0NpuNfh2aiiA/AqBmg4ODb968mTNnDt2BTIxIJdQ11TIRVWNFIlFqairZaGxsjBAalQ0VHPisWbMQQpmZmTjFqNqLShsYGEC/Dk1FkB8BULPy8nIcx93d3YmXTCZzvDNx2pmbm2MYpsgKx9TU1IULF967d49scXZ2NjQ0pP6oUlVVNTAw8O67707Y29y5c1ksVnV1tXJhy0cMx8LCQvWuID8CoAYjIyOdnZ1DQ0P379+Pjo62sbEJCwsjNtnb23d0dBQXFw8ODra1tT179oz6RhMTk+bm5oaGBolEMjg4WFZWNpXrezgcjkAgUKS+OnGWraOjQ22Ji4u7dOlSXl6eWCx+8OBBRESEpaVleHi4Ir1t2bLlm2++ycnJEYvFw8PDTU1NL1++RAiFhIRYWFiocv8iMRwXFxeleyBBfgRgtC+//HL58uUIob17965duzYnJyczMxMhtGTJkidPnpw5cyYuLg4h9NFHH4lEIuItfX19Li4ubDbb29vbwcHh73//O3lRb+fOnatWrfr0008dHR1TU1OJ8z4PDw9iAVBERIS5ufnixYvXrFnT0dEx9YP19fWtra0lLyx+++239vb29fX1y5cv37VrF3VPd3d36rNZEEIHDx48cuRISkqKmZnZypUr58+fX15ebmBggBCacNKysrJiYmLS09NNTU0tLS2jo6M7OzsRQgMDA62trSUlJTKjvX37tpeXl5WVVVVVVU1NjaWl5YoVKyoqKqj73Llzx9raesmSJWqYHer5P7FEEwfaZP369evXr6c7imkMIZSfn6/RjwgPDzcxMdHoR0xIwe9JeHi4tbU1tUUkEjGZzK+//lpjoU3O8PCwt7f3uXPnlHt7e3s7i8U6ceIEtTEqKsrU1HTC946dQzh+BEANJvyJQ3tIpdLvv/9eJBIRv2PY29unpKSkpKR0d3fTHRoaHh4uLi6WSCQhISHK9ZCcnOzm5hYZGYkQwnG8ubm5srKSWHuvBMiPALxdOjo6iPoUW7duJVoSEhKCg4NDQkJoL0VRXl5eVFRUVlYmf0nmeDIyMqqrq0tLS3V1dRFCJSUlRH0KahWiSVE1P27bto3L5WIYpqGfopTzt7/9bfny5Vwud968eVu2bGlpaVHwjdO06t/t27cXLVrEYDAwDLOwsEhLS5uyj6aWC5w9e/aokoJvg8TExNzc3K6uLltbW+1/PPKpU6fIk8e8vDyy/fDhw5GRkUePHqUxNoSQj4/P+fPnydvVJ6WkpKS/v7+8vJzP5xMtAQEB1PNuZQKinmwrd/2RuOf83r17k32jhly4cAEhlJ6e/ubNm3v37gkEAjc3t8HBQUXee+XKFR6Pd/nyZU0HqTjFrz9++OGHCKHOzk5NhzSWnZ2dkZHR1H+uIpDmrz9qA7hOrbq34vrjn//8Zysrqz179hgZGbm5ucXGxlZXV493z9MoM7jqnxpN6+ABUJwa8iOGYap3okaNjY2WlpZkVHPnzkUIjVp0Rjvaq/6pYloHD4DilMmPOI4fP37c0dFRX1/fyMhoz5491K0yy7pNWAzu5s2b7733HofD4fF4Li4uYrF4vK4mJBAIqH+9xMVHgUAw4RtprPqndtoW/P/8z/8sXrzYyMiIxWK5uLh8//33CKFt27YRFy7t7OyIGzO2bNnC4XCMjIwuX76MxvkCfPHFFxwOh8vltra2xsXFWVtbP378WJ1zBwCJerKt4PXHpKQkDMP+9Kc/dXZ29vb2ZmdnI8r1x927d+vr6xcWFnZ2diYmJjIYjDt37hDvQgjduHGjq6urtbXV29vbwMBgYGAAx/Hu7m4ej5eeni6VSltaWtatW9fW1ianK/nKy8t1dXWFQqFYLP75558XLVr04YcfKngBgliye/LkSXKk48WM43h4eLiBgcHDhw/7+vpqa2uJX4SeP39ObN20aZOFhQXZ8/HjxxFCxLhwHA8KCiKq/k1I6euPUxn8hNcfCwoKkpOTOzo6Xr9+7e7uTi5GCwoK0tHRefHiBbnnxo0byeu/8r9LUVFRJ0+eXLdu3T//+U85H43g+iNQjBquP0ql0szMzN/97nexsbHGxsZsNtvExITcOmFZN5nF4BoaGsRisZOTE4vFsrCwKCoqMjMzU7pC3MqVK/fu3RsZGcnj8ZydnSUSydmzZyc7TKopqPqnOVoS/Pr16w8ePMjn801MTPz9/V+/fk2Ut4mIiBgeHiY/VywW37lzh3ia0oRfgGPHjn3++edFRUULFy7UUNjgLTfp/FhXV9fb2+vj4yNzq+Jl3ajF4AQCgbm5eWhoaHJyckNDw2S7GiUpKen06dM3btzo7u5+8uSJp6cneS+XiqZ11T/tCZ5Ym0YsqP7tb3/r4ODwl7/8BcdxhNCFCxdCQkKIm3zVWCJww4YN2ExXWFhYWFhIdxTT29jlWczJftWIe7+J8kRjkWXd9u3bRzZaWlrK75PNZv/3f/93fHz84cOHU1JSPvnkk9zcXOW6evnyZXp6ekJCwm9/+1uEkK2t7ZkzZ/h8/vHjx4VCoUIjVMF0qfonk0aDv3r16vHjx2tra8ViMTVHYxi2Y8eO2NjYGzdu/O53v/uv//qv8+fPE5uU+wLIFB0d7eHhodoItB1xs3NMTAzdgUxjxBxSTTo/Eg856+/vl7mVLOsWHR09qW6dnJy+++67tra2jIyMY8eOOTk5ETcYTbYrkUg0PDxsZWVFtvB4PBMTk9ra2knFo4RpVPVvLE0EX1FR8Y9//CMmJub58+eBgYHr1q37y1/+YmVldfLkyf/4j/8gdwsLC0tMTDx79uzcuXN5PN68efOIdqW/S2N5eHhQHwIzIxUUFCCEZvwwNYqYQ6pJn187OzszGIybN2/K3KpcWbfm5uaHDx8ihGbNmnX06NGlS5c+fPhQua6Iv3CiUBJBIpF0dHQQq3w0ahpV/RtLE8H/4x//IEq5PHjwYHBwcOfOnQKBgMViYf+6IIzP52/YsKG4uPjEiRPbt28n2zVaIhAARUw6P86aNSsoKKiwsPDcuXNisfj+/fvUx/HIKesmR3Nz844dOx49ejQwMHDv3r1nz565u7sr15Wtre2qVavOnDlTUVEhlUobGxuJanR//OMfJztSRair6p8mYpuQ5oIfHBx89eoVWerKxsYGIXT9+vW+vj6RSDR2rX5ERER/f/+VK1eoK/OV+wIAoE7UH7MVXN8jkUi2bdtmampqaGjo5eV14MABhNCcOXNqampwHO/v79+7d6+NjQ2TySSSaW1tbXZ2NnHD+YIFC+rr60+fPs3j8RBC8+bN++WXXxoaGjw9Pfl8vo6OjpWVVVJS0tDQ0HhdTRhee3t7dHS0vb29vr6+oaHhihUrvv32W0V+3T958iSx6I/D4fj7+8uPGcfx8PBwXV1da2trJpPJ4/ECAgLq6+vJ3l6/fr1q1SoWi2Vra7tr1y5ilai9vT2xhubu3bvz5s1js9leXl4tLS1yolJk3cbt27ednJwYDAZCaPbs2YcPH56y4P/zP/9TztPvLl26RHS4d+9eExMTY2Pj4OBgYnmpnZ0duZwIx/F33nknISFh1LhkfgHS09OJEopz585VpCoXgvU9QDFj5xDqPypvaqr+aeh7rw0lC6nWrFnz5MkTTfQM+REo6K24/3oqTaOqf2PRHjx5bn7//n3iWJXeeAAYZZrlx0ePHslZviS/pqYq7wWasHfvXpFI9Msvv2zZsoX6YDygOTt27CC/86OK0V2/fj0hIYFasG7z5s3UHVavXs3lcnV0dJycnFR5PozqRkZGMjMzRxVJuXz5cnp6OvVf/eLiYnKwZmZmynwS9WASzq8Vl5CQQKy4nj9/fkFBgeY+SBPnTVMWvHxJSUkMBmPu3LkaLSiH4PyagriuUlZW9vjx476+PrL9wIEDfn5+YrGYeGlnZ2dqaooQunLlCvXtZWVla9euVW/kk/XLL7+sWLECIeTq6jpqU1ZW1sqVK8lbbEdGRpqamioqKtasWQPPV5hSR44c6e/vx3H86dOn69evpzucydGS4NPS0oaHh58/fz4FBeU0R43V3qamcBybzSbqh5NPEDt27NiFCxcuXrzI5XLJ3YRCIYPBCA8Pp72oOFVNTU18fHxERISbm9vYrVFRUa6urmvWrBkaGkIIYRhG1A9fsGCBch8H+REAlaix2hsthePq6ur2799/6NAh4tYPkqenZ3R09IsXL3bv3j3FIcnh6upaVFS0adMmMrmPkpycXF1dnZWVpZaPg/wIAMJxPCMjg6jWwefzAwICyBu9J1XtTb2F465duzYFz8IWCoU4jvv7+4/dlJaW5uDgcPbs2evXr8t8r5x5m7CkoXLVCyfE5/NXrlyZlZWF47jqvUF+BAAlJycnJCQkJSW1trZWVFQ0NjZ6e3u/evUKISQUCqk37WVnZx86dIh8mZWV5efnR1R7q6uri4yMDAsL6+3tjYqKamhouHv37tDQ0AcffECUR5lUV+jXBQYjIyMaHfvVq1cdHR1lPg+LzWZ/9dVXDAZj+/btxO3wo8iZt507d8bExEilUi6Xm5+fX19fLxAItm/fTi5aiI+P/+KLLzIzM1++fOnn57dx48affvpJLSN65513Xrx4UVNTo3pXkB/B204qlWZkZKxbty40NNTIyMjFxeXUqVPt7e3UG8MmRV2F43x9fcVi8f79+5ULQxE9PT1Pnz6Vs8Lfw8MjJiamoaEhPj5+1CYF501mhT2lqxcqgrja+ODBA9W7gvwI3na1tbXd3d3Lli0jW5YvX66np6fgM4vk0/Kqd62trTiOy3+YalpamqOjY3Z2dmVlJbV9svNGrbCnxuJ1YxHDIQ5jVQT5Ebzt3rx5gxAyNDSkNhobG0skErX0r81V7/r6+hBC4/3WQWCxWLm5uRiGbd26VSqVku2qzBtZvI5cn/js2bPe3l7lRjEKcfspMTQVQX4EbztjY2OE0Ki/anVVe9PyqndEKpnwTioPD4/Y2FiRSERdxq/KvJHF66iLDW/duqXEEMYaGBhAvw5NRZAfwdvO2dnZ0NCQ+uNAVVXVwMDAu+++S7xUpdqblle9Mzc3xzBMkRWOqampCxcuJB6jRphw3uTQaPE6YjgWFhaqdwX5EbztWCxWXFzcpUuX8vLyxGLxgwcPIiIiLC0ticp4aPLV3tRVOK6srEzT63s4HI5AICAeCiAfcZZNPPqCbJE/b/J7G694XUhIiIWFhSr3LxLDcXFxUboHEuRHANDBgwePHDmSkpJiZma2cuXK+fPnk8UrEUI7d+5ctWrVp59+6ujomJqaSpy4kQ81ioiIMDc3X7x48Zo1azo6OhBCfX19Li4ubDbb29vbwcHh73//O3mBb7JdTQFfX9/a2lrywuK3335rb29fX1+/fPnyXbt2Ufd0d3ePjY2ltsiZt5ycHOJxBUuWLHny5MmZM2fi4uIQQh999JFIJEIIZWVlxcTEpKenm5qaWlpaRkdHd3Z2IoQGBgZaW1tLSkpkRnv79m0vLy8rK6uqqqqamhpLS8sVK1ZUVFRQ97lz5461tfWSJUvUMDvU83+4/1oLQd0qFaGpvf+arsJxit9/bW1tTW0RiURMJlORSppTY3h42Nvb+9y5c8q9vb29ncVinThxgtoYFRUF918DoBVoLxwnn1Qq/f7770UiEfE7hr29fUpKSkpKSnd3N92hoeHh4eLiYolEonQ9reTkZDc3t8jISIQQjuPNzc2VlZXEenslQH4E4O3S0dFB1KfYunUr0ZKQkBAcHBwSEkJ7KYry8vKioqKysjL5SzLHk5GRUV1dXVpaSjxDuKSkhKhPcfXqVeXigfwIgNokJibm5uZ2dXXZ2tqOfZiyNjh16hR58piXl0e2Hz58ODIy8ujRozTGhhDy8fE5f/48eYv6pJSUlPT395eXl/P5fKIlICCAet6tRJ+Tfr4rAGA8R44cOXLkCN1RKGn16tWrV6+mOwrlrV27du3atertE44fAQBANsiPAAAgG+RHAACQDfIjAADIJuP3mYsXL059HGA8xM1S8D9FFeoqfKDN4HuiuqamptHFNaiLxdVV4hwAAKajUffPYLg6ntIAgFpgGJafn099CAEANILrjwAAIBvkRwAAkA3yIwAAyAb5EQAAZIP8CAAAskF+BAAA2SA/AgCAbJAfAQBANsiPAAAgG+RHAACQDfIjAADIBvkRAABkg/wIAACyQX4EAADZID8CAIBskB8BAEA2yI8AACAb5EcAAJAN8iMAAMgG+REAAGSD/AgAALJBfgQAANkgPwIAgGyQHwEAQDbIjwAAIBvkRwAAkA3yIwAAyAb5EQAAZIP8CAAAskF+BAAA2SA/AgCAbJAfAQBANsiPAAAgG4bjON0xgLdXeHj448ePyZd37961tbXl8/nESx0dnb/+9a9z5syhKTrwtmPSHQB4q1lYWJw+fZracv/+ffK/BQIBJEdAIzi/BnTauHHjeJv09PTCwsKmMBYARoPza0AzZ2fnhw8fyvwePn782MHBYepDAoAAx4+AZn/4wx90dHRGNWIY5urqCskR0AvyI6DZp59+Ojw8PKpRR0fn3//932mJBwASnF8D+nl6elZVVY2MjJAtGIY1NjZaW1vTGBUAcPwI6Ld582YMw8iXDAbDy8sLkiOgHeRHQL/g4GDqSwzD/vCHP9AVDAAkyI+AfmZmZj4+PuSvNBiGBQYG0hsSAAjyI9ASoaGhxKVwHR2dDz/80NTUlO6IAID8CLTDunXr9PT0EEI4joeGhtIdDgAIQX4EWsLAwOD3v/89QkhPT8/Pz4/ucABACPIj0B6bNm1CCAUGBhoYGNAdCwAITYv1j9SVHwCAGSM/P/+TTz6hOwp5pkf9nujoaA8PD7qjoEdmZiZCKCYmhu5A1OPWrVtZWVn5+fkyt+bl5YWEhDCZ0+NrKd+GDRve5u/thDZs2EB3CBObHseP2v/vjOYQawMLCgroDkQ9Ll68uGHDhvG+dX19fSwWa4pD0pC3/Hs7oWkxP3D9EWiRGZMcwcwA+REAAGSD/AgAALJBfgQAANkgPwIAgGyQH8E0UFpaamRk9N1339EdiKZcv349ISGhqKhIIBBgGIZh2ObNm6k7rF69msvl6ujoODk53b17l644EUIjIyOZmZmenp7UxsuXL6enp4+tczzdQX4E04D2r0JTxcGDB4VCYWJiYlBQ0JMnT+zs7ExNTfPy8q5evUru88MPPxQUFPj5+dXW1i5dupSuUEUi0fvvvx8bG9vb20tt9/f3Z7FYPj4+b968oSs2TYD8CKYBX1/frq6uKbgvWyqVjjoy0rRjx45duHDh4sWLXC6XbBQKhQwGIzw8vKurayqDka+mpiY+Pj4iIsLNzW3s1qioKFdX1zVr1gwNDU19bBoC+RGA/3fu3LnW1tYp+7i6urr9+/cfOnRo1MJPT0/P6OjoFy9e7N69e8qCmZCrq2tRUdGmTZv09fVl7pCcnFxdXZ2VlTXFgWkO5Eeg7SorK21sbDAM+/LLLxFCOTk5BgYGHA6npKTk448/5vF4c+bM+eabb4idhUIhi8UyNzffsWOHpaUli8UiHm5DbI2MjNTT05s9ezbx8rPPPjMwMMAwrL29HSEUHR0dFxdXX1+PYZi9vT1C6Nq1azwe7/DhwxoamlAoxHHc399/7Ka0tDQHB4ezZ89ev35d5ntxHM/IyFi0aJG+vj6fzw8ICHj06BGxSf4UIYSGh4cPHDhgY2PDZrOXLFky3u2ek8Xn81euXJmVlTVjrodAfgTazsvL68cffyRf7ty5MyYmRiqVcrnc/Pz8+vp6gUCwffv2wcFBhFBkZGRYWFhvb29UVFRDQ8Pdu3eHhoY++OCDxsZGhJBQKKTe0JadnX3o0CHyZVZWlp+fn52dHY7jdXV1CCHiBwfqg8PU6+rVq46OjhwOZ+wmNpv91VdfMRiM7du39/T0jN0hOTk5ISEhKSmptbW1oqKisbHR29v71atXaKIpQgjFx8d/8cUXmZmZL1++9PPz27hx408//aSWEb3zzjsvXryoqalRS2+0g/wIpitPT08ejzdr1qyQkJCenp7nz5+Tm5hMJnFgtXjx4pycHIlEkpubq8RH+Pr6isXi/fv3qy/q/9fT0/P06VM7O7vxdvDw8IiJiWloaIiPjx+1SSqVZmRkrFu3LjQ01MjIyMXF5dSpU+3t7adPn6buJnOK+vr6cnJyAgMDg4KCjI2N9+3bp6urq9z8jLVgwQKE0IMHD9TSG+0gP4Jpjyg8Th4cjbJs2TIOh0Oee2qP1tZWHMdlHjyS0tLSHB0ds7OzKysrqe21tbXd3d3Lli0jW5YvX66np0deSRiFOkWPHz/u7e11dnYmNrHZ7NmzZ6trfojhEIexMwDkRzDz6evrt7W10R3FaH19fQih8X7rILBYrNzcXAzDtm7dKpVKyXZiGY2hoSF1Z2NjY4lEMuHnEmfr+/btw3717NmzUet1lMZms9GvQ5sBID+CGW5wcPDNmzdz5syhO5DRiFQy4ZpqDw+P2NhYkUiUmppKNhobGyOERmVDBYc5a9YshFBmZiZOcevWLSWGMNbAwAD6dWgzAORHMMOVl5fjOO7u7k68ZDKZ452JTzFzc3MMwxRZ4Ziamrpw4cJ79+6RLc7OzoaGhtQfVaqqqgYGBt59990Je5s7dy6LxaqurlYubPmI4VhYWGii86kH+RHMQCMjI52dnUNDQ/fv34+OjraxsQkLCyM22dvbd3R0FBcXDw4OtrW1PXv2jPpGExOT5ubmhoYGiUQyODhYVlamufU9HA5HIBA0NTVNuCdxlk0+H5xoiYuLu3TpUl5enlgsfvDgQUREhKWlZXh4uCK9bdmy5ZtvvsnJyRGLxcPDw01NTS9fvkQIhYSEWFhYqHL/IjEcFxcXpXvQLrjWQwjl5+fTHQVt1q9fv379erqjUBtiqd2k3nLy5ElixSKHw/H398/OziZ+BFiwYEF9ff3p06d5PB5CaN68eb/88guO4+Hh4bq6utbW1kwmk8fjBQQE1NfXk729fv161apVLBbL1tZ2165de/bsQQjZ29s/f/4cx/G7d9CpgcMAAA1/SURBVO/OmzePzWZ7eXm1tLSUlpZyudy0tDQlRqrI9zYyMlJXV7e3t5d4eenSJeLnbDMzs88//3zUznv27Fm7di35cmRk5Pjx4wsWLNDV1eXz+YGBgY8fPyY2TThF/f39e/futbGxYTKZs2bNCgoKqq2txXE8MDAQIXTgwAGZ0d66dWvFihWWlpZE6pg9e7anp+fNmzep+/j6+lpbW4+MjKhlfmgH+VHbQX6crPDwcBMTE41+hCIU+d6KRCImk/n1119PTUgTGh4e9vb2PnfunHJvb29vZ7FYJ06cUGTnafF3DefXYAaaLoVk7O3tU1JSUlJSuru76Y4FDQ8PFxcXSySSkJAQ5XpITk52c3OLjIxUb2A0moH5cdu2bVwuF8MwDV2BVs7f/va35cuXc7ncefPmbdmypaWlRY2dU+tiEfT09MzNzf/t3/7t+PHjnZ2davwsoF4JCQnBwcEhISG0l6IoLy8vKioqKyuTvyRzPBkZGdXV1aWlpbq6umqPjS4zMD+ePXv2zJkzdEfxL/Lz8zdt2hQcHNzU1FRSUlJRUfHxxx+rscwJWRfLyMgIx/GRkZHW1taLFy/a2tru3bvXyclJXXePab/ExMTc3Nyuri5bW9vCwkK6w1HI4cOHIyMjjx49Sm8YPj4+58+fJ29On5SSkpL+/v7y8nI+n6/2wOhE9wn+xNDkr1MQt+Lfu3dPQyFN1qpVq6ysrMiL1kSdhcrKSkXeq/j1RzI/UhUUFDAYDHNz8zdv3kwqZg2ZguuPWkKJ7+1bZVrMzww8fkQIYRhGdwj/orGx0dLSkoxq7ty5CKFRK0s0ZP369WFhYa2tradOnZqCjwNgJpkh+RHH8ePHjzs6Ourr6xsZGRGLNkgyqzlNWAPq5s2b7733HofD4fF4Li4uYrF4vK4mJBAIqFUFiYuPAoFALWOfELH0r6ysjHhJ+2wAMG3QfQA7MaTAcXhSUhKGYX/60586Ozt7e3uzs7MR5fx69+7d+vr6hYWFnZ2diYmJDAbjzp07xLsQQjdu3Ojq6mptbfX29jYwMBgYGMBxvLu7m8fjpaenS6XSlpaWdevWtbW1yelKvvLycl1dXaFQKBaLf/7550WLFn344YcKDl/F82scx4lcNnfuXG2YDTi/BoRpMT/T4Js64Tz29vZyOJwPPviAbKFef5RKpRwOJyQkhNxZX19/586d+K8ZQSqVEpuIrFpXV4fj+M8//4wQunLlCvWD5HQ1oX379pH/Js2ZM6exsVHB4aueH3EcxzDM2NgY14LZgPwICNNifphTdZyqQXV1db29vT4+PjK3Kl7NiVoDSiAQmJubh4aGRkVFhYWFzZ8/f1JdjZKUlHT27NkbN2785je/aW1tjY+P9/Dw+PHHH4kLkZrW09OD4zhxB4U2zAZC6OLFi2oYmNZTV9EHQBu6E/TE0ET/zpSWliKEqIv+qceP//u//zt21O7u7viYIyZiVdA///lP4uXPP//8+9//nslkYhi2YcOG3t5eOV3J0dzcrKOjs2/fPrKlq6uLwWDs2rVLkeGrfvxI3E67evVqXAtmA65RApL2Hz/OhN9niGcb9ff3y9yqdDUnJyen7777rrm5ee/evfn5+SdOnFCuK5FINDw8bGVlRbbweDwTE5Pa2lrFx6iKa9euIYQ+/vhjpAWzQdDcF1p7oOnw908jVb7SU2Ym5EdnZ2cGg3Hz5k2ZW5Wr5tTc3Pzw4UOE0KxZs44ePbp06dKHDx8q1xVRko+oj0KQSCQdHR1Tc3Ld0tKSmZk5Z86crVu3Ii2YDQCmkZmQH4kCJIWFhefOnROLxffv36c+hUNONSc5mpubd+zY8ejRo4GBgXv37j179szd3V25rmxtbVetWnXmzJmKigqpVNrY2EgUofrjH/+o+thHwXG8u7ubWIje1taWn5+/YsUKHR2d4uJi4voj7bMBwHRC91H2xJAC5ykSiWTbtm2mpqaGhoZeXl4HDhxACM2ZM6empgYfp5qT/BpQDQ0Nnp6efD5fR0fHysoqKSlpaGhovK4mHEJ7e3t0dLS9vb2+vr6hoeGKFSu+/fZbBYevyPXHy5cvL1myhMPh6OnpMRgMhBDxg/V7772XkpLy+vVr6s70zgb8fg0I02J+MFzrLwRgGJafn099LOdbJTg4GCFUUFBAdyDqcfHixQ0bNmj/t051b/n3dkLTYn5mwvk1AABoAuRHVT169Agbn9Kl9AAAtIP8qKqFCxfKuX5x4cIFugME08D169cTEhKodTw3b95M3WH16tVcLldHR8fJyUmV58OobmRkJDMz09PTk9p4+fLl9PT06VKWWHGQHwGg2cGDB4VCYWJiIlnH09TUNC8v7+rVq+Q+P/zwQ0FBgZ+fX21t7dKlS+kKVSQSvf/++7GxsaOel+3v789isXx8fIgHc88YkB/BTCOVSkcd3WhDV+M5duzYhQsXLl68yOVyyUahUMhgMMLDw2kvKk5VU1MTHx8fERHh5uY2dmtUVJSrq+uaNWvUWPiZdpAfwUxz7tw5ajU5LelKprq6uv379x86dIi4B4zk6ekZHR394sWL3bt3a+7TJ8vV1bWoqGjTpk36+voyd0hOTq6urs7KypriwDQH8iPQRjiOZ2RkLFq0SF9fn8/nBwQEkJUvIiMj9fT0yMcAfPbZZwYGBhiGtbe3I4Sio6Pj4uLq6+sxDLO3txcKhSwWy9zcfMeOHZaWliwWy9PTs6qqSomuEELXrl1T7+OwhUIhjuP+/v5jN6WlpTk4OJw9e/b69euTnaIJq3lqqHAnn89fuXJlVlbWzFm/NQVrLFWEpsM6Us15O5/veuDAAT09va+//vrNmzf3799funSpmZlZS0sLsXXTpk0WFhbkzsePH0cIESUpcRwPCgqys7Mjt4aHhxsYGDx8+LCvr6+2tpZ4ShrxtOvJdnXlyhUul5uSkqLISBX53goEgsWLF49qtLOze/r0KY7jP/74I4PBmD9/fnd3N47jZWVl1Odfy58iOdU8cWXLmJJ+85vfuLq6ytyUkJCAFHu0ybT4u4bjR6B1pFJpRkbGunXrQkNDjYyMXFxcTp061d7eTr1tdFKYTCZxnLV48eKcnByJRJKbm6tEP76+vmKxeP/+/cqFMUpPT8/Tp0/t7OzG28HDwyMmJqahoSE+Pn7UJgWnyNPTk8fjzZo1KyQkpKen5/nz5wihvr6+nJycwMDAoKAgY2Pjffv26erqKjchYy1YsAAh9ODBA7X0RjvIj0Dr1NbWdnd3L1u2jGxZvny5np4eeV6simXLlnE4HAXrVGpUa2srjuPyH6aalpbm6OiYnZ1dWVlJbZ/sFFGreapSuHNCxHBevXqllt5oB/kRaB1ijYihoSG10djYWCKRqKV/fX39trY2tXSlir6+PiIYOfuwWKzc3FwMw7Zu3SqVSsl2Vaaop6cHIbRv3z7yLoZnz56NWq+jNDabjX4d2gwA+RFoHWNjY4TQqD/1N2/eEJXiVDQ4OKiurlREpJIJ11R7eHjExsaKRKLU1FSyUZUpUqVw54QGBgbQr0ObASA/Aq3j7OxsaGj4008/kS1VVVUDAwPvvvsu8ZLJZBKnikooLy/Hcdzd3V31rlRkbm6OYZgiKxxTU1MXLlx47949smXCKZJDo4U7ieFYWFhoovOpB/kRaB0WixUXF3fp0qW8vDyxWPzgwYOIiAhLS0uibiZCyN7evqOjo7i4eHBwsK2tbdSTxE1MTJqbmxsaGiQSCZH7RkZGOjs7h4aG7t+/Hx0dbWNjQzzzdrJdlZWVqXF9D4fDEQgETU1NikxIbm6ujo4OtUX+FMnvbbzCnSEhIRYWFqrcv0gMx8XFReketAs9P5tPBpoO6wA05+1c3zMyMnL8+PEFCxbo6ury+fzAwMDHjx+TW1+/fr1q1SoWi2Vra7tr1y7icef29vbEqp27d+/OmzePzWZ7eXm1tLSEh4fr6upaW1szmUwejxcQEFBfX69cV6WlpVwuNy0tTZGRKvK9jYyM1NXV7e3tJV5eunSJ+DnbzMzs888/H7Xznj17qOt75EyR/Gqe+PiFOwMDAxFCBw4ckBntrVu3VqxYYWlpSaSO2bNne3p63rx5k7qPr6+vtbU1UaFZ9fmhHeRHbfd25kc1Cg8PNzExmcpPJCjyvRWJREwm8+uvv56akCY0PDzs7e1NfdTdpLS3t7NYrBMnTiiy87T4u4bzazDzaW1dGXt7+5SUlJSUlO7ubrpjQcPDw8XFxRKJROmifMnJyW5ubpGRkeoNjEaQHwGgU0JCQnBwcEhICO2lKMrLy4uKisrKyuQvyRxPRkZGdXV1aWmprq6u2mOjC+RHMJMlJibm5uZ2dXXZ2toWFhbSHY5shw8fjoyMPHr0KL1h+Pj4nD9/nrwbfVJKSkr6+/vLy8v5fL7aA6MRk+4AANCgI0eOHDlyhO4oJrZ69erVq1fTHYXy1q5du3btWrqjUD84fgQAANkgPwIAgGyQHwEAQDbIjwAAINv0+H0mMzOzoKCA7ijocfv2bYRQcHAw3YGoB3H/2YwZjnxv8/d2ZsBwra+E/pb8LQHwtomNjfXw8KA7CnmmQX4EAABawPVHAACQDfIjAADIBvkRAABkg/wIAACy/R+aVy+cFUyDnAAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {},
          "execution_count": 43
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ygw7V8OVYxQS"
      },
      "source": [
        "In our case, the model we used only has an input and an output but visualizing more complicated models can be very helpful for debugging."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gebj2eptqwg8"
      },
      "source": [
        "## Visualizing the predictions\n",
        "\n",
        "Now we've got a trained model, let's visualize some predictions.\n",
        "\n",
        "To visualize predictions, it's always a good idea to plot them against the ground truth labels.\n",
        "\n",
        "Often you'll see this in the form of `y_test` vs. `y_pred` (ground truth vs. predictions).\n",
        "\n",
        "First, we'll make some predictions on the test data (`X_test`), remember the model has never seen the test data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RRzj7LJMYftb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ba064840-2194-4147-e5cd-998aa4c73933"
      },
      "source": [
        "# Make predictions\n",
        "y_preds = model.predict(X_test)"
      ],
      "execution_count": 44,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "WARNING:tensorflow:5 out of the last 5 calls to <function Model.make_predict_function.<locals>.predict_function at 0x7f8df63c6290> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_hKpW-KOZiAW",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f31dea6e-0f76-4662-bcba-7cc64ccd8932"
      },
      "source": [
        "# View the predictions\n",
        "y_preds"
      ],
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[53.57109 ],\n",
              "       [57.05633 ],\n",
              "       [60.541573],\n",
              "       [64.02681 ],\n",
              "       [67.512054],\n",
              "       [70.99729 ],\n",
              "       [74.48254 ],\n",
              "       [77.96777 ],\n",
              "       [81.45301 ],\n",
              "       [84.938255]], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 45
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aPRaFncaZnT-"
      },
      "source": [
        "Okay, we get a list of numbers but how do these compare to the ground truth labels?\n",
        "\n",
        "Let's build a plotting function to find out.\n",
        "\n",
        "> 🔑 **Note:** If you think you're going to be visualizing something a lot, it's a good idea to functionize it so you can use it later."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "56euC69rZvNJ"
      },
      "source": [
        "def plot_predictions(train_data=X_train, \n",
        "                     train_labels=y_train, \n",
        "                     test_data=X_test, \n",
        "                     test_labels=y_test, \n",
        "                     predictions=y_preds):\n",
        "  \"\"\"\n",
        "  Plots training data, test data and compares predictions.\n",
        "  \"\"\"\n",
        "  plt.figure(figsize=(10, 7))\n",
        "  # Plot training data in blue\n",
        "  plt.scatter(train_data, train_labels, c=\"b\", label=\"Training data\")\n",
        "  # Plot test data in green\n",
        "  plt.scatter(test_data, test_labels, c=\"g\", label=\"Testing data\")\n",
        "  # Plot the predictions in red (predictions were made on the test data)\n",
        "  plt.scatter(test_data, predictions, c=\"r\", label=\"Predictions\")\n",
        "  # Show the legend\n",
        "  plt.legend();"
      ],
      "execution_count": 46,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Fug5_B6Ab7Ah",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 428
        },
        "outputId": "8ab057a0-528a-43ad-cdee-078608c58da4"
      },
      "source": [
        "plot_predictions(train_data=X_train,\n",
        "                 train_labels=y_train,\n",
        "                 test_data=X_test,\n",
        "                 test_labels=y_test,\n",
        "                 predictions=y_preds)"
      ],
      "execution_count": 47,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAGbCAYAAAAY8u5bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3RV9Z338c+XiyDCE2/xBiWBVuWiGCDFC6PCgEq1VnHVVhtHfWyLWC3qLEermVrsszJLO7byaB+lccZRu1KLj9aqLToK6mCHOjRoHgggxUuiWAZTnEacqNy+zx/n5HgIJ+Eczj6Xvff7tVZWztnnsn/nEvz423t/trm7AAAAEJx+pR4AAABA1BCwAAAAAkbAAgAACBgBCwAAIGAELAAAgIANKPUA0h166KFeXV1d6mEAAADs1cqVK//s7pWZbiurgFVdXa3m5uZSDwMAAGCvzKy9t9vYRAgAABAwAhYAAEDACFgAAAABK6t9sDLZvn27Nm7cqE8++aTUQ0HS4MGDNWLECA0cOLDUQwEAoCyVfcDauHGjhg0bpurqaplZqYcTe+6uLVu2aOPGjRo1alSphwMAQFkq+02En3zyiQ455BDCVZkwMx1yyCHMKAIA0IeyD1iSCFdlhs8DAIC+hSJgAQAAhAkBay+2bNmimpoa1dTU6IgjjtDw4cNT17dt29bnY5ubmzVv3ry9ruOUU04Jari7mTZt2l6LWxcsWKCurq6CrB8AgLgq+53cS+2QQw5RS0uLJGn+/PkaOnSobrjhhtTtO3bs0IABmd/G2tpa1dbW7nUdy5cvD2aw+2DBggW65JJLNGTIkJKNAQCAqIncDFZTk1RdLfXrl/jd1BT8Oi6//HLNnTtXJ554om688UatWLFCJ598siZOnKhTTjlF69evlyS99NJL+vKXvywpEc6uuOIKTZs2TaNHj9bdd9+der6hQ4em7j9t2jR99atf1ZgxY1RXVyd3lyQtXrxYY8aM0eTJkzVv3rzU86b7+OOPddFFF2ns2LGaPXu2Pv7449RtV111lWprazV+/Hj94Ac/kCTdfffd+tOf/qTp06dr+vTpvd4PAADkJlIzWE1N0pw5UvcWr/b2xHVJqqsLdl0bN27U8uXL1b9/f3344Yd6+eWXNWDAAC1ZskS33HKLHn/88T0e8/rrr+vFF1/U1q1bdeyxx+qqq67ao0vqtdde05o1a3TUUUdp6tSp+vd//3fV1tbqyiuv1LJlyzRq1ChdfPHFGcd03333aciQIVq3bp1WrVqlSZMmpW5raGjQwQcfrJ07d2rGjBlatWqV5s2bp5/85Cd68cUXdeihh/Z6vwkTJgT4zgEAEH2RmsGqr/8sXHXr6kosD9qFF16o/v37S5I6Ozt14YUX6rjjjtP111+vNWvWZHzMOeeco0GDBunQQw/VYYcdps2bN+9xnylTpmjEiBHq16+fampq1NbWptdff12jR49O9U71FrCWLVumSy65RJI0YcKE3YLRo48+qkmTJmnixIlas2aN1q5dm/E5sr0fAADoXaQC1jvv5LY8HwcccEDq8ve//31Nnz5dra2tevrpp3vtiBo0aFDqcv/+/bVjx459uk+u3n77bd15551aunSpVq1apXPOOSfjGLO9HwAA5appdZOqF1Sr3239VL2gWk2rC7CvUBYiFbBGjsxteVA6Ozs1fPhwSdKDDz4Y+PMfe+yxeuutt9TW1iZJWrRoUcb7nXbaafrFL34hSWptbdWqVaskSR9++KEOOOAAVVRUaPPmzXrmmWdSjxk2bJi2bt261/sBAFDumlY3ac7Tc9Te2S6Xq72zXXOenlOSkBWpgNXQIPU8GG7IkMTyQrrxxht18803a+LEiYHMOPW0//77695779WsWbM0efJkDRs2TBUVFXvc76qrrtJHH32ksWPH6tZbb9XkyZMlSSeccIImTpyoMWPG6Bvf+IamTp2aesycOXM0a9YsTZ8+vc/7AQBQ7uqX1qtr++77CnVt71L90gLsK7QX1n2UWjmora31nr1N69at09ixY7N+jqamxD5X77yTmLlqaAh+B/dS+OijjzR06FC5u66++modffTRuv7660s2nlw/FwAACq3fbf3k2jPXmEy7frAr8PWZ2Up3z9jHFKkZLCkRptrapF27Er+jEK4k6f7771dNTY3Gjx+vzs5OXXnllaUeEgAAZWVkReZ9gnpbXkiRC1hRdf3116ulpUVr165VU1MTxaAAAPTQMKNBQwbu/t/HIQOHqGFGgfcVyoCABQAAIqHu+Do1ntuoqooqmUxVFVVqPLdRdccXf3NWpIpGAQBANDWtblL90nq90/mORlaMVMOMhozBqe74upIEqp4IWAAAoKx11y90HyHYXb8gqSzCVCZsIgQAAGWtnOoXspVTwDKzB8zsfTNrTVt2sJk9b2Ybkr8PSi43M7vbzN4ws1VmNqn3Zy5fW7ZsUU1NjWpqanTEEUdo+PDhqevbtm3b6+NfeuklLV++PHV94cKFevjhhwMfZ/qJpXvT0tKixYsXB75uAAAK6Z3OzKdk6W15Och1ButBSbN6LPuepKXufrSkpcnrkvQlSUcnf+ZIum/fh1k6hxxyiFpaWtTS0qK5c+emjuZraWnRfvvtt9fH9wxYc+fO1aWXXlrIIfeKgAUACKNyql/IVk4By92XSfqgx+LzJD2UvPyQpPPTlj/sCa9IOtDMjsxnsNkoxjmIVq5cqdNPP12TJ0/WWWedpU2bNkmS7r77bo0bN04TJkzQRRddpLa2Ni1cuFB33XWXampq9PLLL2v+/Pm68847JUnTpk3TTTfdpClTpuiYY47Ryy+/LEnq6urS1772NY0bN06zZ8/WiSeeqJ4FrJL07LPPasyYMZo0aZJ+9atfpZavWLFCJ598siZOnKhTTjlF69ev17Zt23Trrbdq0aJFqqmp0aJFizLeDwCAclNO9QvZCmIn98PdfVPy8n9KOjx5ebikd9PutzG5bFPaMpnZHCVmuDQyz5MGFmMnOHfXd7/7XT355JOqrKzUokWLVF9frwceeEC333673n77bQ0aNEh/+ctfdOCBB2ru3LkaOnSobrjhBknS0qVLd3u+HTt2aMWKFVq8eLFuu+02LVmyRPfee68OOuggrV27Vq2traqpqdljHJ988om+/e1v64UXXtAXvvAFff3rX0/dNmbMGL388ssaMGCAlixZoltuuUWPP/64fvjDH6q5uVk//elPJSXOPZjpfgAAlJPu/4ZncxRhuQj0KEJ3dzPL6dw77t4oqVFKnConn/X3tRNcUB/Cp59+qtbWVp1xxhmSpJ07d+rIIxMTcxMmTFBdXZ3OP/98nX/++X09TcoFF1wgSZo8eXLqZM6/+93vdO2110qSjjvuOE2YMGGPx73++usaNWqUjj76aEnSJZdcosbGRkmJk09fdtll2rBhg8xM27dvz7jubO8HAEAhZFu9IJVP/UK2gjiKcHP3pr/k7/eTy9+T9Lm0+41ILiuYYuwE5+4aP358aj+s1atX67nnnpMk/fa3v9XVV1+tV199VV/84hezOvHzoEGDJEn9+/cP7ETR3//+9zV9+nS1trbq6aef1ieffJLX/QAACFr3Vqf2zna5PLXVqRC79pRCEAHrKUmXJS9fJunJtOWXJo8mPElSZ9qmxIIoxk5wgwYNUkdHh37/+99LkrZv3641a9Zo165devfddzV9+nTdcccd6uzs1EcffaRhw4Zp69atOa1j6tSpevTRRyVJa9eu1erVq/e4z5gxY9TW1qY333xTkvTII4+kbuvs7NTw4cMlSQ8++GBqec+x9HY/AAAKLYzVC7nItabhEUm/l3SsmW00s29Kul3SGWa2QdLM5HVJWizpLUlvSLpf0ncCG3UvirETXL9+/fTYY4/ppptu0gknnKCamhotX75cO3fu1CWXXKLjjz9eEydO1Lx583TggQfq3HPP1RNPPJHayT0b3/nOd9TR0aFx48bp7//+7zV+/HhVVFTsdp/BgwersbFR55xzjiZNmqTDDjssdduNN96om2++WRMnTtxtVmz69Olau3Ztaif33u4HAEChhbF6IRfmntduT4Gqra31nkfLrVu3TmPHjs36OXLZnluudu7cqe3bt2vw4MF68803NXPmTK1fvz6rWohiyfVzAQAgXfWCarV3tu+xvKqiSm3XtRV/QPvAzFa6e22m2yJ3qpyw7QSXSVdXl6ZPn67t27fL3XXvvfeWVbgCACBfDTMadjvyXyr/6oVcRC5gRcGwYcMy9l4BABAVYaxeyAUBCwAABCrb3XWisNWpNwQsAAAQmGKUfodBEDUNAAAAkqJfv5AtAhYAAAhM1OsXskXAykL//v1VU1Oj4447ThdeeKG6urr2/qBeXH755XrsscckSd/61re0du3aXu/70ksvafny5anrCxcu1MMPP7zP6wYAoNCKUfodBgSsLOy///5qaWlRa2ur9ttvPy1cuHC32/e1pPOf/umfNG7cuF5v7xmw5s6dq0svvXSf1gUAQDEUo/Q7DKIXsJqapOpqqV+/xO+mYM9pdOqpp+qNN97QSy+9pFNPPVVf+cpXNG7cOO3cuVN/93d/py9+8YuaMGGCfvazn0lKnLvwmmuu0bHHHquZM2fq/fffTz3XtGnTUnUMzz77rCZNmqQTTjhBM2bMUFtbmxYuXKi77ror1QI/f/583XnnnZKklpYWnXTSSZowYYJmz56t//qv/0o950033aQpU6bomGOOSbXHr1mzRlOmTFFNTY0mTJigDRs2BPq+AAAgJXZkbzy3UVUVVTKZqiqq1HhuY6x2cJeidhRhU5M0Z47UvQmvvT1xXZLq8v9gd+zYoWeeeUazZs2SJL366qtqbW3VqFGj1NjYqIqKCv3hD3/Qp59+qqlTp+rMM8/Ua6+9pvXr12vt2rXavHmzxo0bpyuuuGK35+3o6NC3v/1tLVu2TKNGjdIHH3yggw8+WHPnztXQoUN1ww03SJKWLl2aesyll16qe+65R6effrpuvfVW3XbbbVqwYEFqnCtWrNDixYt12223acmSJVq4cKGuvfZa1dXVadu2bdq5c2fe7wcAIF6oX8hetGaw6us/C1fduroSy/Pw8ccfq6amRrW1tRo5cqS++c1vSpKmTJmiUaNGSZKee+45Pfzww6qpqdGJJ56oLVu2aMOGDVq2bJkuvvhi9e/fX0cddZT++q//eo/nf+WVV3Taaaelnuvggw/uczydnZ36y1/+otNPP12SdNlll2nZsmWp2y+44AJJ0uTJk9XW1iZJOvnkk/UP//APuuOOO9Te3q79998/r/cEABAv3fUL7Z3tcnmqfqFpdbBbiqIiWgHrnV6OUOhteZa698FqaWnRPffckzptzQEHHJC6j7vrnnvuSd3v7bff1plnnpnXevfVoEGDJCV2zu/eP+wb3/iGnnrqKe2///46++yz9cILL5RkbACAcKJ+ITfRClgjezlCobflATrrrLN03333afv27ZKkP/7xj/rv//5vnXbaaVq0aJF27typTZs26cUXX9zjsSeddJKWLVumt99+W5L0wQcfSEqcMmfr1q173L+iokIHHXRQav+qn//856nZrN689dZbGj16tObNm6fzzjtPq1atyuv1AgDihfqF3ERrH6yGht33wZKkIUMSywvsW9/6ltra2jRp0iS5uyorK/XrX/9as2fP1gsvvKBx48Zp5MiROvnkk/d4bGVlpRobG3XBBRdo165dOuyww/T888/r3HPP1Ve/+lU9+eSTuueee3Z7zEMPPaS5c+eqq6tLo0eP1r/8y7/0Ob5HH31UP//5zzVw4EAdccQRuuWWWwJ9/QCAaBtZMVLtne0Zl2NP5u6lHkNKbW2t9zzJ8bp16zR27Njsn6SpKbHP1TvvJGauGhoC2cEdu8v5cwEAhFrPU+BIifqFOB4h2M3MVrp7babbojWDJSXCFIEKAIBAdYeobI4iRBQDFgAAyFq21QsS9Qu5CEXAcneZWamHgaRy2qwMANh3PTf7dVcvSCJI5ansjyIcPHiwtmzZwn/Uy4S7a8uWLRo8eHCphwIAyFMkqxcKfEaXbJX9DNaIESO0ceNGdXR0lHooSBo8eLBGjBhR6mEAAPIUueqFAp/RJRdlH7AGDhyYajgHAADBiVz1Ql9ndClywCr7TYQAAKAwGmY0aMjAIbstGzJwiBpmFL4/siAKdEaXfUHAAgAgpuqOr1PjuY2qqqiSyVRVURXuXqsSntGlJwIWAAAR1LS6SdULqtXvtn6qXlDd60mZ646vU9t1bdr1g11qu64tvOFKSpSLD9l9Rq5YZ3TpiYAFAEDEdNcvtHe2y+Wp+oXeQlYoZHN0YF2d1NgoVVVJZonfjY0lKSAv+1PlAACA3FQvqM6483pVRZXarmsr/oDy1fPoQCkxM1Wi8NStr1PlMIMFAEDERK5+oa+jA8sUAQsAgIjprWYhtPULZXR0YLYIWAAAREzk6hfK6OjAbBGwAACImMjVL5TR0YHZImABABAS2VYvSCGpX8j2vIFldHRgtjiKEACAEOiuXkg/OfOQgUPCOzNVpkcG5qKvowgJWAAAhEDkqheqqxMnY+6pqkpqayv2aPYJNQ0AAIRc5KoXQnhkYC4IWAAAhEDkqhdCeGRgLvIOWGZ2rJm1pP18aGbXmdl8M3svbfnZQQwYAIA4ilz1QgiPDMxF3gHL3de7e42710iaLKlL0hPJm+/qvs3dF+e7LgAA4ipU1QshO29gIQS6k7uZnSnpB+4+1czmS/rI3e/M9vHs5A4AiKOm1U2qX1qvdzrf0ciKkWqY0VCewSkbETg6MFvF3Mn9IkmPpF2/xsxWmdkDZnZQL4ObY2bNZtbc0dER8HAAAChv3fUL7Z3tcrnaO9s15+k5fXZclbUQnjewEAKbwTKz/ST9SdJ4d99sZodL+rMkl/S/JB3p7lf09RzMYAEA4iZy9Qv9+kmZsoWZtGtX8cdTQMWawfqSpFfdfbMkuftmd9/p7rsk3S9pSoDrAgAgEiJXvxDxowOzFWTAulhpmwfN7Mi022ZLag1wXQAARELk6hcifnRgtgIJWGZ2gKQzJP0qbfGPzGy1ma2SNF3S9UGsCwCAKAlV/QJHB2aNU+UAAFBioTiKMEZHB2aLcxECAFACoQhO2YrAuQOD1lfAGlDswQAAEAfd9Qtd2xMzPt31C5LCGbIifu7AoHEuQgAACqB+aX0qXHXr2t6l+qUh7YPi6MCcELAAACiAyNUvcHRgTghYAAAUQOTqFzg6MCcELAAACiA09QvZVC90q6tL7NC+a1fiN+GqVwQsAAAKoO74OjWe26iqiiqZTFUVVWo8t7G8dnDvrl5ob0+c3qa9PXG9r5CFrFDTAABADpqaEuctfuedxP7dDQ0hnsiheiEv1DQAABCAnl2b3RM+UkhDFtULBcMmQgAAslRfv3uRuZS4Xh/S5gWqFwqHgAUAQJYiN+FD9ULBELAAAMhSqCZ8ODFzSRGwAADIUmgmfHI5OpDqhYIgYAEAkKXQTPhEbmex8CFgAQCg7Ps2QzHhE7mdxcKHgAUAiL3I9W2GamexaCJgAQBiL3Jb1EKzs1h0EbAAALEXmi1quWzHDMXOYtFFkzsAIPZGjsx8xpiy2qKWa418XR2BqoSYwQIAxF4otqhFbjtmtBGwAACxF4otaqHZjgmJgAUAiLjI1C9wZGCoELAAAJEVqfqFUGzHRDcCFgAgskKz2xLnDYwcc/dSjyGltrbWm5ubSz0MAEBE9OuXmLnqySyxKbAs9Dw6UErMTBGeyp6ZrXT32ky3MYMFAIisUOy2FJppNuSCgAUAiKxQ7LbE0YGRRMACAERWKHZbCsU0G3JFwAIAhE621QtSCOoXQjHNhlwRsAAAoRKq6gWODowtjiIEAIRKdXXm8wZWVSVmqMoGRwdGHkcRAgAiIzT7hHN0YKwRsAAAoRKafcJDkwRRCAQsAECohGaf8NAkQRQCAQsAECqh2Sc8NEkQhRBYwDKzNjNbbWYtZtacXHawmT1vZhuSvw8Kan0AgOjJtn6h7KsXpBAlQRRCYEcRmlmbpFp3/3Pash9J+sDdbzez70k6yN1v6u05OIoQAOKLg+4QNqU8ivA8SQ8lLz8k6fwCrw8AEFIcdIcoCTJguaTnzGylmc1JLjvc3TclL/+npMN7PsjM5phZs5k1d3R0BDgcAECYcNAdoiTIgPVX7j5J0pckXW1mp6Xf6IltkXtsj3T3RnevdffaysrKAIcDAAgTDrpDlAQWsNz9veTv9yU9IWmKpM1mdqQkJX+/H9T6AADRwkF3iJJAApaZHWBmw7ovSzpTUqukpyRdlrzbZZKeDGJ9AIDo4aA7RElQM1iHS/qdmf0/SSsk/dbdn5V0u6QzzGyDpJnJ6wCAmIlU/QKQhQFBPIm7vyXphAzLt0iaEcQ6AADh1LN+ob09cV0iQCG6aHIHABQU9QuIIwIWAKCgqF9AHBGwAAAFRf0C4oiABQAoKOoXEEcELABAQVG/gDgK5ChCAAD6UldHoEK8MIMFANgn2XZbAXHEDBYAIGd0WwF9YwYLAJAzuq2AvhGwAAA5o9sK6BsBCwCQM7qtgL4RsAAAOaPbCugbAQsAkDO6rYC+EbAAALvJtn6hrk5qa5N27Ur8JlwBn6GmAQCQQv0CEAxmsAAAKdQvAMEgYAEAUqhfAIJBwAIApFC/AASDgAUASKF+AQgGAQsAkEL9AhAMAhYAxAT1C0DxUNMAADFA/QJQXMxgAUAMUL8AFBcBCwBigPoFoLgIWAAQA9QvAMVFwAKAGKB+ASguAhYAxAD1C0BxEbAAIMSyrV6QqF8AiomaBgAIKaoXgPLFDBYAhBTVC0D5ImABQEhRvQCULwIWAIQU1QtA+SJgAUBIUb0AlC8CFgCEFNULQPkiYAFAGcq2foHqBaA85R2wzOxzZvaima01szVmdm1y+Xwze8/MWpI/Z+c/XACIvu76hfZ2yf2z+oW+Oq4AlBdz9/yewOxISUe6+6tmNkzSSknnS/qapI/c/c5sn6u2ttabm5vzGg8AhF11dSJU9VRVlZilAlAezGylu9dmui3volF33yRpU/LyVjNbJ2l4vs8LAHFF/QIQfoHug2Vm1ZImSvqP5KJrzGyVmT1gZgcFuS4AiCrqF4DwCyxgmdlQSY9Lus7dP5R0n6TPS6pRYobrx708bo6ZNZtZc0dHR1DDAYDQon4BCL9AApaZDVQiXDW5+68kyd03u/tOd98l6X5JUzI91t0b3b3W3WsrKyuDGA4AhBr1C0D4BXEUoUn6Z0nr3P0nacuPTLvbbEmt+a4LAMKO+gUgHvLeyV3SVEl/I2m1mbUkl90i6WIzq5HkktokXRnAugAgtLrrF7pP0NxdvyARoICoybumIUjUNACIMuoXgGjpq6aBJncAKBLqF4D4IGABQJFQvwDEBwELAIqE+gUgPghYAFAk1C8A8UHAAoA8ZVu9IFG/AMRFEDUNABBbVC8AyIQZLADIQ339Z+GqW1dXYjmA+CJgAUAeqF4AkAkBCwDyQPUCgEwIWACQB6oXAGRCwAKAPFC9ACATAhYA9CLb+gWqFwD0RE0DAGRA/QKAfDCDBQAZUL8AIB8ELADIgPoFAPkgYAFABtQvAMgHAQsAMqB+AUA+CFgAkAH1CwDyQcACEDvULwAoNGoaAMQK9QsAioEZLACxQv0CgGIgYAGIFeoXABQDAQtArFC/AKAYCFgAYoX6BQDFQMACECvULwAoBgIWgEjItnpBon4BQOFR0wAg9KheAFBumMECEHpULwAoNwQsAKFH9QKAckPAAhB6VC8AKDcELAChR/UCgHJDwAIQelQvACg3BCwAZS3b+gWqFwCUE2oaAJQt6hcAhBUzWADKFvULAMKKgAWgbFG/ACCsCh6wzGyWma03szfM7HuFXh+A6KB+AUBYFTRgmVl/Sf9H0pckjZN0sZmNK+Q6AUQH9QsAwqrQM1hTJL3h7m+5+zZJv5R0XoHXCSAiqF8AEFaFDljDJb2bdn1jclmKmc0xs2Yza+7o6CjwcACUg2yrFyTqFwCEU8l3cnf3RnevdffaysrKUg8HQIF1Vy+0t0vun1Uv9BWyACBsCh2w3pP0ubTrI5LLAMQU1QsA4qDQAesPko42s1Fmtp+kiyQ9VeB1AihjVC8AiIOCBix33yHpGkn/KmmdpEfdfU0h1wmgvFG9ACAOCr4Plrsvdvdj3P3z7s7B1UDMUb0AIA5KvpM7gHihegFAHBCwAAQm2/oFqhcARN2AUg8AQDR01y90HyHYXb8gEaAAxA8zWAACQf0CAHyGgAUgENQvAMBnCFgAAkH9AgB8hoAFIBDULwDAZwhYAAJB/QIAfIaABWCvqF8AgNxQ0wCgT9QvAEDumMEC0CfqFwAgdwQsAH2ifgEAckfAAtAn6hcAIHcELAB9on4BAHJHwALQJ+oXACB3BCwgprKtXpCoXwCAXFHTAMQQ1QsAUFjMYAExRPUCABQWAQuIIaoXAKCwCFhADFG9AACFRcACYojqBQAoLAIWEENULwBAYRGwgIjJtn6B6gUAKBxqGoAIoX4BAMoDM1hAhFC/AADlgYAFRAj1CwBQHghYQIRQvwAA5YGABUQI9QsAUB4IWECEUL8AAOWBgAWEBPULABAe1DQAIUD9AgCECzNYQAhQvwAA4ULAAkKA+gUACBcCFhAC1C8AQLgQsIAQoH4BAMIlr4BlZv9oZq+b2Soze8LMDkwurzazj82sJfmzMJjhAvFE/QIAhIu5+74/2OxMSS+4+w4zu0OS3P0mM6uW9Bt3Py6X56utrfXm5uZ9Hg8AAECxmNlKd6/NdFteM1ju/py770hefUXSiHyeD4ibbLutAADhEuQ+WFdIeibt+igze83M/s3MTu3tQWY2x8yazay5o6MjwOEA5a2726q9XXL/rNuKkAUA4bfXTYRmtkTSERluqnf3J5P3qZdUK+kCd3czGyRpqLtvMbPJkn4taby7f9jXuthEiDiprk6Eqp6qqhIN7ACA8tbXJsK9Nrm7+8y9PPnlkr4saYYn05q7fyrp0+TllWb2pqRjJJGegCS6rQAguvI9inCWpBslfcXdu9KWV5pZ/+Tl0ZKOlvRWPusConbr/cQAAA0BSURBVIZuKwCIrnz3wfqppGGSnu9Rx3CapFVm1iLpMUlz3f2DPNcFRArdVgAQXXmd7Nndv9DL8sclPZ7PcwNR191hVV+f2Cw4cmQiXNFtBQDhR5M7UADZ1i/U1SV2aN+1K/GbcAUA0ZDXDBaAPXXXL3Ql90rsrl+QCFAAEBfMYAEBq6//LFx16+pKLAcAxAMBCwgY9QsAAAIWEDDqFwAABCwgYNQvAAAIWEDA6uqkxsbEKW/MEr8bG9nBHQDihIAF5ID6BQBANqhpALJE/QIAIFvMYAFZon4BAJAtAhaQJeoXAADZImABWaJ+AQCQLQIWkCXqFwAA2SJgAVmifgEAkC0CFmIv2+oFifoFAEB2qGlArFG9AAAoBGawEGtULwAACoGAhVijegEAUAgELMQa1QsAgEIgYCHWqF4AABQCAQuxRvUCAKAQCFiIrGzrF6heAAAEjZoGRBL1CwCAUmIGC5FE/QIAoJQIWIgk6hcAAKVEwEIkUb8AACglAhYiifoFAEApEbAQSdQvAABKiYCF0KF+AQBQ7qhpQKhQvwAACANmsBAq1C8AAMKAgIVQoX4BABAGBCyECvULAIAwIGAhVKhfAACEAQELoUL9AgAgDPIKWGY238zeM7OW5M/ZabfdbGZvmNl6Mzsr/6EiyrKtXpCoXwAAlL8gahrucvc70xeY2ThJF0kaL+koSUvM7Bh33xnA+hAxVC8AAKKmUJsIz5P0S3f/1N3flvSGpCkFWhdCjuoFAEDUBBGwrjGzVWb2gJkdlFw2XNK7affZmFy2BzObY2bNZtbc0dERwHAQNlQvAACiZq8By8yWmFlrhp/zJN0n6fOSaiRtkvTjXAfg7o3uXuvutZWVlTm/AIQf1QsAgKjZ6z5Y7j4zmycys/sl/SZ59T1Jn0u7eURyGbCHhobd98GSqF4AAIRbvkcRHpl2dbak1uTlpyRdZGaDzGyUpKMlrchnXYguqhcAAFGT7z5YPzKz1Wa2StJ0SddLkruvkfSopLWSnpV0NUcQxlO29QtULwAAoiSvmgZ3/5s+bmuQxEaeGKN+AQAQVzS5o2CoXwAAxBUBCwVD/QIAIK4IWCgY6hcAAHFFwELBNDQk6hbSUb8AAIgDAhYKhvoFAEBcEbCwT6hfAACgd3nVNCCeqF8AAKBvzGAhZ9QvAADQNwIWckb9AgAAfSNgIWfULwAA0DcCFnJG/QIAAH0jYCFn1C8AANA3AhZSsq1ekKhfAACgL9Q0QBLVCwAABIkZLEiiegEAgCARsCCJ6gUAAIJEwIIkqhcAAAgSAQuSqF4AACBIBCxIonoBAIAgEbBiINv6BaoXAAAIBjUNEUf9AgAAxccMVsRRvwAAQPERsCKO+gUAAIqPgBVx1C8AAFB8BKyIo34BAIDiI2BFHPULAAAUHwErpLKtXpCoXwAAoNioaQghqhcAAChvzGCFENULAACUNwJWCFG9AABAeSNghRDVCwAAlDcCVghRvQAAQHkjYIUQ1QsAAJQ3AlaZybZ+geoFAADKFzUNZYT6BQAAoiGvGSwzW2RmLcmfNjNrSS6vNrOP025bGMxwo436BQAAoiGvGSx3/3r3ZTP7saTOtJvfdPeafJ4/bqhfAAAgGgLZB8vMTNLXJD0SxPPFFfULAABEQ1A7uZ8qabO7b0hbNsrMXjOzfzOzU3t7oJnNMbNmM2vu6OgIaDjhRP0CAADRsNeAZWZLzKw1w895aXe7WLvPXm2SNNLdJ0r6W0m/MLP/ken53b3R3WvdvbaysjKf1xJ61C8AABANew1Y7j7T3Y/L8POkJJnZAEkXSFqU9phP3X1L8vJKSW9KOqYwLyEcqF8AACA+gqhpmCnpdXff2L3AzColfeDuO81stKSjJb0VwLpCifoFAADiJYh9sC7Snju3nyZpVbK24TFJc939gwDWFUrULwAAEC95z2C5++UZlj0u6fF8nzsqqF8AACBeOFVOEVC/AABAvBCwioD6BQAA4oWAVQTULwAAEC8ErDxkW70gUb8AAECcBFHTEEtULwAAgN4wg7WPqF4AAAC9IWDtI6oXAABAbwhY+4jqBQAA0BsC1j6iegEAAPSGgLWPqF4AAAC9IWBlkG39AtULAAAgE2oaeqB+AQAA5IsZrB6oXwAAAPkiYPVA/QIAAMgXAasH6hcAAEC+CFg9UL8AAADyRcDqgfoFAACQL44izKCujkAFAAD2XaxmsLLttwIAAMhHbGaw6LcCAADFEpsZLPqtAABAscQmYNFvBQAAiiU2AYt+KwAAUCyxCVj0WwEAgGKJTcCi3woAABRLbI4ilOi3AgAAxRGbGSwAAIBiIWABAAAEjIAFAAAQMAIWAABAwAhYAAAAASNgAQAABIyABQAAEDACFgAAQMAIWAAAAAEjYAEAAASMgAUAABAwAhYAAEDAzN1LPYYUM+uQ1F6EVR0q6c9FWE+5ivvrl3gPJN4Difcg7q9f4j2QeA/yef1V7l6Z6YayCljFYmbN7l5b6nGUStxfv8R7IPEeSLwHcX/9Eu+BxHtQqNfPJkIAAICAEbAAAAACFteA1VjqAZRY3F+/xHsg8R5IvAdxf/0S74HEe1CQ1x/LfbAAAAAKKa4zWAAAAAVDwAIAAAhYpAOWmV1oZmvMbJeZ1fa47WYze8PM1pvZWWnLZyWXvWFm3yv+qAvHzBaZWUvyp83MWpLLq83s47TbFpZ6rIViZvPN7L2013p22m0ZvxNRYmb/aGavm9kqM3vCzA5MLo/Nd0CK9t95b8zsc2b2opmtTf67eG1yea9/E1GT/HdvdfJ1NieXHWxmz5vZhuTvg0o9zkIxs2PTPucWM/vQzK6L+nfAzB4ws/fNrDVtWcbP3RLuTv7bsMrMJu3zeqO8D5aZjZW0S9LPJN3g7t1/UOMkPSJpiqSjJC2RdEzyYX+UdIakjZL+IOlid19b5KEXnJn9WFKnu//QzKol/cbdjyvtqArPzOZL+sjd7+yxPON3wt13Fn2QBWRmZ0p6wd13mNkdkuTuN8XsO9BfMfk7T2dmR0o60t1fNbNhklZKOl/S15ThbyKKzKxNUq27/zlt2Y8kfeDutyfD9kHuflOpxlgsyb+D9ySdKOl/KsLfATM7TdJHkh7u/jeut889GS6/K+lsJd6b/+3uJ+7LeiM9g+Xu69x9fYabzpP0S3f/1N3flvSGEv9hnSLpDXd/y923Sfpl8r6RYmamxD+qj5R6LGWkt+9EpLj7c+6+I3n1FUkjSjmeEonF33lP7r7J3V9NXt4qaZ2k4aUdVVk4T9JDycsPKRE642CGpDfdvRhnTykpd18m6YMei3v73M9TIoi5u78i6cDk/5zkLNIBqw/DJb2bdn1jcllvy6PmVEmb3X1D2rJRZvaamf2bmZ1aqoEVyTXJqd8H0jYHxOWzT3eFpGfSrsflOxDHz3o3yRnLiZL+I7ko099EFLmk58xspZnNSS473N03JS//p6TDSzO0ortIu/9Pdly+A916+9wD+/ch9AHLzJaYWWuGn8j/H2kmWb4fF2v3P6xNkka6+0RJfyvpF2b2P4o57iDt5T24T9LnJdUo8bp/XNLBFkA23wEzq5e0Q1JTclGkvgPonZkNlfS4pOvc/UPF4G8izV+5+yRJX5J0dXLTUYon9pmJ7n4zSWa2n6SvSPq/yUVx+g7soVCf+4Cgn7DY3H3mPjzsPUmfS7s+IrlMfSwPhb29H2Y2QNIFkianPeZTSZ8mL680szeV2CetuYBDLZhsvxNmdr+k3ySv9vWdCJUsvgOXS/qypBnJf1gi9x3Yi8h81rkys4FKhKsmd/+VJLn75rTb0/8mIsfd30v+ft/MnlBic/FmMzvS3TclNwW9X9JBFseXJL3a/dnH6TuQprfPPbB/H0I/g7WPnpJ0kZkNMrNRko6WtEKJnV2PNrNRyYR/UfK+UTJT0uvuvrF7gZlVJnd4lJmNVuL9eKtE4yuoHtvSZ0vqPqqkt+9EpJjZLEk3SvqKu3elLY/Nd0Dx+DvfQ3Lfy3+WtM7df5K2vLe/iUgxswOSO/fLzA6QdKYSr/UpSZcl73aZpCdLM8Ki2m0rRly+Az309rk/JenS5NGEJylxMNimTE+wN6GfweqLmc2WdI+kSkm/NbMWdz/L3deY2aOS1iqxmeTq7qPFzOwaSf8qqb+kB9x9TYmGXyg9t7tL0mmSfmhm25U46nKuu/fcITAqfmRmNUpMB7dJulKS+vpORMxPJQ2S9Hziv7d6xd3nKkbfgeQRlFH/O89kqqS/kbTakhUtkm6RdHGmv4kIOlzSE8nv/QBJv3D3Z83sD5IeNbNvSmpX4gCgyEqGyzO0++ec8d/FqDCzRyRNk3SomW2U9ANJtyvz575YiSMI35DUpcQRlvu23ijXNAAAAJRCXDcRAgAAFAwBCwAAIGAELAAAgIARsAAAAAJGwAIAAAgYAQsAACBgBCwAAICA/X8kiTIckrHZWwAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 720x504 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H63NPcgPcwZV"
      },
      "source": [
        "From the plot we can see our predictions aren't totally outlandish but they definitely aren't anything special either."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "orAfIWOrgRDA"
      },
      "source": [
        "## Evaluating predictions\n",
        "\n",
        "Alongisde visualizations, evaulation metrics are your alternative best option for evaluating your model.\n",
        "\n",
        "Depending on the problem you're working on, different models have different evaluation metrics. \n",
        "\n",
        "Two of the main metrics used for regression problems are:\n",
        "* **Mean absolute error (MAE)** - the mean difference between each of the predictions.\n",
        "* **Mean squared error (MSE)** - the squared mean difference between of the predictions (use if larger errors are more detrimental than smaller errors).\n",
        "\n",
        "The lower each of these values, the better.\n",
        "\n",
        "You can also use [`model.evaluate()`](https://www.tensorflow.org/api_docs/python/tf/keras/Model#evaluate) which will return the loss of the model as well as any metrics setup during the compile step."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DPgTdF3ddxiY",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "635fd3f4-00bf-4412-e959-917c831ae5bf"
      },
      "source": [
        "# Evaluate the model on the test set\n",
        "model.evaluate(X_test, y_test)"
      ],
      "execution_count": 48,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "1/1 [==============================] - 0s 137ms/step - loss: 18.7453 - mae: 18.7453\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[18.74532699584961, 18.74532699584961]"
            ]
          },
          "metadata": {},
          "execution_count": 48
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DAXIRyVzegFd"
      },
      "source": [
        "In our case, since we used MAE for the loss function as well as MAE for the metrics, `model.evaulate()` returns them both.\n",
        "\n",
        "TensorFlow also has built in functions for MSE and MAE.\n",
        "\n",
        "For many evaluation functions, the premise is the same: compare predictions to the ground truth labels."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gqoMQ0dJeD2S",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f8938dff-e7d9-4a64-f99f-c62c65413f18"
      },
      "source": [
        "# Calculate the mean absolute error\n",
        "mae = tf.metrics.mean_absolute_error(y_true=y_test, \n",
        "                                     y_pred=y_preds)\n",
        "mae"
      ],
      "execution_count": 49,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(10,), dtype=float32, numpy=\n",
              "array([34.42891 , 30.943668, 27.45843 , 23.97319 , 20.487946, 17.202168,\n",
              "       14.510478, 12.419336, 11.018796, 10.212349], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 49
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7pYp3XNYfkyC"
      },
      "source": [
        "Huh? That's strange, MAE should be a single output.\n",
        "\n",
        "Instead, we get 10 values.\n",
        "\n",
        "This is because our `y_test` and `y_preds` tensors are different shapes."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zeUHwOc2gIV4",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b5f2e2c8-8bf9-469c-aff2-fbbca656fe54"
      },
      "source": [
        "# Check the test label tensor values\n",
        "y_test"
      ],
      "execution_count": 50,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([ 70,  74,  78,  82,  86,  90,  94,  98, 102, 106])"
            ]
          },
          "metadata": {},
          "execution_count": 50
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3aJwlTxugPyc",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "cfa714b2-f083-4d35-c8c0-6dc31b4a9f03"
      },
      "source": [
        "# Check the predictions tensor values (notice the extra square brackets)\n",
        "y_preds"
      ],
      "execution_count": 51,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[53.57109 ],\n",
              "       [57.05633 ],\n",
              "       [60.541573],\n",
              "       [64.02681 ],\n",
              "       [67.512054],\n",
              "       [70.99729 ],\n",
              "       [74.48254 ],\n",
              "       [77.96777 ],\n",
              "       [81.45301 ],\n",
              "       [84.938255]], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 51
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xolZ-lmge_ES",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0ae30863-314f-4699-9ad8-951df7f4d55d"
      },
      "source": [
        "# Check the tensor shapes\n",
        "y_test.shape, y_preds.shape"
      ],
      "execution_count": 52,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "((10,), (10, 1))"
            ]
          },
          "metadata": {},
          "execution_count": 52
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eW1qleu5gHyZ"
      },
      "source": [
        "Remember how we discussed dealing with different input and output shapes is one the most common issues you'll come across, this is one of those times.\n",
        "\n",
        "But not to worry.\n",
        "\n",
        "We can fix it using [`squeeze()`](https://www.tensorflow.org/api_docs/python/tf/squeeze), it'll remove the the `1` dimension from our `y_preds` tensor, making it the same shape as `y_test`.\n",
        "\n",
        "> 🔑 **Note:** If you're comparing two tensors, it's important to make sure they're the right shape(s) (you won't always have to manipulate the shapes, but always be on the look out, *many* errors are the result of mismatched tensors, especially mismatched input and output shapes)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tVtMyw70g4aF",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "1345aaa4-c6da-468f-91a2-4f9151ab2a46"
      },
      "source": [
        "# Shape before squeeze()\n",
        "y_preds.shape"
      ],
      "execution_count": 53,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(10, 1)"
            ]
          },
          "metadata": {},
          "execution_count": 53
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qnYaBnCng-Nq",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5d5a33d4-1fae-4a40-ba2f-06b93d9e208c"
      },
      "source": [
        "# Shape after squeeze()\n",
        "y_preds.squeeze().shape"
      ],
      "execution_count": 54,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(10,)"
            ]
          },
          "metadata": {},
          "execution_count": 54
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HxvVeD64hEX8",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "21c20781-b6cf-4d35-c6a7-c32882a9baf0"
      },
      "source": [
        "# What do they look like?\n",
        "y_test, y_preds.squeeze()"
      ],
      "execution_count": 55,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([ 70,  74,  78,  82,  86,  90,  94,  98, 102, 106]),\n",
              " array([53.57109 , 57.05633 , 60.541573, 64.02681 , 67.512054, 70.99729 ,\n",
              "        74.48254 , 77.96777 , 81.45301 , 84.938255], dtype=float32))"
            ]
          },
          "metadata": {},
          "execution_count": 55
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mfUCIeHyhLk7"
      },
      "source": [
        "Okay, now we know how to make our `y_test` and `y_preds` tenors the same shape, let's use our evaluation metrics."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TvjY6GIJvXBO",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e5617649-57f9-4304-9b66-1f77edf74695"
      },
      "source": [
        "# Calcuate the MAE\n",
        "mae = tf.metrics.mean_absolute_error(y_true=y_test, \n",
        "                                     y_pred=y_preds.squeeze()) # use squeeze() to make same shape\n",
        "mae"
      ],
      "execution_count": 56,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=18.745327>"
            ]
          },
          "metadata": {},
          "execution_count": 56
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EwA9nGJzvbMT",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "9a1c0403-36b3-4a89-f0bc-9d36a83dd8fc"
      },
      "source": [
        "# Calculate the MSE\n",
        "mse = tf.metrics.mean_squared_error(y_true=y_test,\n",
        "                                    y_pred=y_preds.squeeze())\n",
        "mse"
      ],
      "execution_count": 57,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=353.57336>"
            ]
          },
          "metadata": {},
          "execution_count": 57
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X06oo_CIjPbL"
      },
      "source": [
        "We can also calculate the MAE using pure TensorFlow functions."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YxiD6-QBYSzd",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "459b3159-56b8-4067-e548-dd4a2677cc9d"
      },
      "source": [
        "# Returns the same as tf.metrics.mean_absolute_error()\n",
        "tf.reduce_mean(tf.abs(y_test-y_preds.squeeze()))"
      ],
      "execution_count": 58,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float64, numpy=18.745327377319335>"
            ]
          },
          "metadata": {},
          "execution_count": 58
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rmEho4lYofOa"
      },
      "source": [
        "Again, it's a good idea to functionize anything you think you might use over again (or find yourself using over and over again).\n",
        "\n",
        "Let's make functions for our evaluation metrics."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bs1Z2jgNol5f"
      },
      "source": [
        "def mae(y_test, y_pred):\n",
        "  \"\"\"\n",
        "  Calculuates mean absolute error between y_test and y_preds.\n",
        "  \"\"\"\n",
        "  return tf.metrics.mean_absolute_error(y_test,\n",
        "                                        y_pred)\n",
        "  \n",
        "def mse(y_test, y_pred):\n",
        "  \"\"\"\n",
        "  Calculates mean squared error between y_test and y_preds.\n",
        "  \"\"\"\n",
        "  return tf.metrics.mean_squared_error(y_test,\n",
        "                                       y_pred)"
      ],
      "execution_count": 59,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zub5zK7bcl40"
      },
      "source": [
        "## Running experiments to improve a model\n",
        "\n",
        "After seeing the evaluation metrics and the predictions your model makes, it's likely you'll want to improve it.\n",
        "\n",
        "Again, there are many different ways you can do this, but 3 of the main ones are:\n",
        "1. **Get more data** - get more examples for your model to train on (more opportunities to learn patterns).\n",
        "2. **Make your model larger (use a more complex model)** - this might come in the form of more layers or more hidden units in each layer.\n",
        "3. **Train for longer** - give your model more of a chance to find the patterns in the data.\n",
        "\n",
        "Since we created our dataset, we could easily make more data but this isn't always the case when you're working with real-world datasets.\n",
        "\n",
        "So let's take a look at how we can improve our model using 2 and 3.\n",
        "\n",
        "To do so, we'll build 3 models and compare their results:\n",
        "1. `model_1` - same as original model, 1 layer, trained for 100 epochs.\n",
        "2. `model_2` - 2 layers, trained for 100 epochs.\n",
        "3. `model_3` - 2 layers, trained for 500 epochs.\n",
        "\n",
        "**Build `model_1`**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "StVHIIM9csyS",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b9c87603-1d3f-4d74-b246-bb9f567d8fa3"
      },
      "source": [
        "# Set random seed\n",
        "tf.random.set_seed(42)\n",
        "\n",
        "# Replicate original model\n",
        "model_1 = tf.keras.Sequential([\n",
        "  tf.keras.layers.Dense(1)\n",
        "])\n",
        "\n",
        "# Compile the model\n",
        "model_1.compile(loss=tf.keras.losses.mae,\n",
        "                optimizer=tf.keras.optimizers.SGD(),\n",
        "                metrics=['mae'])\n",
        "\n",
        "# Fit the model\n",
        "model_1.fit(tf.expand_dims(X_train, axis=-1), y_train, epochs=100)"
      ],
      "execution_count": 63,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/100\n",
            "2/2 [==============================] - 0s 9ms/step - loss: 15.9024 - mae: 15.9024\n",
            "Epoch 2/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 11.2837 - mae: 11.2837\n",
            "Epoch 3/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 11.1074 - mae: 11.1074\n",
            "Epoch 4/100\n",
            "2/2 [==============================] - 0s 7ms/step - loss: 9.2991 - mae: 9.2991\n",
            "Epoch 5/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 10.1677 - mae: 10.1677\n",
            "Epoch 6/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 9.4303 - mae: 9.4303\n",
            "Epoch 7/100\n",
            "2/2 [==============================] - 0s 8ms/step - loss: 8.5704 - mae: 8.5704\n",
            "Epoch 8/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 9.0442 - mae: 9.0442\n",
            "Epoch 9/100\n",
            "2/2 [==============================] - 0s 6ms/step - loss: 18.7517 - mae: 18.7517\n",
            "Epoch 10/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 10.1142 - mae: 10.1142\n",
            "Epoch 11/100\n",
            "2/2 [==============================] - 0s 8ms/step - loss: 8.3980 - mae: 8.3980\n",
            "Epoch 12/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 10.6639 - mae: 10.6639\n",
            "Epoch 13/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 9.7977 - mae: 9.7977\n",
            "Epoch 14/100\n",
            "2/2 [==============================] - 0s 16ms/step - loss: 16.0103 - mae: 16.0103\n",
            "Epoch 15/100\n",
            "2/2 [==============================] - 0s 9ms/step - loss: 11.4068 - mae: 11.4068\n",
            "Epoch 16/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 8.5393 - mae: 8.5393\n",
            "Epoch 17/100\n",
            "2/2 [==============================] - 0s 7ms/step - loss: 13.6348 - mae: 13.6348\n",
            "Epoch 18/100\n",
            "2/2 [==============================] - 0s 7ms/step - loss: 11.4629 - mae: 11.4629\n",
            "Epoch 19/100\n",
            "2/2 [==============================] - 0s 10ms/step - loss: 17.9148 - mae: 17.9148\n",
            "Epoch 20/100\n",
            "2/2 [==============================] - 0s 6ms/step - loss: 15.0494 - mae: 15.0494\n",
            "Epoch 21/100\n",
            "2/2 [==============================] - 0s 6ms/step - loss: 11.0216 - mae: 11.0216\n",
            "Epoch 22/100\n",
            "2/2 [==============================] - 0s 7ms/step - loss: 8.1558 - mae: 8.1558\n",
            "Epoch 23/100\n",
            "2/2 [==============================] - 0s 9ms/step - loss: 9.5138 - mae: 9.5138\n",
            "Epoch 24/100\n",
            "2/2 [==============================] - 0s 18ms/step - loss: 7.6617 - mae: 7.6617\n",
            "Epoch 25/100\n",
            "2/2 [==============================] - 0s 10ms/step - loss: 13.1859 - mae: 13.1859\n",
            "Epoch 26/100\n",
            "2/2 [==============================] - 0s 6ms/step - loss: 16.4211 - mae: 16.4211\n",
            "Epoch 27/100\n",
            "2/2 [==============================] - 0s 6ms/step - loss: 13.1660 - mae: 13.1660\n",
            "Epoch 28/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 14.2559 - mae: 14.2559\n",
            "Epoch 29/100\n",
            "2/2 [==============================] - 0s 6ms/step - loss: 10.0670 - mae: 10.0670\n",
            "Epoch 30/100\n",
            "2/2 [==============================] - 0s 9ms/step - loss: 16.3409 - mae: 16.3409\n",
            "Epoch 31/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 23.6444 - mae: 23.6444\n",
            "Epoch 32/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 7.6215 - mae: 7.6215\n",
            "Epoch 33/100\n",
            "2/2 [==============================] - 0s 11ms/step - loss: 9.3221 - mae: 9.3221\n",
            "Epoch 34/100\n",
            "2/2 [==============================] - 0s 6ms/step - loss: 13.7313 - mae: 13.7313\n",
            "Epoch 35/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 11.1276 - mae: 11.1276\n",
            "Epoch 36/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 13.3222 - mae: 13.3222\n",
            "Epoch 37/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 9.4763 - mae: 9.4763\n",
            "Epoch 38/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 10.1381 - mae: 10.1381\n",
            "Epoch 39/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 10.1793 - mae: 10.1793\n",
            "Epoch 40/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 10.9137 - mae: 10.9137\n",
            "Epoch 41/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 7.9063 - mae: 7.9063\n",
            "Epoch 42/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 10.0914 - mae: 10.0914\n",
            "Epoch 43/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 8.7006 - mae: 8.7006\n",
            "Epoch 44/100\n",
            "2/2 [==============================] - 0s 3ms/step - loss: 12.2047 - mae: 12.2047\n",
            "Epoch 45/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 13.7970 - mae: 13.7970\n",
            "Epoch 46/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 8.4687 - mae: 8.4687\n",
            "Epoch 47/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 9.1330 - mae: 9.1330\n",
            "Epoch 48/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 10.6190 - mae: 10.6190\n",
            "Epoch 49/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 7.7503 - mae: 7.7503\n",
            "Epoch 50/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 9.5407 - mae: 9.5407\n",
            "Epoch 51/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 9.1584 - mae: 9.1584\n",
            "Epoch 52/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 16.3630 - mae: 16.3630\n",
            "Epoch 53/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 14.1299 - mae: 14.1299\n",
            "Epoch 54/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 21.1247 - mae: 21.1247\n",
            "Epoch 55/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 16.3961 - mae: 16.3961\n",
            "Epoch 56/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 9.9806 - mae: 9.9806\n",
            "Epoch 57/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 9.9606 - mae: 9.9606\n",
            "Epoch 58/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 9.2209 - mae: 9.2209\n",
            "Epoch 59/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 8.4239 - mae: 8.4239\n",
            "Epoch 60/100\n",
            "2/2 [==============================] - 0s 7ms/step - loss: 9.4869 - mae: 9.4869\n",
            "Epoch 61/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 11.4355 - mae: 11.4355\n",
            "Epoch 62/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 11.6887 - mae: 11.6887\n",
            "Epoch 63/100\n",
            "2/2 [==============================] - 0s 3ms/step - loss: 7.0838 - mae: 7.0838\n",
            "Epoch 64/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 16.9675 - mae: 16.9675\n",
            "Epoch 65/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 12.4599 - mae: 12.4599\n",
            "Epoch 66/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 13.0184 - mae: 13.0184\n",
            "Epoch 67/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 8.0600 - mae: 8.0600\n",
            "Epoch 68/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 10.1888 - mae: 10.1888\n",
            "Epoch 69/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 12.3633 - mae: 12.3633\n",
            "Epoch 70/100\n",
            "2/2 [==============================] - 0s 10ms/step - loss: 9.0516 - mae: 9.0516\n",
            "Epoch 71/100\n",
            "2/2 [==============================] - 0s 11ms/step - loss: 10.0378 - mae: 10.0378\n",
            "Epoch 72/100\n",
            "2/2 [==============================] - 0s 9ms/step - loss: 10.0516 - mae: 10.0516\n",
            "Epoch 73/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 12.6151 - mae: 12.6151\n",
            "Epoch 74/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 10.3819 - mae: 10.3819\n",
            "Epoch 75/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 9.7229 - mae: 9.7229\n",
            "Epoch 76/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 11.2252 - mae: 11.2252\n",
            "Epoch 77/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 8.3642 - mae: 8.3642\n",
            "Epoch 78/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 9.1274 - mae: 9.1274\n",
            "Epoch 79/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 19.5039 - mae: 19.5039\n",
            "Epoch 80/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 14.8945 - mae: 14.8945\n",
            "Epoch 81/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 9.0034 - mae: 9.0034\n",
            "Epoch 82/100\n",
            "2/2 [==============================] - 0s 7ms/step - loss: 13.0206 - mae: 13.0206\n",
            "Epoch 83/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 7.9299 - mae: 7.9299\n",
            "Epoch 84/100\n",
            "2/2 [==============================] - 0s 3ms/step - loss: 7.6872 - mae: 7.6872\n",
            "Epoch 85/100\n",
            "2/2 [==============================] - 0s 6ms/step - loss: 10.0328 - mae: 10.0328\n",
            "Epoch 86/100\n",
            "2/2 [==============================] - 0s 13ms/step - loss: 9.2433 - mae: 9.2433\n",
            "Epoch 87/100\n",
            "2/2 [==============================] - 0s 6ms/step - loss: 12.0209 - mae: 12.0209\n",
            "Epoch 88/100\n",
            "2/2 [==============================] - 0s 25ms/step - loss: 10.6389 - mae: 10.6389\n",
            "Epoch 89/100\n",
            "2/2 [==============================] - 0s 13ms/step - loss: 7.2667 - mae: 7.2667\n",
            "Epoch 90/100\n",
            "2/2 [==============================] - 0s 7ms/step - loss: 12.7786 - mae: 12.7786\n",
            "Epoch 91/100\n",
            "2/2 [==============================] - 0s 7ms/step - loss: 7.3481 - mae: 7.3481\n",
            "Epoch 92/100\n",
            "2/2 [==============================] - 0s 6ms/step - loss: 7.7175 - mae: 7.7175\n",
            "Epoch 93/100\n",
            "2/2 [==============================] - 0s 8ms/step - loss: 7.1263 - mae: 7.1263\n",
            "Epoch 94/100\n",
            "2/2 [==============================] - 0s 10ms/step - loss: 12.6190 - mae: 12.6190\n",
            "Epoch 95/100\n",
            "2/2 [==============================] - 0s 9ms/step - loss: 10.0912 - mae: 10.0912\n",
            "Epoch 96/100\n",
            "2/2 [==============================] - 0s 5ms/step - loss: 9.3558 - mae: 9.3558\n",
            "Epoch 97/100\n",
            "2/2 [==============================] - 0s 18ms/step - loss: 12.6834 - mae: 12.6834\n",
            "Epoch 98/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 8.6762 - mae: 8.6762\n",
            "Epoch 99/100\n",
            "2/2 [==============================] - 0s 9ms/step - loss: 9.4693 - mae: 9.4693\n",
            "Epoch 100/100\n",
            "2/2 [==============================] - 0s 4ms/step - loss: 8.7067 - mae: 8.7067\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f8e751f5a10>"
            ]
          },
          "metadata": {},
          "execution_count": 63
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "A-Da56xspOrY",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 465
        },
        "outputId": "97967485-83f2-4463-b2e1-acfac187ac3e"
      },
      "source": [
        "# Make and plot predictions for model_1\n",
        "y_preds_1 = model_1.predict(X_test)\n",
        "plot_predictions(predictions=y_preds_1)"
      ],
      "execution_count": 64,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "WARNING:tensorflow:6 out of the last 6 calls to <function Model.make_predict_function.<locals>.predict_function at 0x7f8df63b8f80> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAGbCAYAAAAY8u5bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3RV9Z338c+XiyDCE2/xBiWBVuWiGCDFC6PCgEq1VnHVVhtHfWyLWC3qLEermVrsszJLO7byaB+lccZRu1KLj9aqLToK6mCHOjRoHgggxUuiWAZTnEacqNy+zx/n5HgIJ+Eczj6Xvff7tVZWztnnsn/nEvz423t/trm7AAAAEJx+pR4AAABA1BCwAAAAAkbAAgAACBgBCwAAIGAELAAAgIANKPUA0h166KFeXV1d6mEAAADs1cqVK//s7pWZbiurgFVdXa3m5uZSDwMAAGCvzKy9t9vYRAgAABAwAhYAAEDACFgAAAABK6t9sDLZvn27Nm7cqE8++aTUQ0HS4MGDNWLECA0cOLDUQwEAoCyVfcDauHGjhg0bpurqaplZqYcTe+6uLVu2aOPGjRo1alSphwMAQFkq+02En3zyiQ455BDCVZkwMx1yyCHMKAIA0IeyD1iSCFdlhs8DAIC+hSJgAQAAhAkBay+2bNmimpoa1dTU6IgjjtDw4cNT17dt29bnY5ubmzVv3ry9ruOUU04Jari7mTZt2l6LWxcsWKCurq6CrB8AgLgq+53cS+2QQw5RS0uLJGn+/PkaOnSobrjhhtTtO3bs0IABmd/G2tpa1dbW7nUdy5cvD2aw+2DBggW65JJLNGTIkJKNAQCAqIncDFZTk1RdLfXrl/jd1BT8Oi6//HLNnTtXJ554om688UatWLFCJ598siZOnKhTTjlF69evlyS99NJL+vKXvywpEc6uuOIKTZs2TaNHj9bdd9+der6hQ4em7j9t2jR99atf1ZgxY1RXVyd3lyQtXrxYY8aM0eTJkzVv3rzU86b7+OOPddFFF2ns2LGaPXu2Pv7449RtV111lWprazV+/Hj94Ac/kCTdfffd+tOf/qTp06dr+vTpvd4PAADkJlIzWE1N0pw5UvcWr/b2xHVJqqsLdl0bN27U8uXL1b9/f3344Yd6+eWXNWDAAC1ZskS33HKLHn/88T0e8/rrr+vFF1/U1q1bdeyxx+qqq67ao0vqtdde05o1a3TUUUdp6tSp+vd//3fV1tbqyiuv1LJlyzRq1ChdfPHFGcd03333aciQIVq3bp1WrVqlSZMmpW5raGjQwQcfrJ07d2rGjBlatWqV5s2bp5/85Cd68cUXdeihh/Z6vwkTJgT4zgEAEH2RmsGqr/8sXHXr6kosD9qFF16o/v37S5I6Ozt14YUX6rjjjtP111+vNWvWZHzMOeeco0GDBunQQw/VYYcdps2bN+9xnylTpmjEiBHq16+fampq1NbWptdff12jR49O9U71FrCWLVumSy65RJI0YcKE3YLRo48+qkmTJmnixIlas2aN1q5dm/E5sr0fAADoXaQC1jvv5LY8HwcccEDq8ve//31Nnz5dra2tevrpp3vtiBo0aFDqcv/+/bVjx459uk+u3n77bd15551aunSpVq1apXPOOSfjGLO9HwAA5appdZOqF1Sr3239VL2gWk2rC7CvUBYiFbBGjsxteVA6Ozs1fPhwSdKDDz4Y+PMfe+yxeuutt9TW1iZJWrRoUcb7nXbaafrFL34hSWptbdWqVaskSR9++KEOOOAAVVRUaPPmzXrmmWdSjxk2bJi2bt261/sBAFDumlY3ac7Tc9Te2S6Xq72zXXOenlOSkBWpgNXQIPU8GG7IkMTyQrrxxht18803a+LEiYHMOPW0//77695779WsWbM0efJkDRs2TBUVFXvc76qrrtJHH32ksWPH6tZbb9XkyZMlSSeccIImTpyoMWPG6Bvf+IamTp2aesycOXM0a9YsTZ8+vc/7AQBQ7uqX1qtr++77CnVt71L90gLsK7QX1n2UWjmora31nr1N69at09ixY7N+jqamxD5X77yTmLlqaAh+B/dS+OijjzR06FC5u66++modffTRuv7660s2nlw/FwAACq3fbf3k2jPXmEy7frAr8PWZ2Up3z9jHFKkZLCkRptrapF27Er+jEK4k6f7771dNTY3Gjx+vzs5OXXnllaUeEgAAZWVkReZ9gnpbXkiRC1hRdf3116ulpUVr165VU1MTxaAAAPTQMKNBQwbu/t/HIQOHqGFGgfcVyoCABQAAIqHu+Do1ntuoqooqmUxVFVVqPLdRdccXf3NWpIpGAQBANDWtblL90nq90/mORlaMVMOMhozBqe74upIEqp4IWAAAoKx11y90HyHYXb8gqSzCVCZsIgQAAGWtnOoXspVTwDKzB8zsfTNrTVt2sJk9b2Ybkr8PSi43M7vbzN4ws1VmNqn3Zy5fW7ZsUU1NjWpqanTEEUdo+PDhqevbtm3b6+NfeuklLV++PHV94cKFevjhhwMfZ/qJpXvT0tKixYsXB75uAAAK6Z3OzKdk6W15Och1ButBSbN6LPuepKXufrSkpcnrkvQlSUcnf+ZIum/fh1k6hxxyiFpaWtTS0qK5c+emjuZraWnRfvvtt9fH9wxYc+fO1aWXXlrIIfeKgAUACKNyql/IVk4By92XSfqgx+LzJD2UvPyQpPPTlj/sCa9IOtDMjsxnsNkoxjmIVq5cqdNPP12TJ0/WWWedpU2bNkmS7r77bo0bN04TJkzQRRddpLa2Ni1cuFB33XWXampq9PLLL2v+/Pm68847JUnTpk3TTTfdpClTpuiYY47Ryy+/LEnq6urS1772NY0bN06zZ8/WiSeeqJ4FrJL07LPPasyYMZo0aZJ+9atfpZavWLFCJ598siZOnKhTTjlF69ev17Zt23Trrbdq0aJFqqmp0aJFizLeDwCAclNO9QvZCmIn98PdfVPy8n9KOjx5ebikd9PutzG5bFPaMpnZHCVmuDQyz5MGFmMnOHfXd7/7XT355JOqrKzUokWLVF9frwceeEC333673n77bQ0aNEh/+ctfdOCBB2ru3LkaOnSobrjhBknS0qVLd3u+HTt2aMWKFVq8eLFuu+02LVmyRPfee68OOuggrV27Vq2traqpqdljHJ988om+/e1v64UXXtAXvvAFff3rX0/dNmbMGL388ssaMGCAlixZoltuuUWPP/64fvjDH6q5uVk//elPJSXOPZjpfgAAlJPu/4ZncxRhuQj0KEJ3dzPL6dw77t4oqVFKnConn/X3tRNcUB/Cp59+qtbWVp1xxhmSpJ07d+rIIxMTcxMmTFBdXZ3OP/98nX/++X09TcoFF1wgSZo8eXLqZM6/+93vdO2110qSjjvuOE2YMGGPx73++usaNWqUjj76aEnSJZdcosbGRkmJk09fdtll2rBhg8xM27dvz7jubO8HAEAhZFu9IJVP/UK2gjiKcHP3pr/k7/eTy9+T9Lm0+41ILiuYYuwE5+4aP358aj+s1atX67nnnpMk/fa3v9XVV1+tV199VV/84hezOvHzoEGDJEn9+/cP7ETR3//+9zV9+nS1trbq6aef1ieffJLX/QAACFr3Vqf2zna5PLXVqRC79pRCEAHrKUmXJS9fJunJtOWXJo8mPElSZ9qmxIIoxk5wgwYNUkdHh37/+99LkrZv3641a9Zo165devfddzV9+nTdcccd6uzs1EcffaRhw4Zp69atOa1j6tSpevTRRyVJa9eu1erVq/e4z5gxY9TW1qY333xTkvTII4+kbuvs7NTw4cMlSQ8++GBqec+x9HY/AAAKLYzVC7nItabhEUm/l3SsmW00s29Kul3SGWa2QdLM5HVJWizpLUlvSLpf0ncCG3UvirETXL9+/fTYY4/ppptu0gknnKCamhotX75cO3fu1CWXXKLjjz9eEydO1Lx583TggQfq3HPP1RNPPJHayT0b3/nOd9TR0aFx48bp7//+7zV+/HhVVFTsdp/BgwersbFR55xzjiZNmqTDDjssdduNN96om2++WRMnTtxtVmz69Olau3Ztaif33u4HAEChhbF6IRfmntduT4Gqra31nkfLrVu3TmPHjs36OXLZnluudu7cqe3bt2vw4MF68803NXPmTK1fvz6rWohiyfVzAQAgXfWCarV3tu+xvKqiSm3XtRV/QPvAzFa6e22m2yJ3qpyw7QSXSVdXl6ZPn67t27fL3XXvvfeWVbgCACBfDTMadjvyXyr/6oVcRC5gRcGwYcMy9l4BABAVYaxeyAUBCwAABCrb3XWisNWpNwQsAAAQmGKUfodBEDUNAAAAkqJfv5AtAhYAAAhM1OsXskXAykL//v1VU1Oj4447ThdeeKG6urr2/qBeXH755XrsscckSd/61re0du3aXu/70ksvafny5anrCxcu1MMPP7zP6wYAoNCKUfodBgSsLOy///5qaWlRa2ur9ttvPy1cuHC32/e1pPOf/umfNG7cuF5v7xmw5s6dq0svvXSf1gUAQDEUo/Q7DKIXsJqapOpqqV+/xO+mYM9pdOqpp+qNN97QSy+9pFNPPVVf+cpXNG7cOO3cuVN/93d/py9+8YuaMGGCfvazn0lKnLvwmmuu0bHHHquZM2fq/fffTz3XtGnTUnUMzz77rCZNmqQTTjhBM2bMUFtbmxYuXKi77ror1QI/f/583XnnnZKklpYWnXTSSZowYYJmz56t//qv/0o950033aQpU6bomGOOSbXHr1mzRlOmTFFNTY0mTJigDRs2BPq+AAAgJXZkbzy3UVUVVTKZqiqq1HhuY6x2cJeidhRhU5M0Z47UvQmvvT1xXZLq8v9gd+zYoWeeeUazZs2SJL366qtqbW3VqFGj1NjYqIqKCv3hD3/Qp59+qqlTp+rMM8/Ua6+9pvXr12vt2rXavHmzxo0bpyuuuGK35+3o6NC3v/1tLVu2TKNGjdIHH3yggw8+WHPnztXQoUN1ww03SJKWLl2aesyll16qe+65R6effrpuvfVW3XbbbVqwYEFqnCtWrNDixYt12223acmSJVq4cKGuvfZa1dXVadu2bdq5c2fe7wcAIF6oX8hetGaw6us/C1fduroSy/Pw8ccfq6amRrW1tRo5cqS++c1vSpKmTJmiUaNGSZKee+45Pfzww6qpqdGJJ56oLVu2aMOGDVq2bJkuvvhi9e/fX0cddZT++q//eo/nf+WVV3Taaaelnuvggw/uczydnZ36y1/+otNPP12SdNlll2nZsmWp2y+44AJJ0uTJk9XW1iZJOvnkk/UP//APuuOOO9Te3q79998/r/cEABAv3fUL7Z3tcnmqfqFpdbBbiqIiWgHrnV6OUOhteZa698FqaWnRPffckzptzQEHHJC6j7vrnnvuSd3v7bff1plnnpnXevfVoEGDJCV2zu/eP+wb3/iGnnrqKe2///46++yz9cILL5RkbACAcKJ+ITfRClgjezlCobflATrrrLN03333afv27ZKkP/7xj/rv//5vnXbaaVq0aJF27typTZs26cUXX9zjsSeddJKWLVumt99+W5L0wQcfSEqcMmfr1q173L+iokIHHXRQav+qn//856nZrN689dZbGj16tObNm6fzzjtPq1atyuv1AgDihfqF3ERrH6yGht33wZKkIUMSywvsW9/6ltra2jRp0iS5uyorK/XrX/9as2fP1gsvvKBx48Zp5MiROvnkk/d4bGVlpRobG3XBBRdo165dOuyww/T888/r3HPP1Ve/+lU9+eSTuueee3Z7zEMPPaS5c+eqq6tLo0eP1r/8y7/0Ob5HH31UP//5zzVw4EAdccQRuuWWWwJ9/QCAaBtZMVLtne0Zl2NP5u6lHkNKbW2t9zzJ8bp16zR27Njsn6SpKbHP1TvvJGauGhoC2cEdu8v5cwEAhFrPU+BIifqFOB4h2M3MVrp7babbojWDJSXCFIEKAIBAdYeobI4iRBQDFgAAyFq21QsS9Qu5CEXAcneZWamHgaRy2qwMANh3PTf7dVcvSCJI5ansjyIcPHiwtmzZwn/Uy4S7a8uWLRo8eHCphwIAyFMkqxcKfEaXbJX9DNaIESO0ceNGdXR0lHooSBo8eLBGjBhR6mEAAPIUueqFAp/RJRdlH7AGDhyYajgHAADBiVz1Ql9ndClywCr7TYQAAKAwGmY0aMjAIbstGzJwiBpmFL4/siAKdEaXfUHAAgAgpuqOr1PjuY2qqqiSyVRVURXuXqsSntGlJwIWAAAR1LS6SdULqtXvtn6qXlDd60mZ646vU9t1bdr1g11qu64tvOFKSpSLD9l9Rq5YZ3TpiYAFAEDEdNcvtHe2y+Wp+oXeQlYoZHN0YF2d1NgoVVVJZonfjY0lKSAv+1PlAACA3FQvqM6483pVRZXarmsr/oDy1fPoQCkxM1Wi8NStr1PlMIMFAEDERK5+oa+jA8sUAQsAgIjprWYhtPULZXR0YLYIWAAAREzk6hfK6OjAbBGwAACImMjVL5TR0YHZImABABAS2VYvSCGpX8j2vIFldHRgtjiKEACAEOiuXkg/OfOQgUPCOzNVpkcG5qKvowgJWAAAhEDkqheqqxMnY+6pqkpqayv2aPYJNQ0AAIRc5KoXQnhkYC4IWAAAhEDkqhdCeGRgLvIOWGZ2rJm1pP18aGbXmdl8M3svbfnZQQwYAIA4ilz1QgiPDMxF3gHL3de7e42710iaLKlL0hPJm+/qvs3dF+e7LgAA4ipU1QshO29gIQS6k7uZnSnpB+4+1czmS/rI3e/M9vHs5A4AiKOm1U2qX1qvdzrf0ciKkWqY0VCewSkbETg6MFvF3Mn9IkmPpF2/xsxWmdkDZnZQL4ObY2bNZtbc0dER8HAAAChv3fUL7Z3tcrnaO9s15+k5fXZclbUQnjewEAKbwTKz/ST9SdJ4d99sZodL+rMkl/S/JB3p7lf09RzMYAEA4iZy9Qv9+kmZsoWZtGtX8cdTQMWawfqSpFfdfbMkuftmd9/p7rsk3S9pSoDrAgAgEiJXvxDxowOzFWTAulhpmwfN7Mi022ZLag1wXQAARELk6hcifnRgtgIJWGZ2gKQzJP0qbfGPzGy1ma2SNF3S9UGsCwCAKAlV/QJHB2aNU+UAAFBioTiKMEZHB2aLcxECAFACoQhO2YrAuQOD1lfAGlDswQAAEAfd9Qtd2xMzPt31C5LCGbIifu7AoHEuQgAACqB+aX0qXHXr2t6l+qUh7YPi6MCcELAAACiAyNUvcHRgTghYAAAUQOTqFzg6MCcELAAACiA09QvZVC90q6tL7NC+a1fiN+GqVwQsAAAKoO74OjWe26iqiiqZTFUVVWo8t7G8dnDvrl5ob0+c3qa9PXG9r5CFrFDTAABADpqaEuctfuedxP7dDQ0hnsiheiEv1DQAABCAnl2b3RM+UkhDFtULBcMmQgAAslRfv3uRuZS4Xh/S5gWqFwqHgAUAQJYiN+FD9ULBELAAAMhSqCZ8ODFzSRGwAADIUmgmfHI5OpDqhYIgYAEAkKXQTPhEbmex8CFgAQCg7Ps2QzHhE7mdxcKHgAUAiL3I9W2GamexaCJgAQBiL3Jb1EKzs1h0EbAAALEXmi1quWzHDMXOYtFFkzsAIPZGjsx8xpiy2qKWa418XR2BqoSYwQIAxF4otqhFbjtmtBGwAACxF4otaqHZjgmJgAUAiLjI1C9wZGCoELAAAJEVqfqFUGzHRDcCFgAgskKz2xLnDYwcc/dSjyGltrbWm5ubSz0MAEBE9OuXmLnqySyxKbAs9Dw6UErMTBGeyp6ZrXT32ky3MYMFAIisUOy2FJppNuSCgAUAiKxQ7LbE0YGRRMACAERWKHZbCsU0G3JFwAIAhE621QtSCOoXQjHNhlwRsAAAoRKq6gWODowtjiIEAIRKdXXm8wZWVSVmqMoGRwdGHkcRAgAiIzT7hHN0YKwRsAAAoRKafcJDkwRRCAQsAECohGaf8NAkQRQCAQsAECqh2Sc8NEkQhRBYwDKzNjNbbWYtZtacXHawmT1vZhuSvw8Kan0AgOjJtn6h7KsXpBAlQRRCYEcRmlmbpFp3/3Pash9J+sDdbzez70k6yN1v6u05OIoQAOKLg+4QNqU8ivA8SQ8lLz8k6fwCrw8AEFIcdIcoCTJguaTnzGylmc1JLjvc3TclL/+npMN7PsjM5phZs5k1d3R0BDgcAECYcNAdoiTIgPVX7j5J0pckXW1mp6Xf6IltkXtsj3T3RnevdffaysrKAIcDAAgTDrpDlAQWsNz9veTv9yU9IWmKpM1mdqQkJX+/H9T6AADRwkF3iJJAApaZHWBmw7ovSzpTUqukpyRdlrzbZZKeDGJ9AIDo4aA7RElQM1iHS/qdmf0/SSsk/dbdn5V0u6QzzGyDpJnJ6wCAmIlU/QKQhQFBPIm7vyXphAzLt0iaEcQ6AADh1LN+ob09cV0iQCG6aHIHABQU9QuIIwIWAKCgqF9AHBGwAAAFRf0C4oiABQAoKOoXEEcELABAQVG/gDgK5ChCAAD6UldHoEK8MIMFANgn2XZbAXHEDBYAIGd0WwF9YwYLAJAzuq2AvhGwAAA5o9sK6BsBCwCQM7qtgL4RsAAAOaPbCugbAQsAkDO6rYC+EbAAALvJtn6hrk5qa5N27Ur8JlwBn6GmAQCQQv0CEAxmsAAAKdQvAMEgYAEAUqhfAIJBwAIApFC/AASDgAUASKF+AQgGAQsAkEL9AhAMAhYAxAT1C0DxUNMAADFA/QJQXMxgAUAMUL8AFBcBCwBigPoFoLgIWAAQA9QvAMVFwAKAGKB+ASguAhYAxAD1C0BxEbAAIMSyrV6QqF8AiomaBgAIKaoXgPLFDBYAhBTVC0D5ImABQEhRvQCULwIWAIQU1QtA+SJgAUBIUb0AlC8CFgCEFNULQPkiYAFAGcq2foHqBaA85R2wzOxzZvaima01szVmdm1y+Xwze8/MWpI/Z+c/XACIvu76hfZ2yf2z+oW+Oq4AlBdz9/yewOxISUe6+6tmNkzSSknnS/qapI/c/c5sn6u2ttabm5vzGg8AhF11dSJU9VRVlZilAlAezGylu9dmui3volF33yRpU/LyVjNbJ2l4vs8LAHFF/QIQfoHug2Vm1ZImSvqP5KJrzGyVmT1gZgcFuS4AiCrqF4DwCyxgmdlQSY9Lus7dP5R0n6TPS6pRYobrx708bo6ZNZtZc0dHR1DDAYDQon4BCL9AApaZDVQiXDW5+68kyd03u/tOd98l6X5JUzI91t0b3b3W3WsrKyuDGA4AhBr1C0D4BXEUoUn6Z0nr3P0nacuPTLvbbEmt+a4LAMKO+gUgHvLeyV3SVEl/I2m1mbUkl90i6WIzq5HkktokXRnAugAgtLrrF7pP0NxdvyARoICoybumIUjUNACIMuoXgGjpq6aBJncAKBLqF4D4IGABQJFQvwDEBwELAIqE+gUgPghYAFAk1C8A8UHAAoA8ZVu9IFG/AMRFEDUNABBbVC8AyIQZLADIQ339Z+GqW1dXYjmA+CJgAUAeqF4AkAkBCwDyQPUCgEwIWACQB6oXAGRCwAKAPFC9ACATAhYA9CLb+gWqFwD0RE0DAGRA/QKAfDCDBQAZUL8AIB8ELADIgPoFAPkgYAFABtQvAMgHAQsAMqB+AUA+CFgAkAH1CwDyQcACEDvULwAoNGoaAMQK9QsAioEZLACxQv0CgGIgYAGIFeoXABQDAQtArFC/AKAYCFgAYoX6BQDFQMACECvULwAoBgIWgEjItnpBon4BQOFR0wAg9KheAFBumMECEHpULwAoNwQsAKFH9QKAckPAAhB6VC8AKDcELAChR/UCgHJDwAIQelQvACg3BCwAZS3b+gWqFwCUE2oaAJQt6hcAhBUzWADKFvULAMKKgAWgbFG/ACCsCh6wzGyWma03szfM7HuFXh+A6KB+AUBYFTRgmVl/Sf9H0pckjZN0sZmNK+Q6AUQH9QsAwqrQM1hTJL3h7m+5+zZJv5R0XoHXCSAiqF8AEFaFDljDJb2bdn1jclmKmc0xs2Yza+7o6CjwcACUg2yrFyTqFwCEU8l3cnf3RnevdffaysrKUg8HQIF1Vy+0t0vun1Uv9BWyACBsCh2w3pP0ubTrI5LLAMQU1QsA4qDQAesPko42s1Fmtp+kiyQ9VeB1AihjVC8AiIOCBix33yHpGkn/KmmdpEfdfU0h1wmgvFG9ACAOCr4Plrsvdvdj3P3z7s7B1UDMUb0AIA5KvpM7gHihegFAHBCwAAQm2/oFqhcARN2AUg8AQDR01y90HyHYXb8gEaAAxA8zWAACQf0CAHyGgAUgENQvAMBnCFgAAkH9AgB8hoAFIBDULwDAZwhYAAJB/QIAfIaABWCvqF8AgNxQ0wCgT9QvAEDumMEC0CfqFwAgdwQsAH2ifgEAckfAAtAn6hcAIHcELAB9on4BAHJHwALQJ+oXACB3BCwgprKtXpCoXwCAXFHTAMQQ1QsAUFjMYAExRPUCABQWAQuIIaoXAKCwCFhADFG9AACFRcACYojqBQAoLAIWEENULwBAYRGwgIjJtn6B6gUAKBxqGoAIoX4BAMoDM1hAhFC/AADlgYAFRAj1CwBQHghYQIRQvwAA5YGABUQI9QsAUB4IWECEUL8AAOWBgAWEBPULABAe1DQAIUD9AgCECzNYQAhQvwAA4ULAAkKA+gUACBcCFhAC1C8AQLgQsIAQoH4BAMIlr4BlZv9oZq+b2Soze8LMDkwurzazj82sJfmzMJjhAvFE/QIAhIu5+74/2OxMSS+4+w4zu0OS3P0mM6uW9Bt3Py6X56utrfXm5uZ9Hg8AAECxmNlKd6/NdFteM1ju/py770hefUXSiHyeD4ibbLutAADhEuQ+WFdIeibt+igze83M/s3MTu3tQWY2x8yazay5o6MjwOEA5a2726q9XXL/rNuKkAUA4bfXTYRmtkTSERluqnf3J5P3qZdUK+kCd3czGyRpqLtvMbPJkn4taby7f9jXuthEiDiprk6Eqp6qqhIN7ACA8tbXJsK9Nrm7+8y9PPnlkr4saYYn05q7fyrp0+TllWb2pqRjJJGegCS6rQAguvI9inCWpBslfcXdu9KWV5pZ/+Tl0ZKOlvRWPusConbr/cQAAA0BSURBVIZuKwCIrnz3wfqppGGSnu9Rx3CapFVm1iLpMUlz3f2DPNcFRArdVgAQXXmd7Nndv9DL8sclPZ7PcwNR191hVV+f2Cw4cmQiXNFtBQDhR5M7UADZ1i/U1SV2aN+1K/GbcAUA0ZDXDBaAPXXXL3Ql90rsrl+QCFAAEBfMYAEBq6//LFx16+pKLAcAxAMBCwgY9QsAAAIWEDDqFwAABCwgYNQvAAAIWEDA6uqkxsbEKW/MEr8bG9nBHQDihIAF5ID6BQBANqhpALJE/QIAIFvMYAFZon4BAJAtAhaQJeoXAADZImABWaJ+AQCQLQIWkCXqFwAA2SJgAVmifgEAkC0CFmIv2+oFifoFAEB2qGlArFG9AAAoBGawEGtULwAACoGAhVijegEAUAgELMQa1QsAgEIgYCHWqF4AABQCAQuxRvUCAKAQCFiIrGzrF6heAAAEjZoGRBL1CwCAUmIGC5FE/QIAoJQIWIgk6hcAAKVEwEIkUb8AACglAhYiifoFAEApEbAQSdQvAABKiYCF0KF+AQBQ7qhpQKhQvwAACANmsBAq1C8AAMKAgIVQoX4BABAGBCyECvULAIAwIGAhVKhfAACEAQELoUL9AgAgDPIKWGY238zeM7OW5M/ZabfdbGZvmNl6Mzsr/6EiyrKtXpCoXwAAlL8gahrucvc70xeY2ThJF0kaL+koSUvM7Bh33xnA+hAxVC8AAKKmUJsIz5P0S3f/1N3flvSGpCkFWhdCjuoFAEDUBBGwrjGzVWb2gJkdlFw2XNK7affZmFy2BzObY2bNZtbc0dERwHAQNlQvAACiZq8By8yWmFlrhp/zJN0n6fOSaiRtkvTjXAfg7o3uXuvutZWVlTm/AIQf1QsAgKjZ6z5Y7j4zmycys/sl/SZ59T1Jn0u7eURyGbCHhobd98GSqF4AAIRbvkcRHpl2dbak1uTlpyRdZGaDzGyUpKMlrchnXYguqhcAAFGT7z5YPzKz1Wa2StJ0SddLkruvkfSopLWSnpV0NUcQxlO29QtULwAAoiSvmgZ3/5s+bmuQxEaeGKN+AQAQVzS5o2CoXwAAxBUBCwVD/QIAIK4IWCgY6hcAAHFFwELBNDQk6hbSUb8AAIgDAhYKhvoFAEBcEbCwT6hfAACgd3nVNCCeqF8AAKBvzGAhZ9QvAADQNwIWckb9AgAAfSNgIWfULwAA0DcCFnJG/QIAAH0jYCFn1C8AANA3AhZSsq1ekKhfAACgL9Q0QBLVCwAABIkZLEiiegEAgCARsCCJ6gUAAIJEwIIkqhcAAAgSAQuSqF4AACBIBCxIonoBAIAgEbBiINv6BaoXAAAIBjUNEUf9AgAAxccMVsRRvwAAQPERsCKO+gUAAIqPgBVx1C8AAFB8BKyIo34BAIDiI2BFHPULAAAUHwErpLKtXpCoXwAAoNioaQghqhcAAChvzGCFENULAACUNwJWCFG9AABAeSNghRDVCwAAlDcCVghRvQAAQHkjYIUQ1QsAAJQ3AlaZybZ+geoFAADKFzUNZYT6BQAAoiGvGSwzW2RmLcmfNjNrSS6vNrOP025bGMxwo436BQAAoiGvGSx3/3r3ZTP7saTOtJvfdPeafJ4/bqhfAAAgGgLZB8vMTNLXJD0SxPPFFfULAABEQ1A7uZ8qabO7b0hbNsrMXjOzfzOzU3t7oJnNMbNmM2vu6OgIaDjhRP0CAADRsNeAZWZLzKw1w895aXe7WLvPXm2SNNLdJ0r6W0m/MLP/ken53b3R3WvdvbaysjKf1xJ61C8AABANew1Y7j7T3Y/L8POkJJnZAEkXSFqU9phP3X1L8vJKSW9KOqYwLyEcqF8AACA+gqhpmCnpdXff2L3AzColfeDuO81stKSjJb0VwLpCifoFAADiJYh9sC7Snju3nyZpVbK24TFJc939gwDWFUrULwAAEC95z2C5++UZlj0u6fF8nzsqqF8AACBeOFVOEVC/AABAvBCwioD6BQAA4oWAVQTULwAAEC8ErDxkW70gUb8AAECcBFHTEEtULwAAgN4wg7WPqF4AAAC9IWDtI6oXAABAbwhY+4jqBQAA0BsC1j6iegEAAPSGgLWPqF4AAAC9IWBlkG39AtULAAAgE2oaeqB+AQAA5IsZrB6oXwAAAPkiYPVA/QIAAMgXAasH6hcAAEC+CFg9UL8AAADyRcDqgfoFAACQL44izKCujkAFAAD2XaxmsLLttwIAAMhHbGaw6LcCAADFEpsZLPqtAABAscQmYNFvBQAAiiU2AYt+KwAAUCyxCVj0WwEAgGKJTcCi3woAABRLbI4ilOi3AgAAxRGbGSwAAIBiIWABAAAEjIAFAAAQMAIWAABAwAhYAAAAASNgAQAABIyABQAAEDACFgAAQMAIWAAAAAEjYAEAAASMgAUAABAwAhYAAEDAzN1LPYYUM+uQ1F6EVR0q6c9FWE+5ivvrl3gPJN4Difcg7q9f4j2QeA/yef1V7l6Z6YayCljFYmbN7l5b6nGUStxfv8R7IPEeSLwHcX/9Eu+BxHtQqNfPJkIAAICAEbAAAAACFteA1VjqAZRY3F+/xHsg8R5IvAdxf/0S74HEe1CQ1x/LfbAAAAAKKa4zWAAAAAVDwAIAAAhYpAOWmV1oZmvMbJeZ1fa47WYze8PM1pvZWWnLZyWXvWFm3yv+qAvHzBaZWUvyp83MWpLLq83s47TbFpZ6rIViZvPN7L2013p22m0ZvxNRYmb/aGavm9kqM3vCzA5MLo/Nd0CK9t95b8zsc2b2opmtTf67eG1yea9/E1GT/HdvdfJ1NieXHWxmz5vZhuTvg0o9zkIxs2PTPucWM/vQzK6L+nfAzB4ws/fNrDVtWcbP3RLuTv7bsMrMJu3zeqO8D5aZjZW0S9LPJN3g7t1/UOMkPSJpiqSjJC2RdEzyYX+UdIakjZL+IOlid19b5KEXnJn9WFKnu//QzKol/cbdjyvtqArPzOZL+sjd7+yxPON3wt13Fn2QBWRmZ0p6wd13mNkdkuTuN8XsO9BfMfk7T2dmR0o60t1fNbNhklZKOl/S15ThbyKKzKxNUq27/zlt2Y8kfeDutyfD9kHuflOpxlgsyb+D9ySdKOl/KsLfATM7TdJHkh7u/jeut889GS6/K+lsJd6b/+3uJ+7LeiM9g+Xu69x9fYabzpP0S3f/1N3flvSGEv9hnSLpDXd/y923Sfpl8r6RYmamxD+qj5R6LGWkt+9EpLj7c+6+I3n1FUkjSjmeEonF33lP7r7J3V9NXt4qaZ2k4aUdVVk4T9JDycsPKRE642CGpDfdvRhnTykpd18m6YMei3v73M9TIoi5u78i6cDk/5zkLNIBqw/DJb2bdn1jcllvy6PmVEmb3X1D2rJRZvaamf2bmZ1aqoEVyTXJqd8H0jYHxOWzT3eFpGfSrsflOxDHz3o3yRnLiZL+I7ko099EFLmk58xspZnNSS473N03JS//p6TDSzO0ortIu/9Pdly+A916+9wD+/ch9AHLzJaYWWuGn8j/H2kmWb4fF2v3P6xNkka6+0RJfyvpF2b2P4o57iDt5T24T9LnJdUo8bp/XNLBFkA23wEzq5e0Q1JTclGkvgPonZkNlfS4pOvc/UPF4G8izV+5+yRJX5J0dXLTUYon9pmJ7n4zSWa2n6SvSPq/yUVx+g7soVCf+4Cgn7DY3H3mPjzsPUmfS7s+IrlMfSwPhb29H2Y2QNIFkianPeZTSZ8mL680szeV2CetuYBDLZhsvxNmdr+k3ySv9vWdCJUsvgOXS/qypBnJf1gi9x3Yi8h81rkys4FKhKsmd/+VJLn75rTb0/8mIsfd30v+ft/MnlBic/FmMzvS3TclNwW9X9JBFseXJL3a/dnH6TuQprfPPbB/H0I/g7WPnpJ0kZkNMrNRko6WtEKJnV2PNrNRyYR/UfK+UTJT0uvuvrF7gZlVJnd4lJmNVuL9eKtE4yuoHtvSZ0vqPqqkt+9EpJjZLEk3SvqKu3elLY/Nd0Dx+DvfQ3Lfy3+WtM7df5K2vLe/iUgxswOSO/fLzA6QdKYSr/UpSZcl73aZpCdLM8Ki2m0rRly+Az309rk/JenS5NGEJylxMNimTE+wN6GfweqLmc2WdI+kSkm/NbMWdz/L3deY2aOS1iqxmeTq7qPFzOwaSf8qqb+kB9x9TYmGXyg9t7tL0mmSfmhm25U46nKuu/fcITAqfmRmNUpMB7dJulKS+vpORMxPJQ2S9Hziv7d6xd3nKkbfgeQRlFH/O89kqqS/kbTakhUtkm6RdHGmv4kIOlzSE8nv/QBJv3D3Z83sD5IeNbNvSmpX4gCgyEqGyzO0++ec8d/FqDCzRyRNk3SomW2U9ANJtyvz575YiSMI35DUpcQRlvu23ijXNAAAAJRCXDcRAgAAFAwBCwAAIGAELAAAgIARsAAAAAJGwAIAAAgYAQsAACBgBCwAAICA/X8kiTIckrHZWwAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 720x504 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EXxHBAtHoSh2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7dee67ef-1c5c-4011-e332-656dece0a41b"
      },
      "source": [
        "# Calculate model_1 metrics\n",
        "mae_1 = mae(y_test, y_preds_1.squeeze()).numpy()\n",
        "mse_1 = mse(y_test, y_preds_1.squeeze()).numpy()\n",
        "mae_1, mse_1"
      ],
      "execution_count": 65,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(18.745327, 353.57336)"
            ]
          },
          "metadata": {},
          "execution_count": 65
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XXELOpdBrE9_"
      },
      "source": [
        "**Build `model_2`**\n",
        "\n",
        "This time we'll add an extra dense layer (so now our model will have 2 layers) whilst keeping everything else the same."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "05vcgEP3rEFi",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "36aea698-e728-4ce4-a41f-9ed0d9f7368f"
      },
      "source": [
        "# Set random seed\n",
        "tf.random.set_seed(42)\n",
        "\n",
        "# Replicate model_1 and add an extra layer\n",
        "model_2 = tf.keras.Sequential([\n",
        "  tf.keras.layers.Dense(1),\n",
        "  tf.keras.layers.Dense(1) # add a second layer\n",
        "])\n",
        "\n",
        "# Compile the model\n",
        "model_2.compile(loss=tf.keras.losses.mae,\n",
        "                optimizer=tf.keras.optimizers.SGD(),\n",
        "                metrics=['mae'])\n",
        "\n",
        "# Fit the model\n",
        "model_2.fit(tf.expand_dims(X_train, axis=-1), y_train, epochs=100, verbose=0) # set verbose to 0 for less output"
      ],
      "execution_count": 67,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f8df6923cd0>"
            ]
          },
          "metadata": {},
          "execution_count": 67
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9xCbDcoDraux",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 428
        },
        "outputId": "5c0391b8-2bbe-4d4f-c1ea-090ee6d93ff5"
      },
      "source": [
        "# Make and plot predictions for model_2\n",
        "y_preds_2 = model_2.predict(X_test)\n",
        "plot_predictions(predictions=y_preds_2)"
      ],
      "execution_count": 68,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAGbCAYAAAAV7J4cAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3df3RU9Z3/8debgCA/FgGpIhSCLQoBY4AU/C0UWy3WKp61lcZVaytitaj7VaiyWuye7NHWVmtdpWnXU92TWvzWumpFV7GyaFlLg+YLAaT4I0Esi1mwURoVCO/vHzMJSZiESTL3zsy9z8c5nGQ+czPzyWQSX37uva9r7i4AAAAEr1e2JwAAABAXBC8AAICQELwAAABCQvACAAAICcELAAAgJL2zPYF0HHnkkV5YWJjtaQAAABzS2rVr/9fdh6e6Ly+CV2FhoaqqqrI9DQAAgEMys7qO7mNXIwAAQEgIXgAAACEheAEAAIQkL47xSmXv3r3atm2bPv7442xPBUn9+vXTqFGj1KdPn2xPBQCAnJS3wWvbtm0aNGiQCgsLZWbZnk7subt27typbdu2aezYsdmeDgAAOSlvdzV+/PHHGjZsGKErR5iZhg0bxgokAACdyNvgJYnQlWP4eQAA0Lm8Dl4AAAD5hODVTTt37lRJSYlKSkp09NFHa+TIkS239+zZ0+nXVlVVacGCBYd8jlNOOSVT021jxowZhyykveeee9TY2BjI8wMAEFd5e3B9tg0bNkzV1dWSpCVLlmjgwIG68cYbW+7ft2+fevdO/fKWlpaqtLT0kM+xevXqzEy2G+655x5dcskl6t+/f9bmAABA1MRmxauyUioslHr1SnysrMz8c1x++eWaP3++pk+froULF2rNmjU6+eSTNXnyZJ1yyinavHmzJGnlypX68pe/LCkR2q644grNmDFDxx57rO69996Wxxs4cGDL9jNmzNDf//3fa/z48SorK5O7S5KWL1+u8ePHa+rUqVqwYEHL47b20Ucf6eKLL9aECRM0Z84cffTRRy33XX311SotLdXEiRP1ve99T5J077336i9/+YtmzpypmTNndrgdAADomliseFVWSvPmSc17zurqErclqawss8+1bds2rV69WgUFBfrggw/00ksvqXfv3lqxYoVuueUWPfbYYwd9zeuvv64XX3xRH374oY4//nhdffXVB3Vhvfbaa9qwYYOOOeYYnXrqqfrDH/6g0tJSXXXVVVq1apXGjh2ruXPnppzTAw88oP79+2vTpk1at26dpkyZ0nJfeXm5hg4dqqamJs2aNUvr1q3TggUL9OMf/1gvvviijjzyyA63Ky4uzuArBwBA9MVixWvx4gOhq1ljY2I80y666CIVFBRIkhoaGnTRRRdp0qRJuuGGG7Rhw4aUX3Puueeqb9++OvLII/WpT31KO3bsOGibadOmadSoUerVq5dKSkpUW1ur119/Xccee2xLb1ZHwWvVqlW65JJLJEnFxcVtAtOjjz6qKVOmaPLkydqwYYM2btyY8jHS3Q4AAHQsFsFr69aujffEgAEDWj6/9dZbNXPmTNXU1Oipp57qsOOqb9++LZ8XFBRo37593dqmq95++23dddddeuGFF7Ru3Tqde+65KeeY7nYAAOSsMI45SkMsgtfo0V0bz5SGhgaNHDlSkvTLX/4y449//PHH66233lJtba0kadmyZSm3O+OMM/SrX/1KklRTU6N169ZJkj744AMNGDBAgwcP1o4dO/TMM8+0fM2gQYP04YcfHnI7AAByXvMxR3V1kvuBY46yEL5iEbzKy6X2J+f1758YD9LChQt18803a/LkyRlZoWrv8MMP1/33369zzjlHU6dO1aBBgzR48OCDtrv66qu1e/duTZgwQbfddpumTp0qSTrxxBM1efJkjR8/Xl//+td16qmntnzNvHnzdM4552jmzJmdbgcAQM4L85ijQ7Dms+NyWWlpqbfvndq0aZMmTJiQ9mNUViZe361bEytd5eWZP7A+G3bv3q2BAwfK3XXNNddo3LhxuuGGG7I2n67+XAAACFyvXomVrvbMpP37M/50ZrbW3VP2RsVixUtKhKza2sTrW1sbjdAlST//+c9VUlKiiRMnqqGhQVdddVW2pwQAQG7J1jFHKcSiTiLKbrjhhqyucAEAkPPKy9v2SknhHHOUQmxWvAAAQEyVlUkVFdKYMYndi2PGJG5nYfcXwQsAAOSvNGsiKoulwuulXt9LfKzMUgc4uxoBAEB+SvPSNJXrKzXvqXlq3JvYrq6hTvOeSmxXdkK4q16seAEAgPyUZk3E4hcWt4Suls32NmrxC+HXSRC8umnnzp0qKSlRSUmJjj76aI0cObLl9p49ew759StXrtTq1atbbi9dulQPP/xwxufZ+oLcHamurtby5csz/twAAAQqzUvTbG1IvV1H40FiV2M3DRs2TNXV1ZKkJUuWaODAgbrxxhvT/vqVK1dq4MCBOuWUUyRJ8+fPD2Se6aiurlZVVZVmz56dtTkAANBlo0cndi+mGm99c/Bo1TUcvN3oweHXScRmxatyfaUK7ylUr9t7qfCeQlWuz/xlAtauXaszzzxTU6dO1dlnn63t27dLku69914VFRWpuLhYF198sWpra7V06VLdfffdKikp0UsvvaQlS5borrvukiTNmDFDixYt0rRp03TcccfppZdekiQ1Njbqq1/9qoqKijRnzhxNnz5d7YtlJenZZ5/V+PHjNWXKFP32t79tGV+zZo1OPvlkTZ48Waeccoo2b96sPXv26LbbbtOyZctUUlKiZcuWpdwOAICck+alacpnlat/n7bb9e/TX+Wzwq+TiMWKVxgH1bm7vvOd7+iJJ57Q8OHDtWzZMi1evFgPPvig7rjjDr399tvq27ev/vrXv+qII47Q/Pnz26ySvfDCC20eb9++fVqzZo2WL1+u22+/XStWrND999+vIUOGaOPGjaqpqVFJSclB8/j444915ZVX6ve//70++9nP6mtf+1rLfePHj9dLL72k3r17a8WKFbrlllv02GOP6fvf/76qqqp03333SUpcmzHVdgAA5JTmA+gPcWma5v/WL35hsbY2bNXowaNVPqs89APrpZgEr84OqsvUi/7JJ5+opqZGX/jCFyRJTU1NGjFihCSpuLhYZWVluuCCC3TBBRek9XgXXnihJGnq1KktF8F++eWXdd1110mSJk2apOLig8+Fff311zV27FiNGzdOknTJJZeooqJCUuKi3Zdddpm2bNkiM9PevXtTPne62wEAkG2VxdLi66WtDdLowVJ5sZTqv+xlJ5RlJWi1F4tdjWEcVOfumjhxoqqrq1VdXa3169frueeekyQ9/fTTuuaaa/Tqq6/qc5/7XFoXzO7bt68kqaCgIGMX2L711ls1c+ZM1dTU6KmnntLHH3/co+0AAAhEut1cyT1adQ11cnnLHq0gDifKlFgEr44OnsvkQXV9+/ZVfX29/vu//1uStHfvXm3YsEH79+/XO++8o5kzZ+rOO+9UQ0ODdu/erUGDBunDDz/s0nOceuqpevTRRyVJGzdu1Pr16w/aZvz48aqtrdWbb74pSXrkkUda7mtoaNDIkSMlSb/85S9bxtvPpaPtAAAIXHM3V11d4sLWzd1cKcJXLtVEpCsjwcvMHjSz98ysptXYUDN73sy2JD8OSY6bmd1rZm+Y2Tozm5KJOXQmjIPqevXqpd/85jdatGiRTjzxRJWUlGj16tVqamrSJZdcohNOOEGTJ0/WggULdMQRR+i8887T448/3nJwfTq+/e1vq76+XkVFRfqnf/onTZw4UYMHD26zTb9+/VRRUaFzzz1XU6ZM0ac+9amW+xYuXKibb75ZkydPbrOKNnPmTG3cuLHl4PqOtgMAIHBpdnNJuVUTkS5z954/iNkZknZLetjdJyXHfiBpl7vfYWbflTTE3ReZ2WxJ35E0W9J0ST9x9+mdPX5paam3P3tv06ZNmjBhQtpzrFxfmRMH1fVEU1OT9u7dq379+unNN9/UWWedpc2bN+uwww7L9tRadPXnAgBAG716JVa62jOT9u9vM1R4T2HKmogxg8eo9vragCZ4aGa21t1LU92XkYPr3X2VmRW2Gz5f0ozk5w9JWilpUXL8YU8kvlfM7AgzG+Hu2zMxl47kykF1PdHY2KiZM2dq7969cnfdf//9ORW6AADosTS7uaTEHq3WrQVS9moi0hXkWY1HtQpT/yPpqOTnIyW902q7bcmxNsHLzOZJmidJo1O82HE0aNCglL1dAABERnl52+svSim7uaTcqolIVyh1Eu7uZtalfZruXiGpQkrsagxkYgAAILeUlenld/6gwh9U6Jj3m/SXIQWqXXiZTitLHabybY9WkMFrR/MuRDMbIem95Pi7kj7dartRyTEAABBzlesrNW//Q2q8rik50qT++x9SxfpT8ypgdSTIOoknJV2W/PwySU+0Gr80eXbjSZIagj6+CwAA5IA0+rnysSKiKzKy4mVmjyhxIP2RZrZN0vck3SHpUTP7pqQ6SV9Nbr5ciTMa35DUKOkbmZgDAADIYc39XM3HbjX3c0ltLvGTjxURXZGRFS93n+vuI9y9j7uPcvd/c/ed7j7L3ce5+1nuviu5rbv7Ne7+GXc/wd3z9mjxgoIClZSUaNKkSbrooovU2L53pAsuv/xy/eY3v5Ekfetb39LGjRs73HblypVavXp1y+2lS5fq4Ycf7vZzAwAQuDT7ucIoPc+mWDTXB+Xwww9XdXW1ampqdNhhh2np0qVt7u9u+egvfvELFRUVdXh/++A1f/58XXrppd16LgAAQrG1gxWrduNhlJ5nU3yCV5rXfequ008/XW+88YZWrlyp008/XV/5yldUVFSkpqYm3XTTTfrc5z6n4uJi/exnP5OUuLbjtddeq+OPP15nnXWW3nvvvZbHmjFjRkttxLPPPqspU6boxBNP1KxZs1RbW6ulS5fq7rvvbmm9X7Jkie666y5JUnV1tU466SQVFxdrzpw5ev/991sec9GiRZo2bZqOO+64lrb8DRs2aNq0aSopKVFxcbG2bNmS0dcFAABJKXu4Uo2XnVCmivMqNGbwGJlMYwaPUcV5FZE4sF4KqU4i69Lcr9xd+/bt0zPPPKNzzjlHkvTqq6+qpqZGY8eOVUVFhQYPHqw//elP+uSTT3Tqqafqi1/8ol577TVt3rxZGzdu1I4dO1RUVKQrrriizePW19fryiuv1KpVqzR27Fjt2rVLQ4cO1fz58zVw4EDdeOONkqQXXnih5WsuvfRS/fSnP9WZZ56p2267TbfffrvuueeelnmuWbNGy5cv1+23364VK1Zo6dKluu6661RWVqY9e/aoqalJAABk2svzZ2vybQ9owN4DY3/rI702f7ZOa7dtvlVEdEU8Vry6cN2nrvjoo49UUlKi0tJSjR49Wt/85jclSdOmTdPYsWMlSc8995wefvhhlZSUaPr06dq5c6e2bNmiVatWae7cuSooKNAxxxyjz3/+8wc9/iuvvKIzzjij5bGGDh3a6XwaGhr017/+VWeeeaYk6bLLLtOqVata7r/wwgslSVOnTlVtba0k6eSTT9a//Mu/6M4771RdXZ0OP/zwHr0mAACkckm/5bryPKl2sLRfiY9XnpcYj5N4rHiluV+5q5qP8WpvwIABLZ+7u37605/q7LPPbrPN8uXhv9H69u0rKXFSQPPxZ1//+tc1ffp0Pf3005o9e7Z+9rOfpQyBAAD0xNaGraorlh4pbjtuETlbMV3xWPFKc79yEM4++2w98MAD2rs3sbb65z//WX/72990xhlnaNmyZWpqatL27dv14osvHvS1J510klatWqW3335bkrRr1y5JiUsHffjhhwdtP3jwYA0ZMqTl+K1///d/b1n96shbb72lY489VgsWLND555+vdevW9ej7BQDEUBrHUUf9bMV0xSN4lZcnrvPUWgfXfcq0b33rWyoqKtKUKVM0adIkXXXVVdq3b5/mzJmjcePGqaioSJdeeqlOPvnkg752+PDhqqio0IUXXqgTTzxRX/va1yRJ5513nh5//PGWg+tbe+ihh3TTTTepuLhY1dXVuu222zqd36OPPqpJkyappKRENTU1nB0JAOia5uOo6+ok9wPHUbcLX1E/WzFd5p77l0EsLS319heH3rRpkyZMmJD+g1RWJo7p2ro1sdJVXp6RA+vRVpd/LgCA/FZYmAhb7Y0ZIyWPJ25Wub4yry5o3V1mttbdS1PeF5vghVDwcwGAmOnVK7HS1Z6ZtH9/+PPJAZ0Fr3jsagQAAIHYfXTqM+47Go+7vA5e+bBaFyf8PAAgfm75fKKPq7W/9UmM42B5G7z69eunnTt38h/7HOHu2rlzp/r165ftqQAAQnTfuF0p+7nuG7cr21PLSXnb4zVq1Cht27ZN9fX12Z4Kkvr166dRo0ZlexoAgBCNHjxajxTXHdTPNSZmNRHpytvg1adPn5ZGdwAAkGFptgGUzyrXvKfmqXHvgSvExLEmIl15u6sRAAAEJM1uLin6F7XOtLytkwAAAAHpQjcXDkadBAAASJtvTRG6OhlH+gheAACgjXePKOjSONJH8AIAAG0smtmUsptr0cym7EwoQgheAACgjT+cPiZlN9cfTh+T7anlPYIXAABxUlmZOHi+V6/ExxRnKpbPKtcTU/tr7A1SwRJp7A3SE1OpiMgEghcAAHGRZk0EFRHBoU4CAIC4oCYiFNRJAAAAaiJyAMELAICYoCYi+wheAADEBDUR2UfwAgAgJqiJyL7e2Z4AAAAIR/mscs1rnKdHihtbxvr36a8KaiJCw4oXAABRkEY/FzUR2UedBAAA+a6yUvu+dYV6f7ynZWhfv8PU+xcPSmWEqrBRJwEAQITtvum6NqFLknp/vEe7b7ouSzNCRwheAADkuf7bd3ZpHNlD8AIAIM9tHdy1cWQPwQsAgDz34y8PS9nP9eMvD8vOhNChQIOXmR1vZtWt/n1gZteb2RIze7fV+Owg5wEAQJRNX/QTXXtBnzb9XNde0EfTF/0k21NDO4H2eLn7ZkklkmRmBZLelfS4pG9Iutvd7wry+QEAiIOyE8qkW6UZpyzW1oatGj14tMpnlVMTkYPC3NU4S9Kb7s6VOAEASMPLd3xb24b21n4zbRvaWy/f8e0Oty07oUy119dq//f2q/b6WkJXjgozeF0s6ZFWt681s3Vm9qCZDWm/sZnNM7MqM6uqr68Pb5YAAOSAl+/4tibf9oBGvd+kXpJGvd+kybc90Gn4Qu4LpUDVzA6T9BdJE919h5kdJel/Jbmkf5Y0wt2v6OjrKVAFAMTNtqG9Ner9gy9evW1IgUbt2peFGSFduVCg+iVJr7r7Dkly9x3u3uTu+yX9XNK0kOYBAEBeOCZF6OpsHPkhrOA1V612M5rZiFb3zZFUE9I8AADIC38ZUtClceSHwIOXmQ2Q9AVJv201/AMzW29m6yTNlHRD0PMAACCf1C6cl7Kbq3bhvOxMCBkRaJ2EJLn73yQNazf2D0E/LwAA+ey0796vlyUV/qBCx7zfpL8MKVDtwnk67bv3Z3tq6AGa6wEACFnl+koV3lOoXrf3UuE9hapcX5lyu9O+e79G7dqnXu4atWsfoSsCAl/xAgAAB1Sur9SKf/6GVj63V6MbpK2D63T76m9It4rurRhgxQsAgBD98c7rdN9/7FVhQ+I/woUN0n3/sVd/vPO6bE8NISB4AQAQon/83U4N2Nt2bMDexDiij+AFAECIRjd0bRzRQvACACBEjSOGdWkc0ULwAgAgRAN/+BPt63dYm7F9/Q7TwB/+JEszQpgIXgAAZEhlpVRYKPXqlfhYmaoloqxMvX/xoDRmjGQmjRmTuF3GGY1xQJ0EAAAZUFkpzZsnNTYmbtfVJW5LKTJVWRlBK6ZY8QIAIAMWLz4Qupo1NibGgWYELwAAMmDr1q6NI54IXgAAZMDo0V0bRzwRvAAAyIDycql//7Zj/fsnxoFmBC8AADKgrEyqqGhzsqIqKjiGHm0RvAAA6ERaFRFJZWVSba20f3/iI6EL7VEnAQBAB7pUEQGkgRUvAAA6QEUEMo3gBQBAB6iIQKYRvAAA6AAVEcg0ghcAAB2gIgKZRvACAKADVEQg0wheAIBYSrcmgooIZBJ1EgCA2KEmAtnCihcAIHaoiUC2ELwAALFDTQSyheAFAIgdaiKQLQQvAEDsUBOBbCF4AQBih5oIZAvBCwAQKdREIJdRJwEAiAxqIpDrWPECAEQGNRHIdQQvAEBkUBOBXEfwAgBEBjURyHUELwBAZFATgVwXePAys1ozW29m1WZWlRwbambPm9mW5MchQc8DABB91EQg14W14jXT3UvcvTR5+7uSXnD3cZJeSN4GACCldCsiJGoikNuytavxfEkPJT9/SNIFWZoHACDHNVdE1NVJ7gcqIjoLX0CuCiN4uaTnzGytmSXbVHSUu29Pfv4/ko4KYR4AgDxERQSiJIwC1dPc/V0z+5Sk583s9dZ3urubmbf/omRImydJozkdBQBii4oIREngK17u/m7y43uSHpc0TdIOMxshScmP76X4ugp3L3X30uHDhwc9TQBAjqIiAlESaPAyswFmNqj5c0lflFQj6UlJlyU3u0zSE0HOAwCQv6iIQJQEveJ1lKSXzez/SVoj6Wl3f1bSHZK+YGZbJJ2VvA0AiJl0zlakIgJRYu4HHV6Vc0pLS72qqirb0wAAZFD7C1pLiZUsQhXynZmtbVWh1QbN9QCArOBsRcQRwQsAkBWcrYg4IngBALKCsxURRwQvAEBWcLYi4ojgBQDICs5WRBwRvAAAGcUFrYGOhXHJIABATLSviGi+oLVEqAIkVrwAABlERQTQOYIXACBjqIgAOkfwAgBkDBURQOcIXgCAjKEiAugcwQsAkDFURACdI3gBANKSbk0EFRFAx6iTAAAcEjURQGaw4gUAOCRqIoDMIHgBAA6JmgggMwheAIBDoiYCyAyCFwDgkKiJADKD4AUAOCRqIoDMIHgBQMxREwGEhzoJAIgxaiKAcLHiBQAxRk0EEC6CFwDEGDURQLgIXgAQY9REAOEieAFAjFETAYSL4AUAMUZNBBAughcARFC6FRESNRFAmKiTAICIoSICyF2seAFAxFARAeQughcARAwVEUDuIngBQMRQEQHkLoIXAEQMFRFA7iJ4AUDEUBEB5C6CFwDkkXRrIqiIAHITdRIAkCeoiQDyX2ArXmb2aTN70cw2mtkGM7suOb7EzN41s+rkv9lBzQEAooSaCCD/BbnitU/S/3H3V81skKS1ZvZ88r673f2uAJ8bACKHmggg/wW24uXu29391eTnH0raJGlkUM8HAFFHTQSQ/0I5uN7MCiVNlvTH5NC1ZrbOzB40syEdfM08M6sys6r6+vowpgkAOY2aCCD/BR68zGygpMckXe/uH0h6QNJnJJVI2i7pR6m+zt0r3L3U3UuHDx8e9DQBIOdREwHkv0CDl5n1USJ0Vbr7byXJ3Xe4e5O775f0c0nTgpwDAOQDaiKAeAjs4HozM0n/JmmTu/+41fgId9+evDlHUk1QcwCAfEBNBBAf5u7BPLDZaZJekrRe0v7k8C2S5iqxm9El1Uq6qlUQS6m0tNSrqqoCmScAZFthYSJstTdmTGJVC0B+MbO17l6a6r7AVrzc/WVJluKu5UE9JwDkI2oigPjgkkEAkGXURADxQfACgCyjJgKID4IXAGQZNRFAfBC8ACAg6VZESNREAHER5LUaASC2qIgAkAorXgAQgMWLD4SuZo2NiXEA8UXwAoAAUBEBIBWCFwAEgIoIAKkQvAAgAFREAEiF4AUAAaAiAkAqBC8A6KJ0ayKoiADQHnUSANAF1EQA6AlWvACgC6iJANATBC8A6AJqIgD0BMELALqAmggAPUHwAoAuoCYCQE8QvACgC6iJANATBC8ASKImAkDQqJMAAFETASAcrHgBgKiJABAOghcAiJoIAOEgeAGAqIkAEA6CFwCImggA4SB4AYCoiQAQDoIXgEhLtyJCoiYCQPCokwAQWVREAMg1rHgBiCwqIgDkGoIXgMiiIgJAriF4AYgsKiIA5BqCF4DIoiICQK4heAHIS+mcrUhFBIBcw1mNAPJOV85WLCsjaAHIHax4Acg7nK0IIF8RvADkHc5WBJCvsha8zOwcM9tsZm+Y2XezNQ8A+YezFQHkq6wELzMrkPSvkr4kqUjSXDMrysZcAOQfzlYEkK+yteI1TdIb7v6Wu++R9GtJ52dpLgDyDGcrAshX2QpeIyW90+r2tuRYCzObZ2ZVZlZVX18f6uQAZE+6F7XmgtYA8lHOHlzv7hXuXurupcOHD8/2dACEoLkmoq5Ocj9QE9FR+AKAfJOt4PWupE+3uj0qOQYgxqiJABB12Qpef5I0zszGmtlhki6W9GSW5gIgR1ATASDqshK83H2fpGsl/aekTZIedfcN2ZgLgNxBTQSAqMvaMV7uvtzdj3P3z7g7J4EDoCYCQOTl7MH1AOKHmggAUUfwAhC4dCsiJGoiAERb72xPAEC0NVdENJ+t2FwRIRGqAMQPK14AAkVFBAAcQPACECgqIgDgAIIXgEBREQEABxC8AASKiggAOIDgBSBQVEQAwAEELwDdlm5NBBURAJBAnQSAbqEmAgC6jhUvAN1CTQQAdB3BC0C3UBMBAF1H8ALQLdREAEDXEbwAdAs1EQDQdQQvAN1CTQQAdB3BC8BBqIkAgGBQJwGgDWoiACA4rHgBaIOaCAAIDsELQBvURABAcAheANqgJgIAgkPwAtAGNREAEByCF4A2qIkAgOAQvICYSLciQqImAgCCQp0EEANURABAbmDFC4gBKiIAIDcQvIAYoCICAHIDwQuIASoiACA3ELyAGKAiAgByA8ELiAEqIgAgNxC8gDyXbk0EFREAkH3USQB5jJoIAMgvrHgBeYyaCADILwQvII9REwEA+YXgBeQxaiIAIL8EErzM7Idm9rqZrTOzx83siOR4oZl9ZGbVyX9Lg3h+IC6oiQCA/BLUitfzkia5e7GkP0u6udV9b7p7SfLf/ICeH4gFaiIAIL8EErzc/Tl335e8+YqkUUE8DxBV6VZESNREAEA+CeMYryskPdPq9lgze83M/qamjo4AAA5hSURBVMvMTu/oi8xsnplVmVlVfX198LMEckRzRURdneR+oCKis/AFAMgP5u7d+0KzFZKOTnHXYnd/IrnNYkmlki50dzezvpIGuvtOM5sq6T8kTXT3Dzp7rtLSUq+qqurWPIF8U1iYCFvtjRmTWNECAOQ2M1vr7qWp7ut2gaq7n3WIJ71c0pclzfJkunP3TyR9kvx8rZm9Kek4SaQqIImKCACIrqDOajxH0kJJX3H3xlbjw82sIPn5sZLGSXoriDkA+YqKCACIrqCO8bpP0iBJz7erjThD0jozq5b0G0nz3X1XQHMA8hIVEQAQXYFcq9HdP9vB+GOSHgviOYGoaD4rcfHixO7F0aMToYuzFQEg/9FcD4Qo3ZoIKiIAIJoCWfECcLDmmojmi1o310RIBCsAiAtWvICQLF58IHQ1a2xMjAMA4oHgBYSEmggAAMELCAk1EQAAghcQEmoiAAAELyAkZWVSRUXi0j9miY8VFRxYDwBxQvACMoCaCABAOqiTAHqImggAQLpY8QJ6iJoIAEC6CF5AD1ETAQBIF8EL6CFqIgAA6SJ4AT1ETQQAIF0EL6CHqIkAAKSL4AV0IN2KCImaCABAeqiTAFKgIgIAEARWvIAUqIgAAASB4AWkQEUEACAIBC8gBSoiAABBIHgBKVARAQAIAsELSIGKCABAEAheiJ10ayKoiAAAZBp1EogVaiIAANnEihdihZoIAEA2EbwQK9REAACyieCFWKEmAgCQTQQvxAo1EQCAbCJ4ITLSOVuRmggAQDZxViMioStnK5aVEbQAANnBihcigbMVAQD5gOCFSOBsRQBAPiB4IRI4WxEAkA8IXogEzlYEAOQDghcigbMVAQD5ILDgZWZLzOxdM6tO/pvd6r6bzewNM9tsZmcHNQfkv3QvaC1xUWsAQO4Luk7ibne/q/WAmRVJuljSREnHSFphZse5e1PAc0Ge4YLWAICoycauxvMl/drdP3H3tyW9IWlaFuaBHEdFBAAgaoIOXtea2Toze9DMhiTHRkp6p9U225JjbZjZPDOrMrOq+vr6gKeJXERFBAAganoUvMxshZnVpPh3vqQHJH1GUomk7ZJ+1JXHdvcKdy9199Lhw4f3ZJrIU1REAACipkfHeLn7WelsZ2Y/l/S75M13JX261d2jkmNAG+XlbY/xkqiIAADktyDPahzR6uYcSTXJz5+UdLGZ9TWzsZLGSVoT1DyQv6iIAABETZDHeP3AzNab2TpJMyXdIEnuvkHSo5I2SnpW0jWc0Rg/6dZEUBEBAIiSwOok3P0fOrmvXBI7jGKKmggAQFzRXI/QURMBAIgrghdCR00EACCuCF4IHTURAIC4InghdOXliVqI1qiJAADEAcELoaMmAgAQVwQvZBQ1EQAAdCywOgnEDzURAAB0jhUvZAw1EQAAdI7ghYyhJgIAgM4RvJAx1EQAANA5ghcyhpoIAAA6R/BCxlATAQBA5wheOKR0KyIkaiIAAOgMdRLoFBURAABkDite6BQVEQAAZA7BC52iIgIAgMwheKFTVEQAAJA5BC90iooIAAAyh+CFTlERAQBA5hC8YizdmggqIgAAyAzqJGKKmggAAMLHildMURMBAED4CF4xRU0EAADhI3jFFDURAACEj+AVU9REAAAQPoJXTFETAQBA+AheEURNBAAAuYk6iYihJgIAgNzFilfEUBMBAEDuInhFDDURAADkLoJXxFATAQBA7iJ4RQw1EQAA5C6CV8RQEwEAQO4ieOWJdCsiJGoiAADIVYHUSZjZMknHJ28eIemv7l5iZoWSNknanLzvFXefH8QcooSKCAAAoiGQ4OXuX2v+3Mx+JKmh1d1vuntJEM8bVZ1VRBC8AADIH4EWqJqZSfqqpM8H+TxRR0UEAADREPQxXqdL2uHuW1qNjTWz18zsv8zs9ICfPxKoiAAAIBq6HbzMbIWZ1aT4d36rzeZKeqTV7e2SRrv7ZEn/KOlXZvZ3HTz+PDOrMrOq+vr67k4zEqiIAAAgGrq9q9Hdz+rsfjPrLelCSVNbfc0nkj5Jfr7WzN6UdJykqhSPXyGpQpJKS0u9u/OMgubjuBYvTuxeHD06Ebo4vgsAgPwS5K7GsyS97u7bmgfMbLiZFSQ/P1bSOElvBTiHnJduTQQVEQAA5L8gD66/WG13M0rSGZK+b2Z7Je2XNN/ddwU4h5xGTQQAAPFi7rm/F6+0tNSrqg7aG5n3CgsTYau9MWMSq1oAACD/mNlady9NdR/N9VlETQQAAPFC8MoiaiIAAIgXglcWURMBAEC8ELyyqKxMqqhIHNNllvhYUcGB9QAARBXBKyDURAAAgPYCvVZjXFETAQAAUmHFKwCLFx8IXc0aGxPjAAAgvgheAaAmAgAApELwCgA1EQAAIBWCVwCoiQAAAKkQvAJATQQAAEiF4NUF6VZESNREAACAg1EnkSYqIgAAQE+x4pUmKiIAAEBPEbzSREUEAADoKYJXmqiIAAAAPUXwShMVEQAAoKcIXmmiIgIAAPQUwUvp10RQEQEAAHoi9nUS1EQAAICwxH7Fi5oIAAAQltgHL2oiAABAWGIfvKiJAAAAYYl98KImAgAAhCX2wYuaCAAAEJbYn9UoJUIWQQsAAAQt9iteAAAAYSF4AQAAhITgBQAAEBKCFwAAQEgIXgAAACEheAEAAISE4AUAABASghcAAEBICF4AAAAh6VHwMrOLzGyDme03s9J2991sZm+Y2WYzO7vV+DnJsTfM7Ls9eX4AAIB80tMVrxpJF0pa1XrQzIokXSxpoqRzJN1vZgVmViDpXyV9SVKRpLnJbQEAACKvR9dqdPdNkmRm7e86X9Kv3f0TSW+b2RuSpiXve8Pd30p+3a+T227syTwAAADyQVAXyR4p6ZVWt7clxyTpnXbj01M9gJnNkzQveXO3mW3O9CRTOFLS/4bwPLks7q9B3L9/iddA4jWI+/cv8RpIvAY9+f7HdHTHIYOXma2QdHSKuxa7+xPdnNAhuXuFpIqgHj8VM6ty99JDbxldcX8N4v79S7wGEq9B3L9/iddA4jUI6vs/ZPBy97O68bjvSvp0q9ujkmPqZBwAACDSgqqTeFLSxWbW18zGShonaY2kP0kaZ2ZjzewwJQ7AfzKgOQAAAOSUHh3jZWZzJP1U0nBJT5tZtbuf7e4bzOxRJQ6a3yfpGndvSn7NtZL+U1KBpAfdfUOPvoPMCnXXZo6K+2sQ9+9f4jWQeA3i/v1LvAYSr0Eg37+5exCPCwAAgHZorgcAAAgJwQsAACAksQxeXOqoLTNbZmbVyX+1ZladHC80s49a3bc023MNipktMbN3W32vs1vdl/I9ESVm9kMze93M1pnZ42Z2RHI8Nu8BKdq/5x0xs0+b2YtmtjH5d/G65HiHvxNRlPzbtz75vVYlx4aa2fNmtiX5cUi25xkEMzu+1c+52sw+MLPro/4eMLMHzew9M6tpNZbyZ24J9yb/Nqwzsyndft44HuNlZhMk7Zf0M0k3unvzL1mRpEeUaNk/RtIKScclv+zPkr6gROnrnyTNdffINe6b2Y8kNbj7982sUNLv3H1SdmcVPDNbImm3u9/Vbjzle6L5ZJGoMLMvSvq9u+8zszslyd0Xxew9UKCY/J63ZmYjJI1w91fNbJCktZIukPRVpfidiCozq5VU6u7/22rsB5J2ufsdySA+xN0XZWuOYUj+HryrRLn5NxTh94CZnSFpt6SHm//GdfQzT4bO70iarcRr8xN3T1kAfyixXPFy903unqoJv+VSR+7+tqTmSx1NU/JSR+6+R1LzpY4ixcxMiT+2j2R7Ljmko/dEpLj7c+6+L3nzFSU69uImFr/n7bn7dnd/Nfn5h5I26cCVRuLufEkPJT9/SIlAGnWzJL3p7nXZnkjQ3H2VpF3thjv6mZ+vREBzd39F0hHJ/2npslgGr06M1MGXNBrZyXjUnC5ph7tvaTU21sxeM7P/MrPTszWxkFybXEJ+sNUuhbj87Fu7QtIzrW7H5T0Qx591G8kVzsmS/pgcSvU7EVUu6TkzW2uJS9ZJ0lHuvj35+f9IOio7UwvVxWr7P99xeg9IHf/MM/b3IbLBy8xWmFlNin+R/z/YVNJ8Peaq7S/cdkmj3X2ypH+U9Csz+7sw551Jh3gNHpD0GUklSnzfP8rqZAOQznvAzBYr0b1XmRyK1HsAHTOzgZIek3S9u3+gGPxOtHOau0+R9CVJ1yR3Q7XwxHE5kT42xxLF5l+R9H+TQ3F7D7QR1M88qItkZx2XOmrrUK+HmfWWdKGkqa2+5hNJnyQ/X2tmbypxzFtVgFMNTLrvCTP7uaTfJW929p7IK2m8By6X9GVJs5J/cCL3HjiEyPysu8rM+igRuird/beS5O47Wt3f+nciktz93eTH98zscSV2Pe8wsxHuvj25W+m9rE4yeF+S9Grzzz5u74Gkjn7mGfv7ENkVr26K86WOzpL0urtvax4ws+HJAy1lZscq8Xq8laX5Bardvvo5kprPcunoPREpZnaOpIWSvuLuja3GY/MeUDx+zw+SPLbz3yRtcvcftxrv6HcicsxsQPLEApnZAElfVOL7fVLSZcnNLpP0RHZmGJo2ez3i9B5opaOf+ZOSLk2e3XiSEiehbU/1AIcS2RWvzlj0LnWUCe3360vSGZK+b2Z7lTgLdL67tz8QMSp+YGYlSiwr10q6SpI6e09EzH2S+kp6PvHfYb3i7vMVo/dA8ozOqP+ep3KqpH+QtN6SVTKSbpE0N9XvREQdJenx5Hu/t6RfufuzZvYnSY+a2Tcl1Slx8lEkJQPnF9T255zy72JUmNkjkmZIOtLMtkn6nqQ7lPpnvlyJMxrfkNSoxBmf3XveONZJAAAAZAO7GgEAAEJC8AIAAAgJwQsAACAkBC8AAICQELwAAABCQvACAAAICcELAAAgJP8fx7d8DXiYq3kAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 720x504 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GxMYu5Frri6o"
      },
      "source": [
        "Woah, that's looking better already! And all it took was an extra layer."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9LnPuGaBrcrP",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "76d99ae4-14e6-4547-86b6-3847d64af935"
      },
      "source": [
        "# Calculate model_2 metrics\n",
        "mae_2 = mae(y_test, y_preds_2.squeeze()).numpy()\n",
        "mse_2 = mse(y_test, y_preds_2.squeeze()).numpy()\n",
        "mae_2, mse_2"
      ],
      "execution_count": 69,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(1.9098114, 5.459232)"
            ]
          },
          "metadata": {},
          "execution_count": 69
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "R8i9yfQGrwHx"
      },
      "source": [
        "**Build `model_3`**\n",
        "\n",
        "For our 3rd model, we'll keep everything the same as `model_2` except this time we'll train for longer (500 epochs instead of 100).\n",
        "\n",
        "This will give our model more of a chance to learn the patterns in the data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ABGwQFsbrvUS",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "94ff92ba-7f6b-4ec3-db2c-c0864285205f"
      },
      "source": [
        "# Set random seed\n",
        "tf.random.set_seed(42)\n",
        "\n",
        "# Replicate model_2\n",
        "model_3 = tf.keras.Sequential([\n",
        "  tf.keras.layers.Dense(1),\n",
        "  tf.keras.layers.Dense(1)\n",
        "])\n",
        "\n",
        "# Compile the model\n",
        "model_3.compile(loss=tf.keras.losses.mae,\n",
        "                optimizer=tf.keras.optimizers.SGD(),\n",
        "                metrics=['mae'])\n",
        "\n",
        "# Fit the model (this time for 500 epochs, not 100)\n",
        "model_3.fit(tf.expand_dims(X_train, axis=-1), y_train, epochs=500, verbose=0) # set verbose to 0 for less output"
      ],
      "execution_count": 71,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f8df67b6d90>"
            ]
          },
          "metadata": {},
          "execution_count": 71
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jEz4bVmasbFk",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 428
        },
        "outputId": "45f04bb9-3057-4f1c-c9a6-a6a038ed5701"
      },
      "source": [
        "# Make and plot predictions for model_3\n",
        "y_preds_3 = model_3.predict(X_test)\n",
        "plot_predictions(predictions=y_preds_3)"
      ],
      "execution_count": 72,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAGbCAYAAAAY8u5bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3RV9Z338c+XiyDC4C3eoBBoUS6KAVK8MCIpqFRrFVdtsbHqY1vEarHOcrTK1GJnZZZ2bPXRPkrjjKN2pRYfrVVbdBTUwQ51aNA8EEAKSkKxDKY4jdio3L7PH+ckHMJJOCdnn8ve+/1aKyvn7HPZv3MLH/b+7c8xdxcAAACC06vYAwAAAIgaAhYAAEDACFgAAAABI2ABAAAEjIAFAAAQsD7FHkCqo48+2svLy4s9DAAAgINauXLln929LN1lJRWwysvLVV9fX+xhAAAAHJSZNXd1GbsIAQAAAkbAAgAACBgBCwAAIGAlNQcrnV27dmnLli36+OOPiz0UJPXv319Dhw5V3759iz0UAABKUskHrC1btmjQoEEqLy+XmRV7OLHn7tq+fbu2bNmiESNGFHs4AACUpJLfRfjxxx/rqKOOIlyVCDPTUUcdxRZFAAC6UfIBSxLhqsTwegAA0L1QBCwAAIAwIWAdxPbt21VRUaGKigodd9xxGjJkSMf5nTt3dnvb+vp6zZs376DrOPPMM4Ma7n6mTZt20OLWe++9V21tbXlZPwAAcVXyk9yL7aijjlJDQ4MkacGCBRo4cKBuuummjst3796tPn3SP42VlZWqrKw86DqWL18ezGB74N5779Xll1+uAQMGFG0MAABETeS2YNXVSeXlUq9eid91dcGv46qrrtLcuXN12mmn6eabb9aKFSt0xhlnaMKECTrzzDO1fv16SdKrr76qL3zhC5IS4ezqq6/WtGnTNHLkSN13330d9zdw4MCO60+bNk1f+tKXNHr0aFVXV8vdJUmLFy/W6NGjNWnSJM2bN6/jflN99NFHmj17tsaMGaNZs2bpo48+6rjs2muvVWVlpcaNG6fvf//7kqT77rtPf/rTn1RVVaWqqqourwcAALITqS1YdXXSnDlS+x6v5ubEeUmqrg52XVu2bNHy5cvVu3dvffDBB3rttdfUp08fLVmyRLfddpueeuqpA27z1ltv6ZVXXtGOHTt00kkn6dprrz2gS+rNN9/UmjVrdMIJJ2jKlCn6z//8T1VWVuqaa67RsmXLNGLECF122WVpx/Tggw9qwIABWrdunVatWqWJEyd2XFZTU6MjjzxSe/bs0fTp07Vq1SrNmzdPP/7xj/XKK6/o6KOP7vJ648ePD/CZAwAg+iK1BWv+/H3hql1bW2J50C699FL17t1bktTa2qpLL71UJ598sm688UatWbMm7W0uuOAC9evXT0cffbSOOeYYbdu27YDrTJ48WUOHDlWvXr1UUVGhpqYmvfXWWxo5cmRH71RXAWvZsmW6/PLLJUnjx4/fLxg98cQTmjhxoiZMmKA1a9Zo7dq1ae8j0+sBAICuRSpgbd6c3fJcHHbYYR2nv/e976mqqkqNjY167rnnuuyI6tevX8fp3r17a/fu3T26TrY2bdqku+++W0uXLtWqVat0wQUXpB1jptcDAKBU1a2uU/m95ep1Ry+V31uuutV5mCuUgUgFrGHDslselNbWVg0ZMkSS9MgjjwR+/yeddJLeeecdNTU1SZIWLVqU9npTp07Vz3/+c0lSY2OjVq1aJUn64IMPdNhhh2nw4MHatm2bnn/++Y7bDBo0SDt27Djo9QAAKHV1q+s057k5am5tlsvV3NqsOc/NKUrIilTAqqmROh8MN2BAYnk+3Xzzzbr11ls1YcKEQLY4dXbooYfqgQce0MyZMzVp0iQNGjRIgwcPPuB61157rT788EONGTNGt99+uyZNmiRJOvXUUzVhwgSNHj1aX/3qVzVlypSO28yZM0czZ85UVVVVt9cDAKDUzV86X2279p8r1LarTfOX5mGu0EFY+1FqpaCystI79zatW7dOY8aMyfg+6uoSc642b05suaqpCX6CezF8+OGHGjhwoNxd1113nUaNGqUbb7yxaOPJ9nUBACDfet3RS64Dc43JtPf7ewNfn5mtdPe0fUyR2oIlJcJUU5O0d2/idxTClSQ99NBDqqio0Lhx49Ta2qprrrmm2EMCAKCkDBucfk5QV8vzKXIBK6puvPFGNTQ0aO3ataqrq6MYFACATmqm12hA3/3/fRzQd4Bqpud5rlAaBCwAABAJ1adUq/bCWg0fPFwm0/DBw1V7Ya2qTyn87qxIFY0CAIBoqltdp/lL52tz62YNGzxMNdNr0gan6lOqixKoOiNgAQCAktZev9B+hGB7/YKkkghT6bCLEAAAlLRSql/IVFYBy8weNrP3zKwxZdmRZvaSmW1I/j4iudzM7D4z22hmq8xsYtf3XLq2b9+uiooKVVRU6LjjjtOQIUM6zu/cufOgt3/11Ve1fPnyjvMLFy7UY489Fvg4U79YuisNDQ1avHhx4OsGACCfNrem/0qWrpaXgmy3YD0iaWanZd+VtNTdR0lamjwvSZ+XNCr5M0fSgz0fZvEcddRRamhoUENDg+bOndtxNF9DQ4MOOeSQg96+c8CaO3eurrjiinwOuUsELABAGJVS/UKmsgpY7r5M0vudFl8k6dHk6UclXZyy/DFPeF3S4WZ2fC6DzUQhvoNo5cqVOvvsszVp0iSdd9552rp1qyTpvvvu09ixYzV+/HjNnj1bTU1NWrhwoe655x5VVFTotdde04IFC3T33XdLkqZNm6ZbbrlFkydP1oknnqjXXntNktTW1qYvf/nLGjt2rGbNmqXTTjtNnQtYJemFF17Q6NGjNXHiRP3yl7/sWL5ixQqdccYZmjBhgs4880ytX79eO3fu1O23365FixapoqJCixYtSns9AABKTSnVL2QqiEnux7r71uTp/5Z0bPL0EEl/TLneluSyrSnLZGZzlNjCpWE5fmlgISbBubu+/e1v65lnnlFZWZkWLVqk+fPn6+GHH9add96pTZs2qV+/fvrLX/6iww8/XHPnztXAgQN10003SZKWLl263/3t3r1bK1as0OLFi3XHHXdoyZIleuCBB3TEEUdo7dq1amxsVEVFxQHj+Pjjj/XNb35TL7/8sj7zmc/oK1/5Ssdlo0eP1muvvaY+ffpoyZIluu222/TUU0/pBz/4gerr6/WTn/xEUuK7B9NdDwCAUtL+b3gmRxGWikCPInR3N7OsvnvH3Wsl1UqJr8rJZf3dTYIL6kX45JNP1NjYqHPOOUeStGfPHh1/fGLD3Pjx41VdXa2LL75YF198cXd30+GSSy6RJE2aNKnjy5x/+9vf6oYbbpAknXzyyRo/fvwBt3vrrbc0YsQIjRo1SpJ0+eWXq7a2VlLiy6evvPJKbdiwQWamXbt2pV13ptcDACAfMq1ekEqnfiFTQRxFuK1911/y93vJ5e9K+lTK9YYml+VNISbBubvGjRvXMQ9r9erVevHFFyVJv/nNb3TdddfpjTfe0Gc/+9mMvvi5X79+kqTevXsH9kXR3/ve91RVVaXGxkY999xz+vjjj3O6HgAAQWvf69Tc2iyXd+x1ysfUnmIIImA9K+nK5OkrJT2TsvyK5NGEp0tqTdmVmBeFmATXr18/tbS06He/+50kadeuXVqzZo327t2rP/7xj6qqqtJdd92l1tZWffjhhxo0aJB27NiR1TqmTJmiJ554QpK0du1arV69+oDrjB49Wk1NTXr77bclSY8//njHZa2trRoyZIgk6ZFHHulY3nksXV0PAIB8C2P1QjayrWl4XNLvJJ1kZlvM7OuS7pR0jpltkDQjeV6SFkt6R9JGSQ9J+lZgo+5CISbB9erVS08++aRuueUWnXrqqaqoqNDy5cu1Z88eXX755TrllFM0YcIEzZs3T4cffrguvPBCPf300x2T3DPxrW99Sy0tLRo7dqz+4R/+QePGjdPgwYP3u07//v1VW1urCy64QBMnTtQxxxzTcdnNN9+sW2+9VRMmTNhvq1hVVZXWrl3bMcm9q+sBAJBvYaxeyIa55zTtKVCVlZXe+Wi5devWacyYMRnfRzb7c0vVnj17tGvXLvXv319vv/22ZsyYofXr12dUC1Eo2b4uAACkKr+3XM2tzQcsHz54uJq+01T4AfWAma1098p0l0Xuq3LCNgkunba2NlVVVWnXrl1ydz3wwAMlFa4AAMhVzfSa/Y78l0q/eiEbkQtYUTBo0KC0vVcAAERFGKsXskHAAgAAgcp0uk4U9jp1hYAFAAACU4jS7zAIoqYBAABAUvTrFzJFwAIAAIGJev1CpghYGejdu7cqKip08skn69JLL1VbW9vBb9SFq666Sk8++aQk6Rvf+IbWrl3b5XVfffVVLV++vOP8woUL9dhjj/V43QAA5FshSr/DgICVgUMPPVQNDQ1qbGzUIYccooULF+53eU9LOv/lX/5FY8eO7fLyzgFr7ty5uuKKK3q0LgAACqEQpd9hEL2AVVcnlZdLvXolftcF+51GZ511ljZu3KhXX31VZ511lr74xS9q7Nix2rNnj/7+7/9en/3sZzV+/Hj99Kc/lZT47sLrr79eJ510kmbMmKH33nuv476mTZvWUcfwwgsvaOLEiTr11FM1ffp0NTU1aeHChbrnnns6WuAXLFigu+++W5LU0NCg008/XePHj9esWbP0P//zPx33ecstt2jy5Mk68cQTO9rj16xZo8mTJ6uiokLjx4/Xhg0bAn1eAACQEhPZay+s1fDBw2UyDR88XLUX1sZqgrsUtaMI6+qkOXOk9l14zc2J85JUnfsLu3v3bj3//POaOXOmJOmNN95QY2OjRowYodraWg0ePFi///3v9cknn2jKlCk699xz9eabb2r9+vVau3attm3bprFjx+rqq6/e735bWlr0zW9+U8uWLdOIESP0/vvv68gjj9TcuXM1cOBA3XTTTZKkpUuXdtzmiiuu0P3336+zzz5bt99+u+644w7de++9HeNcsWKFFi9erDvuuENLlizRwoULdcMNN6i6ulo7d+7Unj17cn4+AADxQv1C5qK1BWv+/H3hql1bW2J5Dj766CNVVFSosrJSw4YN09e//nVJ0uTJkzVixAhJ0osvvqjHHntMFRUVOu2007R9+3Zt2LBBy5Yt02WXXabevXvrhBNO0Oc+97kD7v/111/X1KlTO+7ryCOP7HY8ra2t+stf/qKzzz5bknTllVdq2bJlHZdfcsklkqRJkyapqalJknTGGWfon/7pn3TXXXepublZhx56aE7PCQAgXtrrF5pbm+XyjvqFutXB7imKimgFrM1dHKHQ1fIMtc/Bamho0P3339/xtTWHHXZYx3XcXffff3/H9TZt2qRzzz03p/X2VL9+/SQlJue3zw/76le/qmeffVaHHnqozj//fL388stFGRsAIJyoX8hOtALWsC6OUOhqeYDOO+88Pfjgg9q1a5ck6Q9/+IP++te/aurUqVq0aJH27NmjrVu36pVXXjngtqeffrqWLVumTZs2SZLef/99SYmvzNmxY8cB1x88eLCOOOKIjvlVP/vZzzq2ZnXlnXfe0ciRIzVv3jxddNFFWrVqVU6PFwAQL9QvZCdac7BqavafgyVJAwYklufZN77xDTU1NWnixIlyd5WVlelXv/qVZs2apZdfflljx47VsGHDdMYZZxxw27KyMtXW1uqSSy7R3r17dcwxx+ill17ShRdeqC996Ut65plndP/99+93m0cffVRz585VW1ubRo4cqX/7t3/rdnxPPPGEfvazn6lv37467rjjdNtttwX6+AEA0TZs8DA1tzanXY4DmbsXewwdKisrvfOXHK9bt05jxozJ/E7q6hJzrjZvTmy5qqkJZII79pf16wIACLXOX4EjJeoX4niEYDszW+nulekui9YWLCkRpghUAAAEqj1EZXIUIaIYsAAAQMYyrV6QqF/IRigClrvLzIo9DCSV0m5lAEDPdd7t1169IIkglaOSP4qwf//+2r59O/+olwh31/bt29W/f/9iDwUAkCOqF/Kn5LdgDR06VFu2bFFLS0uxh4Kk/v37a+jQocUeBgAgR1Qv5E/JB6y+fft2NJwDAIDgUL2QPyW/ixAAAORHzfQaDeg7YL9lA/oOUM30/PdHRh0BCwCAmKo+pVq1F9Zq+ODhMpmGDx4e616rIJV80SgAAMheNvUL6Jl4FY0CABBz1C8UH7sIAQCIGOoXio+ABQBAxFC/UHwELAAAIqarmgXqFwqHgAUAQMRQv1B8BCwAACKG+oXio6YBAICQoHqhtFDTAABAyFG9EC7sIgQAIASoXggXAhYAACFA9UK4ELAAAAgBqhfCJeeAZWYnmVlDys8HZvYdM1tgZu+mLD8/iAEDABBHVC+ES84By93Xu3uFu1dImiSpTdLTyYvvab/M3Rfnui4AAOKK6oVwCfoowumS3nb3ZjML+K4BAIimTOsXqk+pJlCFRNBzsGZLejzl/PVmtsrMHjazI9LdwMzmmFm9mdW3tLQEPBwAAEpbe/1Cc2uzXN5Rv1C3uq7YQ0MOAisaNbNDJP1J0jh332Zmx0r6sySX9I+Sjnf3q7u7D4pGAQBxU35vuZpbmw9YPnzwcDV9p6nwA0LGuisaDXIL1uclveHu2yTJ3be5+x533yvpIUmTA1wXAACRQP1CNAUZsC5Tyu5BMzs+5bJZkhoDXBcAAJFA/UI0BRKwzOwwSedI+mXK4h+a2WozWyWpStKNQawLAIAooX4hmgI5itDd/yrpqE7LvhbEfQMAEGXtRwXyJc7REtgk9yAwyR0AECWZ1i8gnLqb5B50DxYAANC++oX2L2hur1+QRMiKAb6LEACAPJi/dH5HuGrXtqtN85fOL9KIUEgELAAA8oD6hXgjYAEAkAfUL8QbAQsAgDygfiHeCFgAAORB9SnVqr2wVsMHD5fJNHzwcNVeWMsE95igpgEAgCzU1Unz50ubN0vDhkk1NVI1mSmWqGkAACAAdXXSnDlSW/LgwObmxHmJkIX9sYsQAIAMzZ+/L1y1a2tLLAdSEbAAAMjQ5i4aFrpajvgiYAEAkKFhXTQsdLUc8UXAAgAgQzU10oD9mxc0YEBiOZCKgAUAQIaqq6XaWmn4cMks8bu2lgnuOBABCwAAJY4QLC+XevVK/K6rS3+96mqpqUnauzfxm3CFdKhpAADEHvULCBpbsAAAsUf9AoJGwAIAxB71CwgaAQsAEHvULyBoBCwAQOxRv4CgEbAAALFH/QKCRsACAEQa9QsoBmoaAACRRf0CioUtWACAyKJ+AcVCwAIARBb1CygWAhYAILKoX0CxELAAAJFF/QKKhYAFAIgs6hdQLAQsAEDoZFq9IFG/gOKgpgEAECpULyAM2IIFAAgVqhcQBgQsAECoUL2AMCBgAQBCheoFhAEBCwAQKlQvIAwIWACAUKF6AWEQWMAysyYzW21mDWZWn1x2pJm9ZGYbkr+PCGp9AIDoybR+geoFlLqgt2BVuXuFu1cmz39X0lJ3HyVpafI8AAAHaK9faG6W3PfVL3TXcQWUqnzvIrxI0qPJ049KujjP6wMAhBT1C4iSIAOWS3rRzFaaWbLyTce6+9bk6f+WdGznG5nZHDOrN7P6lpaWAIcDAAgT6hcQJUEGrL9194mSPi/pOjObmnqhu7sSIUydlte6e6W7V5aVlQU4HABAmFC/gCgJLGC5+7vJ3+9JelrSZEnbzOx4SUr+fi+o9QEAooX6BURJIAHLzA4zs0HtpyWdK6lR0rOSrkxe7UpJzwSxPgBA9FC/gCgJagvWsZJ+a2b/T9IKSb9x9xck3SnpHDPbIGlG8jwAIGaoX0Dc9AniTtz9HUmnplm+XdL0INYBAAin9vqF9iME2+sXJAIUoosmdwBAXlG/gDgiYAEA8or6BcQRAQsAkFfULyCOCFgAgLyifgFxRMACAOQV9QuIo0COIgQAoDvV1QQqxAtbsAAAPZJptxUQR2zBAgBkjW4roHtswQIAZI1uK6B7BCwAQNbotgK6R8ACAGSNbiugewQsAEDW6LYCukfAAgBkjW4roHsELADAfjKtX6iulpqapL17E78JV8A+1DQAADpQvwAEgy1YAIAO1C8AwSBgAQA6UL8ABIOABQDoQP0CEAwCFgCgA/ULQDAIWACADtQvAMEgYAFATFC/ABQONQ0AEAPULwCFxRYsAIgB6heAwiJgAUAMUL8AFBYBCwBigPoFoLAIWAAQA9QvAIVFwAKAGKB+ASgsAhYAhFim1QsS9QtAIVHTAAAhRfUCULrYggUAIUX1AlC6CFgAEFJULwCli4AFACFF9QJQughYABBSVC8ApYuABQAhRfUCULoIWABQgjKtX6B6AShNOQcsM/uUmb1iZmvNbI2Z3ZBcvsDM3jWzhuTP+bkPFwCir71+oblZct9Xv9BdxxWA0mLuntsdmB0v6Xh3f8PMBklaKeliSV+W9KG7353pfVVWVnp9fX1O4wGAsCsvT4SqzoYPT2ylAlAazGylu1emuyznolF33yppa/L0DjNbJ2lIrvcLAHFF/QIQfoHOwTKzckkTJP1XctH1ZrbKzB42syOCXBcARBX1C0D4BRawzGygpKckfcfdP5D0oKRPS6pQYgvXj7q43Rwzqzez+paWlqCGAwChRf0CEH6BBCwz66tEuKpz919Kkrtvc/c97r5X0kOSJqe7rbvXunulu1eWlZUFMRwACDXqF4AcZPMN6HkUxFGEJulfJa1z9x+nLD8+5WqzJDXmui4ACDvqF4AeyuTDU0KH4AaxBWuKpK9J+lynSoYfmtlqM1slqUrSjQGsCwBCq4T+9gOlIdP/cWT64Smhb0DPuaYhSNQ0AIgy6heAFO2hKTUQDRiQfn94ph+eXr0SAawzs8Tm4IB1V9NAkzsAFAj1C4iNTLZMZbO1KdMPTwkdgkvAAoACKaG//UDPBDkPKpv/cWT64SmhQ3AJWABQICX0tx/Yp1jzoLL5H0emH54SOgSXOVgAUEB1dYl/ZzZvTvw7UlPDEYIoomLOg8pm3e3XL7EPD3OwACCPsqndoX4BBVPq86Cy3doUsg8PAQsAckD1Agoq6N15xZ4HFbLQlA0CFgDkoIRqdxBmQZdoMg+q6JiDBQA5KHDtDqIo07lI2RSpxWgeVDExBwsA8oTqBXQryHlQ+didF/F5UMVEwAKAHFC9gC4FPQ8qH7vzJEJTnhCwACAHTDdBl4KeB5VtaOKNWVQELADoQqYHbLEBAGllumUqX5PHeWMWVZ9iDwAASlHnub/te3ck/p1ChoYNSz8pPd08KCmzyePV1bwBQ4KjCAEgjWwO2ALSyvYIPYQORxECQJayOWALSIt5ULHGLkIASCPTvTtAt9ilF1tswQKANKhfAJALAhYApMHeHQC5IGABiB3qFwDkG3OwAMQK9QsACoEtWABiJdNybQDIBQELQKxQvwCgEAhYAGIlm+/LBYCeImABiBXqFwAUAgELQKxQvwCgEAhYACIh0+oFifoFAPlHTQOA0KN6AUCpYQsWgNCjegFAqSFgAQg9qhcAlBoCFoDQo3oBQKkhYAEIPaoXAJQaAhaA0KN6AUCpIWABKGmZ1i9QvQCglFDTAKBkUb8AIKzYggWgZFG/ACCsCFgAShb1CwDCKu8By8xmmtl6M9toZt/N9/oARAf1CwDCKq8By8x6S/o/kj4vaayky8xsbD7XCSA6qF8AEFb53oI1WdJGd3/H3XdK+oWki/K8TgARQf0CgLDKd8AaIumPKee3JJd1MLM5ZlZvZvUtLS15Hg6AUpBp9YJE/QKAcCr6JHd3r3X3SnevLCsrK/ZwAORZe/VCc7Pkvq96obuQBQBhk++A9a6kT6WcH5pcBiCmqF4AEAf5Dli/lzTKzEaY2SGSZkt6Ns/rBFDCqF4AEAd5DVjuvlvS9ZL+XdI6SU+4+5p8rhNAaaN6AUAc5H0OlrsvdvcT3f3T7s7B1UDMUb0AIA6KPskdQLxQvQAgDghYAAKTaf0C1QsAoq5PsQcAIBra6xfajxBsr1+QCFAA4octWAACQf0CAOxDwAIQCOoXAGAfAhaAQFC/AAD7ELAABIL6BQDYh4AFIBDULwDAPgQsAAdF/QIAZIeaBgDdon4BALLHFiwA3aJ+AQCyR8AC0C3qFwAgewQsAN2ifgEAskfAAtAt6hcAIHsELADdon4BALJHwAJiKtPqBYn6BQDIFjUNQAxRvQAA+cUWLCCGqF4AgPwiYAExRPUCAOQXAQuIIaoXACC/CFhADFG9AAD5RcACYojqBQDILwIWEDGZ1i9QvQAA+UNNAxAh1C8AQGlgCxYQIdQvAEBpIGABEUL9AgCUBgIWECHULwBAaSBgARFC/QIAlAYCFhAh1C8AQGkgYAEhQf0CAIQHNQ1ACFC/AADhwhYsIASoXwCAcCFgASFA/QIAhAsBCwgB6hcAIFwIWEAIUL8AAOGSU8Ays382s7fMbJWZPW1mhyeXl5vZR2bWkPxZGMxwgXiifgEAwsXcvec3NjtX0svuvtvM7pIkd7/FzMol/drdT87m/iorK72+vr7H4wEAACgUM1vp7pXpLstpC5a7v+juu5NnX5c0NJf7A+Im024rAEC4BDkH62pJz6ecH2Fmb5rZf5jZWV3dyMzmmFm9mdW3tLQEOBygtLV3WzU3S+77uq0IWQAQfgfdRWhmSyQdl+ai+e7+TPI68yVVSrrE3d3M+kka6O7bzWySpF9JGufuH3S3LnYRIk7KyxOhqrPhwxMN7ACA0tbdLsKDNrm7+4yD3PlVkr4gabon05q7fyLpk+TplWb2tqQTJZGegCS6rQAgunI9inCmpJslfdHd21KWl5lZ7+TpkZJGSXonl3UBUUO3FQBEV65zsH4iaZCklzrVMUyVtMrMGiQ9KWmuu7+f47qASKHbCgCiK6cve3b3z3Sx/ClJT+Vy30DUtXdYzZ+f2C04bFgiXNFtBQDhR5M7kAeZ1i9UVycmtO/dm/hNuAKAaMhpCxaAA7XXL7QlZyW21y9IBCgAiAu2YAEBmz9/X7hq19aWWA4AiAcCFhAw6hcAAAQsIGDULwAACFhAwKhfAAAQsICAVVdLtbRggM8AAAxQSURBVLWJr7wxS/yurWWCOwDECQELyAL1CwCATFDTAGSI+gUAQKbYggVkiPoFAECmCFhAhqhfAABkioAFZIj6BQBApghYQIaoXwAAZIqABWSI+gUAQKYIWIi9TKsXJOoXAACZoaYBsUb1AgAgH9iChVijegEAkA8ELMQa1QsAgHwgYCHWqF4AAOQDAQuxRvUCACAfCFiINaoXAAD5QMBCZGVav0D1AgAgaNQ0IJKoXwAAFBNbsBBJ1C8AAIqJgIVIon4BAFBMBCxEEvULAIBiImAhkqhfAAAUEwELkUT9AgCgmAhYCB3qFwAApY6aBoQK9QsAgDBgCxZChfoFAEAYELAQKtQvAADCgICFUKF+AQAQBgQshAr1CwCAMCBgIVSoXwAAhEFOAcvMFpjZu2bWkPw5P+WyW81so5mtN7Pzch8qoizT6gWJ+gUAQOkLoqbhHne/O3WBmY2VNFvSOEknSFpiZie6+54A1oeIoXoBABA1+dpFeJGkX7j7J+6+SdJGSZPztC6EHNULAICoCSJgXW9mq8zsYTM7IrlsiKQ/plxnS3LZAcxsjpnVm1l9S0tLAMNB2FC9AACImoMGLDNbYmaNaX4ukvSgpE9LqpC0VdKPsh2Au9e6e6W7V5aVlWX9ABB+VC8AAKLmoHOw3H1GJndkZg9J+nXy7LuSPpVy8dDkMuAANTX7z8GSqF4AAIRbrkcRHp9ydpakxuTpZyXNNrN+ZjZC0ihJK3JZF6KL6gUAQNTkOgfrh2a22sxWSaqSdKMkufsaSU9IWivpBUnXcQRhPGVav0D1AgAgSnKqaXD3r3VzWY0kdvLEGPULAIC4oskdeUP9AgAgrghYyBvqFwAAcUXAQt5QvwAAiCsCFvKmpiZRt5CK+gUAQBwQsJA31C8AAOKKgIUeoX4BAICu5VTTgHiifgEAgO6xBQtZo34BAIDuEbCQNeoXAADoHgELWaN+AQCA7hGwkDXqFwAA6B4BC1mjfgEAgO4RsNAh0+oFifoFAAC6Q00DJFG9AABAkNiCBUlULwAAECQCFiRRvQAAQJAIWJBE9QIAAEEiYEES1QsAAASJgAVJVC8AABAkAlYMZFq/QPUCAADBoKYh4qhfAACg8NiCFXHULwAAUHgErIijfgEAgMIjYEUc9QsAABQeASviqF8AAKDwCFgRR/0CAACFR8AKqUyrFyTqFwAAKDRqGkKI6gUAAEobW7BCiOoFAABKGwErhKheAACgtBGwQojqBQAAShsBK4SoXgAAoLQRsEKI6gUAAEobAavEZFq/QPUCAACli5qGEkL9AgAA0ZDTFiwzW2RmDcmfJjNrSC4vN7OPUi5bGMxwo436BQAAoiGnLVju/pX202b2I0mtKRe/7e4Vudx/3FC/AABANAQyB8vMTNKXJT0exP3FFfULAABEQ1CT3M+StM3dN6QsG2Fmb5rZf5jZWV3d0MzmmFm9mdW3tLQENJxwon4BAIBoOGjAMrMlZtaY5ueilKtdpv23Xm2VNMzdJ0j6O0k/N7O/SXf/7l7r7pXuXllWVpbLYwk96hcAAIiGgwYsd5/h7ien+XlGksysj6RLJC1Kuc0n7r49eXqlpLclnZifhxAO1C8AABAfQdQ0zJD0lrtvaV9gZmWS3nf3PWY2UtIoSe8EsK5Qon4BAIB4CWIO1mwdOLl9qqRVydqGJyXNdff3A1hXKFG/AABAvOS8Bcvdr0qz7ClJT+V631FB/QIAAPHCV+UUAPULAADECwGrAKhfAAAgXghYBUD9AgAA8ULAykGm1QsS9QsAAMRJEDUNsUT1AgAA6ApbsHqI6gUAANAVAlYPUb0AAAC6QsDqIaoXAABAVwhYPUT1AgAA6AoBq4eoXgAAAF0hYKWRaf0C1QsAACAdaho6oX4BAADkii1YnVC/AAAAckXA6oT6BQAAkCsCVifULwAAgFwRsDqhfgEAAOSKgNUJ9QsAACBXHEWYRnU1gQoAAPRcrLZgZdpvBQAAkIvYbMGi3woAABRKbLZg0W8FAAAKJTYBi34rAABQKLEJWPRbAQCAQolNwKLfCgAAFEpsAhb9VgAAoFBicxShRL8VAAAojNhswQIAACgUAhYAAEDACFgAAAABI2ABAAAEjIAFAAAQMAIWAABAwAhYAAAAASNgAQAABIyABQAAEDACFgAAQMAIWAAAAAEjYAEAAATM3L3YY+hgZi2SmguwqqMl/bkA6ylVcX/8Es+BxHMg8RzE/fFLPAcSz0Euj3+4u5elu6CkAlahmFm9u1cWexzFEvfHL/EcSDwHEs9B3B+/xHMg8Rzk6/GzixAAACBgBCwAAICAxTVg1RZ7AEUW98cv8RxIPAcSz0HcH7/EcyDxHOTl8cdyDhYAAEA+xXULFgAAQN4QsAAAAAIW6YBlZpea2Roz22tmlZ0uu9XMNprZejM7L2X5zOSyjWb23cKPOn/MbJGZNSR/msysIbm83Mw+SrlsYbHHmi9mtsDM3k15rOenXJb2PRElZvbPZvaWma0ys6fN7PDk8ti8B6Rof867YmafMrNXzGxt8u/iDcnlXX4moib5d2918nHWJ5cdaWYvmdmG5O8jij3OfDGzk1Je5wYz+8DMvhP194CZPWxm75lZY8qytK+7JdyX/Nuwyswm9ni9UZ6DZWZjJO2V9FNJN7l7+wdqrKTHJU2WdIKkJZJOTN7sD5LOkbRF0u8lXebuaws89Lwzsx9JanX3H5hZuaRfu/vJxR1V/pnZAkkfuvvdnZanfU+4+56CDzKPzOxcSS+7+24zu0uS3P2WmL0Heismn/NUZna8pOPd/Q0zGyRppaSLJX1ZaT4TUWRmTZIq3f3PKct+KOl9d78zGbaPcPdbijXGQkl+Dt6VdJqk/6UIvwfMbKqkDyU91v43rqvXPRkuvy3pfCWem//t7qf1ZL2R3oLl7uvcfX2aiy6S9At3/8TdN0naqMQ/rJMlbXT3d9x9p6RfJK8bKWZmSvxRfbzYYykhXb0nIsXdX3T33cmzr0saWszxFEksPueduftWd38jeXqHpHWShhR3VCXhIkmPJk8/qkTojIPpkt5290J8e0pRufsySe93WtzV636REkHM3f11SYcn/3OStUgHrG4MkfTHlPNbksu6Wh41Z0na5u4bUpaNMLM3zew/zOysYg2sQK5Pbvp9OGV3QFxe+1RXS3o+5Xxc3gNxfK33k9xiOUHSfyUXpftMRJFLetHMVprZnOSyY919a/L0f0s6tjhDK7jZ2v8/2XF5D7Tr6nUP7O9D6AOWmS0xs8Y0P5H/H2k6GT4fl2n/D9ZWScPcfYKkv5P0czP7m0KOO0gHeQ4elPRpSRVKPO4fFXWweZDJe8DM5kvaLakuuShS7wF0zcwGSnpK0nfc/QPF4DOR4m/dfaKkz0u6LrnrqIMn5sxEd95MkpkdIumLkv5vclGc3gMHyNfr3ifoOyw0d5/Rg5u9K+lTKeeHJpepm+WhcLDnw8z6SLpE0qSU23wi6ZPk6ZVm9rYSc9Lq8zjUvMn0PWFmD0n6dfJsd++JUMngPXCVpC9Imp78wxK598BBROa1zpaZ9VUiXNW5+y8lyd23pVye+pmIHHd/N/n7PTN7WondxdvM7Hh335rcFfReUQdZGJ+X9Eb7ax+n90CKrl73wP4+hH4LVg89K2m2mfUzsxGSRklaocRk11FmNiKZ8GcnrxslMyS95e5b2heYWVlywqPMbKQSz8c7RRpfXnXalz5LUvtRJV29JyLFzGZKulnSF929LWV5bN4Disfn/ADJuZf/Kmmdu/84ZXlXn4lIMbPDkpP7ZWaHSTpXicf6rKQrk1e7UtIzxRlhQe23FyMu74FOunrdn5V0RfJowtOVOBhsa7o7OJjQb8HqjpnNknS/pDJJvzGzBnc/z93XmNkTktYqsZvkuvajxczsekn/Lqm3pIfdfU2Rhp8vnfe7S9JUST8ws11KHHU51907TwiMih+aWYUSm4ObJF0jSd29JyLmJ5L6SXop8e+tXnf3uYrReyB5BGXUP+fpTJH0NUmrLVnRIuk2SZel+0xE0LGSnk6+7/tI+rm7v2Bmv5f0hJl9XVKzEgcARVYyXJ6j/V/ntH8Xo8LMHpc0TdLRZrZF0vcl3an0r/tiJY4g3CipTYkjLHu23ijXNAAAABRDXHcRAgAA5A0BCwAAIGAELAAAgIARsAAAAAJGwAIAAAgYAQsAACBgBCwAAICA/X/LWEgf8L8BIAAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 720x504 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "38Ki8ZHcsztG"
      },
      "source": [
        "Strange, we trained for longer but our model performed worse?\n",
        "\n",
        "As it turns out, our model might've trained too long and has thus resulted in worse results (we'll see ways to prevent training for too long later on)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BPTUcFe4sbfk",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4cba4687-3bf7-479d-f901-3c4d5cef0dfc"
      },
      "source": [
        "# Calculate model_3 metrics\n",
        "mae_3 = mae(y_test, y_preds_3.squeeze()).numpy()\n",
        "mse_3 = mse(y_test, y_preds_3.squeeze()).numpy()\n",
        "mae_3, mse_3"
      ],
      "execution_count": 73,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(68.68786, 4804.4717)"
            ]
          },
          "metadata": {},
          "execution_count": 73
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UPEeM3UsrxGB"
      },
      "source": [
        "## Comparing results\n",
        "\n",
        "Now we've got results for 3 similar but slightly different results, let's compare them."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mw5RZk-BqLZd"
      },
      "source": [
        "model_results = [[\"model_1\", mae_1, mse_1],\n",
        "                 [\"model_2\", mae_2, mse_2],\n",
        "                 [\"model_3\", mae_3, mae_3]]"
      ],
      "execution_count": 74,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ip7bKH83p5X0",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 142
        },
        "outputId": "200a4d09-1220-4d14-8327-e3d22f768814"
      },
      "source": [
        "import pandas as pd\n",
        "all_results = pd.DataFrame(model_results, columns=[\"model\", \"mae\", \"mse\"])\n",
        "all_results"
      ],
      "execution_count": 75,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>model</th>\n",
              "      <th>mae</th>\n",
              "      <th>mse</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>model_1</td>\n",
              "      <td>18.745327</td>\n",
              "      <td>353.573364</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>model_2</td>\n",
              "      <td>1.909811</td>\n",
              "      <td>5.459232</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>model_3</td>\n",
              "      <td>68.687859</td>\n",
              "      <td>68.687859</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "     model        mae         mse\n",
              "0  model_1  18.745327  353.573364\n",
              "1  model_2   1.909811    5.459232\n",
              "2  model_3  68.687859   68.687859"
            ]
          },
          "metadata": {},
          "execution_count": 75
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o_AtUiwuuLGo"
      },
      "source": [
        "From our experiments, it looks like `model_2` performed the best.\n",
        "\n",
        "And now, you might be thinking, \"wow, comparing models is tedious...\" and it definitely can be, we've only compared 3 models here. \n",
        "\n",
        "But this is part of what machine learning modelling is about, trying many different combinations of models and seeing which performs best.\n",
        "\n",
        "Each model you build is a small experiment. \n",
        "\n",
        "> 🔑 **Note:** One of your main goals should be to minimize the time between your experiments. The more experiments you do, the more things you'll figure out which don't work and in turn, get closer to figuring out what does work. Remember the machine learning practitioner's motto: \"experiment, experiment, experiment\".\n",
        "\n",
        "Another thing you'll also find is what you thought may work (such as training a model for longer) may not always work and the exact opposite is also often the case.\n",
        "\n",
        "## Tracking your experiments\n",
        "\n",
        "One really good habit to get into is tracking your modelling experiments to see which perform better than others.\n",
        "\n",
        "We've done a simple version of this above (keeping the results in different variables).\n",
        "\n",
        "> 📖 **Resource:** But as you build more models, you'll want to look into using tools such as:\n",
        "* [**TensorBoard**](https://tensorboard.dev/) - a component of the TensorFlow library to help track modelling experiments (we'll see this later).\n",
        "* [**Weights & Biases**](https://www.wandb.com/) - a tool for tracking all kinds of machine learning experiments (the good news for Weights & Biases is it plugs into TensorBoard)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Fe5DgNbX6192"
      },
      "source": [
        "## Saving a model\n",
        "\n",
        "Once you've trained a model and found one which performs to your liking, you'll probably want to save it for use elsewhere (like a web application or mobile device).\n",
        "\n",
        "You can save a TensorFlow/Keras model using [`model.save()`](https://www.tensorflow.org/tutorials/keras/save_and_load#save_the_entire_model).\n",
        "\n",
        "There are two ways to save a model in TensorFlow:\n",
        "1. The [SavedModel format](https://www.tensorflow.org/tutorials/keras/save_and_load#savedmodel_format) (default).\n",
        "2. The [HDF5 format](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format).\n",
        "\n",
        "The main difference between the two is the SavedModel is automatically able to save custom objects (such as special layers) without additional modifications when loading the model back in.\n",
        "\n",
        "Which one should you use?\n",
        "\n",
        "It depends on your situation but the SavedModel format will suffice most of the time.\n",
        "\n",
        "Both methods use the same method call."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gg0jD2cUoPsg",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "bde421a5-d4b1-4d8e-9212-c7824f14dcc7"
      },
      "source": [
        "# Save a model using the SavedModel format\n",
        "model_2.save('best_model_SavedModel_format')"
      ],
      "execution_count": 76,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "INFO:tensorflow:Assets written to: best_model_SavedModel_format/assets\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dsCpDYrU7D1j",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7f1b842b-cebb-46a7-ff9c-f6017fabf8eb"
      },
      "source": [
        "# Check it out - outputs a protobuf binary file (.pb) as well as other files\n",
        "!ls best_model_SavedModel_format"
      ],
      "execution_count": 77,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "assets\tkeras_metadata.pb  saved_model.pb  variables\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NGKmWco_SOEU"
      },
      "source": [
        "Now let's save the model in the HDF5 format, we'll use the same method but with a different filename."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "97J6GJMBSM2j"
      },
      "source": [
        "# Save a model using the HDF5 format\n",
        "model_2.save(\"best_model_HDF5_format.h5\") # note the addition of '.h5' on the end"
      ],
      "execution_count": 78,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vB7TmsSGSjdv",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a82e9aa8-2391-461c-d3d5-cf31c9d7eaed"
      },
      "source": [
        "# Check it out\n",
        "!ls best_model_HDF5_format.h5"
      ],
      "execution_count": 79,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "best_model_HDF5_format.h5\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OGA02tY97EUI"
      },
      "source": [
        "## Loading a model\n",
        "\n",
        "We can load a saved model using the [`load_model()`](https://www.tensorflow.org/api_docs/python/tf/keras/models/load_model) method.\n",
        "\n",
        "Loading a model for the different formats (SavedModel and HDF5) is the same (as long as the pathnames to the particular formats are correct)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FzyLIWfs7Fvh",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "9dbc900f-bc63-436b-ba1a-60cf7ed7d793"
      },
      "source": [
        "# Load a model from the SavedModel format\n",
        "loaded_saved_model = tf.keras.models.load_model(\"best_model_SavedModel_format\")\n",
        "loaded_saved_model.summary()"
      ],
      "execution_count": 80,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential_12\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " dense_13 (Dense)            (None, 1)                 2         \n",
            "                                                                 \n",
            " dense_14 (Dense)            (None, 1)                 2         \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 4\n",
            "Trainable params: 4\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MfGO0dCQTeQh"
      },
      "source": [
        "Now let's test it out."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7rehN8ZxTy43",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "6696ffb8-7470-41aa-a83e-778c41241dfa"
      },
      "source": [
        "# Compare model_2 with the SavedModel version (should return True)\n",
        "model_2_preds = model_2.predict(X_test)\n",
        "saved_model_preds = loaded_saved_model.predict(X_test)\n",
        "mae(y_test, saved_model_preds.squeeze()).numpy() == mae(y_test, model_2_preds.squeeze()).numpy()"
      ],
      "execution_count": 81,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 81
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "prjkfX6rUZ6a"
      },
      "source": [
        "Loading in from the HDF5 is much the same."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dQfx-bWKUfRQ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7983df59-27f5-4715-d3a4-6da88b055ebe"
      },
      "source": [
        "# Load a model from the HDF5 format\n",
        "loaded_h5_model = tf.keras.models.load_model(\"best_model_HDF5_format.h5\")\n",
        "loaded_h5_model.summary()"
      ],
      "execution_count": 82,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential_12\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " dense_13 (Dense)            (None, 1)                 2         \n",
            "                                                                 \n",
            " dense_14 (Dense)            (None, 1)                 2         \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 4\n",
            "Trainable params: 4\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "L0kT91h-Uru-",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "44af4a8c-7aab-468a-f6e3-e94f447610f8"
      },
      "source": [
        "# Compare model_2 with the loaded HDF5 version (should return True)\n",
        "h5_model_preds = loaded_h5_model.predict(X_test)\n",
        "mae(y_test, h5_model_preds.squeeze()).numpy() == mae(y_test, model_2_preds.squeeze()).numpy()"
      ],
      "execution_count": 83,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 83
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ABtsYBDtr5Zz"
      },
      "source": [
        "## Downloading a model (from Google Colab)\n",
        "\n",
        "Say you wanted to get your model from Google Colab to your local machine, you can do one of the following things:\n",
        "* Right click on the file in the files pane and click 'download'.\n",
        "* Use the code below."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JV0onjIIr9XC",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "328e1446-c184-40cc-d1bf-bb931b0c5cc6"
      },
      "source": [
        "# Download the model (or any file) from Google Colab\n",
        "from google.colab import files\n",
        "files.download(\"best_model_HDF5_format.h5\")"
      ],
      "execution_count": 84,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/javascript": [
              "\n",
              "    async function download(id, filename, size) {\n",
              "      if (!google.colab.kernel.accessAllowed) {\n",
              "        return;\n",
              "      }\n",
              "      const div = document.createElement('div');\n",
              "      const label = document.createElement('label');\n",
              "      label.textContent = `Downloading \"${filename}\": `;\n",
              "      div.appendChild(label);\n",
              "      const progress = document.createElement('progress');\n",
              "      progress.max = size;\n",
              "      div.appendChild(progress);\n",
              "      document.body.appendChild(div);\n",
              "\n",
              "      const buffers = [];\n",
              "      let downloaded = 0;\n",
              "\n",
              "      const channel = await google.colab.kernel.comms.open(id);\n",
              "      // Send a message to notify the kernel that we're ready.\n",
              "      channel.send({})\n",
              "\n",
              "      for await (const message of channel.messages) {\n",
              "        // Send a message to notify the kernel that we're ready.\n",
              "        channel.send({})\n",
              "        if (message.buffers) {\n",
              "          for (const buffer of message.buffers) {\n",
              "            buffers.push(buffer);\n",
              "            downloaded += buffer.byteLength;\n",
              "            progress.value = downloaded;\n",
              "          }\n",
              "        }\n",
              "      }\n",
              "      const blob = new Blob(buffers, {type: 'application/binary'});\n",
              "      const a = document.createElement('a');\n",
              "      a.href = window.URL.createObjectURL(blob);\n",
              "      a.download = filename;\n",
              "      div.appendChild(a);\n",
              "      a.click();\n",
              "      div.remove();\n",
              "    }\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.Javascript object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "application/javascript": [
              "download(\"download_e4f38d05-ab4b-4b6d-8e64-0d1717768680\", \"best_model_HDF5_format.h5\", 17872)"
            ],
            "text/plain": [
              "<IPython.core.display.Javascript object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7xpVdMtKw6X4"
      },
      "source": [
        "## A larger example\n",
        "\n",
        "Alright, we've seen the fundamentals of building neural network regression models in TensorFlow.\n",
        "\n",
        "Let's step it up a notch and build a model for a more feature rich dataset.\n",
        "\n",
        "More specifically we're going to try predict the cost of medical insurance for individuals based on a number of different parameters such as, `age`, `sex`, `bmi`, `children`, `smoking_status` and `residential_region`.\n",
        "\n",
        "To do, we'll leverage the pubically available [Medical Cost dataset](https://www.kaggle.com/mirichoi0218/insurance) available from Kaggle and [hosted on GitHub](https://github.com/stedy/Machine-Learning-with-R-datasets/blob/master/insurance.csv).\n",
        "\n",
        "> 🔑 **Note:** When learning machine learning paradigms, you'll often go through a series of foundational techniques and then practice them by working with open-source datasets and examples. Just as we're doing now, learn foundations, put them to work with different problems. Every time you work on something new, it's a good idea to search for something like \"problem X example with Python/TensorFlow\" where you substitute X for your problem."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WWK1LBxapgc2"
      },
      "source": [
        "# Import required libraries\n",
        "import tensorflow as tf\n",
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt"
      ],
      "execution_count": 85,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JrnTr5N9blFo"
      },
      "source": [
        "# Read in the insurance dataset\n",
        "insurance = pd.read_csv(\"https://raw.githubusercontent.com/stedy/Machine-Learning-with-R-datasets/master/insurance.csv\")"
      ],
      "execution_count": 86,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QtXPN7cfb4Nm",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "d859454e-45c9-4e6a-e919-65edf4cd139b"
      },
      "source": [
        "# Check out the insurance dataset\n",
        "insurance.head()"
      ],
      "execution_count": 87,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>sex</th>\n",
              "      <th>bmi</th>\n",
              "      <th>children</th>\n",
              "      <th>smoker</th>\n",
              "      <th>region</th>\n",
              "      <th>charges</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>19</td>\n",
              "      <td>female</td>\n",
              "      <td>27.900</td>\n",
              "      <td>0</td>\n",
              "      <td>yes</td>\n",
              "      <td>southwest</td>\n",
              "      <td>16884.92400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>18</td>\n",
              "      <td>male</td>\n",
              "      <td>33.770</td>\n",
              "      <td>1</td>\n",
              "      <td>no</td>\n",
              "      <td>southeast</td>\n",
              "      <td>1725.55230</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>28</td>\n",
              "      <td>male</td>\n",
              "      <td>33.000</td>\n",
              "      <td>3</td>\n",
              "      <td>no</td>\n",
              "      <td>southeast</td>\n",
              "      <td>4449.46200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>33</td>\n",
              "      <td>male</td>\n",
              "      <td>22.705</td>\n",
              "      <td>0</td>\n",
              "      <td>no</td>\n",
              "      <td>northwest</td>\n",
              "      <td>21984.47061</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>32</td>\n",
              "      <td>male</td>\n",
              "      <td>28.880</td>\n",
              "      <td>0</td>\n",
              "      <td>no</td>\n",
              "      <td>northwest</td>\n",
              "      <td>3866.85520</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   age     sex     bmi  children smoker     region      charges\n",
              "0   19  female  27.900         0    yes  southwest  16884.92400\n",
              "1   18    male  33.770         1     no  southeast   1725.55230\n",
              "2   28    male  33.000         3     no  southeast   4449.46200\n",
              "3   33    male  22.705         0     no  northwest  21984.47061\n",
              "4   32    male  28.880         0     no  northwest   3866.85520"
            ]
          },
          "metadata": {},
          "execution_count": 87
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s55oIsYv0KkZ"
      },
      "source": [
        "We're going to have to turn the non-numerical columns into numbers (because a neural network can't handle non-numerical inputs).\n",
        "\n",
        "To do so, we'll use the [`get_dummies()`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.get_dummies.html) method in pandas.\n",
        "\n",
        "It converts categorical variables (like the `sex`, `smoker` and `region` columns) into numerical variables using one-hot encoding."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MqM_BmPkdon8",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 224
        },
        "outputId": "c6a4b1bf-3042-4702-bf7f-bce155e64310"
      },
      "source": [
        "# Turn all categories into numbers\n",
        "insurance_one_hot = pd.get_dummies(insurance)\n",
        "insurance_one_hot.head() # view the converted columns"
      ],
      "execution_count": 88,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>bmi</th>\n",
              "      <th>children</th>\n",
              "      <th>charges</th>\n",
              "      <th>sex_female</th>\n",
              "      <th>sex_male</th>\n",
              "      <th>smoker_no</th>\n",
              "      <th>smoker_yes</th>\n",
              "      <th>region_northeast</th>\n",
              "      <th>region_northwest</th>\n",
              "      <th>region_southeast</th>\n",
              "      <th>region_southwest</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>19</td>\n",
              "      <td>27.900</td>\n",
              "      <td>0</td>\n",
              "      <td>16884.92400</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>18</td>\n",
              "      <td>33.770</td>\n",
              "      <td>1</td>\n",
              "      <td>1725.55230</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>28</td>\n",
              "      <td>33.000</td>\n",
              "      <td>3</td>\n",
              "      <td>4449.46200</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>33</td>\n",
              "      <td>22.705</td>\n",
              "      <td>0</td>\n",
              "      <td>21984.47061</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>32</td>\n",
              "      <td>28.880</td>\n",
              "      <td>0</td>\n",
              "      <td>3866.85520</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   age     bmi  children  ...  region_northwest  region_southeast  region_southwest\n",
              "0   19  27.900         0  ...                 0                 0                 1\n",
              "1   18  33.770         1  ...                 0                 1                 0\n",
              "2   28  33.000         3  ...                 0                 1                 0\n",
              "3   33  22.705         0  ...                 1                 0                 0\n",
              "4   32  28.880         0  ...                 1                 0                 0\n",
              "\n",
              "[5 rows x 12 columns]"
            ]
          },
          "metadata": {},
          "execution_count": 88
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zOHoPzgqgZPq"
      },
      "source": [
        "Now we'll split data into features (`X`) and labels (`y`)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "W_EGj3FxhkAb"
      },
      "source": [
        "# Create X & y values\n",
        "X = insurance_one_hot.drop(\"charges\", axis=1)\n",
        "y = insurance_one_hot[\"charges\"]"
      ],
      "execution_count": 89,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 224
        },
        "id": "-QQFJmDn5ATV",
        "outputId": "eb53df6b-4461-4251-f1c7-47cf81c99da7"
      },
      "source": [
        "# View features\n",
        "X.head()"
      ],
      "execution_count": 90,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>bmi</th>\n",
              "      <th>children</th>\n",
              "      <th>sex_female</th>\n",
              "      <th>sex_male</th>\n",
              "      <th>smoker_no</th>\n",
              "      <th>smoker_yes</th>\n",
              "      <th>region_northeast</th>\n",
              "      <th>region_northwest</th>\n",
              "      <th>region_southeast</th>\n",
              "      <th>region_southwest</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>19</td>\n",
              "      <td>27.900</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>18</td>\n",
              "      <td>33.770</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>28</td>\n",
              "      <td>33.000</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>33</td>\n",
              "      <td>22.705</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>32</td>\n",
              "      <td>28.880</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   age     bmi  children  ...  region_northwest  region_southeast  region_southwest\n",
              "0   19  27.900         0  ...                 0                 0                 1\n",
              "1   18  33.770         1  ...                 0                 1                 0\n",
              "2   28  33.000         3  ...                 0                 1                 0\n",
              "3   33  22.705         0  ...                 1                 0                 0\n",
              "4   32  28.880         0  ...                 1                 0                 0\n",
              "\n",
              "[5 rows x 11 columns]"
            ]
          },
          "metadata": {},
          "execution_count": 90
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kudhkM-0giS1"
      },
      "source": [
        "And create training and test sets. We could do this manually, but to make it easier, we'll leverage the already available [`train_test_split`](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html) function available from Scikit-Learn."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WPGZGk0jhxCZ"
      },
      "source": [
        "# Create training and test sets\n",
        "from sklearn.model_selection import train_test_split\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, \n",
        "                                                    y, \n",
        "                                                    test_size=0.2, \n",
        "                                                    random_state=42) # set random state for reproducible splits"
      ],
      "execution_count": 91,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W8wEC0FPglnN"
      },
      "source": [
        "Now we can build and fit a model (we'll make it the same as `model_2`)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OCXTmz6oh_T6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "da476651-b453-4465-d8ee-9c16961b8ce1"
      },
      "source": [
        "# Set random seed\n",
        "tf.random.set_seed(42)\n",
        "\n",
        "# Create a new model (same as model_2)\n",
        "insurance_model = tf.keras.Sequential([\n",
        "  tf.keras.layers.Dense(1),\n",
        "  tf.keras.layers.Dense(1)\n",
        "])\n",
        "\n",
        "# Compile the model\n",
        "insurance_model.compile(loss=tf.keras.losses.mae,\n",
        "                        optimizer=tf.keras.optimizers.SGD(),\n",
        "                        metrics=['mae'])\n",
        "\n",
        "# Fit the model\n",
        "insurance_model.fit(X_train, y_train, epochs=100)"
      ],
      "execution_count": 92,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 8868.5918 - mae: 8868.5918\n",
            "Epoch 2/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7887.1606 - mae: 7887.1606\n",
            "Epoch 3/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7537.0947 - mae: 7537.0947\n",
            "Epoch 4/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7859.4346 - mae: 7859.4346\n",
            "Epoch 5/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7639.6699 - mae: 7639.6699\n",
            "Epoch 6/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7578.0850 - mae: 7578.0850\n",
            "Epoch 7/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7514.6177 - mae: 7514.6177\n",
            "Epoch 8/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7694.1338 - mae: 7694.1338\n",
            "Epoch 9/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7595.9136 - mae: 7595.9136\n",
            "Epoch 10/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7735.9116 - mae: 7735.9116\n",
            "Epoch 11/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7444.4189 - mae: 7444.4189\n",
            "Epoch 12/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7678.0337 - mae: 7678.0337\n",
            "Epoch 13/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7681.5840 - mae: 7681.5840\n",
            "Epoch 14/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7702.2842 - mae: 7702.2842\n",
            "Epoch 15/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7585.8921 - mae: 7585.8921\n",
            "Epoch 16/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7689.5356 - mae: 7689.5356\n",
            "Epoch 17/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7509.2036 - mae: 7509.2036\n",
            "Epoch 18/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7695.0083 - mae: 7695.0083\n",
            "Epoch 19/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7669.3740 - mae: 7669.3740\n",
            "Epoch 20/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7901.1362 - mae: 7901.1362\n",
            "Epoch 21/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7552.4814 - mae: 7552.4814\n",
            "Epoch 22/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7844.9961 - mae: 7844.9961\n",
            "Epoch 23/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7642.2485 - mae: 7642.2485\n",
            "Epoch 24/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7515.3081 - mae: 7515.3081\n",
            "Epoch 25/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7678.3506 - mae: 7678.3506\n",
            "Epoch 26/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7653.0269 - mae: 7653.0269\n",
            "Epoch 27/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7559.5449 - mae: 7559.5449\n",
            "Epoch 28/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7459.9404 - mae: 7459.9404\n",
            "Epoch 29/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7618.6177 - mae: 7618.6177\n",
            "Epoch 30/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7628.6255 - mae: 7628.6255\n",
            "Epoch 31/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7540.4893 - mae: 7540.4893\n",
            "Epoch 32/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7486.0186 - mae: 7486.0186\n",
            "Epoch 33/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7418.6646 - mae: 7418.6646\n",
            "Epoch 34/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7480.7319 - mae: 7480.7319\n",
            "Epoch 35/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7615.3115 - mae: 7615.3115\n",
            "Epoch 36/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7566.7896 - mae: 7566.7896\n",
            "Epoch 37/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7661.0879 - mae: 7661.0879\n",
            "Epoch 38/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7522.6816 - mae: 7522.6816\n",
            "Epoch 39/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7556.0718 - mae: 7556.0718\n",
            "Epoch 40/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7433.5669 - mae: 7433.5669\n",
            "Epoch 41/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7722.4312 - mae: 7722.4312\n",
            "Epoch 42/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7344.2700 - mae: 7344.2700\n",
            "Epoch 43/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7597.4331 - mae: 7597.4331\n",
            "Epoch 44/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7338.0132 - mae: 7338.0132\n",
            "Epoch 45/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7510.3467 - mae: 7510.3467\n",
            "Epoch 46/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7413.5801 - mae: 7413.5801\n",
            "Epoch 47/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7451.0391 - mae: 7451.0391\n",
            "Epoch 48/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7340.5381 - mae: 7340.5381\n",
            "Epoch 49/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7481.9976 - mae: 7481.9976\n",
            "Epoch 50/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7468.2842 - mae: 7468.2842\n",
            "Epoch 51/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7411.3408 - mae: 7411.3408\n",
            "Epoch 52/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7460.0796 - mae: 7460.0796\n",
            "Epoch 53/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7601.6606 - mae: 7601.6606\n",
            "Epoch 54/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7241.2549 - mae: 7241.2549\n",
            "Epoch 55/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7539.6953 - mae: 7539.6953\n",
            "Epoch 56/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7293.2012 - mae: 7293.2012\n",
            "Epoch 57/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7417.9731 - mae: 7417.9731\n",
            "Epoch 58/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7353.0625 - mae: 7353.0625\n",
            "Epoch 59/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7643.8247 - mae: 7643.8247\n",
            "Epoch 60/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7410.4004 - mae: 7410.4004\n",
            "Epoch 61/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7612.8330 - mae: 7612.8330\n",
            "Epoch 62/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7387.9087 - mae: 7387.9087\n",
            "Epoch 63/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7359.5605 - mae: 7359.5605\n",
            "Epoch 64/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7109.0884 - mae: 7109.0884\n",
            "Epoch 65/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7396.3223 - mae: 7396.3223\n",
            "Epoch 66/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7179.8613 - mae: 7179.8613\n",
            "Epoch 67/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7289.7710 - mae: 7289.7710\n",
            "Epoch 68/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7523.6973 - mae: 7523.6973\n",
            "Epoch 69/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7442.6157 - mae: 7442.6157\n",
            "Epoch 70/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7673.4834 - mae: 7673.4834\n",
            "Epoch 71/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7276.0337 - mae: 7276.0337\n",
            "Epoch 72/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7246.3721 - mae: 7246.3721\n",
            "Epoch 73/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7372.0713 - mae: 7372.0713\n",
            "Epoch 74/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7512.0762 - mae: 7512.0762\n",
            "Epoch 75/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7269.7437 - mae: 7269.7437\n",
            "Epoch 76/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7199.5039 - mae: 7199.5039\n",
            "Epoch 77/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7261.2920 - mae: 7261.2920\n",
            "Epoch 78/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7185.7627 - mae: 7185.7627\n",
            "Epoch 79/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7301.7495 - mae: 7301.7495\n",
            "Epoch 80/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7002.6309 - mae: 7002.6309\n",
            "Epoch 81/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7289.1357 - mae: 7289.1357\n",
            "Epoch 82/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7155.3945 - mae: 7155.3945\n",
            "Epoch 83/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7475.1709 - mae: 7475.1709\n",
            "Epoch 84/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7387.3672 - mae: 7387.3672\n",
            "Epoch 85/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7289.9458 - mae: 7289.9458\n",
            "Epoch 86/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7268.0942 - mae: 7268.0942\n",
            "Epoch 87/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7238.5869 - mae: 7238.5869\n",
            "Epoch 88/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7201.7354 - mae: 7201.7349\n",
            "Epoch 89/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7538.0757 - mae: 7538.0757\n",
            "Epoch 90/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 6967.1187 - mae: 6967.1187\n",
            "Epoch 91/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7314.1299 - mae: 7314.1299\n",
            "Epoch 92/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7192.3115 - mae: 7192.3115\n",
            "Epoch 93/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7530.8770 - mae: 7530.8770\n",
            "Epoch 94/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7187.3579 - mae: 7187.3579\n",
            "Epoch 95/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7561.5635 - mae: 7561.5635\n",
            "Epoch 96/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7263.4648 - mae: 7263.4648\n",
            "Epoch 97/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7146.2896 - mae: 7146.2896\n",
            "Epoch 98/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7247.9238 - mae: 7247.9238\n",
            "Epoch 99/100\n",
            "34/34 [==============================] - 0s 2ms/step - loss: 7200.6694 - mae: 7200.6694\n",
            "Epoch 100/100\n",
            "34/34 [==============================] - 0s 1ms/step - loss: 7301.6870 - mae: 7301.6870\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f8dedf0cad0>"
            ]
          },
          "metadata": {},
          "execution_count": 92
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "B1U7LqpKid0r",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0e322c1b-6c8a-49cd-f724-67afc65bba7f"
      },
      "source": [
        "# Check the results of the insurance model\n",
        "insurance_model.evaluate(X_test, y_test)"
      ],
      "execution_count": 93,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "9/9 [==============================] - 0s 2ms/step - loss: 8628.2393 - mae: 8628.2393\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[8628.2392578125, 8628.2392578125]"
            ]
          },
          "metadata": {},
          "execution_count": 93
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H9doNDToqDru"
      },
      "source": [
        "Our model didn't perform very well, let's try a bigger model.\n",
        "\n",
        "We'll try 3 things:\n",
        "- Increasing the number of layers (2 -> 3).\n",
        "- Increasing the number of units in each layer (except for the output layer).\n",
        "- Changing the optimizer (from SGD to Adam).\n",
        "\n",
        "Everything else will stay the same."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "W59EqfqYimnR"
      },
      "source": [
        "# Set random seed\n",
        "tf.random.set_seed(42)\n",
        "\n",
        "# Add an extra layer and increase number of units\n",
        "insurance_model_2 = tf.keras.Sequential([\n",
        "  tf.keras.layers.Dense(100), # 100 units\n",
        "  tf.keras.layers.Dense(10), # 10 units\n",
        "  tf.keras.layers.Dense(1) # 1 unit (important for output layer)\n",
        "])\n",
        "\n",
        "# Compile the model\n",
        "insurance_model_2.compile(loss=tf.keras.losses.mae,\n",
        "                          optimizer=tf.keras.optimizers.Adam(), # Adam works but SGD doesn't \n",
        "                          metrics=['mae'])\n",
        "\n",
        "# Fit the model and save the history (we can plot this)\n",
        "history = insurance_model_2.fit(X_train, y_train, epochs=100, verbose=0)"
      ],
      "execution_count": 94,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "k9IbYWnOqmoT",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "eab558fc-8905-42fe-89b4-eb6576eae14c"
      },
      "source": [
        "# Evaluate our larger model\n",
        "insurance_model_2.evaluate(X_test, y_test)"
      ],
      "execution_count": 95,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "9/9 [==============================] - 0s 2ms/step - loss: 4924.3477 - mae: 4924.3477\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[4924.34765625, 4924.34765625]"
            ]
          },
          "metadata": {},
          "execution_count": 95
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o9Rf3NosqriS"
      },
      "source": [
        "Much better! Using a larger model and the Adam optimizer results in almost half the error as the previous model.\n",
        "\n",
        "> 🔑 **Note:** For many problems, the [Adam optimizer](https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/Adam) is a great starting choice. See Andrei Karpathy's \"Adam is safe\" point from [*A Recipe for Training Neural Networks*](http://karpathy.github.io/2019/04/25/recipe/) for more. \n",
        "\n",
        "Let's check out the loss curves of our model, we should see a downward trend."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9SE55ANojcF_",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 279
        },
        "outputId": "1c145cb3-71ef-4f0f-bf8e-26e95f3398f0"
      },
      "source": [
        "# Plot history (also known as a loss curve)\n",
        "pd.DataFrame(history.history).plot()\n",
        "plt.ylabel(\"loss\")\n",
        "plt.xlabel(\"epochs\");"
      ],
      "execution_count": 96,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEGCAYAAACkQqisAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZRdZZnv8e9zap7nuVKpCpkHEjIwRRBEAW0VbefrADje22rb2u2A2le7r7ba9mraqVGWoNCigIAttgoCIpMQMhAgcypzVVJJzVWpeXjuH2cHC0hIpXJO7cqp32etWqnznn32efbakF/evd/33ebuiIiITEQk7AJEROTMpRAREZEJU4iIiMiEKURERGTCFCIiIjJhyWEXMNmKi4u9trY27DJERM4o69evb3H3kpe2T7sQqa2tZd26dWGXISJyRjGzfcdr1+UsERGZMIWIiIhMmEJEREQmbNrdExERmaihoSEaGhro7+8Pu5S4SU9Pp7q6mpSUlHFtrxARERmnhoYGcnJyqK2txczCLifm3J3W1lYaGhqoq6sb12d0OUtEZJz6+/spKipKyAABMDOKiopOqaelEBEROQWJGiDHnOrxKUTGwUdHefru/2DjQ7eHXYqIyJSieyLjMDw8ROGWWykZaeLgWedQWTsv7JJEZJrKzs7m6NGjYZfxAvVExiElNY2M9/4XER/l6M/ex+BA4o7MEBE5FQqRcaqatYj6C7/F3OEdbPjxJ8IuR0SmOXfns5/9LIsXL2bJkiXccccdABw6dIiLL76YZcuWsXjxYh577DFGRka45pprXtj2+uuvj1kdupx1Cs654mqe2vU45x+5kw33vYrlV14TdkkiEpJ/+s1mthzsiuk+F1bm8pU3LRrXtvfccw8bN27k2WefpaWlhVWrVnHxxRfz85//nCuuuIIvfelLjIyM0Nvby8aNG2lsbGTTpk0AdHR0xKxm9URO0fIPfY/6pLMoXfMNfHQ07HJEZJp6/PHHec973kNSUhJlZWW8+tWvZu3ataxatYqf/OQnfPWrX+X5558nJyeHWbNmsXv3bj75yU9y3333kZubG7M61BM5Ralp6bQtvoZzn/1Htm14mPkrLwu7JBEJwXh7DJPt4osv5tFHH+W3v/0t11xzDZ/5zGf4wAc+wLPPPsv999/PD3/4Q+68805uvvnmmHyfeiITMP/S99LvKXQ+9bOwSxGRaeqiiy7ijjvuYGRkhObmZh599FHOPfdc9u3bR1lZGR/5yEf48Ic/zIYNG2hpaWF0dJS3ve1tfO1rX2PDhg0xq0M9kQnIzS9ifc5q5rY8wNDgACmpaWGXJCLTzFvf+laefPJJli5dipnxr//6r5SXl3PLLbfw7W9/m5SUFLKzs7n11ltpbGzk2muvZTS4BP+Nb3wjZnWYu8dsZ2eClStXeiweSrXxodtZ9tjH2PiqH7Lste+JQWUiMtVt3bqVBQsWhF1G3B3vOM1svbuvfOm2upw1QYsueivt5DCyUbPYRWT6UohMUEpqGjuKX8ei7ifo6mgNuxwRkVAoRE5D3vnvI92G2PbwbWGXIiISCoXIaZi3/FIarIKMbfeEXYqISCgUIqfBIhEaiy6grn8boyMjYZcjIjLpFCKnycqXkG19HNq3I+xSREQmnULkNOXPWg7A4Z2nP2xYRORMoxA5TTPmrWDEjYGGZ8MuRURk0ilETlNGVg6NSZWkt24OuxQRSXB79+5l/vz5XHPNNcydO5f3vve9PPjgg6xevZo5c+bw9NNP8/TTT3PBBRdwzjnncOGFF7J9+3YARkZG+OxnP8uqVas4++yz+dGPfhSTmrTsSQw0Z86homdL2GWIyGT6/Reg6fnY7rN8Cbz+m6+4SX19Pb/85S+5+eabWbVqFT//+c95/PHHuffee/mXf/kXbr31Vh577DGSk5N58MEH+eIXv8jdd9/NTTfdRF5eHmvXrmVgYIDVq1dz+eWXU1dXd1olK0RiYKh4EZVH/0RXRyu5+UVhlyMiCayuro4lS5YAsGjRIi677DLMjCVLlrB37146Ozu5+uqr2blzJ2bG0NAQAH/4wx947rnnuOuuuwDo7Oxk586dCpGpIKNmGeyFhm1rWXj+lWGXIyKT4SQ9hnhJS/vLgq+RSOSF15FIhOHhYf7xH/+RSy+9lF/96lfs3buXSy65BIg+CfF73/seV1xxRUzr0T2RGKiYtwqA7r3PhFyJiEx3nZ2dVFVVAfDTn/70hfYrrriCG2644YWeyY4dO+jp6Tnt71OIxEBJxUzaycEObwq7FBGZ5j73uc9x3XXXcc455zA8PPxC+4c//GEWLlzI8uXLWbx4MR/72Mde9P5EaSn4GNn0jVeTOtLL3C+vjfm+RWRq0FLwWgo+bo7mz6dmaA/DQ4NhlyIiMmkUIjGSVLGEdBuicZcuaYnI9KEQiZHCYPmT5l3rQ65EROIp0W8BnOrxKURiZMa85Qx6EkONz4VdiojESXp6Oq2trQkbJO5Oa2sr6enp4/5M3OaJmNnNwBuBI+6+OGj7NvAmYBDYBVzr7h3Be9cBHwJGgL919/uD9iuB7wBJwI/d/ZtBex1wO1AErAfe7+6h3ZBITUtnd9IMMtu2hlWCiMRZdXU1DQ0NNDc3h11K3KSnp1NdXT3u7eM52fCnwPeBW8e0PQBc5+7DZvYt4Drg82a2EHg3sAioBB40s7nBZ34AvA5oANaa2b3uvgX4FnC9u99uZj8kGkA3xPF4Tqotew7VXZorIpKoUlJSTnuGd6KJ2+Usd38UaHtJ2x/c/djA5KeAY3F3FXC7uw+4+x6gHjg3+Kl3991BL+N24CozM+A1wF3B528B3hKvYxmvodwZlHirRmiJyLQR5j2RDwK/D36vAg6Mea8haDtRexHQMSaQjrUfl5l91MzWmdm6eHZDI3nVJJnTevjAyTcWEUkAoYSImX0JGAZum4zvc/cb3X2lu68sKSmJ2/ekF80AoP3Qnrh9h4jIVDLpCzCa2TVEb7hf5n8Z4tAIzBizWXXQxgnaW4F8M0sOeiNjtw9NbtlMAHqa1RMRkelhUnsiwUirzwFvdvfeMW/dC7zbzNKCUVdzgKeBtcAcM6szs1SiN9/vDcLnYeDtweevBn49WcdxIoUVswAYat8fciUiIpMjbiFiZr8AngTmmVmDmX2I6GitHOABM9sYjKrC3TcDdwJbgPuAj7v7SNDL+ARwP7AVuDPYFuDzwGfMrJ7oPZKb4nUs45WbX0Svp0HXwbBLERGZFHG7nOXu7zlO8wn/onf3rwNfP07774DfHad9N9HRW1OGRSK0JBWT2qMQEZHpQTPWY6wrpYSs/iNhlyEiMikUIjHWl1FBwbBCRESmB4VIjA1nV1Dk7ZpwKCLTgkIkxjThUESmE4VIjGnCoYhMJwqRGNOEQxGZThQiMaYJhyIynShEYkwTDkVkOlGIxJgmHIrIdKIQiQNNOBSR6UIhEgd9GRXkDyfu4zNFRI5RiMTBcHYFxd6mCYcikvAUInGgCYciMl0oROJAEw5FZLpQiMSBJhyKyHShEIkDTTgUkelCIRIHmnAoItOFQiQONOFQRKYLhUicaMKhiEwHCpE40YRDEZkOFCJxMpxVRpG3MzoyEnYpIiJxoxCJE8suJdlG6WzTJS0RSVwKkThJyS0DoLOlMeRKRETiRyESJ+n5FQAcbdUILRFJXAqROMkuLAegv/NwyJWIiMSPQiRO8kqqABhWiIhIAlOIxEluQQnDHsF7NMxXRBKXQiROIklJtFseSb0KERFJXAqROOpKKiB1oC3sMkRE4kYhEkc9KQVkDraGXYaISNwoROJoIK2InOH2sMsQEYkbhUgcjaQXke+d+Oho2KWIiMSFQiSeskvJsEF6jnaGXYmISFwoROIoKacUgM4WzVoXkcSkEImjtLzorPVuhYiIJCiFSBxlFkbXz+rr0Kx1EUlMCpE4yi2uBGCwsynkSkRE4kMhEkf5xdGeyGi3nikiIolJIRJHqWnpdJJFREufiEiCiluImNnNZnbEzDaNaSs0swfMbGfwZ0HQbmb2XTOrN7PnzGz5mM9cHWy/08yuHtO+wsyeDz7zXTOzeB3L6eiIFJDS3xJ2GSIicRHPnshPgStf0vYF4CF3nwM8FLwGeD0wJ/j5KHADREMH+ApwHnAu8JVjwRNs85Exn3vpd00JR5MLSB/UrHURSUxxCxF3fxR46eqDVwG3BL/fArxlTPutHvUUkG9mFcAVwAPu3ubu7cADwJXBe7nu/pS7O3DrmH1NKQOphWQPaxFGEUlMk31PpMzdDwW/NwFlwe9VwIEx2zUEba/U3nCc9uMys4+a2TozW9fcPLn3J4bSi8kb7ZjU7xQRmSyh3VgPehA+Sd91o7uvdPeVJSUlk/GVLxjNKiGPHgYH+if1e0VEJsNkh8jh4FIUwZ/Hxr42AjPGbFcdtL1Se/Vx2qecSHZ06ZP25ilZnojIaZnsELkXODbC6mrg12PaPxCM0jof6Awue90PXG5mBcEN9cuB+4P3uszs/GBU1gfG7GtKSc2LXrHrbj10ki1FRM48yfHasZn9ArgEKDazBqKjrL4J3GlmHwL2Ae8MNv8d8AagHugFrgVw9zYz+3/A2mC7f3b3Y3ep/4boCLAM4PfBz5STURBdP6unTSEiIoknbiHi7u85wVuXHWdbBz5+gv3cDNx8nPZ1wOLTqXEy5BRF7/cPdmr9LBFJPJqxHmf5JdGlT0a6FCIikngUInGWlZNPr6dBj5Y+EZHEoxCZBB2RPJL7tPSJiCQehcgk6E4qIG2gNewyRERiTiEyCXpTi8gc1vpZIpJ4FCKTYCitkLwRhYiIJB6FyCQYySwh37sYHRkJuxQRkZhSiEwCyy4l2UbpaNVjckUksYwrRMzsU2aWGyxLcpOZbTCzy+NdXKJIzY/OFek40nCSLUVEzizj7Yl80N27iK5dVQC8n+gSJjIOmYXRWetHWxQiIpJYxhsixx49+wbgv9x985g2OYmckuhCxP3tB0OuREQktsYbIuvN7A9EQ+R+M8sBRuNXVmIpKo+GyEiXFmEUkcQy3gUYPwQsA3a7e2/w7PNr41dWYknPzKaLTCJHtX6WiCSW8fZELgC2u3uHmb0P+DLQGb+yEk97pJCUviMn31BE5Awy3hC5Aeg1s6XA3wO7gFvjVlUCOppcRMaA1s8SkcQy3hAZDp75cRXwfXf/AZATv7IST196CbnDWj9LRBLLeEOk28yuIzq097dmFgFS4ldW4hnOKKFwtB0f1XgEEUkc4w2RdwEDROeLNAHVwLfjVlUiyiknwwbp7tIaWiKSOMYVIkFw3AbkmdkbgX531z2RU5CcF5213n74QMiViIjEzniXPXkn8DTwDuCdwBoze3s8C0s06QXREOluUYiISOIY7zyRLwGr3P0IgJmVAA8Cd8WrsESTU1wNQH+bZq2LSOIY7z2RyLEACbSewmcFyC+rAWC4U7PWRSRxjLcncp+Z3Q/8Inj9LuB38SkpMeXmFdLvKdCt5eBFJHGMK0Tc/bNm9jZgddB0o7v/Kn5lJR6LRGiLFJLc1xx2KSIiMTPengjufjdwdxxrSXhdyUVk9CtERCRxvGKImFk34Md7C3B3z41LVQmqN7WYor7dYZchIhIzrxgi7q6lTWJoKKOEgp51YZchIhIzGmE1iTy7nFx66e89GnYpIiIxoRCZRJG8cgBamzThUEQSg0JkEqXnVwLQ3awQEZHEoBCZRFnBrPXetsaQKxERiQ2FyCTKL42GyGCHZq2LSGJQiEyiguIKhjwJ17PWRSRBKEQmUSQpiXbLI6lHz1oXkcSgEJlknclFpGvWuogkCIXIJOtJLSZrqCXsMkREYkIhMskG0kvIH2kLuwwRkZhQiEyy0awyCryL4aHBsEsRETltCpFJFsmtIGJO88E9YZciInLaQgkRM/u0mW02s01m9gszSzezOjNbY2b1ZnaHmaUG26YFr+uD92vH7Oe6oH27mV0RxrGcqry6ZQA0bV8bciUiIqdv0kPEzKqAvwVWuvtiIAl4N/At4Hp3nw20Ax8KPvIhoD1ovz7YDjNbGHxuEXAl8J9mljSZxzIRtQvPY8iT6N+nEBGRM19Yl7OSgQwzSwYygUPAa4C7gvdvAd4S/H5V8Jrg/cvMzIL22919wN33APXAuZNU/4SlZ2azP3km2S3PhV2KiMhpm/QQcfdG4N+A/UTDoxNYD3S4+3CwWQNQFfxeBRwIPjscbF80tv04n3kRM/uoma0zs3XNzeHP0WjNW0TNwHZ8dDTsUkRETksYl7MKiPYi6oBKIIvo5ai4cfcb3X2lu68sKSmJ51eNT+Vy8uihcfeWsCsRETktYVzOei2wx92b3X0IuAdYDeQHl7cAqoFjS902AjMAgvfzgNax7cf5zJRWNPd8AJq2/TnkSkRETk8YIbIfON/MMoN7G5cBW4CHgbcH21wN/Dr4/d7gNcH7f3R3D9rfHYzeqgPmAE9P0jGclpr5K+j3FIYPrA+7FBGR0/KKz1iPB3dfY2Z3ARuAYeAZ4Ebgt8DtZva1oO2m4CM3Af9lZvVAG9ERWbj7ZjO7k2gADQMfd/eRST2YCUpJTWN3ylnktm0KuxQRkdMy6SEC4O5fAb7ykubdHGd0lbv3A+84wX6+Dnw95gVOgo6CxSw58htGhodJSg7lNIiInDbNWA9JpHoFmTbA/h3PhF2KiMiEKURCUjr/QgCatz8ZciUiIhOnEAnJjNlL6PYMvEE310XkzKUQCUkkKYn9aXMp7NwcdikiIhOmEAlRV9ESZg7tZqC/N+xSREQmRCESooxZq0m1EZ674Wr6errDLkdE5JQpREJ09mvexZM1H2VFxwMc/PeLaNytS1sicmbRBIUQRZKSuOCD3+bZh8+l9pFPkXTLZTybsZi+vLOw4rlkVc6lZOZCSipqiSRN+VXuRWQaUohMAUsvfQcHa5fQ+N//l4Kj9cxtepaMw4MQdEz6PJXG5Bm0Z5/FSNF80isXUly7hIra+ZqoKCKhsugyVNPHypUrfd26dWGX8YpGR0Y43FBPy75t9DbtwFvryeysp7x/N6W0vbDdgKfQkFxNe9ZshooXkFGxgIKahVTUzic1LT3EIxCRRGNm69195Uvb9c/YKSiSlETFzHlUzJz3svc625o5tOtZug9sZuTwVjI765nRtYGyrgeiC8c8ASNuHIhU0JI5i4GCeSRXLKBgxiIqz1pCRlbO5B+QiCQshcgZJq+whLzC18Kq176ovbO9hcN7NtHVsJWhIztIa6+nuG8XVUefIKnBIXga70Er43DGWfQXLSStcjHFZ51DZd1CklNSQzgaETnTKUQSRF5BMXkFl8DyS17U3t/Xw/7dm2jbt4XBpq2ktm2npHcnVfufJOmAw5roZbG9yTW0Z89hpHQRWTOWUlK7kNKqWbqhLyKvSPdEpqm+nm4adjxDx77nGGnaQmbHNir7d1FMxwvbDHgKB5Mqac5dhFetoHjehdTMX0FKalqIlYtIGE50T0QhIi/SeriBQ/XP0HNwB966i4zOndT0baOALiAaLPtTamnPnY+XLyV/1gpqFqzSvRaRBKcb6zIuRWXVFJVVv6jNR0dp3LuVQ1ueYLhhI9ntm5nX9kfy2n4DW2DkN8bBSAltqZX0ZM8kUrWCymWXU1k7D4toPqtIIlNPRCbER0dpOrCTpu1r6T+wkZTOPeT0HqB8uIE8egBoooSGnLMZrlxB4bzV1C46X0OPRc5QupwVUIjEl4+Osn/7MzQ99yApB56g6ugmymgFYNCT2ZMym/bCpSTPXEXlooupqJmj3orIGUAhElCITL7DDbtofP5RBvetJbf1WeoGd5BhgwC0kseBzIX0lSwle9Z51Cy5iLzCkpArFpGXUogEFCLhGxocYN/WdbRuf4JI4zpKuzczc7QBgFE39iXN5EjhcpJqL4j2VmbOVW9FJGQKkYBCZGrq6mhl/6Yn6N75BFmH13JW32ayrB+ANnLZn7mI/opzKVhwCbPOXq1hxiKTTCESUIicGYaHBtm3dR0t26K9lfKu55jhBwHo9TR2py+gu3QVOfMuZvby15CemR1yxSKJTSESUIicuVqa9rNvw0MM73mcorZnqBveTZI5A55CfdpCuiovpHDx5Zy19FVaxkUkxhQiAYVI4ujqaGXPhofo2/5HSlrWcNbI7mg7mezKPIfB2kuoXvkmqmYtCLlSkTOfQiSgEElcbUca2bPuPkbqH2ZG21NU0AzAAavkUMEKbOaFVC29jMral6+OLCKvTCESUIhMDz46yoH65zi4/rdk7H+Eur7nyaUXiIZKY8mryFr8Buasupz0jKyQqxWZ+hQiAYXI9DQyPMzereto3vQQGfseZn7fRtJsiD5PZWfGEnqrL6J4yeXMWny+Vi4WOQ6FSEAhIhBdxXjHmt/St/VBKlqfYuboAQA6yWJ35lIGKlaRM+s8Zi65kOzcgpCrFQmfQiSgEJHjOdywiwMb/sDo7kep6lxPlR8GopMf9yfN4HD+MiI151O2cDVVsxbr2fYy7ShEAgoRGY/Www00bP4zvXvXknVkA3V9m8mxPiA6T6UhZSadWXUMZ5UTyaskrbCG/Oq5lM+crzkrkpC0FLzIKYguif9O4J1A9J7K7m3radm5htFDm8jp2EZN5zqKOtpJPjj6os8eoZCWlEp6MqsZzqshKa+K9KIZ5JbNpLCijpzcAi3jIglDISIyDknJycxafB6zFp/3ovaR4WFaWg7SdnA3XQd3MNS8i+SOvWT1NVLTuZaSjvuJ2It7+z2eTktSMd0pxfSllwa9mSrSCmvILp1Jfkk1ecXlWtpFzggKEZHTkJScTHF5DcXlNS97vj3A4EA/rU376GzaR0/LfobaG6CrkdSeJjIHmpnR+QxFHW2kHBx52Wc7yaI9UkRXain96aWM5FQSya0kvWgG2SU1FJbPJL+oTL0aCZVCRCSOUtPSqZg5j4qZJ57gODoyQsvhBtqa9nD0yF6Guo4werSZSM8RUvuayR48QmVHPYXtnS/r1Qx6Mi2RIjqSS+hNL2M4qxxyyknOryCzaAb55XWUVNaqVyNxoxARCVkkKYniypkUV858xe2GBgc4cvgAHU176W05wGB7I3QdJLn3MJn9h6k4uomirkdJbxp60edG3ThiBbQnl9CTXs5gZjmeVUxSdglpBRUUzlhA+cz5ChqZEIWIyBkiJTWN8hmzKZ8x+4Tb+OgonR2ttB/eR9fhffS37meko4Hk7kYy+g5R0rOTou6nyLSBF31uyJNoiJTSmVJMX1opw5mlWNEsMsvnUFg9n7IZZ2lRSzkuhYhIArFIhLzCkujTIRe8bDTmC/p7j9LRcoiOpr10H9zOcPNOUjv3kjnQTPnRzRR3PUr64SHYEt1+2CM0RkpoS62kL6uakbyZpJbMIrdiDqUzF+hplNOYQkRkGkrPzKa8Zg7lNXOA173s/dGREY407ad531Z6Du1gpG0PqV37yelroKrtEQrbumDPX7bvIovWSAndqUUMpJUwXFBHRtXZlM5eTnnNHC0lk8A02VBETtnRrnYO79tO16F6Bo7UYx37SO1tImuwhfzhFkppe2HbAU/hSKSE9rQK+rJnQvFcMisXUjprMaWVdRpddoaYUjPWzSwf+DGwGHDgg8B24A6gFtgLvNPd283MgO8AbwB6gWvcfUOwn6uBLwe7/Zq733Ky71aIiMRfd2cbjTs20Ln3Wbx1F6lHG8jpP0j5UMMLM/8hOmfmUHIVHZm1DOXPIqV0DrlV8ygsr6OwrFo9mClkqoXILcBj7v5jM0sFMoEvAm3u/k0z+wJQ4O6fN7M3AJ8kGiLnAd9x9/PMrBBYB6wkGkTrgRXu3v5K360QEQmPj47S2nSApt3P0nNwG968g8zuPZT076PMW140hHnIkzgcKaUpZxEjlSspmHsBVXOWkZWTH+IRTF9TZtkTM8sDLgauAXD3QWDQzK4CLgk2uwX4E/B54CrgVo+m3VNmlm9mFcG2D7h7W7DfB4ArgV9M1rGIyKmxSGTMcOY3v+i9/r4emvZupaNhBwNtBxjtbCStczc1XRso7XoQtkW3O0QJzem19GVVMZpbRUpxHVWLL37FuTgSP2HcWK8DmoGfmNlSoj2ITwFl7n4o2KYJKAt+rwIOjPl8Q9B2ovaXMbOPAh8FqKmpic1RiEhMpWdkUbtg5ctGlfnoKE2Nuzm45UkGDm4ipW0n+b17mdG6jYLW7ugN/rVw0MpoyFuBVywlZ+YyquatJK+gOJyDmUbCCJFkYDnwSXdfY2bfAb4wdgN3dzOL2XU2d78RuBGil7NitV8RiT+LRE44P6anu4NDuzfTsuVPpDX8mTkdj1HQ8TvYCtwXDZamzHkMliwhq24lNUsu0nDkGAsjRBqABndfE7y+i2iIHDazCnc/FFyuOhK83wjMGPP56qCtkb9c/jrW/qc41i0iU0xWTj6zl65m9tLVQLTXcuTQPg7tWEfv/o2kNm+irGc71XsfjQ7XeTj6eOTDOQsZLj+H/NnnUbv4Ai3ffxrCurH+GPBhd99uZl8Fjj3kunXMjfVCd/+cmf0V8An+cmP9u+5+bnBjfT3RXg3ABqI31tt4BbqxLjL9dLa3cGDTExzdtYa0I89Q1bvthWHIwx5hb3IdrflLiMxYRfmii6g+a4mGHr/EVBudtYzoEN9UYDdwLRAB7gRqgH1Eh/i2BUN8v0/0pnkvcK27rwv280Gio7oAvu7uPznZdytERASg+eBeGjY9Tv++teS0bKS2fzvZwfDjdnLYm7mEgRkXUbHsCmrmnTPtQ2VKhUiYFCIicjwjw8Ps3/EMzVseg4a1L3pMchu57M9YSF/pMnJmX8jsFZdNu0tgCpGAQkRExuvgnm00bPg9dmANpV2bmDHSQMScAU9hZ/oiuitfRdmKN1O3cFXC91QUIgGFiIhMVFdHK3ue+SN92x6itPlJZo3uBaKPRN5bcAFJc17H7PPeQF5R2Svv6AykEAkoREQkVloO7mP3ml+TvOsBZh9dRy69jLhRnzKXtqpLKVv11wnTS1GIBBQiIhIPw0OD1G98lPbn7qPo0CPMHd4BRGfY7yt9DXkr3sa8la89Y9cDU4gEFCIiMhlaDu5j95P3kFb/e7e1XoIAAAhcSURBVBb0rifVhmkhn935F5I05zJmn/+mM+qyl0IkoBARkcnW3dnG9sfvxrb+D7OPriWPHkbc2JSxktFz3s+iS95Falp62GW+IoVIQCEiImF64bLXM/dy1sHfUEobbeSys/i1ZJ/zduafdwVJyVPveYEKkYBCRESmiuGhQTY/dg8jG25jQfdTZNggreRRX3wZ+ee/n7nLL5kyN+UVIgGFiIhMRb1HO9n22D2w+Vcs7P4z6TbEAaukofatzL3ybygqqw61PoVIQCEiIlNdV0cr2//4MzK3/ZJFg88z6Ek8l3cp6eddw4LzXh/K5S6FSEAhIiJnkn3bNnDoof9k0ZH/Icf6aKaAXaWXU/Kqazjr7AsnrQ6FSEAhIiJnor6ebrY8cieRTXezqGcNqTbMtpSFHF32Qc5+7fvjPrpLIRJQiIjIma6zrZmt9/2Q6p23Ue2HOEwRexf+b5a9+ROkpWfG5TsVIgGFiIgkitGREZ5/5G7Snrye+UNbomEy/yOc/aZPkJGVE9PvUogEFCIikmh8dJRNj/+GlMe+yfyhLbSTw/bqdzL7jZ+muHzGyXcwDgqRgEJERBKVj46ybe0D9D/yHyzteZJhkngu7xKyL/o/zFvxmtOac6IQCShERGQ62L9jI4f+8F0WNf+ObOtjV9Is8j7ya4rLaya0vxOFyNSbWy8iIqetZu4yaubeTE93B2vu+zGpe/5IXUlVzL9HISIiksCycvI57x3/APxDXPY/NRZlERGRM5JCREREJkwhIiIiE6YQERGRCVOIiIjIhClERERkwhQiIiIyYQoRERGZsGm37ImZNQP7JvjxYqAlhuWcCabjMcP0PO7peMwwPY97Isc8091LXto47ULkdJjZuuOtHZPIpuMxw/Q87ul4zDA9jzuWx6zLWSIiMmEKERERmTCFyKm5MewCQjAdjxmm53FPx2OG6XncMTtm3RMREZEJU09EREQmTCEiIiITphAZBzO70sy2m1m9mX0h7HrixcxmmNnDZrbFzDab2aeC9kIze8DMdgZ/FoRda6yZWZKZPWNm/xO8rjOzNcE5v8PMUsOuMdbMLN/M7jKzbWa21cwuSPRzbWafDv7b3mRmvzCz9EQ812Z2s5kdMbNNY9qOe24t6rvB8T9nZstP5bsUIidhZknAD4DXAwuB95jZwnCripth4O/dfSFwPvDx4Fi/ADzk7nOAh4LXieZTwNYxr78FXO/us4F24EOhVBVf3wHuc/f5wFKix5+w59rMqoC/BVa6+2IgCXg3iXmufwpc+ZK2E53b1wNzgp+PAjecyhcpRE7uXKDe3Xe7+yBwO3BVyDXFhbsfcvcNwe/dRP9SqSJ6vLcEm90CvCWcCuPDzKqBvwJ+HLw24DXAXcEmiXjMecDFwE0A7j7o7h0k+Lkm+kjwDDNLBjKBQyTguXb3R4G2lzSf6NxeBdzqUU8B+WZWMd7vUoicXBVwYMzrhqAtoZlZLXAOsAYoc/dDwVtNQFlIZcXLfwCfA0aD10VAh7sPB68T8ZzXAc3AT4LLeD82sywS+Fy7eyPwb8B+ouHRCawn8c/1MSc6t6f1d5xCRF7GzLKBu4G/c/euse95dEx4wowLN7M3AkfcfX3YtUyyZGA5cIO7nwP08JJLVwl4rguI/qu7DqgEsnj5JZ9pIZbnViFyco3AjDGvq4O2hGRmKUQD5DZ3vydoPnysexv8eSSs+uJgNfBmM9tL9FLla4jeK8gPLnlAYp7zBqDB3dcEr+8iGiqJfK5fC+xx92Z3HwLuIXr+E/1cH3Oic3taf8cpRE5uLTAnGMGRSvRG3L0h1xQXwb2Am4Ct7v7vY966F7g6+P1q4NeTXVu8uPt17l7t7rVEz+0f3f29wMPA24PNEuqYAdy9CThgZvOCpsuALSTwuSZ6Get8M8sM/ls/dswJfa7HONG5vRf4QDBK63ygc8xlr5PSjPVxMLM3EL1ungTc7O5fD7mkuDCzVwGPAc/zl/sDXyR6X+ROoIboMvrvdPeX3rQ745nZJcA/uPsbzWwW0Z5JIfAM8D53Hwizvlgzs2VEBxOkAruBa4n+wzJhz7WZ/RPwLqIjEZ8BPkz0+n9CnWsz+wVwCdEl3w8DXwH+m+Oc2yBQv0/00l4vcK27rxv3dylERERkonQ5S0REJkwhIiIiE6YQERGRCVOIiIjIhClERERkwhQiIlOcmV1ybHVhkalGISIiIhOmEBGJETN7n5k9bWYbzexHwTNKjprZ9cEzLB4ys5Jg22Vm9lTw/IZfjXm2w2wze9DMnjWzDWZ2VrD77DHP/rgtmCCGmX3Tos9/ec7M/i2kQ5dpTCEiEgNmtoDoTOjV7r4MGAHeS3SRv3Xuvgh4hOjMYYBbgc+7+9lEVwg41n4b8AN3XwpcSHS1WYiuqPx3RJ9pMwtYbWZFwFuBRcF+vhbfoxR5OYWISGxcBqwA1prZxuD1LKLLx9wRbPMz4FXBszzy3f2RoP0W4GIzywGq3P1XAO7e7+69wTZPu3uDu48CG4FaokuZ9wM3mdlfE12yQmRSKUREYsOAW9x9WfAzz92/epztJrrO0Ni1nEaA5OAZGOcSXYH3jcB9E9y3yIQpRERi4yHg7WZWCi88z3om0f/Hjq0Q+7+Ax929E2g3s4uC9vcDjwRPk2wws7cE+0gzs8wTfWHw3Jc8d/8d8Gmij7gVmVTJJ99ERE7G3beY2ZeBP5hZBBgCPk70YU/nBu8dIXrfBKJLcf8wCIljK+hCNFB+ZGb/HOzjHa/wtTnAr80snWhP6DMxPiyRk9IqviJxZGZH3T077DpE4kWXs0REZMLUExERkQlTT0RERCZMISIiIhOmEBERkQlTiIiIyIQpREREZML+P2ut+OYDsgpFAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ckHqtn0srQ5q"
      },
      "source": [
        "From this, it looks like our model's loss (and MAE) were both still decreasing (in our case, MAE and loss are the same, hence the lines in the plot overlap eachother).\n",
        "\n",
        "What this tells us is the loss might go down if we try training it for longer.\n",
        "\n",
        "> 🤔 **Question:** How long should you train for? \n",
        "\n",
        "> It depends on what problem you're working on. Sometimes training won't take very long, other times it'll take longer than you expect. A common method is to set your model training for a very long time (e.g. 1000's of epochs) but set it up with an [EarlyStopping callback](https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/EarlyStopping) so it stops automatically when it stops improving. We'll see this in another module.\n",
        "\n",
        "Let's train the same model as above for a little longer. We can do this but calling fit on it again."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ucs62jV8jl6N"
      },
      "source": [
        "# Try training for a little longer (100 more epochs)\n",
        "history_2 = insurance_model_2.fit(X_train, y_train, epochs=100, verbose=0)"
      ],
      "execution_count": 97,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L2FZA1z1sxxs"
      },
      "source": [
        "How did the extra training go?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cxtiYB3qs0PZ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "9d479dec-66d0-4e02-9b2f-316910fa7b95"
      },
      "source": [
        "# Evaluate the model trained for 200 total epochs\n",
        "insurance_model_2_loss, insurance_model_2_mae = insurance_model_2.evaluate(X_test, y_test)\n",
        "insurance_model_2_loss, insurance_model_2_mae"
      ],
      "execution_count": 98,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "9/9 [==============================] - 0s 2ms/step - loss: 3494.7285 - mae: 3494.7285\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(3494.728515625, 3494.728515625)"
            ]
          },
          "metadata": {},
          "execution_count": 98
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MzUySYEOs-O_"
      },
      "source": [
        "Boom! Training for an extra 100 epochs we see about a 10% decrease in error.\n",
        "\n",
        "How does the visual look?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BtYKSLeQjuzL",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 279
        },
        "outputId": "49c03b0b-a5ce-49ac-b7e8-4f9e69ea17b9"
      },
      "source": [
        "# Plot the model trained for 200 total epochs loss curves\n",
        "pd.DataFrame(history_2.history).plot()\n",
        "plt.ylabel(\"loss\")\n",
        "plt.xlabel(\"epochs\"); # note: epochs will only show 100 since we overrid the history variable"
      ],
      "execution_count": 99,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEGCAYAAACUzrmNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhV5bn38e+dORCSMIQxQIAEEAiTYVAUZ3Cq2tm+tUWrpafHzj16aofXc2zteN5jh9Paeiqttlq1Di11nqu0Ms+DkAABEqaEQBhC5vv9Yy8wIDERsrOSnd/nunKx9rOevda9XJgfa3qWuTsiIiLvJS7sAkREpONTWIiISIsUFiIi0iKFhYiItEhhISIiLUoIu4Bo6NOnj+fk5IRdhohIp7Js2bJyd8861byYDIucnByWLl0adhkiIp2KmW1rbp5OQ4mISIsUFiIi0iKFhYiItCgmr1mIiJyuuro6SkpKqK6uDruUqElJSSE7O5vExMRWfyeqYWFmxcAhoAGod/cCM+sFPArkAMXAx9x9v5kZ8DPgSqAKuNHdlwfLmQN8O1js99z9gWjWLSJdV0lJCT169CAnJ4fIr6XY4u7s27ePkpIShg0b1urvtcdpqIvcfaK7FwSfvwG84u55wCvBZ4ArgLzgZy5wL0AQLncC04CpwJ1m1rMd6haRLqi6uprevXvHZFAAmBm9e/d+30dOYVyzuBY4dmTwAHBdk/YHPWIhkGlmA4DZwEvuXuHu+4GXgMvbu2gR6TpiNSiOOZ3ti3ZYOPCimS0zs7lBWz933xVM7wb6BdODgB1NvlsStDXXfgIzm2tmS81saVlZ2WkVu3t7IW/d90X2lGw+re+LiMSqaIfFee4+mcgpplvNbGbTmR55mUabvFDD3e9z9wJ3L8jKOuUDiC2qPlLJOTsfZNvCv7ZFSSIipyUtLS3sEt4lqmHh7qXBn3uBp4hcc9gTnF4i+HNv0L0UGNzk69lBW3PtbW7oqMnspRcJW1+LxuJFRDqtqIWFmXU3sx7HpoFZwFpgPjAn6DYHOPbP+PnApy1iOlAZnK56AZhlZj2DC9uzgra2rzkujuLM6eQeWUp9XW00ViEi0mruzm233ca4cePIz8/n0UcfBWDXrl3MnDmTiRMnMm7cON58800aGhq48cYbj/e955572rSWaN462w94KriQkgA87O7Pm9kS4DEzuxnYBnws6P8skdtmi4jcOnsTgLtXmNl3gSVBv7vcvSJaRcePvJT0xc/y9oq/M3rqZdFajYh0Av/5t3Ws33mwTZc5ZmA6d35gbKv6Pvnkk6xcuZJVq1ZRXl7OlClTmDlzJg8//DCzZ8/mW9/6Fg0NDVRVVbFy5UpKS0tZu3YtAAcOHGjTuqMWFu6+BZhwivZ9wCWnaHfg1maWNQ+Y19Y1nkrutKtpWPR19q95HhQWIhKiBQsW8IlPfIL4+Hj69evHBRdcwJIlS5gyZQqf+cxnqKur47rrrmPixIkMHz6cLVu28MUvfpGrrrqKWbNmtWkteoL7JBm9+7ExcSS9dr0ZdikiErLWHgG0t5kzZ/LGG2/wzDPPcOONN/K1r32NT3/606xatYoXXniBX//61zz22GPMm9d2/8bW2FCnUDHgfHLrNlG5b0/YpYhIF3b++efz6KOP0tDQQFlZGW+88QZTp05l27Zt9OvXj89+9rPccsstLF++nPLychobG/nwhz/M9773PZYvX96mtejI4hR65l9O/I7fUrToac6+8uawyxGRLuqDH/wgb731FhMmTMDM+PGPf0z//v154IEH+MlPfkJiYiJpaWk8+OCDlJaWctNNN9HY2AjAD37wgzatxSKXCmJLQUGBn8nLj+rraqm6eyhvZ17I1K/8qQ0rE5GObsOGDZx11llhlxF1p9pOM1vWZGimE+g01CkkJCZR1L2AnAML8SClRUS6MoVFM+qHXURfKti2sW3P+4mIdEYKi2YMmfoBAHYvezrkSkREwqewaEb/IXkUxw2m+46/h12KiEjoFBbvYXfWeYyqXk3V4cqwSxERCZXC4j10HzubJKuncHFUhqISEek0FBbvIW/KLI56EtUbFBYi0rUpLN5DSmp3NnWbyMDyf4ZdiohIqBQWLTg65CIG+05Kt2wIuxQR6SKKi4sZPXo0N954IyNHjuSTn/wkL7/8MjNmzCAvL4/FixezePFizjnnHCZNmsS5557Lxo0bAWhoaOC2225jypQpjB8/nt/85jdtUpOG+2jBoCkfgI0/omTp3xg0PPaf6hSRJp77Buxe07bL7J8PV/ywxW5FRUX8+c9/Zt68eUyZMoWHH36YBQsWMH/+fL7//e/z4IMP8uabb5KQkMDLL7/MN7/5TZ544gnuv/9+MjIyWLJkCTU1NcyYMYNZs2YxbNiwMypbYdGC7OFjKbV+JBW/Btwedjki0kUMGzaM/Px8AMaOHcsll1yCmZGfn09xcTGVlZXMmTOHwsJCzIy6ujoAXnzxRVavXs3jjz8OQGVlJYWFhQqLaLO4OEp6zyC/7Blqa6pJSk4JuyQRaS+tOAKIluTk5OPTcXFxxz/HxcVRX1/Pd77zHS666CKeeuopiouLufDCC4HI2/V+8YtfMHv27DatR9csWiF59Cy6WQ2blr4UdikiIkDkiGHQoEEA/P73vz/ePnv2bO69997jRxqbNm3iyJEjZ7w+hUUr5E27gjqP59C6F8MuRUQEgNtvv5077riDSZMmUV9ff7z9lltuYcyYMUyePJlx48bxuc997oT5p0tDlLfShrvPJd7rGPntJS13FpFOS0OUa4jyM3Kg/zmMqCukcn952KWIiLS7qIeFmcWb2Qozezr4fImZLTezlWa2wMxyg/ZkM3vUzIrMbJGZ5TRZxh1B+0Yza9urNq2UMeZS4s3ZsuT5MFYvIhKq9jiy+DLQ9Im2e4FPuvtE4GHg20H7zcB+d88F7gF+BGBmY4DrgbHA5cCvzCy+Heo+wYhJF3DUk6gpfL29Vy0i7SwWT883dTrbF9WwMLNs4Crgt02aHUgPpjOAncH0tcADwfTjwCVmZkH7I+5e4+5bgSJgajTrPpXklG4UpubTf9+i9l61iLSjlJQU9u3bF7OB4e7s27ePlJT39xhAtJ+z+CmRJ9l6NGm7BXjWzI4CB4HpQfsgYAeAu9ebWSXQO2hf2OT7JUHbCcxsLjAXYMiQIW27FYEjA2cwfsvPKd+9nT79o7MOEQlXdnY2JSUllJWVhV1K1KSkpJCdnf2+vhO1sDCzq4G97r7MzC5sMuurwJXuvsjMbgP+m0iAnBF3vw+4DyJ3Q53p8k6lT/5lsOXnFC99nj5Xz43GKkQkZImJiWf8tHMsiuZpqBnANWZWDDwCXGxmzwAT3P3YuZxHgXOD6VJgMICZJRA5RbWvaXsgO2hrd8Pzz+Ug3Wnc/HoYqxcRCU3UwsLd73D3bHfPIXKB+lUi1x8yzGxk0O0y3rn4PR+YE0x/BHjVIycN5wPXB3dLDQPygMXRqvu9xCcksLnbRLIPtO0zHCIiHV27jg0VXIv4LPCEmTUC+4HPBLPvB/5gZkVABZGAwd3XmdljwHqgHrjV3Rvas+6magafx8CN/2Dn1rcZOGx0WGWIiLSrdgkLd38deD2Yfgp46hR9qoGPNvP9u4G7o1dh6w2YODsyZPny5xQWItJl6Anu92nIqEmUk0l88RthlyIi0m4UFu+TxcVRnF5AzqFleGNj2OWIiLQLhcVp8JyZ9KaS4g0aVFBEugaFxWnIPvtyAPas0pDlItI1KCxOw4Choyix/qTsWBB2KSIi7UJhcZpKe04lt2oV9XW1YZciIhJ1CovTlJB7IWl2lKKVuitKRGKfwuI0DSuIXLfYv1bv5RaR2KewOE29+g5ic/ww0nf9M+xSRESiTmFxBsr6TCevZj1HjxwKuxQRkahSWJyB1FEXk2T1FC17OexSRESiSmFxBnKnzKLO4zm84dWwSxERiSqFxRno3iOToqRR9Cl7K+xSRESiSmFxhg70n8GIuiIqK2L3FYwiIgqLM5Qx9lLizNm85PmwSxERiRqFxRnKnXQhVZ5MXaGuW4hI7FJYnKGk5BSKUsczoGJRy51FRDophUUbqMo+jyGNpewt3Rp2KSIiUaGwaANZ42cBsG3JsyFXIiISHQqLNjBs7DT2kw5b/x52KSIiUaGwaANx8fFsSZvM0MoletWqiMSkqIeFmcWb2Qozezr4bGZ2t5ltMrMNZvalJu0/N7MiM1ttZpObLGOOmRUGP3OiXfPpqB86k75UsL1wddiliIi0ufY4svgysKHJ5xuBwcBodz8LeCRovwLIC37mAvcCmFkv4E5gGjAVuNPMerZD3e/LsVet7l6p5y1EJPZENSzMLBu4Cvhtk+bPA3e5eyOAu+8N2q8FHvSIhUCmmQ0AZgMvuXuFu+8HXgIuj2bdp2NgzlnsIovE7f8IuxQRkTYX7SOLnwK3A01P5I8APm5mS83sOTPLC9oHATua9CsJ2pprP4GZzQ2WubSsrP2H3rC4OErTJ5B9ZG27r1tEJNqiFhZmdjWw192XnTQrGah29wLgf4F5bbE+d7/P3QvcvSArK6stFvm+NQwsoC8V7N5RFMr6RUSiJZpHFjOAa8ysmMh1iYvN7I9EjgyeDPo8BYwPpkuJXMs4Jjtoa669w+k1agYApWveDLkSEZG2FbWwcPc73D3b3XOA64FX3f0G4C/ARUG3C4BNwfR84NPBXVHTgUp33wW8AMwys57Bhe1ZQVuHM3TMVKo9kbptGvpDRGJLQgjr/CHwkJl9FTgM3BK0PwtcCRQBVcBNAO5eYWbfBZYE/e5y94r2Lbl1kpJT2JCUR2bFqrBLERFpU+0SFu7+OvB6MH2AyB1SJ/dx4NZmvj+PNrq2EW2VvSYwaffj1NZUk5ScEnY5IiJtQk9wt7GknGkkWx3F63QqSkRih8KijQ3KnwlAxcYFIVciItJ2FBZtrF/2CPbSi4SdJ98xLCLSeSksoqAkbRwDD+vhPBGJHQqLKKjtfzYDfQ/lu3e03FlEpBNQWERBZt65AJTo4TwRiREKiyjIyT+XOo/n6NaFYZciItImFBZRkNItjeLE4aSXLw+7FBGRNqGwiJJ9PScyvGYjdbU1YZciInLGFBZRkjjsHFKtlq1rdSpKRDo/hUWUDJ5wMQAVb78RciUiImdOYRElfQcNYxdZJO1cHHYpIiJnTGERRaXpExh8eA3e2NhyZxGRDkxhEUUNg6aSxX52bdvUcmcRkQ5MYRFFfcZcAMDONa+FXImIyJlRWERRzlkFHPJUGrbpjigR6dwUFlEUn5DA1tQx9N2/MuxSRETOiMIiyo70LWBowzYq95eHXYqIyGlTWERZj7wZxJmzbeXrYZciInLaFBZRNmziBdR7HEc2/yPsUkRETpvCIsq698ikOGEY6XuXhl2KiMhpi3pYmFm8ma0ws6dPav+5mR1u8jnZzB41syIzW2RmOU3m3RG0bzSz2dGuua2V95lKXs16jhw6EHYpIiKnpT2OLL4MbGjaYGYFQM+T+t0M7Hf3XOAe4EdB3zHA9cBY4HLgV2YWH+2i21LauCtIsnoKFz0XdikiIqclqmFhZtnAVcBvm7TFAz8Bbj+p+7XAA8H048AlZmZB+yPuXuPuW4EiYGo0625reVMuo8qTqdnwfNiliIiclmgfWfyUSCg0HRzpC8B8d991Ut9BwA4Ad68HKoHeTdsDJUHbCcxsrpktNbOlZWVlbbcFbSA5pRsbu5/N4Ip/apwoEemUohYWZnY1sNfdlzVpGwh8FPhFW6/P3e9z9wJ3L8jKymrrxZ+x2pyLGeh72b5JD+iJSOcTzSOLGcA1ZlYMPAJcDKwDcoGioL2bmRUF/UuBwQBmlgBkAPuatgeyg7ZOZci0awDYtexvIVciIvL+RS0s3P0Od8929xwiF6hfdfee7t7f3XOC9qrggjbAfGBOMP2RoL8H7dcHd0sNA/KATveSiAFDR7EtbjDdt2tQQRHpfBLCLqCJ+4E/BEcaFUQCBndfZ2aPAeuBeuBWd28Ir8zTtyvrPCbv/jNHDh2ge4/MsMsREWm1Vh1ZmNmXzSzdIu43s+VmNqu1K3H319396lO0pzWZrnb3j7p7rrtPdfctTebd7e4j3H2Uu3fa+091C62IdFatPQ31GXc/CMwi8nzEp4AfRq2qGKVbaEWks2ptWFjw55XAH9x9XZM2aaV3bqF9K+xSRETel9aGxTIze5FIWLxgZj048dkJaaWagdMY6Hso37097FJERFqttWFxM/ANYIq7VwGJwE1RqyqGZY6cAcCO1W+EXImISOu1NizOATa6+wEzuwH4NpEnrOV9yhl3DrUeT/XWRWGXIiLSaq0Ni3uBKjObAHwd2Aw8GLWqYlhKtzSKE0eQvm9F2KWIiLRaa8OiPnhA7lrgf9z9l0CP6JUV2yp6TmBYzSbq62rDLkVEpFVaGxaHzOwOIrfMPmNmcUSuW8hpSBg6lW5WQ/H6JWGXIiLSKq0Ni48DNUSet9hNZHymn0Stqhg3cOxMAPZt1KtWRaRzaFVYBAHxEJARjCZb7e66ZnGaBgwdSTmZxJXqVasi0jm0driPjxEZvO+jwMeARWb2kWgWFsssLo4d3cbQ/+CasEsREWmV1g4k+C0iz1jsBTCzLOBlIm+0k9NQ3W8yg7f+kwPlu8ns0z/sckRE3lNrr1nEHQuKwL738V05hfS8yMN52/Rwnoh0Aq39hf+8mb1gZjea2Y3AM8Cz0Ssr9g0bP4MGN45uWRh2KSIiLWrVaSh3v83MPkzk7XcA97n7U9ErK/Z1S8tgc8IwupctD7sUEZEWtfrlR+7+BPBEFGvpcsozxzO2/AXq62pJSEwKuxwRkWa952koMztkZgdP8XPIzA62V5GxKmHETNLsKJtXLQi7FBGR9/SeYeHuPdw9/RQ/Pdw9vb2KjFXDp1wJQMXaF0OuRETkvemOphD1zBpAUfwI0nfqSW4R6dgUFiErz5pOXs16jh45FHYpIiLNinpYmFm8ma0ws6eDzw+Z2UYzW2tm88wsMWg3M/u5mRWZ2Wozm9xkGXPMrDD4mRPtmttTt9GXkGT1FC19KexSRESa1R5HFl8GNjT5/BAwGsgHUoFbgvYrgLzgZy6Rd2hgZr2AO4FpwFTgTjPr2Q51t4vcgkup9QSObHg57FJERJoV1bAws2zgKuC3x9rc/VkPEBlvKjuYdS3wYDBrIZBpZgOA2cBL7l7h7vuBl4DLo1l3e+qWlkFh8hj6lOnhPBHpuKJ9ZPFT4Hag8eQZwemnTwHPB02DgB1NupQEbc21x4yDA84lt2Ez+8t2hV2KiMgpRS0sgqHM97r7sma6/Ap4w93fbKP1zTWzpWa2tKysrC0W2W565s8CYMsSjaAiIh1TNI8sZgDXmFkx8AhwsZn9EcDM7gSygK816V8KDG7yOTtoa679BO5+n7sXuHtBVlZWW25H1OVOOJ9DnkpD0WthlyIickpRCwt3v8Pds909B7geeNXdbzCzW4hch/iEuzc9PTUf+HRwV9R0oNLddwEvALPMrGdwYXtW0BYzEhKTKOo+iUH7F4ddiojIKYXxnMWvgX7AW2a20sz+b9D+LLAFKAL+F/hXAHevAL4LLAl+7graYkrtkJkM8j2UblkXdikiIu/S6oEEz4S7vw68Hkyfcp3B3VG3NjNvHjAvSuV1CAPPvhLe/iElS59h0PCxYZcjInICPcHdQWSPyGen9SWpWNctRKTjUVh0EBYXx46e08k7soK62pqwyxEROYHCogNJGnUpaXaUwuU6uhCRjkVh0YEMn3oV9R5H5ZrnW+4sItKOFBYdSEbPPhQljab3Hg1ZLiIdi8Kig9k/4Hxy6wo19IeIdCgKiw6m5/jZxJmzZfEzYZciInKcwqKDyZt4AZV0p6HwlbBLERE5TmHRwcQnJLA5rYCcAwvxxncN1isiEgqFRQdUP+xi+lJB8YYlYZciIgIoLDqkYedcB8DuZU+HXImISITCogPKGpjD5vjhZJTo4TwR6RgUFh3U3v4XMLJmHZUVnetFTiISmxQWHVTPCVeRYI0UvfXXsEsREVFYdFR5ky/iAGk0bnox7FJERBQWHVV8QgJF6dMZUfkWjQ0NYZcjIl2cwqIjy5tFLw5StOrNsCsRkS5OYdGB5Z5zLQ1u7Fvxt7BLEZEuTmHRgWX26U9h0ln02fl62KWISBensOjgDgy6iLyGIsp2Foddioh0YQqLDm7guR8DoOjl+0OuRES6sqiHhZnFm9kKM3s6+DzMzBaZWZGZPWpmSUF7cvC5KJif02QZdwTtG81sdrRr7kiGjJzIuqR8hmx9VHdFiUho2uPI4svAhiaffwTc4+65wH7g5qD9ZmB/0H5P0A8zGwNcD4wFLgd+ZWbx7VB3h3E0/1MM8j2s+8f8sEsRkS4qqmFhZtnAVcBvg88GXAw8HnR5ALgumL42+Eww/5Kg/7XAI+5e4+5bgSJgajTr7mjGXXoD++lB3eJ5YZciIl1UtI8sfgrcDhx7MUNv4IC71wefS4BBwfQgYAdAML8y6H+8/RTfOc7M5prZUjNbWlYWW+MppaR2Z2O/q8k/9A/Kd28PuxwR6YKiFhZmdjWw192XRWsdTbn7fe5e4O4FWVlZ7bHKdjXwkn8h0RoofOHXYZciIl1QNI8sZgDXmFkx8AiR008/AzLNLCHokw2UBtOlwGCAYH4GsK9p+ym+02VELnSPZ2jx47rQLSLtLmph4e53uHu2u+cQuUD9qrt/EngN+EjQbQ5wbFjV+cFngvmvursH7dcHd0sNA/KAxdGquyM7Ov5TDPQ9rH7tsbBLEZEuJoznLP4d+JqZFRG5JnHsAYL7gd5B+9eAbwC4+zrgMWA98Dxwq7t3yX9aT5g1h53Wl+4L/5/ezy0i7coi/3iPLQUFBb506dKwy4iKxU/8lKlr7mTVzN8w4eLrwy5HRGKImS1z94JTzdMT3J3MpA98np3Wl27//C8dXYhIu1FYdDKJScmUjLuVvPpCVr+uaxci0j4UFp1Q5Oiin44uRKTdKCw6ocSkZErzI0cXy57TAIMiEn0Ki05q4tX/wsaEUYxa/B12bn077HJEJMYpLDqpxKRketzwB9yMww99itqa6rBLEpEYprDoxAbmjKJo2vcZWb+J5b/7WtjliEgMU1h0cpOvuIlFva9j+u6HeOv336C66nDYJYlIDFJYxIAJN/+SFd3P45zieznw4wksnX+vxo8SkTalsIgBKd3SmHTbM6y77GEOxWdSsPwbLJqn01Ii0nYUFjFk7IyrGPHNxSzJmMXZJX9g28aVYZckIjFCYRFj4uLjGf5/7qHaUjj45Ff00J6ItAmFRQzq3S+bDWd9ifyaFSx//oGWvyAi0gKFRYwq+PC/URQ/gsGL7+LIoQNhlyMinZzCIkbFJyRQf/mP6UsFJT+bxapXH9MpKRE5bQqLGDZ6yqUsmXg3GfUVTHjjs2y5ezLLn/udbqsVkfdNYRHjplz3BXp/cx1LJt5NgtcxedFX2PL9Kax67c860hCRVlNYdAGJSclMue4LZH9zFUsm/YBujYeZ8PdbWPuji9lZvDHs8kSkE1BYdCHxCQlMufZf6fON1Swc9e8Mr95Axu9msujPei+GiLw3hUUXlJScwvRPfJODn3mDLSlnMW3dd1n7o4vZtU1HGSJyagqLLmzA0FGM+/dXWTT2Owyv3kD6PB1liMipRS0szCzFzBab2SozW2dm/xm0X2Jmy81spZktMLPcoD3ZzB41syIzW2RmOU2WdUfQvtHMZker5q7I4uKY9tF/o/KmN9iaMppp675L4fensfBPd7O3dGvY5YlIB2HuHp0FmxnQ3d0Pm1kisAD4MvAgcK27bzCzfwWmuvuNwfR4d/8XM7se+KC7f9zMxgB/AqYCA4GXgZHu3uz9nwUFBb506dKobFcs88ZGljz1M7LW/Z5hjcUAFMbnUpExBvqNI6nnIGrKt2IVW8AbGPyBbzEwZ1S4RYtImzGzZe5ecKp5CdFaqUdS6NjLFRKDHw9+0oP2DGBnMH0t8B/B9OPA/wSBcy3wiLvXAFvNrIhIcLwVrdq7KouLY+qHvwof/irbNq5k5z//RPqutxhd8QoZFfOP9zvkqSTQgP/ueRaO+iJTPnYH8Qkn/lVqqK9n59Z1DBo+jrj4+PbeFBFpY1ELCwAziweWAbnAL919kZndAjxrZkeBg8D0oPsgYAeAu9ebWSXQO2hf2GSxJUHbyeuaC8wFGDJkSHQ2qAsZOmoiQ0dNBCJHHLtLt3CwrJSswSPJ7N2PPSWb2fOnf2X6pv+i6AdPUTbkCnqOvZi+Q0az6cX7GLL5Twz2PaxKmULOZx8io3e/kLdIRM5E1E5DnbASs0zgKeCLwF3Aj4LguA0Y5e63mNla4HJ3Lwm+sxmYRuRoY6G7/zFovx94zt0fb259Og3VPryxkWXP3U/vZb9gWOO2E+atT8qnss9kzi79I+Vxvam67vcMGD6WHRuWULltFV5fg8UlYPGJ9MydQu74GVic7rcQCVMop6GacvcDZvYacAUwwd0XBbMeBZ4PpkuBwUCJmSUQOUW1r0n7MdlBm4TM4uIouOqzcNVn2benhOJlL1G7ez19Cz7ImPHnArBx6XVkPn0LQ5/8APE0MtpO8Y+TtbDrL1ls63cJ8QPyiU/NIDE1nYGjzqZ3v+w2q/dQZQVvv/Ynxl32aVK792iz5Yp0BdG8wJ0F1AVBkQq8CPwI+D1wrrtvMrObgSvd/cNmdiuQ3+QC94fc/WNmNhZ4mHcucL8C5OkCd+exb08JhU9+F09OJ3XwJPqNPJvU7hnU19dRW11FyYoXSNr0DGOqlpJk9ce/V+vxrEm/gNQZn2Po2Ons3b6Ryp2bIS6OQWdNJ2tgzrvW5Y2NrP77ExzduYGxV3+BHhm9AKjYW0rFbz5AbsNmdthAjn7gXkZOvrCd/guIdA7vdWQRzbAYDzwAxBO5Rfcxd7/LzD5I5FRUI7Af+Iy7bzGzFOAPwCSgArje3bcEy/oW8BmgHviKuz/3XutWWHROVYcr2b93J9WH91N9aD+HVv2VMXv/RjpVp+xfRk92po6kqvdYkrMn0FBVSdba35LTuB2A3fRhz8wf0H/k2VTPu+ZWSIUAAA7qSURBVIZ+DXtYOeLz5Gx5mD5ewZIhn2Hk1V9t06MXkc4slLAIk8IidlQdrmTtSw/ScHA3SX2G0WNALo11tRzYsoT43SvJOvQ22Q0lJFjkQcItcTnsm/A50vqPIPWFr5PTuIODdCPOne2X/44x51xB5f5yNv3+80ypfBGAovgRlPWdQVL2BDKHjGPg8LE6TSVdksJCYlp11WG2v72MxroaRk259PiF8prqKpY/9B36l75M/TX/Q97E80/4XtGqf1C+4hnSd75BXs16Ei1yZrPRjdK4AexNG01d33zSc89h+ITzSemW1uqaGurr2bjkJQDOmjZbF++lU1BYiLSguuowu7auo2Lbemp3ryelfB39qzYxgDIgcv1ka2IeRxMziW+sJc7riPNGIo8NQW1CGtXds/H0bOxgCSPKX6UPkTcUrk4pIOO6nzB09OQT1tlQX0/plrVUHdzHyEkX6nkUCZ3CQuQ07S/bxbZVr3N08wIyyleS1HiUBkukIS6RRnvnl3tK/SGyGnaTwRGOehIb0qbjY66lrnIXYzb9im5ezZq0c3GLI66hlm51FWTXbaOb1QCwNW4o+yZ+nvxZc9i88g0OrniKjIo1VOZcTv41X6J7j8xW1+yNjWwvXE1mnwF6vkXeF4WFSDs5eGAfSUnJJ5yyqthbSuGj3yS74i3qLYk6S6I6oQeHM0YRNyAfcPqu+V9yGrdT5/EkWgO1nsDO+IHkNG6nku6sH/hhkgdPJrXXQHr0HkSfgTnH1+GNjews3sDuDW/RUPQqOfvfoi8VVHkyq/tdR84HbqdP/yFsXbuQfRv+DvW1pGaPJWv4JPoPztURjRynsBDp4BobGlj9+p+p2fACCcPPY9R5HyItvSdvL32FqtfuYeLhBcSd9IzKftLZH9eLPo17j98xdpBuFKUVUDf0QuJKFjHxwMsA1JFw/CimqTJ6srnfbPqc+ylG5J/b7LWV6qrDbFz4LFU7VpLUZzi9cvIZOCKf5JRubfxfQsKksBDp5Cr37WHfrmKOVOykuqKU+gMlxB3aSXLVHmq69ccGTKBn7hRyxkwlMSn5+Pd2bdvItud+htUfJWHYDAZPvITkbj3YtWk5ldtXk7T1VcYeWUiSNVBOJgfie3MksRd1SRm4JeAWR3LNPkZWrSDVak+oqcGNPXF9KU/OpqrHcLrnX8VZ51xFQmLSu+o/VFnBxgVPkJk9htwJM6L9n0tOk8JCRJpVuW8Pb7/6R+JKl5BUXU73ugq6NR7GvJE4GqmJS2FXr+mkjr2SoRMvpLykiP3b11C/awOJlVtJr9rOoPoddLMaKkinsPfFeJ+RJKT3JSGpO7Xr/sa4/a8cP7LZmDCag/lz6NE/l6MHdlF7sIz0IfmMnnKZ7hoLmcJCRKKq+ugR1r/xJL7mccYc+ucJRyFHPIV1vS8jbeoNHNyylEGFDzHYd75rGYUJeVROnEv+pTfo9FZIFBYi0m4aGxqorNjLwfKdVB0sZ8iYaSfczeWNjWxY/CL1Rw/TrdcA0nr2ZduivzJgwzyGNJbS6EaFZVCR0JejiT2pT+hOQ0I3PD4ZzHAMT+xGfFYuPQaOIrN/DnHxkWHuEpNS6NlngI5QTpPCQkQ6vMaGBta88SRVWxcTf6iUlKO76VZ3gOTGo6T4UZKoxYK+qV59/CHKkx2kOzsTh3IkpT+pNWX0qt1Nuh+kKHU81blXMuzcD1FXW8OBXVs4WlFK9z5DGDS64Pg4Yl2ZwkJEYkp9XS17dhRSvm09NRUlx9sba45g+wrpcWgzmXV7qUzozZHUQTQkdmdwxVsM9D3NLnOn9aU0fRJxI2eRe+51ZPTsQ2NDA0erDlFWUsT+7euo3r0Ri08kdeBYsoZPeNetx4Ur36Ty1Z8x8NAqEr2OBBo4GJfO3nFzmXj1v5xw80FHpLAQkS7PGxvZsnYhZWteIi4lg9SsoaT1Hkjl7q0cLVlNctlahh9eRiaHaXCjlsR33QF2sipPpiRxKAd65NH9yHbG1q7hsKeyMf2c4NRZEr0OrCGvvpAS60/pmLn0z7+I7Nzx73q75Ola8eIfSc3sx+ipl53xshQWIiKt0FBfT+GK19m/+nmsrgpP7IYldSMhcxAZg8cwYPg4Gupq2Vm0koPb18DeDaRVFjKwditHLYXtuTcw5qovkJ7Z+/gyvbGRVa89Rto/f0xuw2bgnZA5mtiT2qRMGlJ6YT2HktJ3BJkDRpCa3ovk1DRSu/cgKTml2VqX/uZzTCuLvAdufVI+9ed+lfyZHzztazYKCxGRkHljI1vXL2Ff0RIadq6ie2URqfWVpDUcJNMPkGJ1p/zeIU/lQFxPDiX0orLPZDImXMWgUQVsve+TTDy6kIV9Pw6ZQxi2aR792MeyHhdx9tf/clo1KixERDowb2xk394Syrdv5PDerTQcPYTXVeE1h4mrKifhaDndq/cwom4TidZAg0fuCls29g6mfex2AGprqln1zG9ISOvNpFk3nFYdCgsRkRhw8MA+it6aT92WBXQffw3jzr+2TZcf+ju4RUTkzKVn9mbyFTcBN7X7uvXkioiItEhhISIiLVJYiIhIixQWIiLSoqiFhZmlmNliM1tlZuvM7D+DdjOzu81sk5ltMLMvNWn/uZkVmdlqM5vcZFlzzKww+JkTrZpFROTUonk3VA1wsbsfNrNEYIGZPQecBQwGRrt7o5n1DfpfAeQFP9OAe4FpZtYLuBMoABxYZmbz3X1/FGsXEZEmonZk4RGHg4+JwY8DnwfucvfGoN/eoM+1wIPB9xYCmWY2AJgNvOTuFUFAvARcHq26RUTk3aJ6zcLM4s1sJbCXyC/8RcAI4ONmttTMnjOzvKD7IGBHk6+XBG3NtZ+8rrnBMpeWlZVFY3NERLqsqD6U5+4NwEQzywSeMrNxQDJQ7e4FZvYhYB5wfhus6z7gPgAzKzOzbWewuD5A+ZnW1Ml0xW2Grrnd2uau4/1u99DmZrTLE9zufsDMXiNy+qgEeDKY9RTwu2C6lMi1jGOyg7ZS4MKT2l9vYX1ZZ1KvmS1t7pH3WNUVtxm65nZrm7uOttzuaN4NlRUcUWBmqcBlwNvAX4CLgm4XAJuC6fnAp4O7oqYDle6+C3gBmGVmPc2sJzAraBMRkXYSzSOLAcADZhZPJJQec/enzWwB8JCZfRU4DNwS9H8WuBIoAqoIBj9x9woz+y6wJOh3l7tXRLFuERE5SdTCwt1XA5NO0X4AuOoU7Q7c2syy5hG5ttFe7mvHdXUUXXGboWtut7a562iz7Y7JIcpFRKRtabgPERFpkcJCRERapLBowswuN7ONwfhU3wi7nmgws8Fm9pqZrQ/G7Ppy0N7LzF4Kxt96KbjzLOYED4quMLOng8/DzGxRsM8fNbOksGtsS2aWaWaPm9nbwVhs53SFfW1mXw3+fq81sz8FY9XF3L42s3lmttfM1jZpO+X+fa/x91pDYREI7tr6JZExqsYAnzCzMeFWFRX1wNfdfQwwHbg12M5vAK+4ex7wSvA5Fn0Z2NDk84+Ae9w9F9gP3BxKVdHzM+B5dx8NTCCy7TG9r81sEPAloMDdxwHxwPXE5r7+Pe8e/qi5/dt0/L25RMbfazWFxTumAkXuvsXda4FHiIxXFVPcfZe7Lw+mDxH55TGIyLY+EHR7ALgunAqjx8yyidyJ99vgswEXA48HXWJqu80sA5gJ3A/g7rXB3Ygxv6+J3OmZamYJQDdgFzG4r939DeDkRwma27/Njb/XKgqLd7RqDKpYYmY5RG5vXgT0Cx6CBNgN9AuprGj6KXA70Bh87g0ccPf64HOs7fNhQBnwu+DU22/NrDsxvq/dvRT4L2A7kZCoBJYR2/u6qeb27xn9jlNYdFFmlgY8AXzF3Q82nRc88xJT91Sb2dXAXndfFnYt7SgBmAzc6+6TgCOcdMopRvd1TyL/ih4GDAS600VHqm7L/auweEdzY1PFnOD9Ik8AD7n7sXG69hw7JA3+3Nvc9zupGcA1ZlZM5BTjxUTO52cGpyog9vZ5CVASjPYMkVMwk4n9fX0psNXdy9y9jshYdDOI7X3dVHP794x+xyks3rEEyAvumEgickFsfsg1tbngPP39wAZ3/+8ms+YDx95COAf4a3vXFk3ufoe7Z7t7DpF9+6q7fxJ4DfhI0C2mttvddwM7zGxU0HQJsJ4Y39dETj9NN7Nuwd/3Y9sds/v6JM3t3+bG32sVPcHdhJldSeS8djwwz93vDrmkNmdm5wFvAmt459z9N4lct3gMGAJsAz4Wq2NwmdmFwL+5+9VmNpzIkUYvYAVwg7vXhFlfWzKziUQu6CcBW4iMuRZHjO9ri7zG+eNE7v5bQWQMukHE2L42sz8RGZW7D7CHyFtF/8Ip9m8QnP9D5JRcFXCTuy9t9boUFiIi0hKdhhIRkRYpLEREpEUKCxERaZHCQkREWqSwEBGRFiksRDoAM7vw2Ei4Ih2RwkJERFqksBB5H8zsBjNbbGYrzew3wfsxDpvZPcH7E14xs6yg70QzWxi8O+CpJu8VyDWzl81slZktN7MRweLTmrx74qHgISrM7IcWef/IajP7r5A2Xbo4hYVIK5nZWUSeCp7h7hOBBuCTRAaqW+ruY4G/E3mKFuBB4N/dfTyRJ+aPtT8E/NLdJwDnEhkZFSIjAH+FyPtUhgMzzKw38EFgbLCc70V3K0VOTWEh0nqXAGcDS8xsZfB5OJFhUx4N+vwROC94l0Smu/89aH8AmGlmPYBB7v4UgLtXu3tV0Gexu5e4eyOwEsghMrx2NXC/mX2IyDANIu1OYSHSegY84O4Tg59R7v4fp+h3umPoNB2nqAFICN6/MJXIiLFXA8+f5rJFzojCQqT1XgE+YmZ94fi7jocS+f/o2Gim/wdY4O6VwH4zOz9o/xTw9+DthCVmdl2wjGQz69bcCoP3jmS4+7PAV4m8GlWk3SW03EVEANx9vZl9G3jRzOKAOuBWIi8VmhrM20vkugZEhof+dRAGx0Z8hUhw/MbM7gqW8dH3WG0P4K9mlkLkyOZrbbxZIq2iUWdFzpCZHXb3tLDrEIkmnYYSEZEW6chCRERapCMLERFpkcJCRERapLAQEZEWKSxERKRFCgsREWnR/wdjogHMNr0tYwAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HS8xYkvIuVZZ"
      },
      "source": [
        "## Preprocessing data (normalization and standardization)\n",
        "\n",
        "A common practice when working with neural networks is to make sure all of the data you pass to them is in the range 0 to 1.\n",
        "\n",
        "This practice is called **normalization** (scaling all values from their original range to, e.g. between 0 and 100,000 to be between 0 and 1).\n",
        "\n",
        "There is another process call **standardization** which converts all of your data to unit variance and 0 mean.\n",
        "\n",
        "These two practices are often part of a preprocessing pipeline (a series of functions to prepare your data for use with neural networks).\n",
        "\n",
        "Knowing this, some of the major steps you'll take to preprocess your data for a neural network include:\n",
        "* Turning all of your data to numbers (a neural network can't handle strings).\n",
        "* Making sure your data is in the right shape (verifying input and output shapes).\n",
        "* [**Feature scaling**](https://scikit-learn.org/stable/modules/preprocessing.html#preprocessing-scaler):\n",
        "    * Normalizing data (making sure all values are between 0 and 1). This is done by subtracting the minimum value then dividing by the maximum value minus the minmum. This is also referred to as min-max scaling.\n",
        "    * Standardization (making sure all values have a mean of 0 and a variance of 1). This is done by substracting the mean value from the target feature and then dividing it by the standard deviation.\n",
        "    * Which one should you use?\n",
        "      * **With neural networks you'll tend to favour normalization** as they tend to prefer values between 0 and 1 (you'll see this espcially with image processing), however, you'll often find a neural network can perform pretty well with minimal feature scaling.\n",
        "\n",
        "> 📖 **Resource:** For more on preprocessing data, I'd recommend reading the following resources:\n",
        "* [Scikit-Learn's documentation on preprocessing data](https://scikit-learn.org/stable/modules/preprocessing.html#preprocessing-data).\n",
        "* [Scale, Standardize or Normalize with Scikit-Learn by Jeff Hale](https://towardsdatascience.com/scale-standardize-or-normalize-with-scikit-learn-6ccc7d176a02).\n",
        "\n",
        "We've already turned our data into numbers using `get_dummies()`, let's see how we'd normalize it as well."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9v7P20A2d7H6"
      },
      "source": [
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "import tensorflow as tf\n",
        "\n",
        "# Read in the insurance dataset\n",
        "insurance = pd.read_csv(\"https://raw.githubusercontent.com/stedy/Machine-Learning-with-R-datasets/master/insurance.csv\")"
      ],
      "execution_count": 100,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vir8UAIwlUOo",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "306bc0ed-8038-4822-b69e-c0181f5c8f88"
      },
      "source": [
        "# Check out the data\n",
        "insurance.head()"
      ],
      "execution_count": 101,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>sex</th>\n",
              "      <th>bmi</th>\n",
              "      <th>children</th>\n",
              "      <th>smoker</th>\n",
              "      <th>region</th>\n",
              "      <th>charges</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>19</td>\n",
              "      <td>female</td>\n",
              "      <td>27.900</td>\n",
              "      <td>0</td>\n",
              "      <td>yes</td>\n",
              "      <td>southwest</td>\n",
              "      <td>16884.92400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>18</td>\n",
              "      <td>male</td>\n",
              "      <td>33.770</td>\n",
              "      <td>1</td>\n",
              "      <td>no</td>\n",
              "      <td>southeast</td>\n",
              "      <td>1725.55230</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>28</td>\n",
              "      <td>male</td>\n",
              "      <td>33.000</td>\n",
              "      <td>3</td>\n",
              "      <td>no</td>\n",
              "      <td>southeast</td>\n",
              "      <td>4449.46200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>33</td>\n",
              "      <td>male</td>\n",
              "      <td>22.705</td>\n",
              "      <td>0</td>\n",
              "      <td>no</td>\n",
              "      <td>northwest</td>\n",
              "      <td>21984.47061</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>32</td>\n",
              "      <td>male</td>\n",
              "      <td>28.880</td>\n",
              "      <td>0</td>\n",
              "      <td>no</td>\n",
              "      <td>northwest</td>\n",
              "      <td>3866.85520</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   age     sex     bmi  children smoker     region      charges\n",
              "0   19  female  27.900         0    yes  southwest  16884.92400\n",
              "1   18    male  33.770         1     no  southeast   1725.55230\n",
              "2   28    male  33.000         3     no  southeast   4449.46200\n",
              "3   33    male  22.705         0     no  northwest  21984.47061\n",
              "4   32    male  28.880         0     no  northwest   3866.85520"
            ]
          },
          "metadata": {},
          "execution_count": 101
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SHMQiNosg3J3"
      },
      "source": [
        "Now, just as before, we need to transform the non-numerical columns into numbers and this time we'll also be normalizing the numerical columns with different ranges (to make sure they're all between 0 and 1).\n",
        "\n",
        "To do this, we're going to use a few classes from Scikit-Learn:\n",
        "* [`make_column_transformer`](https://scikit-learn.org/stable/modules/generated/sklearn.compose.make_column_transformer.html) - build a multi-step data preprocessing function for the folllowing transformations:\n",
        "  * [`MinMaxScaler`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MinMaxScaler.html) - make sure all numerical columns are normalized (between 0 and 1).\n",
        "  * [`OneHotEncoder`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html) - one hot encode the non-numerical columns.\n",
        "\n",
        "Let's see them in action."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-x9JwbV0hqWh"
      },
      "source": [
        "from sklearn.compose import make_column_transformer\n",
        "from sklearn.preprocessing import MinMaxScaler, OneHotEncoder\n",
        "\n",
        "# Create column transformer (this will help us normalize/preprocess our data)\n",
        "ct = make_column_transformer(\n",
        "    (MinMaxScaler(), [\"age\", \"bmi\", \"children\"]), # get all values between 0 and 1\n",
        "    (OneHotEncoder(handle_unknown=\"ignore\"), [\"sex\", \"smoker\", \"region\"])\n",
        ")\n",
        "\n",
        "# Create X & y\n",
        "X = insurance.drop(\"charges\", axis=1)\n",
        "y = insurance[\"charges\"]\n",
        "\n",
        "# Build our train and test sets (use random state to ensure same split as before)\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
        "\n",
        "# Fit column transformer on the training data only (doing so on test data would result in data leakage)\n",
        "ct.fit(X_train)\n",
        "\n",
        "# Transform training and test data with normalization (MinMaxScalar) and one hot encoding (OneHotEncoder)\n",
        "X_train_normal = ct.transform(X_train)\n",
        "X_test_normal = ct.transform(X_test)"
      ],
      "execution_count": 102,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tz58y3nPiBJ-"
      },
      "source": [
        "Now we've normalized it and one-hot encoding it, what does our data look like now?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VODt2YiziK45",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0aacd372-5097-40d5-d27c-daf5cc32255a"
      },
      "source": [
        "# Non-normalized and non-one-hot encoded data example\n",
        "X_train.loc[0]"
      ],
      "execution_count": 103,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "age                19\n",
              "sex            female\n",
              "bmi              27.9\n",
              "children            0\n",
              "smoker            yes\n",
              "region      southwest\n",
              "Name: 0, dtype: object"
            ]
          },
          "metadata": {},
          "execution_count": 103
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mMYDXdwUnNVt",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "793a3869-9588-4861-8923-ae8adbb530fc"
      },
      "source": [
        "# Normalized and one-hot encoded example\n",
        "X_train_normal[0]"
      ],
      "execution_count": 104,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([0.60869565, 0.10734463, 0.4       , 1.        , 0.        ,\n",
              "       1.        , 0.        , 0.        , 1.        , 0.        ,\n",
              "       0.        ])"
            ]
          },
          "metadata": {},
          "execution_count": 104
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9iI4KtfWib44"
      },
      "source": [
        "How about the shapes?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SFmxzqrWntj7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e13b80e2-e9c6-443b-8178-520113ced7b9"
      },
      "source": [
        "# Notice the normalized/one-hot encoded shape is larger because of the extra columns\n",
        "X_train_normal.shape, X_train.shape"
      ],
      "execution_count": 105,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "((1070, 11), (1070, 6))"
            ]
          },
          "metadata": {},
          "execution_count": 105
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MST951aYijTS"
      },
      "source": [
        "Our data is normalized and numerical, let's model it.\n",
        "\n",
        "We'll use the same model as `insurance_model_2`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TdHnIQqll83Y",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "2efbfe2f-b476-4f39-a35d-3a3f3c7fa81b"
      },
      "source": [
        "# Set random seed\n",
        "tf.random.set_seed(42)\n",
        "\n",
        "# Build the model (3 layers, 100, 10, 1 units)\n",
        "insurance_model_3 = tf.keras.Sequential([\n",
        "  tf.keras.layers.Dense(100),\n",
        "  tf.keras.layers.Dense(10),\n",
        "  tf.keras.layers.Dense(1)\n",
        "])\n",
        "\n",
        "# Compile the model\n",
        "insurance_model_3.compile(loss=tf.keras.losses.mae,\n",
        "                          optimizer=tf.keras.optimizers.Adam(),\n",
        "                          metrics=['mae'])\n",
        "\n",
        "# Fit the model for 200 epochs (same as insurance_model_2)\n",
        "insurance_model_3.fit(X_train_normal, y_train, epochs=200, verbose=0) "
      ],
      "execution_count": 106,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f8df4a6da50>"
            ]
          },
          "metadata": {},
          "execution_count": 106
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-DjwktO6jW41"
      },
      "source": [
        "Let's evaluate the model on normalized test set."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sBcXZu9AnZfP",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d42dbf4e-807b-4de2-c38c-b3cf2dbb5e70"
      },
      "source": [
        "# Evaulate 3rd model\n",
        "insurance_model_3_loss, insurance_model_3_mae = insurance_model_3.evaluate(X_test_normal, y_test)"
      ],
      "execution_count": 107,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "9/9 [==============================] - 0s 2ms/step - loss: 3171.5774 - mae: 3171.5774\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZlHro290jhtX"
      },
      "source": [
        "And finally, let's compare the results from `insurance_model_2` (trained on non-normalized data) and `insurance_model_3` (trained on normalized data)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ybZtnVlNjCJO",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ef1d85b1-f6e5-4d54-9e9a-509040916288"
      },
      "source": [
        "# Compare modelling results from non-normalized data and normalized data\n",
        "insurance_model_2_mae, insurance_model_3_mae"
      ],
      "execution_count": 108,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(3494.728515625, 3171.577392578125)"
            ]
          },
          "metadata": {},
          "execution_count": 108
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gUttViY4jzi8"
      },
      "source": [
        "From this we can see normalizing the data results in 10% less error using the same model than not normalizing the data.\n",
        "\n",
        "This is **one of the main benefits of normalization: faster convergence time** (a fancy way of saying, your model gets to better results faster).\n",
        "\n",
        "`insurance_model_2` may have eventually achieved the same results as `insurance_model_3` if we left it training for longer. \n",
        "\n",
        "Also, the results may change if we were to alter the architectures of the models, e.g. more hidden units per layer or more layers.\n",
        "\n",
        "But since our main goal as neural network practitioners is to decrease the time between experiments, anything that helps us get better results sooner is a plus."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NhIPO_KqocHP"
      },
      "source": [
        "## 🛠 Exercises\n",
        "\n",
        "We've a covered a whole lot pretty quickly.\n",
        "\n",
        "So now it's time to have a **play around** with a few things and start to build up your intuition.\n",
        "\n",
        "I emphasise the words play around because that's very important. Try a few things out, run the code and see what happens.\n",
        "\n",
        "1. Create your own regression dataset (or make the one we created in \"Create data to view and fit\" bigger) and build fit a model to it.\n",
        "2. Try building a neural network with 4 Dense layers and fitting it to your own regression dataset, how does it perform?\n",
        "3. Try and improve the results we got on the insurance dataset, some things you might want to try include:\n",
        "  * Building a larger model (how does one with 4 dense layers go?).\n",
        "  * Increasing the number of units in each layer.\n",
        "  * Lookup the documentation of [Adam](https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/Adam) and find out what the first parameter is, what happens if you increase it by 10x?\n",
        "  * What happens if you train for longer (say 300 epochs instead of 200)? \n",
        "4. Import the [Boston pricing dataset](https://www.tensorflow.org/api_docs/python/tf/keras/datasets/boston_housing/load_data) from TensorFlow [`tf.keras.datasets`](https://www.tensorflow.org/api_docs/python/tf/keras/datasets) and model it.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AyiHG2nubmu7"
      },
      "source": [
        "## 📖 Extra curriculum\n",
        "\n",
        "If you're looking for extra materials relating to this notebook, I'd check out the following:\n",
        "\n",
        "* [MIT introduction deep learning lecture 1](https://youtu.be/njKP3FqW3Sk) - gives a great overview of what's happening behind all of the code we're running.\n",
        "* Reading: 1-hour of [Chapter 1 of Neural Networks and Deep Learning](http://neuralnetworksanddeeplearning.com/chap1.html) by Michael Nielson - a great in-depth and hands-on example of the intuition behind neural networks.\n",
        "\n",
        "To practice your regression modelling with TensorFlow, I'd also encourage you to look through [Lion Bridge's collection of datasets](https://lionbridge.ai/datasets/) or [Kaggle's datasets](https://www.kaggle.com/data), find a regression dataset which sparks your interest and try to model."
      ]
    }
  ]
}