Non_linear_Regression_Exercise_424.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyOSp0rr2nAVKxe7sKfpAM9P",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/jhomolos/data_projects/blob/main/Non_linear_Regression_Exercise_424.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Non-linear Regression Methods**\n",
        "\n",
        "The new mission now is to predict house prices using a non-linear regression method. All the steps of training the models in Random Forest, AdaBoost and Gradient Boosting and extracting the feature importance for each algorithm will be carried out.\n",
        "\n",
        "A description of the features follows:\n",
        "\n",
        "**ID:** unique key for each house\n",
        "\n",
        "**MSSubClass:** The building class\n",
        "\n",
        "**LotFrontage:** Linear feet of street connected to property\n",
        "\n",
        "**LotArea:** Lot size in square feet\n",
        "\n",
        "**OverallQual:** Overall material and finish quality\n",
        "\n",
        "**OverallCond:** Overall condition rating\n",
        "\n",
        "**YearBuilt:** Original construction date\n",
        "\n",
        "**YearRemodAdd:** Remodel date\n",
        "\n",
        "**MasVnrArea:** Masonry veneer area in square feet\n",
        "\n",
        "**ExterQual:** Exterior material quality\n",
        "\n",
        "**ExterCond:** Present condition of the material on the exterior\n",
        "\n",
        "**BsmtFinSF1:** Type 1 finished square feet\n",
        "\n",
        "**BsmtFinSF2:** Type 2 finished square feet\n",
        "\n",
        "**BsmtUnfSF:** Unfinished square feet of basement area\n",
        "\n",
        "**TotalBsmtSF:** Total square feet of basement area\n",
        "\n",
        "**1stFlrSF:** First Floor square feet\n",
        "\n",
        "**2ndFlrSF:** Second floor square feet\n",
        "\n",
        "**LowQualFinSF:** Low quality finished square feet (all floors)\n",
        "\n",
        "**GrLivArea:** Above grade (ground) living area square feet\n",
        "\n",
        "**BsmtFullBath:** Basement full bathrooms\n",
        "\n",
        "**BsmtHalfBath:** Basement half bathrooms\n",
        "\n",
        "**FullBath:** Full bathrooms above grade\n",
        "\n",
        "**HalfBath:** Half baths above grade\n",
        "\n",
        "**BedroomAbvGr:** Number of bedrooms above grade\n",
        "\n",
        "**KitchenAbvGr:** Number of kitches above grade\n",
        "\n",
        "**TotRmsAbvGrd:** Total rooms above grade (does not include bathrooms)\n",
        "\n",
        "**Fireplaces:** Number of fireplaces\n",
        "\n",
        "**GarageYrBlt:** Year garage was built\n",
        "\n",
        "**GarageCars:** Size of garage in car capacity\n",
        "\n",
        "**GarageArea:** Size of garage in square feet\n",
        "\n",
        "**WoodDeckSF:** Wood deck area in square feet\n",
        "\n",
        "**OpenPorchSF:** Open porch area in square feet\n",
        "\n",
        "**EnclosedPorch:** Enclosed porch area in square feet\n",
        "\n",
        "**3SsnPorch:** Three season porch area in square feet\n",
        "\n",
        "**ScreenPorch:** Screen porch area in square feet\n",
        "\n",
        "**PoolArea:** Pool area in square feet\n",
        "\n",
        "**MiscVal:** $Value of miscellaneous feature\n",
        "\n",
        "**MoSold:** Month Sold\n",
        "\n",
        "**YrSold:** Year Sold\n",
        "\n",
        "**SalePrice:** the property's sale price in dollars. This is the target variable\n",
        "that you're trying to predict.\n"
      ],
      "metadata": {
        "id": "njTJBSf4p1Cu"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#importing libraries\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "from sklearn.preprocessing import LabelEncoder\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.ensemble import RandomForestRegressor, AdaBoostRegressor, GradientBoostingRegressor\n",
        "from sklearn import metrics"
      ],
      "metadata": {
        "id": "UiNEHjakqP7Q"
      },
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 255
        },
        "id": "VrqFS6bQpoeo",
        "outputId": "a497e66f-d064-4ed9-c7aa-26cdce4fb73a"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   Id  MSSubClass  LotFrontage  LotArea  OverallQual  OverallCond  YearBuilt  \\\n",
              "0   1          60         65.0     8450            7            5       2003   \n",
              "1   2          20         80.0     9600            6            8       1976   \n",
              "2   3          60         68.0    11250            7            5       2001   \n",
              "3   4          70         60.0     9550            7            5       1915   \n",
              "4   5          60         84.0    14260            8            5       2000   \n",
              "\n",
              "   YearRemodAdd  MasVnrArea ExterQual  ... WoodDeckSF  OpenPorchSF  \\\n",
              "0          2003       196.0        Gd  ...          0           61   \n",
              "1          1976         0.0        TA  ...        298            0   \n",
              "2          2002       162.0        Gd  ...          0           42   \n",
              "3          1970         0.0        TA  ...          0           35   \n",
              "4          2000       350.0        Gd  ...        192           84   \n",
              "\n",
              "   EnclosedPorch  3SsnPorch  ScreenPorch  PoolArea  MiscVal  MoSold  YrSold  \\\n",
              "0              0          0            0         0        0       2    2008   \n",
              "1              0          0            0         0        0       5    2007   \n",
              "2              0          0            0         0        0       9    2008   \n",
              "3            272          0            0         0        0       2    2006   \n",
              "4              0          0            0         0        0      12    2008   \n",
              "\n",
              "   SalePrice  \n",
              "0     208500  \n",
              "1     181500  \n",
              "2     223500  \n",
              "3     140000  \n",
              "4     250000  \n",
              "\n",
              "[5 rows x 40 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-b9cd436c-6120-46df-adcd-a82a727e3751\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Id</th>\n",
              "      <th>MSSubClass</th>\n",
              "      <th>LotFrontage</th>\n",
              "      <th>LotArea</th>\n",
              "      <th>OverallQual</th>\n",
              "      <th>OverallCond</th>\n",
              "      <th>YearBuilt</th>\n",
              "      <th>YearRemodAdd</th>\n",
              "      <th>MasVnrArea</th>\n",
              "      <th>ExterQual</th>\n",
              "      <th>...</th>\n",
              "      <th>WoodDeckSF</th>\n",
              "      <th>OpenPorchSF</th>\n",
              "      <th>EnclosedPorch</th>\n",
              "      <th>3SsnPorch</th>\n",
              "      <th>ScreenPorch</th>\n",
              "      <th>PoolArea</th>\n",
              "      <th>MiscVal</th>\n",
              "      <th>MoSold</th>\n",
              "      <th>YrSold</th>\n",
              "      <th>SalePrice</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>60</td>\n",
              "      <td>65.0</td>\n",
              "      <td>8450</td>\n",
              "      <td>7</td>\n",
              "      <td>5</td>\n",
              "      <td>2003</td>\n",
              "      <td>2003</td>\n",
              "      <td>196.0</td>\n",
              "      <td>Gd</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>61</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>2008</td>\n",
              "      <td>208500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>20</td>\n",
              "      <td>80.0</td>\n",
              "      <td>9600</td>\n",
              "      <td>6</td>\n",
              "      <td>8</td>\n",
              "      <td>1976</td>\n",
              "      <td>1976</td>\n",
              "      <td>0.0</td>\n",
              "      <td>TA</td>\n",
              "      <td>...</td>\n",
              "      <td>298</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>5</td>\n",
              "      <td>2007</td>\n",
              "      <td>181500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>60</td>\n",
              "      <td>68.0</td>\n",
              "      <td>11250</td>\n",
              "      <td>7</td>\n",
              "      <td>5</td>\n",
              "      <td>2001</td>\n",
              "      <td>2002</td>\n",
              "      <td>162.0</td>\n",
              "      <td>Gd</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>42</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>9</td>\n",
              "      <td>2008</td>\n",
              "      <td>223500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>70</td>\n",
              "      <td>60.0</td>\n",
              "      <td>9550</td>\n",
              "      <td>7</td>\n",
              "      <td>5</td>\n",
              "      <td>1915</td>\n",
              "      <td>1970</td>\n",
              "      <td>0.0</td>\n",
              "      <td>TA</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>35</td>\n",
              "      <td>272</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>2006</td>\n",
              "      <td>140000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>60</td>\n",
              "      <td>84.0</td>\n",
              "      <td>14260</td>\n",
              "      <td>8</td>\n",
              "      <td>5</td>\n",
              "      <td>2000</td>\n",
              "      <td>2000</td>\n",
              "      <td>350.0</td>\n",
              "      <td>Gd</td>\n",
              "      <td>...</td>\n",
              "      <td>192</td>\n",
              "      <td>84</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>12</td>\n",
              "      <td>2008</td>\n",
              "      <td>250000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 40 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-b9cd436c-6120-46df-adcd-a82a727e3751')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-b9cd436c-6120-46df-adcd-a82a727e3751 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-b9cd436c-6120-46df-adcd-a82a727e3751');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-c9ad2348-874a-4cfa-91d6-e55933268840\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-c9ad2348-874a-4cfa-91d6-e55933268840')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-c9ad2348-874a-4cfa-91d6-e55933268840 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "house"
            }
          },
          "metadata": {},
          "execution_count": 8
        }
      ],
      "source": [
        "#reading excel file with dataset\n",
        "house = pd.read_excel(\"house.xlsx\")\n",
        "house.head()"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# now we will verity the datatypes to see if some features need to be encoded from categorical to numerical\n",
        "house.dtypes"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-AAp9zqNru9u",
        "outputId": "1798890e-7c73-42eb-dac9-0e96062b6f8d"
      },
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "MSSubClass         int64\n",
              "LotFrontage      float64\n",
              "LotArea            int64\n",
              "OverallQual        int64\n",
              "OverallCond        int64\n",
              "YearBuilt          int64\n",
              "YearRemodAdd       int64\n",
              "MasVnrArea       float64\n",
              "ExterQual          int64\n",
              "ExterCond          int64\n",
              "BsmtFinSF1         int64\n",
              "BsmtFinSF2         int64\n",
              "BsmtUnfSF          int64\n",
              "TotalBsmtSF        int64\n",
              "1stFlrSF           int64\n",
              "2ndFlrSF           int64\n",
              "LowQualFinSF       int64\n",
              "GrLivArea          int64\n",
              "BsmtFullBath       int64\n",
              "BsmtHalfBath       int64\n",
              "FullBath           int64\n",
              "HalfBath           int64\n",
              "BedroomAbvGr       int64\n",
              "KitchenAbvGr       int64\n",
              "TotRmsAbvGrd       int64\n",
              "Fireplaces         int64\n",
              "GarageYrBlt      float64\n",
              "GarageCars         int64\n",
              "GarageArea         int64\n",
              "WoodDeckSF         int64\n",
              "OpenPorchSF        int64\n",
              "EnclosedPorch      int64\n",
              "3SsnPorch          int64\n",
              "ScreenPorch        int64\n",
              "PoolArea           int64\n",
              "MiscVal            int64\n",
              "MoSold             int64\n",
              "YrSold             int64\n",
              "SalePrice          int64\n",
              "dtype: object"
            ]
          },
          "metadata": {},
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "the info method shows that there are two categorical columns: ExterQual and ExterCond, and both cannot be used without conversion."
      ],
      "metadata": {
        "id": "Ap6YFFdK1QS1"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# using Label Encoder to transform categorical features to numerical.\n",
        "categoricalColumns = ['ExterQual', 'ExterCond']\n",
        "le = LabelEncoder()\n",
        "for col in categoricalColumns:\n",
        "    house[col] = le.fit_transform(house[col])\n",
        "house.head()\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 255
        },
        "id": "fX6bHQrXzCUv",
        "outputId": "ad2206de-32f6-4a0c-bb2e-fe30e0d1a9d8"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   Id  MSSubClass  LotFrontage  LotArea  OverallQual  OverallCond  YearBuilt  \\\n",
              "0   1          60         65.0     8450            7            5       2003   \n",
              "1   2          20         80.0     9600            6            8       1976   \n",
              "2   3          60         68.0    11250            7            5       2001   \n",
              "3   4          70         60.0     9550            7            5       1915   \n",
              "4   5          60         84.0    14260            8            5       2000   \n",
              "\n",
              "   YearRemodAdd  MasVnrArea  ExterQual  ...  WoodDeckSF  OpenPorchSF  \\\n",
              "0          2003       196.0          2  ...           0           61   \n",
              "1          1976         0.0          3  ...         298            0   \n",
              "2          2002       162.0          2  ...           0           42   \n",
              "3          1970         0.0          3  ...           0           35   \n",
              "4          2000       350.0          2  ...         192           84   \n",
              "\n",
              "   EnclosedPorch  3SsnPorch  ScreenPorch  PoolArea  MiscVal  MoSold  YrSold  \\\n",
              "0              0          0            0         0        0       2    2008   \n",
              "1              0          0            0         0        0       5    2007   \n",
              "2              0          0            0         0        0       9    2008   \n",
              "3            272          0            0         0        0       2    2006   \n",
              "4              0          0            0         0        0      12    2008   \n",
              "\n",
              "   SalePrice  \n",
              "0     208500  \n",
              "1     181500  \n",
              "2     223500  \n",
              "3     140000  \n",
              "4     250000  \n",
              "\n",
              "[5 rows x 40 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-e5c812ec-3fbd-4f24-bb80-62f6adc204e8\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Id</th>\n",
              "      <th>MSSubClass</th>\n",
              "      <th>LotFrontage</th>\n",
              "      <th>LotArea</th>\n",
              "      <th>OverallQual</th>\n",
              "      <th>OverallCond</th>\n",
              "      <th>YearBuilt</th>\n",
              "      <th>YearRemodAdd</th>\n",
              "      <th>MasVnrArea</th>\n",
              "      <th>ExterQual</th>\n",
              "      <th>...</th>\n",
              "      <th>WoodDeckSF</th>\n",
              "      <th>OpenPorchSF</th>\n",
              "      <th>EnclosedPorch</th>\n",
              "      <th>3SsnPorch</th>\n",
              "      <th>ScreenPorch</th>\n",
              "      <th>PoolArea</th>\n",
              "      <th>MiscVal</th>\n",
              "      <th>MoSold</th>\n",
              "      <th>YrSold</th>\n",
              "      <th>SalePrice</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>60</td>\n",
              "      <td>65.0</td>\n",
              "      <td>8450</td>\n",
              "      <td>7</td>\n",
              "      <td>5</td>\n",
              "      <td>2003</td>\n",
              "      <td>2003</td>\n",
              "      <td>196.0</td>\n",
              "      <td>2</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>61</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>2008</td>\n",
              "      <td>208500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>20</td>\n",
              "      <td>80.0</td>\n",
              "      <td>9600</td>\n",
              "      <td>6</td>\n",
              "      <td>8</td>\n",
              "      <td>1976</td>\n",
              "      <td>1976</td>\n",
              "      <td>0.0</td>\n",
              "      <td>3</td>\n",
              "      <td>...</td>\n",
              "      <td>298</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>5</td>\n",
              "      <td>2007</td>\n",
              "      <td>181500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>60</td>\n",
              "      <td>68.0</td>\n",
              "      <td>11250</td>\n",
              "      <td>7</td>\n",
              "      <td>5</td>\n",
              "      <td>2001</td>\n",
              "      <td>2002</td>\n",
              "      <td>162.0</td>\n",
              "      <td>2</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>42</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>9</td>\n",
              "      <td>2008</td>\n",
              "      <td>223500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>70</td>\n",
              "      <td>60.0</td>\n",
              "      <td>9550</td>\n",
              "      <td>7</td>\n",
              "      <td>5</td>\n",
              "      <td>1915</td>\n",
              "      <td>1970</td>\n",
              "      <td>0.0</td>\n",
              "      <td>3</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>35</td>\n",
              "      <td>272</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>2006</td>\n",
              "      <td>140000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>60</td>\n",
              "      <td>84.0</td>\n",
              "      <td>14260</td>\n",
              "      <td>8</td>\n",
              "      <td>5</td>\n",
              "      <td>2000</td>\n",
              "      <td>2000</td>\n",
              "      <td>350.0</td>\n",
              "      <td>2</td>\n",
              "      <td>...</td>\n",
              "      <td>192</td>\n",
              "      <td>84</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>12</td>\n",
              "      <td>2008</td>\n",
              "      <td>250000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 40 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-e5c812ec-3fbd-4f24-bb80-62f6adc204e8')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-e5c812ec-3fbd-4f24-bb80-62f6adc204e8 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-e5c812ec-3fbd-4f24-bb80-62f6adc204e8');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-5c5899ea-c1d3-452e-8a8b-f258651c9fbf\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-5c5899ea-c1d3-452e-8a8b-f258651c9fbf')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-5c5899ea-c1d3-452e-8a8b-f258651c9fbf button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "house"
            }
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Correcting null values\n",
        "mean_LotFrontage = house['LotFrontage'].mean()\n",
        "house['LotFrontage'].fillna(mean_LotFrontage, inplace=True)\n",
        "\n",
        "mean_MasVnrArea = house['MasVnrArea'].mean()\n",
        "house['MasVnrArea'].fillna(mean_MasVnrArea, inplace=True)\n",
        "\n",
        "mean_GarageYrBlt = house['GarageYrBlt'].mean()\n",
        "house['GarageYrBlt'].fillna(mean_GarageYrBlt, inplace=True)\n",
        "\n",
        "house.isna().sum()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WUSZlGKQ2dpc",
        "outputId": "9fe98f47-ff4c-4ad1-8e6a-1735b185c615"
      },
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Id               0\n",
              "MSSubClass       0\n",
              "LotFrontage      0\n",
              "LotArea          0\n",
              "OverallQual      0\n",
              "OverallCond      0\n",
              "YearBuilt        0\n",
              "YearRemodAdd     0\n",
              "MasVnrArea       0\n",
              "ExterQual        0\n",
              "ExterCond        0\n",
              "BsmtFinSF1       0\n",
              "BsmtFinSF2       0\n",
              "BsmtUnfSF        0\n",
              "TotalBsmtSF      0\n",
              "1stFlrSF         0\n",
              "2ndFlrSF         0\n",
              "LowQualFinSF     0\n",
              "GrLivArea        0\n",
              "BsmtFullBath     0\n",
              "BsmtHalfBath     0\n",
              "FullBath         0\n",
              "HalfBath         0\n",
              "BedroomAbvGr     0\n",
              "KitchenAbvGr     0\n",
              "TotRmsAbvGrd     0\n",
              "Fireplaces       0\n",
              "GarageYrBlt      0\n",
              "GarageCars       0\n",
              "GarageArea       0\n",
              "WoodDeckSF       0\n",
              "OpenPorchSF      0\n",
              "EnclosedPorch    0\n",
              "3SsnPorch        0\n",
              "ScreenPorch      0\n",
              "PoolArea         0\n",
              "MiscVal          0\n",
              "MoSold           0\n",
              "YrSold           0\n",
              "SalePrice        0\n",
              "dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        " It shows also that the columns LotFrontage, MasVnrArea and GarageYrBlt contain null values."
      ],
      "metadata": {
        "id": "CjOS0OAh63FW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#verifying duplicates\n",
        "print(\"There are {} duplicates\".format(house.duplicated().sum()))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lc8LXWV94T6V",
        "outputId": "7c90188c-36f0-4239-ef0c-dbe5fecaa788"
      },
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "There are 0 duplicates\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "For modeling purposes, we will drop the column 'Id', because it doesn't contain relevant information."
      ],
      "metadata": {
        "id": "mrI9wKAL4uBg"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "house_id = house['Id']\n",
        "house.drop('Id', axis=1, inplace=True)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 255
        },
        "id": "mrR1LsW844P9",
        "outputId": "8aaf00cd-b6f0-4cef-ee48-dce57842f5b9"
      },
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   MSSubClass  LotFrontage  LotArea  OverallQual  OverallCond  YearBuilt  \\\n",
              "0          60         65.0     8450            7            5       2003   \n",
              "1          20         80.0     9600            6            8       1976   \n",
              "2          60         68.0    11250            7            5       2001   \n",
              "3          70         60.0     9550            7            5       1915   \n",
              "4          60         84.0    14260            8            5       2000   \n",
              "\n",
              "   YearRemodAdd  MasVnrArea  ExterQual  ExterCond  ...  WoodDeckSF  \\\n",
              "0          2003       196.0          2          4  ...           0   \n",
              "1          1976         0.0          3          4  ...         298   \n",
              "2          2002       162.0          2          4  ...           0   \n",
              "3          1970         0.0          3          4  ...           0   \n",
              "4          2000       350.0          2          4  ...         192   \n",
              "\n",
              "   OpenPorchSF  EnclosedPorch  3SsnPorch  ScreenPorch  PoolArea  MiscVal  \\\n",
              "0           61              0          0            0         0        0   \n",
              "1            0              0          0            0         0        0   \n",
              "2           42              0          0            0         0        0   \n",
              "3           35            272          0            0         0        0   \n",
              "4           84              0          0            0         0        0   \n",
              "\n",
              "   MoSold  YrSold  SalePrice  \n",
              "0       2    2008     208500  \n",
              "1       5    2007     181500  \n",
              "2       9    2008     223500  \n",
              "3       2    2006     140000  \n",
              "4      12    2008     250000  \n",
              "\n",
              "[5 rows x 39 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-5e8ea3ee-0b0e-415e-9e0e-d78d95b59509\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>MSSubClass</th>\n",
              "      <th>LotFrontage</th>\n",
              "      <th>LotArea</th>\n",
              "      <th>OverallQual</th>\n",
              "      <th>OverallCond</th>\n",
              "      <th>YearBuilt</th>\n",
              "      <th>YearRemodAdd</th>\n",
              "      <th>MasVnrArea</th>\n",
              "      <th>ExterQual</th>\n",
              "      <th>ExterCond</th>\n",
              "      <th>...</th>\n",
              "      <th>WoodDeckSF</th>\n",
              "      <th>OpenPorchSF</th>\n",
              "      <th>EnclosedPorch</th>\n",
              "      <th>3SsnPorch</th>\n",
              "      <th>ScreenPorch</th>\n",
              "      <th>PoolArea</th>\n",
              "      <th>MiscVal</th>\n",
              "      <th>MoSold</th>\n",
              "      <th>YrSold</th>\n",
              "      <th>SalePrice</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>60</td>\n",
              "      <td>65.0</td>\n",
              "      <td>8450</td>\n",
              "      <td>7</td>\n",
              "      <td>5</td>\n",
              "      <td>2003</td>\n",
              "      <td>2003</td>\n",
              "      <td>196.0</td>\n",
              "      <td>2</td>\n",
              "      <td>4</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>61</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>2008</td>\n",
              "      <td>208500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>20</td>\n",
              "      <td>80.0</td>\n",
              "      <td>9600</td>\n",
              "      <td>6</td>\n",
              "      <td>8</td>\n",
              "      <td>1976</td>\n",
              "      <td>1976</td>\n",
              "      <td>0.0</td>\n",
              "      <td>3</td>\n",
              "      <td>4</td>\n",
              "      <td>...</td>\n",
              "      <td>298</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>5</td>\n",
              "      <td>2007</td>\n",
              "      <td>181500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>60</td>\n",
              "      <td>68.0</td>\n",
              "      <td>11250</td>\n",
              "      <td>7</td>\n",
              "      <td>5</td>\n",
              "      <td>2001</td>\n",
              "      <td>2002</td>\n",
              "      <td>162.0</td>\n",
              "      <td>2</td>\n",
              "      <td>4</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>42</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>9</td>\n",
              "      <td>2008</td>\n",
              "      <td>223500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>70</td>\n",
              "      <td>60.0</td>\n",
              "      <td>9550</td>\n",
              "      <td>7</td>\n",
              "      <td>5</td>\n",
              "      <td>1915</td>\n",
              "      <td>1970</td>\n",
              "      <td>0.0</td>\n",
              "      <td>3</td>\n",
              "      <td>4</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>35</td>\n",
              "      <td>272</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>2006</td>\n",
              "      <td>140000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>60</td>\n",
              "      <td>84.0</td>\n",
              "      <td>14260</td>\n",
              "      <td>8</td>\n",
              "      <td>5</td>\n",
              "      <td>2000</td>\n",
              "      <td>2000</td>\n",
              "      <td>350.0</td>\n",
              "      <td>2</td>\n",
              "      <td>4</td>\n",
              "      <td>...</td>\n",
              "      <td>192</td>\n",
              "      <td>84</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>12</td>\n",
              "      <td>2008</td>\n",
              "      <td>250000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 39 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-5e8ea3ee-0b0e-415e-9e0e-d78d95b59509')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-5e8ea3ee-0b0e-415e-9e0e-d78d95b59509 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-5e8ea3ee-0b0e-415e-9e0e-d78d95b59509');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-c3e26f3e-68c0-4e79-8bca-35f3ea169238\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-c3e26f3e-68c0-4e79-8bca-35f3ea169238')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-c3e26f3e-68c0-4e79-8bca-35f3ea169238 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "house"
            }
          },
          "metadata": {},
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Now we will split our data into test and training\n",
        "x = house.iloc[:, :-1]\n",
        "y = house.iloc[:, -1]\n",
        "x_list = list(x.columns)\n",
        "x = np.array(x)\n",
        "y = np.array(y)\n",
        "\n",
        "x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=42)"
      ],
      "metadata": {
        "id": "zE3N0PLE7fTD"
      },
      "execution_count": 31,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Random Forest Regression\n",
        "rfr = RandomForestRegressor(n_estimators=100, random_state=42)\n",
        "rfr.fit(x_train, y_train)\n",
        "y_pred_rfr = rfr.predict(x_test)\n",
        "r_sq_score = rfr.score(x_train, y_train)\n",
        "print(f\"R²: {r_sq_score}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nYCIbhbs8V94",
        "outputId": "4e9a5462-ef66-451c-de8e-e12c8088d158"
      },
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "R²: 0.9785355978814705\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Metrics for the Random Forest Regression\n",
        "print('MAE: {}'.format(metrics.mean_absolute_error(y_test, y_pred_rfr)))\n",
        "print('MSE: {}'.format(metrics.mean_squared_error(y_test, y_pred_rfr)))\n",
        "print('RMSE: {}'.format(np.sqrt(metrics.mean_squared_error(y_test, y_pred_rfr))))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hT307XS781vk",
        "outputId": "ee6b4ff0-8b9d-4f5d-e407-201ca1c78b2a"
      },
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "MAE: 17966.99546803653\n",
            "MSE: 839136437.8418005\n",
            "RMSE: 28967.85179887871\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# AdaBoost\n",
        "adb = AdaBoostRegressor(n_estimators=100, random_state=42)\n",
        "adb.fit(x_train, y_train)\n",
        "y_pred_adb = adb.predict(x_test)\n",
        "adb_sq_score = adb.score(x_train, y_train)\n",
        "print(f\"R²: {adb_sq_score}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ynJLuiW5BaU8",
        "outputId": "719249fb-3500-4b59-c704-dbb6c3656644"
      },
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "R²: 0.8719496242910916\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Metrics for AdaBoost Regression\n",
        "print(\"MAE: {}\".format(metrics.mean_absolute_error(y_test, y_pred_adb)))\n",
        "print(\"MSE: {}\".format(metrics.mean_squared_error(y_test, y_pred_adb)))\n",
        "print(\"RMSE: {}\".format(np.sqrt(metrics.mean_squared_error(y_test, y_pred_adb))))\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FMLsfkdfBv8d",
        "outputId": "85c7fd96-a0c3-42a7-8c89-c8c52f76dda5"
      },
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "MAE: 24665.647787939375\n",
            "MSE: 1249478513.4101741\n",
            "RMSE: 35347.96335590177\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Gradient Boosting Regression\n",
        "gbr = GradientBoostingRegressor(n_estimators = 100, random_state = 42)\n",
        "gbr.fit(x_train, y_train)\n",
        "y_pred_gbr = gbr.predict(x_test)\n",
        "gbr_sq_score = gbr.score(x_train, y_train)\n",
        "print(f\"R²: {gbr_sq_score}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bMQ5YusaCYfl",
        "outputId": "e5328327-18f9-456e-b940-5771ffc6813d"
      },
      "execution_count": 36,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "R²: 0.9641021607500599\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Metrics for Gradient Boosting Regression\n",
        "print(\"MAE: {}\".format(metrics.mean_absolute_error(y_test, y_pred_gbr)))\n",
        "print(\"MSE: {}\".format(metrics.mean_squared_error(y_test, y_pred_gbr)))\n",
        "print(\"RMSE: {}\".format(np.sqrt(metrics.mean_squared_error(y_test, y_pred_gbr))))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CYjbsUasCrwG",
        "outputId": "917a4343-a941-458c-fa6d-c892dd4e7bc5"
      },
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "MAE: 17547.37690312577\n",
            "MSE: 822246541.5578891\n",
            "RMSE: 28674.8416134752\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Among all regressors, Gradient Boosting delivered the best results."
      ],
      "metadata": {
        "id": "KrFFiahnC5SK"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Tree Visualization**"
      ],
      "metadata": {
        "id": "-X4cu-DKFbm_"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **Random Forest**"
      ],
      "metadata": {
        "id": "x8M3MSEIFijF"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Import tools needed for visualization\n",
        "from sklearn.tree import export_graphviz\n",
        "import pydot\n",
        "# Pull out one tree from the forest\n",
        "tree = rfr.estimators_[5]\n",
        "# Export the image to a dot file\n",
        "export_graphviz(tree, out_file = 'tree_rfr.dot', feature_names = x_list, rounded = True, precision = 1)\n",
        "# Use dot file to create a graph\n",
        "(graph, ) = pydot.graph_from_dot_file('tree_rfr.dot')\n",
        "# Write graph to a png file\n",
        "graph.write_png('tree_rfr.png')"
      ],
      "metadata": {
        "id": "qsMIc7hIFhp-"
      },
      "execution_count": 42,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **AdaBoost**"
      ],
      "metadata": {
        "id": "-LxtclUmFntU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Pull out one tree from the forest\n",
        "tree = adb.estimators_[5]\n",
        "# Export the image to a dot file\n",
        "export_graphviz(tree, out_file = 'tree_adb.dot', feature_names = x_list, rounded = True, precision = 1)\n",
        "# Use dot file to create a graph\n",
        "(graph, ) = pydot.graph_from_dot_file('tree_adb.dot')\n",
        "# Write graph to a png file\n",
        "graph.write_png('tree_adb.png')"
      ],
      "metadata": {
        "id": "ab1PD4gsFqM8"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **Gradient Boosting**"
      ],
      "metadata": {
        "id": "zBc5vtq2FqmM"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Pull out one tree from the forest\n",
        "tree = gbr.estimators_[5]\n",
        "# Export the image to a dot file\n",
        "export_graphviz(tree, out_file = 'tree_gbr.dot', feature_names = x_list, rounded = True, precision = 1)\n",
        "# Use dot file to create a graph\n",
        "(graph, ) = pydot.graph_from_dot_file('tree_gbr.dot')\n",
        "# Write graph to a png file\n",
        "graph.write_png('tree_gbr.png')"
      ],
      "metadata": {
        "id": "DxUPi2FPGIVd"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Feature Importance**"
      ],
      "metadata": {
        "id": "J0RMZz4sEDl0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Get numerical feature importances\n",
        "importances = list(rfr.feature_importances_)\n",
        "# List of tuples with variable and importance\n",
        "feature_importances = [(feature, round(importance, 2)) for feature, importance in zip(x_list, importances)]\n",
        "# Sort the feature importances by most important first\n",
        "feature_importances = sorted(feature_importances, key = lambda x: x[1], reverse = True)\n",
        "# Print out the feature and importances\n",
        "[print('Variable: {:20} Importance: {}'.format(*pair)) for pair in feature_importances];"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_OGU7r2LEG8e",
        "outputId": "2297baab-b843-4119-8386-aefbc138e137"
      },
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Variable: OverallQual          Importance: 0.56\n",
            "Variable: GrLivArea            Importance: 0.12\n",
            "Variable: TotalBsmtSF          Importance: 0.04\n",
            "Variable: 2ndFlrSF             Importance: 0.04\n",
            "Variable: BsmtFinSF1           Importance: 0.03\n",
            "Variable: 1stFlrSF             Importance: 0.03\n",
            "Variable: LotArea              Importance: 0.02\n",
            "Variable: YearBuilt            Importance: 0.02\n",
            "Variable: GarageArea           Importance: 0.02\n",
            "Variable: LotFrontage          Importance: 0.01\n",
            "Variable: OverallCond          Importance: 0.01\n",
            "Variable: YearRemodAdd         Importance: 0.01\n",
            "Variable: BsmtUnfSF            Importance: 0.01\n",
            "Variable: FullBath             Importance: 0.01\n",
            "Variable: TotRmsAbvGrd         Importance: 0.01\n",
            "Variable: GarageYrBlt          Importance: 0.01\n",
            "Variable: GarageCars           Importance: 0.01\n",
            "Variable: WoodDeckSF           Importance: 0.01\n",
            "Variable: OpenPorchSF          Importance: 0.01\n",
            "Variable: MSSubClass           Importance: 0.0\n",
            "Variable: MasVnrArea           Importance: 0.0\n",
            "Variable: ExterQual            Importance: 0.0\n",
            "Variable: ExterCond            Importance: 0.0\n",
            "Variable: BsmtFinSF2           Importance: 0.0\n",
            "Variable: LowQualFinSF         Importance: 0.0\n",
            "Variable: BsmtFullBath         Importance: 0.0\n",
            "Variable: BsmtHalfBath         Importance: 0.0\n",
            "Variable: HalfBath             Importance: 0.0\n",
            "Variable: BedroomAbvGr         Importance: 0.0\n",
            "Variable: KitchenAbvGr         Importance: 0.0\n",
            "Variable: Fireplaces           Importance: 0.0\n",
            "Variable: EnclosedPorch        Importance: 0.0\n",
            "Variable: 3SsnPorch            Importance: 0.0\n",
            "Variable: ScreenPorch          Importance: 0.0\n",
            "Variable: PoolArea             Importance: 0.0\n",
            "Variable: MiscVal              Importance: 0.0\n",
            "Variable: MoSold               Importance: 0.0\n",
            "Variable: YrSold               Importance: 0.0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Get numerical feature importances\n",
        "importances = list(adb.feature_importances_)\n",
        "# List of tuples with variable and importance\n",
        "feature_importances = [(feature, round(importance, 2)) for feature, importance in zip(x_list, importances)]\n",
        "# Sort the feature importances by most important first\n",
        "feature_importances = sorted(feature_importances, key = lambda x: x[1], reverse = True)\n",
        "# Print out the feature and importances\n",
        "[print('Variable: {:20} Importance: {}'.format(*pair)) for pair in feature_importances];"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "N2HBYySOFRBk",
        "outputId": "1c5322e8-5769-48fc-86f5-425826085bec"
      },
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Variable: OverallQual          Importance: 0.22\n",
            "Variable: GrLivArea            Importance: 0.17\n",
            "Variable: 2ndFlrSF             Importance: 0.14\n",
            "Variable: GarageCars           Importance: 0.08\n",
            "Variable: OpenPorchSF          Importance: 0.04\n",
            "Variable: LotFrontage          Importance: 0.03\n",
            "Variable: LotArea              Importance: 0.03\n",
            "Variable: BsmtFinSF1           Importance: 0.03\n",
            "Variable: TotalBsmtSF          Importance: 0.03\n",
            "Variable: 1stFlrSF             Importance: 0.03\n",
            "Variable: GarageYrBlt          Importance: 0.03\n",
            "Variable: ScreenPorch          Importance: 0.03\n",
            "Variable: YearBuilt            Importance: 0.02\n",
            "Variable: MoSold               Importance: 0.02\n",
            "Variable: YearRemodAdd         Importance: 0.01\n",
            "Variable: BsmtFullBath         Importance: 0.01\n",
            "Variable: FullBath             Importance: 0.01\n",
            "Variable: BedroomAbvGr         Importance: 0.01\n",
            "Variable: Fireplaces           Importance: 0.01\n",
            "Variable: WoodDeckSF           Importance: 0.01\n",
            "Variable: PoolArea             Importance: 0.01\n",
            "Variable: MSSubClass           Importance: 0.0\n",
            "Variable: OverallCond          Importance: 0.0\n",
            "Variable: MasVnrArea           Importance: 0.0\n",
            "Variable: ExterQual            Importance: 0.0\n",
            "Variable: ExterCond            Importance: 0.0\n",
            "Variable: BsmtFinSF2           Importance: 0.0\n",
            "Variable: BsmtUnfSF            Importance: 0.0\n",
            "Variable: LowQualFinSF         Importance: 0.0\n",
            "Variable: BsmtHalfBath         Importance: 0.0\n",
            "Variable: HalfBath             Importance: 0.0\n",
            "Variable: KitchenAbvGr         Importance: 0.0\n",
            "Variable: TotRmsAbvGrd         Importance: 0.0\n",
            "Variable: GarageArea           Importance: 0.0\n",
            "Variable: EnclosedPorch        Importance: 0.0\n",
            "Variable: 3SsnPorch            Importance: 0.0\n",
            "Variable: MiscVal              Importance: 0.0\n",
            "Variable: YrSold               Importance: 0.0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Get numerical feature importances\n",
        "importances = list(gbr.feature_importances_)\n",
        "# List of tuples with variable and importance\n",
        "feature_importances = [(feature, round(importance, 2)) for feature, importance in zip(x_list, importances)]\n",
        "# Sort the feature importances by most important first\n",
        "feature_importances = sorted(feature_importances, key = lambda x: x[1], reverse = True)\n",
        "# Print out the feature and importances\n",
        "[print('Variable: {:20} Importance: {}'.format(*pair)) for pair in feature_importances];"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "uLWAYxCPFW40",
        "outputId": "984cfa0e-ca14-4e15-f4f0-3d8106f9cc9b"
      },
      "execution_count": 41,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Variable: OverallQual          Importance: 0.52\n",
            "Variable: GrLivArea            Importance: 0.14\n",
            "Variable: GarageCars           Importance: 0.05\n",
            "Variable: BsmtFinSF1           Importance: 0.04\n",
            "Variable: TotalBsmtSF          Importance: 0.04\n",
            "Variable: 1stFlrSF             Importance: 0.03\n",
            "Variable: 2ndFlrSF             Importance: 0.03\n",
            "Variable: LotArea              Importance: 0.02\n",
            "Variable: YearBuilt            Importance: 0.02\n",
            "Variable: YearRemodAdd         Importance: 0.02\n",
            "Variable: OverallCond          Importance: 0.01\n",
            "Variable: ExterQual            Importance: 0.01\n",
            "Variable: FullBath             Importance: 0.01\n",
            "Variable: Fireplaces           Importance: 0.01\n",
            "Variable: GarageYrBlt          Importance: 0.01\n",
            "Variable: MSSubClass           Importance: 0.0\n",
            "Variable: LotFrontage          Importance: 0.0\n",
            "Variable: MasVnrArea           Importance: 0.0\n",
            "Variable: ExterCond            Importance: 0.0\n",
            "Variable: BsmtFinSF2           Importance: 0.0\n",
            "Variable: BsmtUnfSF            Importance: 0.0\n",
            "Variable: LowQualFinSF         Importance: 0.0\n",
            "Variable: BsmtFullBath         Importance: 0.0\n",
            "Variable: BsmtHalfBath         Importance: 0.0\n",
            "Variable: HalfBath             Importance: 0.0\n",
            "Variable: BedroomAbvGr         Importance: 0.0\n",
            "Variable: KitchenAbvGr         Importance: 0.0\n",
            "Variable: TotRmsAbvGrd         Importance: 0.0\n",
            "Variable: GarageArea           Importance: 0.0\n",
            "Variable: WoodDeckSF           Importance: 0.0\n",
            "Variable: OpenPorchSF          Importance: 0.0\n",
            "Variable: EnclosedPorch        Importance: 0.0\n",
            "Variable: 3SsnPorch            Importance: 0.0\n",
            "Variable: ScreenPorch          Importance: 0.0\n",
            "Variable: PoolArea             Importance: 0.0\n",
            "Variable: MiscVal              Importance: 0.0\n",
            "Variable: MoSold               Importance: 0.0\n",
            "Variable: YrSold               Importance: 0.0\n"
          ]
        }
      ]
    }
  ]
}