Daria_Petrenko_task_6.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "source": [
    "# Машинное обучение, ФКН ВШЭ\n",
    "\n",
    "## Практическое задание 6\n",
    "\n",
    "### Общая информация\n",
    "Дата выдачи: 4 июня 2018\n",
    "\n",
    "Жёсткий дедлайн: 23:59MSK 14 июня"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### О задании\n",
    "\n",
    "Мы будем решать задачу предсказания опасных событий для страховой компании: [Liberty Mutual Group: Property Inspection Prediction](https://www.kaggle.com/c/liberty-mutual-group-property-inspection-prediction). Обучающая выборка состоит из засекреченных признаков целого и строкового типов. Целевой переменной являются счётчики $y \\in \\mathbb{Z}_+$.\n",
    "\n",
    "Работа состоит из следующих пунктов:\n",
    "* Предобработать данные [1 балл]\n",
    "* Написать свой алгоритм прогнозирования событий [2 балла]\n",
    "* Настроить линейные методы из библиотеки StatsModels для решения задачи [1 балл]\n",
    "* Настроить бустинг из библиотеки lightgbm для решения задачи [1 балл]\n",
    "\n",
    "Задания является дополнительным, то есть само по себе не учитывается в накопленной оценке. Все полученные за задание баллы являются бонусными, то есть их можно прибавить к оценке за любое теоретическое или практическое домашнее задание из курса.\n",
    "\n",
    "\n",
    "### Оценивание и штрафы\n",
    "Каждая из задач имеет определенную «стоимость» (указана в скобках около задачи). Максимально допустимая оценка за работу — 5 баллов.\n",
    "\n",
    "Сдавать задание после указанного срока сдачи нельзя. При выставлении неполного балла за задание в связи с наличием ошибок на усмотрение проверяющего предусмотрена возможность исправить работу на указанных в ответном письме условиях.\n",
    "\n",
    "Задание выполняется самостоятельно. «Похожие» решения считаются плагиатом и все задействованные студенты (в том числе те, у кого списали) не могут получить за него больше 0 баллов (подробнее о плагиате см. на странице курса). Если вы нашли решение какого-то из заданий (или его часть) в открытом источнике, необходимо указать ссылку на этот источник в отдельном блоке в конце вашей работы (скорее всего вы будете не единственным, кто это нашел, поэтому чтобы исключить подозрение в плагиате, необходима ссылка на источник).\n",
    "\n",
    "Неэффективная реализация кода может негативно отразиться на оценке.\n",
    "\n",
    "\n",
    "### Формат сдачи\n",
    "Для сдачи задания переименуйте получившийся файл *.ipynb в соответствии со следующим форматом: homework-practice-06-Username.ipynb, где Username — ваша фамилия и имя на латинице именно в таком порядке (например, homework-practice-06-IvanovIvan.ipynb). \n",
    "\n",
    "Далее отправьте этот файл в AnyTask.\n",
    "\n",
    "Для удобства проверки самостоятельно посчитайте свою максимальную оценку (исходя из набора решенных задач) и укажите ниже."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** Оценка:** 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import statsmodels.api as sm\n",
    "\n",
    "from scipy.optimize import minimize\n",
    "from lightgbm import LGBMModel\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.preprocessing import OneHotEncoder, LabelEncoder\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.metrics import make_scorer\n",
    "\n",
    "pd.set_option('max_rows', 10)\n",
    "pd.set_option('max_columns', None)\n",
    "plt.style.use('bmh')\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "source": [
    "__Задание 1 (1 балл).__ Загрузка и предобработка данных."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Hazard</th>\n",
       "      <th>T1_V1</th>\n",
       "      <th>T1_V2</th>\n",
       "      <th>T1_V3</th>\n",
       "      <th>T1_V4</th>\n",
       "      <th>T1_V5</th>\n",
       "      <th>T1_V6</th>\n",
       "      <th>T1_V7</th>\n",
       "      <th>T1_V8</th>\n",
       "      <th>T1_V9</th>\n",
       "      <th>T1_V10</th>\n",
       "      <th>T1_V11</th>\n",
       "      <th>T1_V12</th>\n",
       "      <th>T1_V13</th>\n",
       "      <th>T1_V14</th>\n",
       "      <th>T1_V15</th>\n",
       "      <th>T1_V16</th>\n",
       "      <th>T1_V17</th>\n",
       "      <th>T2_V1</th>\n",
       "      <th>T2_V2</th>\n",
       "      <th>T2_V3</th>\n",
       "      <th>T2_V4</th>\n",
       "      <th>T2_V5</th>\n",
       "      <th>T2_V6</th>\n",
       "      <th>T2_V7</th>\n",
       "      <th>T2_V8</th>\n",
       "      <th>T2_V9</th>\n",
       "      <th>T2_V10</th>\n",
       "      <th>T2_V11</th>\n",
       "      <th>T2_V12</th>\n",
       "      <th>T2_V13</th>\n",
       "      <th>T2_V14</th>\n",
       "      <th>T2_V15</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>15</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>N</td>\n",
       "      <td>B</td>\n",
       "      <td>N</td>\n",
       "      <td>B</td>\n",
       "      <td>B</td>\n",
       "      <td>D</td>\n",
       "      <td>7</td>\n",
       "      <td>B</td>\n",
       "      <td>B</td>\n",
       "      <td>15</td>\n",
       "      <td>1</td>\n",
       "      <td>A</td>\n",
       "      <td>B</td>\n",
       "      <td>N</td>\n",
       "      <td>36</td>\n",
       "      <td>11</td>\n",
       "      <td>N</td>\n",
       "      <td>10</td>\n",
       "      <td>B</td>\n",
       "      <td>2</td>\n",
       "      <td>37</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>6</td>\n",
       "      <td>Y</td>\n",
       "      <td>N</td>\n",
       "      <td>E</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4</td>\n",
       "      <td>16</td>\n",
       "      <td>14</td>\n",
       "      <td>5</td>\n",
       "      <td>H</td>\n",
       "      <td>B</td>\n",
       "      <td>N</td>\n",
       "      <td>B</td>\n",
       "      <td>B</td>\n",
       "      <td>C</td>\n",
       "      <td>12</td>\n",
       "      <td>B</td>\n",
       "      <td>B</td>\n",
       "      <td>10</td>\n",
       "      <td>3</td>\n",
       "      <td>A</td>\n",
       "      <td>B</td>\n",
       "      <td>Y</td>\n",
       "      <td>78</td>\n",
       "      <td>10</td>\n",
       "      <td>Y</td>\n",
       "      <td>17</td>\n",
       "      <td>C</td>\n",
       "      <td>2</td>\n",
       "      <td>22</td>\n",
       "      <td>1</td>\n",
       "      <td>18</td>\n",
       "      <td>5</td>\n",
       "      <td>Y</td>\n",
       "      <td>Y</td>\n",
       "      <td>E</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>10</td>\n",
       "      <td>10</td>\n",
       "      <td>5</td>\n",
       "      <td>N</td>\n",
       "      <td>K</td>\n",
       "      <td>N</td>\n",
       "      <td>B</td>\n",
       "      <td>B</td>\n",
       "      <td>E</td>\n",
       "      <td>12</td>\n",
       "      <td>H</td>\n",
       "      <td>B</td>\n",
       "      <td>15</td>\n",
       "      <td>1</td>\n",
       "      <td>A</td>\n",
       "      <td>R</td>\n",
       "      <td>Y</td>\n",
       "      <td>71</td>\n",
       "      <td>21</td>\n",
       "      <td>Y</td>\n",
       "      <td>13</td>\n",
       "      <td>C</td>\n",
       "      <td>6</td>\n",
       "      <td>37</td>\n",
       "      <td>2</td>\n",
       "      <td>14</td>\n",
       "      <td>6</td>\n",
       "      <td>Y</td>\n",
       "      <td>Y</td>\n",
       "      <td>E</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>18</td>\n",
       "      <td>18</td>\n",
       "      <td>5</td>\n",
       "      <td>N</td>\n",
       "      <td>K</td>\n",
       "      <td>N</td>\n",
       "      <td>B</td>\n",
       "      <td>B</td>\n",
       "      <td>E</td>\n",
       "      <td>3</td>\n",
       "      <td>H</td>\n",
       "      <td>B</td>\n",
       "      <td>15</td>\n",
       "      <td>1</td>\n",
       "      <td>A</td>\n",
       "      <td>R</td>\n",
       "      <td>N</td>\n",
       "      <td>71</td>\n",
       "      <td>13</td>\n",
       "      <td>N</td>\n",
       "      <td>15</td>\n",
       "      <td>A</td>\n",
       "      <td>2</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>Y</td>\n",
       "      <td>N</td>\n",
       "      <td>C</td>\n",
       "      <td>2</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1</td>\n",
       "      <td>13</td>\n",
       "      <td>19</td>\n",
       "      <td>5</td>\n",
       "      <td>N</td>\n",
       "      <td>H</td>\n",
       "      <td>N</td>\n",
       "      <td>B</td>\n",
       "      <td>B</td>\n",
       "      <td>E</td>\n",
       "      <td>7</td>\n",
       "      <td>H</td>\n",
       "      <td>B</td>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>A</td>\n",
       "      <td>J</td>\n",
       "      <td>N</td>\n",
       "      <td>75</td>\n",
       "      <td>10</td>\n",
       "      <td>Y</td>\n",
       "      <td>11</td>\n",
       "      <td>B</td>\n",
       "      <td>1</td>\n",
       "      <td>22</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>7</td>\n",
       "      <td>N</td>\n",
       "      <td>N</td>\n",
       "      <td>E</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Hazard  T1_V1  T1_V2  T1_V3 T1_V4 T1_V5 T1_V6 T1_V7 T1_V8 T1_V9  T1_V10  \\\n",
       "Id                                                                            \n",
       "1        1     15      3      2     N     B     N     B     B     D       7   \n",
       "2        4     16     14      5     H     B     N     B     B     C      12   \n",
       "3        1     10     10      5     N     K     N     B     B     E      12   \n",
       "4        1     18     18      5     N     K     N     B     B     E       3   \n",
       "5        1     13     19      5     N     H     N     B     B     E       7   \n",
       "\n",
       "   T1_V11 T1_V12  T1_V13  T1_V14 T1_V15 T1_V16 T1_V17  T2_V1  T2_V2 T2_V3  \\\n",
       "Id                                                                          \n",
       "1       B      B      15       1      A      B      N     36     11     N   \n",
       "2       B      B      10       3      A      B      Y     78     10     Y   \n",
       "3       H      B      15       1      A      R      Y     71     21     Y   \n",
       "4       H      B      15       1      A      R      N     71     13     N   \n",
       "5       H      B      10       1      A      J      N     75     10     Y   \n",
       "\n",
       "    T2_V4 T2_V5  T2_V6  T2_V7  T2_V8  T2_V9  T2_V10 T2_V11 T2_V12 T2_V13  \\\n",
       "Id                                                                         \n",
       "1      10     B      2     37      1     11       6      Y      N      E   \n",
       "2      17     C      2     22      1     18       5      Y      Y      E   \n",
       "3      13     C      6     37      2     14       6      Y      Y      E   \n",
       "4      15     A      2     25      1      1       6      Y      N      C   \n",
       "5      11     B      1     22      1      2       7      N      N      E   \n",
       "\n",
       "    T2_V14  T2_V15  \n",
       "Id                  \n",
       "1        2       2  \n",
       "2        2       1  \n",
       "3        6       1  \n",
       "4        2       6  \n",
       "5        1       1  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.read_csv('data.csv', index_col='Id')\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(50999, 33)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "source": [
    "Выделим категориальные и числовые признаки:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "categorical, numerical = list(), list()\n",
    "for col in data.columns[1:]:\n",
    "    if isinstance(data.loc[1, col], str):\n",
    "        categorical.append(col)\n",
    "    if isinstance(data.loc[1, col], np.int64):\n",
    "        numerical.append(col)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 8, 10,  2,  4,  4,  6, 12,  4,  8, 18,  2,  2,  6,  2,  2,  5])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[categorical].nunique().values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 19,  24,   9,   5,   4,   5, 100,  39,  22,   7,   7,   3,  25,\n",
       "         7,   7,  12])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[numerical].nunique().values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "source": [
    "Нарисуем априорное распределение ответов $p(y)$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmcAAAD2CAYAAACTFyDeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3X10XHd95/HP1yONNdGT9VQpTYhJIOCYwiFNaprSJRSniaFt0vWBJdlCw66zuz11tnDYbjdpy0PTZnnqtvR0s5RzTAp0F9I0UOrDhk3YQrrtqUMT2wGa0IAdIqxiSaOn0dPII42/+8dcK+Mb2f5MrPu7upPP6xydaObOjL73PRL8PA93zN0hIiIiIhvDprQHEBEREZHnaHEmIiIisoFocSYiIiKygWhxJiIiIrKBaHEmIiIisoFocSYiIiKygQRbnJnZLjN72syOmNkda2z/FTP7tpk9YWZ/Z2bb67bdGV3vaTO7IdTMIiIiIqFZiOOcmVkOwHcB/CyAEQCPAbjF3Z+qu0yXu89G398I4FfdfVe0SPs8gB0AfhTA/wXwCnev1v+MRx55xDdv3pz4voiIiIicr8XFxYmdO3cOrLWtJdAMOwAccfdnAMDM7gNwE4DVxdmphVmkHcCpVeNNAO5z9xMAvm9mR6LbO1D/AzZv3oxt27ad96DDw8PYunXred9Os1Mnnlrx1IqjTjy14qgTb71aHTp0aPhM20Itzi4CcKzu9AiA18UvZGZ7AbwXQB7Am+qu+2jsuhclMybQ2tqa1E03FXXiqRVPrTjqxFMrjjrxQrQKtTizNc573vOp7n4PgHvM7F8D+G0At7LXHR8fx549e9DS0oJqtYrdu3dj7969GB0dRXt7O3K5HGZnZzEwMICpqSm4OwYGBjA2NoaOjg4AwPz8PDo7O3Hs2DGYGXp7e1EsFtHV1YVqtYqFhQUMDQ1hdHQUra2t6O7uxsTEBLq7u1GpVFAul1e35/N5dHZ2YnJyEj09PSiXy1haWlrd3tbWhkKhgOnpafT19WFubg6VSmV1e6FQQD6fR6lUQn9/P0qlEpaXl1e3N7JPg4ODKBaL67pPbW1tGBsba6p9Sup+MjMMDw831T4ldT8tLi5iZmamqfYpiftpcXER4+PjTbVPSd1Py8vLWFxcbKp9SuJ+WlxcxIkTJ5pqn5K6nzo6OjAyMnLe+3TWRVOg15xdA+CD7n5DdPpOAHD3D53h8psATLt7d/yyZvZQdFunPa154MAB19Oa4agTT614asVRJ55acdSJt45Pax7cuXPn1WttC/VuzccAXG5ml5pZHsDNAPbXX8DMLq87+XMAvhd9vx/AzWa22cwuBXA5gH9IatDu7u6kbrqpqBNPrXhqxVEnnlpx1IkXolWQpzXdfcXMbgfwEIAcgHvd/UkzuwvA4+6+H8DtZnYdgGUA06g9pYnocvej9uaBFQB74+/UXE+VSiWpm24q6sRTK55acdSJp1YcdeKFaBXqNWdw9wcBPBg77/1137/7LNe9G8DdyU33nHK5HOLHZJ468dSKp1YcdeKpFUedeCFa6RMCYoaGhtIeIRPUiadWPLXiqBNPrTjqxAvRSouzmHO9g0Jq1ImnVjy14qgTT6046sQL0SrY05pZkc/nz7jt+n2HG769h2+78nzG2bDO1klOp1Y8teKoE0+tOOrEC9FKj5zFdHZ2pj1CJqgTT614asVRJ55acdSJF6KVFmcxk5OTaY+QCerEUyueWnHUiadWHHXihWilxVlMT09P2iNkgjrx1IqnVhx14qkVR514IVppcRajtxNz1ImnVjy14qgTT6046sTToTRSsLS0lPYImaBOPLXiqRVHnXhqxVEnXohWWpzF6FgvHHXiqRVPrTjqxFMrjjrxdJyzFOhYLxx14qkVT6046sRTK4468UK00uIspq2tLe0RMkGdeGrFUyuOOvHUiqNOvBCttDiLKRQKaY+QCerEUyueWnHUiadWHHXihWilxVnM9PR02iNkgjrx1IqnVhx14qkVR514IVppcRbT19eX9giZoE48teKpFUedeGrFUSdeiFZanMXMzc2lPUImqBNPrXhqxVEnnlpx1IkXopUWZzGVSiXtETJBnXhqxVMrjjrx1IqjTrwQrbQ4i9GxXjjqxFMrnlpx1ImnVhx14uk4ZynQsV446sRTK55acdSJp1YcdeLpOGcp0NuJOerEUyueWnHUiadWHHXi6VAaKcjn82mPkAnqxFMrnlpx1ImnVhx14oVopcVZTKlUSnuETFAnnlrx1IqjTjy14qgTL0QrLc5i+vv70x4hE9SJp1Y8teKoE0+tOOrEC9FKi7MY/euBo048teKpFUedeGrFUSeeHjlLwfLyctojZII68dSKp1YcdeKpFUedeCFaaXEWo2O9cNSJp1Y8teKoE0+tOOrE03HOUqBjvXDUiadWPLXiqBNPrTjqxNNxzlLQ3t6e9giZoE48teKpFUedeGrFUSdeiFbBFmdmtsvMnjazI2Z2xxrb32tmT5nZt8zsr81sa922qpk9EX3tT3LOXC6X5M03DXXiqRVPrTjqxFMrjjrxQrQKsjgzsxyAewC8GcB2ALeY2fbYxQ4DuNrdXwPgAQAfrdtWdvfXRl83Jjnr7OxskjffNNSJp1Y8teKoE0+tOOrEC9Eq1CNnOwAccfdn3L0C4D4AN9VfwN2/7u6L0clHAVwcaLbTDAwMpPFjM0edeGrFUyuOOvHUiqNOvBCtQi3OLgJwrO70SHTemewB8JW6021m9riZPWpmv5jEgKdMTU0lefNNQ514asVTK4468dSKo068EK1aEv8JNbbGeb7mBc3eAeBqANfWnX2Ju//QzC4D8DUz+7a7H62/3vj4OPbs2YOWlhZUq1Xs3r0be/fuxejoKNrb25HL5TA7O4uBgQFMTU3B3TEwMICxsTF0dHQAAObn53Hy5EkcO3YMZobe3l4Ui0V0dXWhWq3i2v4KDs604KotK1isGoYXc7iicwXPLubQ2eLoy59c3T63Yjhe3oTh4WH09PSgXC5jaWkJQ0NDGB0dRVtbGwqFAqanp9HX14e5uTlUKpXV7YVCAfl8HqVSCf39/SiVSlheXl7d3sg+DQ4OolgsrrlPCwsLq7fZ2tqK7u5uTExMoLu7G5VKBeVyeXV7Pp9HZ2cnJicnUalUMDY21lT7lNT9tLi4iOHh4abap6Tup+npaXR0dDTVPiVxP01PTyOfzzfVPiV1P83OzjbdPiVxP01PT2NgYKCp9imp+6larWJkZOS89+msiyb3NddI68rMrgHwQXe/ITp9JwC4+4dil7sOwB8DuNbdx89wW58G8GV3f6D+/AMHDvi2bdvOe9alpSW0tbWtue36fYcbvr2Hb7vyfEfakM7WSU6nVjy14qgTT6046sRbr1aHDh06uHPnzqvX2hbqac3HAFxuZpeaWR7AzQBOe9elmV0J4JMAbqxfmJlZj5ltjr7vB/B6AE8lNejY2FhSN91U1ImnVjy14qgTT6046sQL0SrI05ruvmJmtwN4CEAOwL3u/qSZ3QXgcXffD+BjADoA/IWZAcAPondmXgHgk2Z2ErXF5IfdPbHF2amHJuXs1ImnVjy14qgTT6046sQL0SrUa87g7g8CeDB23vvrvr/uDNf7ewCvTnY6ERERkY1BnxAQMz8/n/YImaBOPLXiqRVHnXhqxVEnXohWWpzFDA4Opj1CJqgTT614asVRJ55acdSJF6KVFmcxxWIx7REyQZ14asVTK4468dSKo068EK20OIuJ3owg56BOPLXiqRVHnXhqxVEnXohWWpzF9Pb2pj1CJqgTT614asVRJ55acdSJF6KVFmcxemiXo048teKpFUedeGrFUSeentZMQVdXV9ojZII68dSKp1YcdeKpFUedeCFaaXEWU61W0x4hE9SJp1Y8teKoE0+tOOrEC9FKi7OYhYWFtEfIBHXiqRVPrTjqxFMrjjrxQrTS4ixmaGgo7REyQZ14asVTK4468dSKo068EK20OIsZHR1Ne4RMUCeeWvHUiqNOPLXiqBMvRCstzmJaW1vTHiET1ImnVjy14qgTT6046sQL0UqLs5ju7u60R8gEdeKpFU+tOOrEUyuOOvFCtNLiLGZiYiLtETJBnXhqxVMrjjrx1IqjTrwQrbQ4i9G/HjjqxFMrnlpx1ImnVhx14umRsxRUKpW0R8gEdeKpFU+tOOrEUyuOOvFCtNLiLKZcLqc9QiaoE0+teGrFUSeeWnHUiReilRZnMTrWC0edeGrFUyuOOvHUiqNOPB3nLAU61gtHnXhqxVMrjjrx1IqjTjwd5ywF+Xw+7REyQZ14asVTK4468dSKo068EK20OIvp7OxMe4RMUCeeWvHUiqNOPLXiqBMvRCstzmImJyfTHiET1ImnVjy14qgTT6046sQL0UqLs5ienp60R8gEdeKpFU+tOOrEUyuOOvFCtNLiLEZvJ+aoE0+teGrFUSeeWnHUiadDaaRgaWkp7REyQZ14asVTK4468dSKo068EK20OIvRsV446sRTK55acdSJp1YcdeLpOGcp0LFeOOrEUyueWnHUiadWHHXiNdVxzsxsl5k9bWZHzOyONba/18yeMrNvmdlfm9nWum23mtn3oq9bk5yzra0tyZtvGurEUyueWnHUiadWHHXihWgVZHFmZjkA9wB4M4DtAG4xs+2xix0GcLW7vwbAAwA+Gl23F8AHALwOwA4AHzCzxN4qUSgUkrrppqJOPLXiqRVHnXhqxVEnXohWoR452wHgiLs/4+4VAPcBuKn+Au7+dXdfjE4+CuDi6PsbAHzV3afcfRrAVwHsSmrQ6enppG66qagTT614asVRJ55acdSJF6JVS+I/oeYiAMfqTo+g9kjYmewB8JWzXPei+BXGx8exZ88etLS0oFqtYvfu3di7dy9GR0fR3t6OXC6H2dlZDAwMYGpqCu6OgYEBjI2NoaOjAwAwPz+Pjo4OHDt2DGaG3t5eFItFdHV1oVqt4tr+Cg7OtOCqLStYrBqGF3O4onMFzy7m0Nni6MufXN0+t2I4Xt6E4eFh9PT0oFwuY2lpCUNDQxgdHUVbWxsKhQKmp6fR19eHubk5VCqV1e2FQgH5fB6lUgn9/f0olUpYXl5e3d7IPg0ODqJYLK65TwsLC6u32draiu7ubkxMTKC7uxuVSgXlcnl1ez6fR2dnJyYnJ9HW1oaxsbGm2qek7qfW1lYMDw831T4ldT9VKhXMzMw01T4lcT9VKhWMj4831T4ldT9Vq1UsLi421T4lcT9VKhWcOHGiqfYpqfupvb0dIyMj571PZ2PuftYLrAczexuAG9z9tuj0OwHscPf/uMZl3wHgdgDXuvsJM/vPADa7++9F298HYNHd/1v99Q4cOODbtm0771mPHz+OCy+8cM1t1+873PDtPXzblec70oZ0tk5yOrXiqRVHnXhqxVEn3nq1OnTo0MGdO3devda2UE9rjgB4Sd3piwH8MH4hM7sOwG8BuNHdTzRy3fVSqVSSuummok48teKpFUedeGrFUSdeiFahFmePAbjczC41szyAmwHsr7+AmV0J4JOoLczG6zY9BOB6M+uJ3ghwfXReInSsF4468dSKp1YcdeKpFUedeE1znDN3X0HtqcqHAHwHwP3u/qSZ3WVmN0YX+xiADgB/YWZPmNn+6LpTAH4XtQXeYwDuis5LhI71wlEnnlrx1IqjTjy14qgTL0SrUG8IgLs/CODB2Hnvr/v+urNc914A9yY33XP0dmKOOvHUiqdWHHXiqRVHnXgb6lAaZvZrZtaf5DAbQT6fT3uETFAnnlrx1IqjTjy14qgTL0SrRp7WvA7As2b2ZTN7u5ltTmqoNJVKpbRHyAR14qkVT6046sRTK4468UK0ohdn7n4jgK2oHX/sPQBGzWyfmb0hqeHS0N/f9A8Orgt14qkVT6046sRTK4468UK0augNAe4+6e73uPs1AK4F8BMAvm5mz5rZb5lZRyJTBqR/PXDUiadWPLXiqBNPrTjqxNtQj5ydYmY7zexPATwCYAzALwN4J4Ar8dxR/TNreXk57REyQZ14asVTK4468dSKo068EK3od2ua2e+jdnyyEoDPAvhtd//nuu2PAsj8h3PpWC8cdeKpFU+tOOrEUyuOOvE22nHO2gD8S3d/lbt/pH5hBgDuvgxgzY8hyBId64WjTjy14qkVR514asVRJ95GO87ZhwAs1p8RHbG/4O4/BAB3/6d1nC0V7e3taY+QCerEUyueWnHUiadWHHXihWjVyCNnX0Ltcy3rXQzgL9dvnPTlcrm0R8gEdeKpFU+tOOrEUyuOOvFCtGpkcfZKd/92/RnR6W3rO1K6Zmdn0x4hE9SJp1Y8teKoE0+tOOrEC9GqkcXZuJm9vP6M6PTk+o6UroGBgbRHyAR14qkVT6046sRTK4468UK0amRxdi+AL5jZz5vZdjP7BQAPANiXzGjpmJpK7DPVm4o68dSKp1YcdeKpFUedeCFaNfKGgA8DWAbw+wBeAuAYaguzP0hgrtS4e9ojZII68dSKp1YcdeKpFUedeCFa0Yszdz8J4GPRV9PSQ7scdeKpFU+tOOrEUyuOOvE22tOaMLNXmtm/MrN/W/+V1HBpGBsbS3uETFAnnlrx1IqjTjy14qgTL0SrRj4h4DcBvB/AN3H68c4ctdejNYWOjsx/PGgQ6sRTK55acdSJp1YcdeKFaNXIa87eA2CHu38rqWFEREREXuwaeVqzDCDznwBwLvPz82mPkAnqxFMrnlpx1ImnVhx14oVo1cji7H0A/tjMLjSzTfVfSQ2XhsHBwbRHyAR14qkVT6046sRTK4468UK0amRh9WkA/w7ACGqH1FgGsBL9t2kUi8W0R8gEdeKpFU+tOOrEUyuOOvFCtGrkNWeXJjbFBmJmaY+QCerEUyueWnHUiadWHHXihWjVyHHOhgEgehpz0N2PJzZVinp7e9MeIRPUiadWPLXiqBNPrTjqxAvRin5a08y2mNnnACwBOBKdd6OZ/V5Sw6VBD+1y1ImnVjy14qgTT6046sQL0aqR15z9CYASgK0AKtF5BwC8fb2HSlNXV1faI2SCOvHUiqdWHHXiqRVHnXghWjXymrOdAH7U3ZfNzAHA3Ytm9iPJjJaOarWa9giZoE48teKpFUedeGrFUSdeiFaNPHJWAtBff4aZXQKgqV57trCwkPYImaBOPLXiqRVHnXhqxVEnXohWjSzO9gH4gpn9DIBNZnYNgM+g9nTnOZnZLjN72syOmNkda2x/g5kdMrMVM3trbFvVzJ6IvvY3MHPDhoaGkrz5pqFOPLXiqRVHnXhqxVEnXohWjSzOPgLgfgD3AGhF7fM0/wrAH53rimaWi673ZgDbAdxiZttjF/sBgHcB+NwaN1F299dGXzc2MHPDRkdHk7z5pqFOPLXiqRVHnXhqxVEnXohWjRxKwwF8PPpq1A4AR9z9GQAws/sA3ATgqbrbfzbadvIF3P66aW1tTfPHZ4Y68dSKp1YcdeKpFUedeCFa0YszM3vTmba5+9fOcfWLAByrOz0C4HXszwbQZmaPo/aJBB929y81cN2GdHd3J3XTTUWdeGrFUyuOOvHUiqNOvBCtGnm35qdipwcA5FFbaF12juuudThdb+BnX+LuPzSzywB8zcy+7e5H6y8wPj6OPXv2oKWlBdVqFbt378bevXsxOjqK9vZ25HI5zM7OYmBgAFNTU3B3DAwMYGxsDB0dHQBqH2a6srKCfD4PM0Nvby+KxSK6urpQrVZxbX8FB2dacNWWFSxWDcOLOVzRuYJnF3PobHH05U+ubp9bMRwvb8Lw8DB6enpQLpextLSEoaEhjI6Ooq2tDYVCAdPT0+jr68Pc3Bwqlcrq9kKhgHw+j1KphP7+fpRKJSwvL69ub2SfBgcHUSwW19ynhYWF1dtsbW1Fd3c3JiYm0N3djUqlgnK5vLo9n8+js7MTk5OTOHHiBLq7u5tqn5K6nyYmJlAoFJpqn5K6n44ePYpLLrmkqfYpifvp6NGjGBoaaqp9Sup+mpmZwcte9rKm2qck7qexsTG88pWvbKp9Sup+Wl5eRltb23nv09lY7dnKxkWvI/ttAHPu/gfnuOw1AD7o7jdEp+8EAHf/0BqX/TSAL7v7A2e4rTW3HzhwwLdt2/YC9uR0MzMz2LJly5rbrt93uOHbe/i2K893pA3pbJ3kdGrFUyuOOvHUiqNOvPVqdejQoYM7d+68eq1tjbwh4DTuXgVwN4DfIC7+GIDLzexSM8sDuBkA9a5LM+sxs83R9/0AXo+616qtt0qlcu4LiTo1QK14asVRJ55acdSJF6LVC16cRX4WwDlfwO/uKwBuB/AQgO8AuN/dnzSzu8zsRgAws58wsxEAbwPwSTN7Mrr6FQAeN7NvAvg6aq85S2xxVi6Xk7rppqJOPLXiqRVHnXhqxVEnXohWjbwh4BhOf53YBQDaAPwqc313fxDAg7Hz3l/3/WMALl7jen8P4NXsnOdLx3rhqBNPrXhqxVEnnlpx1Im30Y5z9g4A76z72oXaxzl9NonB0qJjvXDUiadWPLXiqBNPrTjqxNtoxzn7myQH2Sjy+XzaI2SCOvHUiqdWHHXiqRVHnXghWjXytOafgTj8hbv/8nlNlLLOzs60R8gEdeKpFU+tOOrEUyuOOvFCtGrkac0ZAL8IIIfasc02oXaU/xkAR+u+Mm1ycjLtETJBnXhqxVMrjjrx1IqjTrwQrRo5CO0rAPycu//tqTPM7KcBvO/U8cuaQU9PT9ojZII68dSKp1YcdeKpFUedeCFaNfLI2U8CeDR23jcAXLN+46RPbyfmqBNPrXhqxVEnnlpx1IkXolUji7PDAP6rmRUAIPrv3QCeSGKwtCwtLaU9QiaoE0+teGrFUSeeWnHUiReiVSOLs3ehdnT+kpmNASgB+GkAtyYwV2p0rBeOOvHUiqdWHHXiqRVHnXgb6jhn7v6su/8UgJcBuBHAy939p9z9+4lNlwId64WjTjy14qkVR514asVRJ16IVg19fJOZ9QF4I4Br3f0HZvajZva8o/pnWVtbW9ojZII68dSKp1YcdeKpFUedeCFa0YszM7sWwNMAfgnA+6KzLwfwiQTmSk2hUEh7hExQJ55a8dSKo048teKoEy9Eq0YeOfs4gLe7+y4AK9F53wCwY92nStH09HTaI2SCOvHUiqdWHHXiqRVHnXghWjWyOHupu/919P2pTwqooLFjpW14fX19aY+QCerEUyueWnHUiadWHHXihWjVyOLsKTOLH2z2OgDfXsd5Ujc3N5f2CJmgTjy14qkVR514asVRJ16IVo086vWfAHzZzP43gIKZfRLAL6D2EU5No1KppD1CJqgTT614asVRJ55acdSJF6JVI4fSeBTAawA8CeBeAN8HsMPdH0totlToWC8cdeKpFU+tOOrEUyuOOvE2zHHOzCxnZo8AmHT3j7r7Xnf/sLuPJDteeDrWC0edeGrFUyuOOvHUiqNOvBCtqKc13b1qZpeiweOiZdF6v0X2+n2HG77Ow7ddua4zJEFvu+apFU+tOOrEUyuOOvE22qE0fgfAJ8xsa/RI2qZTX0kNl4Z8Pp/2CJmgTjy14qkVR514asVRJ16IVo0srPYB+GXUXmtWAbCM2vHOlhOYKzWlUintETJBnXhqxVMrjjrx1IqjTrwQrc75tKaZDbn7KIBLE59mA+jv7097hExQJ55a8dSKo048teKoEy9EK+aRs+8CgLsPu/swgD889X3deU1D/3rgqBNPrXhqxVEnnlpx1IkXohWzOLPY6TcmMMeGsbzcVM/SJkadeGrFUyuOOvHUiqNOvBCtmMWZn/sizUPHeuGoE0+teGrFUSeeWnHUibdRjnPWYmY/Y2ZvMrM3xU9H5zUNHeuFo048teKpFUedeGrFUSfeRjnO2ThqnwhwymTstAO4bD2HSlN7e3vaI2SCOvHUiqdWHHXiqRVHnXghWp1zcebuL018ig0kl8ulPUImqBNPrXhqxVEnnlpx1IkXolWwA8ia2S4ze9rMjpjZHWtsf4OZHTKzFTN7a2zbrWb2vejr1iTnnJ2dTfLmm4Y68dSKp1YcdeKpFUedeCFaBVmcmVkOwD0A3gxgO4BbzGx77GI/APAuAJ+LXbcXwAcAvA7ADgAfMLOepGYdGBhI6qabijrx1IqnVhx14qkVR514IVqFeuRsB4Aj7v6Mu1cA3AfgpvoLuPuz7v4tACdj170BwFfdfcrdpwF8FcCupAadmppK6qabijrx1IqnVhx14qkVR514IVqFWpxdBOBY3emR6Lykr9sw9xfVkUNeMHXiqRVPrTjqxFMrjjrxQrRi3q25HuIHsgX446dR1x0fH8eePXvQ0tKCarWK3bt3Y+/evRgdHUV7eztyuRxmZ2cxMDCAqakpuDsGBgYwNjaGjo4OAMD8/Dy2bNmCY8eOwczQ29uLYrGIrq4uVKtVXNtfwcGZFly1ZQWLVcPwYg5XdK7g2cUcOlscffmTq9vnVgzHy5vwis4qji7k0Jc/iS2tvrp9ZtkwWdmEl7VX8d25HC4snERnS2378PAwCoUC8vk8SqUS+vv7USqVsLy8jKGhoYb3aXBwEMVicc19WlhYWL3N1tZWdHd3Y2JiAt3d3ahUKiiXy6vb8/k8Ojs7MTk5ifb2doyNjWFpaWl1e1tbGwqFAqanp9HX14e5uTlUKpXV7Rt9n3p6elAul9d9n/L5PIaHh5tqn5K6nyqVCmZmZppqn5K4nyqVCsbHx5tqn5K6n6rVKhYXF5tqn5K4nyqVCk6cONFU+5TU/dTd3Y2RkZHz3qezLnxCrADN7BoAH3T3G6LTdwKAu39ojct+GsCX3f2B6PQtAN7o7v8hOv1JAI+4++frr3fgwAHftm3bec86PDyMrVu3rrnt+n2Hz/v2GQ/fdmWQn3M+ztZJTqdWPLXiqBNPrTjqxFuvVocOHTq4c+fOq9faFuppzccAXG5ml5pZHsDNAPaT130IwPVm1hO9EeD66LxEnFr9ytmpE0+teGrFUSeeWnHUiReiVZDFmbuvALgdtUXVdwDc7+5PmtldZnYjAJjZT5jZCIC3AfikmT0ZXXcKwO+itsB7DMBd0XkiIiIiTSfYcc7c/UF3f4W7v8wBmCDCAAAQT0lEQVTd747Oe7+774++f8zdL3b3dnfvc/dX1V33Xnd/efT1p0nOOT8/n+TNNw114qkVT6046sRTK4468UK0CrY4y4rBwcG0R8gEdeKpFU+tOOrEUyuOOvFCtNLiLKZYLKY9QiaoE0+teGrFUSeeWnHUiReilRZnMWZrHblD4tSJp1Y8teKoE0+tOOrEC9FKi7OY3t7etEfIBHXiqRVPrTjqxFMrjjrxQrTS4ixGD+1y1ImnVjy14qgTT6046sTT05op6OrqSnuETFAnnlrx1IqjTjy14qgTL0QrLc5iqtVq2iNkgjrx1IqnVhx14qkVR514IVppcRazsLCQ9giZoE48teKpFUedeGrFUSdeiFZanMUMDQ2lPUImqBNPrXhqxVEnnlpx1IkXopUWZzHn+qR4qVEnnlrx1IqjTjy14qgTL0QrLc5iWltb0x4hE9SJp1Y8teKoE0+tOOrEC9FKi7OY7u7utEfIBHXiqRVPrTjqxFMrjjrxQrTS4ixmYmIi7REyQZ14asVTK4468dSKo068EK20OIvRvx446sRTK55acdSJp1YcdeLpkbMUVCqVtEfIBHXiqRVPrTjqxFMrjjrxQrTS4iymXC6nPUImqBNPrXhqxVEnnlpx1IkXopUWZzE61gtHnXhqxVMrjjrx1IqjTjwd5ywFOtYLR514asVTK4468dSKo048HecsBfl8Pu0RMkGdeGrFUyuOOvHUiqNOvBCttDiL6ezsTHuETFAnnlrx1IqjTjy14qgTL0QrLc5iJicn0x4hE9SJp1Y8teKoE0+tOOrEC9FKi7OYnp6etEfIBHXiqRVPrTjqxFMrjjrxQrTS4ixGbyfmqBNPrXhqxVEnnlpx1ImnQ2mkYGlpKe0RMkGdeGrFUyuOOvHUiqNOvBCttDiL0bFeOOrEUyueWnHUiadWHHXihWjVkvhPyJjR0VFs3bo11Rmu33e4ocs/fNuVCU1yZhuhU1aoFU+tOOrEUyuOOvFCtNIjZzFtbW1pj5AJ6sRTK55acdSJp1YcdeKFaBVscWZmu8zsaTM7YmZ3rLF9s5n9ebT9G2b20uj8l5pZ2cyeiL7+JMk5C4VCkjffNNSJp1Y8teKoE0+tOOrEC9EqyOLMzHIA7gHwZgDbAdxiZttjF9sDYNrdXw7gDwF8pG7bUXd/bfT1K0nOOj09neTNNw114qkVT6046sRTK4468UK0CvXI2Q4AR9z9GXevALgPwE2xy9wE4DPR9w8A2GlmFmi+VX19faF/ZCapE0+teGrFUSeeWnHUiReiVajF2UUAjtWdHonOW/My7r4CoATgVIFLzeywmf2Nmf2LJAedm5tL8uabhjrx1IqnVhx14qkVR514IVqFerfmWo+AOXmZ4wAucfdJM7sKwJfM7FXuPlt/wfHxcezZswctLS2oVqvYvXs39u7di9HRUbS3tyOXy2F2dhYDAwOYmpqCu2NgYABjY2Po6OgAAMzPz2NlZQXHjh2DmaG3txfFYhFdXV2oVqu4tr+CgzMtuGrLCharhuHFHK7oXMGzizl0tjj68idXt8+tGI6XN+EVnVUcXcihL38SW1p9dfvMsmGysgkva6/iu3M5XFg4ic6W57ZPVjZhbsXw0guq+M5cC7ZeUMUFuee2j53YhOWTwMWFk1hcXDzrPg0ODqJYLK65TwsLCxgaGsLo6ChaW1vR3d2NiYkJdHd3o1KpoFwur27P5/Po7OzE5OQkTpw4gU2bNmFpaWl1e1tbGwqFAqanp9HX14e5uTlUKpXV7YVCAfl8HqVSCf39/SiVSlheXl7d3sj9lMQ+9fT0oFwur/s+lUolVCqVptqnpO6nYrGIQqHQVPuUxP1ULBaRy+Waap+Sup9mZmZWf7eaZZ+SuJ+KxSJ6e3ubap+Sup+Wl5cxMjJy3vt01kWTe3yNtP7M7BoAH3T3G6LTdwKAu3+o7jIPRZc5YGYtAEYBDHhsQDN7BMCvu/vj9ecfOHDAt23bdt6znjhxAps3b15zW6OHuAgljUNpnK2TnE6teGrFUSeeWnHUibderQ4dOnRw586dV6+1LdTTmo8BuNzMLjWzPICbAeyPXWY/gFuj798K4Gvu7mY2EL2hAGZ2GYDLATyT1KDnWs1KjTrx1IqnVhx14qkVR514IVoFeVrT3VfM7HYADwHIAbjX3Z80s7sAPO7u+wF8CsCfmdkRAFOoLeAA4A0A7jKzFQBVAL/i7lNJzaq3E3PUiadWPLXiqBNPrTjqxAvRKtgnBLj7gwAejJ33/rrvlwC8bY3rfQHAFxIfMJLP50P9qExTJ55a8dSKo048teKoEy9EK31CQEypVEp7hExQJ55a8dSKo048teKoEy9EKy3OYvr7+9MeIRPUiadWPLXiqBNPrTjqxAvRSouzGP3rgaNOPLXiqRVHnXhqxVEnXohWwV5zlhXLy8tpj9CwF3KIj/M9/EYWO6VFrXhqxVEnnlpx1IkXopUeOYsZGhpKe4RMUCeeWvHUiqNOPLXiqBMvRCstzmJ0rBeOOvHUiqdWHHXiqRVHnXghWmlxFtPe3p72CJmgTjy14qkVR514asVRJ16IVlqcxeRyubRHyAR14qkVT6046sRTK4468UK00uIsZnZ29twXEnVqgFrx1IqjTjy14qgTL0QrLc5iBgYG0h4hE9SJp1Y8teKoE0+tOOrEC9FKh9KImZqawgUXXJD2GIk738NvvFg6rQe14qkVR514asVRJ16IVnrkLMbd0x4hE9SJp1Y8teKoE0+tOOrEC9FKi7MYPbTLUSeeWvHUiqNOPLXiqBNPT2umYGxsDFu3bk17jA2p/qnQa/sr+JuJ/Fkvf76fQtAs9DvFUyuOOvHUiqNOvBCt9MhZTEdHR9ojZMLxJf3qsPQ7xVMrjjrx1IqjTrwQrfT/sCIiIiIbiBZnMfPz82mPkAkXtp1Me4TM0O8UT6046sRTK4468UK00uIsZnBwMO0RMuGJkl6uyNLvFE+tOOrEUyuOOvFCtNLiLKZYLKY9Qia8umsl7REyQ79TPLXiqBNPrTjqxAvRSouzGDNLe4RMWHF1Yul3iqdWHHXiqRVHnXghWmlxFtPb25v2CJnwvXl9SC5Lv1M8teKoE0+tOOrEC9FKi7MYPbTL+TE9rUnT7xRPrTjqxFMrjjrxQrTSq7pjurq60h4hE0bKyazrz/czPzci/U7x1IqjTjy14qgTL0QrPXIWU61W0x4hE1r1m0PT7xRPrTjqxFMrjjrxQrTS/8XGLCwspD1CJgxu1nHOWPqd4qkVR514asVRJ16IVlqcxQwNDaU9QiYcnNEz4iz9TvHUiqNOPLXiqBMvRCstzmJGR0fTHiETrtqiNwSw9DvFUyuOOvHUiqNOvBCtgj38YWa7APwRgByAfe7+4dj2zQA+C+AqAJMA3u7uz0bb7gSwB0AVwK+5+0NJzfmlL30J7373u5O6+abx+Ne/Arz6prNe5oW8uP+FaPTnhH4DgX6neGrFUSeeWnHUiReiVZBHzswsB+AeAG8GsB3ALWa2PXaxPQCm3f3lAP4QwEei624HcDOAVwHYBeB/RLeXiC9+8YtJ3XRTOfzIV9IeIajr9x1u+OsU/U7x1IqjTjy14qgTL0SrUI+c7QBwxN2fAQAzuw/ATQCeqrvMTQA+GH3/AID/brXD8N4E4D53PwHg+2Z2JLq9A0kMurKip+sYhQw/IR76Eb3x2aXEfuZGP4xIo/T3x1Ennlpx1IkXopW5e/I/xOytAHa5+23R6XcCeJ273153mX+MLjMSnT4K4HWoLdgedff/GZ3/KQBfcfcH6n/Ggw8+OHf8+PHVJUNXV1ext7d3otFZp6am+l/I9V5s1ImnVjy14qgTT6046sRbx1Zbd+7cObDWhlCPnK31QVTxVeGZLsNcF295y1s6X8BcIiIiIhtKqCenRgC8pO70xQB+eKbLmFkLgG4AU+R1RURERJpCqMXZYwAuN7NLzSyP2gv898cusx/ArdH3bwXwNa8957ofwM1mttnMLgVwOYB/CDS3iIiISFBBFmfuvgLgdgAPAfgOgPvd/Ukzu8vMbowu9ikAfdEL/t8L4I7ouk8CuB+1Nw/8HwB73X3dPzvBzHaZ2dNmdsTM7ljv288yM7vXzMaj1wWeOq/XzL5qZt+L/tuT5owbgZm9xMy+bmbfMbMnzezd0flqFWNmbWb2D2b2zajV70TnX2pm34ha/Xn0j7kXPTPLmdlhM/tydFqd1mBmz5rZt83sCTN7PDpPf39rMLMtZvaAmf1T9L9Z16jV85nZK6Pfp1Nfs2b2nqRbBXlDwEYXHZrjuwB+FrWnUR8DcIu7P3XWK75ImNkbAMwD+Ky7/1h03kcBTLn7h6PFbI+7/5c050ybmV0I4EJ3P2RmnQAOAvhFAO+CWp0meid2u7vPm1krgL8D8G7U/mH2RXe/z8z+BMA33f0Tac66EZjZewFcDaDL3X/ezO6HOj2PmT0L4Gp3n6g7T/9btQYz+wyAv3X3fdHi/gIAvwm1OqNorfDPqL1ZcS8SbJXhAyKsq9VDfbh7BcCpQ30IAHf/f6i9/q/eTQA+E33/GdQWIS9q7n7c3Q9F38+h9ijxRVCr5/Ga+ehka/TlAN6E2qF0ALUCAJjZxQB+DsC+6LRBnRqhv78YM+sC8AbUnrGCu1fcfQZqdS47ARx192Ek3EqLs5qLAByrOz0SnSdnNujux4HaogTAj6Q8z4ZiZi8FcCWAb0Ct1hQ9VfcEgHEAXwVwFMBM9DIIQH+Hp3wcwG8AOBmd7oM6nYkDeNjMDprZv4/O09/f810GoAjgT6Ony/eZWTvU6lxuBvD56PtEW2lxVkMdrkOEYWYdAL4A4D3uPpv2PBuVu1fd/bWovQN7B4Ar1rpY2Kk2FjP7eQDj7n6w/uw1Lvqi7lTn9e7+46h9Gs3e6CUZ8nwtAH4cwCfc/UoAC4he5y1ri576vRHAX4T4eVqc1ehwHY0bi15jdeq1VuMpz7MhRK+f+gKA/+Xupz7jQ63OIno65REAPwlgS3QoHUB/hwDwegA3Rq+lug+1pzM/DnVak7v/MPrvOIC/RG3Rr7+/5xsBMOLu34hOP4DaYk2tzuzNAA65+1h0OtFWWpzVMIf6kNPVH/rkVgB/leIsG0L0WqBPAfiOu/9B3Sa1ijGzATPbEn1fAHAdaq/R+zpqh9IB1Arufqe7X+zuL0Xtf5e+5u6/BHV6HjNrj96Ig+gpuusB/CP09/c87j4K4JiZvTI6aydqR0RQqzO7Bc89pQkk3Erv1oyY2VtQ+xdpDsC97n53yiNtGGb2eQBvBNAPYAzABwB8CbVDnFwC4AcA3ubu8TcNvKiY2U8D+FsA38Zzrw/6TdRed6ZWdczsNai9iDaH2j8S73f3u8zsMtQeIeoFcBjAO6LP1X3RM7M3Avj16N2a6hQTNfnL6GQLgM+5+91m1gf9/T2Pmb0WtTeZ5AE8A+DfIPpbhFqdxswuQO116Ze5eyk6L9HfKy3ORERERDYQPa0pIiIisoFocSYiIiKygWhxJiIiIrKBaHEmIiIisoFocSYiIiKygWhxJiIiIrKBaHEmIiIisoFocSYiIiKygfx/DPgLNvPWB9cAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "data['Hazard'].plot(kind='hist', figsize=(10, 4), bins=40, density=True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "source": [
    "В обобщённых линейных моделях, как и в машинном обучении в целом, мы проводим основную работу с апостериорным распределением $p(y|x)$, ведь именно в нём заключается информация о конкретной задаче. Здесь же мы знаем, что количество несчастных случаев во многом подчиняется распределению Пуассона, поэтому будем стараться моделировать именно его."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "В выборке могут присутствовать шумовые признаки. Попробуем простейшим способом избавиться от них.\n",
    "\n",
    "Исследуйте абсолютное значение корреляции:\n",
    "* Признаков и отклика\n",
    "* Признаков и логарифма отклика"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlYAAAEQCAYAAAByTDrzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xt05XV97//nZ7KTmdxnkokz6AhYGJxSLyACg4Pc5pRLL6JUFK1WOXDqsWD7a1f1yKpSq57aemytS1xHrResrtYLqyJVKLZVQOzIUQZaRARmKAMzTjK5TW4ks5Odz++P7NmEfN/fZGey893v7Lyea82aJN+dvR/fS3Y+2fu7PzvEGFFKKaWUUktvTbUBSimllFK1kgZWSimllFIVSgMrpZRSSqkKpYGVUkoppVSF0sBKKaWUUqpCaWCllFJKKVWhctUGANx1111x7dq11WYopZRSSi3YM88807dz584ua5mLgdXatWvZtm3bkq5j3759nHDCCRUSLS1Z7GSxk8WvA2RJSxY7Wey8WCrl2L179760ZTXzVGB9fX21CaVksZPFTpZkXhwgS1qy2Mli58WShaNmBlbt7e3VJpSSxU4WO1mSeXGALGnJYieLnRdLFo6aGVj19fVVm1BKFjtZ7GRJ5sUBsqQli50sdl4sWThcnGNlFWNkdHSUct/LsLm5meHh4WVWlZcsdku1hBBoaWkhhLBki5e/nkAWKy8OkCUtWexksfNiycLhdmA1OjrK2rVraWhoKOvyk5OTbp7DlcVuqZZ8Ps/o6Citra1LtuTz+SVfR6WSJZkXB8iSlix2sth5sWThcPtUYIyx7EEVwPT09DJqFpcsdku1NDQ0lP0I5kKNj49X5HoqkSzJvDhAlrRksZPFzoslC4fbgdVi8/KoDMiSlifL5s2bq00oJUsyLw6QJS1Z7GSx82LJwlEzA6vJycnMbuuFL3zhvMuztCzUUcsDDzzAe9/7XgDuvfde7rvvvtJlrrvuOr71rW/Nez1DQ0N8/vOfr4hloeb6lqPu7u5lvf7FJEsyLw6QJS1Z7GSx82LJwuH2HKu5Xfy5Byp6fd+99vSKXt/sKnFy9dwKhQJ1dXWlz6empsjlFt59Ry2nn346p58+s8733nsvzc3NnH322WXf/tGB1TXXXFP298QYiTGyZs2a51gW6lh8i20xTzMvd7Ik8+IAWdKSxU4Wu6wsC40VXrF+kt3/MjDvZZY6PqiZR6yWo7e85S1ceOGFnHPOOdx8883PWfa+972PCy64gNe+9rWll29+5jOfYfv27Vx00UXmAKRQKPD+97+fHTt2cO655/LZz34WgLvvvpvzzz+fHTt2cP3113PkyBEAXv7yl/PRj36Uyy67jFtvvZXf/M3f5EMf+hC/8Ru/wac//ennXPeOHTsYGhoixshJJ53EV7/6VQDe9a53cdddd3Hvvfdy1VVX8dRTT3HzzTfz6U9/mvPOO49du3YBsGvXLi655BJOP/1089GrP/uzP+PJJ5/kvPPO48Ybb2R0dJTXvva1XHDBBezYsYPbb78dgKeeeoqzzz6bP/7jP+aCCy7gwIEDfPnLX+bMM8/kiiuu4A/+4A94z3veA8y87PV3fud32LlzJzt37uRHP/pRqq/SVeIE+EolSzIvDpAlLVnsZLHzYjk4vvzDnhXziFU1+uQnP8mGDRsYHx9n586dvOY1r6Gjo4OxsTFe/vKX8+EPf5iPfvSjpX+f+MQneOCBmdHyxMRE4vq+9KUvsW/fPu6++25yuRyDg4NMTExw3XXXceutt3LyySfzzne+ky984Qu8853vBGDdunXccccdANx8880MDQ3x7W9/O3HdZ599Nvfddx9btmzhxBNPZNeuXVx11VX85Cc/4a/+6q948MEHATj++ON5+9vfTnNzM+9617sA+MpXvkJ3dzd33HEHjz32GL/927/N5Zdf/pzr/9M//VMeeeQR7rnnHmDmEbO/+7u/o62tjf7+fi6++GIuu+wyAPbs2cNNN93Exz72MQ4ePMjHPvYx7rrrLurr63nDG97AS17yEgBuuOEGfu/3fo/t27ezf/9+fuu3fov77rsv4VuO+vv7aWlpWbbrX0yy+HWALGnJYieLnRfLKa0FDh6pW/iCS6isoVsI4dIQwqMhhD0hhPcay88LIewOIUyFEF4/Z9nbQgiPF/+9rVLwLPrMZz7Dq1/9ai6++GIOHDjA3r17AVizZg2ve93rAHjDG97Aj370IwBOPfVUfvd3f5dvfvObz3na7mh33XUXV199dekpvA0bNrBnzx5OOOEETj75ZACuuuqq5zxKc/R20j4/2jnnnMO///u/s2vXLq6++mp+9rOf8Ytf/IINGzaUdTD/+q//OmvWrGHbtm309vYuePkYIx/+8Ic599xzed3rXsfBgwc5dOgQMHMO2plnngnA7t272bFjBxs2bKCxsfE5A7a7776b97znPZx33nm8+c1vZnR0lJGRkQVvuxJt2LAhk9spJ1mSeXGALGnJYieLnRfL3rHlHVRBGQOrEEId8CngMuBU4E0hhFPnXOwp4O3A38/53g7gT4GzgbOAPw0h+Ni6C3Tvvfdy9913c+edd/KDH/yAl73sZaWn6OZ29Nyhr33ta1x77bU8+OCDXHTRRUxNTaVe9mgLTR/Q1NQ07+dHO+ecc9i1axe7du3i3HPPZePGjdx2221ln6c0+/nvcqY0+MY3vkFfXx/f//73ueeee+jq6iptn9nG2dc1d7qF6elp7rzzTu655x7uueceHn744cweLvby0l+QxcqLA2RJSxY7Wey8WDobln8KonIesToL2BNjfCLGmAe+CjzneaIY45Mxxv8E5oovAf4lxjgQYxwE/gW4tALuZW94eJj169fT1NTEY489xk9+8pPSsunp6dJ5SLfccgvbt29nenqaAwcO8OpXv5r3ve99DA0NMTY29pzrvPDCC/niF79YGnANDg6ydetWnnrqKZ544gkAvv71r/OqV71q0d4tW7YwMDDAE088wYknnsj27du56aabOOussxKXbWlpYXR0dFHXP/d7hoeH6erqor6+nh/84Ac8/fTT5ve94hWv4Ic//CGHDx8mn8/zT//0T6VlF154IX/7t39b+vyhhx46Zt9is56qrVayJPPiAFnSksVOFjsvlvX1lZkLcb7KGVi9AJj9W3N/8WvltJTvrWo7d+5kamqKc889lz//8z/nla98ZWlZc3MzP//5z7nwwgv5wQ9+wLvf/W4KhQLveMc72LFjBxdffDHvfOc7E1Pnv/Wtb2XLli2ce+65vPrVr+aWW25h3bp13HTTTVx99dXs2LGDEAJXX331MZnPOOMMTjrpJAC2b9/OwYMH2bFjR+Jyl156Kd/5zncWdXJ4R0cHZ599Nq961au48cYbufLKK3nggQe46KKL+MY3vsHWrVvN73v+85/PH/7hH/Krv/qrvPGNb+TFL34xbW1tAPzFX/wFDz74IOeeey7bt2/ni1/84jH7FpuXOVVAFisvDpAlLVnsZLHzYrn/8PKfWh4WetonhHAlcEmM8dri528FzooxJs4sDiHcDHw7xnhL8fN3A2tjjB8ufv5+4JkY41/N/r5vfetb8YYbbiCXy1EoFLjiiiu4+uqraWpqYs2aNYQQKBQKpeUAuVyOycnJ0rlM+XyedevWlR4NqqurY2pqirq6OmKMTE9PU19fz+TkJCGEspfncjmmp6efs3zNmjWsWbOmtLxQKBBjLC0/etnZ5tnLy12nQqFAfX39ktZp1r5Z0jpZ5nLX6fDhw7S1tTE+Ps61117LW97yFi655JJjWqe+vj46Ojro7u6moaGB1tZW+vv7Sy8ymJiYYPPmzXR3d7Nu3ToaGxsZHByks7OTkZER8vk8mzdv5tFHH2Xz5s00NDQwNDTExo0bGRoaYnJysvT9zc3N1NXVlR6dGxgYIMZIV1cXPT09pXPXRkdH2bRpE729vYQQ6OjooLe3l7a2NgqFAmNjY6XrrK+vp729nb6+Ptrb28nn83R3d/PiF794yevU3d1NY2PjktZpfHycrq6uJa/T+Ph4afmxrFNvb29pmyx1nZa6nyYmJkrupaxTJfZTPp8nl8tV7Nhbyjrt27ePlpaWih17S1mnvXv3cvzxx1fk2Fvqfjr6TEQl7yOOdZ3y+TwtLS0VvY841nUaGxujoaGhYvcRaev0J/f0cUJTgaa6yP2Hc5yxfoqeI2uYnIYtjdM01kUOT64hFyIPDec4rX2KgxMzjzEdt26aB4dy/PVFz1twnfr7++/fuXPns4+4zB4LlTGwOgf4QIzxkuLnNwDEGD9iXPZmnjuwehNwQYzxHcXPPwPcFWP8h9nft2vXrrht27bnXNfw8HDpkY1yqqX3xKtkHizvf//7ufvuu5mYmOCiiy7iIx/5yDHP9bXY4yKtnp4eNm3atOTrqUSy+HWALGnJYieLXVaWheaxenn7JP8xNP/vxHLmsdq9e3fqwKqcx8R+DGwNIbwIOABcBby5jO8DuBP481knrF8M3FDm9y6qo5NQekiW5/ahD30ISE5yWs0aGxurTSglSzIvDpAlLVnsZLHzYunPL//vxAVvIcY4BVzPzCDpEeDrMcaHQwgfDCG8BiCEcGYIYT9wJfCZEMLDxe8dAD7EzODsx8AHi1+reNYr8KqVLHaeLIODg9UmlJIlmRcHyJKWLHay2HmxnNRcWPbbKOssrhjj7cDtc75246yPfwxsSfneLwBfWIKxrMp5e5esksXOk6Wzs7PahFKyJPPiAFnSksVOFjsvlsdGHMxjVa1CCOTz+bIvf/RkaQ/JYrdUSz6fr9j7MGY1EWk5yZLMiwNkSUsWO1nsvFiOa1z+eaz8PIQwp6NzGZU798XY2BjNzc3LrCovWeyWagkhVOwtERYzaF/uZEnmxQGypCWLnSx2XiytueWfx8rtwCqEsKhZuNeuXcvatWuXUVR+sth5sniZUwVksfLiAFnSksVOFjsvlizmsXL7VOBi6+7urjahlCx2stjJksyLA2RJSxY7Wey8WM5Yv/wvoqqZgZWXl3KCLGnJYidLMi8OkCUtWexksfNicTHdwkpp9psIVztZ7GSxkyWZFwfIkpYsdrLYebGMTFXmBVDzVTMDq6GhoWoTSsliJ4udLMm8OECWtGSxk8XOi+XEpuV/pXzNDKw2btxYbUIpWexksZMlmRcHyJKWLHay2HmxPDKik9fLzstoGGRJSxY7WZJ5cYAsacliJ4udF8sJesSq/CYnJ6tNKCWLnSx2siTz4gBZ0pLFThY7L5amuuWfx6pmBlZe5sgAWdKSxU6WZF4cIEtastjJYufFonmsFpGXOTJAlrRksZMlmRcHyJKWLHay2HmxaB6rReTlbVtAlrRksZMlmRcHyJKWLHay2Hmx9BzRPFZlV1e3/O9YXW6y2MliJ0syLw6QJS1Z7GSx82KZXP73YK6dgdXw8HC1CaVksZPFTpZkXhwgS1qy2Mli58WypXH5R1Y1M7Dq6uqqNqGULHay2MmSzIsDZElLFjtZ7LxYfjqsk9fLbmBgoNqEUrLYyWInSzIvDpAlLVnsZLHzYtnaonmsyi7G5Z+botxksZPFTpZkXhwgS1qy2Mli58WSC5rHquy8PMwIsqQli50sybw4QJa0ZLGTxc6L5SE9FVh+PT091SaUksVOFjtZknlxgCxpyWIni50Xy2ntmseq7FpaWqpNKCWLnSx2siTz4gBZ0pLFThY7L5aDE5rHSimllFJqxVQzA6vR0dFqE0rJYieLnSzJvDhAlrRksZPFzovluHWax6rsNm3aVG1CKVnsZLGTJZkXB8iSlix2sth5sTw4pJPXy663t7fahFKy2MliJ0syLw6QJS1Z7GSx82J5aZtOXi+7EEK1CaVksZPFTpZkXhwgS1qy2Mli58UyFZffUTMDq46OjmoTSsliJ4udLMm8OECWtGSxk8XOi+Xx0eV/M+iaGVh5eZgRZElLFjtZknlxgCxpyWIni50Xy0v0VGD5tbW1VZtQShY7WexkSebFAbKkJYudLHZeLPvHNY9V2RUKy//GiuUmi50sdrIk8+IAWdKSxU4WOy+W+gxGPTUzsBobG6s2oZQsdrLYyZLMiwNkSUsWO1nsvFg2rXUyj1UI4dIQwqMhhD0hhPcay9eGEL5WXH5fCOHE4tfrQwhfCiE8FEJ4JIRwQ2X5z7Z58+bluupFJ4udLHayJPPiAFnSksVOFjsvlvsPO5jHKoRQB3wKuAw4FXhTCOHUORe7BhiMMZ4MfBz4y+LXrwTWxhhfCpwBvOPooKvSdXd3L8fVHlOy2MliJ0syLw6QJS1Z7GSx82I5Y72Pk9fPAvbEGJ+IMeaBrwKXz7nM5cCXih/fAuwMM5NWRKA5hJADGoE8MFwR+Zzq6+uX42qPKVnsZLGTJZkXB8iSlix2sth5sTxT8DGP1QuAp2d9vr/4NfMyMcYpYAjoZGaQNQYcBJ4CPhZjHFii2ay9vX05rvaYksVOFjtZknlxgCxpyWIni50Xy75nln8eq3KebLSGd7HMy5wFFIDnAxuAH4QQ/jXG+MTsCx46dIhrrrmGXC5HoVDgiiuu4LrrrqO7u5vm5mbq6uoYHh6mq6uLgYEBYox0dXXR09NDS0sLAAcOHOCUU06ht7eXEAIdHR309vbS1tZGoVBgbGyMzZs3093dTX19Pe3t7fT19dHe3k4+n2d8fLy0vKGhgdbWVvr7+9mwYQPj4+NMTEyUlq9bt47GxkYGBwfp7OxkZGSEfD5fWj42Nsbzn/98hoaG2LhxI0NDQ0xOTpaWl7tOo6OjbNq0aUnrVCgUqKurW/I6NTY20tDQsKR16u3tpbGxccnrVIn9tHfvXjZv3rzkdarEfuru7ubFL35xRY69pe6n8fFxurq6KnLsLWU/9fb2lrZJJY69peyniYmJkrtS9xHHuk75fJ5cLlfR+4hjXad9+/bR0tJS0fuIY12nvXv3cvzxx1f8vvxY1mlwcJCtW7dW/L78WNYpn8/T0tJS8fvyY1mnsbExGhoaluV37ux1et7aaU5oKtBUF7n/cI4z1k/Rc2QNk9OwpXGaxrrI4ck15ELkoeEcp7VPcXBi5jGm49ZN8+BQjqeffnrBdZp30BTj3DHSnAuEcA7wgRjjJcXPbwCIMX5k1mXuLF5mV/Fpv26gC7gJ+FGM8cvFy30B+OcY49dn38auXbvitm3b5nUs1OHDh1m/fv2SrqNSyWIni50sfh0gS1qy2Mlil5Xl4s89MO/yE5oKCz5q9d1rT1/wdnbv3n3/zp07X2ktK+epwB8DW0MILwohNABXAbfNucxtwNuKH78e+F6cGbE9BVwUZmoGtgM/L+M2F10+n1+Oqz2mZLGTxU6WZF4cIEtastjJYufF0pqb/8GkSrTgwKp4ztT1wJ3AI8DXY4wPhxA+GEJ4TfFinwc6Qwh7gD8Cjk7J8CmgBfgpMwO0L8YY/7PC6wDA+Pj4clztMSWLnSx2siTz4gBZ0pLFThY7L5bOhuWfx6qsCR1ijLcDt8/52o2zPp5gZmqFud83an19OfIyRwbIkpYsdrIk8+IAWdKSxU4WOy8WF/NYrZS8zJEBsqQli50sybw4QJa0ZLGTxc6Lxcs8ViuihoaGahNKyWIni50sybw4QJa0ZLGTxc6LZWTKxzxWK6LW1tZqE0rJYieLnSzJvDhAlrRksZPFzovl4PjyD3tqZmDV399fbUIpWexksZMlmRcHyJKWLHay2HmxnNJaWPbbqJmB1YYNG6pNKCWLnSx2siTz4gBZ0pLFThY7L5a9Y8s/83rNDKy8vJQTZElLFjtZknlxgCxpyWIni50XSxbTLdTMwGpiYqLahFKy2MliJ0syLw6QJS1Z7GSx82JZX7/8E4Qu/4QOGeVljgyQJS1Z7GRJ5sUBsqQli91qtCz0NjIALblpRqcGUpeX8zYylUjzWC0iL3NkgCxpyWInSzIvDpAlLVnsZLHLYv6octI8Voto3bp11SaUksVOFjtZknlxgCxpyWIni93hyeWfP6qcsnDUzMCqsbGx2oRSstjJYidLMi8OkCUtWexksevP+xhuZOHwsaYVaHBwsNqEUrLYyWInSzIvDpAlLVnsZLE7qXn5548qpywcNTOw6uzsrDahlCx2stjJksyLA2RJSxY7WeweG1n++aPKKQtHzQysRkZGqk0oJYudLHayJPPiAFnSksVOFrvjGpd//qhyysJRMwOrfD5fbUIpWexksZMlmRcHyJKWLHay2LXmln/+qHLKwlEzA6vVOHdIOcliJ4udF4sXB8iSlix2sthlMX9UOWXh8LGmFai7u5sTTjih2gxAlrRksZPFrwNkSUsWu6ws5UzKef7GPHf3NaQuz2pSTpiZP2o+Sy05auYRK08vK5XFThY7WZJ5cYAsacli58niZYoD8GPRdAuLqKGh+iPho8liJ4udLMm8OECWtGSx82QZmfIxKSf4sWThqJmB1dDQULUJpWSxk8VOlmReHCBLWrLYebKc2ORj7ijwY8nCUTMDq40bN1abUEoWO1nsZEnmxQGypCWLnSfLIyN+TqP2YsnCUTMDK09/JchiJ4udLMm8OECWtGSx82Q5wcmjRODHkoWjZgZWk5OT1SaUksVOFjtZknlxgCxpyWLnydJU52PuKPBjycJRMwMrT/N1yGIni50sybw4QJa0ZLHzZPEydxT4sWThqJmBVXd3d7UJpWSxk8VOlmReHCBLWrLYebKcsX6q2oRSXixZOGpmYNXc3FxtQilZ7GSxkyWZFwfIkpYsdp4sPUf8/Ir3YsnC4WNNK1BdnY93zgZZ0pLFTpZkXhwgS1qy2HmyTPp432PAjyULR80MrIaHh6tNKCWLnSx2siTz4gBZ0pLFzpNlS6OT0Qx+LFk4amZg1dXVVW1CKVnsZLGTJZkXB8iSlix2niw/HfZxwjj4sWThqJmB1cDAQLUJpWSxk8VOlmReHCBLWrLYebJsbfExdxT4sWThqJmBVYw+5sgAWdKSxU6WZF4cIEtasth5suSCLHPLwlHWwCqEcGkI4dEQwp4QwnuN5WtDCF8rLr8vhHDirGUvCyHsCiE8HEJ4KISwrnL8Z/P08KssdrLYyZLMiwNkSUsWO0+Wh5w8/QZ+LFk4FhxYhRDqgE8BlwGnAm8KIZw652LXAIMxxpOBjwN/WfzeHPAV4H/GGH8FuABYlmlpe3p6luNqjylZ7GSxkyWZFwfIkpYsdp4sp7X7mDsK/FiycJTziNVZwJ4Y4xMxxjzwVeDyOZe5HPhS8eNbgJ0hhABcDPxnjPE/AGKM/THGZXmCs6WlZTmu9piSxU4WO1mSeXGALGnJYufJcnDCz9k+XixZOMq5hRcAT8/6fH/xa+ZlYoxTwBDQCZwCxBDCnSGE3SGE9yydrJRSSinls3KebAzG1+ae/ZV2mRxwLnAm8AzwbyGE+2OM/zb7gocOHeKaa64hl8tRKBS44ooruO666+ju7qa5uZm6ujqGh4fp6upiYGCAGCNdXV309PSU/jo4cOAAzc3N9Pb2EkKgo6OD3t5e2traKBQKjI2NsXnzZrq7u6mvr6e9vZ2+vj7a29vJ5/OMj4+Xljc0NNDa2kp/fz8bNmxgfHyciYmJ0vJ169bR2NjI4OAgnZ2djIyMkM/nS8vHxsaoq6tjaGiIjRs3MjQ0xOTkZGl5ues0OjrKpk2blrROhUKB0dHRJa9TY2MjDQ0NS1qn3t7eiqxTJfbTgQMHKBQKS16nSuyn7u5uWlpaKnLsLXU/jY+PV+zYW8p+6u3tLW2TShx7S1mniYkJCoVCRe8jjnWd8vk8o6OjFb2PONZ16u7uZnR0tKL3Ece6TgcOHKCurq7i9+XHsk6Dg4M0NzdX/L587jqdvzHP3rE6OhumWV8fuf9wjjPWT3F4MtCfX8NJzQVa6iItuUhr7tnl/fk1jEwFTmyaud1K7KftHZM8PlrHS9qm2D++hvo1sGntdOk2nykEuhqmOW7dNE8+U0drLtLZ8OzykanA6OhoRfbT89ZOc0JTgaa6Z9e558gaJqdn5rBqrIt0NERyIfLQcI7T2qdKj2Idt26aB4dyPP300wvup3kHTQu9giGEcA7wgRjjJcXPbwCIMX5k1mXuLF5mV/G8qm6gC3gjcGmM8e3Fy70fmIgx/p/Zt7Fr1664bdu2eR0LNTExwbp1y3Je/KKTxU4WO1n8OkCWtGSxy8py8eceWPAy7fXTDE2mPzH13WtPX3WWhRzlWnbv3n3/zp07X2ktK+epwB8DW0MILwohNABXAbfNucxtwNuKH78e+F6cGbHdCbwshNBUHHCdD/ysjNtcdL29vctxtceULHay2MmSzIsDZElLFjtPlpe2+ThhHPxYsnAs+FRgjHEqhHA9M4OkOuALMcaHQwgfBH4SY7wN+Dzw5RDCHmCAmcEXMcbBEMJfMzM4i8DtMcbvLMeKzJwr7yNZ7GSxkyWZFwfIkpYsdp4sU1GWuWXhKGtChxjj7cDtc75246yPJ4ArU773K8xMubCsdXR0LPdNlJ0sdrLYyZLMiwNkSSsrSzlPM3U2TNOfn/+Roko81VQJS6We8iqnx0f9vCG0F0sWDh+vf6xAnh5+lcVOFjtZknlxgCxpebK8xMnTTCBLWl4sWThqZmDV1tZWbUIpWexksZMlmRcHyJKWJ8v+cT+/ymSx82LJwuFjTStQoeDjDR5BlrRksZMlmRcHyJKWJ0u9o99ksth5sWThcLKqS29sbKzahFKy2MliJ0syLw6QJS1Plk1rp6tNKCWLnRdLFo6aGVht3ry52oRSstjJYidLMi8OkCUtT5b7D/t4g1+QJS0vliwcNTOwWmgm1CyTxU4WO1mSeXGALGl5spyx3seJ0SBLWl4sWThqZmBVX19fbUIpWexksZMlmRcHyJKWJ8szBR9zJIEsaXmxZOHw8dhcBWpvb682oZQsdrLYyZLMiwNWp6Wc+Zqet3aaQ0d65r1MVnM27XvGxxxJIEtaXixZOGrmEau+vr5qE0rJYieLnSzJvDhAlrR+udXHUzsgS1qyJMvCUTMDq9X4V2U5yWIni50XixcHyJLWk04egQBZ0pIlWRaOmhlY5fP5ahNKyWIni50sybw4QJa0WnOx2oRSstjtZj39AAAgAElEQVTJkiwLR80MrMbHx6tNKCWLnSx2siTz4gBZ0ups8DEvEciSlizJsnDUzMDK05wqstjJYidLMi8OkCUtL/MSgSxpyZJM81gtIk9zqshiJ4udLMm8OECWtLzMSwSypCVLMs1jtYgaGhqqTSgli50sdrIk8+IAWdIamfIxLxHIkpYsybJw1MzAqrW1tdqEUrLYyWInSzIvDpAlrYPjfn59yGInS7IsHD7WtAL19/dXm1BKFjtZ7GRJ5sUBsqR1Smuh2oRSstjJkiwLR80MrDZs2FBtQilZ7GSxkyWZFwfIktbeMR/zEoEsacmSLAtHzQysPL0MWRY7WexkSebFAbKk5eXl8yBLWrIk03QLi2hiYqLahFKy2MliJ0syLw6QJa319T4mfARZ0pIlWRaOmhlYeZrfRRY7WexkSebFAbKk5WVeIpAlLVmSaR6rReRpfhdZ7GSxkyWZFwfIkpaXeYlAlrRkSaZ5rBbRunXrqk0oJYudLHayJPPiAFnSOjzpY14ikCUtWZJl4aiZgVVjY2O1CaVksZPFTpZkXhwgS1r9eT+/PmSxkyVZFg4fa1qBBgcHq00oJYudLHayJPPiAFnSOqnZx7xEIEtasiTLwlEzA6vOzs5qE0rJYieLnSzJvDhAlrQeG/ExLxHIkpYsybJw1MzAamRkpNqEUrLYyWInSzIvDpAlreMafcxLBLKkJUuyLBw1M7DK5/PVJpSSxU4WO1mSeXGALGm15nzMSwSypCVLsiwcNTOw8jS/iyx2stjJksyLA2RJy8u8RCBLWrIk0zxWi8jT/C6y2MliJ0syLw6QJS0v8xKBLGnJkkzzWC0iTy9DlsVOFjtZknlxgCxpeXn5PMiSlizJ3Ey3EEK4NITwaAhhTwjhvcbytSGErxWX3xdCOHHO8uNDCKMhhD+uDDtZQ0PDcl31opPFThY7WZJ5cYAsaY1M+ZjwEWRJS5ZkWTgWHFiFEOqATwGXAacCbwohnDrnYtcAgzHGk4GPA385Z/nHgTuWzk1vaGhoOa9+UcliJ4udLMm8OECWtE5s8jEvEciSlizJsnCU84jVWcCeGOMTMcY88FXg8jmXuRz4UvHjW4CdIYQAEEJ4LfAE8HBlyHYbN25czqtfVLLYyWInSzIvDpAlrUdGfJyMDLKkJUuyLBzlDKxeADw96/P9xa+Zl4kxTgFDQGcIoRn4X8CfLZ06f57+kpPFThY7WZJ5cYAsaZ3g5BEIkCUtWZJl4Shn6GY9ITl3Ioi0y/wZ8PEY42jxASyzQ4cOcc0115DL5SgUClxxxRVcd911dHd309zcTF1dHcPDw3R1dTEwMECMka6uLnp6emhpaQGgr6+Pjo4Oent7CSGUPm5ra6NQKDA2NsbmzZvp7u6mvr6e9vZ2+vr6aG9vJ5/PMz4+Xlre0NBAa2sr/f39bNiwgfHxcSYmJkrL161bR2NjI4ODg3R2djIyMkI+ny8tHxsbo6mpiaGhITZu3MjQ0BCTk5Ol5eWu0+joKJs2bVrSOhUKBfbt27fkdWpsbKShoWFJ6zQ8PMzk5OSS16kS+6mvr49cLrfkdarEfurr66Ozs7Mix95S99P4+Dhr166tyLG3lP00e5tU4thbyn6amJjg0KFDFb2PONZ1yufz7Nu3r6L3EdY6bWks0Nkwzfr6yP2Hc5yxforDk4H+/BpOai7w2EgdJzVP0VT37PL+/BpGpgInNhV4ZCTHCU0z9z1L3U+dDdO8pG2K/eNrqF8Dm9ZOl27zmUJg3zN1/HLrFIeOrKE1F+lseHb5yFTg4PgaTmktMDw8vOT9dOaGyeesc8+RNUxOw5bGaX46nGNrS4GtzVM8NlrHae1THJyYeeziuHXTPDiU46VtU+zfv78i++n8jXn2jtXNu59a6iKvWD9Ja87eT2NjYxX5edreMcnjo3Xz7qeuhmnO35jnyWfqzP00OjpakZ+n562d5oSmQup+aqyLbO+YJBciDw3nzP309NNPL/jzNO+gKcb5J8sKIZwDfCDGeEnx8xsAYowfmXWZO4uX2RVCyAHdQBdwD/DC4sXWA9PAjTHGm2bfxq5du+K2bdvmdSzUkSNHWLt27ZKuo1LJYieLnSx+HbA6LRd/7oEFL9OSm2Z0av4nPb577emyLINDlmO3VOpY2b179/07d+58pbWsnKcCfwxsDSG8KITQAFwF3DbnMrcBbyt+/Hrge3GmV8cYT4wxngj8DfDncwdVlcrT/C6y2MliJ0syLw6QJS0v8xKBLGnJkiwLx4JPBcYYp0II1wN3AnXAF2KMD4cQPgj8JMZ4G/B54MshhD3AADODr0xrbm7O+iZTk8VOFjtZknlxgCxp9RzxMS8RyJKWLMmycJR1enyM8Xbg9jlfu3HWxxPAlQtcxweOwVd2dXU+3jkbZElLFjtZknlxgCxpTfp4T11AlrRkSZaFw8cQsgINDw9Xm1BKFjtZ7GRJ5sUBsqS1pdHJb0pkSUuWZFk4amZg1dXVVW1CKVnsZLGTJZkXB8iS1k+HfcxLBLKkJUuyLBw1M7AaGBioNqGULHay2MmSzIsDZElra4uPeYlAlrRkSZaFw8cQsgItNG1ElsliJ4udLMm8OCA7SzkvWT+3M8+9/9w772Uq9bL1hcoFP/tIFjtZkmXhqJlHrDw9RC6LnSx2siTz4gBfloecPJ0CsqQli50XSxaOmhlY9fT0VJtQShY7WexkSebFAb4sp7X7mAsIZElLFjsvliwcNTOwOjrNvodksZPFTpZkXhzgy3L0rTc8JIudLHZeLFk4fKypUkoppVQNVDMDq9HR0WoTSsliJ4udLMm8OMCX5bh1PuYCAlnSksXOiyULR80MrDZt2lRtQilZ7GSxkyWZFwf4sjw45OMEYJAlLVnsvFiycNTMwKq3d/6XIGeZLHay2MmSzIsDfFle2ubjBGCQJS1Z7LxYsnDUzMAqhFBtQilZ7GSxkyWZFwf4skxFWaxksZMlWRaOmhlYdXR0VJtQShY7WexkSebFAb4sj4/6eRNmWexksfNiycJRMwMrTw/Xy2Ini50sybw4wJflJU6eTgFZ0pLFzoslC0fNDKza2tqqTSgli50sdrIk8+IAX5b9437usmWxk8XOiyULh481rUCFgo83eARZ0pLFTpZkXhzgy1Lv6B5bFjtZ7LxYsnD4eP1jBRobG2Pjxo3VZgCypCWLnSzVdSz05sfnb8xzd9/T814mqzc+3rR2mp+PZHJTCyaLnSx2XixZOJyMIZfe5s2bq00oJYudLHayJPPiALj/sJ+/P2Wxk8VOlmRZOGpmYNXd3V1tQilZ7GSxkyWZFwfAGet9nHQLsqQli50sybJw1MzAqr6+vtqEUrLYyWInSzIvDoBnCj7m3wFZ0pLFTpZkWThqZmDV3t5ebUIpWexksZMlmRcHwL5nfMy/A7KkJYudLMmycNTMwKqvr6/ahFKy2MliJ0syLw6AX2718RQGyJKWLHayJMvCUTMDK09/4cpiJ4udLMm8OACedPKXNsiSlix2siTLwlEzA6t8Pl9tQilZ7GSxkyWZFwdAay5Wm1BKFjtZ7GRJloWjZgZW4+Pj1SaUksVOFjtZknlxAHQ2TFebUEoWO1nsZEmWhaNmBlae5r2RxU4WO1mSeXGAn/l3QJa0ZLGTJZnmsVpEnua9kcVOFjtZknlxgJ/5d0CWtGSxkyWZ5rFaRA0NDdUmlJLFThY7WZJ5cQCMTPmYfwdkSUsWO1mSZeGomYFVa2trtQmlZLGTxU6WZF4cAAfH/dxNymIni50sybJw+FjTCtTf319tQilZ7GSxkyWZFwfAKa2FahNKyWIni50sybJw1MzAasOGDdUmlJLFThY7WZJ5cQDsHfMx/w7IkpYsdrIky8JRMwMrTy/PlsVOFjtZknlxgJ+XiYMsacliJ0syN9MthBAuDSE8GkLYE0J4r7F8bQjha8Xl94UQTix+/VdDCPeHEB4q/n9RZfnPNjExsVxXvehksZPFTpZkXhwA6+t9TGwIsqQli50sybJwLDiwCiHUAZ8CLgNOBd4UQjh1zsWuAQZjjCcDHwf+svj1PuA3Y4wvBd4GfLlS8Ll5mvdGFjtZ7GRJ5sUBfubfAVnSksVOlmRe5rE6C9gTY3wixpgHvgpcPucylwNfKn58C7AzhBBijA/EGH9R/PrDwLoQwtpKwOfmad4bWexksZMlmRcH+Jl/B2RJSxY7WZJl4Shn6PYC4OlZn+8Hzk67TIxxKoQwBHQy84jV0X4LeCDGeGTuDRw6dIhrrrmGXC5HoVDgiiuu4LrrrqO7u5vm5mbq6uoYHh6mq6uLgYEBYox0dXXR09NDS0sLAGNjY0xMTNDb20sIgY6ODnp7e2lra6NQKDA2NsbmzZvp7u6mvr6e9vZ2+vr6aG9vJ5/PMz4+Xlre0NBAa2sr/f39bNiwgfHxcSYmJkrL161bR2NjI4ODg3R2djIyMkI+ny8tz+fzHD58mKGhITZu3MjQ0BCTk5Ol5eWu0+joKJs2bVrSOuVyOfbt27fkdWpsbKShoWFJ6zQ1NcW+ffuWvE6V2E9jY2McOnRoyetUif00NjbGkSNHKnLsLXU/TU1N0d/fX5Fjb7799LldT3L/4RxnrJ/i8GSgP7+Gk5oLPDZSx3GN05zcPMXt3+/jjPVT9OfXMDIVOLGpwCMjOU5oKtBUF/njy06ryH46qXmK+jWwae10yfRMIbDvmTp+uXWKuhB5SdsUnQ3PLh+ZChwcX8MprQX2jtXR09Oz5P10/sZ86fp7jqxhchq2NE7z0+EcW1sK5ELkyDScvzHPwYmZv4mPWzfNg0M5Xto2xVQMPD5ax759+5a8n7Y0FuhsmGZ9fUzdT+31088xW/tp3759S/556myY5iVtU+wfX5O6nzatLXBCU4HWXEzdT8PDw0v+eTpzwyRNdXHe/dTVME17/TSntU+Z+2n//v0V+Xk6f2OevWN18+6nGCOvWD9Jay6a+2lsbKwi93vbOyZ5fLRu3v1UiJHzN+Z58pk6cz+Njo5W5H7veWunS/cR1n6amobtHZPkQuSh4Zy5n55++ukF7/fmK8Q4//ONIYQrgUtijNcWP38rcFaM8V2zLvNw8TL7i5/vLV6mv/j5rwC3ARfHGPfOvY1du3bFbdu2zetYqOHhYdra2pZ0HZVKFjtZ7LKyXPy5Bxa8zJbGAvvH0181891rT8/EspCjFi2V2D+y+LdkddzKcmyOci27d+++f+fOna+0lpXzVOB+4IWzXcAv0i4TQsgB7cBA8fMtwDeB37EGVZVqcHBwua560cliJ4udJ8tJzT7mmvHiAFnSksVOFjsvliwc5QysfgxsDSG8KITQAFzFzKNPs7uNmZPTAV4PfC/GGEMI64HvADfEGH9YKbRVZ2fncl79opLFThY7T5bHRnzMNePFAbKkJYudLHZeLFk4FhxYxRingOuBO4FHgK/HGB8OIXwwhPCa4sU+D3SGEPYAfwQcnZLheuBk4P0hhAeL/55X8bUARkZGluNqjylZ7GSx82Q5rtHHXDNeHCBLWrLYyWLnxZKFo6zXHcYYbwdun/O1G2d9PAFcaXzfh4EPL9FYVvl8PoubKStZ7GSx82RpzfmYa8aLA2RJSxY7Wey8WLJw1MzM657mvZHFThY7T5bVNNdMucliJ4udLHZeLF7msVoReZr3RhY7Wew8WVbTXDPlJoudLHay2HmxZOGomYFVY2NjtQmlZLGTxc6TpT/v4y7BiwNkSUsWO1nsvFiycPhY0wrU0NBQbUIpWexksfNkGZkK1SYAfhwgS1qy2Mli58WShaNmBlZDQ0PVJpSSxU4WO0+WE5t8zDXjxQGypCWLnSx2XixZOGpmYLVx48ZqE0rJYieLnSfLIyM+TjD14gBZ0pLFThY7L5YsHDUzsPL0V78sdrLYebKc4OSvSi8OkCUtWexksfNiycJRMwOrycnJahNKyWIni50nS1Odj7lmvDhAlrRksZPFzoslC0fNDKw8zQUki50sdp4sq2mumXKTxU4WO1nsvFg0j9Ui8jQXkCx2sth5sqymuWbKTRY7WexksfNi0TxWi6i5ubnahFKy2Mli58nSc8THXYIXB8iSlix2sth5sWTh8LGmFaiuzsc7Z4Msacli58ky6eN9Ut04QJa0ZLGTxc6LJQtHzQyshoeHq00oJYudLHaeLFucvAO9FwfIkpYsdrLYebFk4aiZgVVXV1e1CaVksZPFzpPlp8M+TjD14gBZ0pLFThY7L5YsHD7WtAINDAzQ1NRUbQYgS1qr0XLx5x5Y8DLbOyb50UB96vLvXnt6JUnztrWlQP9A9f/e8uIAWdKSxU4WOy+WLBzVX8sKFaOPOTJAlrRkscsFWebmxQGypCWLnSx2XixZOGpmYOXp6RRZ7GSxe8jJQ+Tgx+LFAbKkJYudLHZeLFk4amZg1dPTU21CKVnsZLE7rd3H/C7gx+LFAbKkJYudLHZeLFk4amZg1dLSUm1CKVnsZLE7OOHnx9CLxYsDZElLFjtZ7LxYsnD4WFOllFJKqRqoZgZWo6Oj1SaUksVOFrvj1vmY3wX8WLw4QJa0ZLGTxc6LJQtHzQysNm3aVG1CKVnsZLF7cMjHSZ3gx+LFAbKkJYudLHZeLFk4amZg1dvbW21CKVnsZLF7aZuPkzrBj8WLA2RJSxY7Wey8WLJw+BhCVqAQQrUJpWSxy8pS9qScd/alLs9yUs6p6GcfebF4cYAsacliJ4udF0sWjpp5xKqjo6PahFKy2HmyPD7q542PZUnmxQGypCWLnSx2XixZOGpmYOXpqR1Z7DxZXuLkYWmQxcqLA2RJSxY7Wey8WLJw1MxTgW1tbdUmlFqNlnKefjupeYq9/zIw72Wyegpu/7ifvylkSebFAbKkJYudLHZeLFk4fKxpBSoUCtUmlJLFrt7R0SaLnReLFwfIkpYsdrLYebFk4XCyqktvbGys2oRSsthtWutjHhOQJS0vFi8OkCUtWexksfNiycJRM08Fbt68udqEUllZynn6rSU3zeiUj6ff7j/s53CTxc6LxYsDZElLFjtZ7LxYsnDUzCNW3d3d1SaU8mQ5Y72PEwZBlrRkSebFAbKkJYudLHZeLFk4yhpYhRAuDSE8GkLYE0J4r7F8bQjha8Xl94UQTpy17Ibi1x8NIVxSOfpzu/XWW5frqhedJ8tPvn9HtQmlZLGTJZkXB8iSlix2sth5sWThWPAxsRBCHfAp4FeB/cCPQwi3xRh/Nuti1wCDMcaTQwhXAX8JvDGEcCpwFfArwPOBfw0hnBJjXPQZ1Qs97fXYF/6e7zSfN+9lKvGUVzlPv2VlKacH7rqDU156eSa3tVCy2Mni1wGypCWLnSx2XixZOMp5xOosYE+M8YkYYx74KjBXdTnwpeLHtwA7w8w025cDX40xHokx/hewp3h9Fa/R0ZOastjJYidLMi8OkCUtWexksfNiycIRYozzXyCE1wOXxhivLX7+VuDsGOP1sy7z0+Jl9hc/3wucDXwA+FGM8SvFr38euCPGeMvs27j99ttHDh48WFrdtra23o6OjvT3GzEaGBjYuNjvWa5ksZPFTha/DpAlLVnsZLHzYqmg44SdO3d2WQvKOT3eemOduaOxtMuU87382q/9WmsZDqWUUkop15XzoNh+4IWzPt8C/CLtMiGEHNAODJT5vUoppZRSNVE5A6sfA1tDCC8KITQwczL6bXMucxvwtuLHrwe+F2eeY7wNuKr4qsEXAVuB/1cZulJKKaWUrxZ8KjDGOBVCuB64E6gDvhBjfDiE8EHgJzHG24DPA18OIexh5pGqq4rf+3AI4evAz4Ap4LpjeUWgUkoppdRKaMGT11dKIYSOGOP8U4yvgkIIm4AXMHMu2y9ijD1VJrlI20UdayGE1xT/gFRqxeTpd+Jq+xnyMcf8Igsh7AA+B0wD/x34MHBSCKEeeEOMcVdGjl8C3sfMeWN/AXwcOAd4BHh3jPHJLBxFy2nAp5k5v+1A8ctbQgiHgd+LMe7O0PKPwD8Ct8YYR7O63RSLp+1SB1zLzLmG/xxj/OGsZe+LMX44K8t8hRA+G2P83Qxv73pmpmXpCyGcDHwBeBnwKHBtjPGhDC1XzP0S8KniuaPEGP8xK8vcQgiPxRhPqcLtrgHeDvwWM8fuFPA48OkY411Ze9KqwnHbBFzPzB9rn2TmmZorgJ8DH8zyvm/2/Udx/shbgfritEdvjDHel6HFzc9Qte5zV+QjViGE/8fMpKQtwD8Br40x3htCeAXwyRjjjowc9wD/wMwv7bcAXwS+DlwM/HaM8aIsHEXLg8A75v4AhRC2A5+JMb48Q8sBYBdwEfCvzGyj7xTnQcs0Z9vlc0ATM+cZvhW4O8b4R8Vlu2OMr8jQ0pG2CPiPGOOWDC0Pxxh/pfjxd4DPxRi/GUK4APjfWf08F29/Cvhn4BDPvqr59czMzxdjjP89I8cIz76C+qijCXim6GjLwlG0fBHYx8zP8uuBYeAHwP8CvhVj/GSGFk/H7deBp4FG4MXM/EH9deA3gc0xxrdmaCndfxR/hm6KMd4RQjgL+JsY46sytLj4GSpaqnOfG2Nccf+AB2Z9/MicZbur5HgqbVlGlsfnWbanGvsHaC0ezLcDvcwMPC9exdvlP2d9nAM+y8wje2urcLwUgCeA/5r17+jn+Ywtj876+Mdp2ywjy5nAvwHv5Nk/PP8rS0PxNj8J/B2wadbXMndY+4CZuQkpHrePZGzxdNw+WPw/AN2zjpdQheN296yPH5izLOv7Fhc/Q8Xbrcp9rpO5UBfdbPcNc5Y1ZOiYDiGcEkI4E2gKIbwSoPh0Rl2GDoA7QgjfCSG8MYTwquK/Nxb/evnnjC0RIMY4EmP8cozx15j5i+4+IPFek8ucp+1SOjZjjFNx5mmLB4HvMfPoa5Y9AVwQY3zRrH+/FGN8EZD1+We3hBBuLj61/s0Qwv8XQjg+hHA18FSWkBjjj5l5+64G4HvFv/gzf1g/xvgu4BPAP4QQfr/4dFy1nl6YDCGcBFB8ViBfNB6pgsnTcQvMPAQD3F78/+jnWW+XXwoh3BZC+CdmTnVomrWsPkuIl5+hYlW5z12R51gB7w8hNMUYn4kxlt7xuPjD/3cZOt7DzFOR08BrgRtCCC8H2oD/kaGDGOPvhxAuY+ZthF7AzF9N+4FPxRhvz9ICJM4tiDMnUX66+C+znG2Xn4QQLo0xlgZ0McYPhhB+AfzfjC1/A2zAHrh8NEtIjPFPioOofwBOYuavyd9l5jyR387SUvRMA58IIXyDme1UlWKM94cQ/hsz5/HcDayrEuXdwPdDCBPM/JK+CiCE0AV8O2OLm+OWmZ/nlhjjaJz19Fbx99BIxpa5bzO3pmjZRPb3LW5+hqjSfe6KPMfKSyGE+hjj5JyvbWTmDalX7bQS1nZRvraLLL6bu01CCMcBp1fhjwGKLwqaAjpjjFV9SxJPx8p8lhBCiPrlWvWqdbys1KcCUwshfDbDmzsQQvjbEMKFxVdfEGPs8zaoynibwLPb5aKj28Vjq3y7eLRc6MCSWpXuWy4q/pI+WI1B1VELM+emvNTB/vF43CYsngZVVbifS2213OeuyEesvLwyJITQycyrHa5iZlb5W4B/iBm+tHWWxcU2KVq0XWyLp+0ii21xcbw42yay+Le4OG4dWqqyj1bqwKrAzMt/Z49Aj77p8wtijFmewH7U9HzgSmZ24POYmZfnTzK8fXfbpOjSdrFdVd0usqTevrvjpdrbRBb/Fk/HrSfLHFdm+2ilDqweB3bGGBMnMIYQno4xvtD4tmUvhNDCzARxfwQcF2PclOFtu9wmxdvXdjGq5naRJfW2XR4v2j+yLHDbbo5bTxbj9jPZRyv1HKujrwyxyvSVISGEdSGEK8PMbON7gZ3MTAHx/CwdONomoO2SlqPtIoudm+PF0TaRxb/FzXGLL0tV9tFKfcTKxStDQgh/D/w34B7gq8C3Y4wTVbK42Cag7ZKWs+0ii21xcbw42yay+Le4OG7BnaU6+yhWYTbUpf5jZqr8v2XmLVNCFR1vA1rLudxq2SbaLitmu8ji+Hhxtk1k8W9xcdw6tFRlH63UR6zcvBqjnEIG7wO30rYJaLuklcV2KbfVZllpx8tq2z/lttosno5bT5Zyq/Q+WpHnWMUY+2OMn4kxXgicxcz7RP1NCGFvCOF/V5lntezzZ6zAbQLaLmlVe36e2a0qywo8XlbV/llEq8ri6bj1ZFlEFd1HK3JgNbsY4y+AzzMzPf0IcG11RWaZPiy4QrYJaLuk5elh5FVrWSHHy6rdPwu0ai2ejltPlgWq6D5asQMrR6/GKKdM/npaYdsEtF3SWlV/bS8iHS/JVt3+KbNVZ/F03HqylFlF99GKfBPmOWf6/z3w5lilV2OU2Q+X+wZW4DYBbZe0ln27LKJVZVmBx8uq2j+LaFVZPB23niyLqLL7qJpn7K+0M/1TbmMbM6Pxljlfv3S1bhNtF//bRRb/x4uXbeLNMo/x6mobqmVxdty6sVRrH1VtpTLacLuX+fp/H3gUuBV4Erg8q9v2uk20XVbGdpHF9/HiaZt4sizgfKraBo+WOS5P+6uqluXcRyvyqcBFtNzPbf8P4IwY42gI4UTglhDCiTHGT2Rw28daFi5tFztP20WWpbWa7lvcWEII/5m2CMj07Ww8WRaRp5+nZbdUax/V+sBquV+NURdjHAWIMT4ZQriAmTudE/B1AM8ui1eoaLvYedousiyt1XTf4smyCbgEGJzz9QD8+yq2lNtqe7VkVfbRin1VYJkt9w99dwjhtKOfFO98fgPYCLx0mW/7WMvijlDbxc7TdpFlaa2m+xZPlm8zc57Xvjn/ngTuWsWWcvP0h0oWlqrso1ofWC33qzGOBw7O/kKMcSrG+DvAect828daFq+W0Xax87RdZFlaq+m+xZPl9BjjvdaCGOObV7Gl3FbVqyWp0j6quYFVCOHqox/HGK9f5psbjkHG56EAAAJMSURBVDH2WAtijG4O4Iy3CTjbLiGEbSGEnSGEljlfv3SWa7VtF1lScnK8eNomniwqJSfH7bxV4XdRVVqR7xU4XyGEp2KMx2d0W/uBv05bHmNMXZZlWW6T4u252S4hhN8HrgMeAU4D/iDG+K3iskzfT8zZdpHFtrg4XpxtE1n8W1wctwu1Wn4XrciT1x29GqMOaMHB89aOtgk42i44ekUTvraLLHZejhdP20QWO08WL8etfhexQh+xCiH0MM+Z/jHGTKbNd/aXgIttUrR42i4/izGeOuvzFmbebf1nwEUxxtNSv7nyFk/bRRYjL8eLs20ii5Ezi4vjtnjbq/530Uo9x8rLqzE8/KVyNC/bBHxtF0+vaPK0XWSx83K8eNomsth5sng5bkG/i1bsI1Yu/lIIIXTEGAeq7QA/2wTcbZc9wA7r5NsQwo4sT751tl1kMfJyvDjbJrIYObO4OG6Lt7fqfxet1EesXOTlh8pbzraLm1c0edousqTm4njxtE1ksfNkwclx661q7aMVefI68LwQwh+lLfTyaryM0zax03ZRi0nHi1qJeTpuPVmq0kodWHl6NYaXtE3stF3UYtLxolZino5bT5aqpHOsaiRtEzttF7WYdLyolZin49aTpVqt1HOsVu1IeJ60Tey0XdRi0vGiVmKejltPlqq0Uh+xcvNqDC9pm9hpu6jFpONFrcQ8HbeeLNVqRQ6slFJKKaU8tlKfClRKKaWUcpcGVkoppZRSFUoDK6WUUkqpCqWBlVJKKaVUhdLASimllFKqQv3/XHY3Kv4o2M4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "corrs = [data[col].corr(data['Hazard']) for col in numerical]\n",
    "corrs = pd.DataFrame(np.abs(corrs), numerical, \n",
    "    ['abs corr with target']).sort_values('abs corr with target')\n",
    "corrs.plot(kind='bar', figsize=(10, 4))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlYAAAEQCAYAAAByTDrzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XuUnHWZ6Pvv0119v6VvdBiRBLkYGS9gEAjhIkS5zKhxUBx0Dyo7ODgn6Jn7lrUddJjZenSdpeOoc8QtiqNLkWEJBgcGZ0AIl8AGAgqIQECaAOlO39K3dKe6qn/nj6oums7zdlenbk9XPd+1WHTXW131eS9d+XXVW7+SEAKe53me53le7lWVGuB5nud5nlcu+cDK8zzP8zwvT/nAyvM8z/M8L0/5wMrzPM/zPC9P+cDK8zzP8zwvT/nAyvM8z/M8L0/FSg0AuOuuu0JdXV2pGZ7neZ7neUu2f//+wU2bNnVry0wMrOrq6li3bl1Ot9Hb28uaNWvyJMott+i5Rc8telYsVhzglqjcoucWvXxYdu7c2Ru1rGxeCqypqSk1IZNb9Nyi5xY9KxYrDnBLVG7Rc4teoS1lM7Bqa2srNSGTW/TcoucWPSsWKw5wS1Ru0XOLXqEtZTOwGhwcLDUhk1v03KLnFj0rFisOcEtUbtFzi16hLSbOsdIKITAxMUG2n2XY1NTE2NhYgVXZ5RY9S5aGhgZCCIhIqSkV9ZfccrJiseIAt0TlFj236BXaYnZgNTExQV1dHbW1tVldf2ZmxsxruG7Rs2SJxWJMTEzQ0tJSagrxeLzUhExuOTgrDnBLVG7Rc4teoS1mXwoMIWQ9qAKYnZ0toGZ5uUXPkqW6ujrrZ0ML3dTUVKkJmdxycFYc4Jao3KLnFr1CW8wOrJablWdCwC1RuUVv9erVpSZkcsvBWXGAW6Jyi55b9AptKZuB1czMTNHu6/Wvf/2iy4tpWao5y6OPPspnPvMZAO69914efPDBzHW2bt3Kz372syVva6n1ztbyox/9iD179uR0W9k0OjrKtddeu6jFQn19faUmZHLLwVlxgFuicoueW/QKbTF7jtXCzv3Oo3m9vV9cdmJeb29+hTghOplMUl1dnfk+kUgQiy29++YsJ554IieemFrne++9l6amJk455ZS8O7Ox/PjHP+ZNb3oThx9+eNY/m+36zm9uYLVly5ZIi4WW85J3oXPLwVlxgFuicoteJVqyGSu8fdUMO/9zOHJ5ruODsnnGqhD9yZ/8CWeffTYbNmzguuuue82yz372s7zzne/k/e9/f+atm9dccw2nnnoq55xzjvqPeTKZ5O/+7u/YuHEjp59+Ot/+9rcBuPvuuznrrLPYuHEjV1xxBQcOHADgbW97G1/+8pe54IILuPnmm3nve9/LP/zDP/Ce97yHb33rW6+57Y0bNzI6OkoIgaOPPprrr78egE996lPcdddd3HvvvVx88cW8+OKLXHfddXzrW9/izDPPZMeOHQDs2LGD8847jxNPPHHJZ69CCFx11VWcdtppbNy4kZ/+9KdA6hyqv/7rv2bDhg1cfPHFfOhDH3rNbVVXV/Ozn/2Mxx57jMsvv5wzzzyTqakpvvzlL7Np0yZOO+00/vzP/zxz7tPC9f3d737Hu9/9bjZt2sQXvvCF1zyD9s///M9s2rSJ008/nS9+8YsA/P3f/z0vvPACZ555JlddddVr1mH+ILXUWTiBfi63HJwVB7glKrfouUVvz1Rhhz4+sFqkr3/96/zyl7/kzjvv5Nvf/jbDw6kR7uTkJG9729u46667OO200/jyl78MwNe+9jXuvvtu7rjjDr7yla8cdHvf//736e3t5e677+bee+/loosuYnp6mq1bt3Lttddy3333kUwm+e53v5v5mfr6em677TY+8IEPAKlnYX7+859zxRVXvOa2TznlFB588EGeeuop1q5dmxkwPfzww5x00kmZ6x155JF8/OMf55Of/CTbt29nw4YNQOqp0dtuu40f//jHXH311Ytul1tuuYXHH3+ce+65h5tuuonPfe5z9PX1ccstt/Diiy9y33338bWvfY2HHnroNT+XSCTYvHkzJ5xwAtdccw3bt2+noaGBT3ziE9xxxx3cf//9TE9Pc/vtt2d+Zv76XnnllXzyk5/kjjvueM2zXXfeeSfPP/88//Vf/8X27dv51a9+xf3338/nPvc51q5dy/bt2w9ap0Qiseg6FrOhoaFSEzK55eCsOMAtUblFzy16x7UkC3r7WQ2sROR8EXlaRHaJyGeU5WeKyE4RSYjIB+ddfoKI7BCRJ0Xk1yLyx/nEF7prrrmGM844g3PPPZeXX36Z5557DoCqqir+6I/+CIAPfehDPPDAAwAcf/zx/Omf/ik33XST+ozIXXfdxaWXXpp5Sau9vZ1du3axZs0ajjnmGAAuvvjizKAIyNxP1Pdzbdiwgfvvv58dO3Zw6aWX8pvf/IZXXnmF9vZ2mpubl1zXP/zDP6Sqqop169YxMDCw6HUfeOABPvCBD1BdXc1hhx3Gxo0befTRR3nggQfYvHkzVVVV9PT0cMYZZ7zm56Jeyrvnnnt417vexcaNG7nnnnv47W9/q67vQw89xObNmwEyA02AX/7yl/zyl7/krLPO4p3vfCfPPvtsZl9FtdyXFQtZe3t7qQmZ3HJwVhzglqjcoucWvecmC/uKxZIDKxGpBr4JXAAcD3xYRI5fcLUXgY8DP1pw+X7goyGE3wfOB/5JRFblii5G9957L3fffTe3334799xzD29961szL9EtbO58nZ/85CdcdtllPPbYY5xzzjnqsyILz+1Z6i3/jY2Ni34/14YNG9ixYwc7duzg9NNPp6uri23btmV9HtX817+XMkUtX+rntOkWpqen+Zu/+Ruuu+467rvvPi655BKmp6czy6PWd+H9/sVf/AXbt29n+/btPPLII1xyySXLtpSqSnob8nKyYrHiALdE5RY9t+h11hb28T+bZ6xOBnaFEJ4PIcSB64HN868QQnghhPBrYHbB5c+EEJ5Nf/0KsBfozou8wI2NjbFq1SoaGxt55plnePjhhzPLZmdnM+cO3XjjjZx66qnMzs7y8ssvc8YZZ/DZz36W0dFRJicnX3ObZ599Nt/73vcyA66RkRGOPfZYXnzxRZ5//nkAbrjhBk477bRle4844giGh4d5/vnnWbt2Laeeeirf+MY3OPnkkw+6bnNzMxMTE8u+j7lOO+00brrpJpLJJIODg9x///28/e1v59RTT+WWW25hdnaWvXv3cu+9977m5+YGM/Pvf26w2tnZycTEBNu2bYu835NOOimzfO68LoBzzjmHH/7wh5nbfOWVVxgYGFh0PS0NrOYPJEudWw7OigPcEpVb9Nyit6qmsHMYZvN6yOuA3fO+fwlY9tvJRORkoBZY/DUaI23atInvfe97nH766RxzzDGvOU+pqamJ3/72t5x99tm0trZy7bXXkkwmufzyyxkbGyOEwJ/92Z8dNG3+JZdcwq5duzj99NOpqanhox/9KJ/4xCf4xje+waWXXkoikeDEE0/k0ksvPSTz+vXrSSZTrx2feuqpXH311WzcuPGg651//vl8/OMf57bbbuNLX/rSsu/nPe95Dw899BBnnHEGIsLnP/95enp6eN/73sf27ds57bTTOOaYY1i/fj2tra2Zn5ubO+rDH/4wf/VXf0V9fT233347H/3oRzn99NM58sgjM+9c1PrCF77A5Zdfzr/8y7/w7ne/O3Pb55xzDs888wznnXcekNo/11xzDUcddRSnnHIKp512Gu9617tec55VTU1N5DOQxa6S5ndZTlYsVhzglqjcoucWvUf2FfZUEFnq5RsRuQg4L4RwWfr7S4CTQwifUq57HfDzEMKNCy4/HLgL+FgI4YGFP/ezn/0sXHnllcRiMZLJJBdeeCGXXnopjY2NVFVVISIkk8nMckidIzMzM5M5lykej1NfX595Nqi6uppEIpGZYXt2dpaamhpmZmYQkayXx2IxZmdnX7O8qqqKqqqqzPJkMkkIIbN87rrzzfOXZ7tOyWSSmpqanNZp3j7IaZ00s7Z8bGyMtrY2BgYGuOCCC7j11lvp6Oiguro6c71DXafJycnMxxz927/9GzfffDM//OEPD2mdpqenmZiYoLa2ltHRUbq6uhgdHWVmZobVq1fT19dHU1MT1dXVjI2N0d3dzfDwMCEEuru76e/vz5y7NjExQU9PDwMDA4gIHR0dDAwM0NraSjKZZHJyMnObNTU1tLW1MTg4SFtbG/F4nL6+Pt74xjfS19dHbW0tLS0tDA0N0d7eztTUFNPT05mfr6+vp6GhgZGRETo7OxkfHycej2eWNzQ05LROU1NTdHd357xOU1NTmeWHuk5zU4zkuk657qfp6emMO9d1ynU/xeNxYrFY3o69XNapt7eX5ubmvB17uazTc889x5FHHpm3Yy+XdZp7JSKfjxGHuk7xeJzm5ua8PkYc6jpNTk5SW1ub18cIbZ3+5/ZB1jQmaawOPLIvxvpVCfoPVDEzC0c0zPLEWIzzew7Qu7+ax8dinNCWYM906sW7w+tneWw0xlfOOWzJdRoaGnpk06ZNrz7jMn/Mk8XAagPw+RDCeenvrwQIIXxRue51LBhYiUgrqUHVF0MI/6bdx44dO8K6detec9nY2Nhrnu1YKkufQ1fJlve+972Mjo4Sj8f59Kc/zUc+8pG8WXbs2MHf/u3fEkKgra2Nr3/967zhDW84pNuamZlhampqWcdYoerv76enp6fUDMAtlh3glqjcoleJlmzmsXpb2wy/Go3+tyibeax27twZObDK5vmwh4BjReQo4GXgYuAji/9IKhGpBW4C/jVqUJWvqqrszBxRyZZbbrklclmulg0bNnDPPffkdBv5suSzhoaGUhMyueXgrDjALVG5Rc8tekPxEs9jFUJIAFcAtwNPATeEEJ4UkatF5H0AIvIOEXkJuAi4RkSeTP/4h4AzgY+LyGPp/04oxIpYmpfILXpu0RsZGSk1IZNbDs6KA9wSlVv03KJ3dFNh57HK6gyuEMKtwK0LLrtq3tcPAUcoP/dD4Ic5GrPK0rxEbtGzZrFy8npnZ2epCZnccnBWHOCWqNyi5xa9Z8ZLPI9VqRIR4vF41tefOwHcQm7Rs2SZnp4283mB4+PjpSZkcsvBWXGAW6Jyi55b9A5vKOx0O3aeQljQ3BxE2c59MTk5SVNTU4FV2eUWPUuW0dFRjjjioCdZS9Jy/oAodG45OCsOcEtUbtFzi15LrPTzWJUkEVnWhzbW1dVRV1dXQFH2uUXPmsXKM1aW5ndxy8FZcYBbonKLnlv0Cj2PldmXApdbX19fqQmZ3KLnFj236FmxWHGAW6Jyi55b9NavKuybl8pmYGXprZxu0XOLnlv0rFisOMAtUblFzy16JZ9uYaU0/0OES51b9Nyi5xY9KxYrDnBLVG7Rc4veeKKwp4GUzcBqdHS01IRMbtFzi55b9KxYrDjALVG5Rc8temsbC/sO9bIZWHV1dZWakMktem7Rc4ueFYsVB7glKrfouUXvqXE/eT2rLI2G3aLnFj236FmxWHGAW6Jyi55b9Nb4M1bZNTMzU2pCJrfouUXPLXpWLFYc4Jao3KLnFr3G6sLOY1U2AytLc2S4Rc8tem7Rs2Kx4gC3ROUWPbfo+TxWWWZpjgy36LlFzy16VixWHOCWqNyi5xY9n8cqy6x8VAq4JSq36LlFz4rFigPcEpVb9Nyi13/A57HKqurqwn5a9XJyi55b9NyiZ8VixQFuicotem7RmynsZzCXz8BqbGys1IRMbtFzi55b9KxYrDjALVG5Rc8tekc0FHZkVTYDq+7u7lITMrlFzy16btGzYrHiALdE5RY9t+g9MeYnr2fV8PBwqQmZ3KLnFj236FmxWHGAW6Jyi55b9I5t9nmssiqEws5LsZzcoucWPbfoWbFYcYBbonKLnlv0YuLzWGWVpacZ3aLnFj236FmxWHGAW6Jyi55b9B73lwKzq7+/v9SETG7Rc4ueW/SsWKw4wC1RuUXPLXontPk8VlnV3NxcakImt+i5Rc8telYsVhzglqjcoucWvT3TPo+V53me53neiqhsBlYTExOlJmRyi55b9NyiZ8VixQFuicotem7RO7ze57HKqp6enlITMrlFzy16btGzYrHiALdE5RY9t+g9Nuonr2fVwMBAqQmZ3KLnFj236FmxWHGAW6Jyi55b9N7SauDkdRE5X0SeFpFdIvIZZfmZIrJTRBIi8sEFyz4mIs+m//tYvuCKoVA3vezcoucWPbfoWbFYcYBbonKLnlv0EqGwliUHViJSDXwTuAA4HviwiBy/4GovAh8HfrTgZzuAzwGnACcDnxOR9tzZB9fR0VGImz2k3KLnFj236FmxWHGAW6Jyi55b9J6dKOwHQmfzjNXJwK4QwvMhhDhwPbB5/hVCCC+EEH4NLDwj7DzgP0MIwyGEEeA/gfPz4D4oS08zukXPLXpu0bNiseIAt0TlFj236L3ZwEuBrwN2z/v+pfRl2ZTLzy6r1tbWQtzsIeUWPbfouUXPisWKA9wSlVv03KL30lRhTy/P5tR47cXIbD9oJ6uf3bt3L1u2bCEWi5FMJrnwwgvZunUrfX19NDU1UV1dzdjYGN3d3QwPDxNCoLu7m/7+/sykY3v37qWhoYGBgQFEhI6ODgYGBmhtbSWZTDI5Ocnq1avp6+ujpqaGtrY2BgcHaWtrIx6PMzU1lVleW1tLS0sLQ0NDtLe3MzU1xfT0dGZ5fX09DQ0NjIyM0NnZyfj4OPF4PLM8kUggIoyOjtLV1cXo6CgzMzOZ5dmu08TEBD09PTmtU21tLb29vTmvU0NDA7W1tTmt09jYGGNjYzmvU77208zMTM7rlI/9NDQ0RGNjY17WKdf9NDs7y+zsbF6OvVz309yxm49jL5f9BDAzM5PXY+9Q12nusnw+RhzqOg0ODjI2NpbXx4hDXae+vj5EJO+PEYeyThMTEzQ0NOT9sfxQ1qm6uprp6em8P5YfyjrNzs4yNjZWkMfy+et0WN0saxqTNFYHHtkXY/2qBP0HqpiZhSMaZnliLMbvtyY4vH6Wx8dinNCWyEwYenj9LI+Nxti9e/eS67TowGepD0YUkQ3A50MI56W/vxIghPBF5brXAT8PIdyY/v7DwDtDCJenv78GuCuE8OP5P7djx46wbt26RR1L1dvby5o1a3K6jXzlFj236LlFz4rFigPcEpVb9CrRcu53Hl3yOmd1xbl7sDZy+S8uO3HJ29i5c+cjmzZtOklbls3zYQ8Bx4rIUSJSC1wMbMvi5wBuB84Vkfb0Sevnpi/Le6tXry7EzR5SbtFzi55b9KxYrDjALVG5Rc8teo/sK/E8ViGEBHAFqQHRU8ANIYQnReRqEXkfgIi8Q0ReAi4CrhGRJ9M/Owz8A6nB2UPA1enL8t5ST80VM7fouUXPLXpWLFYc4Jao3KLnFr31qwp78npWw7YQwq3ArQsuu2re1w8BR0T87HeB7+ZgzKqamppC30XWuUXPLXpu0bNiseIAt0TlFj236O1Plngeq5VSW1tbqQmZ3KLnFj236FmxWHGAW6Jyi55b9Hr3l34eqxXR3Lt3LOQWPbfouUXPisWKA9wSlVv03KL3ppbSz2O1IrI0GnaLnlv03KJnxWLFAW6Jyi16btF7wZ+xyq54PF5qQia36LlFzy16VixWHOCWqNyi5xa9lli2U3EeWmUzsJqamio1IZNb9Nyi5xY9KxYrDnBLVG7Rc4teZ+3CT9/Lb2UzsLI0R4Zb9Nyi5xY9KxYrDnBLVG7Rc4teyeexWilZmiPDLXpu0XOLnhWLFQe4JSq36LlFr9DzWJXNwKq2Nnp6+mLnFj236LlFz4rFigPcEpVb9NyiN57weayyqqWlpdSETG7Rc4ueW/SsWKw4wC1RuUXPLXp7pgo79CmbgdXQ0FCpCZncoucWPbfoWbFYcYBbonKLnlv0jmtJFvT2y2Zg1d7eXmpCJrfouUXPLXpWLFYc4Jao3KLnFr3nJn0eq6yy9FZOt+i5Rc8telYsVhzglqjcoucWPZ9uIcump6dLTcjkFj236LlFz4rFigPcEpVb9Nyit6rGJwjNKktzZLhFzy16btGzYrHiALdE5RY9t+j5PFZZZmmODLfouUXPLXpWLFYc4Jao3KLnFj2fxyrL6uvrS03I5BY9t+i5Rc+KxYoD3BKVW/TcordvxuexyqqGhoZSEzK5Rc8tem7Rs2Kx4gC3ROUWPbfoDcV9HqusGhkZKTUhk1v03KLnFj0rFisOcEtUbtFzi97RTT6PVVZ1dnaWmpDJLXpu0XOLnhWLFQe4JSq36LlF75lxn8cqq8bHx0tNyOQWPbfouUXPisWKA9wSlVv03KJ3eIPPY5VV8Xi81IRMbtFzi55b9KxYrDjALVG5Rc8tei0xn8cqqyzNkeEWPbfouUXPisWKA9wSlVv03KLn81hlmaU5Mtyi5xY9t+hZsVhxgFuicoueW/R8Hqsss/RWTrfouUXPLXpWLFYc4Jao3KLnFj0T0y2IyPki8rSI7BKRzyjL60TkJ+nlD4rI2vTlNSLyfRF5XESeEpEr88t/tdra2kLd9LJzi55b9NyiZ8VixQFuicotem7RG0+UeIJQEakGvglcABwPfFhEjl9wtS3ASAjhGOCrwJfSl18E1IUQ3gKsBy6fG3Tlu9HR0ULc7CHlFj236LlFz4rFigPcEpVb9Nyit7ax9PNYnQzsCiE8H0KIA9cDmxdcZzPw/fTXNwKbRESAADSJSAxoAOLAWF7kC+rq6irEzR5SbtFzi55b9KxYrDjALVG5Rc8tek+Nl/7k9dcBu+d9/1L6MvU6IYQEMAp0khpkTQJ7gBeB/zeEMJyjWc3SaNgtem7Rc4ueFYsVB7glKrfouUVvTYGfscpm2Ka9GLlwEoio65wMJIHfA9qBe0Tkv0IIz8+/4t69e9myZQuxWIxkMsmFF17I1q1b6evro6mpierqasbGxuju7mZ4eJgQAt3d3fT399Pc3AzA4OAgHR0dDAwMICKZr1tbW0kmk0xOTrJ69Wr6+vqoqamhra2NwcFB2traiMfjTE1NZZbX1tbS0tLC0NAQ7e3tTE1NMT09nVleX19PQ0MDIyMjdHZ2Mj4+TjwezyyfnJyksbGR0dFRurq6GB0dZWZmJrM823WamJigp6cnp3VKJpP09vbmvE4NDQ3U1tbmtE5jY2PMzMzkvE752E+Dg4PEYrGc1ykf+2lwcJDOzs68HHu57qepqSnq6urycuzlup/mjt18HHu57Kfp6Wn27t2b18eIQ12neDxOb29vXh8jDnWdRkZGXmMu9X5qbGzM+2P5oazTyMgIHR0deX8sP5R1isfj7NmzJ++P5YeyTpOTk/T29hbk39z563RY3SxrGpM0Vgce2Rdj/aoE/QeqmJmFIxpmeWIsxnHNCeqqAo+PxTihLcGe6dRzTIfXz/LYaIzdu3cvuU6LDppCWHyiLBHZAHw+hHBe+vsrAUIIX5x3ndvT19mRftmvD+gGvgE8EEL4Qfp63wX+I4Rww/z72LFjR1i3bt2ijqU6cOAAdXV1Od1GvnKLnlv03KJnxWLFAW6Jyi16lWg59zuPLnmd5tgsE4noF+x+cdmJS97Gzp07H9m0adNJ2rJsXgp8CDhWRI4SkVrgYmDbgutsAz6W/vqDwJ0hNWJ7EThHUjUBpwK/zeI+l52lOTLcoucWPbfoWbFYcYBbonKLnlv0Sj6PVfqcqSuA24GngBtCCE+KyNUi8r701a4FOkVkF/CXwNyUDN8EmoEnSA3QvhdC+HWe1wGApqamQtzsIeUWPbfouUXPisWKA9wSlVv03KLXf6Cw81hldWp8COFW4NYFl1017+tpUlMrLPy5Ce3yQlRdXdhPq15ObtFzi55b9KxYrDjALVG5Rc8tejOF/Qzm8pl5fWysILM4HFJu0XOLnlv0rFisOMAtUblFzy16RzQUdmRVNgOr7u7uUhMyuUXPLXpu0bNiseIAt0TlFj236D0xVvp5rFZEw8MFmR7rkHKLnlv03KJnxWLFAW6Jyi16btE7trn0M6+viJaaNqKYuUXPLXpu0bNiseIAt0TlFj236MWksJbCPh9WxCw9zegWPbfouUXPisWKA9wSlVv0imXJZu6otppZRmcGI5dnM3dUvnrcXwrMrv7+/lITMrlFzy16btGzYrHiALdE5RY9S5YT2go7d9RyKrSlbAZWc9PsW8gtem7Rc4ueFYsVB7glKrfoWbLMfWyMhQptKZuXAj3P8zyvUsrm5bfjmhM8M/Fi5PJivvxWSdkZQubYxMREqQmZ3KLnFj236FmxWHGAW6Jyi97h9QWeCXMZVZKlbAZWPT09pSZkcoueW/TcomfFYsUBbonKLXqPjdp5UaqSLGUzsBoYGCg1IZNb9Nyi5xY9KxYrDnBLVG7Re0urnRPGK8lSNgMrESk1IZNb9Nyi5xY9KxYrDnBLVG7RSwS3aBXaUjYDq46OjlITMrlFzy16btGzYrHiALdE5Ra9ZyfsfPBxJVnKZmBl6elXt+i5Rc8telYsVhzglqjcovdmQy+/VZKlbAZWra2tpSZkcoueW/TcomfFYsUBbonKLXovTdn5J76SLHbWNMeSycJ+qOJycoueW/TcomfFYsUBbonKLXo1hv6FrySLoVXNrcnJyVITMrlFzy16btGzYrHiALdE5Ra9njo7c0dVkqVsBlarV68uNSGTW/TcoucWPSsWKw5wS1Ru0Xtkn525oyrJUjYDq76+vlITMrlFzy16btGzYrHiALdE5Ra99avsnDBeSZayGVjV1NSUmpDJLXpu0XOLnhWLFQe4JSq36O1P2pk7qpIsZTOwamtrKzUhk1v03KLnFj0rFisOcEtUbtHr3W9n7qhKspTNwGpwcLDUhExu0XOLnlv0rFisOMAtUblF700tdl5+qyRL2QysLP2V4BY9t+i5Rc+KxYoD3BKVW/ReMPQsUSVZymZgFY/HS03I5BY9t+i5Rc+KxYoD3BKVW/RaYqHUhEyVZCmbgdXU1FSpCZncoucWPbfoWbFYcYBbonKLXmetnbmjKsmS1cBKRM4XkadFZJeIfEZZXiciP0kvf1BE1s5b9lYhQA4bAAAgAElEQVQR2SEiT4rI4yJSnz/+q1maO8Qtem7Rc4ueFYsVB7glKrfoVdLcUcup0JYlb11EqoFvAu8GXgIeEpFtIYTfzLvaFmAkhHCMiFwMfAn4YxGJAT8ELgkh/EpEOoGZvK8FqblD1qxZU4ibXnZu0XOLnlv0rFisOMAtURXLcu53Hl3yOmd1xbl7sHbR6/zishPzRVq09asSS1qKVSVZsnnG6mRgVwjh+RBCHLge2LzgOpuB76e/vhHYJCICnAv8OoTwK4AQwlAIoSAfpFRba2OHgVuicoueW/SsWKw4wC1RWbKMJ+zM1+QWvUJbsnk+7HXA7nnfvwScEnWdEEJCREaBTuA4IIjI7UA3cH0I4csL72Dv3r1s2bKFWCxGMpnkwgsvZOvWrfT19dHU1ER1dTVjY2N0d3czPDxMCIHu7m76+/tpbm4GYGJigunpaQYGBhAROjo6GBgYoLW1lWQyyeTkJKtXr6avr4+amhra2toYHBykra2NeDzO1NRUZnltbS0tLS0MDQ3R3t7O1NQU09PTmeX19fU0NDQwMjJCZ2cn4+PjxOPxzPKqqir27dvH6OgoXV1djI6OMjMzk1m+nHXq6enJaZ1aW1vp7e3NeZ0aGhqora3NaZ1mZ2fp7e3NeZ3ysZ8mJibYu3dvzuuUj/00MTHBgQMH8nLs5bqfYrEYQ0NDeTn2ct1Pc8duPo69XPZTbW0te/fuzetjxKGuU2NjI729vXl9jDjUdTpw4AC9vb15fYw41HWamJhg3759eX8sX7hO72ifobE68Mi+GOtXJeg/UMXMLBzRMMsTYzGObU7SHpulrWaWE9oS7JlOPXdxeP0sj43GeEtrgkQQ9u/fn/N+OqsrznOT1XTWzrKq5lXTvhlhKF7F0U1Jhg8Ib181Q0vs1eVD8SrGE8LaxtT95mM/ndoxw7MT1by5NcFLU1XUVKU+j2/uPvcnhcmZ1LN5L+yvpiUW6Kx9dfl4QpiYmMjLfjqsbpY1jclF91NXzSynd8Z5fCym7qfdu3cveewtloSw+NnxInIRcF4I4bL095cAJ4cQPjXvOk+mr/NS+vvnSD3TdSmwFXgHsB+4A/hsCOGO+fexY8eOsG7dukUdS9Xb22vmaWm36LlFzy16VixWHOCWqIplsfRSYD4s+XpJshItO3fufGTTpk0nacuyeSnwJeD1874/Angl6jrp86ragOH05XeHEAZDCPuBW4G3Z3Gfy669vb0QN3tIuUXPLXpu0bNiseIAt0RlyfLcpJ35mtyiV2hLNgOrh4BjReQoEakFLga2LbjONuBj6a8/CNwZUk+F3Q68VUQa0wOus4DfUIAsvcXVLXpu0XOLnhWLFQe4JSpLlkqaVmA5VZJlyXOs0udMXUFqkFQNfDeE8KSIXA08HELYBlwL/EBEdpF6puri9M+OiMhXSA3OAnBrCOHfC7Ei09PThbjZQ8otem7Rc4ueFYsVB7glKkuWVTV2JsJ0i16hLVlN5hBCuJXUy3jzL7tq3tfTwEURP/tDUlMuFDRLc4e4Rc8tem7Rs2Kx4gC3RGXJUknzNS2nSrKUzczrS52lX8zcoucWPbfoWbFYcYBborJkWb/KzocNu0Wv0JayGVjV1xdkQvdDyi16btFzi54VixUHuCUqS5Z9M3bma3KLXqEtZTOwamhoKDUhk1v03KLnFj0rFisOcEtUlixDcTv/rLpFr9AWO2uaYyMjI6UmZHKLnlv03KJnxWLFAW6JypLl6KaCfLjIIeUWvUJbymZg1dnZWWpCJrfouUXPLXpWLFYc4JaoLFmeGbczX5Nb9AptKZuB1fj4eKkJmdyi5xY9t+hZsVhxgFuismQ5vMHOfE1u0Su0xc77H3MsHo+XmpDJLXpu0XOLnhWLFQdUpiX7jyhZ/J2B+frIlKVqidmZr8kteoW2lM0zVpbmMXGLnlv03KJnxWLFAW6JqpLmSFpObtHzeayyzNI8Jm7Rc4ueW/SsWKw4wC1RVdIcScvJLXo+j1WWWXq7rVv03KLnFj0rFisOcEtUlfRW/uXkFj2fbiHLamtrS03I5BY9t+i5Rc+KxYoD3BLVeMLO5JNu0askS9kMrEZHR0tNyOQWPbfouUXPisWKA9wS1dpGO3MkuUWvkixlM7Dq6uoqNSGTW/TcoucWPSsWKw5wS1RPjds5MdotepVkKZuBlaW/ntyi5xY9t+hZsVhxgFuiWmPo2RC36FWSpWwGVjMzM6UmZHKLnlv03KJnxWLFAW6JqrHazhxJbtGrJEvZDKwszaniFj236LlFz4rFigPcElUlzZG0nNyi5/NYZZmlOVXcoucWPbfoWbFYcYBboqqkOZKWk1v0fB6rLGtqaio1IZNb9Nyi5xY9KxYrDnBLVP0H7PxT5ha9SrLYWdMcq66288nZbtFzi55b9KxYrDjALVHN2Pl8X7dEVEmWshlYjY2NlZqQyS16btFzi54VixUHuCWqIxrs/KvtFr1KspTNwKq7u7vUhExu0XOLnlv0rFisOMAtUT0xZufEaLfoVZKlbAZWw8PDpSZkcoueW/TcomfFYsUBbonq2GY7cyS5Ra+SLGUzsArBzhwZbtFzi55b9KxYrDjALVHFxC1abtErtKVsBlaWnpZ2i55b9NyiZ8VixQFuiepxQy8zuUWvkixZDaxE5HwReVpEdonIZ5TldSLyk/TyB0Vk7YLlR4rIhIj8dX7YB9ff31+om152btFzi55b9KxYrDjALVGd0GZnjiS36FWSZcmBlYhUA98ELgCOBz4sIscvuNoWYCSEcAzwVeBLC5Z/Fbgtd250zc3Nhbz5ZeUWPbfouUXPisWKA9wS1Z5pOy++uEWvkizZ3PrJwK4QwvMhhDhwPbB5wXU2A99Pf30jsElEBEBE3g88DzyZH7LneZ7neZ7NshlYvQ7YPe/7l9KXqdcJISSAUaBTRJqA/wH8fe7UxZuYmCj0XWSdW/TcoucWPSsWKw5wS1SH19uZI8ktepVkyeYMLlEuW3hKfdR1/h74aghhIv0EltrevXvZsmULsViMZDLJhRdeyNatW+nr66OpqYnq6mrGxsbo7u5meHiYEALd3d309/dnno5OJBJMT08zMDCAiNDR0cHAwACtra0kk0kmJydZvXo1fX191NTU0NbWxuDgIG1tbcTjcaampjLLa2traWlpYWhoiPb2dqamppiens4sr6+vp6GhgZGRETo7OxkfHycej2eWx2Ix9u3bx+joKF1dXYyOjjIzM5NZnu06TUxM0NPTk9M6tbe309vbm/M6NTQ0UFtbm9M61dXV0dvbm/M65WM/JRIJ9u7dm/M65WM/JRIJDhw4kJdjL9f9VFdXx9DQUF6OvVz309yxm49jL5f91NDQwN69e/P6GHGo67Rq1Sp6e3vz+hhxqOtUVVVFb29vXh8jtHXqrJ3lza0JXpqqoqYKeupmeWRfjPWrEuxPCr37q6mrCqxpTNISC3TWvrp8PCHsmariuJYkY2NjOe+nd7TP0FgdMrfff6CKmdnUhJNPjMU4tjlJjQTaamY5oS2Recnp8PpZHhuN8ZbWBIkg7N+/P+f9dFZXnOcmq+msnWVVzaumfTPCULyKo5uSvDJVxdtXzdASe3X5ULyK8YSwtjF1v/nYT6d2zPDsRPWi+2n3VBVndcV5YX+1up8mJiby8vt0WN0saxqTi+4nIXB6Z5zHx2Lqftq9e/eSv0+LDpqWesusiGwAPh9COC/9/ZUAIYQvzrvO7enr7BCRGNAHdAPbgdenr7YKmAWuCiF8Y/597NixI6xbt25Rx1Lt3r2b17/+9UtfsQi5Rc8tem7Rs2Kx4oDKtJz7nUeXvM7pnXHuHapd9Dq/uOxEtxTAUamWnTt3PrJp06aTtGXZPGP1EHCsiBwFvAxcDHxkwXW2AR8DdgAfBO4MqRHbGXNXEJHPAxMLB1X5arFnxIqdW/TcoucWPSsWKw5wS1SJ4BYtt+gV2rLkwCqEkBCRK4DbgWrguyGEJ0XkauDhEMI24FrgByKyCxgmNfgqah0dHcW+y8jcoucWPbfoWbFYcYBbonp2ws4HQrtFr5IsWc2SFUK4Fbh1wWVXzft6Grhoidv4/CH4sm5gYIA1a9YU8i6yzi16btFzi54VixUHFM+SzcspZ3XFuXuw8C95ZdObWxNLWoqVW/QqyWJnYokca21tLTUhk1v03KLnFj0rFisOsGV5acrOPx9u0XOLXqEtdtY0x5JJOx/w6BY9t+i5Rc+KxYoDbFlqDP3r4RY9t+gV2mLnw3tybHJykq6urlIzALdE5RY9t+hZsRTTsdRLcKmX33Yvep1ivfzWUzfLb8eLcldL5hY9t+gV2mJoDJlbq1evLjUhk1v03KLnFj0rFisOgEf22flb2C16btGrJEvZDKyWmrCrmLlFzy16btGzYrHiAFi/ys4H2bpFzy16lWQpm4FVTU1NqQmZ3KLnFj236FmxWHEA7E/amQvILXpu0askS9kMrNra2kpNyOQWPbfouUXPisWKA6B3v525gNyi5xa9SrKUzcBqcHCw1IRMbtFzi55b9KxYrDgA3tRi5+UUt+i5Ra+SLGUzsLL0V6Vb9Nyi5xY9KxYrDoAXDP3V7xY9t+hVkqVsBlbxeLzUhExu0XOLnlv0rFisOABaYqHUhExu0XOLXiVZymZgNTU1VWpCJrfouUXPLXpWLFYcAJ21s6UmZHKLnlv0KslSNgMrS3PNuEXPLXpu0bNiseKAypoLaDm5Rc8tej6PVZZZmmvGLXpu0XOLnhWLFQdU1lxAy8ktem7R83mssqy21sanZoNbonKLnlv0rFisOADGE3bmAnKLnlv0KslSNgOrlpaWUhMyuUXPLXpu0bNiseIA2DNl5yHbLXpu0aski501zbGhoaFSEzK5Rc8tem7Rs2Kx4gA4riVZakImt+i5Ra+SLGUzsGpvby81IZNb9Nyi5xY9KxYrDoDnJu3MBeQWPbfoVZKlbAZWlt4S7RY9t+i5Rc+KxYoDKust68vJLXpu0Su0xc77H3Nsenq61IRMbtFzi55b9IplOfc7jy66/KyuOHcPvrLodX5x2Yn5JEW2qsbOJItu0XOLXiVZyuYZK0tzzbhFzy16btGzYqmk+XeWk1v03KJXSZayGVhZmmvGLXpu0XOLnhVLJc2/s5zcoucWvUqylM3Aqr6+vtSETG7Rc4ueW/SsWPbN2Jl/xy16btFzi16hLWUzsGpoaCg1IZNb9Nyi5xY9K5ahuJ2HSbfouUXPLXqFtthZ0xwbGRkpNSGTW/TcoucWPSuWo5vszL/jFj236LlFr9CWrAZWInK+iDwtIrtE5DPK8joR+Ul6+YMisjZ9+btF5BEReTz9/3Pyy3+1zs7OQt30snOLnlv03KJnxfLMuJ35d9yi5xY9t+gV2rLkwEpEqoFvAhcAxwMfFpHjF1xtCzASQjgG+CrwpfTlg8B7QwhvAT4G/CBf8IWNj48X6qaXnVv03KLnFj0rlsMb7My/4xY9t+i5Ra/QlmyesToZ2BVCeD6EEAeuBzYvuM5m4Pvpr28ENomIhBAeDSHMTQDzJFAvInX5gC8sHo8X4mYPKbfouUXPLXpWLC0xO/PvuEXPLXpu0Su0JZvJHF4H7J73/UvAKVHXCSEkRGQU6CT1jNVcHwAeDSEcOHRudFbmvAG3ROUWvUq0LDUpJ0BzbJaJxHDk8mJNyllJ8+8sJ7fouUWvkizZ3Lr2vsSFw71FryMiv0/q5cFztTvYu3cvW7ZsIRaLkUwmufDCC9m6dSt9fX00NTVRXV3N2NgY3d3dDA8PE0Kgu7ub/v5+mpubAXj55Zc57rjjGBgYQETo6OhgYGCA1tZWkskkk5OTrF69mr6+Pmpqamhra2NwcJC2tjbi8ThTU1OZ5bW1tbS0tDA0NER7eztTU1NMT09nltfX19PQ0MDIyAidnZ2Mj48Tj8czyycnJ/m93/s9RkdH6erqYnR0lJmZmczybNdpYmKCnp6enNYpmUxSXV2d8zo1NDRQW1ub0zoNDAzQ0NCQ8zrlYz89/fTTrF69Oud1ysd+6uvr441vfGNejr1c99PU1BTd3d15OfYW209ndcV5ZF+M9asS7JsRhuJVHN2U5Jnxag5vmKUlFqitCsRnU8vGE8LaxiRPjcdY05iksTpw4MCBvOyno5sS1FRBT91sxrQ/KfTur+ZNLQnaamZ5fjJGZ+2ry8cTwp6pKo5rSfLcZDX9/f152U/NsVnWr0rQf6CKmVk4omGWJ8ZiHNucJCYhtd6zwp7p1IsNh9fP8thojLe0JkgE4dmJanp7e3PeT0c0JOmsnWVVTYjcT2d1x9kzXZ1Zru2n3t7enH+fOmtneXNrgpemqiL30/k9B7h7sJaWWIjcT2NjYzn/Pr2jfYbG6le3ibafjm1KsK2vjhPaEpH7af/+/Tn/Pp3VFee5yepF91NzdWDPgSpaYkHdT5OTk3l53Du1Y4ZnJ6oX3U/dtbNMJoUX9ler+2liYiIvj3uH1c1mHiOi9tP5PQfo3V/N42MxdT/t3r17yce9RQdNISz+lJiIbAA+H0I4L/39lQAhhC/Ou87t6evsEJEY0Ad0hxCCiBwB3AlcGkK4T7uPHTt2hHXr1i3qWKq9e/dy2GGH5XQb+cotem7Rq0RLNs9Yvbk1wRNj0X/75esZq6UsSznK0ZKP/eMW+5ZiHbflaNm5c+cjmzZtOklbls05Vg8Bx4rIUSJSC1wMbFtwnW2kTk4H+CBwZ3pQtQr4d+DKqEFVvqqtrS3kzS8rt+i5Rc8teuMJGxMKWnGAW6Jyi55b9AptWXJgFUJIAFcAtwNPATeEEJ4UkatF5H3pq10LdIrILuAvgbkpGa4AjgH+TkQeS/9XkD+HR0dHC3Gzh5Rb9Nyi5xa9tY025r2x4gC3ROUWPbfoFdqS1RlcIYRbgVsXXHbVvK+ngYuUn/tH4B9zNGZVV1dXMe4mq9yi5xY9t+g9NW7jZFcrDnBLVG7Rc4teoS1lM/O6pb+03aLnFj236K0x8heuFQe4JSq36LlFr9CWshlYzczMlJqQyS16btFzi15jtY15b6w4wC1RuUXPLXqFtpTNwKoS5wLKJrfouUXPksXKvDdWHOCWqNyi5xa9QlvKZmC11LwSxcwtem7Rc4ve+lWJUhMAOw5wS1Ru0XOLXqEtZTOwampqKjUhk1v03KLnFr3+AzYenqw4wC1RuUXPLXqFtthZ0xyrrrbzydlu0XOLnlv0Zox8ZqsVB7glKrfouUWv0BY7L3rm2NjYGO3t7aVmAG6Jyi16xbJkMyPxWV1x7h6MniS0WJ/PB6mPn3husmh3Z94BbonKLXpu0Su0pWyeseru7i41IZNb9NyiZ8my1EdxFDMrFisOcEtUbtFzi16hLWUzsBoeHi41IZNb9NyiZ8lybLOduWasWKw4wC1RuUXPLXqFtpTNwGqpD5MuZm7Rc4ueJUtM3LIwKw5wS1Ru0XOLXqEtZTOwsvRyilv03KJnyfK4oafrrVisOMAtUblFzy16hbaUzcCqv7+/1IRMbtFzi54lywltduaasWKx4gC3ROUWPbfoFdpSNgOr5ubmUhMyuUXPLXqWLHum7TwkWLFYcYBbonKLnlv0Cm2xs6ae53me53krvLIZWE1MTJSakMktem7Rs2Q5vN7OLH5WLFYc4Jao3KLnFr1CW8pmYNXT01NqQia36LlFz5LlsVE7J5hasVhxgFuicoueW/QKbSmbgdXAwECpCZncoucWPUuWt7TaOcHUisWKA9wSlVv03KJXaEvZDKxEpNSETG7Rc4ueJUsiuGVhVhzglqjcoucWvUJb7Dw3l2MdHR2lJmRyi14lWrL5fL7O2lmG4tHPWhXz8/menbDzIcxWLFYc4Jao3KLnFr1CW8pmYDUwMMCaNWtKzQDcElWxLPn4sGEo3oDmza2JJS3Fyi12HeCWqNyi5xa9QlvK5qXA1tbWUhMyuUXPkuWlKTuHvlv0rFisOMAtUblFzy16hbbYWdMcSybtfMCjW/QsWWoMHflu0bNiseIAt0TlFj236BXaYmhVc2tycrLUhExu0bNk6amzM6eKW/SsWKw4wC1RuUXPLXqFtpTNOVarV68uNSFTJVqyOa+pOTbLRGJ40esU67ymR/bZOfTdomfFYsUBbonKLXpu0Su0pWyeserr6ys1IZNb9NavsjOPiVv03HJwVhzglqjcoucWvUJbshpYicj5IvK0iOwSkc8oy+tE5Cfp5Q+KyNp5y65MX/60iJyXP/pru/nmmwt108vOLXoP//K2UhMyuUXPLQdnxQFuicotem7RK7RlyYGViFQD3wQuAI4HPiwixy+42hZgJIRwDPBV4Evpnz0euBj4feB84F/St5f3fvrTnxbiZg8pt+g9epedXyy36Lnl4Kw4wC1RuUXPLXqFtmTzQuPJwK4QwvMAInI9sBn4zbzrbAY+n/76RuAbkppOejNwfQjhAPA7EdmVvr0d+eG/WiJRvKcZlzqfaO/Y9JLXyce5RNmc11QsSzY1GHrh2S16bjk4Kw5wS1Ru0XOLXqEtEkJY/AoiHwTODyFclv7+EuCUEMIV867zRPo6L6W/fw44hdRg64EQwg/Tl18L3BZCuHH+fdx6663je/bsyaxqa2vrQEdHx+ByVmR4eLhruT9TqNyi5xY9t+hZsVhxgFuicoueW/TyZFmzadOmbm1BNs9YaR+qs3A0FnWdbH6WP/iDP2jJwuF5nud5nme6bJ4Qewl4/bzvjwBeibqOiMSANmA4y5/1PM/zPM8ri7IZWD0EHCsiR4lILamT0bctuM424GPprz8I3BlSrzFuAy5Ov2vwKOBY4P/kh+55nud5nmerJV8KDCEkROQK4HagGvhuCOFJEbkaeDiEsA24FvhB+uT0YVKDL9LXu4HUie4JYGsIwc7nmnie53me5+WxJU9et5yI9ACvI3Xe1ishhP4Sk0wlIh0hhMWnOq+Q/FjxDjUReV/6D0jPWzFZevyvtN8hO3PMLyMROQH4FqlzuV5OX3yEiOwD/q8Qws4iWt4AfJbUuWP/D6l5vDYATwF/E0J4oUiOjcB3gFngvwP/CBwtIjXAh0IIeZ/iYhHLT4GfAjeHECaKdb8RFjPHymKJyLdDCH9axPurBi4jdd7jf4QQ7pu37LMhhH8souUKUtOyDIrIMcB3gbcCTwOXhRAeL6LlwoUXAd9MnztKCMHExHDFPl6U+38mhHBcCe63Cvg48AFSx24CeBb4VgjhriJbGoErSP2x9nVSr9RcCPwWuLqYj33zf2fT80feDNSkpz364xDCg0W0mPkdKtXj3Ip8xkpEHgMuX3iwiMipwDUhhLcV0bId+DGpf7j/BPgecANwLvDfQgjnFMnxf0hN1NoM3AK8P4Rwr4i8Hfh6CGFjMRxpy8uk5io7B/gvUtvn30MI8WIZ5lksHSsdUYuAX4UQjiii5TtAI6lzHi8B7g4h/GV62c4QwtuLaHkyhPD76a//HfhOCOEmEXkn8L+KfOwmgP8A9vLqu5o/SGp+vhBC+O9FtJg4XkRknFffzT23TRqB/aS2SWsxHGnL94BeUo8rHwTGgHuA/wH8LITw9SJabgB2Aw3AG0n9MX0D8F5gdQjhkiJaMr+z6d+hb4QQbhORk4F/CiGcVkSLpd+h0jzOhRBW3H/As4ss21Vky6Pzvn4xalmRHU8tWLazFNsEaEkfzLcCA6QGnedW8LGSBJ4Hfjfvv7nv40W2/Hre1zHg26SeZawr5nGbvv+n5339UJSzSJZ3AHcAf8arf3j+rpgGa8cLqWdj/hXomXdZqbbJrxd8/0D6/3ULH/eKYHks/X8B+uYdL1KC43bnvK8fXbCs2L/Pln6HSvI4Z2gu1GV1m4j8u4j8sYiclv7vj9Mj9f8osmVWRI4TkXcAjSJyEkD6JY2CfHxPRPP35ZULltUW0QHpv25DCOMhhB+EEP6A1F90DwIHfdZkgbN0rDwPvDOEcNS8/94QQjgKKPY5X5ljIoSQCKmXlR4D7iT1rGcxu1FErku/rH6TiPy5iBwpIpcCLxYTEkJ4CHg3qe1zZ/ov/lI9rW/ieAkhfAr4GvBjEfl0+uW4Um2TGRE5GiD9bHw8bTxQKlNI/at9a/r/c98X2/IGEdkmIreQOtWhcd6ymmJCjP0OleRxbkWeYxVC+LSIXEDqI3NeR+ovhJeAb4YQbi0y529JvfQ2C7wfuFJE3ga0Ap8oouPvRKQxhLA/hJD55OX0g9C/FtEBcNC5BSF1EuW30v8VLWPHyj8B7eiDhS8X2fKwiJwfQsgMLkMIV4vIK8D/V0xICOF/pgdRPwaOJvXX5J+SOk/kvxXTkvbMAl8TkX8jtc9KlZnjJYTwiIi8i9Q5RXcD9cW8/3n9DfBLEZkmNWC4GEBEuoGfF9nysIg0hxAmwryXt9KPueNFtmxe8H1V2tJDkX+fwdTvUEke51bkOVaWEpGaEMLMgsu6SH0odUVOLaFtE8/WdnGL/axsl4UOETkcOLEEf5iQfjNOAugMIZT041EW2z8iIsH/cS15pfodWqkvBUYmIt8u8l2+LCL/W0TOTr8DgxDCoKVBVQm3yTlz28RiFb5dLFrONmCJrIKPl9c4Qgh7SjGomrOQOk/mLQaOlcj9Y2lQVYLjNrJK+R1akc9YWXm3TNrSSeodDxeTmln+RuDHoYhvb007fJvoFt8ublmOxY8Xow63LGqxdNxaspRkH63UgVWS1Ftu549A5z70+XUhhGKfrD3n+j3gIlI78TBSc/P8zyLdt28T/f59u7hlOffvx8sKcLjloPs3c9xasixwFW0frdSB1bPAphDCQSd1isjuEMLrlR8rSiLSTGqSuL8EDg8h9BTpfn2b6Pft28Uty7lvP15WiMMtr7lvM8etJYty/0XZRyv1HKu5d8toFfvdVYhIvYhcJKkZx58DNpGa8uD3isjwbaLn28Uty8mPF8MOt0Rm6bi1ZCnJPlqpz1iZeLcMgMgY/J4AAAUISURBVIj8CHgXsB24Hvh5CGG6BA7fJrrFt4tblmPx48Wowy2LWiwdt5YspdlHoQSzoeb6H6mp8v83qY9MkRJbPga0ZHM93ybF3Sa+XdxyCBY/Xow63LLofVg6bi1ZSrKPVuozVmbejZFtUuDPX/NtEnkfvl1yqNIsfrysfAdUnsXScWvJkm353kcr8hyrEMJQCOGaEMLZwMmkPjvrn0TkORH5XyXmRVXQOTR8m+j5dsm5irL48ZJTVhxQYRZLx60lyzLK6z5akQOr+YUQXgGuJTU9/ThwWWlFkRXtqUHfJhF35tvlUKpYix8vy86KAyrYYum4tWRZorzuoxU7sDL0boxsK/hfLb5NIu7Et0suVZzFj5dDzooDKtBi6bi1ZMmyvO6jFfkhzAvO9P8R8JFQondjLKP7Cnnjvk30fLvkXEVZ/HjJKSsOqDCLpePWkmUZ5XcflfKM/ZV2pv8i97OO1Ii8ecHl5/s2Kd028e3ilpV+vCxy/5dW4v6xZjG0f8wct5YspdpHJVupIm24nUW4j08DTwM3Ay8Am4t5/75NfLu4pTyPlyXu/8VK3D+WLFb2zzJdlrZR2f4OrciXApdRMV7b/gSwPoQwISJrgRtFZG0I4WtFuv/l5ttEr9K2i1tyqxjnTP56kfsu5ke3WNo/ZiyG9s9ysvT7VLa/Q+U+sCrGuzGqQwgTACGEF0TknaR+2ddg6yCey7eJXqVtF7fkVjGOlx7gPGBkweUC3F+E+5/L0v6xZLGyf5ZTpb1bsiT7aMW+KzDLivGL1iciJ8x9k/6lfw/QBbylCPe/3Hyb6FXadnFLbhXjePk5qfOIehf89wJwVxHufy5L+8eSxcr+WU6W/lAp29+hch9YFeOdIUcCe+ZfEEJIhBA+CpxZhPtfbr5N9Cptu7glt4pxvJwYQrhXWxBC+EgR7n8uS/vHksXK/llOFfVuSUq0j8puYCUil859HUK4ogh3ORZC6NcWhBBMHMSVvk1EZJ2IbBKR5gWXnz/PVWnbxS0RGTperGRp/1iymGolHLcl+LeoJK3IzwpcLBF5MYRwZBHv7yXgK1HLQwiRy4pVJW8TEfk0sBV4CjgB+L9DCD9LLyvq54kZ2y5u0S1+vBh1uGVRi5njdrEq5d+iFXnyurF3Y1QDzZT4tWvfJpGZeRcRtraLW/T8eLHrALdEZea49X+LVugzViLSzyJn+ocQijZtvpW/Bnyb6InIb0IIx8/7vpnUp63/BjgnhHBC5A/n32Jpu7hFyY8Xuw5wS1TGjtuK/7dopZ5jZendGBb+WgHfJlFZeheRpe3iFj0/Xg7OigPcEpWl47bi/y1aqc9YWfpLoSOEMGzA4dtESUR2ARu1E15FZGMxT3g1tl3couTHi10HuCUqY8dtxf9btFKfsTKTlV8sSxnbJmbeRWRpu7glMj9eFmTFAW5ZJDPHraVKtY9W5MnrwGEi8pdRCy28E68E+TbR8+3iLSc/XryVmKXj1pKlJK3UgZWld2NYybeJnm8Xbzn58eKtxCwdt5YsJcnPsSqTfJvo+XbxlpMfL95KzNJxa8lSqlbqOVYVOxJeJN8mer5dvOXkx4u3ErN03FqylKSV+oyVmXdjWMm3iZ5vF285+fHircQsHbeWLKVqRQ6sPM/zPM/zLLZSXwr0PM/zPM8zlw+sPM/zPM/z8pQPrDzP8zzP8/KUD6w8z/M8z/PylA+sPM/zPM/z8tT/DzUPRgfcteucAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "corrs_log = [data[col].corr(np.log(data['Hazard'])) for col in numerical]\n",
    "corrs_log = pd.DataFrame(np.abs(corrs_log), numerical, \n",
    "    ['abs corr with log target']).sort_values('abs corr with log target')\n",
    "corrs_log.plot(kind='bar', figsize=(10, 4))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "source": [
    "Уберите несколько наиболее неинформативных признаков. Лучше сделать это число гиперпараметром и потом настраивать его по функционалу качества. Может быть разумно также исследовать взаимосвязь признаков с логарифмом целевой переменной, поскольку мы предполагаем, что она неотрицательна и имеет распределение Пуассона."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "def remove_non_inform(df, corr_df, num_remove=3):\n",
    "    return df.drop(columns=corr_df.index.values[:num_remove])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_orig = data\n",
    "data = remove_non_inform(data, corrs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "source": [
    "Закодируем значения категориальных признаков в числа:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "label_encoder = LabelEncoder()\n",
    "data[categorical] = data[categorical].apply(label_encoder.fit_transform)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "source": [
    "Выделим столбец целевой переменной из наших данных. Множество значений случайной величины с распределением Пуассона начинается с нуля $\\{0, 1, 2...\\}$, поэтому вычтем единицу из целевой переменной. На самом деле, помимо прочего это приводит к существенному росту качества на валидации."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "objects = data.loc[:, 'T1_V1':]\n",
    "labels = data['Hazard'] - 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "source": [
    "Сделаем бинарное кодирование категориальных признаков:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "mask = objects.columns.isin(categorical)\n",
    "onehot_encoder = OneHotEncoder(categorical_features=mask, sparse=False)\n",
    "objects_encoded = onehot_encoder.fit_transform(objects)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "source": [
    "Разделим выборку на обучающую и тестовую:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Для нормальной работы функции train_test_split с параметром stratify отдельно выделим объекты, для которых целевая переменная принимает уникальное значение, и занесем их в обучающую выборку, а к остальным применим  train_test_split."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "label_values = data.groupby('Hazard')['Hazard'].count()\n",
    "ones_index = label_values.index[label_values == 1]\n",
    "mask = np.isin(data['Hazard'].values, ones_index)\n",
    "\n",
    "train_objects, test_objects, train_labels, test_labels = train_test_split(\n",
    "    objects_encoded[np.logical_not(mask)], labels.values[np.logical_not(mask)], \n",
    "    random_state=1, test_size=0.2, stratify=labels.values[np.logical_not(mask)]\n",
    ")\n",
    "\n",
    "train_objects = np.concatenate((train_objects, objects_encoded[mask]), axis=0)\n",
    "train_labels = np.concatenate((train_labels, labels.values[mask]), axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "source": [
    "__Задание 2 (2 балла).__ Обучение регрессии с распределением Пуассона.\n",
    "\n",
    "Будем считать, что апостериорное распределение — это распределение Пуассона:\n",
    "\n",
    "$$p(y|\\lambda(x)) = \\frac{e^{-\\lambda(x)}\\lambda(x)^y}{y!}.$$\n",
    "\n",
    "Реализуйте функции для вычисления функционала качества (через метод максимального правдоподобия) и его градиентов — они были выведены на [семинаре](https://github.com/esokolov/ml-course-hse/blob/master/2017-spring/seminars/sem22-glm.pdf).\n",
    "\n",
    "Численные алгоритмы должны работать по возможности быстро, поэтому циклов быть не должно, и все операции должны быть векторными. Дальше мы будем использовать эту функцию в качестве аргумента другой функции. Можете попробовать добавить в модель регуляризатор."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "def oracle(w, X, y):\n",
    "    \"\"\"\n",
    "    :param w: weights\n",
    "    :param X: features\n",
    "    :param y: target\n",
    "    :yield: loss, grad\n",
    "    \"\"\"\n",
    "    loss = np.sum(np.exp(np.dot(X, w)) - y * np.dot(X, w) , axis=0) \n",
    "    grad = np.dot(X.T, (np.exp(np.dot(X, w)) - y)) \n",
    "    return loss, grad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "source": [
    "Добавьте к признакам столбец единиц, чтобы учесть вектор сдвига. Это важно. Библиотечные алгоритмы уже учитывают это внутри себя, поэтому им на вход нужно подавать исходную выборку."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "train_objects_bias = np.concatenate(\n",
    "    (np.ones(np.size(train_objects, 0))[:, np.newaxis], train_objects),\n",
    "    axis=1\n",
    ")\n",
    "\n",
    "test_objects_bias = np.concatenate(\n",
    "    (np.ones(np.size(test_objects, 0))[:, np.newaxis], test_objects), \n",
    "    axis=1\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "source": [
    "Дальше воспользуйтесь функцией _scipy.optimize.minimize_, в ней реализовано множество градиентных методов оптимизации. Рекомендуется взять [L-BFGS-B](http://www.machinelearning.ru/wiki/images/6/65/MOMO17_Seminar6.pdf). Начальное приближение весов $w^{(0)}$ возьмите из стандартного нормального распределения как _np.random.randn_. Возможно, придётся запустить функцию несколько раз, прежде чем начальное приближение окажется удачным, и алгоритм покажет хороший результат. \n",
    "\n",
    "Сделайте прогноз для тестовых объектов. В качестве прогноза мы будем брать матожидание распределения $p(y | \\lambda(x))$ в данной точке, поэтому не забудьте взять экспоненту от выхода линейной модели, предсказывающей значение натурального параметра. Измерьте качество с помощью коэффициента Джини (чем выше – тем лучше) — именно он является целевой метрикой в соревновании."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def gini(y_true, y_pred):\n",
    "    # check and get number of samples\n",
    "    assert y_true.shape == y_pred.shape\n",
    "    n_samples = y_true.shape[0]\n",
    "    \n",
    "    # sort rows on prediction column \n",
    "    # (from largest to smallest)\n",
    "    arr = np.array([y_true, y_pred]).transpose()\n",
    "    true_order = arr[arr[:,0].argsort()][::-1,0]\n",
    "    pred_order = arr[arr[:,1].argsort()][::-1,0]\n",
    "    \n",
    "    # get Lorenz curves\n",
    "    L_true = np.cumsum(true_order) / np.sum(true_order)\n",
    "    L_pred = np.cumsum(pred_order) / np.sum(pred_order)\n",
    "    L_ones = np.linspace(1/n_samples, 1, n_samples)\n",
    "    \n",
    "    # get Gini coefficients (area between curves)\n",
    "    G_true = np.sum(L_ones - L_true)\n",
    "    G_pred = np.sum(L_ones - L_pred)\n",
    "    \n",
    "    # normalize to true Gini coefficient\n",
    "    return G_pred/G_true"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.3181171503352283"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w_0 = np.random.randn(train_objects_bias.shape[1])\n",
    "poisson_results = minimize(oracle, w_0, args=(train_objects_bias, train_labels),\n",
    "                           method=\"L-BFGS-B\", jac=True, tol=1e-6) \n",
    "pred_labels = np.exp(np.dot(test_objects_bias, poisson_results.x))\n",
    "gini(test_labels, pred_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "source": [
    "__Задание 3 (1 балл).__ Линейные модели из коробки.\n",
    "\n",
    "Запустите базовую линейную регрессию *sklearn.linear_model.LinearRegression* для предсказания логарифма целевой переменной и измерьте качество. Сравните полученный результат с работой вашего алгоритма."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.3189047972150875"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reg = LinearRegression()\n",
    "reg.fit(train_objects, np.log(train_labels + 1))\n",
    "pred_labels = np.exp(reg.predict(test_objects)) - 1\n",
    "gini(test_labels, pred_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Результат не зависит от начального приближения, качество немного выше."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "source": [
    "Познакомимся теперь с библиотекой [StatsModels](http://www.statsmodels.org/dev/index.html). Она предназначена скорее для описательной статистики, проверки гипотез и построения доверительных интервалов, чем для прогнозирования, — в отличие от scikit-learn, который создан специально для решения задач машинного обучения. В то же время в StatsModels реализован очень сильный алгоритм прогнозирования временных рядов – [SARIMAX](http://www.statsmodels.org/dev/generated/statsmodels.tsa.statespace.sarimax.SARIMAX.html), который при правильной настройке работает очень хорошо и занимает первые места в конкурсах ([Запись трансляции ML тренировки 03.02.18 | TrainMyData Ascott](https://www.youtube.com/watch?v=9MQEEyYDCQc&t=1101s)). \n",
    "\n",
    "Мы же попробуем обучить обобщённые линейные модели (модуль [GLM](http://www.statsmodels.org/dev/glm.html)) с различными вероятностными распределениями. Запустите алгоритм _sm.GLM_ на нескольких распределениях family, посмотрите на качество и проинтерпретируйте результаты. Синтаксис StatsModels немного отличается от scikit-learn тем, что здесь объекты и метки задаются в конструкторе модели, метод _fit()_ идёт без аргументов, и после обучения сохраняется новая модель с результатами и методом _predict_."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Gaussian__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.31428514376005035"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gaussian_sm_model = sm.GLM(train_labels, train_objects, family=sm.families.Gaussian())\n",
    "gaussian_sm_results = gaussian_sm_model.fit()\n",
    "pred_labels = gaussian_sm_results.predict(test_objects)\n",
    "gini(test_labels, pred_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Poisson__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.317737973766086"
      ]
     },
     "execution_count": 105,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "poisson_sm_model =sm.GLM(train_labels, train_objects, family=sm.families.Poisson())\n",
    "poisson_sm_results = poisson_sm_model.fit()\n",
    "pred_labels = poisson_sm_results.predict(test_objects)\n",
    "gini(test_labels, pred_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__NegativeBinomial (Pascal)__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.31851360919827537"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pascal_sm_model =sm.GLM(train_labels, train_objects, family=sm.families.NegativeBinomial())\n",
    "pascal_sm_results = pascal_sm_model.fit()\n",
    "pred_labels = pascal_sm_results.predict(test_objects)\n",
    "gini(test_labels, pred_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "source": [
    "Как видно, качество в последнем случае получилось лучше. На практике чаще используют именно отрицательное Биномиальное распределение. Оно является обобщением геометрического распределения и даёт некоторую свободу в выборе среднего и дисперсии для $p(y|x)$, тогда как в распределении Пуассона среднее и дисперсия совпадают. Если вам будет интересно, можете прочитать подробнее на вики [NegativeBinomial](https://en.m.wikipedia.org/wiki/Negative_binomial_distribution)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Задание 4. (1 балл)__ Прогнозирование с помощью бустинга."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Что бы мы ни делали, бустинг по-прежнему остаётся лучшим подходом для широкого круга задач — особенно если мы не работаем со структурированными данными вроде последовательностей или картинок. Бустинг хоть и сложный по своей структуре алгоритм, но вероятностные распределения не чужды и ему. Запустите _LGBMModel_, используя классическую регрессию и регрессию Пуассона. Настройте параметры, чтобы добиться наилучшего качества. В особенности обратите внимание на *objective*, *n_estimators*, *num_leaves* и *colsample_bytree*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [],
   "source": [
    "gini_score = make_scorer(gini, greater_is_better=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=3, error_score='raise',\n",
       "       estimator=LGBMModel(boosting_type='gbdt', class_weight=None, colsample_bytree=1.0,\n",
       "     importance_type='split', learning_rate=0.1, max_depth=-1,\n",
       "     min_child_samples=20, min_child_weight=0.001, min_split_gain=0.0,\n",
       "     n_estimators=100, n_jobs=-1, num_leaves=31, objective='regression',\n",
       "     random_state=None, reg_alpha=0.0, reg_lambda=0.0, silent=True,\n",
       "     subsample=1.0, subsample_for_bin=200000, subsample_freq=0),\n",
       "       fit_params=None, iid=True, n_jobs=1,\n",
       "       param_grid={'num_leaves': [10, 100, 1000, 10000], 'colsample_bytree': [0.5, 0.6000000000000001, 0.7000000000000001, 0.8, 0.9, 1.0], 'learning_rate': [0.001, 0.01, 0.1, 1, 10, 100]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring=make_scorer(gini), verbose=0)"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# objective='regression'\n",
    "args_dict = {\n",
    "    'num_leaves': [10 ** x for x in range(1, 5)],\n",
    "    'colsample_bytree': [0.1 * x for x in range(5, 11)],\n",
    "    'learning_rate': [10 ** x for x in range(-3, 3)]\n",
    "}\n",
    "\n",
    "model = LGBMModel(objective='regression')\n",
    "gs = GridSearchCV(model, args_dict, scoring=gini_score, cv=3)\n",
    "gs.fit(train_objects, train_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'colsample_bytree': 1.0, 'learning_rate': 0.1, 'num_leaves': 10}"
      ]
     },
     "execution_count": 120,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gs.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=3, error_score='raise',\n",
       "       estimator=LGBMModel(boosting_type='gbdt', class_weight=None, colsample_bytree=1.0,\n",
       "     importance_type='split', learning_rate=0.1, max_depth=-1,\n",
       "     min_child_samples=20, min_child_weight=0.001, min_split_gain=0.0,\n",
       "     n_estimators=100, n_jobs=-1, num_leaves=31, objective='regression',\n",
       "     random_state=None, reg_alpha=0.0, reg_lambda=0.0, silent=True,\n",
       "     subsample=1.0, subsample_for_bin=200000, subsample_freq=0),\n",
       "       fit_params=None, iid=True, n_jobs=1,\n",
       "       param_grid={'num_leaves': [10, 20, 30, 40], 'learning_rate': [0.1, 0.2, 0.30000000000000004, 0.4]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring=make_scorer(gini), verbose=0)"
      ]
     },
     "execution_count": 121,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "args_dict = {\n",
    "    'num_leaves': [10 * x for x in range(1, 5)],\n",
    "    'learning_rate': [0.1 * x for x in range(1, 5)]\n",
    "}\n",
    "\n",
    "model = LGBMModel(objective='regression')\n",
    "gs = GridSearchCV(model, args_dict, scoring=gini_score, cv=3, colsample_bytree=1.0)\n",
    "gs.fit(train_objects, train_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'learning_rate': 0.1, 'num_leaves': 30}"
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gs.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=3, error_score='raise',\n",
       "       estimator=LGBMModel(boosting_type='gbdt', class_weight=None, colsamle_bytree=1.0,\n",
       "     colsample_bytree=1.0, importance_type='split', learning_rate=0.1,\n",
       "     max_depth=-1, min_child_samples=20, min_child_weight=0.001,\n",
       "     min_split_gain=0.0, n_estimators=1000, n_jobs=-1, num_leaves=30,\n",
       "     objective='regression', random_state=None, reg_alpha=0.0,\n",
       "     reg_lambda=0.0, silent=True, subsample=1.0, subsample_for_bin=200000,\n",
       "     subsample_freq=0),\n",
       "       fit_params=None, iid=True, n_jobs=1,\n",
       "       param_grid={'learning_rate': [0.001, 0.01, 0.1, 1, 10, 100]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring=make_scorer(gini), verbose=0)"
      ]
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "args_dict = {\n",
    "    'learning_rate': [10 ** x for x in range(-3, 3)]\n",
    "}\n",
    "\n",
    "model = LGBMModel(objective='regression', n_estimators=1000, num_leaves=30, colsamle_bytree=1.0)\n",
    "gs = GridSearchCV(model, args_dict, scoring=gini_score, cv=3)\n",
    "gs.fit(train_objects, train_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'learning_rate': 0.01}"
      ]
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gs.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=3, error_score='raise',\n",
       "       estimator=LGBMModel(boosting_type='gbdt', class_weight=None, colsamle_bytree=1.0,\n",
       "     colsample_bytree=1.0, importance_type='split', learning_rate=0.1,\n",
       "     max_depth=-1, min_child_samples=20, min_child_weight=0.001,\n",
       "     min_split_gain=0.0, n_estimators=1000, n_jobs=-1, num_leaves=30,\n",
       "     objective='regression', random_state=None, reg_alpha=0.0,\n",
       "     reg_lambda=0.0, silent=True, subsample=1.0, subsample_for_bin=200000,\n",
       "     subsample_freq=0),\n",
       "       fit_params=None, iid=True, n_jobs=1,\n",
       "       param_grid={'learning_rate': [0.01, 0.02, 0.03, 0.04]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring=make_scorer(gini), verbose=0)"
      ]
     },
     "execution_count": 131,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "args_dict = {\n",
    "    'learning_rate': [0.01 * x for x in range(1, 5)]\n",
    "}\n",
    "\n",
    "model = LGBMModel(objective='regression', n_estimators=1000, num_leaves=30, colsamle_bytree=1.0)\n",
    "gs = GridSearchCV(model, args_dict, scoring=gini_score, cv=3)\n",
    "gs.fit(train_objects, train_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'learning_rate': 0.02}"
      ]
     },
     "execution_count": 130,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gs.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.3613379982349759"
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = LGBMModel(objective='regression', n_estimators=1000, num_leaves=30, colsamle_bytree=1.0, \n",
    "                  learning_rate=0.02)\n",
    "model.fit(train_objects, train_labels)\n",
    "pred_labels = model.predict(test_objects)\n",
    "gini(test_labels, pred_labels) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=3, error_score='raise',\n",
       "       estimator=LGBMModel(boosting_type='gbdt', class_weight=None, colsample_bytree=1.0,\n",
       "     importance_type='split', learning_rate=0.1, max_depth=-1,\n",
       "     min_child_samples=20, min_child_weight=0.001, min_split_gain=0.0,\n",
       "     n_estimators=100, n_jobs=-1, num_leaves=31, objective='poisson',\n",
       "     random_state=None, reg_alpha=0.0, reg_lambda=0.0, silent=True,\n",
       "     subsample=1.0, subsample_for_bin=200000, subsample_freq=0),\n",
       "       fit_params=None, iid=True, n_jobs=1,\n",
       "       param_grid={'num_leaves': [10, 100, 1000, 10000], 'colsample_bytree': [0.5, 0.6000000000000001, 0.7000000000000001, 0.8, 0.9, 1.0], 'learning_rate': [0.001, 0.01, 0.1, 1, 10, 100]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring=make_scorer(gini), verbose=0)"
      ]
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# objective='poisson'\n",
    "args_dict = {\n",
    "    'num_leaves': [10 ** x for x in range(1, 5)],\n",
    "    'colsample_bytree': [0.1 * x for x in range(5, 11)],\n",
    "    'learning_rate': [10 ** x for x in range(-3, 3)]\n",
    "}\n",
    "\n",
    "model = LGBMModel(objective='poisson')\n",
    "gs = GridSearchCV(model, args_dict, scoring=gini_score, cv=3)\n",
    "gs.fit(train_objects, train_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'colsample_bytree': 0.6000000000000001,\n",
       " 'learning_rate': 0.1,\n",
       " 'num_leaves': 100}"
      ]
     },
     "execution_count": 134,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gs.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=3, error_score='raise',\n",
       "       estimator=LGBMModel(boosting_type='gbdt', class_weight=None, colsample_bytree=0.6,\n",
       "     importance_type='split', learning_rate=0.1, max_depth=-1,\n",
       "     min_child_samples=20, min_child_weight=0.001, min_split_gain=0.0,\n",
       "     n_estimators=100, n_jobs=-1, num_leaves=31, objective='poisson',\n",
       "     random_state=None, reg_alpha=0.0, reg_lambda=0.0, silent=True,\n",
       "     subsample=1.0, subsample_for_bin=200000, subsample_freq=0),\n",
       "       fit_params=None, iid=True, n_jobs=1,\n",
       "       param_grid={'num_leaves': [100, 200, 300, 400], 'learning_rate': [0.1, 0.2, 0.30000000000000004, 0.4]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring=make_scorer(gini), verbose=0)"
      ]
     },
     "execution_count": 136,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "args_dict = {\n",
    "    'num_leaves': [100 * x for x in range(1, 5)],\n",
    "    'learning_rate': [0.1 * x for x in range(1, 5)]\n",
    "}\n",
    "\n",
    "model = LGBMModel(objective='poisson', colsample_bytree=0.6)\n",
    "gs = GridSearchCV(model, args_dict, scoring=gini_score, cv=3)\n",
    "gs.fit(train_objects, train_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'learning_rate': 0.1, 'num_leaves': 100}"
      ]
     },
     "execution_count": 137,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gs.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=3, error_score='raise',\n",
       "       estimator=LGBMModel(boosting_type='gbdt', class_weight=None, colsample_bytree=0.6,\n",
       "     importance_type='split', learning_rate=0.1, max_depth=-1,\n",
       "     min_child_samples=20, min_child_weight=0.001, min_split_gain=0.0,\n",
       "     n_estimators=1000, n_jobs=-1, num_leaves=100, objective='poisson',\n",
       "     random_state=None, reg_alpha=0.0, reg_lambda=0.0, silent=True,\n",
       "     subsample=1.0, subsample_for_bin=200000, subsample_freq=0),\n",
       "       fit_params=None, iid=True, n_jobs=1,\n",
       "       param_grid={'learning_rate': [0.001, 0.01, 0.1, 1, 10, 100]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring=make_scorer(gini), verbose=0)"
      ]
     },
     "execution_count": 138,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "args_dict = {\n",
    "    'learning_rate': [10 ** x for x in range(-3, 3)]\n",
    "}\n",
    "\n",
    "model = LGBMModel(objective='poisson', colsample_bytree=0.6, num_leaves=100, n_estimators=1000)\n",
    "gs = GridSearchCV(model, args_dict, scoring=gini_score, cv=3)\n",
    "gs.fit(train_objects, train_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'learning_rate': 0.01}"
      ]
     },
     "execution_count": 139,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gs.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=3, error_score='raise',\n",
       "       estimator=LGBMModel(boosting_type='gbdt', class_weight=None, colsample_bytree=0.6,\n",
       "     importance_type='split', learning_rate=0.1, max_depth=-1,\n",
       "     min_child_samples=20, min_child_weight=0.001, min_split_gain=0.0,\n",
       "     n_estimators=1000, n_jobs=-1, num_leaves=100, objective='poisson',\n",
       "     random_state=None, reg_alpha=0.0, reg_lambda=0.0, silent=True,\n",
       "     subsample=1.0, subsample_for_bin=200000, subsample_freq=0),\n",
       "       fit_params=None, iid=True, n_jobs=1,\n",
       "       param_grid={'learning_rate': [0.01, 0.02, 0.03, 0.04]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring=make_scorer(gini), verbose=0)"
      ]
     },
     "execution_count": 140,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "args_dict = {\n",
    "    'learning_rate': [0.01 * x for x in range(1, 5)]\n",
    "}\n",
    "\n",
    "model = LGBMModel(objective='poisson', colsample_bytree=0.6, num_leaves=100, n_estimators=1000)\n",
    "gs = GridSearchCV(model, args_dict, scoring=gini_score, cv=3)\n",
    "gs.fit(train_objects, train_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'learning_rate': 0.01}"
      ]
     },
     "execution_count": 141,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gs.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.36953202538297636"
      ]
     },
     "execution_count": 143,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = LGBMModel(objective='poisson', colsample_bytree=0.6, num_leaves=100, n_estimators=1000, \n",
    "                 learning_rate=0.01)\n",
    "model.fit(train_objects, train_labels)\n",
    "pred_labels = model.predict(test_objects)\n",
    "gini(test_labels, pred_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Бустинг с настроенными параметрами показал наилучшее качество среди всех рассмотренных моделей."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  },
  "widgets": {
   "state": {},
   "version": "1.1.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}