chapter2/2.2-deep-learning-basic-mathematics.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2.2 深度学习基础及数学原理\n",
    "深度学习并没有想象的那么难，甚至比有些传统的机器学习更简单。所用到的数学知识也不需要特别的高深，本章将会一边讲解深度学习中的基本理论，一边通过动手使用PyTorch实现一些简单的理论，本章内容很多，所以只做一个简短的介绍\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2.1 监督学习和无监督学习\n",
    "监督学习、无监督学习、半监督学习、强化学习是我们日常接触到的常见的四个机器学习方法：\n",
    "\n",
    "- 监督学习：通过已有的训练样本（即已知数据以及其对应的输出）去训练得到一个最优模型（这个模型属于某个函数的集合，最优则表示在某个评价准则下是最佳的），再利用这个模型将所有的输入映射为相应的输出。\n",
    "- 无监督学习：它与监督学习的不同之处，在于我们事先没有任何训练样本，而需要直接对数据进行建模。 \n",
    "- 半监督学习 ：在训练阶段结合了大量未标记的数据和少量标签数据。与使用所有标签数据的模型相比，使用训练集的训练模型在训练时可以更为准确。\n",
    "- 强化学习：我们设定一个回报函数（reward function），通过这个函数来确认否越来越接近目标，类似我们训练宠物，如果做对了就给他奖励，做错了就给予惩罚，最后来达到我们的训练目的。\n",
    "\n",
    "这里我们只着重介绍监督学习，因为我们后面的绝大部们课程都是使用的监督学习的方法，在训练和验证时输入的数据既包含输入x,又包含x对应的输出y，即学习数据已经事先给出了正确答案。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2.2 线性回归 （Linear Regreesion）\n",
    "线性回归是利用数理统计中回归分析，来确定两种或两种以上变量间相互依赖的定量关系的一种统计分析方法，运用十分广泛。其表达形式为y = w'x+e，e为误差服从均值为0的正态分布。 \n",
    "\n",
    "回归分析中，只包括一个自变量和一个因变量，且二者的关系可用一条直线近似表示，这种回归分析称为一元线性回归分析。如果回归分析中包括两个或两个以上的自变量，且因变量和自变量之间是线性关系，则称为多元线性回归分析。\n",
    "摘自[百度百科](https://baike.baidu.com/item/线性回归/8190345)\n",
    "\n",
    "简单的说：\n",
    "线性回归对于输入x与输出y有一个映射f，y=f(x),而f的形式为aX+b。其中a和b是两个可调的参数，我们训练的时候就是训练a，b这两个参数。\n",
    "\n",
    "下面我们来用pyTorch的代码来做一个详细的解释"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.0.0'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 引用\n",
    "# 注意，这里我们使用了一个新库叫 seaborn 如果报错找不到包的话请使用pip install seaborn 来进行安装\n",
    "import torch\n",
    "from torch.nn import Linear, Module, MSELoss\n",
    "from torch.optim import SGD\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "torch.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面定义一个线性函数，这里使用y = 5x + 7，这里的5和7就是上面说到的参数a和b，我们先使用matplot可视化一下这个函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1cae5b6e630>]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl4VPXd/vH3F0iAQFhCWAIhhH1LQDHsLigugAgial2KuJVq61NrWyGKC+6gtta2Lg+u2Fo3EhYFVETcFQUK2QgQIEAgJEAgCWTPfH9/ZHgufjRBYGZyZib367pyZTJzJufmzMydkzMnH4y1FhERCV6NnA4gIiK+paIXEQlyKnoRkSCnohcRCXIqehGRIKeiFxEJcip6EZEgp6IXEQlyKnoRkSDXxOkAAJGRkTY2NtbpGCIiAWXdunUHrLXtf245vyj62NhY1q5d63QMEZGAYozZeSrL6dCNiEiQU9GLiAQ5Fb2ISJBT0YuIBDkVvYhIkFPRi4gEORW9iEiQU9GLiDigtKKap1ZsIudQic/X5Rd/MCUi0pB8t+0AiUmp7CooIbptGNNGdPPp+lT0IiL1pKiskqeWb+KdH3cT2y6Md2eMYESPdj5fr4peRKQerMzI44HFqewvLufXF/Tgnov70Cykcb2sW0UvIuJDB46UM2dpOh+l5NKvUziv3JTAoOg29ZpBRS8i4gPWWpZs2MsjH6ZztLyaP17Sh19f0JPQJvV/DoyKXkTEy/YeLuWBxWl8npnP2TFteHrqIHp3DHcsj4peRMRLXC7Lv3/cxdwVmVS7LA9NHMD0UbE0bmQczaWiFxHxgh0HjjIrKYUfdxRwbq9Inroqnq4RYU7HAlT0IiIeqap28eo3O3hu5RZCmzTi6amDuCYhGmOc3Ys/nopeROQMZewtYlZSCql7Crl0QEceuzKOjq2aOR3rv/xs0RtjXgcmAvnW2jj3dRHAe0AskA1ca609ZGp+hD0PTABKgJuttet9E11ExBnlVdX84/MsXvpiG23CQnjhhiFMiO/kV3vxxzuV83zeBMadcF0isMpa2xtY5f4aYDzQ2/0xA3jJOzFFRPzDup2HuPxv3/D3z7OYdFZnVt5zAZcPivLbkodT2KO31n5ljIk94erJwBj35QXAF8As9/VvWWst8IMxpo0xJspam+utwCIiTiipqOKZTzbz5nfZdG7dnDdvGcqYvh2cjnVKzvQYfcdj5W2tzTXGHPvXdgF2H7dcjvs6Fb2IBKxvth4gMTmFnEOl3DSyGzPH9aNl08B5i9PbSWv73cXWuqAxM6g5vENMTIyXY4iIeK6wpJInlmfw/tocekS24P1fj2RY9winY522My36vGOHZIwxUUC++/ocoOtxy0UDe2v7Btba+cB8gISEhFp/GIiIOOXjtH08uCSNgqMV3DmmJ3eP7V1vQ8i87UyLfikwHZjr/rzkuOvvMsa8CwwHCnV8XkQCyf7imiFky1JzGRDVijduHkpcl9ZOx/LIqZxe+Q41b7xGGmNygIepKfj3jTG3AbuAa9yLL6fm1Mosak6vvMUHmUVEvM5aS/L6PTz6UQalFdXce1lfZpzfg5DGgf8f8Z3KWTfX13HT2FqWtcBvPQ0lIlKf9hwu5f7kVL7csp9zurVl3tRB9OrQ0ulYXhM4bxuLiHiZy2X515qdzFuRiQUemTSQaSO60cjhIWTepqIXkQZp2/4jJCal8FP2Ic7rHcmTU/xnCJm3qehFpEGprHbxytfb+etnW2ke0phnrxnM1CFd/PovWz2loheRBiNtTyGzklJI31vE+LhOPDJ5IB3C/W8Imbep6EUk6JVVVvP3z7fy8pfbaRsWyks3DmF8fJTTseqNil5Egtra7AJmJqWwff9RrjknmtmX96dNWKjTseqVil5EgtKR8iqe+TiTt37YSefWzXnr1mGc36e907EcoaIXkaDz5Zb93J+cyt7CUqaPjOXey/rSIoCGkHlbw/2Xi0jQOVxSwWMfbSJpfQ4927fgg1+PJCE28IaQeZuKXkSCworUXB5cks6hkgruurAXd13UK2CHkHmbil5EAlp+URkPLUnn4/R9xHVpxYJbhzKwc2APIfM2Fb2IBCRrLQvX5fDYRxmUVbmYNa4fvzqvO02CYAiZt6noRSTg7C4o4f5FqXy99QDDYiOYOzWeHu2DZwiZt6noRSRgVLssb32fzTOfbMYAj00eyI3Dg28Imbep6EUkIGTlFzMrKZV1Ow9xQZ/2PHlVPF3aNHc6VkBQ0YuIX6usdvG/X27jb6uyCGvamL9cO5gpZwf3EDJvU9GLiN9KzSlkZlIKm3KLuHxQFHOuGEj78KZOxwo4KnoR8TtlldX89bOtvPL1dtq1COV/p53DZQM7OR0rYKnoRcSvrNl+kMTkVHYcOMovErpy/+X9ad08xOlYAU1FLyJ+obiskqc/3sw/f9hJ14jmvH37cEb3inQ6VlBQ0YuI41Zvzmd2ciq5RWXcOro7f7qsD2Ghqidv0ZYUEcccOlrBYx9lkPyfPfTu0JKkO0cxJKat07GCjopeROqdtZZlqbk8vCSdwtJKfje2N7+9sCdNm2gImS+o6EWkXuUVlfHA4jRWZuQxKLo1/7p9OP2jWjkdK6ip6EWkXlhreX/tbh5ftomKKhf3T+jHraM1hKw+qOhFxOd2HSwhMTmF77YdZHj3COZNHURsZAunYzUYKnoR8Zlql+XN77J59pPNNG5keGJKHNcPjdEQsnqmohcRn9iSV8zMhSls2H2Yi/p14IkpcUS11hAyJ6joRcSrKqpcvPTFNv6xeistmzbh+evOYtLgzhpC5iAVvYh4zcbdh5mVlELmvmImDe7Mw1cMoF1LDSFzmopeRDxWWlHNc59t4dWvt9MhvBmv3pTAxQM6Oh1L3DwqemPMPcDtgAVSgVuAKOBdIAJYD0yz1lZ4mFNE/NT32w5yX3IK2QdLuH5YDPdN6EerZhpC5k/O+ARWY0wX4HdAgrU2DmgMXAfMA56z1vYGDgG3eSOoiPiXorJK7l+UyvWv/IAF/v2r4Tx1VbxK3g95euimCdDcGFMJhAG5wEXADe7bFwBzgJc8XI+I+JFVm/KYvSiN/OIyfnVed/5wSV+ah2p8gb8646K31u4xxjwL7AJKgU+BdcBha22Ve7EcoIvHKUXELxw8Us4jH2awdONe+nYM5+Vp53BW1zZOx5KfccZFb4xpC0wGugOHgQ+A8bUsauu4/wxgBkBMTMyZxhCRemCtZenGvTzyYQbFZZXcc3Ef7hzTk9AmGl8QCDw5dHMxsMNaux/AGJMMjALaGGOauPfqo4G9td3ZWjsfmA+QkJBQ6w8DEXFebmEpDyxKY1VmPoO7tuHpqYPo2ync6VhyGjwp+l3ACGNMGDWHbsYCa4HVwNXUnHkzHVjiaUgRqX8ul+Xdn3bz1PJNVLpcPHB5f24Z3Z3GGl8QcDw5Rr/GGLOQmlMoq4D/ULOHvgx41xjzuPu617wRVETqT/aBoyQmp/DD9gJG9mjH3KnxdGunIWSByqOzbqy1DwMPn3D1dmCYJ99XRJxRVe3ijW+z+fPKzYQ0asTcq+L5xdCuGl8Q4PSXsSICQOa+ImYtTGFjTiEX9+/I41fG0al1M6djiReo6EUauPKqal5YvY0XV2fRunkIf7/+bCYOitJefBBR0Ys0YP/ZdYhZSSlsyTvClLO78ODEAUS0CHU6lniZil6kASqpqOLPn27h9W930KlVM16/OYGL+mkIWbBS0Ys0MN9lHSAxOZVdBSX8ckQMs8b1I1zzaYKail6kgSgsreSp5Zt496fddI9swbszRjCiRzunY0k9UNGLNACfpu/jgcVpHDhSzq8v6ME9F/ehWYiGkDUUKnqRIHbgSDlzlqbzUUou/TqF8+r0BAZFawhZQ6OiFwlC1loWb9jDIx9mUFJezR8v6cMdY3oS0lhDyBoiFb1IkNl7uJTZi1JZvXk/Z8fUDCHr3VFDyBoyFb1IkHC5LG//uIt5KzKpdlkemjiA6aNiNYRMVPQiwWD7/iMkJqXyY3YB5/aK5Kmr4ukaEeZ0LPETKnqRAFZV7eLVb3bw3MotNG3SiKevHsQ150RrfIH8f1T0IgEqY28RM5M2kraniMsGduSxyXF0aKUhZPLfVPQiAaa8qpp/fJ7FS19so01YCC/eOITxcZ20Fy91UtGLBJB1OwuYlZRKVv4RrhrShQcvH0BbDSGTn6GiFwkAR8ureOaTzSz4PpvOrZvz5i1DGdO3g9OxJECo6EX83Ndb93Nfcio5h0qZPrIb947rR8umeunKqdOzRcRPFZZU8viyDD5Yl0OP9i344I6RDI2NcDqWBCAVvYgf+jhtHw8uSaPgaAW/GdOT343trSFkcsZU9CJ+JL+4jDlL01meuo8BUa144+ahxHVp7XQsCXAqehE/YK0laf0eHvsog9LKau69rC8zzu+hIWTiFSp6EYflHCrh/kVpfLVlPwnd2jJ36iB6dWjpdCwJIip6EYe4XJZ//rCTeR9nAvDIpIFMG9GNRhpCJl6mohdxwLb9R5i1MIW1Ow9xfp/2PDkljui2GkImvqGiF6lHldUu5n+1nedXbaV5SGOevWYwU4d00fgC8SkVvUg9SdtTyMyFKWTkFjEhvhNzJg2kQ7iGkInvqehFfKyssprnV21l/lfbaRsWysu/HMK4uCinY0kDoqIX8aGfsguYtTCF7QeOcs050Txw+QBah4U4HUsaGBW9iA8cKa/i6Y8zeev7nUS3bc4/bxvGeb3bOx1LGiiPit4Y0wZ4FYgDLHArsBl4D4gFsoFrrbWHPEopEkC+3LKf+5NT2VtYys2jYrn3sr600BAycZCnf3b3PPCxtbYfMBjYBCQCq6y1vYFV7q9Fgt7hkgr+8P4Gpr/+I81CGrHwjpHMmTRQJS+OO+NnoDGmFXA+cDOAtbYCqDDGTAbGuBdbAHwBzPIkpIg/s9ayIm0fDy1J43BJJXdd2Iu7LuqlIWTiNzzZ1egB7AfeMMYMBtYBdwMdrbW5ANbaXGOM/ncECVr5RWU8uCSNT9LziOvSigW3DmNgZw0hE//iSdE3AYYA/2OtXWOMeZ7TOExjjJkBzACIiYnxIIZI/bPW8sG6HB7/KIPyKheJ4/tx+7ndaaIhZOKHPCn6HCDHWrvG/fVCaoo+zxgT5d6bjwLya7uztXY+MB8gISHBepBDpF7tLijhvuRUvsk6wLDYCOZOjadHew0hE/91xkVvrd1njNltjOlrrd0MjAUy3B/Tgbnuz0u8klTEYdUuy1vfZ/P0x5tpZOCxK+O4cViMhpCJ3/P0dID/Ad42xoQC24FbqDmT531jzG3ALuAaD9ch4ris/GJmLkxh/a7DjOnbniemxNOlTXOnY4mcEo+K3lq7AUio5aaxnnxfEX9RWe3i5S+28ffPswhr2pjnfjGYK8/SEDIJLDrBV6QOqTmF3LtwI5n7ipk4KIo5kwYS2bKp07FETpuKXuQEZZXVPPfZFl75ajuRLZsyf9o5XDqwk9OxRM6Yil7kOGu2HyQxOZUdB45y3dCu3DehP62bawiZBDYVvQhQXFbJvI8z+dcPu+ga0Zy3bx/O6F6RTscS8QoVvTR4qzPzuX9RKvuKyrjt3O788dI+hIXqpSHBQ89mabAKjlbw6IfpLN6wl94dWpJ05yiGxLR1OpaI16nopcGx1vJRSi5zlqZTWFrJ78b25rcX9qRpEw0hk+CkopcGJa+ojNmL0vhsUx6Dolvz9q+G069TK6djifiUil4aBGst7/20myeWb6KiysXsCf25ZXSshpBJg6Cil6C362AJickpfLftIMO7RzBv6iBiI1s4HUuk3qjoJWhVuyxvfLuDZz/dTJNGjXhySjzXDe2qIWTS4KjoJSht3lfMzKQUNu4+zEX9OvDElDiiWmsImTRMKnoJKhVVLl78IosXVmcR3iyE5687i0mDO2sImTRoKnoJGht3H2bmwhQ25xUz+azOPDRxAO00hExERS+Br7Simr+s3Mxr3+ygQ3gzXr0pgYsHdHQ6lojfUNFLQPtu2wHuS05l58ESbhgeQ+L4frRqpiFkIsdT0UtAKiqr5Knlmbzz4y66tQvj378azqieGkImUhsVvQSczzLymL04lf3F5cw4vwf3XNyH5qEaXyBSFxW9BIyDR8p55MMMlm7cS79O4cyflsDgrm2cjiXi91T04vestSzduJc5S9M5Ul7FPRf34c4xPQltovEFIqdCRS9+LbewlAcWpbEqM5+zurbh6asH0adjuNOxRAKKil78kstleeenXTy1PJMql4sHLu/PLaO701jjC0ROm4pe/M6OA0dJTEphzY4CRvVsx9yrBhHTLszpWCIBS0UvfqOq2sXr3+7gz59uIbRJI+ZNjefahK4aXyDiIRW9+IVNuUXMSkohJaeQSwZ05PEr4+jYqpnTsUSCgopeHFVeVc0Lq7fx4uosWjcP4R83nM3l8VHaixfxIhW9OGb9rkPMWpjC1vwjTDm7Cw9NHEDbFqFOxxIJOip6qXclFVU8+8kW3vhuB51aNeONm4dyYb8OTscSCVoqeqlX32YdIDE5hd0FpfxyRAyzxvUjXEPIRHxKRS/1orC0kieXbeK9tbvpHtmC92aMYHiPdk7HEmkQVPTic5+m7+OBxWkcPFrBHRf05PcX96ZZiIaQidQXj4veGNMYWAvssdZONMZ0B94FIoD1wDRrbYWn65HAs7+4nDkfprMsJZf+Ua14bfpQ4qNbOx1LpMHxxlSou4FNx309D3jOWtsbOATc5oV1SACx1pK8PodLnvuSlel5/OnSPiy9a7RKXsQhHhW9MSYauBx41f21AS4CFroXWQBc6ck6JLDsOVzKLW/+xB/e30iPyBYsv/tc7rqoNyGNNWlSxCmeHrr5KzATODZOsB1w2Fpb5f46B+ji4TokALhclrfX7GTuikxcFh6+YgA3jYzVEDIRP3DGRW+MmQjkW2vXGWPGHLu6lkVtHfefAcwAiImJOdMY4ge27z9CYlIqP2YXcF7vSJ6cEk/XCA0hE/EXnuzRjwYmGWMmAM2AVtTs4bcxxjRx79VHA3tru7O1dj4wHyAhIaHWHwbi36qqXbzy9Q6e+2wLzZo04pmrB3H1OdEaXyDiZ8646K219wH3Abj36P9krb3RGPMBcDU1Z95MB5Z4Iaf4mYy9RcxM2kjaniIuG9iRxybH0UFDyET8ki/Oo58FvGuMeRz4D/CaD9YhDimrrOYfn2fx8pfbaBMWyks3DmF8fJTTsUTkJLxS9NbaL4Av3Je3A8O88X3Fv6zbWcDMhSls23+UqUOieXBif9qEaQiZiL/TX8bKzzpaXsUzn2xmwffZdG7dnAW3DuOCPu2djiUip0hFLyf11Zb93Jecyt7CUm4a0Y17x/WjZVM9bUQCiV6xUqvCkkoeW5bBwnU59Gjfgvd/PZKhsRFOxxKRM6Cil//ycVouDy5Jp+BoBb8Z05PfjdUQMpFApqKX/5NfXMbDS9JZkbaPAVGteOPmocR10XwakUCnohestSxcl8PjyzZRWlnNvZf1Zcb5PTSfRiRIqOgbuN0FJdy/KJWvtx4goVtb5k4dRK8OLZ2OJSJepKJvoFwuy1vfZ/P0J5sxwKOTB/LL4d1opCFkIkFHRd8AZeUfITEphbU7D3F+n/Y8OSWO6LYaQiYSrFT0DUhltYv5X23n+c+20jy0MX++ZjBXDemiIWQiQU5F30Ck7Slk5sIUMnKLmBDfiUcmxdE+vKnTsUSkHqjog1xZZTXPr9rK/K+2E9EilJd/OYRxcRpCJtKQqOiD2E/ZBcxamML2A0e5NiGa2RMG0DosxOlYIlLPVPRB6Eh5FU9/nMlb3+8kum1z/nXbcM7tHel0LBFxiIo+yKzenM/s5FRyi8q4ZXQsf7q0Ly00hEykQVMDBIlDRyt47KMMkv+zh14dWrLwjlGc062t07FExA+o6AOctZblqft4eGkah0sq+Z+LenHXRb1o2kRDyESkhoo+gOUXlfHA4jQ+zcgjvktr3rp1OAM6t3I6loj4GRV9ALLW8sHaHB5blkFFlYv7xvfjtnO700RDyESkFir6ALO7oIT7klP5JusAw7pHMPeqeHq01xAyEambij5AVLssC77L5plPNtO4keHxK+O4YViMhpCJyM9S0QeArXnFzExK4T+7DjOmb3uenBJP5zbNnY4lIgFCRe/HKqpcvPzlNv7xeRYtmjbmr784i8lnddYQMhE5LSp6P5WSc5iZC1PI3FfMFYM78/AVA4hsqSFkInL6VPR+pqyymudWbuGVr7fTPrwpr9yUwCUDOjodS0QCmIrej/yw/SCJSSlkHyzh+mFdSRzfn9bNNYRMRDyjovcDxWWVzF2RydtrdhETEca/bx/OqF4aQiYi3qGid9jnmXnMXpRGXlEZt5/bnT9c2oewUD0sIuI9ahSHFByt4NEP01m8YS+9O7TkxTtHcXaMhpCJiPep6OuZtZYPU3KZszSd4rJK7h7bm99c2FNDyETEZ1T09WhfYc0Qss825TE4ujXzrh5Ov04aQiYivnXGRW+M6Qq8BXQCXMB8a+3zxpgI4D0gFsgGrrXWHvI8auCy1vLuT7t5ctkmKl0uZk/oz63ndqexxheISD3wZI++CvijtXa9MSYcWGeMWQncDKyy1s41xiQCicAsz6MGpp0Hj5KYlMr32w8yokcEc68aRGxkC6djiUgDcsZFb63NBXLdl4uNMZuALsBkYIx7sQXAFzTAoq92Wd74dgfPfrqZkEaNeHJKPNcN7aohZCJS77xyjN4YEwucDawBOrp/CGCtzTXGdKjjPjOAGQAxMTHeiOE3Nu+rGUK2cfdhxvbrwONT4ohqrSFkIuIMj4veGNMSSAJ+b60tOtWBW9ba+cB8gISEBOtpDn9QUeXixS+yeGF1FuHNQvjb9WdzxaAoDSETEUd5VPTGmBBqSv5ta22y++o8Y0yUe28+Csj3NGQg2LD7MLMWprA5r5jJZ3Xm4SsGEtEi1OlYIiIenXVjgNeATdbavxx301JgOjDX/XmJRwn9XGlFNX/+dDOvf7uDDuHNeG16AmP7awiZiPgPT/boRwPTgFRjzAb3dfdTU/DvG2NuA3YB13gW0X99t+0AiUmp7Coo4YbhMSSO70erZhpCJiL+xZOzbr4B6jr4PPZMv28gKCqr5Knlm3jnx910axfGO78awcie7ZyOJSJSK/1l7Gn6LCOP2YtT2V9czozze3DPxX1oHqrxBSLiv1T0p+jgkXLmfJjBhxv30q9TOPOnJTC4axunY4mI/CwV/c+w1rJkw14e+TCdI+VV/OGSPtxxQU9CmzRyOpqIyClR0Z/E3sOlPLA4jc8z8zmraxuevnoQfTqGOx1LROS0qOhr4XJZ/v3jLuauyKTaZXlw4gBuHhWrIWQiEpBU9CfYceAoiUkprNlRwOhe7XhqyiBi2oU5HUtE5Iyp6N2qql289s0O/rJyC6FNGjFvajzXJnTV+AIRCXgqemBTbhGzklJIySnkkgEdefzKODq2auZ0LBERr2jQRV9eVc0Ln2fx4hfbaBMWwgs3DGFCfCftxYtIUGmwRb9u5yFmJaWQlX+Eq87uwoMTB9BWQ8hEJAg1uKIvqajimU828+Z32US1asYbtwzlwr61jswXEQkKDarov9l6gMTkFHIOlTJtRDdmjutLuIaQiUiQaxBFX1hayRPLMnh/bQ7dI1vw3owRDO+hIWQi0jAEfdF/kr6PBxencfBoBXeO6cndY3vTLERDyESk4Qjaot9fXM6cpeksS82lf1QrXps+lPjo1k7HEhGpd0FX9NZaktfv4dGPMiitqObey/oy4/wehDTWEDIRaZiCquj3HC7l/uRUvtyynyExNUPIenXQEDIRadiCouhdLsu/1uxk3opMLDDnigFMG6khZCIiEARFv23/ERKTUvgp+xDn9Y7kySnxdI3QEDIRkWMCuujf/2k3DyxJo1mTRjxz9SCuPida4wtERE4Q0EXfvX0LxvbrwCOTB9IhXEPIRERqE9BFPzQ2gqGxEU7HEBHxazrnUEQkyKnoRUSCnIpeRCTIqehFRIKcil5EJMip6EVEgpyKXkQkyKnoRUSCnLHWOp0BY8x+YOcZ3j0SOODFON6iXKdHuU6fv2ZTrtPjSa5u1tr2P7eQXxS9J4wxa621CU7nOJFynR7lOn3+mk25Tk995NKhGxGRIKeiFxEJcsFQ9POdDlAH5To9ynX6/DWbcp0en+cK+GP0IiJycsGwRy8iIicRMEVvjBlnjNlsjMkyxiTWcntTY8x77tvXGGNi6yFTV2PMamPMJmNMujHm7lqWGWOMKTTGbHB/POTrXO71ZhtjUt3rXFvL7cYY8zf39koxxgyph0x9j9sOG4wxRcaY35+wTL1tL2PM68aYfGNM2nHXRRhjVhpjtro/t63jvtPdy2w1xkz3caZnjDGZ7sdpkTGmTR33Pelj7qNsc4wxe457vCbUcd+Tvn59kOu94zJlG2M21HFfn2yzurrBseeXtdbvP4DGwDagBxAKbAQGnLDMb4CX3ZevA96rh1xRwBD35XBgSy25xgAfObDNsoHIk9w+AVgBGGAEsMaBx3QfNecBO7K9gPOBIUDacdc9DSS6LycC82q5XwSw3f25rftyWx9muhRo4r48r7ZMp/KY+yjbHOBPp/BYn/T16+1cJ9z+Z+Ch+txmdXWDU8+vQNmjHwZkWWu3W2srgHeByScsMxlY4L68EBhrfPwfyFprc621692Xi4FNQBdfrtOLJgNv2Ro/AG2MMVH1uP6xwDZr7Zn+oZzHrLVfAQUnXH3882gBcGUtd70MWGmtLbDWHgJWAuN8lcla+6m1tsr95Q9AtDfWdbrq2F6n4lRevz7J5e6Aa4F3vLW+U8xUVzc48vwKlKLvAuw+7usc/rtQ/28Z94uiEGhXL+kA96Gis4E1tdw80hiz0RizwhgzsJ4iWeBTY8w6Y8yMWm4/lW3qS9dR94vPie11TEdrbS7UvFiBDrUs4+S2u5Wa38Rq83OPua/c5T6s9HodhyKc3F7nAXnW2q113O7zbXZCNzjy/AqUoq9tz/zE04VOZRmfMMa0BJKA31tri064eT01hycGA38HFtdHJmC0tXYIMB74rTEm3xNmAAACcklEQVTm/BNud3J7hQKTgA9qudmp7XU6HNl2xpjZQBXwdh2L/Nxj7gsvAT2Bs4Bcag6TnMix5xpwPSffm/fpNvuZbqjzbrVc59H2CpSizwG6Hvd1NLC3rmWMMU2A1pzZr5mnxRgTQs0D+ba1NvnE2621RdbaI+7Ly4EQY0ykr3NZa/e6P+cDi6j59fl4p7JNfWU8sN5am3fiDU5tr+PkHTuE5f6cX8sy9b7t3G/ITQRutO4DuSc6hcfc66y1edbaamutC3iljnU68lxz98BVwHt1LePLbVZHNzjy/AqUov8J6G2M6e7eG7wOWHrCMkuBY+9OXw18XtcLwlvcx/9eAzZZa/9SxzKdjr1XYIwZRs02P+jjXC2MMeHHLlPzZl7aCYstBW4yNUYAhcd+pawHde5lObG9TnD882g6sKSWZT4BLjXGtHUfqrjUfZ1PGGPGAbOASdbakjqWOZXH3BfZjn9fZ0od6zyV168vXAxkWmtzarvRl9vsJN3gzPPL2+82++qDmrNEtlDz7v1s93WPUvPkB2hGzaGALOBHoEc9ZDqXml+pUoAN7o8JwB3AHe5l7gLSqTnT4AdgVD3k6uFe30b3uo9tr+NzGeAF9/ZMBRLq6XEMo6a4Wx93nSPbi5ofNrlAJTV7UbdR877OKmCr+3OEe9kE4NXj7nur+7mWBdzi40xZ1ByzPfYcO3Z2WWdg+cke83rYXv90P39SqCmxqBOzub/+r9evL3O5r3/z2PPquGXrZZudpBsceX7pL2NFRIJcoBy6ERGRM6SiFxEJcip6EZEgp6IXEQlyKnoRkSCnohcRCXIqehGRIKeiFxEJcv8Pye3TH8lSsHMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x=np.linspace(0,20,500)\n",
    "y=5*x + 7\n",
    "plt.plot(x,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我生成一些随机的点，来作为我们的训练数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = np.random.rand(256)\n",
    "noise = np.random.randn(256) / 4\n",
    "y = x * 5 + 7 + noise\n",
    "df = pd.DataFrame()\n",
    "df['x'] = x\n",
    "df['y'] = y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在图上显示下我们生成的数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\envs\\pytorch1\\lib\\site-packages\\scipy\\stats\\stats.py:1713: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.\n",
      "  return np.add.reduce(sorted[indexer] * weights, axis=axis) / sumval\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<seaborn.axisgrid.FacetGrid at 0x1cadecb7780>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAFgCAYAAACFYaNMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3Xm83HV96P/X57vMdmbOlnNO9kACgSCbYEToRYxor9jaopZWY63WFsFeK623VPTW7YIL/LS/tlp6S1xq1V68GrXQFlxpinhJISSELYHACWQjOfs5M2e27/K5f3xnJjNzZs6Zs2/vp+VBM2eWb4K8/cz7+16U1hohhBBzz5jvCxBCiOVKArAQQswTCcBCCDFPJAALIcQ8kQAshBDzRAKwEELMEwnAQggxTyQACyHEPJEALIQQ88Sa7wtoxDXXXKN/9KMfzfdlCCFEo1QjT1oUJ+C+vr75vgQhhJhxiyIACyHEUiQBWAgh5okEYCGEmCcSgIUQYp5IABZCiHkiAVgIIeaJBGAhhJgnEoCFEGKeSAAWQoh5IgFYCCHmiQRgIYSYJxKAhRBiBmmtG36uBGAhhJghWmtOjeQafr4EYCGEmAGerzkxnGXfkcGGXyMBWAghpsn1fE4MZXjoUC8f/u7jDb9uUQxkF0KIhSrv+pwcznLfkyf4wk+ew/MbzwFLABZCLCm7DvZw14PdHB1Ms74txo1XbWLblq5Z+ays43FyOMM//ecRvvKLwwBs6mxq+PWSghBCLBm7DvbwyXufpieZpTVq05PM8sl7n2bXwZ4Z/6x03uX4YIa/+fnzpeD7yvWt/PU7Xtnwe0gAFkIsGXc92I1tKmIhC6WCv9um4q4Hu2f0c1I5l6MDaW7912f44b7jALz+3E5uf/uFrGmNNvw+koIQQiwZRwfTtEbtiseitsmxwfSMfcZw2uGl/lE+cc9T7D82DMDbL13Lf9t2Fp3xCC0xe4J3OE0CsBBiyVjfFqMnmSUWOh3aMo7HurbYhK9tJHc8MJrn0KkkH/vBk3T3jQJw41WbeMer19PVHCEenlxIlRSEEGLJuPGqTTieJp130Tr4u+Npbrxq07iv23Wwh5t37mff0UFOjWTZd3SQm3fur8gd9yZz7D86yIfu3kd33yimofjYm7fwrtdsYE1rdNLBFyQACyGWkG1burj1N8+nKxFhOOPQlYhw62+eP2EVxO33H2Ao7aB9MJVC+zCUdrj9/gOF7rYsD7/Qx03feZyeZI6obfL5t13Ar124mtUtUSK2OaXrlRSEEGJJ2bala9JlZ4f70xgKDEMBoBRoX9PdN8rLw1l+9swpPnPfAfKuT1vM5vNvv5AL1rawqjmCZU79HDtrAVgp9XXgLUCP1vqCwmNfAH4DyAMvAO/TWg/N1jUIIZaXmawB1lqjlOJ7e47yNz8/hK9hbWuUO37rQs7uStCVCJcC9lTNZgriG8A1VY/9FLhAa30R8BzwsVn8fCHEMjKdGuBNHU34Gnyt0Wg838fzIRay+KufBcH33FUJvrz9lZy7qpmVzdMPvjCLAVhr/SAwUPXYT7TWbuGXu4F1s/X5QojlZTo1wLdcs4W2mI0CHNdHa41lKoYyDgCXbWzn//+di9nUGaczEUap6QdfmN+bcH8A3F/vh0qpG5RSe5RSe3p7e+fwsoQQi9HRwTTRqpthyYzD7u5+zvof93HRp3/Ml372XM3XbtvSxReuu5iL1rXSEQ8RDVnkvWCmw5vOX8ln33oBG9pjtMZCM3rN8xKAlVJ/AbjAP9V7jtZ6h9Z6q9Z6a2dn59xdnBBiUVrfFiPjeKVf94xk6UnlAbCMoB74bx54vm4Q3rqxnf/xa1tYEQ8zkg2+qP/uazbw0Wu2sK4tRiLSeINFo+Y8ACul3ktwc+539WRGxwshxDiqa4B7ksFgdMtQGMrAMgwMBV996PCY1w5nHJ44OsSH7t7HMy8nUcCHrj6bG686izVtUaKhqZWZTWROy9CUUtcAtwCv01rPXG+gEGLZ27ali1sJcsHHBtNogpNveZmYoWA071W8bnA0z54XB7jlB08yMJrHNhV/8Wvn8cZXrJx2mdlEZrMM7W5gG9ChlDoGfIqg6iEM/LSQxN6ttf7AbF2DEGJ5Ka8BvujTP65ISQD4GprKTrN9qRy7nu3hU/c8zWjeIx62uO2t53P5phWsTERmpNJhPLMWgLXW22s8/LXZ+jwhhCh3/ZUb+ZsHnsf1fQxFocwseFxrzT37jvO5+w6U8sSmobj+yo38l7M76IzPXKXDeKQVWQixJN30xnP4k6vPJmqbuH4wFe1Prj6bP756Mz/ce5yP/fDJUvCFoPPtW7tf5JnjI3MSfAHUYrgPtnXrVr1nz575vgwhxDyaiS63YHFmhrff+X/pTQU36RRgm0HANZTikg1t3H3D5dO93IYiuJyAhRAL3kxsunA8n5f6R/nQ/95XCr4ApgoCr2EoXN+f0dnBE5EALIRY8Kq73Dxf0zOS5cZvP8b2HbsnDMQ51+OFnhQ3fPMxHj9aOX7GLeSGtQbLMBqaHTxTZBqaEGLBK990kcw6nBjKAhoNHO5LceO3HyNsBedJ21Slm2jJnMualihvekUX39lzjOd7UwAkIhaprEsxAZv3fCxD0RazJ5wdPJMkAAshFrzyTRe9yRxKAVqh0PSP5vG1Jpf1sU3FaF4zOOpgGIrVLWGODIzyufufxSvc7+pKhGiLhcnGXE4OZ8l7QSDf3BXnlmu2zNoG5VokAAsh5sxUb6TdeNUmPnnv06TzLjnXw1AKTSF3qykFV8swyBZqf11fc3QwW3oPQ8EZ7U34OgjUIStEczREOu/SlYjMxI23SZMcsBBiTkznRlr5pgvTMDAMxZqWKL7WqEKNb7FnQhf+qtYUMvnTN25Go8g43qRWFs0WCcBCiDkx3ZXx27Z0cfcNl3PXu19FVyKCZSpCpoGnNYrgNFxPocqMlqiN73l0943y1IkRXh7Oct2la+c07VBOUhBCiDkxUyvjy2c+DKfzuL6mLWaRzLrkXW/M8y1FoSpXc9PdexnJnX5OzvH41u6XAHi4e2BGNmlMhgRgIcScmM7K+GrlMx92Hezh7//jBZ48PsRovjL5YCoIWQbNUZuRjFuaDRHcxANPw0A6z527XmBdW7QiNXJr4XNmk6QghBBzYqor4ydy1TmdvPWSNXj+6ccUQXBb2xphdWuUkGXieH5lblgFz/P8oENuqqmR6ZAALISYE1NdGT8e1/M5eGqE2/71AFk3iMBtUZs1LWFsU3EqmS99jmGouv3BxRrioqmkRqZCUhBCiDlTb2V8dXnaFZvaJ8zJ5l2fJ44NcfP39pdm/HbGQ6yIh4OmiqYwwxmnVF62cUWM506lgiqJqjKJTN6juzdFRzxMc9SecmpksiQACyHmVbE8zTYVrVGbF/tTPPLiAJ3xEB3xMD3JLDfv3E9nPEwy57K+Lcb7fuVMQrbBzd/bT18qjwLamuxS8FVKkc67FUH0o28+j5t37i/cuDtdqhYPGeRcTd7zOTGcIed6hCxzTkrTJAALIeZVeXkawEAqj+9rTo3kSGZdYiGTobRDKutydleckyMZPvbDJxnNu2Qdn1jI5F2v3sD9T5/E8Xwsw6yZX962pYsvXndxaWPGcMahKWzSEY+QzDr0JnNkXY903uP2t18kVRBCiKWvvDxtJOOQ807nB1xP05fKY6igYsHXkHV8+keDOb7tTSHu+K0Lec2mFVy5uaMUXNfVSVuUp0CuvOOB0ucmIjaJiI3WmuGMM2d1wRKAhRDzqrw8rS+VK90oUwoMQ6G9IPCGDUVfMkdPYZSkaSju3H4JF29oJRay6uaXG/ncornK/RZJFYQQYl6Vl6flPR/TCPKzimCMJAS/djy/FHwVsK4lwqVntlUE0Kl+7ny1JUsAFkLMq/LyNEMpLMOgNWqXdrgVOYU632Jj22je5eHn+2fkc2eqLG6yZCWREGJWTWYCWmlgz0gWX2u0r3F15XCdiKVYWVgXP19TzBrQ0EoiyQELIWZNdYnZRG2+xTkPN3xrD56v8TkdfI1C59rZXQmUUmitxzRLzMTeuLkkAVgIMWUTBbw7fnSQnpEsntaETIPORLjU5lsvMF51TiebV8Z59mQKXchB2IZCowlbZmljcfUNs0aD/UIK0hKAhRCTUgxgh3qSJLMubTG71DBRHvB2HezhuZ4UpgoqFlxfc2IoS2vU4thghivveGBMAHQ9nzsfeJ6DL6dKQ9bbYha2adA/6tActdBak3G8MTfMquuJYyGLdN6tCPaTPZHPNgnAQoiGlQewdM7F18FKoLBl0hy1KwJeMSBqHxQKpcDxfXpSeSK2MSYAup7PJ+55mpdHgi0WCjANGM64nNMV592vOYOHuwcq6nwBtu/YzdHBNL3JHKuawwClxoqc63FsMMOugz0V1zRekJ5LEoCFEA0rD2COr4OTrac5OpjGHFbYhmI44wDw3KkRtK/JeRrlgWUE24wBViYiJLMufakcOdfnA9/eg2kYpZkOEOR8V7dECVkGbU1hbnrjOdxUdi3Vp9m+VI7jQ1myjs9g2gnqiFUQ+ItBfqZmEs8UKUMTQjTs6GCaqG0CBNsofI3ra3wdpBkcX5PMunzpZ8+Rynn4gF2IMo4fLL8MFX59YjiD62kMpcm6uhR8FRAyFaZSDIzm6wbI6g0bKxMRAHqSOUAX/4+ViUgp77y+LVaaCVw0180X5SQACyEaVh7AOhNh3ELbsKFA+0Hdru/7/NXPDqG1xvc1hmEQtg1sMziNrkiE6UvlMAimort+5WdoIO9p8n4QlJ/vTdEUMsdcS/n/GAA0R23WtkZKO+EsM9gb1xy1S0F8ITRflJMALMQysutgD9t37ObKOx5g+47dDS3ELFcewOJhC6XKCl4VaK2DErHiQ0qB1ni+xjYUibCFbZrkXB9f++QLp+fx5AuzH6qvtdZp1jINmiMWG9pjbOqM01xINxRPuQuh+aKcBGAhlonpbCUuqg5g8bDFyuYw569pwTIUtmGglApqdlWQRlAqWJ6ZK6ysuO7StYQtheOfnssbKzvhVncwFFcKVW+oqHeavf7KjeOecovLPX9xy9XcfcPl81onLJ1wQiwT23fsHjN8Jp13sQvDyyeqi61VPwuUboQdGUgXgqeiLWYzmHbwtY/rg11YS7yiKQQo8q7HQDq4WZcIm+Qcj7xPISkRpDQsw0DrIMdsGmAaBne9+1U1a3qrJ6DVe3wONdQJJwFYiGWiOH5Rla1vH8nkOTaU4cwVTURts1RfW/21vLzioPp5ENwQ23tkEAWsaomQiNgksw4v9Qc3z2IhkxVNIUzT4MhAGs/XGCoIyP2jeUKWgdbga43raSxToVDkPR+lgkYMw1DznjKYhIYCsKQghFgmauVMTyVz2IYx4ULK6oqD8ucVv9Lf9e5X0dUcwTSCNmHTUBgGbGiPsrGjCZTipf5RPF+jgM+/7UI2dsTpSoQxlcLxfFwv+JnjBRsqAEzGVjMsFRKAhVgm6uVMVxaaF4rKy76KN+0eeXGAl4cyJLNOzedB7elimzvjmIZiMO1wdCCNr4P0wuauOG+9dC0v9CbpH83jekGrsqlOj6IM6nghbJtjqhmWCmnEEGKZKA66Kc+NhkyjdNIsKlYMlKcdIlbwvBNDWda0BhskatXPVg9F/9kzJ/mz7+1nOOMCQa3wipjNR645l7Bl4hTL2IzgG7ttmSjfJ2qbnL+mhcN9KZJZlxPDGfpSORIRi40d8Vn8U5pbEoCFWEaqA2QxyKbzbkVu98arNlWkHTriYU4MZ9BoekayQdPFBPWzWcfl4e7+UvC1DMV5qxN8+A3ncPUrVgJBhUMm7+FrTaFiDXTw+BWb2nnkxQGMwkk47/n0pvK867L2Wf0zmksSgIVYxmqdiosVAx+/56lS226xnrYvlSPr+nQlImMqC8qrJFYXcsG7Dw8AQcqhKWTRm8rylYcOYxiKbVu62NyV4MX+FCOZYBtGyDRobrI5c0Wch7sH6EqEKn8WtXi4e6CiJXkxkwAsxDJXb5da9c605qiNZaqaQ9DL0xVNIZOnTgyTKaywuGR9K6cKA9bbYqGKATw3XrWJT977NKtaLKK2SV8qx8Cog+MFk9ZWNYfZ1Hk65VBrBvBiJjfhhBA11bppN5JxGBzNjemkK879fbFvlEM9qVLw7YqHMRVEQ2bdCorijbuTwxkG0w7tTTarmiMoBceHshU3/uZzbsNskAAshKipuqohZBrBckxfV3TSfelnz/FcTwrX8yv2uDVHTGxLcTKZq5jZAJUVFMUyts0rm1nXFqUjHqkYrnNyOLsg5jbMhllLQSilvg68BejRWl9QeOy3gU8D5wGXaa2lu0KIgoW0qaGoPD2xfcdu8p4/ZpbuVx86jKF0aWkmgKkg5/icv6YJYNz178Xf9yMvDhA2FV3NEbQO8s2+DsZZnhzJsrkrsSD+TGbSbOaAvwH8LfDNsseeAt4O3DWLnyvEojNXmxqmE+RrzdKNWAbJrFvxmGUEdbyu1hXtyrUqLWqVuh0dSJfmSFiGwiikLZZa8IVZTEForR8EBqoeO6C1fna2PlOIxWq8TrOZMt1hPNWddFprTiVzpfGPhoKwFdTzmqbB5s546QRdbwJZdakbBCkMz9eFZl7Fqpal1wFXtGCrIJRSNwA3AGzYsGGer0aI2TUXmxqmu46nWLGQzrtELIMTw1kGCwN1TEOxKhGiNRYi6/o4nuajbz6v9Np6lRblv+9iqduRgXQwz9dQdCbCJCL2kqt+KFqwN+G01ju01lu11ls7Ozvn+3KEmFX1NjU0hcxpze8tVz3AHCYX5IsnWRPNgZPJUvDd0B7jM9eez4YVcUay7qQG5lT/vpujNmHbIGIZbOqMk4hUzvNdahbsCViI5aT8dFnMkw5nnGAwTVXVQfnW4cnkc6vreqGy7biR98o4HkeHsqVKh4ht4Ho+q1uiY2qDp/r7joctFNTMGS81C/YELMRyUitP2hkP0xy1a+aFp5LPrTeM54pN7Q29V38qx0e+/0SpxrclarFxRRPRkDnl/Gyt3/cXr7uYL1x38YLZWjGbZm0esFLqbmAb0AGcAj5FcFPuy0AnMAQ8rrV+00TvJfOAxXJUa36v1prhjMO6GqfZdN6t2aVWrtag8rse7K75XiHToDUW4uhgms54mN5UjmODGYCgWSIRwTSN0jX94parZ+FPYdFqaB7wrKUgtNbb6/zoh7P1mUIsJeOlDKZ6067WzbDymQ9FrudzuG+UkGXguH4p8AK0x2xWt0QxCv/DsFTzs3NBUhBCLFDjbfCdyfXqtd7r2GAGX0PW8fHKviS3Ri2awhZZx1uy3WlzSQKwEAvUePWzM7levfq9epPZiqBbFNwY87jt2guWRX52LshOOCEWkfJqhXjIRClFKudOe/FkeW54OOMwUtXdVhS2DJ79zJun81tYLuY3ByyEmFnV7cpBeZbPbddeMO0TaDE37Ps+F9/607rP29TRNK3PEZUkBSHEIjHb7cqO6/PRHzw5ZrZDkQLefMGqGfksEZATsBCLxGy2K4/mHN7ypYc43F/7vRRB3e/Ovce5aF2r5HxniJyAhVgkZrLyoVx/Kscb/vLBcYNvRzzE+vamJTsUZ75IABZikZjJyoeil/pH+Z27HubkSLbi8bLeD0KmQTofBP6lthZ+vkkKQohFYrwFmlPx5PEhrv/GHk4lc0AwRL1m+VlhIzFM/cS9EIfNLwQSgIVYROqNdZysXzzXywf/915Gsi4RO/gi7Ho+BqdXChV5WhdOwVM7cc/VsPnFSFIQQiwz/7zvGH/4zT2MZF1aozY7fm8r/+11Z6EpnzkR/D0eNjGUImobU266mIth84uVnICFWARm6iv8Vx58gc/ffxBfw5rWCH//7ldx0bpWrjqnk8N9Ke594mQp+sZCJheubZ12umAuhs0vVhKAhVjgZuIrvO/7fO6+g3z1ocNAEAA9X/P5+w6WUgqPHRnmzBWxMTN4p5smGG+o0HInKQghFrjpfoXPOx4f/u7+UvANWwZdzSE64+FSML/9/gOzliaYjeqNpUICsBAL3HRWCaWyLn/wj3u45/ETQDBKck1LmOZIqCLQHu6f3rqi8Yw3VGi5kxSEEPNgvJxu9c/ioSAlMNmv8L3JLO/7h0d56sQIAL/7mg38+8FTNIXH5mOL7zlbaYKZqt5YaiQACzHHxsvpAmN+NpJxKFaGNbojrbs3xTt37KanUOPbEQ+x53A//aMOA6MOq1oiFQsvN3U0MZr3lsUetoVExlEKMce279hdd50QMO56oGIDxhWb2nm4e6DmCXr/0SF+72v/WRop2R6zgwWfStEWsxhMB4+vbY1gmQaOp7n1N88HZq7JQ8g4SiEWpPHKsjTU/NlwxuH+P70KGP8ErYEPfPsxcm7QuRY2FaM5F8s08Dyf/lGn9L4nhrJsPbO9ItBKwJ1bEoCFmGET1exOVJY13s92Hezhpu/sYzTvErFMOhNhEhGbdN7ltn97hhf703iFVjbLAFDkPB8DjV92jQrwmH6ZmbQYT49UQQgxgxpZFz9eWdZ4Pyu+dzrvYRkK19ecGMqSzDoksw4v9I7i+RrLUEQshWkYGIZCwZjgC0HL8R0/Ojirv1cxPgnAQsygYs2u62kO941yZCBNTzLL7fcfKD1nvLKs8X5WfO+wZYBWha3EmmODaXqSeQBsU3FWZxOrWqJoDb7WhZNwIHgFpZt6B08lpxwwpcV4+iQFIcQMOjqYxlRwYjiLgcJUCt/XHOpNsetgT0Wutd5X9Xo/K+aOO+JhTgxn8F0fj9PDc85f3UwsZDKQzpOI2Kxphd5kjmzZCOHqW+5aM+XBONJiPH1yAhZiBq1vi3EqmcNABV//VfCXbRiTOhnuOtjD9h27ufKOB9i+Yze7DvaUBrI3R21WJsIVwffyje18/4+u4IOvP7uUwoiHLZqjFiiFWeeefMQypnxqna0B8cuJBGAhZlAxh1v8j681WsPK5nDDJ8N6udUrNrXjeJqRTJ7eVL4UfK85fxX/dP1riISsMSmM0ZxHZzzEurYYRlUQtgzFqpbIlE+t0mI8fRKAhZhB27Z0cU5XHEOp0g2xNYV620ZPhvVyqw93D3DjVRt5eSRXKjN7x9Z1/K93X4ppnv5XeduWLu6+4XJ+ccvVNBdSFs1Rm/VlQVgB69qiJCL2lE+t0mI8fZIDFmKG3XLNFv58536SWZec63FyOEsiYvGJX39FQ6+vl1t9oSfJF34yQt71CVsGt7/9Qt526bpx36u85K05arOeGMeHMliGIh62pn1qlRbj6ZEALMQs0AAKlFKgxt78Gk+tOuHeVI6+0SDt0ByxuPNdl/LaczonfK8rNrVz564X8HxN2DJIRCxaYzad8TDDGUc63uaZBGAhZthdD3bTErVZ3RItPZbOu9z1YHdDge7GqzYV6n2DuQwnR7L0pYIys5XNYb7ynq1ctK51wvfZdbCHnXuP095kM5x2yLoeblrzwW1ncdMbzxn3ddJcMTckByzEDJvO+Mhi8EvnXXpGsrzQmyoF36htoIDP33ewodrdYi65Ix7hrK4Er1jdwrq2KA93D4z7+dJcMXfkBCzEDGt0A0T1SfOKTe18c/dLpHIuruej9ektxSHTYGUiTDxyOihed2yo7kAemFqdbvkNQIBYyJrU6V1MjpyAhZhhjZRn1TppfumBQwyO5vFcH9+vWhGvfSjUFMdCFo7nceeuF8Y9qU6lTnc6p3cxeRKAhZhh1eVZtqFoCpl8/J6nSk0Vd/zoID0jWY4MpDncF8xwcAtB19WVsxsAUKo09wFgOO3g+XrcNuCp1OlKc8XckgAsxCwo1uLedu0FpB2fvOeXTqp/vnM/B08m8bXGLBuqU1SrYkKhUCpoLQbIeUEpWlEy6/DyUIZHXhwoBfmp1OlKc8XckhywELOoVk71+GCGoDpNlQKr63tjXlscnGObCh+N0pBzfdJ5F8sISsogCL4nhrJoNBHLGLM1eTK5221burgVGcw+VyQACzGLat0Ic30fpYKgig++9nHG5ByC4GsqWFMoZzuVzKK0oisR4dqL17Bz7/FStYQunJs74uFp3ziT5oq5IwFYiFlUqyLCMgxQsDIR4eXhTCn4GgriYYu85+N4Gt8P5kiAxjKNMSmEi9a1cteD3bzYnyZiGaWWY5AbZ4uF5ICFmEXFnGpfKssLPUmeeXkYT2tChiKVc8iXlzpoyHs+KxMRXrG6mQ3tMUK2wcmRXM38bTHPfNmZ7axqiZSCL8iNs8VCArAQs2jbli6uu3QtA6NBsI1YJu0xm7yvS/vZFNDZZGMY4Lg+RwbSPPPyCH2pHCsTYboSYe6+4fK6aYHyG2cjmTyHepK82J9mKJ2XBooFTlIQQsyw6gaLowOjwU03BVprkjmPbCHv0ByxaG+ySURCjGRHyBVmTPq+xvU0x4eynN3ZNO7nFW+c3X7/AV7sz2AbButaI+Q9f8rD1sXcmLUArJT6OvAWoEdrfUHhsXbg/wBnAi8Cv6O1HpytaxBiLuw62MPt9x/gcH8a39egYEVTiI54mMN9KU6O5LAMMJUi4/ilMrOobRAPm8TDQepAFVYMlcb2FsoggsfHV1xZdGahNrhIutgWttlMQXwDuKbqsY8CP9dabwZ+Xvi1EIvWroM93LxzP8/3jqK1xvE1jqfpTeVJZl2SWRcA14e8p0vB11Rw0doWzlgRLzU+OIXTb3Fnm+P5tMUsUjm3oWuRLrbFZ9ZOwFrrB5VSZ1Y9fC2wrfD//yOwC7hltq5BiNl214PdpHIupiqsIPKCYOr7mr5UjrznYwIepxssDAUoxR9tOxsIdrL1pbKldfJFvoaBUYfNXWFg+uvuxcIz1znglVrrlwG01i8rpep+L1JK3QDcALBhw4Y5ujwhxlcdBA/1JPF8jVmWJiiG0XTewzSC4FtkqGCwzsaOplLwvBW46Tv7Kj6n+G5eIQVRnB1hm6pi9kN5frd6jGXG8aSLbYFbsFUQWusdWuutWuutnZ0TD54WYrbVGqCTzLpBqlaD5+uKNmJNkHqAIKCe0R5lY0cTXc0RbrlmS8V7F1cMFZ8LwU07U0Eq5za0Al5WBC0+c30CPqWUWl04/a4GpEZGLBq12or5L8Y9AAAgAElEQVTbYjZ9qWA+Q3UKochQ8IrVzaWgWJ46KAZ1xenWYwDbNArtyrCuLdbwaEnpYltc5joA3wu8F7i98Pd75vjzhZiyWkGwIx7G9Xw64mEOnkqhCAJu+Rzf9iabf73ptTXfsxjUV7VEODqQxtNBEM57PpahaIvZ3HjVJu56sFvyu0vQrKUglFJ3Aw8D5yqljiml/pAg8P6qUuoQ8KuFXwuxKNQb1bh5ZTM/+vDruHxjO4mIVQq+UdtkdUuYjR3xuu9ZrFxIRGzWt8cIm6qUgtjcFecL113Mti1dMqVsiZrNKojtdX70htn6TCFm03g3ufKuj+NrRgplZ/GwSUc8hK/VhPN3iyfbRMQmscomnXfpSkS4+4bLS8+TKWVLk9J6Mvta58fWrVv1nj175vsyxDJV3WhhWwZmYeOxbSo2dcYZyTg8eyoFQGc8TMiEDSvidYNksZriuVMjpHIe7U02K5rCpaAuN88WvYm7Z5BWZLEMTGfLb7HRYijtYBTaiXOOhwa6EmGaIxb7jgzhFm7ArWmJ8Nm3XoBSQYXCx+95ivUPxmreeLNNxeqWKH2pHAOjDo6n2dyVkJPtMiInYLGklQe78rTBeCfM8oA9knHI5F0MZWAYwaEmWwjAEUvheOAV/h2yDdiwoonhjIPj+eQcH9f3S8PTi/nc7Tt2j7mhVivtIBa1hk7AC7YOWIiZ0Ej9bLnqWt/RvItTGJpeVDyyZF1dCr6WARTefyTjMJxx0YBlGmhgMO1wx48OAtIyLE6TACyWtMkGu+qAHbGC1xb7JGrV+toGmIZByAz+dXIKZRCGClYOaQ2urzlwMsn2HbuJh0xZfCkAyQGLJWC8HG/1fISRjMOpZBatYfuO3WPyrdW1vp2JMNlCfa7jebg1Vgf5GpQOngvBCbnUSuxrHC94kQJ6kllGMk7ZRDRpGV7O5AQsFrXylIGpYN/RQf7wm3t4818/yK6DPWOGlR8fyuB6mlXN4dI8hfKh5dW1vomITVdzmKhlVARfU0HUCmp2fQ0r4jbxsFVYmBk0Y/iF4FsMtqHCybo5atMZD0vLsJAALBa3YsrA9TQnhrNoPwiOh/tG+eS9TwOU5iMEc3kV69qiNEdDNfPBtRoeLMPg1RvbS8+JhUzOWZng7JXNnL+mmRVNNmeuiJeC6U1Xb6atKYQyTp+GTQWrW4PlmlHbJJVzufuGy/nFLVePu+1CLG2SghCLWjFlcHh4FINgJKQm+OpfDK7FAHflHQ/QGrUrBpxX54OrGx5Wt0TJuR4PHuoDguB75ooYphGcXYqdcNXVC8WFmXuPDKKAVS0REhG79BrJ9wqQACwWuWKON+/5pZGQWgczGIrBtZgj7k3m6EvmJgyGxYE2w2mH9/7DI+w/NgzA1ed28typJDnXJ2qrcXO3xfcopkhMQ6G1lnyvqCApCLGoFVMGpqHwtcbXwSr3zkTQVRYPW6Uc8armMK6vOTaYYSSTH3eewvHBNG/7u1/y+NEhAP7sV8/h6++7jM+89cJxc7e7DvawfcdurrzjAbbv2A0gIyJFXdKIIRa9Yqvwod4UtmGwsjmMZRo4niZmGzhle9LKqyAu3dBWs+vswMsjvOfrj9CbDHLGn33bBbzj1RMvBZhK04dYsqQVWSwP5V/3i7nb4tzdj9/zVEVZWXPUJhGxGM44NbvOfvl8Hzd+6zFSOZdYyOTL2y/hDeetbOg6as0LlqWYYjwSgMWSUWsY+foHG9+Tdu/jx7n5e/vJe5r2phBfe+9WLtnQ1vDnNzo0XYgiCcBiSajXjNHonrSPff8J7n70aOnXLRGT4bQzqWuQpZhisuQmnFj0au1qKzZYTLQnTWvNB771WCn4KsBScHQwy80791c0aUxEhqaLyZKbcGLRm+p0Mcfzufl7+7nn8RNAEHxDpsIwDHxfowy4ZH3bpCaUleehZWj6stbQTTgJwGLRq9VgMZLJc3IkR2ciXHMG8I+ffJmP/OAJhjNu6bGQCaYRDN/RWuP4GstQJCIWedfHNhXnrGyWoCoaMTPjKJVSf6yUavxOhBBzrHp+w0jG4fhQFgVjUhIA9+w7xh9/Z18p+JqFf1UcL5j1m3M9HM/H9zVaa4bTDhnHYyTrcrgvNWZ+RHXt72TSFmJ5ayQHvAp4VCn1XaXUNar8mCHEPKgOeFdsaq/IvZ5KZoGg/bd6BvDhvlE+8v0nSyMjVzdHWNEUAoK5DZpguM7p7cTBSdjzgwaPZNatmB8xXv5ZiIlMGIC11h8HNgNfA34fOKSU+pxS6qxZvjYhxqgV8HbuPc51l64lZBoc6kmRdfzSqTaZdejuTfFS/yiPvtjPb3z5IXKujwLWtUXpSIQZzXul5yvGfndUUJrpm3X9itKyyQ58F6JcQ2VoWmutlDoJnARcoA3YqZT6qdb6I7N5gWL5qlVaVq/Z4f6nTjKa91jXFuXkcDAb4thgBqAwhyE41aZyLoaClc1h2mLByTfv+ZiGImIabOoMVsg/fWIYXxfGSpbdJvF8TV8qV1o1L7W/YjoayQHfpJR6DPj/gF8CF2qt/wh4FfBbs3x9Ypmq99X+UE+y5oaL7r7RUmDuiAeD0V1fB395PoWMA6ah2NAewzbNUsrCVApfU3odALpwEq5xj7o3leeKTcF4yur8M0jtr2hcIzngDuDtWus3aa2/p7V2ALTWPvCWWb06sWzV+2qfd/2aAQ+oCMzl/8UuBt+QZXBWRwzP1xW1wRs7mmiN2VimKtXv2pZBa8wak48wFHQlQjzcPQBI7a+YnglTEFrrT47zswMzezliOaqVaqj31T5kqlLAK+9s27giOIkGg9kzqKqTq2UoTDTdfWlioSBQl9f3VtfvXnvxGnbuPU4y6xHE9SCNsaY1QjxslVIM1fODpfZXTIbUAYt5VW+CWFPIJO/5NZsrirng8oAHlFIWXiHlUGN/JgDtMZt4xJ5wStmugz3c9J19jOZdIpZJZyJMImLLCnnRCJmGJhaOerMa6t1U01rXPOkWX1crcN4KvP+bj+KU7W6LWAbZwjI3QwWbilM5j7BtTDilbNuWLr70zksq/gdCUgxiJsksCDHrxquVrbc2fjTvTXqQ+bmrEljm6ffqjIcwCucQQ0HYMrFNA6VgOO00VKkw0SwJIaZDTsBi1o03J3e8CWL1Trq1PHdyhPf+w6OlG3IdTSFWNkc4cHIEReVJQynIeX7DlQqTuQ4hJkNOwGLW1TvlHhtMz0gVwSOH+/ntu3bz8nCWiG3wodefxeaVCYYzDk0hi5aohTJUqbXY8zWWYYz5DGkpFnNNTsBi1k10yp1OFcH9T73Mn37ncXKuT2vM5iu/t7VihXwx/RGxPZJZl5wbNF18cNtZY3a5FXO95WmSW0FOv2LWSAAWs26ioehT/Yr/j//3Rf7nvzyNXygP+8f3XcbmlYmK5zQa4GWdkJgPEoDFrJvpWlmtNV/48bP83a4XADhnZZxv/sFlrGqJ1v38iT5LWorFfJAALObUdKvOPV/zkZ37+f7e4wBctrGdr75nK81VwXOyZJ2QmA9yE07Mupka2ZjJu/zBNx4tBd9rLljFt/7gsmkHX5CWYjE/5AQsJlSviaJRM5FfHUjl+P1vPMoTx4YBeM8VZ/Dp3zgfwzjdcFTvOhu5fmkpFvNBWpHFuOq1CtdrRqgV7D5+z1NjVgZprRnOOPzilqsnvIYj/aO85+uP8GJ/GgXc/KZz+eDrz674vOdOjZDKebQ32axoCpeu87pL17Jz7/GGr1+IGTIzK4nE8jaZgeP1Ug3xkDnlkY1PHhvmLV9+iBf7g5thGzubOH9185jPyzo+vtb0pxxSObd0nV996LAMTBcLlqQgxLgmUx1QL9WglMLx/LplaPVSBA8+18sN39pD1gk2WGxoi6KgVJ9b/nl5z8dUCg30JnMkInappXlDnSYQIeabBGAxrslUB9QL1sMZh9uuvaBmfrVeA8R/fb6Pf3z4RRxPYyo4s6OpdA19qWxhSplHxDLoiIcJmQauF6ySz3t+6TqbCqdvqW4QC9G8BGCl1J8A7yfIk3xFa/3X83EdYmITNVGUm8pch1qn5uODab760GEg2GCxcUWMaOHnyaxDXzKPBsKmIu/5nBjO0Bq1Gco44INtqFIVw/VXbmTn3uMNXb8Qc23Oc8BKqQsIgu9lwMXAW5RSm+f6OkRjJjMNbCqlXMU5ESMZhxd6kjx1fJiBtAPABWuaeeW6lora4d5kDhSELYOu5giqcK8jnfdY0RTCUIpY2Cpd501vPEemmYkFaz5OwOcBu7XWaQCl1H8AbyPYOScWoEZbhadSyrW+LcbhvhS9yRxan27UUMAHX38WUduqOIFnXQ9DKTriwXD0Na3QM5Il6/pc1BHn9jolZhJwxUI052VoSqnzgHuAK4AM8HNgj9b6Q1XPuwG4AWDDhg2veumll+b0OsXc2HWwh/d/81Fc/3TwNYDOhM2mzmbuvuHyinVBwxmHWMikMxEpvYdsqBAL0MLciKG1PqCUugP4KZAC9hOsuq9+3g5gBwR1wHN6kWLOnLc6gY9CF8KvbSjWtEZIROyKvWvFE2zxpp3kdMVSMO+NGEqpzwHHtNZ/V+850oixcE22S678+SuaQhwbzNA/mgdgVUuYznhwsu1NZknnPZqj9pj3rV6gKR1rYgFq6AQ8LwFYKdWlte5RSm0AfgJcobUerPd8CcAL01S65IrP97Xmpf40vg4qHVqjNq2xoHa3L5WjN5WnKxGq6GqTm2diEVnQnXDfV0o9A/wL8MHxgq9YuCbTJVf+fMfTvNiXLm0tDpuK915xRqlSIZ336EqE6IhHpHtNLGnzUgestX7tfHyumFmTnaF7dDCN72tODGdLj9lGsJ9t597jpRPulXc8ILN5xbIgsyDElK1vizU840FrjdaUgq8iOPkahkHEMitOuJN5XyEWMwnAApjaQspGGy88z+ejP3iC40OZ0mOWASiF1tCZCFeccGU2r1guJACLaQ1MbwqZvNQ/ytMvj3CkP01TqHLwTSbvcsO3H+P/PHoMgHg4+LnjA1qXSs7KT7iT6b4TYjGTYTxiSgPTi0E773oopbAUoGEoky9NK3vlhlbe941H2XdkCAiCdWc8xIomOD6UxdPUPeFK95pYDiQAiyktpCwG7f6Ui0FQVuZqzcnhHGHb4NZ/eQoMg+7eUQDWtkYJW4qmcPA5SilODmc5OZLj0g1tk67lne6WDiEWAklBiCnd9CoO0cl7Pr72cXxdmuXguj7d/Rm6e0cxDcVt156PUlRMSUtEbM7uitOVCHP3DZdPOvjOxI45IeabBGAxpZtexaAdMg3cYPxuqfLcLdT3GgrufNclrG+LMZJxOHByhO7eFMlsMO1sqpUNk60/FmKhkgAspnTTqxi0ExGrNESnvKfSULCiKUTEMvnkvU8TC5kYKpjfe3wwQ18qO+XKhuLpu5zUCYvFSHLAApj8Ta/y0ZP9qTyOfzr8moZidUuY9W1NpdNqSzRC2ArajHOuz2jO40vvvGhKedvJbOkQYiGTE7CYsm1buvin6y/j6rIgGrYM1rVGMJTBjVdtqjitNkdtNnXG2bIqQUvUnvJNM6kTFkuFBGAxZXnH48Pf3c+PnzkFQCJisaLJZk1rrJTCmI2uNqkTFkuFpCDElCQzDh/4p8f45fP9AFxzwSr++h2vJFKVm53MTrnJkDphsRRIABYVatXXAhWPvWPrOr760GGeOjECwHuvOINP/cYrMAyj5vskwhZaa4YzjszvFaKMBOBlrjpQ9qZytETtUn3tzTv3owjyt61Rm6ODo/z37+0vjZJc3xZl2zmdY4Jv+ar54qn3tmsvkMArRBnJAS9j1Q0Nh/tGGUo7uJ5GKYXraQZG8/Sm8pwcztKbzHFiKFsKvmtaIoQsg0/9yzMVTRBSpytEY+QEvIxVz4DwtMZQ0JfKAXBiOFMKtjnHYzR/+maaqWBFPAwwZm5EvdbmQ6dG2L5jt7QPC1EgAXiJamRWQnWgDJkGjueT93z6UjkMFEoFLcZuWZeFpSBcdrOtugmiVp1uXypHMueNaR++FSQIi2Vr3pdyNkJ2wk3OeLva4PQNtZGMQ1PYpKOwCDOZdTg2mMEyFZ6vQeuKdfGFgWdYhmJta5TmqM1IxuFUMovWlIbqAGM+/9hghvYmu/RZIOvkxZK2oHfCiVlULwd7x48OVuR8YyGTnmSevlQWrTWmoWiL2ZzZHgOt8fXp4GsU/usUC5m0xmwsUzGSyXN8KIPraVY1h0unWmBMnW5QIxyuuE5pHxbLnaQglqC6OdieFOvaoqXUQGciOI2O5jwsIygR+8Svv4JXb2zn9772n+wtn+ObCOFrVXGK3ntkEMtQrGoJhqrD6Xxw+YSzXQd7uOk7+zhwcoSIZdKZCI8Zwi7EciQBeAmqNysBGDPEpiMeZjjj8ItbrgaCXO17v/5IKfh2NIUIW4rVLZV55PLlmUqd/rZVfaotpkNioSAVURzG05HwsE1T2ofFsiYBeAkp3ng71JMkmXVpi9l0xMOlHPDGFUFbcDEwnxrO0Deax9dw0ad/zG+/ai2/eL6f506lAHj/azfy0Wu2YJq1M1WNDMWZjWE8QiwVEoCXiPIbb6uaI9hmjoFRB9fz2byyueLmWDrvMpLJ05tySq8fybp87ZcvAUG+92NvPo/rX7ux4nRbrdhm3JvMksy65Fwf01Bce/Ga0nPK0yHNUZvmqF3qipPgK5Y7CcBLRHVNb0c8QixkjakyuBW4/f4DFcG32hd/+2Lefum6CT9z25Yurjs2xJ27XsD1fcKmQUvMZufe41y0rrU0jOfF/hQjGZe85xMyDZqjFmeuiE/79yzEYidVEEvEZIaUpx1/3PdqJPgWPdw9wLq2KK9Y3cJZXQk64pGKrrcrNrXTk8yT93wMBXnPpyeZ54pN7Q1/hhBLlQTgJaLRsY/Fk3I942Qcapoo8D/cPUBnPETINPB10OzRGQ/xcPfA5D5IiCVIUhBLxBWb2rlz1ws4no/WQR7XMo2KfCwEAbM5bGIo8Gv04KxtiYx9cBwT3Yg7OpimIx4ulbxBsIpe6n+FkBPwkrDrYA879x4nahulzcS+hljIYOfe4xWDcta0RDgymKkZfJvDJp9564WT+uyJtlPMxkB2IZYKCcBLQDGt4HiakGkQtU1ClkHe1RX52KHRPOm8RyoXBMTyf/jrWsJ8afulk65MmGg7hawPEqI+SUEsAcVSr7znYxaSuKpww6uYjz0xlOED336sNETdKAx2aAqZJCIWhmmO8wnjG287RfnyzmODaRnILkQZGcazBGzfsZueZJaTw1lcT2MYCl/rUptwU8gimXM53DcKwNrWCGHLoCl8ul05nXexDUVbU1jGRQoxfTKMZ7kofs1PRCx8NK7v4/ua5qhFMuvy0kCaw32jWIbifxZmOZTfNANwPZ9Dvakx4yJ3Hexh18Eetu/YzZV3PMD2HbsrcspCiKmTFMQSUP413/VGyHtB7jcRtjk6mCHjeERtky/89kX8+oWr+dFTJ8dULpxK5rANo/RYLGSRzrvcfv8B0o5fWi8kc3yFmDkSgOdBI8PSp/L64nt4vua7e47yqXufJu/6tMVs/vZdl/Bfzu4ETpeslXevOZ5mXWtlCVqtCWrFwFy+AUMIMTWSgphj1XvYyr/qz8Tr867P3/378/zFD58k7/qsa4vy7T98TSn4FkvW2mI2Ecsk72kGRh3WtESwqobu1JugJnN8hZgZcgKeY9UzG+qdKOudcsd7/Ws2tfP5+w/yzYeDoTpbViW46/dexRkrmsZ8fks0QmcieKx4Ay7t+KTzbsUWjU0dTRUT1EDqeIWYKXICnmONzGyod8r90s+eY++RQY4MpOnuTTGScUqvf6k/xZ99b38p+IYsg6aQxeHe0YY+fzTv1aznveWaLVLHK8QskRPwHJuodbe4PWI071Zsj+hLZblz1wsogvoW19OcGM4AoNGkch73PXkSCGp7N7RHGcrkx9wwG+/z69XzSh2vELND6oDn2EQLMz9579OcGMoQpGMVWsOa1gg9I1nynmZdW5QTQ1mUCmYqBOE36IIDaI5YrGuLYhrBl5vqxZfjfb4EVSFmjNQBL0Tjte4W87NhywCtMJRCKehN5sh5PmHLIBGxWdMawVTBKTjvgeNpFEHwXV8WfGFsemOi1mEhxNyZlxSEUurDwPUEc2OeBN6ntc7Ox7XMh3pf9YstxR3xcJBe8AGlybo+lmGQiAT/uOJhC52IcHQwDRpsU/Guyzbwg73HePZUirBl0BEP0xytvfhyvNZhIcTcmfMTsFJqLXATsFVrfQFgAu+c6+tYiIqTw5qjNmtaolimwvOhKWTxwW1nEbJMRnMOQ+k8RwbS+IWxk2HT4O5HjmCbBlprRvMeLw2kOfjyMCMZR26YCbFAzVcKwgKiSikLiAEn5uk6FpTyyWGJiMWqlghrWqN86Z2XcNMbz+ETv34eng/HhrJogoE7K5vD+FqjCfa6FR8H8AqjKYUQC9OcB2Ct9XHgi8AR4GVgWGv9k+rnKaVuUErtUUrt6e3tnevLnBfj5WczeY99R4d4aSDI50Ysg7UtETriERxfYyqF5weBOGKZRCwD01C0RO3SOEohxMIy5zlgpVQbcC2wERgCvqeUerfW+tvlz9Na7wB2QFAFMdfX2YhGWoon23ZcKz87nM7z2fsO8N09xwC4cG0LvcksrbEQEKz5cb0g+KrCn5QurP+RrjUhFq75SEG8ETiste7VWjvAD4BfmYfrmJZGWorHe06jE8Z6RrJ8+Lv7S8H3tZs7+Iff38rGjnipVbgjHsYvJBs04PsaH01HPCxda0IsYPNRBXEEuFwpFQMywBuARVfkO1FLcb2GivEmjF13bIiHuwc4OphmXWuUt12ylu/uOcpjR4YAuPaVa/jMWy8gEbG58apNfPLep0v54hVuiP7RPJ6vUQasSUSwClsy5CacEAvTnAdgrfV/KqV2AnsBF9hHIdWwmBRLxsoVv+4XT77pvIdlKFxfc2Ioy5rWoISs1oSx3mTQ6bauLUpLxOLYYJq/+OencAvL266/ciN/9l/PIVp4TfWmiY0dcW5/exBoi491JSLStSbEAiadcFNU3EJR3tJb7DoDxt1QcWwww+auOKpsB3x3b4qs63HeqmZGcx5HB9Ol4PvRa7bwvivPJGxNfW2QEGJOSSfcbBpv2WRx4E0xN+v7Qctw1q2cMFYu5/qEDMVI1uXIwCiuH3S3tUUt/vC1GyX4CrEESQCeovFKxsZrqKg3YcxQYFsmRwfSeBpMBV3NYc5d1Yxtyj8mIZYimYY2DfVaeos3yHqTWZJZl5zrYxqK66/cWHr+rcDt9x/gUE8KrTUR22Ak6wJgGYquRAjTMPjA686ay9+SEGIOSQCeBdu2dHHdsaExa3927j3ORetaS0E47fisag4zMJpnJBukJBQQD5tsaG/iA687q/Tc8eqJp7viSAgxPyQAN2AqAe7h7oGKSgegokwt6E7T9KXyjOZPB1/bhOZoaEzwLY6QrF6MCdT9mQRhIRY2SS5OYKo73MbbfKG15nBfiv7R08HXUGAboFHYpqpoHy6vOVYq+HvxOeP9TAixsMkJeAL1Gi5uv//AuKfiepsn1rZGOXgqSTLrVgXfYNdFcVB6efvweDXHGur+TAixsMkJeAK1TrKu53OoNzXuqbi8TG0kk+fQqSSH+0Y5MZTmvV97pBR8TRX85WnIez4Zx+P53hRNodOfWayqKFdsMR7vZ0KIhU0C8ARqBbhTyRy2YdT92r/rYA+333+AE0MZuntHeWkgQ9b18TUcGczSk8yhgO2vXs/ZXXE8DZ6vMVRQAeF6mv7RfCmgj1dzPN7PhBALmwTgCdQLcCubwxXPK29Dvnnnfp7vHcVQ9efxaqAzEeLHH34d565MELYMDENhm0Zhp5vipu/s48o7HuCuB7u57tK1NWuOZcWQEIuXtCI3oFgFUdwKPJTOk/f8um3I+44Oov0gyOY9v+77Nkcsnvj0m7jyjgdojdql1uSRjMOJ4Qy+1py3qlkWZwqx+DTUiiw34RpQ3XBxetiOW7FZ+MarNvHxe57CKwxIz7v1g6+CUh64+oZdXyoHBIPViymOYgkbIDW/QiwRyz4F0ehc3nITtSEbCjzPn3AdUPFGW3WaI+cGx+fOxOk0R9Q2OdSTnFJJnBBiYVrWJ+DxGhwmOlXWa0N+/2s38t+/O8yw6477ek0wYrL4XuWjJWMhk6awSSJyurws43jkXZ+WaP0ZxEKIxWVZB+CJhqpPluv5rG+PsbI5wlAmNe5zmyMWN73xnNKvywN6vRRHsUa4nNT8CrF4LesUxHjdapNNTeRcj2dOjPDHd+/j2VNB8H3PFWdwVkeMiB0syDRU0HQRsQzWtkbrvle9FMc5K5ul5leIJWRZn4ATYYvne1J4WhMyDToTYUxD0RQyJ5WayOQ99h0Z5CPff4JjgxkMBX/yhs38/q9s5Opzu/jznfsZGM1TmK9O1vU5NpjmSz97jvuefJnD/cEJdlNHE2++YBX3P3WS7r5RADauqLzRVu/mnxBi8Vm2ZWjFet2htINRKBjxNbTFbDri4bplZnffcHnF+ySzDg+/0M8t33+CwbRDyDL4xK+fx1svWVvK4V75+Z9xfDiHJjgBGwQ5YK3BKJyMgdJmY0OBWXjQ19Aas/nidReXds2Vl8RJFYQQC5KUoY3nrge7aYnaNIUs+lI58p6PZSpWNIVI5tyG5isMjub56TMn+dS9z5BxPBIRi8++7ULeeF5XRfDuHXWwTYVpnM74ZB0PDdhKYRSCreN7aA0+ECo8V/maVO50XrrezT8hxOKzbANwccCNCimaC8FWa81wxqk7SKeYa9Va05vKcc++49zxo2dxfU1XIswXrruI12xaQcSeeH1Q8XtH2Vo4Sl9Gyr6UKBW0KcuNNiGWnmV7E268ITbjzVfwfc3JkSxff+gwn73vIK6v2djRxG4cU+kAAA0vSURBVN/97qVccVZHzeC7qaMJX4OvNRqNX4i0irKgS1kwrgrKpqHkRpsQS9CyDcDjBdl6VQhXbu7g2FCav/zJs/z9fwRdaReta+HOd13CK9e3ErJq/3Hecs0W2mI2iqBUTQEtUYt42MTTGs/38XwfpYPYa0DpMU9r4mFLbrQJsQQt25twMHbGw3g3tHKux9H+DJ+97xn+/dleAK46p4NP/+b5nNHeVLppNpnPgmAv3ERVEB9983mS9xVicWnoJtyyDsCNyuQ9XuhN8fF/forHjw4B8NZXruHP37SFNa2R0hAdIYQokCqImZDMOhw8meSW7z9Bd29wKn3/azdy/ZUb6WqW4CuEmDoJwOMYSud5/OgQt3z/CU6N5DANxZ+/6Vyue9U6OuLBoBzZSCyEmKplexNuIn2pHL841MtNd+/j1EiOiG3wubddwDu2rq8IvjKdTAgxVXICrqK1pieZ46fPnOS2fz1AzvVpjdp8/u0X8l82d9BcNqGs0WE+ckoWQtQiJ+Aynq85MZzlu48e5ZP3PE3O9VnTGuFv33UJrz2nsyL4wvjDfIrklCyEqEcCcIHj+RwfTPOVB1/gL3/6HL6Gc1bG+dvtl/KqM9qJh8d+WWhkI3H5KbnWAk8hxPIlKQiCuQwnhjL81U+f41+eeBmArWe0cdtbz2djR7xua/GNV22qOZ3sik3tbN+xm6ODaXqTOVbVWeAphFjeln0ATuddjvanufXfnuGXz/cD8KuvWMnrz+3g0/c+w/GhTN28bfUmi3VtMa7Y1M7OvcdLoyz7UjmOD2VRSpWmo8kMXyEELPMAPJJ16O4JGiyeOjECwDtfvZ5L17fy5X9/npBlTDgPuHo62fYduytuzK1MRDg+lOHkcJZ42JIZvkKIkmWbAx4YzfP08WH+5DuP89SJERSwtiXKrud6uO3fnsH1/SnlbatvzDVHbda2RtAwZoGnEGJ5W3YnYK01vckcTxwb4qM/eJK+VB7TUCTCFpYJreEQfck8WSdP2Dq9GLPRvG2tUZaWaXDphrYxw9yFEMvbsgrAnq/5533H+F+7unmhN4UGwpbButYoru+TiNgopQhbBnnP5+Rwlt5kMKzdVIqNHU0Tfka9G3OSchBCVFs2KQjH8/nBY8e47d8O8Hwh+JoKEhGLwUy+FHwBOuJhfK3Juj5OYXyk6wdD2Ceq3603ylJSDkKIasviBJx1PE6NZPnrnx9iKO0AYBkK0wjWCqEUITNHZyICBHlbYwi0CpZThEyDjngYq5AHniiYytogIUQjlnwAHs25nBrJ8pVfdHN8KAOAbSp8X6N9hWUqXE/Tm8oDwek343j4KNa3RWiOhkrvpbWsBhJCzJwlHYCH0w6nRjJ88SfP8ZNnTgEQsQ3QGl+Di8Z3gy3ELRGT9P9r715j5KrLOI5/n529tmx3W7qVLS2XlZbbEigpgr4AucRQXrSRENNGgiaNCCgajAkiCTFoxJh4CQajfWG8JJQCom6MvFFAGrBIQ5dSiVRawG1L2y3tbm97mZ15fHFO2+m63T27OzP/OTu/T7LJXP47+zw5M8+e+Z//ZThH/0CWRXNnUZ+J+oELafyuiBRT2QuwmV0MbCh4qAN4xN1/Wsy/89HRIfb2D/LAhm627z8KRP29GYP+gZFo38t4LfoaM44O5WhprGXjgzcBp9Zw0MU0ESmVoDtimFkG2A1c6+4fnKndZHbEODHMrOfQcb6+vptdcbfDvNl1zK7PMDCc46Nj2VO7EgN1mRpy+TyY0d7SeHLmG5B4yyIRkQKp2BHjZmDHeMV3MnJ5Z9/hQXb0HuXB329lT98gAAuaG5g7q566jJHNDZ1WfIGTXQ0Z/PSZbysv19hdESmZ0MPQVgPrx3rCzO42s81mtrm3t3fCF8rm8uzpG+DNnj7uf3LLyeLb3nKq+JoZRwZHgGgUxOjdhBrrMlqxTETKJlgBNrN6YCXwzFjPu/s6d1/u7svb2trGfa0Tq5m9uuMADzzdTd9AljmNtSxpO4v6TM3J4gswNJKnvgbMoK6mhoaCreRP7HQBWrFMREov5BnwCuANd983nRc5PjzC3v5Bnt+2l2//YRuD2TwLmht4fPUy7rvx4+Q9Gr3g7hwfHiFTY5zd3MDCliZqM0bOnRqDuppo/O8JGvEgIqUWsg94DWfofkjq8GCW3sODbHi9h3Ub3wOgo202P7j9Ci6cfxZtcd9v4YW0VVcu5Nk3dlObiaYWD2SjoWcGGvEgImUVZBSEmc0CeoAOd++fqP1YoyAOHhvm4LEhfv7iDp7bshuAqxa38OiqThbNncW82fVjvRRwao+2wtENoBEPIlI0iUZBBB2GllRhAXaP1mQ4eHSYx57/N3/fHl2gu2FpGw+tuIT21iZamurGezkRkVJLxTC0ScnnnX1HBuk9PMQjXdvo7olOnm+/+ly+cuNFfGxO45h7t4mIVKLUVKuRXJ69hwfZfWiAh557i50HjgFw9/UdrLlmMee0NNFUP/bebYW0RbyIVIrQ44ATcYc9fYP8Z98R7l+/hZ0HjpGpMR5acQmfv/Y82luTF19tES8ilSIVBTiby9Pdc4ivPdXN/iNDNNVl+P5nO1nR2U57S9MZdy0eTVvEi0glSUUXxJGhEb757FaGR/LMnVXHY7dfweULW2hvaaQ2k/x/SM+h47SOukCnCRciEkoqzoD39A0wPJLn3NYmfrZmGVcsamVha9Okii9E+7UNZHOnPaYJFyISSioKMMDF5zTz+JqruGhBM+1zGsnUJBrlcZovX99BNhfNiDsxM04TLkQklFSMA553/qW+8dVNtDU30NbccHJdh6kYaxKGRkGISJHNnIkYnVcu85de2XTaYjkiIhUsUQFORRdEXbwppojITJKKAjyV/l4RkUqXigIsIjITqQCLiASiAiwiEogKsIhIICrAIiKBqACLiASiAiwiEogKsIhIICrAIiKBqACLiASiAiwiEogKsIhIICrAIiKBpGI9YDPrBT6Y4q/PBw4UMZxKobzSRXmly3TzOuDut07UKBUFeDrMbLO7Lw8dR7Epr3RRXulSrrzUBSEiEogKsIhIINVQgNeFDqBElFe6KK90KUteM74PWESkUlXDGbCISEVSARYRCWTGFGAzu9XM3jGzd83sW2M832BmG+LnXzOzC8of5eQlyOsbZva2mW01s7+Z2fkh4pysifIqaHeHmbmZpWKoU5K8zOxz8TH7l5k9We4YJyvBe/A8M3vRzLbE78PbQsQ5WWb2KzPbb2bbzvC8mdnjcd5bzezqogfh7qn/ATLADqADqAfeBC4b1eY+4Bfx7dXAhtBxFymvG4FZ8e17Z0pecbtm4GVgE7A8dNxFOl5LgC3A3Pj+gtBxFyGndcC98e3LgPdDx50wt+uBq4FtZ3j+NuB5wIDrgNeKHcNMOQP+BPCuu+9092HgKWDVqDargN/Et58FbjYzK2OMUzFhXu7+orsfj+9uAhaVOcapSHK8AL4L/BAYLGdw05Akry8BT7j7IQB331/mGCcrSU4OzIlvtwB7yhjflLn7y8DBcZqsAn7rkU1Aq5m1FzOGmVKAzwV6Cu7vih8bs427jwD9wNlliW7qkuRVaC3Rf+xKN2FeZrYMWOzufy5nYNOU5HgtBZaa2StmtsnMJpyuGliSnL4D3Glmu4C/APeXJ7SSm+znb9Jqi/liAY11Jjt6fF2SNpUmccxmdiewHLihpBEVx7h5mVkN8BPgi+UKqEiSHK9aom6ITxN9W9loZp3u3lfi2KYqSU5rgF+7+4/M7JPA7+Kc8qUPr6RKXjNmyhnwLmBxwf1F/P/XoJNtzKyW6KvSeF8/KkGSvDCzW4CHgZXuPlSm2KZjoryagU7gJTN7n6j/rSsFF+KSvg//5O5Zd38PeIeoIFeqJDmtBZ4GcPd/AI1Ei9mkXaLP33TMlAL8OrDEzC40s3qii2xdo9p0AV+Ib98BvOBxT3sFmzCv+Kv6L4mKb6X3J54wbl7u3u/u8939Ane/gKhve6W7bw4TbmJJ3od/JLpwipnNJ+qS2FnWKCcnSU7/BW4GMLNLiQpwb1mjLI0u4K54NMR1QL+7f1jUvxD6SmQRr2jeBmwnumL7cPzYo0QfXIjeFM8A7wL/BDpCx1ykvP4K7AO645+u0DEXI69RbV8iBaMgEh4vA34MvA28BawOHXMRcroMeIVohEQ38JnQMSfMaz3wIZAlOttdC9wD3FNwrJ6I836rFO9BTUUWEQlkpnRBiIikjgqwiEggKsAiIoGoAIuIBKICLCISiAqwiEggKsAiIoGoAEvVMbNr4vVdG81sdrwub2fouKT6aCKGVCUz+x7R7MgmYJe7PxY4JKlCKsBSleJ1DV4nWmv4U+6eCxySVCF1QUi1mgecRbTyWmPgWKRK6QxYqpKZdRHt7nAh0O7uXw0cklShmbIgu0hiZnYXMOLuT5pZBnjVzG5y9xdCxybVRWfAIiKBqA9YRCQQFWARkUBUgEVEAlEBFhEJRAVYRCQQFWARkUBUgEVEAvkf7MRYXb0XKH0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 360x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.lmplot(x='x', y='y', data=df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们随机生成了一些点，下面将使用PyTorch建立一个线性的模型来对其进行拟合，这就是所说的训练的过程，由于只有一层线性模型，所以我们就直接使用了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "model=Linear(1, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "损失函数我们使用均方损失函数：MSELoss，这个后面会详细介绍"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "criterion = MSELoss()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "优化器我们选择最长见的优化方法 SGD，就是每一次迭代计算mini-batch的梯度，然后对参数进行更新，学习率0.01 ，优化器本章后面也会进行介绍"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "optim = SGD(model.parameters(), lr = 0.01)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "训练3000次"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "epochs = 3000"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "w，b是我们所需要训练的模型参数，就是5和7，记录下来，然后最后显示一下我们这个模型最后训练的结果是多少"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "[w, b] = model.parameters()\n",
    "x_train = x.reshape(-1, 1).astype('float32')\n",
    "y_train = y.reshape(-1, 1).astype('float32')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "开始训练了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 100, loss 0.4731859564781189\n",
      "epoch 200, loss 0.06617910414934158\n",
      "epoch 300, loss 0.0634840652346611\n",
      "epoch 400, loss 0.06299427896738052\n",
      "epoch 500, loss 0.06262210011482239\n",
      "epoch 600, loss 0.062333136796951294\n",
      "epoch 700, loss 0.062108736485242844\n",
      "epoch 800, loss 0.06193448230624199\n",
      "epoch 900, loss 0.06179914250969887\n",
      "epoch 1000, loss 0.06169406324625015\n",
      "epoch 1100, loss 0.06161244958639145\n",
      "epoch 1200, loss 0.061549071222543716\n",
      "epoch 1300, loss 0.06149986386299133\n",
      "epoch 1400, loss 0.06146164610981941\n",
      "epoch 1500, loss 0.061431970447301865\n",
      "epoch 1600, loss 0.06140892207622528\n",
      "epoch 1700, loss 0.06139101833105087\n",
      "epoch 1800, loss 0.06137711927294731\n",
      "epoch 1900, loss 0.06136634573340416\n",
      "epoch 2000, loss 0.06135796383023262\n",
      "epoch 2100, loss 0.06135144084692001\n",
      "epoch 2200, loss 0.06134640425443649\n",
      "epoch 2300, loss 0.06134248524904251\n",
      "epoch 2400, loss 0.06133943423628807\n",
      "epoch 2500, loss 0.06133706122636795\n",
      "epoch 2600, loss 0.06133522093296051\n",
      "epoch 2700, loss 0.061333801597356796\n",
      "epoch 2800, loss 0.06133269891142845\n",
      "epoch 2900, loss 0.06133183836936951\n",
      "epoch 3000, loss 0.06133117899298668\n"
     ]
    }
   ],
   "source": [
    "for i in range(epochs):\n",
    "    i+=1\n",
    "    # 整理输入和输出的数据，这里输入和输出一定要是torch的Tensor类型\n",
    "    inputs = torch.from_numpy(x_train)\n",
    "    labels = torch.from_numpy(y_train)\n",
    "    #使用模型进行预测\n",
    "    outputs = model(inputs)\n",
    "    #梯度置0，否则会累加\n",
    "    optim.zero_grad()\n",
    "    # 计算损失\n",
    "    loss = criterion(outputs, labels)\n",
    "    # 反响传播\n",
    "    loss.backward()\n",
    "    # 使用优化器默认方行优化\n",
    "    optim.step()\n",
    "    if (i%100==0):\n",
    "        #每 100次打印一下损失函数，看看效果\n",
    "        print('epoch {}, loss {}'.format(i,loss.data.item()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "训练完成了，看一下训练的成果是多少\n",
    "\n",
    "我们期望的数据 w=5，b=7 可以做一下对比"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4.924540996551514 7.020828723907471\n"
     ]
    }
   ],
   "source": [
    "print (w.data.item(),b.data.item())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "再次可视化一下我们的模型，看看我们训练的数据，如果你不喜欢seaborn，可以直接使用matplot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3Xt8VNd56P3fnvtNmtFodJeQBEggbuImwAZsE3zDJnaJ7aRO0tpvatOk581J80ma9j3JSXqauE2aNk3iJCcH6tbpaWKnvmDiGLBjG2MbGyGuBiGEJITQSEia0WhGmvttv3/IM9aI0QXdQPL6/uNoz569l/jk8+ylZz/rWZIsywiCIAizn+J6D0AQBEGYGiKgC4IgzBEioAuCIMwRIqALgiDMESKgC4IgzBEioAuCIMwRIqALgiDMESKgC4IgzBEioAuCIMwRqpm8mc1mk8vKymbyloIgCLPe8ePHnbIs54x13owG9LKyMo4dOzaTtxQEQZj1JElqG895IuUiCIIwR4iALgiCMEeIgC4IgjBHzGgOPZ1IJILdbicYDF7vocx6Op2O4uJi1Gr19R6KIAjXwXUP6Ha7nYyMDMrKypAk6XoPZ9aSZZne3l7sdjvl5eXXeziCIFwH1z2gB4NBEcyngCRJZGdn43A4rvdQBGHOsHvs1HXW4fA5yDHmUFNYQ7G5+HoPa0Q3RA5dBPOpIf4dBWHq2D129jbuxR/xk2fKwx/xs7dxL3aP/XoPbUQ3REAXBEG40dR11mHRWcjUZqKQFGRqM7HoLNR11l3voY3ouqdcbjR/+7d/i8lk4utf/3raz1966SUqKytZsmTJDI9MEISZ5PA5yDPlpRwzaUx0e7uv6TozmbaZdQH9eue0XnrpJbZv3y4CuiDMcTnGHLxhL5nazOQxb9hLjnHMFfhJdfY6dp/YTUyOkWPMIRgN0jnQyf2L7p+WuDVmykWSpH+TJKlHkqSzQ479UJKk85IkfSBJ0h5JkixTPrI0piun9cQTT7Bo0SJuv/12GhsbAdi9ezc1NTVUV1fzwAMP4Pf7ee+99/jd737HX/3VX7Fy5UpaWlrSnicIwuxXU1iDO+imP9RPXI7TH+rHHXRTU1gzru/bPXZ2n9yNSqGiwFRAJBbhnOMcsXhs2tI248mhPw3cPezYH4BlsiyvAC4A/98Ujyut6chpHT9+nGeffZaTJ0/y4osvUlc3eK1PfepT1NXVcfr0aaqqqnjqqae4+eabue+++/jhD3/IqVOnWLBgQdrzBEGYG7RKLe9cfocDTQfwR/zXNLOu66wjGouSpc9CkiQMagNGjZFObycO3/RUo42ZcpFl+W1JksqGHXttyI9HgAendljpTVVOa6h33nmHHTt2YDAYALjvvvsAOHv2LN/61rdwu914vV7uuuuutN8f73mCIMweiWyARWdh28JteMNe3EH3NV0jEa+C0SAG9WB80av0XPFeYX3R+ukY9pTk0L8A/HYKrjOmqchppZOu3O/RRx/lpZdeorq6mqeffpq33nor7XfHe54gCLPH0GwAkPzvgeYDZBuyx/UOL5EzP+c4B4BOpaMv0IdSUo47bXOtJlW2KEnSN4Eo8OtRztkpSdIxSZKOTXbRy2RzWunccsst7Nmzh0AgwMDAAC+//DIAAwMDFBQUEIlE+PWvP/r1MjIyGBgYSP480nmCIMxeDp8Dk8aUciwYDfJy48scajvEqa5THGo7xK9O/WrEd3g1hTUoFUqW5CxBo9RwZeAKUTnK46sfn7ZCjgkHdEmSHgG2A5+TZVke6TxZlnfJsrxWluW1OTmTm0kXm4u5f9H9GNQGur3dGNSGSb8tXr16NZ/5zGdYuXIlDzzwAJs3bwbgu9/9LuvXr+eOO+5g8eLFyfP/+I//mB/+8IesWrWKlpaWEc8TBGH2SmQDhnq//X08YQ9KSUmWPgulpOSC6wIHmg+kvUYiXknRebTZq9m28JN8+5ZvU1M8PbNzAGmUWPzRSYM59N/Lsrzsw5/vBn4E3CrL8rin3WvXrpWHb3DR0NBAVVXVNQxZGI349xSEyRuaQzdpTHjDXv73sf9NVXYV+Rn5yfN8YR/+qJ9fbv/lVdcIhGPc+sOD9AyEADj8N5+gyKKf0HgkSTouy/Lasc4bM4cuSdIzwG2ATZIkO/AdBqtatMAfPsw/H5Fl+YsTGqkgCMINJjG7ruuso9vbTY4xh/LMckza1DSMjIwkf/QOzu6xs795P8+85+dix8Lk8X9/tGbCwfxajKfK5eE0h0VtniAIs861LEwsNhenfOb0OzncfhiFpECv0hOIBugL9rGxZGPy2t9/8zl+X1uZ/E5Bbgv3rh6goqB6en+xD4leLoIgfCxMdmHitoXbWJi1kJgcwxVwEZNjLMxayLaF2/AEItz2g1PJYK5QxPnMljpuXtqGM+iYsf4vs27pvyAIwkSkK0Xs9ffy5NEnWZC1YFwz9kdXPpoyw19bsJYf7G/n5VOngcHUy8YVtczPH3w3qZIHSxWnayHRcCKgC4LwsTB8YaLT76S+p56oHOXmkpvxhr08feppNEoNF90XkWSJ1YWrWZm3EvuA/ao0zcHGHjZ//6OZd1VpO7LpJa6Ew9iCy8nUZRKMBtGoNJNeKzNeIqALgjBrTKY53/CFiU2uJpQKJdm6bBSSgnA0zOnu0zj9Tlblr0JG5rWW1/jd+d9xx4I7KLWU4g17+c3pl/nFvnnJ62bo43znARiIaHmjxcbl/stcdF+kwlpBb6CXSmvltC0kGk7k0KeByTT4Jryzs5MHHxy9K8KPf/zja27o9dZbb7F9+/YJj08QZqPJ5sCHL0zsGugiJseosFYAgwE+GA2iUqgwaoyYNCZC0RD94X4OtR3i1ebX+OXr0ZRg/oWtPXzvM3G0arAZbGxdsJXlucvpD/bjj/rZPG8zj6x8ZMY6wooZ+jjFYjGUSuU1faewsJDnn39+1HN+/OMf8/nPfz7ZS0YQhPRGWo5f11k3roCZrhSxKKMIm9EGgCfkwR/2k2XISn5nIDyAw+cg5F3G8Y57ksc3LO7i2Uf/jH989x/56ZH/pMvXhVapZUX+Clbnr+b2BbdTU1hDXWcd+5r2zVirbzFDBy5dusTixYt55JFHWLFiBQ8++CB+v5+ysjL+7u/+jk2bNvHcc8/R0tLC3XffzZo1a9i8eTPnz58HoLW1lZtuuomamhr+5//8nynXXbZsGTD4QPj617/O8uXLWbFiBU8++SQ//elP6ezsZMuWLWzZsgWA1157jZtuuonVq1fz0EMP4fUOrlY7cOAAixcvZtOmTbz44osz/C8kCNdfuuX4Jo3pml44FpuL2VG1g51rd/LldV9GqVAmZ+xqpZo4caw6a/J8j0+J3/5Duj4M5haTj7tv3kdp0QXq7HW81vwabZ42NEoNyPBu27u8fOFl1JL6umxfd0PN0P/Xy/Wc6+yf0msuKczkO59cOuZ5jY2NPPXUU2zcuJEvfOEL/OIXvwBAp9Px7rvvArB161Z++ctfUlFRQW1tLX/xF3/Bm2++yVe+8hW+9KUv8ad/+qf8/Oc/T3v9Xbt20draysmTJ1GpVLhcLqxWKz/60Y84ePAgNpsNp9PJ9773PV5//XWMRiM/+MEP+NGPfsQ3vvENHn/8cd58800WLlzIZz7zman7BxKEWWKqm/MNn7EvzVmKVqGly9fFlf4u3j9Tg8+3IXn+J28+iUbbhzccJUuXxfMNz6NSqlhgXYA37CUUC5GpzSQej/NO+zusyFsx4b8mJuqGCujXU0lJCRs3Di4Q+PznP89Pf/pTgGTw9Hq9vPfeezz00EPJ74RCHy7pPXyYF154AYA/+ZM/4a//+q+vuv7rr7/OF7/4RVSqwX9yq9V61TlHjhzh3LlzyXGEw2Fuuukmzp8/T3l5ORUVFcnx7dq1a0p+b0GYLWoKa9jbuBcguRzfHXRza+mtE77m8MVDdo+db/1+P6+fKUwem1d8GH3mcTzRPIoNxZTmlFJiLuGFcy8gSRKZmsF9GQDicpzL7su83/4+A6EBLDoLFdYKbEbbpFt9j8cNFdDHM5OeLsNb6CZ+NhqNAMTjcSwWC6dOnRrX94eTZXlc59xxxx0888wzKcdPnTo15ncFYa5LlwO/tfTWZEAeXgFTnFGcttxwJC0OL1v/+TQwGMwr8mX+eLODuk4XCmkBmbpMluYuTXZ4PWI/gtPvJBwLo1VpAXD6nPiiPnJMOehUOkKxELUdtawvWj8j5Ysih/6hy5cv8/777wPwzDPPsGnTppTPMzMzKS8v57nnngMGg+/p06cB2LhxI88++yzAiC1077zzTn75y18SjUYBcLlcQGo73g0bNnD48GGam5sB8Pv9XLhwgcWLF9Pa2kpLS0tyfILwcTQ0B76jakdKMB+as/6g6wP+8tW/5Nmzz3LRfZF2TztPn3qafz3+r+w6tos9DXuS+exwNM5d//I2W//5UPI+33kwyn/fJpNrsrG+eD1mnRm7x57S4fXBqgdRK9S4Ai4CkQD9wX7sA3bKzGVsKd2CPzJYvaZX6zndc3rSrb7HQwT0D1VVVfGrX/2KFStW4HK5+NKXvnTVOb/+9a956qmnqK6uZunSpezdO/jn309+8hN+/vOfU1NTg8fjSXv9xx57jHnz5rFixQqqq6v5zW9+A8DOnTvZtm0bW7ZsIScnh6effpqHH36YFStWsGHDBs6fP49Op2PXrl3ce++9bNq0idLS0un7hxCEWWhoBYzL7+I9+3voVXpi8RiRWIS6jjpOd5/mrONsykvKH7x6gspv7aexe3BS9eTDq/iXR8JoNB+1zrUZbCzLXcYDSx5IeYjUFNfw7Vu/zdKcpXiCHuLEWZ6znB2Ld1Bpq2R98Xq0Ki2haIhgNDhtG0MPNa72uVPlRm2fe+nSJbZv387Zs2fHPvkGdyP8ewrCTNt1bBd5pjwUkoL3299PBnh/xM/qgtWc7jqNjMw88zzWFqyl9lInB46uTH5/27J8fvG51UiSlLZ1rjvoHldA3tOwB3/En/Litj/Uj0FtYEfVjgn/fuNtnytm6IIgzHpDN6TwhDyYtWZ8YR/IcN55nhZXC23uNjz+AN9/KSMlmP/FPZf55ifzku+pJrORznTsqnYtbqiXotdLWVnZnJidC8JcM96l/kMrYDK0GQyEBrjsuYxaqUalVKFRaejv3cK7l7cmv7NxxSlK83wUWZZeVU44vPplvMZ6cTvdboiAPp4KEGFsM5k+E4TpNjT1kWfKwxv2srdxb9rZ8tBAatFacKldzM+ajy/io9djoe/SzuS5tuzz3LK8E3/ER4V1/YjlhON9mKQ7bzLplcm47gFdp9PR29tLdna2COqTIMsyvb296HS66z0UQbjKREoK6zrriMVj1Dvq8QQ9mHVm8o35Iy7OGTqrrrPX8T/+8F2aLjwG8mCYUypkli75T3qD7ejUt7I8bxk2g43+UP9V5YTjfZhcy0NnJlz3gF5cXIzdbsfhmJl+wXOZTqejuHjm/08kCKMZHvTa3G08e/ZZNpZsTHYwHB4E7R47L557kUueS2Tpsyg1lxKKhqjvqefKwBUAGp2N9AX7sOqsVNo+6mi4v+kAu9/y0uP88+QY1i07zLYlFQyEb+as4yxLc5Zi0piSOe5bS29Neei09LVQlFGUfLkZjoVpcjXxxNtPcOfCO5MPoMn2l5lq1z2gq9VqysvLr/cwBEGYJsODXpevi2x9NhdcF+jydeEJelAr1RxoPsBjax5LPgD6Qn0YNUYUkoKm3iYWZS8iEA1w7MoxCjIKaHW3opSUuANu9Go99T312B2ZHDi2IHnvjKz3uKnKgVFj5HTPaSqsFTy+6nHsA/aUHDeQ8tCp7ajFHXCToc0AoNZei0FtQJKkZMnj/Yvuv6rHOjAjK0JHct0DuiAIc9vwoOcJelBICj7o+oCaohosOguBSICXG18G4NiVY+hUOrQKLZFYBGTQqrS0ulsJRoNk6bLo8nZh0pgwqA34I34u9fXy4qF1yXuo1T4WVv4ab8RFXYeXTF0mOpWOzy37HDXFNdSQWnWyp2FPykMnz5iHO+imydUEgFEzuGJ86Dl1nXVT3l9mskRAFwRhRJPZUCJheNAz68wcv3Ici96CQT3YNtob8tIf7ues4ywKFEhIeCNesnXZROQIvrCPOHGy9FnMz5qPJ+QhS5eFLMPR+mrae7KT9yud/wwGQ+/g7D3oJhKPsNa6FqPWyNHOoxRkFFz1Owx/6FRYKzjScYSuga7Bh4tKSyASYFnRYPfUxCz8nop7pry/zGSIgC4IQlojvfBbV7hu3D1S7B47Tr+Tg60HyTXlsjx3OfnGfHr9vazMW4ksywSiAdo8bSzIWkAkFsGitxCKhig1l9LobMSsNxOOhdGpdSglJQWmAiQkLnRkcPTcsuS98vLfx6/ei0afTzASxRf1oVaqkWWZuit1VOdV0+RqSqZ2hhr+0LEZbSzNWUrHQAeekAcZmfVF65O90xOz8OtdpjicWFgkCEJaQ3PfCklBpjaTWDzG7pO7x9XnO/FAMKgNbC0frP9+/eLrGNQGHlryECatib5gH1qlljxTHpnaTMw6MxXWCnxhXzLYh2IhwrEwecY8ijKKuOIO8pvXNyaDuUHvZnX1r5hX2IherUeFijhxBoIDeENe4nKcDHUGxZnFKFDwRusbV4033YIgpULJl9d9mW9t/hYV1go0Kk3axUIj9Ze5HsQMXRCEtNK98Ov0dhKNRcdV1TH0gZCpzWSraWtyGfzdC+9OWV7vv+SnN9DLbXm3YTMMNsR64dwLZGgzWJO/hgprBVa9jR/vj9Hao07eY9uGOrpDHxCOhbGqrGTrs4nKUTRKDWadmQxtBiqFCpPGhEJSIEkSeaa8tAuJRptp30iz8NGIgC4IQlrpXviNp6ojkXd/4dwLlJhLqMyuxGawpZybbnOJHm8PGuXgLFij1JCpzWTH4h3kmnI53AjPvqcgkVT4+x3L+ez6edg91XzjD9+g1FyKUW1Ep9JxznGOxbbFhGNhml3NaFVaKrIq8Ef8+MI+agpr0u5yNNrq0ImuHJ1pIqALgpBWug0lEjnsoYZWdQzNuxebi/EEPdTaa1lfvB6bwZZybrrNJYbOgreUb2EgYOC7z32UGS7PjfD/3iXzwNLBjZrrOuvINeWmzL5h8C+J4sxiqvOrCcfCuENufAM+TBoTZxxnWJpz/fZemE4ihy4IQlrpmlQ9vvrxlH04h+eTh6ZZFlkXEZfjKCQFF3ovjNioyu6xs6dhD/ua9gFwT8U93FtxP8+8NZ8f//6jvw42rt5PxcJ9zBvyEHD4HCzPXY4v7MMf8ROX45g0JjI0GXxr87f45uZvUm4pR6FQMM8yj2xDNn2BPnq8PdO+v+f1IGbogiCMKF2qoSCjYMR88tCUjM1oY33RehpdjbR72llXtO6q3HOdvY7dJ3cTjUXJM+URjAbZc9zL22c/2qJxRWUdNutl8kx5FJgKUkoPc4w5+CN+1hevp8nVhDvoTtaq72vaR44xB41Sg0VnIRwLY9aZuS3vNjRKzXVbzTmdxgzokiT9G7Ad6JFledmHxx4C/haoAtbJsnxs5CsIgjCXjJZPTlf+p1FpWJi1ECAZZBOz9N0ndqNSqMjJyKGrT8Uzr3+0ynPbsnzuWH2ZQLSCTO2a5PH+UH8yGCfSQhadhfVF62lzt3G4/TDVJdXJUsuTXSe5ff7t5Bpzk9eIy/HrtppzOo1nhv408DPgP4YcOwt8Cvg/0zAmQRDSmIpFPtN9z3R590t9l5CR0av1KfXsWqWWmBzDqsnnxbdXEQhpktf5yvYrfHXTvew6dnzUl7CJtNCB5gO8c/kdLrkuUWQuIkObgcvvosnVRI+vhz0Ne3hgyQPJl7PXczXndBozhy7L8tuAa9ixBlmWG6dtVIIgpBi+Z+Zo9d/X857p8u65plzKs8pT6tktOgvHrhyj80oNz765PhnMb1t1jk9seIGy7MGVn0M3rkhIF4xDsRCb521mvnU+Zq2ZN1re4M3WNwnFQlRlV9Eb6OWtS2/R4+uZ8U0nZpLIoQvCLHA9uvpN9J7DUzK7ju3CpDGlnNPdZ+LAu59M/lxe2MHGpZdxBVxEZVUy2Kab8Q9fWj90nBadhVAsRH+kH2QoyyoDYE3BGlRKFXWdddy14K4bto58sqY9oEuStBPYCTBv3rzpvp0gzEkj1X83OBrY07BnWtIwU9VJMDHLDkfD1Hdf5MW3a4h/2KNcpYiz8+42XGE7V7wOlJKSx1c/nvwdxrO0fug4K6wV1HbU4ov40EiaZO35+uL1WPVWur3d123ziZkw7QFdluVdwC4Y3CR6uu8nCHNRukU+be42LrovUmIuGXFzhcnk3UfrJHgt160prOHfTz7NvhOZXOnZkDz+yfVN/GnNRuwDURw+DeuL1qe9zliLeoaOM1FZYx+w44/40aq0LMsdeSOLuUakXARhFkiXejjbc5ZluctGTIlMdjedkdIdldbKEa+buP/QQN/UpeGpA6uS111S1s5D67VolDnYB+yTnjEPH6dGpWFD0QYkJMqyyq7ayGIuk8bah1KSpGeA2wAb0A18h8GXpE8COYAbOCXL8l1j3Wzt2rXysWOiwlEQJmL4rPiC8wKLcxajkD6qbUiU4+1cu5M9DXvwR/wpM+xEL5XxBtF0M/G6zrq01w1EAnR5u3AGnISjYeS4kf3v3ZY8x2yQ+fYDMhrV1WOdrHTjhKsfLrM1by5J0nFZlteOdd6YM3RZlh8e4aM91zwqQRAmbHjqYU/DnlE3V5iKHHi6dMe+pn1pr/tC/Qv0hfpQSiqcnX+E21OW/Pxr2yNYM31oVNOzEcRIaZnZGsAnSiz9F4RZKl3L16HleOMt+btW6a7b5m7jfO95Qr5lNDf8ZTKYL5x3lm0bf8+O5WtHHaswNURAF4RZKl3NdyI/nthY4pULr/BG6xtTWn+d7kFS196C3/5DOtvvBsCgd7Fi+VMYLIeRJXnUsQpTZ8wc+lQSOXRBmH5DX4YGI0HOOM7Q7e1ma/lW7l5495QE0UTO+kj7UV6pnY/XV5j8LKPoX9Bo+ojEIgRjQR5b9Rjf2PSNSd/z42zKcuiCIMwuo20sMVUz4mJzMf/3yEV+e3BT8pjS/Cx9vII/qEIdVmPRWsjWZxOOhbF77GI2PgNEQBeEOWaqFgSNpMXhZes/H0r+nJHRSUHJC/QGHIS8BqLxKFn6LFSSigcXP0h+Zv6kV7Rejz42s5EI6IIwx4y2IGgywtE4n3zyXRq7B5LH7t9UR7OnDqM6E61KTV+wj3gkzuLsxRjVRipzKifd2XCy9fQfJyKgC8IcM57+J9fql4da+P7+88mff/bZVbzT/RMGQgMY1AbCsTAGtQGL1oJSr6TUUopWqQUm/zC5Hn1sZisR0AVhjhialtAoNQQiAXxh36Q2Nb7QPcCd//J28ud7lufz88+uRpIkOL+Zfzj8D0RjUUKxEEa1EaWkJM+QR2+gl80lm6dkheZ0p5DmEhHQBWEOSJeWcAfdE05LRGJxnj58iSf2NSSP/d2nw3xiQR6SJGH32Gnrb+OuBXdxqusUbe42vBEvm0s2s7JgJTIyMTmGQW2YdGfD6UohzUUioAvCHDCVaYl3m5z87cv1NPd4mZ/nZ/uaKAtyDXjD/mTuOnG/eeZ5rCoY7NNyrW0Fxms6UkhzlQjognCDGauiI93nU5GWsPf5eeKVBvaf7SLbJLOmqo6SPA9ZGdUoJFPKQ2Im0yDjaaErDBILiwThBjI0dTJ0Njp0BWi6zzVKDQa1ISUt0drXSsdABwuyFoxa6heMxPinP5ziV4evIMsyZUWN3LNSS3NfAzqVLrkJs81gS1asJDZnnkzjL2H8xMIiQZiFxkqdjPR5IBLAHXQDgzPlxGbJG0s2jljq1+5u5+mjx3nuiEy/X0NFoY+q+Q1E6eHElX56g71E41H0aj3HrxznrgV3JXPXIg1yYxIBXRBmyHgWx4yVyhjpc1/Yl0xLNDgaqLXXolFp6PJ1kaHNSG6OnHgwvN/awjdfOsHFbgOZRj/rlp3AanbgDvsxa81c8V0hFAnhjXiJxCM0OZvINeSSbchOpjsmkwYRC4WmhwjogjADEqmSWDxGp7eT2o5a9jft5/HVj1NT/FGzrLEqOkb7PBEQOwc6sRqs5BnzCEVD1Nprk1uwtbt7+McD5/k/bzejUujZURMnqDmE1WAmGDXi8Dtwh9yoJBVdoS6KMopwB914I17eaXuHv9/69ynbw00kCIuFQtNHdFsUhBlQ11lHLB7jnOMckViEAlMBKoWK3Sd3Y/fYk+eN1RJ3rM8TKZk8Yx6hWAiD2oBRY+RCbxPvNYX419cK+MVbLSwpCfDNT8X4xDLIMmQSjAbRq/QoUNDsaqbeUU8oGkKSJPQqPeXmcvxRP883PJ8y3on+WyTSRgpJkdzcua6zblLXFURAF4QZ4fA56PR2YtQYMagNSJJElj6LaCyaEsjGajOb+DwQCbC/eT/vtr2LRqlJuY9JY6LCWoEv7MMf8RMIZLH/aAW/PWwkJ0PP81+8iS/cpkKpGuxpnjj3sucynqCHHEMOMTmGRqnB7rETlsPEiZNvysfhc7C3ce+kgnpijEOZNCYcPseErykMEikXQZgBOcYcajtqKTAVJI8Fo0HyTHlXBbLxpDJCsRCb521OvpBMpCwSKRmb0UZ1zgZeOh7hQns+alWMr91VxF/cWo1SIZHv+eilplVvZUnOEl6+8DJWg5UcfQ4yMka1kV5/L8FIEFkjk63Pxma0JWfTE02PiIVC00cEdEGYATWFNexv2s9lz2X8UT+eoAeFpGBjycZrDmSjVcLUFNbw0vm9nLio4dVTNnxBWFHu5Yn7NrCsoCx5jeEvNUvMJWwt30pVThUKSUG5pZzXLr5Gf7gfvVLPPPM8FJKCCmvFpOvNRYXM9BEBXRBmQLG5mPsq7+P7h7+PQlKQY8zBqDZS76hn28Jt13St0Sph+gYy2F+7hPpOP0XWMDtvl/jU8k0j7rc59Pju47s5eOkgkVgEs9bMnfPv5ODlg/gjfmxGGxXWCmwGG/2h/knNpsVCoekjArogzJCIHOHTSz9Nl7cLT8iDWWsm35SPfcBODePfFi5dyqK738dpsYCBAAAgAElEQVTBs/n8wwvvkm3U8E8PVfOpVUUoFNK4rmn32Onx9tAX6CNbn00gGuCM8wxV2VVYdBbKssowaUxpm21NpARxohUywuhEQBeEGeLwOSi1lFKeVZ48NlKv8JGCpN1jp6m3id9d+B1alZZFWYsJDtRwuCGLcFRBzUIvf7RWw4YyUCikcQfbus46yrLKKMwspMnVhCfowaKzsMi2iLsX3j3ibFqUIN5YREAXhBky3peBdfY6dp/cTTQWJc+URzAapHOgk3WF63i15VWa+5pZnL2Yiz0Sr9atIhbOQW9oZ8vKdjaWz0eWdOxt3Mu6wnUc7Tw6rmCbSOMoJEVyEVLiYTPabFr0Kr+xiLJFQZgh6WrIW/ta6fX3suvYLvY07BkM5id2o5JUFGQUEI6FOec4Rywe4/mG53EGnBgUBVy8tImLLX8EcROGnGdYsOAVskxB6jrqCMfCWHQWnm94ftz13omHzVDjqTwRJYg3FhHQBWGGDK8x90f8SEjo1XryTHn4I352n9yNJ+TBqreikBTJhUFXvFdod3dy/lIRf6jdwKWubAryTlO1+Dkk7UniH/YeN2qMNLmaMGlMdPR3pARbp9/J2Z6zvHDuBfY07LmmBU0jmeiDQJgeIqALwgwqNhezo2oHO9fuxGawUZZVljKDjsaihGNhAtFA8js6lY6GDpmzDZ/k/KUl6AwdzK/4DWHDC4Tjg1vAqZSq5LmeoAdv2EtRZlEy2Dr9TmrttXiCHorNxfgj/pQFQmMtaBrJRB8EwvQQOXRBuE7SlR/mmfLwRXz4wj4AYhEzRxoK6eotwJYhU1q6D1l7BoPOQm8ozsW+i1RaK8lUZ+KP+JFlGbVSjTvo5sGqBznaeRSAC70XUEgK4nKcRdZFaXPdE6k8ESWINxYR0AXhOkmkK8KxcLKyJBgNopSUVFqXcrBeQX1rMTIyy+dfYtG8duKylgaHkZa+FpDBrDVTkFnAzcU3c8Zxhm5vN1vLt3L3wrspNhdTkFFAXWcd7Z52is3FLLIuwmYcfOkZjAR5t+3dSXc8FCWIN44xA7okSf8GbAd6ZFle9uExK/BboAy4BHxaluW+6RumIMx+w0sI1ZKaV5peobWvFZvBRp4xj2g8hhRawX+8OQ+3T0VedgefXBNleWExv/7gEJ0DnSyxLWFN0RoC0QBdA12Eo2FicoxbS2+9KigPDbZDN6Rw+py80/6OKDecY8YzQ38a+BnwH0OO/Q3whizL35ck6W8+/Pmvp354gjA72T129jfv50TnCWRJZr5lPuFYmPKscvJMeckNKCQkcgw5BKIBWnr6iQ88QFuPiTyzzJfujFBsMyaDcEyOoVPp6Av1kZeRh0FtwKwzk6HNYOfanaOOZ/hy+9M9p5Flmeq86mT+HkS54Ww3ZkCXZfltSZLKhh2+H7jtw//9K+AtREAXBGAwmD996mma+5rJ0mUhIbG/eT82g42ijCIUWgVd3i6y9dlcdF+kOreG+tYSWi4VoFDE+aOaGItKulAqwKQZzLE7/U5cAReXPZdp7mvG6XeSoc0gQ5VBqbl0zDENz3UHo0FuKb0lWXMO07cnqDBzJppDz5Nl+QqALMtXJEnKncIxCcKskW4lZl1nHc6Ak2x9Nga1AYBoPMrFvov89txvWZ2/mo7+DgozihhwV/C7ppUEQjrmF/RQPu8MNQvWYFAPlv0lcuy19lqUCiUKFMjI9Hh7UClVaJVarAbrqONJ99JzT8Me/BF/yu8iyg1nv2kvW5QkaackScckSTrmcIjFBsLckVj27o/4k3Xkexv3csF5gXA0jF6lBwY3T/aGvQQjQRQoCMVCtDnD7Du6kA773ahUIW5ddZzqRR+QoY8ny/4SJYHvXn6XzoFOOjwdeCNeSjJLWJi9EL1KP9hbHWnU8aTrXS7KDeemiQb0bkmSCgA+/G/PSCfKsrxLluW1siyvzckRT39h7hhp5x1X0IVGpUnWkncOdGLRWVAoFEgYqG9ZzKWLn6Xfl8nWagf/fVuALLOLbm83S3OWJl9MFpuLWVe4jgvOC/gjfvRqPRVZFfSH++kP9hOJRril9BbicnzU8aRbGTrRunPhxjbRlMvvgEeA73/4371TNiJBmCVGamObpcsiGo/S3NdMXI7j8rtAlrDK99DUuJ5wRMPCoi5WzG9j8/zVOHyOtBUqAPYBO2uK1qCUlFz2XCYSi2CVrMTlOMtyl6FT6ZJpnbE2mB5OlBvOPeMpW3yGwRegNkmS7MB3GAzk/yVJ0p8Bl4GHpnOQgnAjSeSpT145iVatpTqvGpvBhtPv5HT3aUKREKsLV6NRamjtayUaLqCv5076vdmU5cg8tEHGkqHHoF7Njqodo97L4XOwPHc5dR11ZOmyaPO0IUkSvrCPfGN+SitbsROQMJ4ql4dH+GjrFI9FEG54Q9vFzs+az77mfdTaa1mUvQhv2ItJa2JzyWZ0ah1tLiddV7bQdtGARh3irtU91MyPc9Z5hu4rgwuA7B771bPyIS82W/paKMooYn3xeppcTQSjQVxBF2XmMkrMJSmzerETkCDJsjxjN1u7dq187NixGbufIAw1kY0Yhn8vEWAzNBnUdtQSj8e54rvCJfclbAYb2xduZ2F2Ja+e8fHaaS2xmIpbl8DS8jZOdr+H0+9Eo9Rg1VuT+3c+svKRtP3FTRpTsl59Y8lGSi2lySA9Ur57or+jcGOTJOm4LMtrxzpPLP0XPhYmuhHD8O8dsR/BHXSjltQYNUYMagNatZaLfRdRSkpeO9/Cc84FdLkzsFlcLJl/lgeqNwGlnO89SSQeYXnecnQqHcFokJNdJ+l4p4OV+SvJMebg9DtT+osnNsPoGOhAp9KN2StF5MU/3kRAFz4WJroRw/Dv5Wfk4w64aXY3s7ZgLf2hfuod9WilXJxX7sXtrkSnCVC1oI7FxQH0Gl3yWh3eDpSS8qPa9FgUV8BFX6iPuxbehTfs5WDrQbaWb03Jg5daStGpdGOuBhUEEdCFOSORbmh0NtIX7MOqs1Jpq6SmsCZtBch4mlMN/16FtYIj7UeIxCP4Ij5aXG14etficd5MTJbIzT3B+kVuWj2NuILZ3JZ/W/K74WgYo8aY/LnT24lGpQFIlhnmmnI54zjDVtNHr6jEi01hvERAF+aERGokFo/R6m5FKSlxB9zo1Xo6BzrRKDUpFSDjbU41vHLEZrCxNHcpCoWCxg6JprZPEQlnk291Ulx0hHyLkpgcx6K1UGmtRKPUEJfjeMNe8k35ROKRwZpylR530E0sHmNJzpLk/ZbnLuf1i6/TH+oXLzaFayYCujAnJFIj9T31mDQmDGoD/oifLl8XS3OWEogEuNR3CUfAQTga5nL/ZfRKPbeW3jpqc6p0lSPeoIaA43Oca/Jg1AdYsvg9lhbLVFjXYTPa6A/1Y1Abkm0AEn3Cv7rhq7za8irOwGBfFq1Ki06pY03hmuT9dCodW8u3Jhf8iP7iwrUQAV2YExKpEU/IQ5YuCxgMju6gG5PGhN1jR+ajiq5ANIBZY065RrpFOEObWnV4ujnTmsebZ7OQpAG+dkcl96zUcqClO1mVklhCnwjCQ6tX6jrrUCkG+68UGAtYVbCKHm9Pyix+tAoWQRiLCOjCnJBIjZi1ZgLRAAa1gWA0iFlnxhv24gq6WJG3gur8agAs7RbcQTdNrqZkx8GRctVFmUU0dKh59tA5Lrv83LM8j2/eu4Qiy2CvlvtVo+/YM7RSpiqnKhm4ty3cBiB2+xGmjAjowpyQSI3km/Kpd9QTiASIxWOUmktxB91k6bJSNkyusFZwpOMIXQNdKbPj4bnqVqeP//VyPW81OliYa+LXj61n40JbyjljlQqOVmGzo2qHCODClBEBXZgTEo2snm94nh7vYK+4xbbFydWUdZ11qS83jTaW5iylY6Djqtmx3WPn3bY6flvr4fTFbJRKmfvXynz1E/Mps9pGG0Za19pjRRAmSgR0YU6we+wc7TzKirwV3Fxyc3LGPbQUcfjLTaVCyZfXfTllhtzubucf33ydgx9Y8QZzKMq1U1Faz4KihbzSbJ9Qflv0WBFmigjowpww1sKh0XanT7ywrO/s43d1WnrcuWRleLm1qpF5ORH8ETVd3i7yTfk8efRJFmQtuKZl9aLHijBTREAXZp10/UpGSms0OBrY07BnxMVDdo+d3555meNNeRy5kItCEWHBvDpiurdxRHT4nUYKTAU4/U56/b1E5SiV2ZUcajvEc/XPsaV8C9sWbhu1wVaOMYd1heuwD9jFy09hWonmXMKsMrx5VWK2q1FqMKgNKWmN1r5WzjrOcmvprQSjQc70nKHH25MMwoUZRfyP3/+Ol08o8Aclbl4ERusbnOutIxQNUZhRiCRJ9Af70al0yRa53ogXi85CriEXk9ZEhbUiJRUz0hhFOaIwUaI5lzCrjdQ1cKTUSiASwB100+vv5Yr3Ct3ebtr729k8bzPhWJi6jjqMGiN5pjzqHfVcdsLxxoWc61QN9ii/Q2aeDV5tjqBRaJCV8uCWcYrBTb26vd30h/rRqXWDGz9LEu2edvJN+VgKLSkLkibaN0YQJksEdOGGM1pnxJFSK76wj3WF69h9YjcxOUauKRdfxEdHfwcOvyPZGTEQUlLbmIe9qwCjzsfDG2MsmefFohsMujIy87Pm0xfoS86ubXobroCLcks5rqCLWCxGX6gPd9CNJ+ThltJb8IV9yfGIqhbhehEBXbjhjDbDHa1ixD5g59ayW5OfeUNemvqasHvsLLYtITKwnoZLC4nGVNy6RGZ1RRefWrKNvY17UUiDQVetVOOL+NheuR2bcbBEsT/Uj4yMQWMYnP37u9FIGjQKDUqFknfa32FjycbkeERVi3C9THSTaEGYEnaPnT0Ne9h1bBd7GvZg99hx+Bwpi4BgMNg6fI5Rd6sf+j2nz4k74CYcDSOH5nO24V4+aK5Cq3Nwy8p3UZpfosPbDJCyWfKynGUszFqIRqVJuf5tZbexNGcpaqWacDSMUqkkx5hDtiEbWZaRkJJjHW2MgjCdxAxdmFaj7aAzUmpFq9SOOMMdrfxw6My4ydVEpqaElt5N+B0lKFX92Ar2IulO0RXJIuALAPCrU7/ikZWPpOztmRjz0OvDYB17YUYhC7IWcLn/Mp6gh0pbJavyVxGLx5LfH22MgjCdREAXps1YuwSNlFrxR/y4g24gfd32SEvtE/Xe0RicaLbS3F5BLC6xqcrH5iVBXjrfTqPLSUFmPpX5g61tL7gucKD5AI+teSx5nZGuf/+i+7nsuYzD52BZ7jIqrBXYDB91VxxK7BwkXA8ioAvTZqxqj9FecE5khltsLqZQ8wn+YX8LPR4LeVYHn96goDIvC9ATiodYlrOM6rzq5Hey9dkcu3KMx3jsquul++viy+u+nFKSOLS7oiBcbyKgC9NmrGqP0V4eXusM197n54lXGth/tot5ViM/eDAPZ/wSFp0l2XwrEo2Qa8lN+Z6MjCRLV19vlL8uRDpFuFGJgC5Mm7GqPaZiSXwwEmPX2xf5xVuDLzi/dkclj98yH51aid2TmRJ4ty/aToOzIbljUCAaoC/Yl1KhkiA6JAqzkQjowrSpKazh6VNP4ww4CUfDaFQabHobj658FJjcy0NZlnm9oYfv/j7Rozw/pUd54vrDl/knxuMKuNCoNCzMWpjsSz6UqCUXZiMR0IVpNbScL93PE3l5OJ4e5ekUm4t5dOWjI1bdDCVqyYXZSAR04SqjlRpei7rOOsqyyliRvyJ5rD/UP+El8L5QlJ8dbOapd1rRqBR8694qHrm5DLXyo+UUY5VJjvf3Eh0ShdlIBHQhxVilhqN9b7wdEK81bdHubmf3eyfYUycxEFBy97Is/u6+1eRm6lLu3ehs5KL7IstyllFqKU0ZO3BNv5eoJRdmIxHQhRQTaSw10kNAo9SMmbYYa9b8dnMT33zpNO1OPUVWmYc3DWDOuERYngcUp9zbHXKjklScc5wjQ5uR3Cu0rrMO4Jp/L1FLLsw2kwrokiR9BXgckIDdsiz/eEpGJVw3E5lVj9UBMXGN4WmL0f4ayNDk8ePXL/Cr91rRqrV8+qY4GytBoTDS2teT3Giipa+FoowiMrWZDIQGsOqtBKKB5ObPQ8cuXnIKc92EA7okScsYDObrgDBwQJKkV2RZbpqqwQkzbyIvAye6QCjdgyAuw88OHef1Dwz0+sKsLPfz4DotmfrBPLnT76S+p56oHOXmkps5Yj+CO+gmQ5OBWWsmEA2gU+mSD5KhYxcvOYW5bjIz9CrgiCzLfgBJkg4BO4B/nIqBCdfHRF4GTnSBUOJB4PQ5aXI10eaE8xeX09uvYtU8A//+6DqaB97EH/ECg9ducjWhVCjJ1mWjkBTkZ+TjDrhpcjVRYa2gtqOWQCSAWWe+ahWneMkpzHWT6bZ4FrhFkqRsSZIMwD1AydQMS7heEi8DE90HDWrDmC9EJ9pdMMeYQ5u7jXcuneS9+jIOn7qZgYCWpRUn+Mln57G82HzVtbsGuojJMSqsFQBUWCuIxWN0+7qxGqwsyVlCNB7ForWkjH0iv5cgzDaT2oJOkqQ/A/4b4AXOAQFZlr867JydwE6AefPmrWlra5v4aIUbQroXmcA1lzq29bXz5//1f2lpryIaU7OgqJ3SonqqCyrRq/XYDDYcPgcKSYGMjCzLyZx5eVZ58jqtfa10DHRc8+bNgjBbjHcLuinbU1SSpL8H7LIs/2Kkc8SeorPfRPfLHP4Q0MSq+PkbDs5d6Sc3y83i8g8ozlZSYa0gHo/zRusb3Ft571X3AMR+ncLHzozsKSpJUq4syz2SJM0DPgXcNJnrCTe+yZY16hR57HojQIO9BaMuwk3LmtiwQMt866bk+W+0vkGuKXfEPiqiPlwQ0ptsHfoLkiRlAxHgv8my3DcFYxJuYBMtazSpLdRdyGTfKYjEDCwobmZ1RQ8l5jwOtx9GkkguBur2dnP7/NtHvIeoDxeE9CYV0GVZ3jxVAxFmzmSW9k+krPHoxX7e/CCbbo9EYbaL5QsbsWXKuINuyrM2ANAx0IFOpSPHmMPW8q3oVLqUa4gSQ0EYm1gp+jEz0aX9ie/2+nt5o/UNDGoDOqWOYCyIUlLy+OrHrz4/2aPchkHnZ+2Sc3gV76PVVBKMqjHrzMDgzFyn0rFz7c6UMYIoMRSEayEC+sfMRHLgkPogWJ2/mn3N++gP9pNjysGsNbP7xG4AaoprUnqUy7LMmoorGCyHyTFm0eRSc6r7FPMy57F1wVbg6tn3ePqoTFUDMUGYS0RA/5iZaMOsoQ+C+mA9FdYKWvpa0Cg0VFgrcAVc7Dqxm5YuA794syfZo3xt5WW0Wj3h2C00uZqw6q14w160Ki1WvXXELdxGy5NP5q8MQZjLRED/mJlon++hDwJPyIMr4MKkNhGVo0iShDJeQO25Qv7w3qVkj/LSnCDfe2c3ChRY9BYqrBXcVHwTPb6eSVWpTPSvDEGY60RA/5iZaJ/voQ8Cs9bMhd4LGNQGtFImJ5pKaLhUgEIRZ+sKD7/8zDa6vZ3sbdyLTqVDQiIUDVFrr2V98Xp0Kh13LbiLHVU7JvQ7iN2EBCG9ySz9F2ahiS6BH7oEf0HWAiKxKO09uZxreJD61iLybR184Q4721fpUSsVyVl0dW41gUgAAIPawOnu0+NqCzCaxMNlKFEFIwhihv6xNJE67qEvKlsdIcLOL+LqNWEyuLl9RSMrik0oFcpkoE7MohVaBeuL1tPkasIddCPL8qRz3WI3IUFITwR04SojVZBkaPL4oLmS/3i/DZNWxdfuyqUwN0xvwHZVpcnQFI3NaMNmtNEf6segNkw6zy12ExKE9ERA/xgZHqiLM4qxD9ivarI1vILkpfN78XuW8x/v9eMNwk2Vcb65bQnLCsqAlWnvccF5gZa+FpblfrQd3FTOosVqUUG4mgjoHxPDS/3a3G08e/ZZNpZspNRSSpu7jf1N+wlEAmQbs6nOrUahVeAeyOQ/39XR1TdASXaML94OWZleDrW/gsVw/1W14Yl7LM5ZjF6t56zjLP6In0W2RWIWLQjTTAT0WexaFtcML/Xr8nWRrc+my9tFhiaDc45zqBQqevw92Iw23rl0ErfzFk5e1KHRKFiy4Bh/vnk1CgkSm00MLxMcfo/yrHKyDdn4I34A9jXtE4uABGEaiYA+S421uGZ4sL/gvMDinMXJ73uCHiw6C56QhyZXE0aNcbDE0KWktbOE+ovzicZU3LYUYvr9FFqsHwZzcPqcNLoasXvsAMkAna6cMBgJcrD1IPdW3isWAQnCNBNli7PU0NmwQlKQqc3EorNQ11mXDPb+iJ88Ux7+iJ+Wvhba3B9tLmLWmXEH3Zi1ZjwhD3qVHrtTS2fbn3DqwiIspgE2rXyb26s96LRQaCoEBoN5bUct/cF+Sswl+CN+9jbuxe6xpy0nPOM4k2yFO3ycgiBMLRHQZymHz4FJY0o5ZtKYcPgcaYP9stxlnHWcTW7llm/MpzfQS74pHxVW3jxVwqGTayBu5MGbPNxcfRyTYQCD2sDjqx5HqVDSH+qn0dWIQlIQl+NUZlemBOjh28W19rVyvOM4Lr+L9+3v4/Q7U8YpCMLUEimXWSaRSjnRdQKdSkd1bjU2ow34aHFNutRHKBqi0dHIecd59Go9W8u38pV1X+PZo1fYf6qMaBzWVfbxYE0GEVnGHaxISYsUZBQkZ/8l5hIqsyuxGQbvm1ilObScsMHRwEX3xcGgr8tMWSmqUWrEIiBBmAYioM8iQ/PmNYU1vN32NocuH2JzyWZ0al2yLLCusw5v2Es4GqbJ1cTprtMc7zpOhiaDFXkrMKqNvNnYyd7DLjr6VGxZlMOf35bNleAHOHzp67qHlgn6I37C0TDvt7+PJ+RBrVSzNGdpynl7GvZQYi4hHA1T21GLUWNMrhStsFaIRUCCMA1EQJ9FhqZSMrWZ3FZ2G6e7T3Os8xh3LrwzJQg/feppmvuaUUkqznSfAUCn1OEagAs9axjoX4hR7+WpRzaztSoxm18w5hhqCmuS187SZaFVaukN9NLj7cHusSfv7/A5UCqUtPS14I/4cfgdGNVGjGqjeCEqCNNEBPRZZHgqxWawsaVsC93e7pRGV8XmYvJN+TgDThodjcSJU2QqI+TZzCX7RkCisOA4ZYXNbK168JrGMPTa4VgYs87MbXm3oVFqUsoYFZKCt9vexmawUWIuIRgN4vQ7WVO4RgRzQZgmIqDPEnaPnZa+Fo7Yj2DQGJCQiMtx1Eo1y3KWXXV+XI6zpWwL4WiELlcOffY7iEetaAz1VM1vwB25SIllw4TGkri2QlKkHBva7VBGRpIG6xwlPvyvJCEjT+iegiCMTVS5zAKJ3HlRRhGhWIjTXac523OWUDSEO+imy9uVrAlPyDHmcMnpp65+LX1XPkNUDiJbfkJG7m/xxdqJxWM8eI2z86HXHqvboSzLbC7ZjFappS/Yh1apZXPJZmRZBHRBmC5ihj4LDM2dX3BeYEA/QCASoC/Yx/aK7WhUqekOXyjK8QvF/OZIF0g6rLkH0Zjep9PXTo9fxqAx8Dcb/4aa4om1sB1Pt8McYw7+iJ+bSm5KHks05xIEYXqIgD4LDM2dy5LMitwVyMi4g25sRlsy3SHLMr//4ApPvNJAV3+Q5aVBigrrCMQdeMP5VNoWYDVYKTQVEpEj7Dq2a0JL8cfT7VC0uBWEmScC+iwwfLegQHRwwwizzgwMpjui4Vwe3n2EIxddLCnI5GefXcUJ5/PkmTam5Lp7vD280foGRZlFIy7FH0+PmLG6HYoWt4Iw80RAn0JTtRN9uja3RzuPArAgawHvtL+DLMvcUnoL3QMD7D+p4NRFNRn6Ab73R8t4eN08lAqJU70KDl46SCQWwawzU2GtSFmKD1fvxzmVGzCLFreCMLNEQJ8i4w2EYwX9dNc52nmUdYXrsA/Y8YV9bCzZCLLEiYtaDp3NxBeU+Oz6eXz9zkVkGTXJ63R5u3AH3WTpsghGgrx16S3cATf3Lb4vZexD9+MUGzALwuwlAvoUGU8gHE/QH+k69gF7stb89fMX+O7vG2lzKijLifOThyvZuqjyqvGUZ5VTlFFEk6sJT8hDlj4Lg9qATqVLOXdohYrYgFkQZi8R0KfIWIHQ7rHz5NEncfgc5GfkU2GtSPZCGZrueK35NSRJwqKzDJ5jtCWv4/KF+c7Lx3n5VC8mHXxuU4zFJQOc9/yBRZ7Urd2G7umZ6PUSl+M0OBpwB93J8Q1/WTk0X58gNmAWhNlB1KFPkdFqsxMzc6ffSb4pP9moyul3JjsPJs7RqrXoVDpCsRC1HbU4fU76g14a2/PY8k9v8crpXjYuCvHtB2BDhYRFl74d7UjjWWRbxP2L7segNtDt7cagNqT8hTC8Y2J/qB930J3cnk4QhBuXNJmFHpIkfRV4DJCBM8D/I8tycKTz165dKx87dmzC97uRDU2nDJ35Jio9/BE/9T31hGIhDGoD/ogfrUrL0pylydpsf8RPOBam1j7YzCoux3G6zTS2rqDHo2VNqQm99fdkGn0pM/hE2eLOtTvHNZ6xcuFT9XJXEISpIUnScVmW14553kQDuiRJRcC7wBJZlgOSJP0XsE+W5adH+s5cDugwciDcdWwXeaY8XH5XsvOgVqmly9vFmsI13L/ofvY17RtMkUgKnH4nH1y5xOH6XHp6yzDqIty1cgCr5TLOoANfyEe3rxt3yM2K3BVUZldSYi5J6ecy2ngEQZhdxhvQJ5tDVwF6SZIigAHonOT1ZrWRyvQS6Q+b0cb6ovU0uZro/rBNbWLGnDjHoMrkdIuNfadsRGIylfMu8tjmebzXcZymvj5KM0s53jnYCteitdDsasbhd/D14q+PezyCIMxNE86hy7LcAfwTcBm4AnhkWX5t+HmSJO2UJOmYJEnHHI6P5y41Q/PSVoOVpblLWV2wmi+v+3JK7vpUW5C/f0nmpWMKsjJ7qVnxKsGhmVcAABHPSURBVA/fZEKvURCJRcjWZ1PvrKfKVoVRYyQajyJJEhtLNmIfsI8xCkEQ5roJz9AlScoC7gfKATfwnCRJn5dl+T+HnifL8i5gFwymXCYx1mk3nhTFaOeM9NlYqybtfX6eeKWb/WfzsWXIPHRzLwOKI9QU1iQrYRIrRHv9vSy2LU7uFapVaim1lIqyQkEQJpVyuR1olWXZASBJ0ovAzcB/jvqtG9S4asTtdew+uZtoLEqeKY9gNEjnQCf3L7ofYMTvA2kDfTASY9fbF/nFW80AfP3OSh7bPB+dWsmehgz8EX9yfBXWCg5dPoRJYyIQCSBJEr6wj2VFy0RZoSAI8P+3d+fBcZfnAce/z15aSavDOm1JPkA2xifYWMYMh00gFMMUSgcCZNKGhqOQtJlJMkOTkGnaMhnazJR0miZtTNoJ0EKgMGCnKYZiDJgEX8gYXxjL91q3tdaxh7TH2z9W+lkyknZlrVba1fOZ8Yx+u+/uPq9Xeva37+9535fxJfRTwBoRyQOCwE1Axl7xTDQxyNvp5Zn6Z3DYHJQXlBOKhDjYdpDF5YutksHhHv9Gwxv0RfuGJPrXP91ImX0dP3+nlVMdAW5fNovv376I6uJcK54LF7dyOVzMnzGfxWWL2dO8h0pPJXVVdbgcLl30SikFjCOhG2N2iMgrQD0QAfbQP7SSiRJNDNrVuIuoiVKRW4GIWKWGTT1N1szL4R7/wckPuG7udVaCD4UK2bQ9n8+aTjC/wsN/PXQ1184v+1w8ww3TPHDlA9aHy8AZf54zTxe9UkoB46xyMcb8EPhhimKZVIlmSA4MlwQjQSuZux1umrqbWF292mpfmFNIu7/dqmTxdnlZVb2KHFshb+4Vth4Au93BzcvP8a/3rsdpH/m69EhVKlq9opQajk7975do/e7y/Pgwy64zu+js7SQSixAlSmluqTWLcuPhjZwNnOVA2wHsYschDqo9NbxSfxzvmXK6gzZW1xq+sLyLysK8UZO5UkqN1bRK6KNVqCSqRKmrquNA6wFC0RAGQ1+sj0gsQo1n6ON/uvOn8RLDglKKbJezabebM2fzceW0cPOqZpbXeIiKnbqqG5KOLZn7lVJqXFP/x2oyZ4qOZyr8gF9+9Ev2t+0fsr64y+4iz5lnzdLcsHsDBc5K3txr571D4LCHqZ19EJP7ATVFVdjFzsMrHx6y/Vui2FIRu1Iqc6VrpmjGSKaKJdEZcKLd7mMxw7HGSn5TbyMQgtqaZi6fd4QcZ4Qc+wqumX0NXb1deLu91HE+oSeKTdcoV0olY9ok9NGqWJLdnEJErB2ARITm7mYOtx8mSpR3jxynrXkdR1ocVJWE+Oq6MCd66glF+zjScZpKTyUfnv6Q2hm1+Pv8SceWzP1KKQXTKKGPVsWSzBnwLu8utp7YygnfCWxi41TnKTpCHeTbKimL/Rm/P30lDkcXf3FzJV9atZyPmnaz72w33m4vC0oWUJFfQTASZNvpbSwqW8Rrh16zvg2IyKgVNrpGuVIqGdOmzGK0db7b/G14XJ4h7QfWKQesSUXFOcUsLF3I6a7TtAc68IRvI7fzKfxdyyku2cO8Bc/RZn7LnOL4yodr561lTtEcCnIKMMSvVfT09rC3ZS+BcMCavt/a08px3/ER1yDXNcqVUsmYNmfoo1WxJDoDHjyp6HDgMAWygljwi9ij84g6DpIzYxP2/ABOWwVnus5YzxEzMW6YewNHfUc5FzpHkbuISk8lwXBwyLeBeTPmEQzH69sPtR3CF/JR4i6xZqAmqsBRSinIkoSebEnfSBNyEtWgDzxvo8/BrvoH4g+Ss3S5f0Ke51Nszjya/X7yc/JZXrjcet7y/HIC4QDX1Fxj3fb6p68POx7u7/NTV1VHY3cjs4tmW3EMHsvXBK6UGk3GJ/RkLmgmSviJzoALXeVs2lZJoPf8f1eo6HuEIj48tjKisSgxEyPQG+DuRXdbbYb7oLCLnVmeWUP6MJaxfKWUGknGJ/TNDZs50nFkSG34wB6bF9Zwj1bBMtIZ8FNvHOIX77ms47olO/mo43lKxEOlbQYtgRY6I53UFNSwpmaNVV8+8CHS3dvNqc5TlLhLKMkroSyvjJcPvkyOLYfLyy9nYelC7DY7a+eutXYtGkyrWZRSycrohO7t9LLl+BZmeWZR7C4mFAmxw7uDuuo6qzTwYs96dxw7y70btlvHf7SilLXLm2jzV1PW9Af4Qj6O+44zzzUPj9ODy+miI9SBt9NLU3fTkGV2qwuq6ertouFsA83+ZhaVLqLZ30x9cz2+oI9vrflWUmP5Sik1moxO6Lsad1HpqUREsInNWjRrX+u+IePfYznr7QyGuerJ/yMSi1eluJ02dj5xM4Vup9XG2+nl8bcfx+VwUZJbAgZ6wj2EY2Fe2PcCDR0Nn1tmNxwN0+JvYWHZQvKcecwsmEkgHCBqotZEo0Rj+UopNZqMLlts87exrHwZ/j4/gXAAYwwxE6O1p9Uq6Rs46x1suLNeYwx/9conXPG3b1nJ/JVHr+HTJ9cPSeYQH56xGzvF7mIisQguh4sl5UuoKaxhy/EtRE2UktwS60Mm35XPmZ4zdPd2W0vtAuQ6cumL9FnlkQNj+XnOPFp6Wshz5un0fqVU0jL6DH2gimRg42VfyIfT7uTGS24csqhWorPedz5t4Wu/Or/GzNfX1fL4rZeP+tr57nzKPeXku/Kt2/x9foLh4LDL7PZF+ijIKSAUCVm3ByNBXA7XkA8XrWZRSl2sjE7oA8m62F3M1TVXW8l6/fz1VpvRKljaunup+9HbVtuqIjdbvrOOXJc94WuvmrWKbae2ISK4HW5CkRBng2dZUrGEKk8VB9sOAvGzcF/Qx0zPTGYXzqbZ30xpbikGgy/kY/6M+TpBSCmVEhm/2uLFLCtrjOHPn/+Itw6eH0f/7TevY0lV0Zhe99mPn6Ut2EZfpC9+pp1bzi21t7CzcSfRWJTGnkba/G3WCouzCmaxuWEzu5t2I0ZYWbWS9fPX6xm5UmpUya62mPEJfaw27W3kmy/usY6/f9vlPHJDrXU8lg+Ikdrq2uVKqVTS5XMv4PUFuO4ftlrHi2YVsvEb1+JynL8unGzN+kDb0TbL0ASulEq3rE/o0Zjh/g3b2Xmiw7rtne+s5dJyz+faJluzPpbEr5RS6ZLVCf0/t5/kB6/vt47//o+Xcd/qOSO2T7ZmXafoK6WmoqxM6A2tPdz89HvW8bXzS3nua1djt8mojxttpubgIZb65nrqquqGtNMp+kqpyZbRE4su1BuJ8sWn3xuSzF/9xiLuvraJf69/htcOvYa30zvi40dad7ymoIaNhzdaa5i7HW7eP/k+7YF267E6RV8pNdmyJqH//N0GFv5gM0da47NC71pzlifuaWTzsReHbCax8fDGEZP6SDM1vd1ea4jFJjauqLgCEWFvy17dcEIpNWVk/JDLPm8nf/gvH1jHC6v9/MkNYQpyith6Yiu+oI9cZy4HggfoDHXitDvZ3LCZh656aNjnG65C5cJVEMvyy7h+9vXsbtytG04opaaMjEzo3k4vH5zcxZOv2vH3np/V+eS9fdjsYWtsOxwN47K5ePPomyytWEqxu5hgOMiW41u4df6tSSfg4cbW3U43t8y/hbsW3ZXaziml1EXKuCEXb6eXb7/6Pt99wWUl85WLPuSx2/fze+9bhMIhq21RThHN/maisSh5zjxsYkNEqPRUWtu7JUP39FRKZYKMSug7jp3luqf2svOz+BT9q2oD3Hrtb5hZ2sG53nPkOHPYdnob7f74xcoFJQtoD7aT68zFGEMgHMDf52dZ+TJrhcNk6CqISqlMcNFDLiKyEHhp0E2XAn9tjPmncUc1iLfTy7YTu3ji105iJl526LQbfnSf4eOWPbQHYrQH2jnYdpCllUvp6e1hb+tebpx3Iy6Hi0sKLyHXmYsv5KMop4il1UtxOVzWiofJ0tmfSqmp7qITujHmMHAlgIjYgTPAaymKCzg/I/OTY5XETHwbuLpl27i+topc1yUcbj/MUd9RIiZCgbOAvnAfUROlvrGedn871YXVfHnZlznZdZJid7FuGqGUymqpuih6E3DUGHMyRc8HnJ+R+YXFOSyqilFbCcd9Vexvjc/+POo7SjgWxmV3Uegu5LOOzwiEA1R4KrhnyT309PVwsuskq6tW4+32akWKUiqrpSqh3we8ONwdIvII8AjAnDkjT7sfzsBUfJtAbX/V4NziuQTDQc50n8Fhi4dfkV9BUU4RDf4G+qJ9lOaWYhObVZXi7fZqNYpSKuuNO6GLiAu4A/jecPcbYzYAGyC+fO5YnnukqfiXlV1Gm78NqRU6Ah34Qj78YT8IFLmLqC6ottrrlHyl1HSRiiqX9UC9MSblWXO0csHy/HKqPFXYxMacojmsmLmC8vxyXHYXC0oWWM+hU/KVUtNFKhL6/Yww3DJeo5UL1lXVYbfZWVy+GKfdSVNPE6XuUupm1eFyuLReXCk17YxrxyIRyQNOA5caYzoTtU/1jkXDbTIB6G5BSqmskpYdi4wxAaB0PM9xMS5M5LctuG1I0tYErpSajjJqpiicr01PdgVFpZSaLjIuoQ/eLWigNLHYXTymtVmUUiobZVxCb/O34XEN3Q/U4/KMaW0WpZTKRhmX0Adq0wfT0kSllMrAhK5L2Sql1PAyLqHrUrZKKTW8jNyxSJeyVUqpz8u4M3SllFLD04SulFJZQhO6UkplCU3oSimVJTShK6VUlhjXaotjfjGRNuBitqkrA9pTHM5Up32eHrTP08d4+j3XGJNw9mRaE/rFEpHdySwdmU20z9OD9nn6SEe/dchFKaWyhCZ0pZTKEpmS0DdMdgCTQPs8PWifp48J73dGjKErpZRKLFPO0JVSSiUwpRK6iNwqIodFpEFEvjvM/Tki8lL//TtEZF76o0ytJPr8bRE5KCKfiMgWEZk7GXGmUqI+D2p3t4gYEcn4iohk+iwiX+p/rw+IyAvpjjHVkvjdniMiW0VkT//v922TEWcqich/iEiriOwf4X4RkX/u/z/5RERWpjQAY8yU+AfYgaPApYAL2AssvqDN14F/6//5PuClyY47DX2+Ecjr//mx6dDn/nYFwPvAdmDVZMedhvd5AbAHmNF/XDHZcaehzxuAx/p/XgycmOy4U9DvG4CVwP4R7r8NeAMQYA2wI5WvP5XO0FcDDcaYY8aYPuDXwJ0XtLkTeLb/51eAm0RE0hhjqiXsszFmqzEm0H+4Hcj0dYOTeZ8BngR+DITSGdwESabPDwM/M8b4AIwxrWmOMdWS6bMBCvt/LgIa0xjfhDDGvA90jNLkTuA5E7cdKBaRWal6/amU0KuB04OOvf23DdvGGBMBOoHStEQ3MZLp82APEv90z2QJ+ywiK4DZxpj/SWdgEyiZ9/ky4DIR+Z2IbBeRW9MW3cRIps9/A3xFRLzA/wJ/mZ7QJtVY/+bHZCptcDHcmfaFJTjJtMkkSfdHRL4CrALWTmhEE2/UPouIDfgJ8EC6AkqDZN5nB/Fhl3XEv4VtE5GlxphzExzbREmmz/cDvzLG/KOIXAM839/n2MSHN2kmNIdNpTN0LzB70HENn/8KZrUREQfxr2mjfb2Z6pLpMyJyM/AEcIcxpjdNsU2URH0uAJYC74rICeLjjJsy/MJosr/bG40xYWPMceAw8QSfqZLp84PAywDGmA8BN/H1TrJZUn/zF2sqJfRdwAIRuUREXMQvem66oM0m4Kv9P98NvGP6rzRkqIR97h9++AXxZJ7p46qQoM/GmE5jTJkxZp4xZh7x6wZ3GGN2T064KZHM7/brxC+AIyJlxIdgjqU1ytRKps+ngJsARGQR8YTeltYo028T8Kf91S5rgE5jTFPKnn2yrwoPcwX4M+JXx5/ov+3viP9BQ/wN/2+gAdgJXDrZMaehz28DLcDH/f82TXbME93nC9q+S4ZXuST5PgvwNHAQ2AfcN9kxp6HPi4HfEa+A+Ri4ZbJjTkGfXwSagDDxs/EHgUeBRwe9zz/r/z/Zl+rfbZ0pqpRSWWIqDbkopZQaB03oSimVJTShK6VUltCErpRSWUITulJKZQlN6EoplSU0oSulVJbQhK6UUlni/wE6CjVTC4pXJwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "predicted =model.forward(torch.from_numpy(x_train)).data.numpy()\n",
    "plt.plot(x_train, y_train, 'go', label = 'data', alpha = 0.3)\n",
    "plt.plot(x_train, predicted, label = 'predicted', alpha = 1)\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上就是一个使用PyTorch做线性回归的简单样例了，下面我们会对上面的内容做详细的介绍\n",
    "##  2.2.3 损失函数(Loss Function)\n",
    "损失函数（loss function）是用来估量模型的预测值(我们例子中的output)与真实值（例子中的y_train）的不一致程度，它是一个非负实值函数,损失函数越小，模型的鲁棒性就越好。\n",
    "我们训练模型的过程，就是通过不断的迭代计算，使用梯度下降的优化算法，使得损失函数越来越小。损失函数越小就表示算法达到意义上的最优。\n",
    "\n",
    "这里有一个重点：因为PyTorch是使用mini-batch来进行计算的，所以损失函数的计算出来的结果已经对mini-batch取了平均\n",
    "\n",
    "常见（PyTorch内置）的损失函数一下几个：\n",
    "### nn.L1Loss:\n",
    "输入x和目标y之间差的绝对值，要求 x 和 y 的维度要一样（可以是向量或者矩阵），得到的 loss 维度也是对应一样的\n",
    "\n",
    "$ loss(x,y)=1/n\\sum|x_i-y_i| $\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### nn.NLLLoss:\n",
    "用于多分类的负对数似然损失函数\n",
    "\n",
    "$ loss(x, class) = -x[class]$\n",
    "\n",
    "NLLLoss中如果传递了weights参数，会对损失进行加权，公式就变成了\n",
    "\n",
    "$ loss(x, class) = -weights[class] * x[class] $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### nn.MSELoss:\n",
    "均方损失函数 ，输入x和目标y之间均方差\n",
    "\n",
    "$ loss(x,y)=1/n\\sum(x_i-y_i)^2 $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### nn.CrossEntropyLoss:\n",
    "多分类用的交叉熵损失函数，LogSoftMax和NLLLoss集成到一个类中，会调用nn.NLLLoss函数,我们可以理解为CrossEntropyLoss()=log_softmax() + NLLLoss()\n",
    "\n",
    "\n",
    " $ \\begin{aligned} loss(x, class) &= -\\text{log}\\frac{exp(x[class])}{\\sum_j exp(x[j]))}\\ &= -x[class] + log(\\sum_j exp(x[j])) \\end{aligned}  $\n",
    " \n",
    " 因为使用了NLLLoss，所以也可以传入weight参数，这时loss的计算公式变为：\n",
    " \n",
    " $ loss(x, class) = weights[class] * (-x[class] + log(\\sum_j exp(x[j]))) $\n",
    " \n",
    " 所以一般多分类的情况会使用这个损失函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### nn.BCELoss:\n",
    "计算 x 与 y 之间的二进制交叉熵。\n",
    "\n",
    "$ loss(o,t)=-\\frac{1}{n}\\sum_i(t[i]* log(o[i])+(1-t[i])* log(1-o[i])) $ \n",
    "\n",
    "与NLLLoss类似，也可以添加权重参数： \n",
    "\n",
    "$ loss(o,t)=-\\frac{1}{n}\\sum_iweights[i]* (t[i]* log(o[i])+(1-t[i])* log(1-o[i])) $\n",
    "\n",
    "用的时候需要在该层前面加上 Sigmoid 函数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2.4 梯度下降\n",
    "在介绍损失函数的时候我们已经说了，梯度下降是一个使损失函数越来越小的优化算法，在无求解机器学习算法的模型参数，即约束优化问题时，梯度下降（Gradient Descent）是最常采用的方法之一所以梯度下降是我们目前所说的机器学习的核心，了解了它的含义，也就了解了机器学习算法的含义。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 梯度\n",
    "在微积分里面，对多元函数的参数求∂偏导数，把求得的各个参数的偏导数以向量的形式写出来，就是梯度。\n",
    "例如函数f(x,y), 分别对x,y求偏导数，求得的梯度向量就是(∂f/∂x, ∂f/∂y)T,简称grad f(x,y)或者▽f(x,y)。\n",
    "\n",
    "几何上讲，梯度就是函数变化增加最快的地方，沿着梯度向量的方向，更加容易找到函数的最大值。反过来说，沿着梯度向量相反的方向梯度减少最快，也就是更加容易找到函数的最小值。\n",
    "\n",
    "我们需要最小化损失函数，可以通过梯度下降法来一步步的迭代求解，得到最小化的损失函数，和模型参数值。\n",
    "### 梯度下降法直观解释\n",
    "梯度下降法就好比下山，我们并不知道下山的路，于是决定走一步算一步，每走到一个位置的时候，求解当前位置的梯度，沿着梯度的负方向，也就是当前最陡峭的位置向下走一步，然后继续求解当前位置梯度，向这一步所在位置沿着最陡峭最易下山的位置走一步。这样一步步的走下去，一直走到觉得我们已经到了山脚。\n",
    "\n",
    "如下图所示，（此图摘自百度百科）\n",
    "![](1.png)\n",
    "\n",
    "这样走下去，有可能我们不能走到山脚，而是到了某一个局部的山峰低处（局部最优解）。\n",
    "\n",
    "这个问题在以前的机器学习中可能会遇到，因为机器学习中的特征比较少，所以导致很可能陷入到一个局部最优解中出不来，但是到了深度学习，动辄百万甚至上亿的特征，出现这种情况的纪律几乎为0，所以我们可以不用考虑这个问题。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Mini-batch的梯度下降法\n",
    "对整个训练集进行梯度下降法的时候，我们必须处理整个训练数据集，然后才能进行一步梯度下降，即每一步梯度下降法需要对整个训练集进行一次处理，如果训练数据集很大的时候处理速度会很慢，而且也不可能一次的载入到内存或者显存中，所以我们会把大数据集分成小数据集，一部分一部分的训练，这个训练子集即称为Mini-batch。\n",
    "在PyTorch中就是使用这种方法进行的训练，可以看看上一章中关于dataloader的介绍里面的batch_size就是我们一个Mini-batch的大小。\n",
    "\n",
    "为了介绍的更简洁，使用 吴恩达老师的 [deeplearning.ai](https://www.deeplearning.ai/deep-learning-specialization/) 课程板书。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于普通的梯度下降法，一个epoch只能进行一次梯度下降；而对于Mini-batch梯度下降法，一个epoch可以进行Mini-batch的个数次梯度下降。\n",
    "![](2.png)\n",
    "普通的batch梯度下降法和Mini-batch梯度下降法代价函数的变化趋势，如下图所示：\n",
    "![](3.png)\n",
    "- 如果训练样本的大小比较小时,能够一次性的读取到内存中，那我们就不需要使用Mini-batch，\n",
    "- 如果训练样本的大小比较大时，一次读入不到内存或者现存中，那我们必须要使用 Mini-batch来分批的计算\n",
    "- Mini-batch size的计算规则如下，在内存允许的最大情况下使用2的N次方个size\n",
    "![](4.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`torch.optim`是一个实现了各种优化算法的库。大部分常用优化算法都有实现，我们直接调用即可。\n",
    "### torch.optim.SGD\n",
    "随机梯度下降算法,带有动量（momentum）的算法作为一个可选参数可以进行设置，样例如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "#lr参数为学习了率对于SGD来说一般选择0.1 0.01.0.001，如何设置会在后面实战的章节中详细说明\n",
    "##如果设置了momentum，就是带有动量的SGD，可以不设置\n",
    "optimizer = torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### torch.optim.RMSprop\n",
    "除了以上的带有动量Momentum梯度下降法外，RMSprop（root mean square prop）也是一种可以加快梯度下降的算法，利用RMSprop算法，可以减小某些维度梯度更新波动较大的情况，使其梯度下降的速度变得更快"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "#我们的课程基本不会使用到RMSprop所以这里只给一个实例\n",
    "optimizer = torch.optim.RMSprop(model.parameters(), lr=0.01, alpha=0.99)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### torch.optim.Adam\n",
    "Adam 优化算法的基本思想就是将 Momentum 和 RMSprop 结合起来形成的一种适用于不同深度学习结构的优化算法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 这里的lr，betas，还有eps都是用默认值即可，所以Adam是一个使用起来最简单的优化方法\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=0.001, betas=(0.9, 0.999), eps=1e-08)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2.5 方差/偏差\n",
    "- 偏差度量了学习算法的期望预测与真实结果的偏离程序, 即 刻画了学习算法本身的拟合能力\n",
    "- 方差度量了同样大小的训练集的变动所导致的学习性能的变化, 即 模型的泛化能力\n",
    "![](5.png)\n",
    "\n",
    "从图中我们可以看出\n",
    "- 高偏差（high bias）的情况，一般称为欠拟合（underfitting）,即我们的模型并没有很好的去适配现有的数据，拟合度不够。\n",
    "- 高方差（high variance）的情况一般称作过拟合（overfitting），即模型对于训练数据拟合度太高了，失去了泛化的能力。\n",
    "\n",
    "如何解决这两种情况呢？\n",
    "\n",
    "欠拟合：\n",
    "- 增加网络结构，如增加隐藏层数目；\n",
    "- 训练更长时间；\n",
    "- 寻找合适的网络架构，使用更大的NN结构；\n",
    "\n",
    "过拟合 ：\n",
    "- 使用更多的数据；\n",
    "- 正则化（ regularization）；\n",
    "- 寻找合适的网络结构；\n",
    "\n",
    "例如我们上面的例子，可以计算出我们的偏差:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.07545900344848633 -0.020828723907470703\n"
     ]
    }
   ],
   "source": [
    "print (5-w.data.item(),7-b.data.item())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2.6 正则化\n",
    "利用正则化来解决High variance 的问题，正则化是在 Cost function 中加入一项正则化项，惩罚模型的复杂度,这里我们简单的介绍一下正则化的概念"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### L1正则化\n",
    "损失函数基础上加上权重参数的绝对值\n",
    "\n",
    "$ L=E_{in}+\\lambda{\\sum_j} \\left|w_j\\right|$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### L2正则化\n",
    "损失函数基础上加上权重参数的平方和\n",
    "\n",
    "$ L=E_{in}+\\lambda{\\sum_j} w^2_j$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "需要说明的是：l1 相比于 l2 会更容易获得稀疏解\n",
    "\n",
    "[知乎](https://www.zhihu.com/question/37096933/answer/70507353)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "pytorch 1.0",
   "language": "python",
   "name": "pytorch1"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}