6.3-advanced-usage-of-recurrent-neural-networks.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'2.2.4'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import keras\n",
    "keras.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 순환 신경망의 고급 사용법\n",
    "\n",
    "이 노트북은 [케라스 창시자에게 배우는 딥러닝](https://tensorflow.blog/케라스-창시자에게-배우는-딥러닝/) 책의 6장 3절의 코드 예제입니다. 책에는 더 많은 내용과 그림이 있습니다. 이 노트북에는 소스 코드에 관련된 설명만 포함합니다. 이 노트북의 설명은 케라스 버전 2.2.2에 맞추어져 있습니다. 케라스 최신 버전이 릴리스되면 노트북을 다시 테스트하기 때문에 설명과 코드의 결과가 조금 다를 수 있습니다.\n",
    "\n",
    "---\n",
    "\n",
    "이 절에서는 순환 신경망의 성능과 일반화 능력을 향상시키기 위한 세 가지의 고급 기술을 살펴보겠습니다. 이 절을 마칠 때면 케라스에서 순환 신경망을 사용하는 대부분의 방법을 알게 될 것입니다. 온도 예측 문제로 세 가지 개념을 모두 시연해 보겠습니다. 이 시계열 데이터는 건물 옥상에 설치된 센서에서 취득한 온도, 기압, 습도 같은 데이터입니다. 이 데이터를 사용하여 마지막 데이터 포인트에서부터 24시간 이후의 온도를 예측하겠습니다. 이 문제는 시계열 데이터에서 일반적으로 나타나는 여러 가지 어려운 점을 가지고 있습니다. 전형적이고 꽤 도전적인 문제입니다.\n",
    "\n",
    "다음 기법들을 적용하겠습니다.\n",
    "\n",
    "* 순환 드롭아웃 : 순환 층에서 과대적합을 방지하기 위해 케라스에 내장되어 있는 드롭아웃을 사용합니다.\n",
    "* 순환 층 스태킹 : 네트워크의 표현 능력을 증가시킵니다(대신 계산 비용이 많이 듭니다).\n",
    "* 양방향 순환 층 : 순환 네트워크에 같은 정보를 다른 방향으로 주입하여 정확도를 높이고 기억을 좀 더 오래 유지시킵니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 기온 예측 문제\n",
    "지금까지 다룬 시퀀스 데이터는 IMDB 데이터셋이나 로이터 데이터셋과 같은 텍스트 데이터입니다. 시퀀스 데이터는 이런 언어 처리 분야뿐만 아니라 훨씬 많은 문제에서 등장합니다. 이 절에 있는 모든 예제는 날씨 시계열 데이터셋을 사용합니다. 이 데이터는 독일 예나 시에 있는 막스 플랑크 생물지구화학 연구소( http://www.bgc-jena.mpg.de/wetter/ )의 지상 관측소에서 수집한 것입니다.\n",
    "\n",
    "이 데이터셋에는 수년간에 걸쳐 (기온, 기압, 습도, 풍향 등과 같은) 14개의 관측치가 10분마다 기록되어 있습니다. 원본 데이터는 2003년부터 기록되어 있지만 이 예제에서는 2009~2016년 사이의 데이터만 사용합니다. 이 데이터셋은 시계열 수치 데이터를 다루는 법을 익히는 데 안성맞춤입니다. 최근 데이터(몇 일치 데이터 포인트)를 입력으로 사용해 모델을 만들고 24시간 이후의 기온을 예측하겠습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "데이터를 살펴보죠:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['\"Date Time\"', '\"p (mbar)\"', '\"T (degC)\"', '\"Tpot (K)\"', '\"Tdew (degC)\"', '\"rh (%)\"', '\"VPmax (mbar)\"', '\"VPact (mbar)\"', '\"VPdef (mbar)\"', '\"sh (g/kg)\"', '\"H2OC (mmol/mol)\"', '\"rho (g/m**3)\"', '\"wv (m/s)\"', '\"max. wv (m/s)\"', '\"wd (deg)\"']\n",
      "420551\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "data_dir = './datasets/jena_climate/'\n",
    "fname = os.path.join(data_dir, 'jena_climate_2009_2016.csv')\n",
    "\n",
    "f = open(fname)\n",
    "data = f.read()\n",
    "f.close()\n",
    "\n",
    "lines = data.split('\\n')\n",
    "header = lines[0].split(',')\n",
    "lines = lines[1:]\n",
    "\n",
    "print(header)\n",
    "print(len(lines))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "420,551개 데이터 전체를 넘파이 배열로 바꿉니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "float_data = np.zeros((len(lines), len(header) - 1))\n",
    "for i, line in enumerate(lines):\n",
    "    values = [float(x) for x in line.split(',')[1:]]\n",
    "    float_data[i, :] = values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "예를 들어 다음은 시간에 따른 기온(섭씨) 그래프입니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from matplotlib import pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD8CAYAAAB0IB+mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJztnXd4FVX6x79vGjUhlFADBhAIRWqkiaIogoJl/eEuuuu6irru2nbVVSzr2kW36NrW1cW2igtWkCZdRZHeSyD0QCChBQIkIeT8/rhzk8nN3DZzZubcue/nefLk3nNn5rz33Jl3zrznLSSEAMMwDON9EtwWgGEYhnEGVvgMwzBxAit8hmGYOIEVPsMwTJzACp9hGCZOYIXPMAwTJ7DCZxiGiROkKXwiSiSi1UQ0XXvfnoiWEtE2IppMRCmy+mIYhmGiR+YM/z4Am3XvXwTwshCiE4CjAMZJ7IthGIaJEpIRaUtEmQA+APAcgPsBXAWgCEBLIUQFEQ0C8KQQYkSo4zRr1kxkZWVZlodhGCaeWLly5SEhREa47ZIk9fcKgIcApGrvmwI4JoSo0N7nA2hjtCMR3QHgDgBo164dVqxYIUkkhmGY+ICIdkeynWWTDhGNBlAohFipbzbY1PBRQgjxthAiRwiRk5ER9gbFMAzDmETGDP8CAFcT0ZUA6gJIg2/Gn05ESdosPxPAfgl9MQzDMCaxPMMXQjwihMgUQmQBGAtggRDilwAWAhijbXYzgKlW+2IYhmHMY6cf/sMA7ieiPPhs+hNt7IthGIYJg6xFWwCAEGIRgEXa6x0A+ss8PsMwDGMejrRlGIaJE1jhMwzDxAms8BmGUZo9h0/h+21FbovhCaTa8BmGYWRz0V8XAgB2TRjlsiSxD8/wGYaJCUrPnHVbhJiHFT7DMDHFu4t3Yt+x026LEZOwwmcYJmY4eLwUT0/fhFveW+a2KDEJK3wmrvj9xyvx4ZJdbovBmORspS8l14nSijBbMkawwmfiipnrD+CJqRvdFgMAcKL0DN5clIfKSuspyuMFHilrsMJnGJd4fuYWvDQ7F3M2HXBbFCZOYIUfQ5SUVeDuSatwuKTMbVEYCZwq95klSs9UuiwJEy+wwo8hJi/fi+nrCvDagjy3RWEkkEC+shGVEqrOxRtGBTeY8LDCjyH8tt6kBD7dY5HNBcfR8dGZVS6Fmr7H/VPWuihV7JD959lYtvOw22IAAL5eux/D/r7IcP3l9g9XYNLSPS5IFR5W+DFEhXZy7Tx0EjJqETPOMmnpHpytFJi/+SAAgHieGjVzNx10WwQAwAOfrsWOopMoP1vbHDd300E8+uV6F6QKDyv8GKJCO7nmbynEO9/vcFkaxiqkgL7fffgk+jw9B3uPnKpqyz96CsWnz7goVXD085yPftqNJ6ZucEWO8orYXHdhhR9DzFhfUPV6bX6xi5IwMlDBMjd5+V4cPXUGU9fsq2ob8uJCjHzlOxelCo7/JnnkVDke/2oDPlwSUe1uRoMVfoR8u7WoKujDLbYcOOFq/9GwYV8xxvzrR85/EoIEBab4/jOaAmQpKC51XpgIOHrS9+TBnk3mYIUfAYtyC3Hzu8vw5kJ1vGPcVxWheWLqBqzYfRQb97v/JDJx8U5c9dpit8Wowm+WCFSybrBhn+/32VF00mVJImPJDjUWbf3E2lIaK/wIKDzh83vfrbNzus3pcrVnzqv2HAMAJWzBz0zfhPX7ilFSplY4vgL6Ht1apQEAOmQ0cFkSxglY4cco87cUui1CRKhkhlq5+6jbIgCoVvQK6HslnjIY52CFHwUqPb5d1au12yJEhFuuh0IIbC8qCZBFDfznkQo2fAVEMORQSRm2HlRnshAMEZDd54yBm6ZKsMJnPMlnK/Nx6d+/rVEaz+37daByVclEqBqX/G0RLn9ZTU8hPYGTQFb4HsLN2ZDqJ1Iw3Boz/2Lk9sKSMFu6x3dbq29GbuVHOqStT/204zBmri/A3ZNWuSJHIF5If5w1fgYWKmZ6tazwiaguES0jorVEtJGIntLa2xPRUiLaRkSTiSjFurjxy9FT5W6LYAo39P2p8gqsUMRer6fwuE+5btp/vNZn8za7E0G6ao9vnL7fdgi//3gVpq8rCLMHoyfcU+Mt7y93RI5IkTHDLwMwTAjRC0BvACOJaCCAFwG8LIToBOAogHES+nIVN234lbE5wbd9hv/Jsj34ZFnNvCUPTFmLjZpS1f9kbqejWJjrm+1NXrG31mduLZ4eKonNiYQbPDltI2ZvqJnK+vmZmzHmXz9WpctQHcsKX/jwPzcna38CwDAAn2ntHwC41mpfbvHTdvd9f+dvqXlCKbrW5jiPfLEej3xRM2/JRoMZNKC2r3miSwpfBbfZWOH9H3fhzo9W1mibtHQPVuw+inEfrHBJquiQYsMnokQiWgOgEMBcANsBHBNC+A1x+QDayOjLDb5Y7Qs7/3xVPl6avcUVGSrO1pydur0AGSlueOnolZi+d/06yOGSMhQUO1sIuyJEpHYCr6Z5glOKx8dIOc2EEGeFEL0BZALoD6Cr0WZG+xLRHUS0gohWFBUVGW2iFG8u2u5ofyt2HcHsDQW1TCNuLgZVnK2MOPDLqYmrPpmVXuHrTzr9xdjv2XkY9MICJ0SrIlRqDhVcNAO5VTH7cyywdu8xt0UIidR5hRDiGIBFAAYCSCeiJO2jTAD7g+zzthAiRwiRk5GRIVMcTzDmrSW486PanhP+qNHp6/Zj2c4jjsp016RV6PrEbEf7DEfnx2eh6ERtT5eDx2OjOliilknt1fnbsHK3s79nMBYo5mESC6h449Yjw0sng4jStdf1AFwGYDOAhQDGaJvdDGCq1b7imWCFt++etBo///cSR2X5ZmPoBao8nSukk4uR/sIiet761tknMrP4bfj/mLsV//cvZ39PlYm55Htq63skhd8kLK0AfEBEifDdQKYIIaYT0SYA/yOiZwGsBjBRQl9MDPCibp1DpfP/n/O3uS1CUDjFgTH9npnrtgjRofjimgwvnXVCiD5CiJ5CiB5CiKe19h1CiP5CiHOFENcLIWLj2ZqJisnL9yBr/IyqgtxAzapET0/fhPUO5e5/ZvomCCHQrGEdR/qTyXH2ljHkpOKLoIEEplpQDfYNYEwzefmeqkXs+ZuD23uvet2Z1MQrdx9F4Yky1EuJvdO6bkqi2yIYsrnA2MVVJVTKHKt62crYuzIcZGFuIbLGz3BbDGV5+PNq//d7PlkNAFiwxd0AlJdm5yqV5C5S7v1kNZYqlusdAK745/eO9nf0ZHnUAXInStV4OsorLFHLhmkAK/wA9h07jazxM7B6z1Hc8p7abmm5CqQe1nslCCFw6/vuBqB8virf1f4j5biBkjKKwLWLA8Wlyk1mCopPo88zc6N2fVbl/n7bB8vx4Y+73BYjJHGv8CsrBV5fsK3qAlysZVcMDNd3g3AznTFv/eiQJMHZeag6etVf9MRtYmH9c+mO2q6Xc8J4P8nk73NyHesrUvxlFfVrQJGgyhPdrsOnsDBX7ViiuFf487cU4m9ztuKpaZtqtKvgvx2uhG5g9K37qCaPuhgVMHeyIlevtumO9WU3lapo/Bgg7hW+P0Lz9BnfxeY/d77d6v6dOtyJfFo5H+UYmForgtsBOql1ZXhkqwGr+8iJe4UfyH9/2u22CFWECsVXkVgwpajCCZfr65ZVxGj6VQMqY+w6cZO4V/j7jtWsOqRSDdZYe1I9HCLVbuHxUhzQbLR2o56pqzb3al5NbjHx+52u9m+Ef76g+q/XTbG0ItEQ1wq/8EQpnp9ZM/tlJLPqdfnOLE7Gmm3y9g+De+j0f34+Br4w3xE5CkzcWKat3Y/nZ26WLstl//gW73y3Q/pxrZKrYL3YqmjjKM97py8T1TNihiKuFf6xU+b8d6c45D53NsYUfixz7yer8bYkxXyopKyqZGFeYQmes+FG4jTj3l+OiYvVeyoA1I9uVYm4Vvh6orlrFxxzxjQhvGNmjStynp2Hfs/Oc1sMqczfUohnpm8Kv6EL8LwocuJa4evXGFfsirwG6nyH0saqZtK5f8oaDHjeW4qMiY7ZGwrwp0/X2nJsfz6maM96ta4StYlrha8iWeNn4MlpvlTIqin8L1btsxyfsHqPesXFmci586NV+HSlPdHMN76zFED0M3bVrhOVYYWvoZJL4fuKh2dbYeVutRT+D3mHpB7v4HFnzH1e5mSULqus7yOHFb7GidIKfBdFsFW5h/yYnUQ1/+9f/mep1OPpzwsZxVeKTpThpolLcfRkcJdXr7HjUHTF5m+auFTpYuz9n5vnaBR1KOJa4QfO6n/97rKI9/39x7XLDrrJoRL5qSAmLd2DXk/NiTp7IeNjwqxql1+zQXT/WbwD3287hE+Wu5/byS46Pz4LT31tXNEtEgqKS/HfJbukyRPI+vxiS4VYCk+UYcM+Z2pChCPuFP6xU+Uoq/B75Ji348zb7G4a4EA+X5kvvZzfo1+uV3rmpCLB7o1mffz9KRhUu+fKTINdXlGJ937YZekYf5uzVY4wBlz1+mIctviEpYpFIO4Ufu+n5+LW99VOewz40udGcxK/MGtLjRmlqqj4tLCjqCT8RhESzCd8xroCU8dLqIpFsjZuR0zkmQ+F22mwY406SWqoWjWkcJgf8tQrNBFIzyfnmErRbEeOcwV1tFRkFoEPNlYHTC7m+isoWUkXs6OoBH2fmYsPTDoD/Ch5YZtxj7hS+Mt2Vucg19dgNUvW+BlYsat2XnOvcaKsAit3y/meKhbrPlkmL1Re9r0xwVy2gRrsOuxbBDWbAfbGIAvb329zP6NsrKDKnCmuFL5+JvfQZ+ukrJx/ssxamoXyikrlXfl6PTUH//cvObNgK2aFk2UVePizddLXFWSG5gf7fsmJJm905J/hq6Iyqrlp4jKUKpeimwlFXCl8PdPXFeD+KWssH2fjfmur7w99thYDnp+PsoqzWOhQBG+s8uGS3Zi8Yi/+FWUJvHCUnrF/Qe2MyQyeMjJI+tOG2OEaqOKNiAlO3Cp8ANhRFJ2/rxFdW6VZ2n+OVs6t4qzA6wvzLMsDoCpxl4pYMenEgnKRb9KxbtPxe8As33UUX6/dj+UeNUN2f2K2EoWLVCauFb4MZFYualQvWcpxzKQHdgoZniLVbrXysVpM49AJuTdbvw3filj6GIB7PlmN69+St0h99es/YO1eNWoZnyw/i3/MleOeeeZspdTCKqoUabGs8ImoLREtJKLNRLSRiO7T2psQ0Vwi2qb9b2xdXCYSVJ4IW5HNH7xi1Wc7FFYL4MiuUuafT2y1kL9+0/7jkqSpTV5hCZ6doWYWTSt0emwWxn0gz337jFcUPoAKAA8IIboCGAjgLiLqBmA8gPlCiE4A5mvvmQD89lWB+KgIa+WB6Hip+kFgsi9rvwnMb/qLlsnL96D8rBpBP0ZIr4ImcbazMFeeeUiVa9uywhdCFAghVmmvTwDYDKANgGsAfKBt9gGAa632xcQ+Kj99ANY9dmR/P6sWw4c/Xy9HEJuQHgSpoNsv4FG3TCLKAtAHwFIALYQQBYDvpgCgucy+mOCoXAHIimRO3Cys9iF77GWuEdkFWZi/Hj1lT1I4IYQy6QwAdSLMpSl8ImoI4HMAfxBCRGw0JKI7iGgFEa0oKorfFXaZJ4Qi55Z0vPq9QqG+ulcL/wLyS9/kovPjs5SJE5gwawuG/X2R22LIUfhElAyfsv9YCPGF1nyQiFppn7cCYOhkLoR4WwiRI4TIycjIkCGOo8ia0Z2WeGJe88YPOKOw3dYsKj+5+JF9U7Iyw//MpkIlsYA/LclpEwXH7ZiNbzlwQoobuFVkeOkQgIkANgsh/qH7aBqAm7XXNwOYarUvFVmfLyft6QNT5JaNU2VmIxP9dfjnrza4J0gIZKsKKy62D9pUilAmdj/BmPk9ihSOY7GKjBn+BQBuAjCMiNZof1cCmABgOBFtAzBce+85thXKybT4/bZD2C/RY0HVubCVzJT67/Tfn3ajoPi0dYEkI3t2+O4PO6UezxYsaG2Z57yes1pks6nfQ9WLRwJJVg8ghFiM4D/5pVaPH09sLpDnL62qvfurNfvxytg+pvbVJ78DgEEvLJAhUg2sL9rK4WylQGJCjFjwFTzXTlhII9GwrmW1qCwcaetR9AFA5RWVyngJqI7ldQJJw9zx0ZmYtd5cDn2nWaZYqgb9uZ5rMZDOzqhuN2CF71H6aiXZjpwsR+fHZ+Gd73e4LFE1v3lvGZ6cZr6kXbwwc8MBt0WwjYqzldLXrfzoa+L+bU6upWNNWuqt0pKs8C2i+lO3P5Lxi1X7XJakmkW5RXjfZDEO1ZHpSfT12v3SjqUa6/cV4/NV9ngR6Z9uV+2xlufHa95urPAtonpgzJerfRcVW3ScodJb+iEu8fK1wgrfIior/JW7j+Cd731eHrkWkm/FE16+2JnoeX6m+nWio4EVvkWs6Pv9x+x1K9SX7mvfrIGtfZlhYa56BV+s6nu+X4Tnx7xDVWUX7SAW6ia4hXf9jxzCygx/vs0VrvSi1UlS797+xoI8XNLFWymWZKdH9iLBauTKYsY6a95Ne46ckiSJeqinBWIM/aLt9HX78ez0yHODq1IUwS06tWjotgi1MOO+WqQrevLnqfZFAOc8O8+2Y9vBotxCrMt3vjiK1SjzXYfse/r4y9QNptI9yIIVvkUSdBr/7kmr8Z/FkUdGyq7NGshNE5eZ2m9hbmHVYi8TmsXbDuH85+ZhzkafC2WR5IpXeg7FWMj/b95bjqtf/8FtMaK+ids5DftgyW58uGSXjT2EhhW+Raws2R447lwpwiMnI09De8t7y/HHyfbnYfGCqXWtNoNdrUiZP6Y2S7Yfjmp7u8/Lsy6e+KzwLaJyNSE9hTbOPM0ydY16fuZmL0Uv3Ly8gt8zzU+016iXF31Z4Vuk9ExsKHwVkZkSWhZG1/r324qCRlz6F8ZjIXVzvOIvEymEiMi+b7fCd/N+wgrfJaIxsTDuctPEZXj0S+NSgVaqPdmFEMLx1BV5herGefiX2SYv34vsP8/GbhtdQlWHFb4LbNhXXJXrhomMJ2z0fjFizd5j2LAviloHCk3w520udDx1xWX/+A75R9V0Z/Tb8Md/4btp7wjjheNhiw4rfKu0alQ36n22ctRr1Hy4ZLdDPfmu9mvf+AGjX1scdutqk446/G+ZOwm/hry40JV+w/FmgDdcYpjYGbvNc2VnzrpWy4EVvkUOHC9FZaWo4fr1fiwUrWBM8/zMzZimSGKz46VncNekVTimFQM/UXoG+UfVKwzjNv+Yu7Xqdbg6A3bnQ3p1QR4GvbAAJ0rP2NuRAazwLdK6UT10eHQmnpm+uapt8gr2Yfcyb3+3A/d+stptMXCguBTPTd+MGesK8PLcrag4W4nznpzjet6kEgvFR+zi1fnbql6HU/hO2finuKAnWOFbZJ+WD0dfii7cMp6XbYTRcqq8AndPWoVCB2MSQhHtb+P/rZ0uMCOEwMAX5mPyir0AfAE9KrgIb9xfjB5/+UaZJyAj/Ar/wyW7kP3nWSgpq8A3WuDc+vxivLogzxE53ChKxArfBhROoFmD/cdOu77QNm3NfkxfV2C5UIUsor0E3cqO8fDn62q1qZCpY+N+X5nO77YWuSxJcPz5r56YuhGlZyrx0Gdr8dv/rkReYQmuej38uo0s9h07jazxM7A+PwrnAIt4XuEXnz7j+J00VhT+4AkLai207Tns7A3A/8uo6N4YCa8v8JkKVu4+amr//90x0NR+RuYAlRK3qfwUG2jS8SdLO1XurClqgZY8cZKDi+yeVvj7j51Gr6fmOF7eLxaV1+7DJ3HweKnjmQL9isF/k3xl3tbo3CFtkidSTmqJsA4eNxfJPLBDU1P7GSFzYpNqspB3LJz5R0+W14gbcOt6deOm6GmF7/dWmLvpoKP9xsoMX8/Qvy7CgOfnm77QzeKPtq1W+NtquUP+mHfIUZnMoEKReJkT/L+O6SnvYIrx9PRNKCiuXjM66fDM3o8baZg9rfDdYp2DNjmz6BXUU1+7V1D8map00sHvkg8Z2KvtYvG2IlvT49qJTJPOsOwWlvZXOdXEzkMna4zVjiLrv/cdF3Uwve8ny/ag2xOzLcsQCazw45T3fthl+NrNp5Ngs2QnJ8+vLsjDxX9b5FyHURLqSULmU0ZKUgLaNakf9X6HSnzxAN/mqrtoSwRUnK09VlaGLyXRmio95VCOfCkKn4jeJaJCItqga2tCRHOJaJv2v7GMvsxw5GR5Vb5yFVBh7vP5KrViBYjUXugDQqfZdUr0LQeC+9jLXrOtn5IY9T5+vXdY4VxRSQmECoPoqvmbnTX9uoGsGf77AEYGtI0HMF8I0QnAfO29K2wvOok7/rvSre5r8cZCZ/x89YzoXvMR3e8+F0huCIViJwkUXGmqkq72hnd+CvqZUyKeCeFrL2uc/jSiCwDgPzfnRL1v89TQqUbcrPbkh4jw7dba60LFp52PfHUaKSt0QojviCgroPkaABdrrz8AsAjAwzL6c4rUOkk4YUPU4E6FbcQ/7TjiSr8ECmqSCKXkVMEpm/Xx08HPR1kK/65LzgUAZDaO3qQTzCRYUlaBce8vx9Kd7pxfegjA6j213WjJgj1T5TULPXba8FsIIQoAQPsfM9Wqr+vTBrsmjEI9E4+0sY6MEzcjtU7U+5SUVQTt2W8XVhmnZvg/7QhhVlJY58zbdFAJZe+nvEL9SYQduL5oS0R3ENEKIlpRVKTGQs/Pz28LoDoizwx2enp0zGgQ9T4R+xpLUBpf/G5w1PtsLypRWmGFw4zoP44fFvU+r4cwB9odeJXdMjXsNnbW9NXTt1266X3LKirRzGBSkpwYg/7UUWKnwj9IRK0AQPtfaLSREOJtIUSOECInIyNDSscVZytRUlZh2uMkWVt5Oi+zkWkZ7PT08D9y24EMs4CZcSciWx6L7720k/RjyqJ1er0a75++prul47lZK9XPszM2h99IAh+OG4CrerU2vX+jesm12gJLI3oROxX+NAA3a69vBjDVxr5q8MCna9HjL9+Y3r9Fmu/uf1FnOTcg2QzLjt46FqkSjnaSOLpnK4O+otf4a/cew5sLt4ffMEruH95Z+jFlMLhj7Qjb3m3T8duh5v25K22e4fc9x7yj3QHJyfEa1klC+2bRP+n6seLC+ouctqb3dRtZbpmfAFgCoAsR5RPROAATAAwnom0AhmvvHcFfHNvsb9oyzedpMPb8tvjjZZ2x+s/DZYlmmd5t05FePwW7JozC13cPkX78o6eis5en1vXNlB4a2aWqzeyD8T91KWxjjWjPtWBPUpH4vgczm/xq4tLohIiSJvVTTO23ZPthvPOdvPQmfTRzjhUDTL0U8/4qD1+RjXn3D4U+JU9jk2PjNLK8dG4I8tGlMo7vFsmJCbjvMnVMAs1T6+Cruy6oel8nOfL7daQFw7/fZi6NgX69w8raR6xSFmVBdr29/bw2jbBeyx90Y/92eOzL0OUc3w1SYMdsPp9IMWsyWhJioTladk0YVfW6V1vzJtevLaRvbtIgBU0aVCv4CdedhzH9Mh0zZ1nB9UVbOzly0twFYMU9S8+GfcVSfXsDxercIhX3RWijXpRbhLIKO3ygayuBFml1cMsFWZaO+qmW5z1WiNZ9t2kDY0+mSM69fy2Sb/qKBLPrOwTrvgC/v7gj3vl1zbiAtibcRq0yoH2Tqtf+32pMv0wkWYy0dYrYkNIkZivKGFXEGdm9ZdTHGf3aYvzyP8GDdaIlvV7tx8abBp0T8f52uKJVZbvUtRERHh6Zbem4f/rMlz/H7epJh0pqThqspi+4sFMzAEDThrV/S7fXXJ//2XnY+cKVQT/PaBi9u60sHhqZjeHdagYPZlmw4Zslq6nzfcrE0wq/pDR6ZWG0eg8AjRsYt4djwz7jiFYzGE3+olESdphaLtEWkHu3Tcd/x/XH36/vJfX4x6JcU/CTY7DA+NoNffDqDX3w0bgBER9nxMvf1Xhv1fVxZA/fxEF/GFUsYGP6ZYZ8wri2TxsHpalm1n0XGrYnuzCrfvTKrrXajM6I3wzOsl0WMzibC9dhjpsoEhzcF9f9q9LoySMaV0bZisVvT1335OVIq1vzhiirL6MkV5Hw33EDcOy072bx1V0XIK1uEjpkNAQAFJ+K/LwIzAmjT6trBv9NV/+k8MoveuOtb7eje+s0S8e2wsIHL0ZKkj0K1Oq54PaTj56GuvThob7Wk1d3x/s/7rJdnmjx1Ay/pKwC7y6uXtAKlWgqWoLN/J3Abwa4/UIDlz2XZ/gAail7AEiU1FeFyRl1vZREtGrk83Pv3Ta9StkD1qKJ1+Yfq3ptpjxk5xa+4KW+7aqfQDpkNMRLY3q5age24uIYjoPHS1ERA+kxIkF/Vl/a1fd0G+y6qhuFU4VTeGqG//TXGyVUgjf+8do28SmP6/q2wRer9lnsIzrqJPlSPBhlL0xz8UYUChnKa/G2Q2iWqpa7m96DZtvBkqj373dOY/wwfhhaNwqdZEw1Xv6FeVPd7sOncNyEedWP/9pTjX+O7YNDJWW1nryn/HYQAKBnZjqWKZROAvDYDP9YFI/qwQg2MfWnJqhj02NvaILPSOsmR57vRxVbcaRsOXAcJywoCjvQe11F67Xid6ltk15PmieY3fh12dW92pg2rVhxFtg1YVRVrIdq1E1ONEww11/z5Bmo8+iJhILi01LkCoWnFL4MrutrvDDlvz6t2hOj9fKYfs8QXd1Xa0oi1mrtHjlZjkW5hhk5lCDac6F3W/P5X9zirV/1Q/+sJlr6anMnv50pHxY+eDEWPnixbccPJMFgHU0W09cW2HZsP54y6ciYNA3tZJxOwe8RkGQxwdLExcZBM8Ho0aZR1WUWW+raOm8u2o5re5vPlxIMv4nMKgqtJVoiVKqOy7u3xOUmXJL1mF14jwQ71x6c5qDk9BNGeGqGb3aRVp+iYPC5zQy3uaZ3a/x2aAc8ZMG/fMa6AkvReFZvaDFiRajBGRvyw9RLScTFXaznSQo06RyRVOVp7ROXSzlOpJzXxnzEaiSYXXiPeaK84AJjPuzAUwp/92FzVeD9PvZt0oMvDiUnJuCRK7oaeqSE41S5zw5916RVpuTrp/mUt2mOuXYqAAAZYklEQVSszuLVoA61k3/ZgV3eHS3CVGaKhEBLRd9n5lo+JgA0qu+czXrGvUMizyhqUm+fNSgn6EW++cNFeEaX8bRzi4Yhtq7NV2vMp3uIFM8o/NIoc5n4eePGvpIlqY3VxeTfDe2IhQ9ejOyW7vlpm+H5n52HT+8cZOkY32xUuc5o7M9cu7duZBjfYYT+26bWjdwabDY6tVur2Drfu7RMxU2Dsqrej+7ZOmjQmFt4RuGbfWxs16R+1Itvfx3T01Rf0eIPxElIIGm2yk1BatnawY0D2uH8rCboquCFe//l1tMmx7qlwmytgOapdaIyba7YXbucYCSc2zy6GbKKqHbue0bhm8WM58H1OW3x+KjaIdaqM23Nflz56veYvcG6N8DPgngzGaHi0kGLNOsmHbNPlapgpVbAVQZ1EIJhdm0jxu+nNWhuouynHcS9wgd8xRQAYGAUdunbjKJegyBgTjmYWWR99toeQT/bcsA3u99eZK384q4Jo/DzKIpAGBX78AL3T1kb8bbLHovpTOEAgHRtbeFPI7og3ab87+c0rfZrl1WUXRY39DdX+CT32ZFY/HD05SztwFNumWYQAmjcIAXzHxiKTJsWRSfM2mIq/3akfvObnh6BlbuP4kLNpfTxr4zzqftNEN9sPBC1LFYYf0U2fj0oCxf9daGj/drBpdnNMX9LdLEBv8hpi+YSFondpk5SYo189HagT8nRuXn4GrpO8sJ15ky5fjfg5ETCGRtdVCMh7hW+n44Z9tkLrRRbiIT6KUlVyj4UuZrb6rr8YlvlCSQpMQHtmtqXu7xNej3sO+aLUnzsyq4otzFvi5kMjXbU6vUsujnO3cPsq93sBtkt06oK3bhF3Jt04ulS3HnImilHVfSLe7df1CHiIu+LH74Ec/94UVR9pUfoMvn13UPwq4Htojo2U03OOY0j9h6KFVSIg/GMwldgLKVjpYSbEZGWOYw1zN60MxvXR6cW0ZkNIrUrn5fZCLde0B6ALw8NExnDu/qKnIyOYlHYCTpI8JJTIX+SZ0w6J8vVSrJllfuHd8adQztKPaasSFDVcHIiGI0rZoeMhrbbvL3G+Cuy8cfhnV1KUhic+nWsp+PYXOCcS3Qw1BpVC6yPwi694akR6JChdg6O7q3TbCtI4TVuGugr82h3igAA+Gyl1fTbTCiICHWTE5WYDeuR4TA06jz3n1riUqM0rJOEVM0V02qNUrtQVCwlqaeliPa71zKMbGQUD7rjoshdue3CMwqfFSTDxC9/vMx65HQourRUy0XULJ5R+F4jy0NpX+0mWwtfv3VIe5clYdwiOcleE9CV51lLEQ3EiZcOEY0kolwiyiOi8Xb3FzEqjH4QVj5+mSfyiDhFkwYp2DVhFIZ3a2Fq/5n3Xoihna2nS7aLVjFWDtEN7H7Cj7XiQcGwVeETUSKANwBcAaAbgBuIqJsdfUUbhp2ksI9vY5vC1mXQwKCubqzTrXUaPri1v9tiGHJN79YoKLa/MIYZVFozqYyBTHYqFGuxe4bfH0CeEGKHEKIcwP8AXGNznxHxz7G9MW5Ie/TKlFd2rqPinj9myXvuCrdFiFtSEhOUzUWk0pTJdhOohC8rq9KaFexW+G0A7NW9z9faqiCiO4hoBRGtKCoqMt3RoZLofMwzG9fHn0d3s1SjMjAh1vR75OS+Vs3alJSYUJXtr7sDro9MdW3lrq3SpJ8Pdw7tiE9uHyjteC//ope0Y5lldM9WuEZyOcxG9ZJxkWbqU+ySNI3dCt9onGo8ewkh3hZC5AghcjIyzNtR3/0hulqxMghMiFVPkrlDNR9kABikzTLNZgwEgPOzGssSx/PcM6wT2qTXsyXidPwV2VW/pxX8p+mwbHNrJ4E8eZV5ay8RVVWGk4XZNSGVsVvh5wPQa4hMALZkEssrLLHjsDFJn3bVZqrslqnKBJlFklK3aQN11y+cpH2zBvhh/DA0T6ur7ILhlDsH4c6hHZEWRfWrUPzmAjleVjf0l5fDyB+no+IkzAx2K/zlADoRUXsiSgEwFsA0m/uMe778/QVVr1+47jykmMjwGIiM0z0S6xlHF9dGVV2T3TIN46/IVk4ZygqmpCCvYxlbry4hRAWAuwF8A2AzgClCiI129snUJNVE0fVQWLmWIolWtDuAxiyxVl81kOREr6is4Pi/YbAbUJv06OpdtGtiX0pvt7B9OiWEmCmE6CyE6CiEeM7u/phAhJSwcBmzuEjkGNChieV+7ODz3w12re/+WWqOiZ7bFAp6k+Fx/d5vzsfvLpabvFAF+PlZEr8ZnOW2CEFJkPAr+5W1Fb0fyb5JEsxPdiBrQd4Md11yLt6/5XxT+75+Yx8A9gcOPT7aWniNallFL8lujqTEBLw0piduGniOsq6x0aLm1RUlq/ccdVsEPHl1d7dFqMGrN/gu9Nbp9aTY8B8b1RU3DTwHo84z7/rWWZd73mhxtlPzhlE/dscDCQmEi7s0N7XvZV2952niJK0a1cMz1/ZQdiISLZ74FgeiiERs28RehXJ9v0zT+3aSmE7h6l6tsWvCKNRPScJrN/Y1fRy/7bdJgxQ8c20PS4uq/jTGAFA3ufaM2cws6sHL1bT528HGp0ZEtN1r2s0eqPlU9e5vcvDWr8yfC3bx0bgBcg4kafH4PZNPU7GAOrHRFgj3O/9qYDt89NMeAMDs+6IraRctL/5fTyzOOxRVOPz9wzujYZ0kjO3fFifL5FelsjJrlll8u7FuVt+rbaOqOrR+IplFdW+dhtdu6INhf/8WAHD3sE7S5FOdBgapDJY9ein6Pz+/RttVvVqjQ0YDlFVUIlmz590z7Fxp/vIy+eDW/hjSqZmUY/kXWf1Pkk0bpOCwiaI/l5h8moqEhnWSUFLmXrEmjyj80Bp/cMdmVQrf6KKRSUIC4c6hHfGXaZE7I917abXSqp+i1k+y4MGhUo+XkpSA8opKwwXcDC2aNxR92zVGBxsLzscazdOMb8jdW1dHRKtmHweAxATC2UohtR7F0M4Z+OquC9ArsxEGdWyKpg1S0O/ZeVWfv/yL3vj5v5dI688MbvtKecKkE877Q4aXSjR8s/GAo/3Ziez8H/5fwqhA9UWdwkdaX97d/lnqqzf0iQnb97JHLw2/kaJccK5vVi875VnvtukgInRukYqmDasnEP8c2xv927vv7TS6l7tVrzyh8EOp85ZpdQ2Vi538etA54TeKcxIDbsK7JoxCt9ahfd13TRiFCyO4KVjl6l6tMbKH9fzndhNsdu8Wj13ZNeJtu7TwPaXZHVl9+4U+d9HBHSMzG3125yA7xcEz1/Sw9fjhUMt+YJKKysqgnwmIiEwFZvn4tgFoEXDhJcnwg1QAfYoGWXz+u8H4eu3+qO2Ys/9QMzGdf93DLuIhUEk2kRb6blgnCQ+NzMalXVugp8RstUY8NqobHhsVuctojs0xD257+3hC4X+8dE/Qz85WAr0yG+GWC7Jwy2D5wSH+R1M9qZJyi7jJRZ0zqny4ZdKjTSP0aNMIczcdDPm7BZLdsubsX7/uoRot0uybYMQidZMTUHqmelI2674LkZyYgIEdnPdtb1QvGcWnzzjeryp4Yipaeia4Z4sQAkSEv1zVHe2aOhMqPcCFE1k2dw7tgDTJaRn0DO/WAn+xkB1RZRY+eLHtfXTXmb+GGEw63MAouOvj2wZg3v3VC/8NUhLR1sWUBWcDCqX8rE+bIFt6E08o/FBRhOrXwVEUBwbuliDZEVW4CK2kknDC02r6PUOqXn90myQ/dosYDdkF5zZDZmN1ctIErq9dn2M+biYWiX3bA0L74Udb+jDeyG6Zii0HTtRqd2rUAh/3AWMPHqdpk67WgmggwW5IvdraaxMPxdW9WuPb3CLMDuGl9uCILg5KVJvRPVvjzUXbq94P7tgMuyaMQnlFJUQcTA+9McMPoR8yG3Oofii6tEw1bHfqPjnz3gvx1zE9a7S5r+6Bfue478JnBjfHrkGdJLx1U7+q90bXntvmp26t07DzhStrtackJbheglBWXYFQeELhh/Kz79oyttPauoVTs50OGQ1xfU7NKlpjLKSnsELnFhzQJYNHrsgG4AuECkRm5LZZVMvfv+Lxyxzry5MKf9Z91S58iv22MYOblrABHZri67uHhN9QMlPvGoKVUV58b+tmtE4QzlVWhfO9vpZZ1EiWRvXtcwSIVZIS/Jlo7f/xPKHwA8epq65Yharl4VQhmGJ325rphuKql5JYIzozEroGFEaRmQDPiI9vG4DFD19iax92kWpzWpNY4d8OTxL0eOIXcDp1gpcIpthl5jjxMoEuhrILaQdSPyVJuXxLkfDJ7QOR1Uwdbx03GdHdvSju2DtzDGB9Lx+39b1KN/FJt0fu9uhGMJEedUatJoM8UkDEDhrVS8ZtQ9rjur72r115w6QT6jNVrwBFCDaTd9tFTQXXTD/RBKCda7NJJxyqLUgyxnzzh+o07USEx0d3C5tLSgaeUPihZoN8/ocmuEnHUTFqoeLvVt/FMoexiNvnkMoEc4e2G08o/PlbCt0WQXnGnt/WsD2YXnX7YlVJ3wcWamHCoOLdmgHgEYXPhCfYNXhem0a12n6R0xZDu9ifhjgUKumMI1rVJIVECooKMqbX85nAovV4YuzHE4u27ZrUx54jp4J8qsIloALG41DfwFXuxYDIVzdQwRZN5HvSCYwODX2+uUs7FxOT+RndsxXKKipxdS/zBe8Ze/DEDN8od/ktF2QBUGumaIRT0Z3BxkHV4VFBrhZaVGhSwPk1497gQWFumcJuG+JLROeWbVgPEWFMv0xLBe/jgbVPXI61T1zuaJ+WfhEiup6INhJRJRHlBHz2CBHlEVEuEY2wJmZojDw6Oipe99QfsHND/3aO9BfM6UXVG2JaPXUiMt1ez4gGldxZmdA0qp/seOSx1VvwBgDXAfhO30hE3QCMBdAdwEgAbxKRbS4OsXSSn59lb2BOMIJFHKsaidxMQfuvCmamYPjTvCssIqMAlhS+EGKzECLX4KNrAPxPCFEmhNgJIA9Afyt9hcLo0dE/KVPt/HdLwX69br9heystDfDNXIc3YkJN+O0spxkKt+MmmNjALiNbGwB7de/ztbZaENEdRLSCiFYUFRWZ6uyNG/vWbtSew1Wd8TiduuDYqdpl3a7vl4lLujTHpNsG4C9XdXdUnlgg8Nzxm8WEAP45tjf+FJDb/dLs5mjZyJ1skMO7tQDAEa1MaMJ66RDRPABGyR8eE0JMDbabQZuhhhNCvA3gbQDIyckxpQXdLJkWNQrdgPz1eAcrUiJPVfwn5ZQ7B+Gr1fuRVjcJ1/SuPX9xs/aCv5AHw4QirMIXQphJ1pwPQB/pkwnA2KbAuAZXA4uO7JZpGH9F7fD31DpJOFFWgdsu7OCCVAwTOXaZdKYBGEtEdYioPYBOAJbZ1FcN2qRrsyzteVy1BV2VpGF9H5qC4lIAQOmZsyG3q5Psu4zqJnPqBUZtLAVeEdHPALwGIAPADCJaI4QYIYTYSERTAGwCUAHgLiFE6KtGEv4w+Ov7ZSL3wHE8MNzdGprhcFPpsr6PjMMl5ejobuAxw0jBksIXQnwJ4Msgnz0H4Dkrx7dC3eREPHvteW51HxNwzvvIaNYwxW0RGEYKnguFUyG0PBJ+rtVx9XtXuAHr+9D4F2GTEyO7TNg1klEdT+TS0aN6Clv/kkJ2q1TXvSp40TY02S1TkX/0dJWNPjh+f03bRWIYS3hO4ascDekmbdLr1Urzy/opNK+M7YP1+cVonuqObz3DyMZzJp2YUfcOa1uj++BgDtIJScM6SRzIxHgKzyl81XErtYKReypnM5QDP1QysQJf8Q6z+/BJAEBZRaWj/Rply6xkm44U/LEfKtXhZRgjPGjDd1uC0OzXgnmW7TqCS7KbO9av0Qyf3TLlMPHmHCzZcZgrPDHK47kZvupVdjpmNADgvP3cKAqU9b0cmjasg9E91T7vmJo8c20P11KVu4nnFH7rdPcSWEVCizSfx4fTKR9uv8hXEal9swZVbazwmXjlpoHn4NM7B7sthuN4TuGrbtIZp5Wiy3a4FF3j+r5o0czG9dC2ie+mGBgo1KQBR5QyjJfxng1fMcfMr+8egqteX1z1/tKuLVwNuCKiqjEKnOHPu38ojp4qd0EqhmGcgGf4NtOjTRoeGN7ZbTGqEEJUF/II+KxJgxTlawEzDGMe7yl8twUIgIhwz6Wd3BajRgRyal1f4WTVxophGHvxnkmHtZghLdJ8LoM9MxvhVwPPwbQ1+3FO09hINMcwjBw8p/D7ndPEbRGUJLtlGmbcOwTZLdOQmED47dCObovEMIzDeE7hh89sGL90b93IbREYhnER1o4MwzBxgucUPpvwGYZhjPGcwvd7oDAMwzA18ZzCV4UGilfeYhgm/vDcoq0qrHtyBMp1KZDf+lVfLMotclEihmHiHVb4NpGYQKinm+WP7NEKI3u0clEihmHiHTbpMAzDxAmWFD4R/ZWIthDROiL6kojSdZ89QkR5RJRLRCOsi8owDMNYweoMfy6AHkKIngC2AngEAIioG4CxALoDGAngTSLiVUyGYRgXsaTwhRBzhBAV2tufAGRqr68B8D8hRJkQYieAPAD9rfTFMAzDWEOmDf9WALO0120A7NV9lq+1MUzUNONasQwjhbBeOkQ0D0BLg48eE0JM1bZ5DEAFgI/9uxlsb1hQj4juAHAHALRr1y4CkZl44/qczPAbMQwTlrAKXwhxWajPiehmAKMBXCpEVQ2lfABtdZtlAtgf5PhvA3gbAHJycrjKKlMLDmJjGDlY9dIZCeBhAFcLIU7pPpoGYCwR1SGi9gA6AVhmpS8mfqmo5HkAw8jAqg3/dQCpAOYS0RoiegsAhBAbAUwBsAnAbAB3CSHOWuyLiTNeGtMTANArMz3MlgzDRIKlSFshxLkhPnsOwHNWjs/ENz/PaYtLujRHRiov2jKMDDjSllEaVvYMIw9W+AzDMHECK3yGYZg4gRU+wzBMnMAKn2EYJk7wTD78D2/tj+OlZ9wWg2EYRlk8o/Av6pzhtggMwzBKwyYdhmGYOIEVPsMwTJzACp9hGCZOYIXPMAwTJ7DCZxiGiRNY4TMMw8QJrPAZhmHiBFb4DMMwcQJVVyV0HyIqArDb5O7NABySKI4X4TEKDY9PeHiMQuPW+JwjhAgbfaqUwrcCEa0QQuS4LYfK8BiFhscnPDxGoVF9fNikwzAMEyewwmcYhokTvKTw33ZbgBiAxyg0PD7h4TEKjdLj4xkbPsMwDBMaL83wGYZhmBB4QuET0UgiyiWiPCIa77Y8siGid4mokIg26NqaENFcItqm/W+stRMRvaqNxToi6qvb52Zt+21EdLOuvR8Rrdf2eZWIKFQfqkFEbYloIRFtJqKNRHSf1s5jpEFEdYloGRGt1cboKa29PREt1eSfTEQpWnsd7X2e9nmW7liPaO25RDRC1254HQbrQ0WIKJGIVhPRdO29t8ZHCBHTfwASAWwH0AFACoC1ALq5LZfk73gRgL4ANujaXgIwXns9HsCL2usrAcwCQAAGAliqtTcBsEP731h73Vj7bBmAQdo+swBcEaoP1f4AtALQV3udCmArgG48RjXGiAA01F4nA1iqffcpAMZq7W8B+J32+vcA3tJejwUwWXvdTbvG6gBor117iaGuw2B9qPgH4H4AkwBMDyV7rI6P6wMs4QcaBOAb3ftHADzitlw2fM8s1FT4uQBaaa9bAcjVXv8bwA2B2wG4AcC/de3/1tpaAdiia6/aLlgfqv8BmApgOI9R0PGpD2AVgAHwBQklae1V1xKAbwAM0l4nadtR4PXl3y7YdajtY9iHan8AMgHMBzAMwPRQssfq+HjBpNMGwF7d+3ytzeu0EEIUAID2v7nWHmw8QrXnG7SH6kNZtEfrPvDNYHmMdGjmijUACgHMhW/GeUwIUaFtov9eVWOhfV4MoCmiH7umIfpQjVcAPASgUnsfSvaYHB8vKHwyaItn16Ng4xFte8xBRA0BfA7gD0KI46E2NWjz/BgJIc4KIXrDN5PtD6Cr0Wbaf1ljFBNjR0SjARQKIVbqmw02jenx8YLCzwfQVvc+E8B+l2RxkoNE1AoAtP+FWnuw8QjVnmnQHqoP5SCiZPiU/cdCiC+0Zh4jA4QQxwAsgs+Gn05ESdpH+u9VNRba540AHEH0Y3coRB8qcQGAq4loF4D/wWfWeQUeGx8vKPzlADppK90p8C2gTHNZJieYBsDvRXIzfHZrf/uvNU+UgQCKNVPDNwAuJ6LGmifJ5fDZCgsAnCCigZrnya8DjmXUh1Jock8EsFkI8Q/dRzxGGkSUQUTp2ut6AC4DsBnAQgBjtM0Cx8j/vcYAWCB8RuZpAMZqXirtAXSCb0Hb8DrU9gnWhzIIIR4RQmQKIbLgk32BEOKX8Nr4uL1QImmx5Ur4PDO2A3jMbXls+H6fACgAcAa+mcI4+Gx/8wFs0/430bYlAG9oY7EeQI7uOLcCyNP+btG15wDYoO3zOqoD8gz7UO0PwBD4HoPXAVij/V3JY1RjjHoCWK2N0QYAT2jtHeBTSHkAPgVQR2uvq73P0z7voDvWY9o45ELzVtLaDa/DYH2o+gfgYlR76XhqfDjSlmEYJk7wgkmHYRiGiQBW+AzDMHECK3yGYZg4gRU+wzBMnMAKn2EYJk5ghc8wDBMnsMJnGIaJE1jhMwzDxAn/D0MA15+1ZpeKAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "temp = float_data[:, 1]  # 온도(섭씨)\n",
    "plt.plot(range(len(temp)), temp)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 그래프에서 온도에 주기성이 있다는 것을 잘 볼 수 있습니다.\n",
    "\n",
    "다음은 기간을 좁혀서 처음 10일간의 온도 데이터를 나타낸 그래프입니다. 10분마다 데이터가 기록되므로 하루에 총 144개의 데이터 포인트가 있습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD8CAYAAAB0IB+mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJztnXecG+W193+P+korbe/e9a7ttY17xxhMNb0TSCC5BAJ5CaQn75sE4pt7Q0u4gRRCCpBAyKUmgZBCiQE7YKqNjcG9t117vb1JWvXn/WNmtCOtumZUz/fz8cfSzEhzPJZ+OnOeUxjnHARBEETho8m2AQRBEERmIMEnCIIoEkjwCYIgigQSfIIgiCKBBJ8gCKJIIMEnCIIoEkjwCYIgigQSfIIgiCKBBJ8gCKJI0GXbADnV1dW8tbU122YQBEHkFZs3b+7jnNfEO051wWeMXQDgQQBaAL/nnN8X7djW1lZs2rRJbZMIgiAKCsbYkUSOUzWkwxjTAvg1gAsBzAJwHWNslprnJAiCICKjdgx/GYD9nPODnHMPgOcAXK7yOQmCIIgIqC34TQA6ZM87xW0EQRBEhlFb8FmEbSH9mBljtzDGNjHGNvX29qpsDkEQRPGituB3AmiWPZ8E4Lj8AM75o5zzJZzzJTU1cReZCYIgiBRRW/A/BNDOGGtjjBkAXAvgHyqfkyAIgoiAqmmZnHMfY+yrANZASMt8nHO+Q81zEgRBEJFRPQ+fc/4KgFfUPg+Rn3xwsB8Wgw5zJ5Vl2xSCKHiotQKhGgd77Wi9/WVc++j7Effv6x7FtY9+gEt/9U6GLSOI4oQEn8B7B/rw1Wc+QiCg3EB7p8eHzzz6AQDgg4MD4Hz8vdfu6sbv3z6Ic3++PrjN4wsodm6CICKTU710iOzw2d9tAADce+VclJXoFXnP772wDb2j7uDz4TEvys0GDDo8uPmPE9tn7OwawYLmckXOTRBEZMjDJ4I4PT5F3sft82Ptrm40lJlw1+WzAQAH+xz486YO3P/anpBjf/bp+dAwYN2ubkXOTRBEdMjDJ4I43MoI/sZDA3B6/HjouoXQawWf4vt/3YbdJ0aDx1wyrwG3njEVc5rK8OKWY3jx42P41rnTwVikWj2CIJSAPHwiiMPtV+R91u3ugVGnwYqp1agvMwEAdp8YRZ3NCAC4YkEjfvXZRZjTJGTmXLmwCR0DY9h0ZFCR8xMEERny8IucDw8PBB8n4+G/va8XJr0WS1srg9s45/jHJ8fxv+8fwent1SgxaNFeWwq9lsHr57jvqnlYMa0KOk2on3HurDoAwDUPv4+nv3gyTp1WDUDI4qm1mfCVpz+C1aTD8ilV+PSSZpQYtACA7ceGMaXGArOBPsYEkQj0TSlyrnl4PGXSnqDgu7x+XP/YRgDA4fsuDm5/ccsxfPvPnwAAPn9KKwCAMYbHbliKnV0jOH16DbSaiSEbq0mPhS3l2HJ0CJ/7/QbcesZUWE063L8mNN7/6vYT+O9/7MCTNy/DoNOLrz+7BWfOqMETX1iW1L+ZIIoVEvwixuUNDeE4PYmFdN7c0zNhmz/Acec/dwIAHr1+Mc6aWRvcd/r0Gpw+PXafpGf/z3Ks292Du1/aiYffOhDcvnhyBdprS+H1c9hKdPjDu4eDPzaCLb346OggFrVUgHOOAEfEHxWCIEjwi5ohpzfkeaIe/vsH+oOPXV4/dBqGR9YfxPCYF6tOqsN5s+uTtsWk1+KiuQ04e2YtLnnoHezvseOpm0/Gae3VIcfdesZUvLqtC2t2dOP7F52Em/74IW54bCP+z+lT8KcPO+APcPzja6ei1mpK2gaCKHRI8IuYoTFPyPNEYvicc/xrx4ng81GXD//acSIYfvnVZxemZZNJr8Ub3z4j6v46mwk3ntqGG09tAwD833On4/a/bsPPXt8bPObfu3vwmaUtadlBEIUICX4RE+7hOxII6byxqwfdI260VplxuN8Ju9uHHceGAQDXL58Mk16riq3RuHZZC6bWlqLSYkBblQXz73oNWzuH8ZmlGTWDIPICSsvMEXz+ADYfGcSgwxP/YIWYIPgJePibxdTJ75w/EwBw1gNv4rkPO3ByWyXuvmKO8kYmwNLWSkytKYVGwzBvUhm2iT9ABEGEQoKfA2w/Noxpq1/Fp377Hr767EcZO++WjtC89z2ywqhoHB1wYEq1BdWlhpDtp02rjvKKzDK3qRy7ukbg9ilTU0AQqbK3O/73KdOQ4OcAlzw03i1yzwl7xs77yFsHQ56/s78P/jgN1I4OONFcaYbVFNpz55L5jYrblwrzJ5XB6+fBOxGCyAYvbunEeT9fj39HyGjLJiT4OYY/EAjpLJkpWirNAICeUVfUYwIBjkO9DrRVW9BYHpoF01plVtW+RDlzRi3KzXq8sPlYtk0hipiNhwSHo2PAmWVLQiHBzzLhnuig05sx79SkH//vv375ZACx2ysc7nfA4fFjVoMN5WYDXvraacF9udIDp8SgxawGGw72Ze5OiSDCkVKcSzKcxBAPEvws889Pxme611iN0DBg/b6+jJxb/mGsE3veRFu49Qc4vvP8VgAI9sCZ01SG/750Fm5c0aquoUnSUmnOOc+KKC6k71GixYyZgtIys8wnnUOY31yO65Y2Y2lbJb7x3BZ8cKAffBVX3WuWfxilRdhIgu/1B3DTEx9i85FBtFSacVKDNbjvC2I+fC7RXGlGn92DUZd3wloDQWSCgBiW3X1iJMuWhEIefhbpGXFhy9EhzG2yCfnkNaU4e0YtNh4ewMW/fAdev3pToAIBDrcvgIvnNuCeK+ag1Cj89kfKxX//QD/e3teHJZMr8MJtK3ImfBON2Y02AMBD6/Zn2RKiWJEmuO3tzq3QIgl+Fnlqw1EAwMVzxzNcvnp2O246tQ07u0Zw78u7VDv3mNhHZ+6kMvzH8skw6oTwTqRRg1K65sPXL0aN1aiaTUohddt8dP1BSs8ksoJb/B7lWmiRBD9L+AMcv1y7DwAwp8kW3G7QafD9i2ZixdQqPPHeYby3X514viT4ZrHVsEEnfBQ8/okCufnIIJorS1BdmvtiDwB6rQa//uwiAMD/vncky9YQxYjUmLBn1J1wj6pMQIKfJXYcH68GDY8z67Qa/O7zS2AxaPHYO4fAOYdP4fDOmBi6kVohGEXBd3tDz8M5x6Yjg1g6uRL5xKnTqgAA9/1rd8hsXYLIBPJOtI/Iur9mGxL8LPGO6Llv+s9VEfdbjDp8Y1U71u7uwcK7X8e01a/irb29ip0/uocfKviPv3sYfXY3Frbk14DxcrMBL355BfwBjqX3voGu4bFsm0QUEW5fABfNFbrGRgqTZgsS/Cyxu2sUTeWxwyQ3nzYFy6dUBnvefP+v2xCIUwmbKJKHXxLm4Yd/OD86KtQEnH1SnSLnzSQLWypw52XCEPWnPqDQDpE5XN4AykoMKCvRT5g7kU1I8LPErq6RkPTGSGg1DM98cTk2rj4H91wxB8eGxvCbN5XJPJE8fEnwJQ/fHSb4NpMe1aVGNJWXKHLeTHPDilYsba3AK9tO5NQXj8gMIy6vqtlu0XD7/DDqNDDpNXB5ycMvalxePw72OTCz3hb3WI2GodZqwudObsH0ulK8uUeZsE7Qw5dCOlophh8qiiNjXpSV5He5xg0rWnGoz4FLH3oHoy5v/BcQBcO8H76G//j9hoyf1+0NwKTXwqTXwpVDmWIk+Flgf48d/gDHSQ3xBV+CMYZTplRhV9eIImGdoIcvCj5jDDaTDsNjgiB2DDjRevvLeHlbV94XL10yrxE/vHQW9vXYsX5vZqqYidxhw6GBjJ5vxOWFxx+ASa+BSacNOle5AAl+hnn8nUP4f38RBn3PjBPSCWd2YxkcHj8O9zvStiM8hg8A1VYj+uweuH1+XP/YuFck/QjkM589eTJ0GhaSHUUUNkqtdyXLqp++BQAw6rRCSIcWbYuTruEx3PXSTuw+MYq5TWVoq7Ik9fpZYgXp9uPpl2s7wzx8AKguNaLP7sa/tp/A4f7xgpF7szTYREkMOg3aqi3Y15NblY+FjNcfQL89eymx4etRmaJHTAPWaxkMOg08FNLJf+xuX0IfZofbF+ynseGgcGt5wymT8diNS6DRJNeiYHqdFeVmPV6TzZRNFVckD7/UgD67G2t39aDKYsC/vrkSO+48HytyZLhJukytKcXrO7tp8TZDrH5xGxbf80bW0hKz8f8snycR4BxGnZbSMguBT/3mPSy+543g8yP9Dvzgb9snjCi87emPcMEv3sagw4NnNhxFS6UZ/33pbNRaTeFvGReDToOT2yoVGeHnjCD4NaVGHOh14NXtXThzRi1m1ttgMeb3gq2cxZMrAAC/+Tf12MkEL24RZhIc7ndg0+GBuMN1lCYbi6XyqlqDViN4+FnIEooGCX6K7BHHl1344NsYcnpwxv1v4skPjuDMB97EDY9vxKDDg0CAY71YLLXw7tex8fAAPndyS9KevZzqUiOO9Duxdld3WvaPef0waDXQacc/As3iEBSvn+OyBbkxwUpJbjqtDaVGHV7bmd61IxJD6s/0pw87cPXD7+MP7x7K6Pm3dWZ+veZdWSsUi1EHg1ZDHn6+s/nI+Kr/rq4RLLjr9eDz4TEv3trbi79uOYYPDvZPeO2Np7amde7rlrUAAO5fsyet93F5/SEDUABgsmxNQZqAVUhoNQzfOnc6dp8YxaG+9Be+idiUlQjZXY+9Iwj9kf7MNRLjnOPvH4/PmjjQm5m1my8/LcykNuk1uGReoxjDJ8HPa9btnjinstSowyf/dR6qLEJf+f099uDkqvduPxt3XDgTO+48P+j1pMqcpjLcdfls7D4xmlaPDqfHF7JgC4SOKay0GMJfUhBcNLceGga8sLkz26YUPLaS0HTeUlPmwoPrdvfg5W1dwefniJkzmeL3n1+KEoO2eASfMfZDxtgxxtjH4p+L1DpXptl8ZBDzJpVhzTdPD2575PrFKDPr8eZ3zsRJDTY8u/Eofvr6XgBAvc2EL50xVbF4+JULmwAAf9rUkfJ7jHkDMBtC7WmWefW2DH45M0lDWQlObqvKueHShciYJ7RLpF6bOf9yIGwtLdMYxbvnYovh/5xzvkD884rK51IVl9ePQIDjxS2d+ODgAOY2lWFGvRUv3LYCXzpjCk5uE7pJWk16fEE28q+t2pJWzD4SVpMeXz5zKo72O1MuGx/z+IOdMiVMei2uXjwJD1wzP+eHnKTDsrZK7OwaQc9I9IHtRPqEj/ezuzLXJtgYYZZsJrJ2pGLKRS1CgoBBq8laemgkCtONUxi3z4+zHngTXcPjAiHlxC+eXBHM/pBYNasOeEF4/MA181SxaVptKXwBjiP9TkyrLU369WNeX7BTppwHrpmvhHk5zfmz6/Hg2n14c08vPr20OdvmFCxOjz8kpNE94oLXH8iIpy81A5QTyclRmkqLHksmV0ArOnnGYgnpiHyVMbaVMfY4Y6wi/uG5ybf/9EmI2APAZfOjZ7FUWgzBZmPtdclV0yZKo/j+3Sl6qWMef0hKZjExo96KEr0Wu8VJXoTycM7h9PhQLVsLenlbF259cnNGzh/p/jQToRWXNxAM5wDjIR3Os1P1G05aHj5j7A0A9RF2rQbwWwB3A+Di3z8FcFOE97gFwC0A0NLSko45qtA56Awu/ly1sAktVWbcdFpb3P4yz92yHDotg02lPjTS4nB/irFKp8ePSkt+TLBSGq2GYUa9FZvF1s+E8rh9AQQ4UFVqxHGZs7Q2QsKDGnj9EwU2fLiPGrh9/mB2EiCEdDgHfAEOvTb7YdK0BJ9zHnl6RxiMsd8BeCnKezwK4FEAWLJkSW78DEJIt/zThx144r3DAIBL5zfi/mvmB2/V4tGsclqjlEXzzIYjuHhuQ8J2Sbi8/oghnWLhrBm1+PkbezHi8qr2o1zMSL2aqkqzk+0VaW0rE/ON3d5ASDjJIJszkclF62iomaXTIHt6JYDtap1LaR556wAufPDtoNhfvXgSHrpuYdKiqiblZuGL9MHBgaCdyTDmLd6QDgAsECd4bVegapmYiNSrqaEsO3MU5HFz6XubicVTty8Qsk5giDJYKFuo+ZPzE8bYNsbYVgBnAfiWiudSlB+/ujvkeS4uZMp/fA6nUETk9Pgn5OEXE3ObygAAW7NQjVkMOMUWA8unVOI/Lz4Jf7hxKQBgQXNmRmXK4/WPi+fOhIfv8voje/g5kpqpWpYO5/x6td5bTQ6KFXlVFgP++bXTMOjMbj5vLK5d2oznPuxAIIUFIZe3uAW/0mLAzHor/rblGL50+pSCTkPNBlJKpsWgw+UrhbqRk9sqkamYrTykY4wyzU0N3L6wkI62eDz8vMIntnL98yahAvN7F85EY3kJZjeWZdmy6Nz3qXmYUWfF/iRb/nr9AXj9vKhDOgDwqUWTsPvEaMoL30R0JME3G8c/Yya9dsJENbXInuCHpn5KxZbhNQnZgvLwRW7/6zY8L5bbT6oowRULmrJsUWKcMrUKz2w8ikCAJ1zgFT7PtliZN0n4MV+3q4fy8RVmzCuEdOTV3CZ95oqQJI+63mYKtjNRO0uHcy6kZco8/FJR8HNltCZ5+CLPy3qr/PGmZcHYW67TXlcKjy+AE0nk4w/YBY+2okD75STKsrZKWI06bD02lG1TCo6ghy8LGxp12oz1qPeIaZnvfO+s4HdZ7Ri+1KCtxjbe+txqkgQ/c1XGscgPVVOZ8KKIqTXJV65mi1axw+WK+9bhobX7EnpNrzi4pcZanHn4EowJ+fh7qABLcSLNWzDpNXBlIBceAHpHXdBqGHRaTcZCOieGhe/VDFmxpVSvM+omwc8ZRsRf36+f045DP86vHm+TZR0uf/r6Xvz63/vj3j72iSPYakqLW/ABYHFrBT46OoShHF6cz0fGonj4mciUsbt9eHZjR3DgilT5qrbgO8RmcRbZusW4h08hnZxBmlLVWmXOu2yN8Dzn+9fswR/ePRzzNZKHX20t7pAOAJwxvQb+AMcnlJ6pCGt3deOmJz4MTn6Sd4jNlIcfPnVOiuGrnSnjlARftm5BIZ0cZMCZvzHtSMVgkQavyOkbdUPDgKoiba0g56R6oQneXgrrKMLNf9yEdbt7cGxoDIyFNjEz6QUPX+2+MsNjod60MUMxfId74l1NiV4LrYZltFNoLEjwMe4RVJrzT/AjEW+yUK/djUqLIacqh7NFhcWAGqsxOLKSUIYj/Q6Y9dqQO2ajToMAj9znRklGogm+yncXkodvlt3VMMZQXqLPmdRfEnwAB3uFStWG8uQHi+cCK9urAQC3nD4Fnzu5BceHx2JmQ/SOelBN8fsgM+qs2EuCryjv7u+f0LZbyk9Xe7i4W8zBf/DaBQAE0TXo1E8JlTz88HTnpooSdA5mbrxjLEjwIYRAplRbUGvNT8F/5PrFePu7Z+H7F52EhS0V4BwT2jnLGXR6UFEgdzNKMKepDFs7h/FJB6VnpkMgEOq5y2ckA5nztL2isMt/cIxajeohHafHFwzhyGmuNKNzcEzVcydK0Qs+5xxbOoYmDDHJJ8wGXbA7Z7nYmjX8tlbOkNODCgt1iJT41CKhyO6hdfuzbEl+E+65h89FlqZQqZ2LL4WMDLLulMYMFH05PP6QDB2J5gozOgedwayhbFK0gi8tHHUOjmHA4cH8DDV1UhtpcPRIjDSwIac32G2TEIbULG2tQM8ojTxMh7Gw9gGnTasOeS6FdNQWXqmtgrwdsVGnVT9Lx+2bMCcaAJorS+D185SHFSlJUQo+5xzn/nw9TvnxWmwRb+PnTyoMwZeGLww5Iws+5xxDY15UmMnDl9NaZcmJL2Q+Mxbmua+aVRfyXArpqO3hS50p9Tq54Kvv4Q86I3+vmiuEu++OgezH8YtS8LtH3NjfY0fXsAt/2dQBg06DGfXqjCLMNNLC87GhyDHDEZcP/gBHeQl5+HJqrEb02T05M4ouH4kn5FK6otqNxMY9/PFYukGnUb1xW7/DjaoIyRBSuDUXEgOKUvCPyn5p397Xh+VTqvKmd048bCY9Ki0GHOmP3CN/WPT8y8nDD6GsRA9/gMORI10N85ExjyC0Z82owft3nD1hv9TKo1es9FYLadE2NIavVd3D77d7gqNH5UyuNKPOZsQHhwZUPX8iFIbKJcnxMO93TqMtS5aoQ0ulOWouvtTfn7J0QilLYLG7EOGcY8+JUUUWFKWQzs2nTYk46apOzIJTO3QmLdqGxvDVzdLhnAuCH8HD12gYWqss6MmBkGFRCv7HYtz+i6e1ARAEspBorUpA8ClLJwRpEbvfnhsFMpniu89vxfm/WI+Xth5P+72CbbejDNYpN+th0GlUF3xPxEVbjaqLtqNuHzz+AKqjzPCtLzMl1dFWLQpK8O/85w489s6hmMdsPzYcnAH73Qtm4r6r5uJTiydlwLrM0VxpxvHhsYiDnIeCIR3y8OVMqxVyxvf3Zj/OmilcXj/+IrYFP9Cb/JjMcMbEStNocxYYY6izGdUXfN/EGL7ai7ZSQ8JoQ9vrbSZ0j7izvkZUUIL/h3cP4+6XdsY8Rj4dyqDT4NplLTkxTV5JGspKwHnkWKnUFVLK1ycEJldZYNBqsLuIeur8/eNjir6f1CBMahgWiTqr+p6u1x+AXsvC2jqoG8OXWidE609VZzPB4wtgMEr2XKYoGKVLND4nVaA+dfPJapqTVRrKhFjps+IkLDnSB66MBD8EvVaDOU02bDiY/YW1TNEzMu4Q/HLtPjjS7NkudcgsNcYQ/DJTyHnVQBD8UGlTO4YfLxmiXvxOnohRAZ8JCkLw7W4f/r27N/g81m1Tx6AT5WY9TmuvjnpMviN9uB5atx+v7TwRsm/Q6YHNpIOuwO5qlGBlew0+6RyKWbRWSDz3YUfI842H0/uxkzz80jgefiYWbcMF32LUYWRMvY6VUpVxtHBWnS0zC9bxKIhv/f4eO259anPwebSe26MuL57ZcBRt1ZaI+wsFycMHgM1HBkP29dndqC7ySVfRWDRZ6EO0vUh640u1Gk9/Ubjb9aYZ8rC7fTDpNTFDpPVlRjg8flUHgngiePiN5SUYHvOmfRcTDanK2BRF8CUnjARfAWxhHsWoO/KHSVrQXdZaqbpN2UQerjkclq3TZ6dOmdGY1yQMNd/ZNZJlSzLHxXMbgllqQ2mmpI66fCg1xg4Vjnu66oV1vL4ADNrQBmZNFUKaaLSCxHSRis6iCX6t1YhSow6fdGa3QV9hCH5YPDradBlpwfabq6arblM2YYzh+uWTAUzMK++zu2m0YRQqLAaUGnU509lQTQbERcbZTbbx/ktpCr7d7Yu5YAuMC/7RgfSzgqLh9QdC2ioAwCRR8OPNikgVKaoQLSVVr9VgSo0lZhfbTFAQgh/+IYs2XWZftx3nzKyN+p9SSNx9xRysOqkuOK8XAPrtbhzsdURNHSOEcFjXcOEL/rWPvg9AGMdnNerAGPDEe4fT+rePurwxF2wBYEFzOUx6Ddbv7Uv5PPGIFMOfWW+FVsOwfm9vlFelh1SDYIpRsW8x6OB0Z7eSuyAEX5pZKWGPEKfz+QM42GdHe11h9MxJhHKzHgMO4dbZ4wtg8T1vABBuL4nINJaX4PhQ9gtk1CQQ4NjbLdztnj+7HhoNA+dC59jbX9iW8vvaXfE9fJNei2VtVXhtx4mYx6VDpBi+2aCDP8Dx5AdHcLhP+bsLp8cPg04TMxnCYtQGB51ni4IQfAB4/MYl+K9LZgEAfviPHRM60x0ZcMLr52gPm8JTyEyqKEH3iBtPfXAEQ2PjFaT/IYZ7iInU23KjIlJNpFqD//nU3OBiooQvkPrCrd3ti+vhA8Cy1gocH3ap1jXT658YwwfG17bUiOM7Pb6QWbaRMBt0qjeOi0fBCP7ZM+twrtiOdV+PHV955qOQ/dIYw6lFJPifWdoMANhxfCQYn/3ldQupyjYG1VYDBhyeCfULhYTUtXHx5PHkhRduWwEgeoZbIoy6fDFTMiUay4V4enhPK6WIlIcPjGcjDagwX9bh9sMSoRe+HItRGzH6kEkKRvCB8W58ALC1cxgPvrEv+HyNeAsZqZtdodJQVoKTGmzoHXVheIwKrhKhutQIf4AHew4VIn12Icwn/74snlyBC+fUp7VwO+rywpqAh99Urm7GjNc3MYYPjIcy081GikTCHj4JvnKY9Fp8ZklzcNLOz9/YC0Bolva82DPEksAHspCotRrRPeKGXVwsKo0wgo0YR0pZ7SvgJmq9o24YtJoJ6cxlJfqgY5AsLq8fIy5fQim/wRRJlbKhPBGydADAahKcHTVqABweP8xxtMVi1MHpFWoQUr3O6VJQgg8A/3P1PDx58zIsEWfUdgw4Q/6DI82cLGRqrUb0jLpkja2K6wcvWSTB6rerW/6fTXrtbtRYjSG9ZgAhvTnVKmOpoCh8TSASdTYTNExFDz9KDN+k10CnYVHTttPB6fbBEsfDtxi04Bw464E3Mf/O1xS3IREKTvABIQ999cUnAQB2dY2EpGmGZ/QUOnU2E3pHxz38eLedxU6NVQj59Raw4AvFdxNDm2Uleri8gZR6zhzoFbJ+ptTEr2LXazUoNxtUC5t5fJFj+IwxWE069Tz8ODF86Q5Aunv0RehmqzYFKfjA+OLsI+sPqhKzyxdqbUYE+Pg8TRL82BRLSCdS6EUK8Vz04NtJv6dUUNRUnthsCUF41YlnR1u0BYTmgU99cFTxJmZOjy9u9KAyLFkiG2GdghV8mxiv23xkEAd77XGOLlxqxSlDD64VFrCLoegsHcpKMjOkI1v02d3Y1TUSMuZTokT0UA/0OpLu2y6Jt60ksZChmoIfzcOXs3Z3t2Ln45zD7oq/aBte8JiNVskFK/hyfve20EPn55+Zn2VLMk+tLdSTi3fbWewwxtBcUTKhjqNQkNqLrGyvmbDPKFvoTLZ3/KjLC62GRe0WGU5ZiT44m0FJhse8ODHiiltNrmTWbdewC/0OD9prYxd1hofR1Pj3xyMtwWeMXcMY28EYCzDGloTtu4Mxtp8xtocxdn56ZqbGW985M+T5lQsLa7JVIki9SyS0momLWUQoLZXmiB5wIdAjDsW5dlnzhH0GmeAn21XS7hKKrsIXgqPRVF6iSs+iPrsbAQ7MjjKn+jvnzwCQ/L/ypCPLAAAgAElEQVQvFtIUucby2AvWlWHDUYby0MPfDuAqAOvlGxljswBcC2A2gAsA/IYxlvFYwuQqSzDXONoHoNChRmnJ01JpxtF+Z9bH0amBNEhbGiguRy74iVaE+gMcIy4vRhNoqyCnoawEPaPuiGM408EeZ+rWbWdMBQBFq3yldgnxUr7Dp8xlo9YjLcHnnO/inO+JsOtyAM9xzt2c80MA9gNYls65UmWu2PL2K2dNy8bps44hRjMnIjLNlWaMun1Z8cDUpmfUDaNOEzHWLg/pJNrz5cn3D2PeD1/DtmPDwTz3RJBmMgwqXPUqVbJGq3rVaBgMWk1aFcXRzhkvXKrRMPz1yyuw/jtnASisRdsmAPJxOp3itoxz6xlT0VJpLloPHwBe+9bp2TYhr5D6wxdiWOeZDUfBGCKGXkIEP8Gujmt39wAQ2pnokggXVosV70pnQwXHLMa42zDpNYp6+FIHzERqfBa1VAQLz174SNmZwokQ9x6MMfYGgPoIu1Zzzv8e7WURtkW8P2aM3QLgFgBoaWmJZ07SLGurxPrvnqX4++YT0+us+O4FMzClwCd9KUVLlSD4h/sdmN9cnmVrlMPu9sXs5aLVyEM6iXn48swUTRKCXyUVuDmUrXeQQjqxmriZ9FpF59tKrZHNCRY1Sutou7pGcKjPkdEJfHEt5JyvSuF9OwHIV4UmATge5f0fBfAoACxZsqTwgqY5wpfPLM6QVipMrSmFQafBts5hXL4gKzemqiC1BV4ZZZ5ze20pGstMOD7sStjDH3R6sbClHItaKnDlwsSvlZRF06+Whx9D8EsMWkW7Vkp3C0Z98gGTvd2jGRV8tUI6/wBwLWPMyBhrA9AOYKNK5yIIRdFrNZjTaMv6ODqlkVJNb79wZsT9FqMOz96yHEDiHn6f3Y2GMhN+cMkszBHXyxKh2iIVuCns4bvjL6CWl+gVzYEPjjdMoYo/0+m/6aZlXskY6wRwCoCXGWNrAIBzvgPAnwHsBPAvAF/hnGe3ETRBJMHClgp80jkczFsvBKQ0yEkV0athpYVHR4IecF+Uqt142EqEKVvPbDgKv4JJ8Xa3D3otC1mPCKfSYggOBlICqWYhGQ9fWlfrHBzDC5s7sfxHazPSkjvdLJ0XOeeTOOdGznkd5/x82b57OedTOeczOOevpm8qQWSOm09rg1Gnwa/W7Yt/cJ7QOeiEzaSL2SJbWnj86Mggbn7iw5hN5Ny+xDtkhsOYMGXrYJ8Dr27vSvr10XC449cDVJgNimZgubx+MIaYPzLhTK+zYma9FR0DTvzfv3yCEyMu2DMwDYty9ggiAo3lJThlShW2dg5n2xTF6LN7QnrgR0IKS7y45RjW7u7BOT97K+qx0iCRVAQfACaLi+NvKzjf1u7yxc2Ht4iD6pUahOLy+mHUaRIuOpNoLC8JGWqu9HpGJEjwCSIKc5rKcLDPoWhVZjYZdfvi5sqHZ9rE8oT7RiXBT22o0MtfX4mT2yrx/sH+lF4fidEExixKPwin/HitIud0eQMwJdhSQk6dzRisfAaAm574UBF7YkGCTxBRkFr9KpGP//ePj2Hx3a+rNsc1Eewub1LVsAAwqyF6/Yq04Fod564hGqVGHeY3l+PEiEuxqmZHAoIvDQFKtl9QNFxef0oLtlaTHnb3+A/qjLrYvXiUgASfIKIwuVIQ/CP96Qv+N577GP0OD/aIA8SzQaJDxu+7am7wsSaGQkgzA6otqbfvqCk1wuMLKBZTt7vjz9VNNvQSD5cvAFMKKZlmgzak4veuy2craVZESPAJIgpSAdbRAYdi73nLk5sUe69kkRqcxeOqRZPwk0/Nw4Vz6qPmq3PO8d3ntwIQBr+nilTYtuHQQMrvISeRf2OFWdm51i6vP6WQjtzO65dPRq0t/rSwdCHBJ4golJXoUWrU4fiQcr3xu0eyN0lr1B1/QRMQ+i99emkzrCZdsG1AOIf6xn8E02m5PW9SGRiDYnc+idzFXLNE6Jpbr5DAurx+GFMQfPl1SybDJx1I8AkiBg1lJnQNqzN7NZNwzmF3J9fR0mzQRS3AWr+3FwBwxYLGtOwy6bUoL9Gj167Mj2oigq/XanDdsmb4FVo3cHsDMKUg2FJPHQBQOMoUFRJ8gohBQ1jqXL7i9PjBeeyWA+GUGLTBPjESXn8AHl8AXSMu6LUMP/v0grRtqyo1onc0/Tsff4DD6fEndBdj0mvhUqi9gsuXWkhnwaTxPk0lGRpMRIJPEDFosJkUEXx5/5pMVFSGk0gXyXAsBi28fo5+uzuYkXP2T9/E3B+uQfewC7VWU1IN06LhdPuwZkc3frf+YFrvI7V0TuQuxmzQYtTtU6RFsdub2qJtmVmPD1evwpdOn4IvrmxL245EIMEniBg0lJvQZ3fDk2YKX0AWPnAp2KkxUUYT6CIZjuR1Lr7nDSy55w0AQMfAGNy+ALpH3KgvUyYGfuflcwAAv3hjb1rpmVKnzEQ8fGkU4/w7X0t7fnGqHj4A1FiNuOOik4IzuNWGBJ8gYtBQZgLnSFsUfP5xIVOyU2OiSB5+MjH8cPGVP+8ecSm26HnurDrcceFMODz+tK6NI4FOmRJygV4n9vRPlVTz8LMBCT5BxKChTFhYSzesI28Q9rctxzI+PvFvW4RhG6XGxD3J08LaKI+4xhdwD/Y5JsxLTgcp1BSrX388RpMIW5XI+vi/uacnrf8PV4ohnWyQH1YSRJaQBlOnm6njkwn+PS/vwt8+zty0o02HB/DEe4cBCOX8iTKzPrTKdiQs3l1fpty8ZMkrH3WlLviJDD+RaCofz5BZs6Mba3acSOmcHl8AIy5vyiGdTEOCTxAxqBc9/HRz8X2BAOTrm0pU7ybKYdm5pDuWRLHIPOERV6jgK+nhSzHsUVfqi6hD4g9SIoJ/6rRqfGvV9ODz3SnWAew5MQrOgfYMtEVQAhJ8gohBqVGHKosBh/rS64vv83PYZG2J9drMffXkvd+THWr/0tdX4urFQqHSoCNUjCdVJPfjEQslQjofiE3YKi3xK2n1Wg2+sao9+PxEiiE7yd5GhRaw1YYEnyDiMKvRhh3HR9J6D3+AhyyYjmVw4XZAFOobV7Qm/dq2akvwdZ2Dwp2ClOEyoz56Y7Vkka5NOiEdjy8Am0mX0p3HsaHUQnZjXsFe+ZpALkOCTxBxmNVow75ue1qpmb4AR42sb7wjA8MuJFxeP8pK9PjhZak155IGpkhhj3uvnINtPzwvqRTPeEjvZU9D8F1ef9K9+T97cgsA4O19ffhXCoNYpKyidNpLZBISfIKIw5zGMnj8Aew+kbqX7/b6MaWmFD+5eh6AzHr4bp8/rV4tUihKWvgtN+vj9tVPlnKxodmAM/UhIKk0MfvRlXPx/YuEGb+3PvVR0uccF3zy8AmiIFjaWglAGPuXKm5fAEadBp9e0oy2akvCM2OVwO0LJB27lxM+ErGsRNluk4Dg4ZsN2pRj6UDq6ZGfP6U1+PhwX3KdUaUfbgrpEESBUGczwmbSYV8aA80FwRdEwWzQwpnBKVrSj006fGZJc/BxrJm46TC70YaXtnbB608tdDY8llp6pEmvxZM3LwMAfPvPHyf1WvLwCaLAYIxhep0V+7rTEXw/jKL3aTHoMlpt6/aO/9ikSqMsb73crI7gX7VoEvrs7pS8/C1HB7Ht2DA2p3gXNq9JaGT20dGhpF43Jq7FUKUtQRQQ7XVW7O0ZTaki0x/g8Pp50Ms2G7VR2w6rgdvnTyukAwh96yXU8vClYqgTKbSxeHufMAg91bGFZbIfsWT+j50eP0r0WkWayGUCEnyCSICFzeUYcnpTSs+UsnskL9tm0qe1OJnK+dMN6Zw5oyb4WK0agoYyqao5ecF3K9CQ7geXzAIQe3B7OE6vP2/COQAJPkEkxKpZddBqGP61PfkSfMmbLxFDOpOrzOgYGEs5/JAsbl8gpYlMchhj2Lj6HLxw2ykKWTURqfvm8RRy4qWQivyHKVmkH5zjSbTRGPP482bBFiDBJ4iEqLQYcHJbJV5NIVd7vFOlEDa4ZrGwALpud7dyBsbA7QvAoIBXXms1YfHkSgUsiozVpEedzYi9KbQ5kFoV/eZzi9I4v5BLn8z6itPjgyVPcvABEnyCSJiL5jbgQK8DO44PJ/U6qXpUEpSWKjPqbSb0ZGi+rUe2YJzrzGqwYWdXCmEzvx8all4BlFRBnJzgk4dPEAXJebPrAACPv3M4qYW94PARWWuFSosBA47MxPHdvgCMGezdkw6zGm3Y32NPOibvSbPWABjPpf/gYH/C/79jHorhE0RBUms14ZbTp+CFjzqxpSPx9L1gSEfWi77UpEurUVgyCDH8/Piqz2oogy/Ak06B9fp52mErycP/7ZsH8PePjyf0GicJPkEULl89exoMOg0ee/tQwl6g1PJX3jzNasyc4Ht86efhZ4pZjUJDtm3HkgubCdXE6f0b5eGgb/7p44T+f8e8/owNIFcCEnyCSAKbSY8vnNqKl7d1JZxlE2mAeCY9fJc3/Tz8TDG50oxysx6vJpkNpUTqaVVpaMuIDYcG4r7G6fHBnCfDTwASfIJImltPnwpgvP96PMIXbQGhd0w6nSETxecPwO0L5E3YQaNhOGdmHXYluXDr8Qeg16ZX/BReX3CwN35fHVq0JYgCp8JiwMx6K9aL1Z3xGHX5YNBqQsIqmfLwpSZtSrYyVpvpdaXoHXVjeCzxAiiPAtXEwPggE8YSq/ilRVuCKAIaykzYeGgAb+3tjXus3e2dMFjbatTB7Quk1WM/ERzuxOe85grTaksBAPuTaFbn9XNFBP/1b5+BLT84FzaTHr9cuw93/XNn1GM9vgB8AU6CTxCFztfOEcbjvbI1fiHWW3t7gxkgEsGBHyp7+ZLgW/JI8GfUC/NhX9mWeJGbR6HiMotRhwqLIXh38fi7h3DWA2/CF6GD53hr5Py5tiT4BJECi1oqsLK9GlvjZJN0DDjRMTA2YYSeNPBjUOWeOvY89PAnVZhx3bJm/OHdQ+hKsM2BEnn40TjU50Dn4EQ7nOJ4Q/LwCaIIWNhcjj0nRoJedCQ+FvP1v7Vqesh2aRRf36i61bb2PPTwAeCW06ciwIE1CWbruP0BRZu6PXjtgpDnuyO0e3C486sXPpCm4DPGrmGM7WCMBRhjS2TbWxljY4yxj8U/D6dvKkHkFgtbKhDgsXPGv/bsFgDAF1e2hWyvtgoefp9dXQ9/PKSTP6IEAK1VQnrm3gTj+F4F0jLlXL6gCRtXn4P/vPgkAMCtT21GIBCalx8M6RRRWuZ2AFcBWB9h3wHO+QLxz61pnocgco6ZDUKsOdokLJdXEISm8pIJHrbk4d/3r10qWgjY3fmXpQMI3Tmn1ZQmvHDr8Ssf0qm1mvDFlVOCz/+9pwe/f/sgAGDI6cGlv3oHQP4MMAfSFHzO+S7O+R6ljCGIfKLeZoLZoMXB3sii1CuGa247c+qEfRViDL9jIPlWwMkw4BBsqLAoP4dWbVqqzNh4aABbjsYvcHN5/apNnfrxVXMBADf/cRPueXkXnB4fjg44g/spD1+gjTG2hTH2FmNsZbSDGGO3MMY2McY29fbGT3EjiFyBMYa2asuEiluX14/pq1/Fyp/8G0DkkYBaDcPK9moAmBAqUJKeETdK9FpY88zDB8avyxee+DDusUNOb8jUKiWZP6k85Pn+HntIz/58iuHH/RQwxt4AUB9h12rO+d+jvKwLQAvnvJ8xthjA3xhjsznnE8rnOOePAngUAJYsWaLeJ58gVGDI6cWxoTEc7nOgtdoCQPDsPbI0vnDBkFjZXo239/VhzOtXbVG1e9SNWpsRjOXHCD45Wo3gj/rj/CB6fAHY3T5UmtW5iwlvuXDZr94NeZ5Pgh/Xw+ecr+Kcz4nwJ5rYg3Pu5pz3i483AzgAYHq04wkiX/nuBTMAAJ9/fGNw25h3vLXv+u+cheZKc8TXSvnbDhXn2/aMuFBnNan2/mry5bOEUNhssaFaNEbE5nQ2lWbtVsT5ISn6kA5jrIYxphUfTwHQDuCgGuciiGxy+YImGHUaHB1wBqtmR2U9clqqIos9gGDTrbEkBm4kS8+oGzU2o2rvryZTa0px3qw6DDpit1iQrp9ad0nxFoPzaUE83bTMKxljnQBOAfAyY2yNuOt0AFsZY58AeB7ArZzz+K3nCCIPeeCa+QCAvd1CrraU+37p/MaYr5NCAclMWEqWfPbwAaDWZkSvPXatgnSHpGZo5bMnt6DWGvmHM5+ydNKylHP+IoAXI2x/AcAL6bw3QeQLUsime8SFOU1lONwndFn87vkzYr6uJCj46oR0XF4/HB7/hBh0PmEz6THg8KBz0IlJFZHvlpzBFgfqCf6PrpyLaxZPwpW/eS+47blblqOpvES1c6oBVdoSRJpInl+PmIb55p4eVJj1ccVACkGo5eHnY+O0cKT12nteil6v4BRrDdQeJi735F/62mlYPqUq6vpMrkKCTxBpIi0WSpOtDvc7sWJqNTSa2JkxUoVmsuP8EkX6IcmnLJJwvnb2NACx/w3ODIR0gNCK2jlNZaqeSy1I8AkiTaTFV7vbj44BJw71OYKj+mIhDUS566XoLXjTwanyYmYmsBh1aKu24K9bjuG1HSfQb3fj0fUHQmoXpKwo1QU/j384JUjwCSJNNBoGs0ELp9uHv398DIwBVyxsivu6Flk4INH5uMmQicXMTNAgDiX5/TuHcMuTm/GjV3bjgFjd/OzGo/jGcx8DUH/xVPqB/uaqdlXPoyYk+AShABajDg6PD7u6RtFaZUloMU9eDOVWYRBKMLadxx4+ADx03UIAQHttKToHhZYGXcMuuLx+/OfftgePU9sDN+m12H/vhfjGOST4BFHUlBp1sLv96BoeQ2N54mmQd18xBwAwksQ4v0QpFA+/qtSIJZMr8HHHEHRi9e3nH9+Izz++MaQoKxP/Tp1Wk5dVyxIk+AShABajENIZ8waSCi2UiQu+UrWokkiLmWpnr2SCRZMrJnQl3XhoAHW28R9XJfvhFyp0hQhCAcwGYSi5y+uHKYn+6FJ8emfXxAEb6RIc0JFnvfAjMa22FB5fYMLksDiJUEQYJPgEoQClRh26R1xim97Ev1aLWypQZzPitR2JTXZKhkLy8NvFwebhuLzC2se7t5+dSXPyFhJ8glCA6XVWHO53omvYldTioUbDMLuxLOFBH8kgefj5NJEpGu111ojb39rbi/ba0ryreM0WJPgEoQCnTqsKPk4mpAMI4YqDfY64bYCTxenxwWzQxi0AywdKjTpsXH1OxH3RJo4REyHBJwgFWNZWmfJrp9UI8ekO2RQlJXB4/HmfoSOn1mrCoR9fNGH7tCjhHmIiJPgEoQBGnRaf/Nd5uGx+I25c0ZrUa6eKgqV0WMfh9hVEdaicSCmRL355RRYsyU9I8AlCIcrMevzyuoVoTDKeLHmoB6LMxk2VY4NjaLAVXmz77Jm1AIAbV7Tiz186BVaTOoNPCpH8X74niDynrESPcrMeHYPKhnQ6B8dwmjg3t5D4zecWYcDhSfqHlSAPnyByguYKMzoGxuIfmAQjLi/KVRr7l01Mei2JfYqQ4BNEDjCpoiTYJ0YJvP4AnB6/anNeifyEBJ8gcgBB8McU65opzdWVOjwSBECCTxA5waQKM9y+AH70yq6QXu+pUgjTrgjlIcEniByguVKISf/u7UN4fVd32u8ndcokwSfkkOATRA4gH9A9psCM2/HGaST4xDgk+ASRA8h7wSjRYkEK6VgKrPCKSA8SfILIAeRTqZQQ/GCnTPLwCRkk+ASRYxwZcKT1eo8vgONDLgCF0RqZUA4SfILIEV647RQAwPZjI2m9z21PbcZdL+0EUBjDTwjlIMEniBxh8eRKXL14EnYcH04rH3/t7p7gY8rSIeSQ4BNEDtFeW4o+uwd2cdE1XYxJTN8iCh/6NBBEDlFrMwIAukdcKb+HQRR5q1EXsZ0wUbyQ4BNEDtFYJqRnrpOFZZJlZr0wDnBUobsEonAgwSeIHEKanDXo9Kb8HrVWEwCgACYbEgpDgk8QOQRjDDVWIwbsnpTfIyAu+L789ZVKmUUUCCT4BJFjVFkM6HekLvhefwCLWspxUoNNQauIQoAEnyByjH09dryxqzvloeZuXwB6LX21iYnQp4IgcgyptcK2Y8Mpvd7rDwQzdQhCDn0qCCLH+POXhIrbVHPxvf4ADOThExFI61PBGLufMbabMbaVMfYiY6xctu8Oxth+xtgextj56ZtKEMVBe20pAMDuSlHwfZxCOkRE0v1UvA5gDud8HoC9AO4AAMbYLADXApgN4AIAv2GMUVMPgkiAUnEsYToevp5COkQE0vpUcM5f45xLn8oPAEwSH18O4DnOuZtzfgjAfgDL0jkXQRQLeq0GJr0Go67UcvE9/gD0WkrCJyaipBtwE4BXxcdNADpk+zrFbRNgjN3CGNvEGNvU29uroDkEkb9YTXqK4ROKE/dTwRh7gzG2PcKfy2XHrAbgA/C0tCnCW0Vs/8c5f5RzvoRzvqSmpiaVfwNBFBxWow6jYgzf5w8kNdjcQ2mZRBTi9k7lnK+KtZ8xdgOASwCcw8d7unYCaJYdNgnA8VSNJIhio9Q0LvjTVr+KBc3l+NtXTk3otV4/LdoSkUk3S+cCAN8DcBnnXF4l8g8A1zLGjIyxNgDtADamcy6CKCasJh3sbh8GxIrbjzuGEn6th/LwiSikOx3hVwCMAF4X27B+wDm/lXO+gzH2ZwA7IYR6vsI596d5LoIoGkqNOry7vxuL7n49qddxzsUYPi3aEhNJS/A559Ni7LsXwL3pvD9BFCtWkz6l1/kDHJyDQjpEROhTQRA5SIU5NcH3+oVlNMrDJyJBnwqCyEEmV1kmbNt+bDhmLN/rD2DlT9YBIA+fiAx9KggiB7l0XuOEbZc89A6u+PW7UV/TZ3ejT+yjTzF8IhIk+ASRg5SlENIZkPXQJw+fiAR9KggiR/njTcl1I/nVuv3Bx5SWSUSCPhUEkaOcMb0GzZUlCR/fbycPn4gNfSoIIod58qaTJ3jrXn8g5PnqF7fh0fUH0Gt3B7eR4BORSLfwiiAIFWmttuDb507Hfa/uDm4b8/qDgu71B/D0hqMAhGItCSOFdIgI0KeCIHKcEn3oKAmXZ7xo3SHrqCnvrmnU01ebmAh9Kggix/nM0mZ86Ywp+MElswAATo8fz208CpfXH7WFcviPBEEAFNIhiJzHpNfijgtPwivbugAAL2/rwv1r9uCTziEsnlwZ9TUEEQ4JPkHkCZLX3jU8BgB4dmMHnt3YEfFYEnwiEhTSIYg8QRJxhzt+41kTxfCJCJCHTxB5QolBEPwXtxyLeszK9mp0DbtQZzVlyiwijyDBJ4g8IZGF2NvOnIoVU6szYA2Rj9B9H0HkCYkIPuXfE7GgTwdB5AkmQ+Sva1v1eCtlo44Wa4noUEiHIPKE8hJD8PFvP7cIF85tAOfCwJO2O14BAMxqsGXFNiI/IMEniDxB3lOn1iYsyoqzpLHx++cgwAGNhvrgE9EhwSeIPKS+LDQLR/oBIIhYUAyfIPKQOqsx2yYQeQh5+ASRRzx188k4PjQGHbU/JlKABJ8g8ojT2inHnkgdchMIgiCKBBJ8giCIIoEEnyAIokggwScIgigSSPAJgiCKBBJ8giCIIoEEnyAIokggwScIgigSmNRtLxdgjPUCOJLGW1QD6FPIHDUhO5UlX+wE8sdWslN51LR1Mue8Jt5BOSX46cIY28Q5X5JtO+JBdipLvtgJ5I+tZKfy5IKtFNIhCIIoEkjwCYIgioRCE/xHs21AgpCdypIvdgL5YyvZqTxZt7WgYvgEQRBEdArNwycIgiCiUBCCzxi7gDG2hzG2nzF2e5ZtaWaM/ZsxtosxtoMx9g1xeyVj7HXG2D7x7wpxO2OM/VK0fStjbFGG7dUyxrYwxl4Sn7cxxjaIdv6JMWYQtxvF5/vF/a0ZtrOcMfY8Y2y3eG1PycVryhj7lvj/vp0x9ixjzJQr15Qx9jhjrIcxtl22LelryBi7QTx+H2PshgzZeb/4f7+VMfYiY6xctu8O0c49jLHzZdtV1YVIdsr2/T/GGGeMVYvPs3Y9Q+Cc5/UfAFoABwBMAWAA8AmAWVm0pwHAIvGxFcBeALMA/ATA7eL22wH8j/j4IgCvAmAAlgPYkGF7vw3gGQAvic//DOBa8fHDAG4TH38ZwMPi42sB/CnDdv4RwBfFxwYA5bl2TQE0ATgEoER2LW/MlWsK4HQAiwBsl21L6hoCqARwUPy7QnxckQE7zwOgEx//j8zOWeJ33gigTdQCbSZ0IZKd4vZmAGsg1BRVZ/t6htiWiS+Cyh/iUwCskT2/A8Ad2bZLZs/fAZwLYA+ABnFbA4A94uNHAFwnOz54XAZsmwRgLYCzAbwkfhj7ZF+s4LUVP8CniI914nEsQ3baRCFlYdtz6ppCEPwO8curE6/p+bl0TQG0hglpUtcQwHUAHpFtDzlOLTvD9l0J4Gnxccj3XbqmmdKFSHYCeB7AfACHMS74Wb2e0p9CCOlIXzKJTnFb1hFv0RcC2ACgjnPeBQDi37XiYdm0/xcAvgsgID6vAjDEOfdFsCVop7h/WDw+E0wB0AvgD2L46feMMQty7Jpyzo8BeADAUQBdEK7RZuTmNZVI9hrmwvftJgjeMmLYkxU7GWOXATjGOf8kbFdO2FkIgs8ibMt66hFjrBTACwC+yTkfiXVohG2q288YuwRAD+d8c4K2ZPM66yDcOv+Wc74QgANC+CEa2bqmFQAuhxBaaARgAXBhDFty8rMrEs22rNrMGFsNwAfgaWlTFHsybidjzAxgNYD/irQ7ij0ZtbMQBL8TQsxMYhKA41myBQDAGNNDEPunOed/FTd3M8YaxP0NAHrE7dmy/1QAlzHGDgN4DkJY5xcAyhlj0nB7uS1BO8X9ZQAGMmCndO5OzvkG8fnzEH4Acu2argJwiHPeyzn3AvgrgBXIzd1jTDYAAAHDSURBVGsqkew1zNr3TVzQvATA57gY/8gxO6dC+LH/RPxeTQLwEWOsPlfsLATB/xBAu5gJYYCw+PWPbBnDGGMAHgOwi3P+M9mufwCQVuBvgBDbl7Z/XlzFXw5gWLrFVhPO+R2c80mc81YI12wd5/xzAP4N4Ooodkr2Xy0enxHPjnN+AkAHY2yGuOkcADuRY9cUQihnOWPMLH4OJDtz7prKSPYargFwHmOsQryjOU/cpiqMsQsAfA/AZZxzZ5j914oZT20A2gFsRBZ0gXO+jXNeyzlvFb9XnRASOE4gV66nWosDmfwDYQV8L4RV+dVZtuU0CLdkWwF8LP65CEJsdi2AfeLfleLxDMCvRdu3AViSBZvPxHiWzhQIX5j9AP4CwChuN4nP94v7p2TYxgUANonX9W8QMhpy7poCuBPAbgDbATwJIXskJ64pgGchrC14IYjRzalcQwgx9P3iny9kyM79EGLd0nfqYdnxq0U79wC4ULZdVV2IZGfY/sMYX7TN2vWU/6FKW4IgiCKhEEI6BEEQRAKQ4BMEQRQJJPgEQRBFAgk+QRBEkUCCTxAEUSSQ4BMEQRQJJPgEQRBFAgk+QRBEkfD/ATyGJJoJE7qrAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(range(1440), temp[:1440])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 그래프에서 일별 주기성을 볼 수 있습니다. 특히 마지막 4일간을 보면 확실합니다. 이 데이터는 아주 추운 겨울 중 10일입니다.\n",
    "\n",
    "지난 몇 달간의 데이터를 사용해 다음 달의 평균 온도를 예측하는 문제는 쉬운 편입니다. 연간 데이터 주기성은 안정적이기 때문입니다. 하지만 하루 하루 데이터를 살펴보면 온도 변화는 매우 불안정합니다. 일자별 수준의 시계열 데이터를 예측할 수 있을까요? 직접 확인해 보겠습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preparing the data\n",
    "\n",
    "이 문제의 정확한 정의는 다음과 같습니다. `lookback` 타임스텝(하나의 타임스텝은 10분입니다)만큼 이전으로 돌아가서 매 `steps` 타임스텝마다 샘플링합니다. 이 데이터를 바탕으로 `delay` 타임스텝 이후의 온도를 예측할 수 있을까요? 사용할 변수는 다음과 같습니다.\n",
    "\n",
    "* `lookback = 1440` : 10일 전 데이터로 돌아갑니다.\n",
    "* `steps = 6` : 한 시간마다 데이터 포인트 하나를 샘플링합니다.\n",
    "* `delay = 144` : 24시간이 지난 데이터가 타깃이 됩니다.\n",
    "\n",
    "시작하기 전에 두 가지 작업을 처리해야 합니다:\n",
    "\n",
    "* 신경망에 주입할 수 있는 형태로 데이터를 전처리합니다. 데이터가 이미 수치형이므로 추가적인 벡터화가 필요하지 않습니다. 하지만 데이터에 있는 각 시계열 특성의 범위가 서로 다릅니다(예를 들어 온도는 일반적으로 -20도에서 +30도 사이이고, 밀리바(mb)로 측정된 기압은 1,000 근처의 값입니다). 각 시계열 특성을 개별적으로 정규화하여 비슷한 범위를 가진 작은 값으로 바꾸겠습니다.\n",
    "* float_data 배열을 받아 과거 데이터의 배치와 미래 타깃 온도를 추출하는 파이썬 제너레이터를 만듭니다. 이 데이터셋에 있는 샘플은 중복이 많습니다(샘플 `N`과 샘플 `N + 1`은 대부분 타임스텝이 비슷합니다). 모든 샘플을 각기 메모리에 적재하는 것은 낭비가 심하므로 대신에 원본 데이터를 사용해 그때 그때 배치를 만들겠습니다.\n",
    "\n",
    "각 시계열 특성에 대해 평균을 빼고 표준 편차로 나누어 전처리합니다. 처음 200,000개 타임스텝을 훈련 데이터로 사용할 것이므로 전체 데이터에서 200,000개만 사용하여 평균과 표준 편차를 계산합니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "mean = float_data[:200000].mean(axis=0)\n",
    "float_data -= mean\n",
    "std = float_data[:200000].std(axis=0)\n",
    "float_data /= std"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "다음은 여기서 사용할 제너레이터입니다. 이 제너레이터 함수는 `(samples, targets)` 튜플을 반복적으로 반환합니다. `samples`는 입력 데이터로 사용할 배치이고 `targets`은 이에 대응되는 타깃 온도의 배열입니다. 이 제너레이터 함수는 다음과 같은 매개변수가 있습니다:\n",
    "\n",
    "* `data` : 코드 6-32에서 정규화한 부동 소수 데이터로 이루어진 원본 배열\n",
    "* `lookback` : 입력으로 사용하기 위해 거슬러 올라갈 타임스텝\n",
    "* `delay` : 타깃으로 사용할 미래의 타임스텝\n",
    "* `min_index`와 `max_index` : 추출할 타임스텝의 범위를 지정하기 위한 `data` 배열의 인덱스. 검증 데이터와 테스트 데이터를 분리하는 데 사용합니다.\n",
    "* `shuffle` : 샘플을 섞을지 시간 순서대로 추출할지 결정합니다.\n",
    "* `batch_size` : 배치의 샘플 수\n",
    "* `step` : 데이터를 샘플링할 타임스텝 간격. 한 시간에 하나의 데이터 포인트를 추출하기 위해 6으로 지정하겠습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generator(data, lookback, delay, min_index, max_index,\n",
    "              shuffle=False, batch_size=128, step=6):\n",
    "    if max_index is None:\n",
    "        max_index = len(data) - delay - 1\n",
    "    i = min_index + lookback\n",
    "    while 1:\n",
    "        if shuffle:\n",
    "            rows = np.random.randint(\n",
    "                min_index + lookback, max_index, size=batch_size)\n",
    "        else:\n",
    "            if i + batch_size >= max_index:\n",
    "                i = min_index + lookback\n",
    "            rows = np.arange(i, min(i + batch_size, max_index))\n",
    "            i += len(rows)\n",
    "\n",
    "        samples = np.zeros((len(rows),\n",
    "                           lookback // step,\n",
    "                           data.shape[-1]))\n",
    "        targets = np.zeros((len(rows),))\n",
    "        for j, row in enumerate(rows):\n",
    "            indices = range(rows[j] - lookback, rows[j], step)\n",
    "            samples[j] = data[indices]\n",
    "            targets[j] = data[rows[j] + delay][1]\n",
    "        yield samples, targets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이제 generator 함수를 사용해 훈련용, 검증용, 테스트용으로 세 개의 제너레이터를 만들어 보죠. 각 제너레이터는 원본 데이터에서 다른 시간대를 사용합니다. 훈련 제너레이터는 처음 200,000개 타임스텝을 사용하고 검증 제너레이터는 그 다음 100,000개를 사용하고 테스트 제너레이터는 나머지를 사용합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "lookback = 1440\n",
    "step = 6\n",
    "delay = 144\n",
    "batch_size = 128\n",
    "\n",
    "train_gen = generator(float_data,\n",
    "                      lookback=lookback,\n",
    "                      delay=delay,\n",
    "                      min_index=0,\n",
    "                      max_index=200000,\n",
    "                      shuffle=True,\n",
    "                      step=step, \n",
    "                      batch_size=batch_size)\n",
    "val_gen = generator(float_data,\n",
    "                    lookback=lookback,\n",
    "                    delay=delay,\n",
    "                    min_index=200001,\n",
    "                    max_index=300000,\n",
    "                    step=step,\n",
    "                    batch_size=batch_size)\n",
    "test_gen = generator(float_data,\n",
    "                     lookback=lookback,\n",
    "                     delay=delay,\n",
    "                     min_index=300001,\n",
    "                     max_index=None,\n",
    "                     step=step,\n",
    "                     batch_size=batch_size)\n",
    "\n",
    "# 전체 검증 세트를 순회하기 위해 val_gen에서 추출할 횟수\n",
    "val_steps = (300000 - 200001 - lookback) // batch_size\n",
    "\n",
    "# 전체 테스트 세트를 순회하기 위해 test_gen에서 추출할 횟수\n",
    "test_steps = (len(float_data) - 300001 - lookback) // batch_size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 상식 수준의 기준점\n",
    "\n",
    "블랙 박스 같은 딥러닝 모델을 사용해 온도 예측 문제를 풀기 전에 간단한 상식 수준의 해법을 시도해 보겠습니다. 이는 정상 여부 확인을 위한 용도이고 고수준 머신 러닝 모델이라면 뛰어 넘어야 할 기준점을 만듭니다. 이런 상식 수준의 해법은 알려진 해결책이 없는 새로운 문제를 다루어야 할 때 유용합니다. 일부 클래스가 월등히 많아 불균형한 분류 문제가 고전적인 예입니다. 데이터셋에 클래스 A의 샘플이 90%, 클래스 B의 샘플이 10%가 있다면, 이 분류 문제에 대한 상식 수준의 접근법은 새로운 샘플을 항상 클래스 'A'라고 예측하는 것입니다. 이 분류기는 전반적으로 90%의 정확도를 낼 것입니다. 머신 러닝 기반의 모델이라면 90% 이상을 달성해야 유용하다고 볼 수 있습니다. 이따금 이런 기본적인 기준점을 넘어서기가 아주 어려운 경우가 있습니다.\n",
    "\n",
    "이 경우 온도 시계열 데이터는 연속성이 있고 일자별로 주기성을 가진다고 가정할 수 있습니다(오늘 온도는 내일 온도와 비슷할 가능성이 높습니다). 그렇기 때문에 상식 수준의 해결책은 지금으로부터 24시간 후의 온도는 지금과 동일하다고 예측하는 것입니다. 이 방법을 평균 절댓값 오차(MAE)로 평가해 보겠습니다:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "np.mean(np.abs(preds - targets))\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "다음은 평가를 위한 반복 루프입니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.2897359729905486\n"
     ]
    }
   ],
   "source": [
    "def evaluate_naive_method():\n",
    "    batch_maes = []\n",
    "    for step in range(val_steps):\n",
    "        samples, targets = next(val_gen)\n",
    "        preds = samples[:, -1, 1]\n",
    "        mae = np.mean(np.abs(preds - targets))\n",
    "        batch_maes.append(mae)\n",
    "    print(np.mean(batch_maes))\n",
    "    \n",
    "evaluate_naive_method()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.5672247338393395"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "0.29 * std[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "출력된 MAE는 0.29입니다. 이 온도 데이터는 평균이 0이고 표준 편차가 1이므로 결괏값이 바로 와 닿지는 않습니다. 평균 절댓값 오차 0.29에 표준 편차를 곱하면 섭씨 2.57°C가 됩니다. 평균 절댓값 오차가 상당히 크네요. 이제 딥러닝 모델이 더 나은지 시도해 봅시다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 기본적인 머신 러닝 방법\n",
    "\n",
    "머신 러닝 모델을 시도하기 전에 상식 수준의 기준점을 세워 놓았습니다. 비슷하게 RNN처럼 복잡하고 연산 비용이 많이 드는 모델을 시도하기 전에 간단하고 손쉽게 만들 수 있는 머신 러닝 모델(예를 들면 소규모의 완전 연결 네트워크)을 먼저 만드는 것이 좋습니다. 이를 바탕으로 더 복잡한 방법을 도입하는 근거가 마련되고 실제적인 이득도 얻게 될 것입니다.\n",
    "\n",
    "다음 코드는 데이터를 펼쳐서 두 개의 `Dense` 층을 통과시키는 완전 연결 네트워크를 보여줍니다. 전형적인 회귀 문제이므로 마지막 `Dense` 층에 활성화 함수를 두지 않았습니다. 손실 함수는 MAE입니다. 상식 수준의 방법에서 사용한 것과 동일한 데이터와 지표를 사용했으므로 결과를 바로 비교해볼 수 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "500/500 [==============================] - 7s 14ms/step - loss: 1.5177 - val_loss: 0.7086\n",
      "Epoch 2/20\n",
      "500/500 [==============================] - 6s 13ms/step - loss: 0.4895 - val_loss: 0.4523\n",
      "Epoch 3/20\n",
      "500/500 [==============================] - 6s 13ms/step - loss: 0.3034 - val_loss: 0.3101\n",
      "Epoch 4/20\n",
      "500/500 [==============================] - 6s 13ms/step - loss: 0.2717 - val_loss: 0.3208\n",
      "Epoch 5/20\n",
      "500/500 [==============================] - 6s 13ms/step - loss: 0.2583 - val_loss: 0.3127\n",
      "Epoch 6/20\n",
      "500/500 [==============================] - 6s 13ms/step - loss: 0.2476 - val_loss: 0.3118\n",
      "Epoch 7/20\n",
      "500/500 [==============================] - 6s 13ms/step - loss: 0.2404 - val_loss: 0.3217\n",
      "Epoch 8/20\n",
      "500/500 [==============================] - 6s 13ms/step - loss: 0.2357 - val_loss: 0.3160\n",
      "Epoch 9/20\n",
      "500/500 [==============================] - 6s 13ms/step - loss: 0.2285 - val_loss: 0.3320\n",
      "Epoch 10/20\n",
      "500/500 [==============================] - 6s 13ms/step - loss: 0.2263 - val_loss: 0.3452\n",
      "Epoch 11/20\n",
      "500/500 [==============================] - 6s 13ms/step - loss: 0.2216 - val_loss: 0.3388\n",
      "Epoch 12/20\n",
      "500/500 [==============================] - 6s 13ms/step - loss: 0.2189 - val_loss: 0.3429\n",
      "Epoch 13/20\n",
      "500/500 [==============================] - 6s 13ms/step - loss: 0.2148 - val_loss: 0.3640\n",
      "Epoch 14/20\n",
      "500/500 [==============================] - 6s 13ms/step - loss: 0.2122 - val_loss: 0.3490\n",
      "Epoch 15/20\n",
      "500/500 [==============================] - 6s 13ms/step - loss: 0.2091 - val_loss: 0.3400\n",
      "Epoch 16/20\n",
      "500/500 [==============================] - 6s 13ms/step - loss: 0.2069 - val_loss: 0.3536\n",
      "Epoch 17/20\n",
      "500/500 [==============================] - 6s 13ms/step - loss: 0.2039 - val_loss: 0.3348\n",
      "Epoch 18/20\n",
      "500/500 [==============================] - 6s 13ms/step - loss: 0.2040 - val_loss: 0.3511\n",
      "Epoch 19/20\n",
      "500/500 [==============================] - 6s 13ms/step - loss: 0.2008 - val_loss: 0.3419\n",
      "Epoch 20/20\n",
      "500/500 [==============================] - 6s 13ms/step - loss: 0.1991 - val_loss: 0.3359\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.Flatten(input_shape=(lookback // step, float_data.shape[-1])))\n",
    "model.add(layers.Dense(32, activation='relu'))\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=20,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "훈련 손실과 검증 손실의 그래프를 그려 보죠:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3XmYFNW9//H3FxhBYFgEVARhQI3KMsA4MXhdAPXnD01crwsI7ooaNYvJ75GrRo2JT9yiXJQYSeJyw0Tkmrhcg+Fm4YZorsuwg4iADjqCbMomIM7w/f1xumeaoWe6h+npnqn5vJ6nn+6qOlV1uqbnU6dPVVeZuyMiItHSKtcVEBGRzFO4i4hEkMJdRCSCFO4iIhGkcBcRiSCFu4hIBCncJSkza21m282sTybL5pKZHWlmGT/318xON7OyhOHlZnZyOmX3Y12/NrPb93f+Opb7UzN7JtPLldxpk+sKSGaY2faEwfbAl0BlbPh6dy+pz/LcvRLomOmyLYG7H52J5ZjZtcB4dx+ZsOxrM7FsiT6Fe0S4e1W4xlqG17r7X2orb2Zt3L0iG3UTkexTt0wLEfva/byZPWdm24DxZnaCmb1pZpvNbK2ZTTazvFj5NmbmZlYQG54Wm/6amW0zs/81s371LRubfqaZvW9mW8zsMTN7w8yurKXe6dTxejNbaWafm9nkhHlbm9mjZrbJzFYBo+vYPnea2fQa46aY2SOx19ea2bLY+1kVa1XXtqxyMxsZe93ezH4bq9tS4Lgk6/0gttylZnZObPxg4HHg5FiX18aEbXtPwvw3xN77JjN7ycx6prNtUjGz82L12WxmfzOzoxOm3W5ma8xsq5m9l/Beh5vZvNj4dWb2ULrrk0bg7npE7AGUAafXGPdTYDdwNmGnfiDwdeAbhG9w/YH3gZtj5dsADhTEhqcBG4FiIA94Hpi2H2UPBrYB58am3Qp8BVxZy3tJp44vA52BAuCz+HsHbgaWAr2BbsCc8JFPup7+wHagQ8Ky1wPFseGzY2UMOBXYCRTGpp0OlCUsqxwYGXv9MPA/QFegL/BujbIXAz1jf5NLY3U4JDbtWuB/atRzGnBP7PUZsToOBdoBvwD+ls62SfL+fwo8E3t9bKwep8b+RrfHtnseMBBYDRwaK9sP6B97/Q4wNvY6H/hGrv8XWvJDLfeW5XV3/y933+PuO939HXd/y90r3P0DYCowoo75X3D3Unf/CighhEp9y34LWODuL8emPUrYESSVZh1/5u5b3L2MEKTxdV0MPOru5e6+Cbi/jvV8ACwh7HQA/g+w2d1LY9P/y90/8OBvwF+BpAdNa7gY+Km7f+7uqwmt8cT1znD3tbG/ye8IO+biNJYLMA74tbsvcPddwERghJn1TihT27apyxjgFXf/W+xvdD/QibCTrSDsSAbGuvY+jG07CDvpo8ysm7tvc/e30nwf0ggU7i3Lx4kDZnaMmf3RzD41s63AvUD3Oub/NOH1Duo+iFpb2cMS6+HuTmjpJpVmHdNaF6HFWZffAWNjry8l7JTi9fiWmb1lZp+Z2WZCq7mubRXXs646mNmVZrYw1v2xGTgmzeVCeH9Vy3P3rcDnQK+EMvX5m9W23D2Ev1Evd18O/IDwd1gf6+Y7NFb0KmAAsNzM3jazs9J8H9IIFO4tS83TAJ8ktFaPdPdOwF2EbofGtJbQTQKAmRl7h1FNDanjWuDwhOFUp2o+D5wea/meSwh7zOxA4AXgZ4Quky7Af6dZj09rq4OZ9QeeAG4EusWW+17CclOdtrmG0NUTX14+ofvnkzTqVZ/ltiL8zT4BcPdp7n4ioUumNWG74O7L3X0Moevt58DvzaxdA+si+0nh3rLlA1uAL8zsWOD6LKzzVaDIzM42szbAd4EejVTHGcD3zKyXmXUDbqursLuvA14HngaWu/uK2KS2wAHABqDSzL4FnFaPOtxuZl0s/A7g5oRpHQkBvoGwn7uW0HKPWwf0jh9ATuI54BozKzSztoSQ/Ye71/pNqB51PsfMRsbW/f8Ix0neMrNjzWxUbH07Y49Kwhu4zMy6x1r6W2LvbU8D6yL7SeHesv0AuILwj/skoeXaqGIBegnwCLAJOAKYTzgvP9N1fILQN76YcLDvhTTm+R3hAOnvEuq8Gfg+8CLhoOSFhJ1UOu4mfIMoA14D/iNhuYuAycDbsTLHAIn91H8GVgDrzCyxeyU+/58I3SMvxubvQ+iHbxB3X0rY5k8QdjyjgXNi/e9tgQcJx0k+JXxTuDM261nAMgtnYz0MXOLuuxtaH9k/Fro8RXLDzFoTugEudPd/5Lo+IlGhlrtknZmNNrPOsa/2PyKcgfF2jqslEikKd8mFk4APCF/tRwPnuXtt3TIish/ULSMiEkFquYuIRFDOLhzWvXt3LygoyNXqRUSapblz525097pOHwZyGO4FBQWUlpbmavUiIs2SmaX6pTWgbhkRkUhSuIuIRJDCXUQkgnQnJpEW4quvvqK8vJxdu3bluiqShnbt2tG7d2/y8mq7tFDdFO4iLUR5eTn5+fkUFBQQLsYpTZW7s2nTJsrLy+nXr1/qGZJoVt0yJSVQUACtWoXnknrd8lmkZdu1axfdunVTsDcDZka3bt0a9C2r2bTcS0pgwgTYsSMMr14dhgHGNfg6eCItg4K9+Wjo36rZtNzvuKM62ON27AjjRURkb80m3D/6qH7jRaRp2bRpE0OHDmXo0KEceuih9OrVq2p49+70Lvt+1VVXsXz58jrLTJkyhZIM9dmedNJJLFiwICPLyrZm0y3Tp0/oikk2XkQyr6QkfDP+6KPwf3bffQ3rAu3WrVtVUN5zzz107NiRH/7wh3uVcXfcnVatkrc7n3766ZTruemmm/a/khHSbFru990H7dvvPa59+zBeRDIrfoxr9Wpwrz7G1RgnMaxcuZJBgwZxww03UFRUxNq1a5kwYQLFxcUMHDiQe++9t6psvCVdUVFBly5dmDhxIkOGDOGEE05g/fr1ANx5551MmjSpqvzEiRM5/vjjOfroo/nnP/8JwBdffMG//uu/MmTIEMaOHUtxcXHKFvq0adMYPHgwgwYN4vbbbwegoqKCyy67rGr85MmTAXj00UcZMGAAQ4YMYfz48RnfZuloNuE+bhxMnQp9+4JZeJ46VQdTRRpDto9xvfvuu1xzzTXMnz+fXr16cf/991NaWsrChQv585//zLvvvrvPPFu2bGHEiBEsXLiQE044gaeeeirpst2dt99+m4ceeqhqR/HYY49x6KGHsnDhQiZOnMj8+fPrrF95eTl33nkns2fPZv78+bzxxhu8+uqrzJ07l40bN7J48WKWLFnC5ZdfDsCDDz7IggULWLhwIY8//ngDt87+aTbhDiHIy8pgz57wrGAXaRzZPsZ1xBFH8PWvf71q+LnnnqOoqIiioiKWLVuWNNwPPPBAzjzzTACOO+44ysrKki77ggsu2KfM66+/zpgxYwAYMmQIAwcOrLN+b731Fqeeeirdu3cnLy+PSy+9lDlz5nDkkUeyfPlyvvvd7zJr1iw6d+4MwMCBAxk/fjwlJSX7/SOkhmpW4S4i2VHbsazGOsbVoUOHqtcrVqzg3//93/nb3/7GokWLGD16dNLzvQ844ICq161bt6aioiLpstu2bbtPmfrepKi28t26dWPRokWcdNJJTJ48meuvvx6AWbNmccMNN/D2229TXFxMZWVlvdaXCQp3EdlHLo9xbd26lfz8fDp16sTatWuZNWtWxtdx0kknMWPGDAAWL16c9JtBouHDhzN79mw2bdpERUUF06dPZ8SIEWzYsAF356KLLuLHP/4x8+bNo7KykvLyck499VQeeughNmzYwI6afVxZ0GzOlhGR7Il3eWbybJl0FRUVMWDAAAYNGkT//v058cQTM76OW265hcsvv5zCwkKKiooYNGhQVZdKMr179+bee+9l5MiRuDtnn3023/zmN5k3bx7XXHMN7o6Z8cADD1BRUcGll17Ktm3b2LNnD7fddhv5+fkZfw+p5OweqsXFxa6bdYhkz7Jlyzj22GNzXY0moaKigoqKCtq1a8eKFSs444wzWLFiBW3aNK32brK/mZnNdffiVPM2rXciIpIF27dv57TTTqOiogJ358knn2xywd5Q0Xo3IiJp6NKlC3Pnzs11NRqVDqiKiESQwl1EJIIU7iIiEaRwFxGJoJThbmZPmdl6M1uSotzXzazSzC7MXPVEJCpGjhy5zw+SJk2axLe//e065+vYsSMAa9as4cILk8fLyJEjSXVq9aRJk/b6MdFZZ53F5s2b06l6ne655x4efvjhBi8n09JpuT8DjK6rgJm1Bh4AMv9TMhGJhLFjxzJ9+vS9xk2fPp2xY8emNf9hhx3GCy+8sN/rrxnuM2fOpEuXLvu9vKYuZbi7+xzgsxTFbgF+D6zPRKVEJHouvPBCXn31Vb788ksAysrKWLNmDSeddFLVeedFRUUMHjyYl19+eZ/5y8rKGDRoEAA7d+5kzJgxFBYWcskll7Bz586qcjfeeGPV5YLvvvtuACZPnsyaNWsYNWoUo0aNAqCgoICNGzcC8MgjjzBo0CAGDRpUdbngsrIyjj32WK677joGDhzIGWecsdd6klmwYAHDhw+nsLCQ888/n88//7xq/QMGDKCwsLDqgmV///vfq25WMmzYMLZt27bf2zaZBp/nbma9gPOBU4Gvpyg7AZgA0Ed32RDJme99DzJ9g6GhQyGWi0l169aN448/nj/96U+ce+65TJ8+nUsuuQQzo127drz44ot06tSJjRs3Mnz4cM4555xa7yP6xBNP0L59exYtWsSiRYsoKiqqmnbfffdx0EEHUVlZyWmnncaiRYv4zne+wyOPPMLs2bPp3r37XsuaO3cuTz/9NG+99Rbuzje+8Q1GjBhB165dWbFiBc899xy/+tWvuPjii/n9739f5/XZL7/8ch577DFGjBjBXXfdxY9//GMmTZrE/fffz4cffkjbtm2ruoIefvhhpkyZwoknnsj27dtp165dPbZ2apk4oDoJuM3dU172zN2nunuxuxf36NEjA6sWkeYksWsmsUvG3bn99tspLCzk9NNP55NPPmHdunW1LmfOnDlVIVtYWEhhYWHVtBkzZlBUVMSwYcNYunRpyouCvf7665x//vl06NCBjh07csEFF/CPf/wDgH79+jF06FCg7ssKQ7i+/ObNmxkxYgQAV1xxBXPmzKmq47hx45g2bVrVL2FPPPFEbr31ViZPnszmzZsz/gvZTCytGJge28N2B84yswp3fykDyxaRRlBXC7sxnXfeedx6663MmzePnTt3VrW4S0pK2LBhA3PnziUvL4+CgoKkl/lNlKxV/+GHH/Lwww/zzjvv0LVrV6688sqUy6nr+lrxywVDuGRwqm6Z2vzxj39kzpw5vPLKK/zkJz9h6dKlTJw4kW9+85vMnDmT4cOH85e//IVjjjlmv5afTINb7u7ez90L3L0AeAH4toJdRJLp2LEjI0eO5Oqrr97rQOqWLVs4+OCDycvLY/bs2axOdsPkBKecckrVTbCXLFnCokWLgHC54A4dOtC5c2fWrVvHa6+9VjVPfn5+0n7tU045hZdeeokdO3bwxRdf8OKLL3LyySfX+7117tyZrl27VrX6f/vb3zJixAj27NnDxx9/zKhRo3jwwQfZvHkz27dvZ9WqVQwePJjbbruN4uJi3nvvvXqvsy4pW+5m9hwwEuhuZuXA3UAegLv/MqO1EZHIGzt2LBdccMFeZ86MGzeOs88+m+LiYoYOHZqyBXvjjTdy1VVXUVhYyNChQzn++OOBcFelYcOGMXDgwH0uFzxhwgTOPPNMevbsyezZs6vGFxUVceWVV1Yt49prr2XYsGF1dsHU5tlnn+WGG25gx44d9O/fn6effprKykrGjx/Pli1bcHe+//3v06VLF370ox8xe/ZsWrduzYABA6ruKpUpuuSvSAuhS/42Pw255K9+oSoiEkEKdxGRCFK4i7QgueqGlfpr6N9K4S7SQrRr145NmzYp4JsBd2fTpk0N+mGT7sQk0kL07t2b8vJyNmzYkOuqSBratWtH796993t+hbtIC5GXl0e/fv1yXQ3JEnXLiIhEkMJdRCSCFO4iIhGkcBcRiSCFu4hIBCncRUQiSOEuIhJBCncRkQhSuIuIRJDCXUQkghTuIiIRpHAXEYkghbuISAQp3EVEIkjhLiISQSnD3cyeMrP1ZraklunjzGxR7PFPMxuS+WqKiEh9pNNyfwYYXcf0D4ER7l4I/ASYmoF6iYhIA6S8E5O7zzGzgjqm/zNh8E1g/+8LJSIiGZHpPvdrgNdqm2hmE8ys1MxKdR9HEZHGk7FwN7NRhHC/rbYy7j7V3YvdvbhHjx6ZWrWIiNSQkRtkm1kh8GvgTHfflIlliojI/mtwy93M+gB/AC5z9/cbXiUREWmolC13M3sOGAl0N7Ny4G4gD8DdfwncBXQDfmFmABXuXtxYFRYRkdTSOVtmbIrp1wLXZqxGIiLSYPqFqohIBCncRUQiSOEuIhJBCncRkQhSuIuIRJDCXUQkghTuIiIRpHAXEYkghbuISAQp3EVEIkjhLiISQQp3EZEIUriLiESQwl1EJIIU7iIiEaRwFxGJIIW7iEgEKdxFRCJI4S4iEkEKdxGRCFK4i4hEUMpwN7OnzGy9mS2pZbqZ2WQzW2lmi8ysKPPVFBGR+kin5f4MMLqO6WcCR8UeE4AnGl4tERFpiJTh7u5zgM/qKHIu8B8evAl0MbOemaqgiIjUXyb63HsBHycMl8fG7cPMJphZqZmVbtiwIQOrFhGRZDIR7pZknCcr6O5T3b3Y3Yt79OiRgVWLiEgymQj3cuDwhOHewJoMLFdERPZTJsL9FeDy2Fkzw4Et7r42A8sVEZH91CZVATN7DhgJdDezcuBuIA/A3X8JzATOAlYCO4CrGquyIiKSnpTh7u5jU0x34KaM1UhERBpMv1AVEYkghbuISAQp3EVEIkjhLiISQQp3EZEIUriLiESQwl1EJIIU7iIiEaRwFxGJIIW7iEgEKdxFRCJI4S4iEkEKdxGRCFK4i4hEkMJdRCSCFO4iIhGkcBcRiSCFu4hIBCncRUQiSOEuIhJBCncRkQhKK9zNbLSZLTezlWY2Mcn0PmY228zmm9kiMzsr81UVEZF0pQx3M2sNTAHOBAYAY81sQI1idwIz3H0YMAb4RaYrKiIi6Uun5X48sNLdP3D33cB04NwaZRzoFHvdGViTuSqKiEh9pRPuvYCPE4bLY+MS3QOMN7NyYCZwS7IFmdkEMys1s9INGzbsR3VFRCQd6YS7JRnnNYbHAs+4e2/gLOC3ZrbPst19qrsXu3txjx496l9bERFJSzrhXg4cnjDcm327Xa4BZgC4+/8C7YDumaigiIjUXzrh/g5wlJn1M7MDCAdMX6lR5iPgNAAzO5YQ7up3ERHJkZTh7u4VwM3ALGAZ4ayYpWZ2r5mdEyv2A+A6M1sIPAdc6e41u25ERCRL2qRTyN1nEg6UJo67K+H1u8CJma2aiIjsL/1CVUQkghTuIiIRpHAXEYkghbuISAQp3EVEIkjhLiISQc0u3Csr4a9/zXUtRESatmYX7s88A6efDm+/neuaiIg0Xc0u3C+6CDp2hClTcl0TEZGmq9mFe6dOcPnl8PzzsHFjrmsjItI0NbtwB/j2t+HLL+Gpp3JdExGRpqlZhvvAgTBiBDzxRDjAKiIie2uW4Q5w001QVgavvZbrmoiIND3NNtzPOw8OOwx+oVtxi4jso9mGe14eTJgAf/oTrFqV69qIiDQtzTbcAa67Dlq3Dn3vIiJSrVmH+2GHwfnnh7NmduzIdW1ERJqOZh3uEA6sfv55OO9dRESCZh/up5wSTo2cMgV011YRkaDZh7tZ+FHT3Lm63oyISFyzD3eAyy6D/HydFikiEpdWuJvZaDNbbmYrzWxiLWUuNrN3zWypmf0us9WsW36+rjcjIpIoZbibWWtgCnAmMAAYa2YDapQ5Cvg34ER3Hwh8rxHqWqcbbwzXm/nNb7K9ZhGRpiedlvvxwEp3/8DddwPTgXNrlLkOmOLunwO4+/rMVjO1gQNh5Ehdb0ZEBNIL917AxwnD5bFxib4GfM3M3jCzN81sdLIFmdkEMys1s9INGzbsX43rcNNNsHq1rjcjIpJOuFuScTVPOmwDHAWMBMYCvzazLvvM5D7V3YvdvbhHjx71rWtK554bftikG3mISEuXTriXA4cnDPcG1iQp87K7f+XuHwLLCWGfVYnXm1m5MttrFxFpOtIJ93eAo8ysn5kdAIwBXqlR5iVgFICZdSd003yQyYqma8IEaNNG15sRkZYtZbi7ewVwMzALWAbMcPelZnavmZ0TKzYL2GRm7wKzgf/n7psaq9J16dkTLrgAnn5a15sRkZbLPEe/2S8uLvbS0tJGWfacOeFOTb/5DVx9daOsQkQkJ8xsrrsXpyoXiV+o1nTyybrejIi0bJEMd7NwWuS8efDWW7mujYhI9kUy3AHGj9f1ZkSk5YpsuCdeb6YRfi8lItKkRTbcIVwKePduXW9GRFqeSIf7gAEwahT88pe63oyItCyRDneovt7MzJm5romISPZEPtzPOaf6ejMlJVBQAK1aheeSklzXTkSkcbTJdQUaW14eXH893H03/P3vsGtXGL96dbhUAcC4cbmrn4hIY4h8yx3guuvCczzY43bsgDvuyH59REQaW4sI9549a5/20UfZq4eISLa0iHAHOOSQ5OP79MluPUREsqHFhPvDD4fLEiRq3x7uuy839RERaUwtJtzHj4crr6we7tsXpk7VwVQRiaYWE+4AkyeHyxKMHw9lZQp2EYmuFhXuHTvCFVfAjBnw8cepy4uINFctKtwBvv/9cO77pZdCRUWuayMi0jhaXLj37w9PPgmvvw533ZXr2oiINI4WF+4Q+tqvvRZ+9jN47bVc10ZEJPNaZLhDOLg6eDBcdhmUl+e6NiIimdViw/3AA+E//xO+/BLGjIGvvsp1jUREMietcDez0Wa23MxWmtnEOspdaGZuZinvzN0UHH10ONf9jTfgRz/KdW1ERDInZbibWWtgCnAmMAAYa2YDkpTLB74DNKtbUo8dG64O+cAD8Mc/5ro2IiKZkU7L/Xhgpbt/4O67genAuUnK/QR4ENiVZFqTNmkSDBkS7rmq899FJArSCfdeQGLklcfGVTGzYcDh7v5qXQsyswlmVmpmpRua0F2rDzww/LBp9271v4tINKQT7pZknFdNNGsFPAr8INWC3H2quxe7e3GPHj3Sr2UWfO1r8KtfwT//qWu8i0jzl064lwOHJwz3BtYkDOcDg4D/MbMyYDjwSnM5qJpozBi44QZ46CF4tc7vICIiTVs64f4OcJSZ9TOzA4AxwCvxie6+xd27u3uBuxcAbwLnuHtpo9S4kT36KAwdGq5Boxt5iEhzlfIequ5eYWY3A7OA1sBT7r7UzO4FSt39lbqX0Ly0axf63487Di65BObMCdeiEcmlPXtg5UqYOxeWLIFu3eCoo+DII6Ffv/C5FUlk7p66VCMoLi720tKm27ifMSOE+w9/GLppRLKlshJWrAhBHn/Mnw/btoXprVqFsI8zg8MPD0Eff8SDv3//cFMaiQ4zm+vuKbu9U7bcW6qLL4a//z3cwemUU+Dss3NdI4miykpYvnzvIF+wALZvD9PbtQvdhJddFr5NHnccDBgQgn7lyn0ff/gDbNy49zp69do7+I88MtxX+OCDoUcP6Nx537uUNYR7qENZGXz44b7PH38c1j948N6PI4+ENo2QSJWVsHp12M7vvRe2U9u24f0ne0RlZ6iWex127YJ/+ZfwoZw/P9y9SZqPykpYuzb8Y5eVVT9/8kn45+7SJQRb586pX6fTNecePjM7d8KOHdWPmsNbt8LixdVBvmNHmL99+xDkxx0HRUXh+dhj6x94mzfDqlXJw//TT/ctn5cXQr5Hj+rAjz8nG9epU1hHbeFdVgZffLH3Og46CAoKQhdS797hb7B4cfiGEv8W0rZt2HHVDP2ePdPb+WzeHAI8/njvvfC8cmW4zEhc587hsxHfgdbUoUPtwR9/dO0atkPnzuG5bdvM7iDrkm7LXeGewqpV4R/t2GND//sBB+S6RhJXURFComZ4x19/9NG+v1k4+OAQLrt3hzDYsqW6u6MuBx5YHfSdOoXl1gzvnTtDwKejQwcYNmzvID/mGGjdup4boZ62bYMPPoB162D9+vDYsCH5c23h17p1CMdE+fkhuPv1qw7x+HPfvmG7JbNzJyxbFoI+8bF2bXWZbt1g0KDqsB84ED7/fN8QX7++ep42beCII8IlRmo+uncPQbxjR/V7Xb9+722S7FHzPSfKy6v+bCSGfm2vCwvDY38o3DPohRfgoovg1lvh5z/PdW1alt27Q2vw/ffDY/ny0NorKwtf72v+w/XsGcKkoCA8El/36ZP8K3dlZWhNb9lSHfh1vd66Nezk27cPod++/b6v6xru0CHsYBo7yBtq584QfjVDf9OmEJCJId61a2Zbrhs3hgPHixfDokXhecmSfb8R9OiRPMD798/siRB79oQdSjzoN2+u/sxs3Vr9SByu+Trx5kATJ4ZLju8PhXuG3XILPP44vPQSnJvs4gtpcg8f0Pbtw4Gx5uTLL8MHe+PGEFT5+aEV0qFDw96LO6xZE4I7HuLxIP/ww70DvHv3cLCwf/99Q7xPH501EmV79oSd+rJloZvn6KPDc3MQ77KLB32nTnDoofu3LIV7hn35Zfja/NFH4UPWty/cd1/1TbbdQ4tmzZrwlbK257VrQ2u0VavwwezePXztjD8Sh2u+PuigzB9w+uqr0CL79NPwtTTxuea4zz9PvgyzcH/aTp2qA7+253j/ZPwA1/vvh5Z4YovswANDgB99dPjlcOKjufwzizQWhXuGlZSEuzftSrgsWuvWodX41VchtJNdk6ZLFzjssNBdEH/u1i30fW7cGHYImzZVv964ce+DPzV17lwd8q1a1f1o3Tr5+D17QqCvW7fvmRVx+fmhZXHIIXs/H3poqP+uXeE9bN1a/Zz4Otm4xBZ4q1bhK31icMfDvFev5vetRiRbdCpkht1xx97BDiGsPvkknDaZGN41XVNdAAAH+ElEQVTx5549Qyu0PtzDgZ7EwK/5+rPPwrr37Kn/I97vd9RRcPLJycP7kEMyfzpY4tfSHTvCNmrbNrPrEJFqCvc01XYpgi+/hGefzdx6zEIfdocOoQ85KszCjq6+OzsR2T/68pum2oI2SgEsItGhcE/Tffft21XRvn0Yn66SktBH36pVeC4pyWQNRUSqKdzTNG5cuN9q376hi6Fv3zAcP1smlZKScDu/1atD//Pq1WFYAS8ijUHhXg/jxoXzbOPn26Yb7BAOyMZ/Zh63Y0f9bgyilr+IpEsHVLOktgOy6V4zPt7yj+8g4i1/qN9ORkRaBrXcs6ShB2Qz0fIHtf5FWgqFe5Y09IBsQ1v+kJl+/4buHLRzEckSd8/J47jjjvOWZto097593c3C87Rp6c/bt697iOS9H337Zm8Z06a5t2+/97zt26f/Pho6v4i4E+6AlzJj1XLPooYckM3EqZgNbf03tGuoKRxU1jcHaTHS2QM0xqMlttwbqiEtf/eGt9zNks9vlp35m8I3h4b+DRo6vwhpttwV7i1IQ8OtoTuH5j6/di7aOTUFCndJqiH/nLkOt1x/c2jpOxftnJqGjIY7MBpYDqwEJiaZfivwLrAI+CvQN9UyFe7NUy7/uXIdri1956KdU9PYuWQs3IHWwCqgP3AAsBAYUKPMKKB97PWNwPOplqtwl/rK9T93rsMx1zsX7Zxyv3Nxz2y4nwDMShj+N+Df6ig/DHgj1XIV7rI/ctlyyvU/d67DTTun3G6/uEyG+4XArxOGLwMer6P848CdtUybAJQCpX369KnfOxJpAlryzkU7p9zuXOIyGe4XJQn3x2opOx54E2ibarlquYvUX677fLVzilbLPa1uGeB0YBlwcDorVriLtDzaOe3//HGZDPc2wAdAv4QDqgNrlBkWO+h6VDordYW7iDRDud45uacf7hbK1s3MzgImxc6cecrd7zOze2MrecXM/gIMBtbGZvnI3c+pa5nFxcVeWlqact0iIlLNzOa6e3Gqcmldz93dZwIza4y7K+H16fWuoYiINBpdOExEJIIU7iIiEaRwFxGJIIW7iEgEpXW2TKOs2GwDsDonK0+tO7Ax15WoQ1OvHzT9Oqp+DaP6NUxD6tfX3XukKpSzcG/KzKw0nVONcqWp1w+afh1Vv4ZR/RomG/VTt4yISAQp3EVEIkjhntzUXFcghaZeP2j6dVT9Gkb1a5hGr5/63EVEIkgtdxGRCFK4i4hEUIsNdzM73Mxmm9kyM1tqZt9NUmakmW0xswWxx13JltWIdSwzs8Wxde9zCU0LJpvZSjNbZGZFWazb0QnbZYGZbTWz79Uok/XtZ2ZPmdl6M1uSMO4gM/uzma2IPXetZd4rYmVWmNkVWazfQ2b2Xuxv+KKZdall3jo/D41Yv3vM7JOEv+NZtcw72syWxz6PE7NYv+cT6lZmZgtqmbdRt19tmZKzz1861wWO4gPoCRTFXucD77Pvjb9HAq/msI5lQPc6pp8FvAYYMBx4K0f1bA18SvhxRU63H3AKUAQsSRj3IDAx9noi8ECS+Q4i3LfgIKBr7HXXLNXvDKBN7PUDyeqXzuehEet3D/DDND4Dq4D+VN/3YUA26ldj+s+Bu3Kx/WrLlFx9/lpsy93d17r7vNjrbYS7SPXKba3q7VzgPzx4E+hiZj1zUI/TgFXunvNfHLv7HOCzGqPPBZ6NvX4WOC/JrP8X+LO7f+bunwN/BkZno37u/t/uXhEbfBPonen1pquW7ZeO44GV7v6Bu+8GphO2e0bVVT8zM+Bi4LlMrzcddWRKTj5/LTbcE5lZAeFuUm8lmXyCmS00s9fMbGBWKwYO/LeZzTWzCUmm9wI+ThguJzc7qDHU/g+Vy+0Xd4i7r4XwDwgcnKRMU9mWVxO+jSWT6vPQmG6OdRs9VUu3QlPYficD69x9RS3Ts7b9amRKTj5/LT7czawj8Hvge+6+tcbkeYSuhiHAY8BLWa7eie5eBJwJ3GRmp9SYbknmyeq5rWZ2AHAO8J9JJud6+9VHU9iWdwAVQEktRVJ9HhrLE8ARwFDC3dZ+nqRMzrcfMJa6W+1Z2X4pMqXW2ZKMa9D2a9HhbmZ5hD9Cibv/oeZ0d9/q7ttjr2cCeWbWPVv1c/c1sef1wIuEr76JyoHDE4Z7A2uyU7sqZwLz3H1dzQm53n4J1sW7q2LP65OUyem2jB1A+xYwzmOdsDWl8XloFO6+zt0r3X0P8Kta1pvr7dcGuAB4vrYy2dh+tWRKTj5/LTbcY/1zvwGWufsjtZQ5NFYOMzuesL02Zal+HcwsP/6acNBtSY1irwCXx86aGQ5siX/9y6JaW0u53H41vALEzz64Ang5SZlZwBlm1jXW7XBGbFyjM7PRwG3AOe6+o5Yy6XweGqt+icdxzq9lve8AR5lZv9i3uTGE7Z4tpwPvuXt5sonZ2H51ZEpuPn+NdeS4qT+AkwhfexYBC2KPs4AbgBtiZW4GlhKO/L8J/EsW69c/tt6FsTrcERufWD8DphDOUlgMFGd5G7YnhHXnhHE53X6EHc1a4CtCa+gaoBvwV2BF7PmgWNli4NcJ814NrIw9rspi/VYS+lvjn8NfxsoeBsys6/OQpfr9Nvb5WkQIqp416xcbPotwhsiqbNYvNv6Z+OcuoWxWt18dmZKTz58uPyAiEkEttltGRCTKFO4iIhGkcBcRiSCFu4hIBCncRUQiSOEuIhJBCncRkQj6/9WzTe+sIwnWAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss) + 1)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "일부 검증 손실은 학습을 사용하지 않은 기준점에 가깝지만 안정적이지 못합니다. 앞서 기준 모델을 만든 것이 도움이 됩니다. 이 문제는 기준 모델의 성능을 앞지르기가 쉽지 않습니다. 우리가 적용한 상식에는 머신 러닝 모델이 찾지 못한 핵심 정보가 많이 포함되어 있습니다.\n",
    "\n",
    "간단하고 괜찮은 성능을 내는 모델(상식 수준의 기준 모델)이 데이터와 타깃을 매핑할 수 있다면 왜 훈련한 모델은 이를 찾지 못하고 성능이 낮을까요? 훈련 과정이 찾는 것은 간단한 이 모델이 아니기 때문입니다. 문제 해결을 위해 탐색하는 모델의 공간, 즉 가설 공간은 우리가 매개변수로 설정한 두 개 층을 가진 네트워크의 모든 가능한 가중치의 조합입니다. 이 네트워크는 이미 매우 복잡합니다. 복잡한 모델 공간에서 해결책을 탐색할 때 간단하고 괜찮은 성능을 내는 모델은 찾지 못할 수 있습니다. 심지어 기술적으로 봤을 때 이 가설 공간에 포함되어 있을 때 조차도 말이죠. 이것이 일반적으로 머신 러닝이 가진 심각한 제약 사항입니다. 학습 알고리즘이 특정한 종류의 간단한 모델을 찾도록 하드코딩되지 않았다면, 모델 파라미터를 학습하는 방법은 간단한 문제를 위한 간단한 해결책을 찾지 못할 수 있습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 첫 번째 순환 신경망\n",
    "\n",
    "첫 번째 완전 연결 네트워크는 잘 작동하지 않았습니다. 그렇다고 이 문제에 머신 러닝이 적합하지 않다는 뜻은 아닙니다. 앞선 모델은 시계열 데이터를 펼쳤기 때문에 입력 데이터에서 시간 개념을 잃어버렸습니다. 대신 인과 관계와 순서가 의미가 있는 시퀀스 데이터 그대로 사용해 보겠습니다. 이런 시퀀스 데이터에 아주 잘 들어맞는 순환 시퀀스 처리 모델을 시도해 보겠습니다. 이 모델은 앞선 모델과 달리 데이터 포인터의 시간 순서를 사용합니다.\n",
    "\n",
    "이전 절에서 소개한 `LSTM` 층 대신에 2014년에 정준영 등이 개발한 `GRU` 층을 사용하겠습니다. GRU 층은 LSTM과 같은 원리로 작동하지만 조금 더 간결하고 그래서 계산 비용이 덜 듭니다(LSTM 만큼 표현 학습 능력이 높지는 않을 수 있습니다). 계산 비용과 표현 학습 능력 사이의 트레이드오프(trade-off)는 머신 러닝 어디에서나 등장합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "500/500 [==============================] - 109s 219ms/step - loss: 0.3072 - val_loss: 0.2738\n",
      "Epoch 2/20\n",
      "500/500 [==============================] - 109s 218ms/step - loss: 0.2878 - val_loss: 0.2662\n",
      "Epoch 3/20\n",
      "500/500 [==============================] - 109s 218ms/step - loss: 0.2810 - val_loss: 0.2681\n",
      "Epoch 4/20\n",
      "500/500 [==============================] - 109s 218ms/step - loss: 0.2762 - val_loss: 0.2658\n",
      "Epoch 5/20\n",
      "500/500 [==============================] - 109s 218ms/step - loss: 0.2709 - val_loss: 0.2635\n",
      "Epoch 6/20\n",
      "500/500 [==============================] - 109s 218ms/step - loss: 0.2667 - val_loss: 0.2639\n",
      "Epoch 7/20\n",
      "500/500 [==============================] - 109s 218ms/step - loss: 0.2602 - val_loss: 0.2644\n",
      "Epoch 8/20\n",
      "500/500 [==============================] - 109s 218ms/step - loss: 0.2570 - val_loss: 0.2690\n",
      "Epoch 9/20\n",
      "500/500 [==============================] - 109s 218ms/step - loss: 0.2515 - val_loss: 0.2736\n",
      "Epoch 10/20\n",
      "500/500 [==============================] - 109s 218ms/step - loss: 0.2486 - val_loss: 0.2758\n",
      "Epoch 11/20\n",
      "500/500 [==============================] - 109s 218ms/step - loss: 0.2431 - val_loss: 0.2727\n",
      "Epoch 12/20\n",
      "500/500 [==============================] - 109s 218ms/step - loss: 0.2395 - val_loss: 0.2744\n",
      "Epoch 13/20\n",
      "500/500 [==============================] - 109s 218ms/step - loss: 0.2358 - val_loss: 0.2775\n",
      "Epoch 14/20\n",
      "500/500 [==============================] - 109s 218ms/step - loss: 0.2322 - val_loss: 0.2841\n",
      "Epoch 15/20\n",
      "500/500 [==============================] - 109s 218ms/step - loss: 0.2281 - val_loss: 0.2844\n",
      "Epoch 16/20\n",
      "500/500 [==============================] - 109s 218ms/step - loss: 0.2253 - val_loss: 0.2841\n",
      "Epoch 17/20\n",
      "500/500 [==============================] - 109s 218ms/step - loss: 0.2227 - val_loss: 0.2905\n",
      "Epoch 18/20\n",
      "500/500 [==============================] - 109s 218ms/step - loss: 0.2181 - val_loss: 0.2945\n",
      "Epoch 19/20\n",
      "500/500 [==============================] - 109s 217ms/step - loss: 0.2145 - val_loss: 0.2959\n",
      "Epoch 20/20\n",
      "500/500 [==============================] - 109s 218ms/step - loss: 0.2114 - val_loss: 0.2976\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.GRU(32, input_shape=(None, float_data.shape[-1])))\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=20,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "결과를 확인해 보죠:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3XmcVOWV//HPYRNZBATiArIZY1iFtkM0oqASB1SWKEYR4hIdgtFEB83ICEmUxN+omGhwGBUdjEYiMTomaFR0gBEdBW0Qm00ECWgLgRZlR6Dh/P54bkPRVHdX0921ft+vV7266t6nbp263X3qqeee+1xzd0REJDfUSXUAIiKSPEr6IiI5RElfRCSHKOmLiOQQJX0RkRyipC8ikkOU9KVKzKyumW03s3Y12TaVzOzrZlbjtctm1t/M1sQ8XmFmZyfS9ghe63Ezu+NIn1/Bdn9tZr+v6e1K6tRLdQBSu8xse8zDRsBuYF/0+EfuPq0q23P3fUCTmm6bC9z91JrYjpldD4x0934x276+JrYt2U9JP8u5+4GkG/Ukr3f3/ymvvZnVc/eSZMQmIsmn4Z0cF319/5OZPWNm24CRZnammc0zs81mtt7MJplZ/ah9PTNzM+sQPX46Wv+KmW0zs3fMrGNV20brB5rZR2a2xcweMrP/M7Nryok7kRh/ZGarzOxLM5sU89y6ZvaAmW0ys4+BARXsn/FmNr3Msslm9tvo/vVmtjx6Px9HvfDytlVkZv2i+43M7A9RbEuB0+O87upou0vNbHC0vDvwH8DZ0dDZ5zH79s6Y54+O3vsmM/uLmZ2QyL6pjJkNjeLZbGazzezUmHV3mNk6M9tqZh/GvNczzGxhtHyDmU1M9PWkFri7bjlyA9YA/css+zWwBxhE6AQcDXwL+Dbhm2An4CPgpqh9PcCBDtHjp4HPgXygPvAn4OkjaPs1YBswJFo3BtgLXFPOe0kkxr8CzYAOwBel7x24CVgKtAVaAnPDv0Lc1+kEbAcax2x7I5AfPR4UtTHgPGAX0CNa1x9YE7OtIqBfdP9+4H+BFkB7YFmZtt8HToh+J1dGMRwXrbse+N8ycT4N3BndvyCKsSfQEPhPYHYi+ybO+/818PvofucojvOi39Ed0X6vD3QF1gLHR207Ap2i++8Bw6P7TYFvp/p/IZdv6ukLwFvu/qK773f3Xe7+nrvPd/cSd18NTAH6VvD859y9wN33AtMIyaaqbS8GFrn7X6N1DxA+IOJKMMZ/d/ct7r6GkGBLX+v7wAPuXuTum4B7Knid1cASwocRwHeBze5eEK1/0d1XezAbmAXEPVhbxveBX7v7l+6+ltB7j33dZ919ffQ7+SPhAzs/ge0CjAAed/dF7v4VMBboa2ZtY9qUt28qcgUww91nR7+je4BjCB++JYQPmK7REOHfo30H4cP7FDNr6e7b3H1+gu9DaoGSvgB8GvvAzL5pZn8zs3+Y2VZgAtCqguf/I+b+Tio+eFte2xNj43B3J/SM40owxoRei9BDrcgfgeHR/SsJH1alcVxsZvPN7Asz20zoZVe0r0qdUFEMZnaNmX0QDaNsBr6Z4HYhvL8D23P3rcCXQJuYNlX5nZW33f2E31Ebd18B3Er4PWyMhguPj5peC3QBVpjZu2Z2YYLvQ2qBkr5A+Lof61FC7/br7n4M8AvC8EVtWk8YbgHAzIxDk1RZ1YlxPXBSzOPKSkr/BPSPespDCB8CmNnRwHPAvxOGXpoDryUYxz/Ki8HMOgEPAzcALaPtfhiz3crKS9cRhoxKt9eUMIz0WQJxVWW7dQi/s88A3P1pdz+LMLRTl7BfcPcV7n4FYQjvN8DzZtawmrHIEVLSl3iaAluAHWbWGfhREl7zJSDPzAaZWT3gZqB1LcX4LHCLmbUxs5bA7RU1dvcNwFvAE8AKd18ZrToKaAAUA/vM7GLg/CrEcIeZNbdwHsNNMeuaEBJ7MeHz73pCT7/UBqBt6YHrOJ4BrjOzHmZ2FCH5vunu5X5zqkLMg82sX/TaPyMch5lvZp3N7Nzo9XZFt32EN/ADM2sVfTPYEr23/dWMRY6Qkr7EcytwNeEf+lFCT7dWRYn1cuC3wCbgZOB9wnkFNR3jw4Sx98WEg4zPJfCcPxIOzP4xJubNwL8ALxAOhg4jfHgl4peEbxxrgFeAp2K2WwhMAt6N2nwTiB0Hfx1YCWwws9hhmtLnv0oYZnkhen47wjh/tbj7UsI+f5jwgTQAGByN7x8F3Ec4DvMPwjeL8dFTLwSWW6gOux+43N33VDceOTIWhk5F0ouZ1SUMJwxz9zdTHY9ItlBPX9KGmQ0ws2bREMHPCRUh76Y4LJGsoqQv6aQPsJowRDAAGOru5Q3viMgR0PCOiEgOUU9fRCSHpN2Ea61atfIOHTqkOgwRkYyyYMGCz929ojJnIA2TfocOHSgoKEh1GCIiGcXMKjuzHNDwjohITlHSFxHJIUr6IiI5RElfRCSHKOmLiOSQrEn606ZBhw5Qp074Oa1Kl/sWEckNaVeyeSSmTYNRo2DnzvB47drwGGBEtecWFBHJHlnR0x837mDCL7VzZ1guIiIHZUXS/+STqi0XEclVWZH025VzsbvylouI5KqsSPp33w2NGh26rFGjsFxERA7KiqQ/YgRMmQLt24NZ+Dllig7iioiUlRXVOxASvJK8iEjFsqKnLyIiiVHSFxHJIUr6IiI5RElfRCSHKOmLiOQQJX0RkRyipC8ikkMSSvpmNsDMVpjZKjMbG2f9aDNbbGaLzOwtM+sSs+7fouetMLN/qsngRUSkaipN+mZWF5gMDAS6AMNjk3rkj+7e3d17AvcBv42e2wW4AugKDAD+M9qeiIikQCI9/d7AKndf7e57gOnAkNgG7r415mFjwKP7Q4Dp7r7b3f8OrIq2JyIiKZDINAxtgE9jHhcB3y7byMxuBMYADYDzYp47r8xz28R57ihgFEA7TY0pIlJrEunpW5xlftgC98nufjJwOzC+is+d4u757p7funXrBEISEZEjkUjSLwJOinncFlhXQfvpwNAjfK6IiNSiRJL+e8ApZtbRzBoQDszOiG1gZqfEPLwIWBndnwFcYWZHmVlH4BTg3eqHLSIiR6LSMX13LzGzm4CZQF1gqrsvNbMJQIG7zwBuMrP+wF7gS+Dq6LlLzexZYBlQAtzo7vtq6b2IiGSczz6Dd96Bt9+Gxo3hV7+q3dcz98OG2FMqPz/fCwoKUh2GiEiN27sXFi06mOTfeefgtbwbNoRBg+DZZ49s22a2wN3zK2uXNRdRERFJNxs3hsRemuQLCmDXrrDupJPgO9+BMWPCz9NOgwYNaj8mJX0RkRqwbx8sWXKwB//22/Dxx2Fd/fqQlwejR8OZZ4Zb27apiVNJX0SkCtxh/XpYvPjQ27JlsHt3aHP88SGxlyb5008PwzfpQElfRKQcW7bA0qWHJvclS+CLLw62OfFE6N4dzj8fevUKSb5DB7B4ZymlASV9ERFCFc0bbxya4EsPsgI0bRqS+2WXQbdu4X63btCyZepiPhJK+iKS09atg7vvhsceC9U19evDN78JffocTO7du0O7dunbe68KJX0RyUkbN8I998DDD0NJCVx3HdxwA3TunJwqmlRR0heRnPLFFzBxIjz0UCifvOoq+PnPoVOnVEeWHEr6IpITtmyBBx4It23b4Ior4Je/hFNPTXVkyaWkLyJZbfv20KufOBG+/BIuuQTuuiuM1+ciJX0RyUq7doXx+nvugeJiuOgimDAhnCSVy3RhdBHJKrt3w+TJcPLJcOutYXqDd96Bl15Swgf19EUkS+zdC08+GWap/OQTOPtseOYZ6Ns31ZGlFyV9Ecl4a9fC0KFhBsveveHxx6F//+yoq69pSvoiktHefhu+970wrPPcc+FArZJ9+TSmLyIZ68kn4dxz4ZhjYN48uPRSJfzKKOlHpk0LkyTVqRN+TpuW6ohEpDz79sG//itcc02YLmH+/DB1glROwzuEBD9qFOzcGR6vXRseA4wYkbq4RORwW7eG/8uXXoIf/xgefDDMlyOJUU8fGDfuYMIvtXNnWC4i6WP16nCVqVdeCWWZkycr4VeVevocOn1qIstFJPneeCOM2e/fDzNnhvnrperU0ydMmVqV5SKSXKUlmK1ahfF7Jfwjp6RPmEu7UaNDlzVqFJaLSOqUlMAtt8A//3NI9PPmwSmnpDqqzKakTzgoNGUKtG8fyr3atw+PdRBXJHU2b4aLL4bf/Q5uvjkcuG3ePNVRZT6N6UdGjFCSF0kXK1fCoEHw8cehA/bP/5zqiLKHkr6IpJXZs2HYsHDOzP/8j+bOqWlK+iJSIXfYswd27Ahz01f00wyaNIHGjQ//GXu/vMsRPvww/OQn4USrGTNy52pWyaSkLyLs2QO/+EWYxyZeMi8pqdnXq1//8A8GMygoCPPe//GPYWoFqXlK+iI57ssvQ/37nDlw1llwwgnl99Yr++l++AdGIt8QSu/feSeMHw9166Z6r2QvJX2RHLZqVehZr1kDTz0FP/hB9bfZqlX1tyG1R0lfJEfNnRumJDYLB0zPPjvVEUkyqE6/hmiWTskkTz0VznBt3Tqc8KSEnzuU9GtA6Syda9eGMc3SWTqV+CXd7N8fxsyvvjok+nfega9/PdVRSTIllPTNbICZrTCzVWY2Ns76MWa2zMwKzWyWmbWPWXefmS01s+VmNsks+y5xoFk6JRPs2gXDh4fpRa6/Hl59FVq0SHVUkmyVJn0zqwtMBgYCXYDhZtalTLP3gXx37wE8B9wXPfc7wFlAD6Ab8C0g60610Cydku42bAhXmPrzn2HixHCWq6Ykzk2J9PR7A6vcfbW77wGmA0NiG7j7HHcv7evOA9qWrgIaAg2Ao4D6wIaaCDydaJZOSWeLF4eLhS9eDP/933DbbbqkYC5LJOm3AT6NeVwULSvPdcArAO7+DjAHWB/dZrr78rJPMLNRZlZgZgXFxcWJxp42NEunpKtXXw219yUl8OabMHRoqiOSVEsk6cfrE3jchmYjgXxgYvT460BnQs+/DXCemZ1z2Mbcp7h7vrvnt27dOtHY04Zm6ZR0NHlyqME/+eQwB31eXqojknSQSJ1+EXBSzOO2wLqyjcysPzAO6Ovuu6PF3wPmufv2qM0rwBnA3OoEnY40S6eki5ISGDMGHnoIBg8OVWRNmqQ6KkkXifT03wNOMbOOZtYAuAKYEdvAzHoBjwKD3X1jzKpPgL5mVs/M6hMO4h42vCMiNWPbNhgyJCT8MWPCGL4SvsSqtKfv7iVmdhMwE6gLTHX3pWY2AShw9xmE4ZwmwJ+jisxP3H0woZLnPGAxYUjoVXd/sXbeikhu++STcNGRZcvg0UfDuSIiZZl73OH5lMnPz/eCgoJUh5F006aFuv5PPglVP3ffreEiqZw7/N//wWOPhXLMBg3Cz+9+N9WRSbKZ2QJ3z6+snebeSQOlZ/SWnuBVekYvKPFLfJs2hakUHnsMli+Hpk3DWba33qozbKVimoYhDeiMXkmEe5j+ePhwOPHEMGbfvDlMnQrr14cLkCjhS2XU008DOqM3tTZtgtdeg1mzQrntddeFpJouNmyA3/8eHn88TIXcogWMHh2uG9utW6qjk0yjpJ8G2rULQzrxlkvN278/XKHplVfC7d13Qy+6eXPYvBnuuitUwNxwA5x3Xpg5NRUxvv56ON9jxoxQhnnOOfDLX4YLnhx9dPJjkuyg4Z00oDN6a19xMTz9dDhGctxx8O1vh+QOIZHOnw+ffx560rfeGuaa/+534dRT4f77w7pk+Owz+NWvwrVhBwwIcdx8cxi3f+MNGDlSCV+qR9U7aULVOzVr377Qg3/11dCbLygIvfnWrUMyHTgQLrgAWraM//zdu+H558M4+VtvwVFHwWWXhWGV73yn5uau2bo1fOC8/XZ4ndmzQy+/f/8wfDNkSHhtkcokWr2TNUnfHf7938N47HHH1UJgkvaKiw8O2bz2GnzxRRiaOeOMkOQHDAhTEVR1uGbJklD3/tRTIUl37x6S/8iRVbt4tzt8/HFI8KW3JUvCcrOw3YsuCn/DJ59ctRhFci7pr1gR/qFPOikckGtT0ZRwklW++ioMwfy//xfmjD/uuIO9+e9+F449tmZeZ8cOeOaZ0PtfuDBcCPzKK8PYf69eh7fftSt8wyhN8O+8Ez6YIHxYnHlm+Nbwne+EWTCr8gEiUlbOJX0IswheeGH4p589WwdCc8HLL8NPfxp60JdeCnfcAT171v7B14KCkPyfeSYk99694Uc/ClMelCb4hQvDAViAb3zjYII/80zo0iU1B4gle+Vk0odwvc8BA0IlxuzZ4YCYZJ/Vq+GWW+DFF8PB1oceSs1ZqJs3wx/+AI88EqY/gHCgtXfvgz35M84IxxJEalOiST/r+hpnnBGGd7ZtCyVuH32U6oiSI1cuzL5rF9x5Z+gpz54N994LhYWpm3ageXP4yU/C2Py8eeHg8ZYt8L//G44xDRqkhC/pJeuSPsDpp4czF/fsgb59D/bAslUuXJjdPdSrd+0aSi2HDoUPP4R//dcw30yqmYUy0G99S5chlPSWlUkfoEeP0NuCkPg/+CCl4dSqbJ/GYdWqUNUyZEgYOpk9G6ZPh7ZtK3+uiBwqa5M+hCGAuXOhYcNwUegFC1IdUe3I1mkcduyA8eND7/6tt+A3v4FFi8LvUkSOTFYnfYBTTgmJv1kzOP/8MO6abbLtwuzu4cSozp3DSWrf/34oyR0zRkMnItWV9UkfoGPHcAp7q1bhgN+bb6Y6opqVTdM4fPgh/NM/wbBh4SDp3LmhOuaEE1IdmUh2yImkD6HXO3duGAceMCBU+GSLbLgw+/btcPvt4VjM/Pnwu9+FOvezz051ZCLZJevq9CuzYUOY12TVKnjhhfABIKnjHk5w+tnPYN26cCGQe+/VVBoiVZWzdfqVOe64UM7ZuXOoBpkxo/LnVGTXrjB09KtfhZrsW28NVUN799ZIuFmtsBD69QvfSE44IZzJ+vvfK+GL1KacS/oQxvZnzQqn6196KTz3XOLP3bo1zNx4xx3Qp08Yd+7XL0zP+9FH8B//EapLvva1MC/LM8/Al1/W2lupMck8uevLL8MJTb16wdKlYTKz+fPDGawiUsvcPa1up59+uifLli3uZ53lXqeO+9NPx2+zcaP788+733KLe15eaAvu9eq5f/vb7j/7mfuLL7p/8UVov3VraH/NNe6tW4e2deu69+vn/pvfuH/0UdLeXsKeftq9UaMQa+mtUaPy98mRKilxnzLFvVWrsB9vvNF906aafQ2RXAUUeAI5NufG9Mvavj0My7zxBvzXf4Xx/rlzQ4XP3Lnh4hUQav3PPDMcWDznnDDdQ+PGFW+7dE73F18MtyVLwvJTTw2vOWhQmJulXoqvX9ahQ/wrd7VvD2vW1MxrzJ8PN90UJirr0yfMldOzZ81sW0RyeMK1I7FzZzit//XXDy475piQnM45JyT6/Pzqn+7/97/DSy+FD4DScf9jjw1TAA8aFA4qN2tWvdc4EnXqhP59WWbhgh7VsWED/Nu/wRNPhHH7iRPDsFdNXYRERAIl/Sr66it44IHQez/77FA6WLdu7b3e1q3hQh8vvgh/+1u4OHe9euFD5uKLw+2UU2rv9WPVRk9/7174z/+EX/wiHOy+5Rb4+c+hadPqRCoi5VHSzyD79oUzhV98MXwTWLo0LP/GNw5+APTpU3tno5ZO2BY7f0+jRkde6z9nTjhQu3RpuCThpElhSEtEao9KNjNI3bpw1llwzz1h3H/16jDm3alTqAY677xQcXT55eHs1Jq+SHdNnNz11Veheunyy0O8O3aE8yBefVUJXySdqKef5rZvD+WlL70Ubv/4R0jMZ5558FtAt27VHyN3D8MwmzeHksqq3r76KmynYcMwhv+zn4UZMUUkOTS8k4X274f33z/4AVC6m9q1C1MPX3QRtGgRjhds3RouJFN6P/ZW3vJ9+yp+/WbNwvZLb82bH/q49KB0+/a1vy9E5FBK+jlg/fpwjdiXXgqVRzt2lN+2ceNQkRTv1rTpwZ+xSTz21qxZ7R7YFpHqUdLPMbt3h2kM9uw5PKk3aaKELZLtEk36KT4tSGrKUUfp4iIiUjlV74iI5BAlfakRyZywTUSOXEJJ38wGmNkKM1tlZmPjrB9jZsvMrNDMZplZ+5h17czsNTNbHrXpUHPhSzooPblr7dpQ+rl2bXisxC+SfipN+mZWF5gMDAS6AMPNrEuZZu8D+e7eA3gOuC9m3VPARHfvDPQGNtZE4JI+xo079GxeCI/HjUtNPCJSvkR6+r2BVe6+2t33ANOBIbEN3H2Ou5f+288D2gJEHw713P31qN32mHaSJT75pGrLRSR1Ekn6bYBPYx4XRcvKcx3wSnT/G8BmM/tvM3vfzCZG3xwOYWajzKzAzAqKi4sTjV3SRLt2VVsuIqmTSNKPd4J/3OJ+MxsJ5AMTo0X1gLOB24BvAZ2Aaw7bmPsUd8939/zWrVsnEJKkk7vvDhO0xWrUKCwXkfSSSNIvAk6KedwWWFe2kZn1B8YBg919d8xz34+GhkqAvwB51QtZ0k1NTNgmIsmRSNJ/DzjFzDqaWQPgCuCQy4mbWS/gUULC31jmuS3MrLT7fh6wrPphS7oZMSLMvb9/f/hZ1YSvkk+R5Kj0jFx3LzGzm4CZQF1gqrsvNbMJhGsyziAM5zQB/mxhusdP3H2wu+8zs9uAWRZWLAAeq603I5mp7Hz+pSWfoG8LIjVNc+9IyiXjGr0i2U4XUZGMoZJPkeRR0peUU8mnSPIo6UvKqeRTJHmU9CXlVPIpkjyaT1/SwogRSvIiyaCevohIDlHSFxHJIUr6IiI5RElfsoKmcRBJjA7kSsbTNA4iiVNPXzKertwlkjglfcl4msZBJHFK+pLxNI2DSOKU9CXjaRoHkcQp6UvGq4lpHFT9I7lC1TuSFaozjYOqfySXqKcvOU/VP5JLlPQl56n6R3KJkr7kPFX/SC5R0pecp+ofySVK+pLzdBEXySWq3hFBF3GR3KGevohIDlHSFxHJIUr6IiI5RElfRCSHKOmL1ADN3SOZQtU7ItWkuXskk6inL1JNmrtHMomSvkg1ae4eySRK+iLVpLl7JJMo6YtUk+bukUySUNI3swFmtsLMVpnZ2Djrx5jZMjMrNLNZZta+zPpjzOwzM/uPmgpcJF1o7h7JJJUmfTOrC0wGBgJdgOFm1qVMs/eBfHfvATwH3Fdm/a+AN6ofrkh6GjEC1qyB/fvDz6omfJV8SrIk0tPvDaxy99XuvgeYDgyJbeDuc9y9tH5hHtC2dJ2ZnQ4cB7xWMyGLZJfSks+1a8H9YMmnEr/UhkSSfhvg05jHRdGy8lwHvAJgZnWA3wA/q+gFzGyUmRWYWUFxcXECIYlkD5V8SjIlkvQtzjKP29BsJJAPTIwW/Rh42d0/jdf+wMbcp7h7vrvnt27dOoGQRLKHSj4lmRI5I7cIOCnmcVtgXdlGZtYfGAf0dffd0eIzgbPN7MdAE6CBmW1398MOBovkqnbtwpBOvOUiNS2Rnv57wClm1tHMGgBXADNiG5hZL+BRYLC7byxd7u4j3L2du3cAbgOeUsIXOZRKPiWZKk367l4C3ATMBJYDz7r7UjObYGaDo2YTCT35P5vZIjObUc7mRKQMlXxKMpl73OH5lMnPz/eCgoJUhyEiklHMbIG751fWTmfkimQB1flLojS1skiG09TOUhXq6YtkONX5S1Uo6YtkONX5S1Uo6YtkOE3tLFWhpC+S4VTnL1WhpC+S4VTnL1WhpC+SBTS1syRKJZsiOU4ln7lFPX2RHKeSz9yipC+S41TymVuU9EVynEo+c4uSvkiOU8lnblHSF8lxKvnMLareERFGjFCSzxXq6YtItanOP3Oopy8i1aI6/8yinr6IVIvq/DOLkr6IVIvq/DOLkr6IVIvq/DOLkr6IVIvq/DOLkr6IVEtN1Pmr+id5VL0jItVWnTp/Vf8kl3r6IpJSqv5JLiV9EUkpVf8kV0YM7+zdu5eioiK++uqrVIciCWjYsCFt27alfv36qQ5FMkC7dmFIJ95yqXkZkfSLiopo2rQpHTp0wMxSHY5UwN3ZtGkTRUVFdOzYMdXhSAa4++5Dx/RB1T+1KSOGd7766itatmyphJ8BzIyWLVvqW5kkTNU/yZURPX1ACT+D6HclVaXqn+TJiJ6+iEh5VP1TNVmZ9Gv6q96mTZvo2bMnPXv25Pjjj6dNmzYHHu/ZsyehbVx77bWsWLGiwjaTJ09mWg19L+3Tpw+LFi2qkW2JpDNV/1RNxgzvJKo2vuq1bNnyQAK98847adKkCbfddtshbdwdd6dOnfifo0888USlr3PjjTceWYAiOUzVP1WTUE/fzAaY2QozW2VmY+OsH2Nmy8ys0MxmmVn7aHlPM3vHzJZG6y6v6TdQVjK/6q1atYpu3boxevRo8vLyWL9+PaNGjSI/P5+uXbsyYcKEA21Le94lJSU0b96csWPHctppp3HmmWeyceNGAMaPH8+DDz54oP3YsWPp3bs3p556Km+//TYAO3bs4NJLL+W0005j+PDh5OfnV9qjf/rpp+nevTvdunXjjjvuAKCkpIQf/OAHB5ZPmjQJgAceeIAuXbpw2mmnMXLkyBrfZyI1TXP/VE2lPX0zqwtMBr4LFAHvmdkMd18W0+x9IN/dd5rZDcB9wOXATuAqd19pZicCC8xsprtvrvF3Ekn2V71ly5bxxBNP8MgjjwBwzz33cOyxx1JSUsK5557LsGHD6NKlyyHP2bJlC3379uWee+5hzJgxTJ06lbFjD/ssxd159913mTFjBhMmTODVV1/loYce4vjjj+f555/ngw8+IC8vr8L4ioqKGD9+PAUFBTRr1oz+/fvz0ksv0bp1az7//HMWL14MwObN4Vdy3333sXbtWho0aHBgmUg6K/0GP25c+D9v1y4kfB3EjS+Rnn5vYJW7r3b3PcB0YEhsA3ef4+6l/et5QNto+UfuvjK6vw7YCLSuqeDjSfY0ryeffDLf+ta3Djx+5plnyMvLIy8vj+XLl7Ns2bLDnnP00UczcOBAAE4//XTWrFkTd9uXXHLJYW3eeustrrjiCgBOO+00unbtWmEVkkItAAANZUlEQVR88+fP57zzzqNVq1bUr1+fK6+8krlz5/L1r3+dFStWcPPNNzNz5kyaNWsGQNeuXRk5ciTTpk3TyVWSMUaMgDVrYP/+8FMJv3yJJP02wKcxj4uiZeW5Dnil7EIz6w00AD6Os26UmRWYWUFxcXECIZUv2V/1GjdufOD+ypUr+d3vfsfs2bMpLCxkwIABcevVGzRocOB+3bp1KSkpibvto4466rA27l6l+Mpr37JlSwoLC+nTpw+TJk3iRz/6EQAzZ85k9OjRvPvuu+Tn57Nv374qvZ6IpLdEkn68ouu4mcTMRgL5wMQyy08A/gBc6+77D9uY+xR3z3f3/Natq/dFoCZO9DhSW7dupWnTphxzzDGsX7+emTNn1vhr9OnTh2effRaAxYsXx/0mEeuMM85gzpw5bNq0iZKSEqZPn07fvn0pLi7G3bnsssu46667WLhwIfv27aOoqIjzzjuPiRMnUlxczM6yB0hEslAundyVSPVOEXBSzOO2wLqyjcysPzAO6Ovuu2OWHwP8DRjv7vOqF25iqnOiR3Xk5eXRpUsXunXrRqdOnTjrrLNq/DV+8pOfcNVVV9GjRw/y8vLo1q3bgaGZeNq2bcuECRPo168f7s6gQYO46KKLWLhwIddddx3ujplx7733UlJSwpVXXsm2bdvYv38/t99+O02bNq3x9yCSTnLt5C6rbLjAzOoBHwHnA58B7wFXuvvSmDa9gOeAAaVj+NHyBoShnhfd/cFEAsrPz/eCgoJDli1fvpzOnTsn9IayXUlJCSUlJTRs2JCVK1dywQUXsHLlSurVS6/qW/3OJFN06BC/5LN9+3B8IFOY2QJ3z6+sXaWZwt1LzOwmYCZQF5jq7kvNbAJQ4O4zCMM5TYA/R6fgf+Lug4HvA+cALc3smmiT17i7zho6Qtu3b+f888+npKQEd+fRRx9Nu4Qvkkly7eSuhLKFu78MvFxm2S9i7vcv53lPA09XJ0A5VPPmzVmwYEGqwxDJGrl2cldWTsMgIpKoXDu5S0lfRHJaKiv+UkGDwSKS81JV8ZcK6umLiFRTJtX5K+knoF+/foedaPXggw/y4x//uMLnNWnSBIB169YxbNiwcrddtkS1rAcffPCQk6QuvPDCGpkX58477+T++++v9nZEcllpnf/ateB+sM4/XRO/kn4Chg8fzvTp0w9ZNn36dIYPH57Q80888USee+65I379skn/5Zdfpnnz5ke8PRGpOZl2EZeMG9O/5Rao6WuD9OwJD1Zw6tiwYcMYP348u3fv5qijjmLNmjWsW7eOPn36sH37doYMGcKXX37J3r17+fWvf82QIYfMR8eaNWu4+OKLWbJkCbt27eLaa69l2bJldO7cmV27dh1od8MNN/Dee++xa9cuhg0bxl133cWkSZNYt24d5557Lq1atWLOnDl06NCBgoICWrVqxW9/+1umTp0KwPXXX88tt9zCmjVrGDhwIH369OHtt9+mTZs2/PWvf+Xoo48u9z0uWrSI0aNHs3PnTk4++WSmTp1KixYtmDRpEo888gj16tWjS5cuTJ8+nTfeeIObb74ZCJdGnDt3rs7clZyVaXX+6uknoGXLlvTu3ZtXX30VCL38yy+/HDOjYcOGvPDCCyxcuJA5c+Zw6623Vjgp2sMPP0yjRo0oLCxk3Lhxh9Tc33333RQUFFBYWMgbb7xBYWEhP/3pTznxxBOZM2cOc+bMOWRbCxYs4IknnmD+/PnMmzePxx57jPfffx8Ik7/deOONLF26lObNm/P8889X+B6vuuoq7r33XgoLC+nevTt33XUXEKaKfv/99yksLDwwffT999/P5MmTWbRoEW+++WaFHyYi2S7ZM/tWV8b19Cvqkdem0iGeIUOGMH369AO9a3fnjjvuYO7cudSpU4fPPvuMDRs2cPzxx8fdzty5c/npT38KQI8ePejRo8eBdc8++yxTpkyhpKSE9evXs2zZskPWl/XWW2/xve9978BMn5dccglvvvkmgwcPpmPHjvTs2ROoePpmCPP7b968mb59+wJw9dVXc9lllx2IccSIEQwdOpShQ4cCcNZZZzFmzBhGjBjBJZdcQtu2bRPZhSJZ6e67D527B9K7zl89/QQNHTqUWbNmsXDhQnbt2nXg4iXTpk2juLiYBQsWsGjRIo477ri40ynHiqaqOMTf//537r//fmbNmkVhYSEXXXRRpdup6BtF6bTMUPH0zZX529/+xo033siCBQs4/fTTKSkpYezYsTz++OPs2rWLM844gw8//PCIti2SDWqizj+Z1T9K+glq0qQJ/fr144c//OEhB3C3bNnC1772NerXr8+cOXNYG+987hjnnHPOgYufL1myhMLCQiBMy9y4cWOaNWvGhg0beOWVg5ckaNq0Kdu2bYu7rb/85S/s3LmTHTt28MILL3D22WdX+b01a9aMFi1a8OabbwLwhz/8gb59+7J//34+/fRTzj33XO677z42b97M9u3b+fjjj+nevTu33347+fn5SvqS86pzEZdkV/9k3PBOKg0fPpxLLrnkkEqeESNGMGjQIPLz8+nZsyff/OY3K9zGDTfcwLXXXkuPHj3o2bMnvXv3BsJVsHr16kXXrl0Pm5Z51KhRDBw4kBNOOOGQcf28vDyuueaaA9u4/vrr6dWrV4VDOeV58sknDxzI7dSpE0888QT79u1j5MiRbNmyBXfnX/7lX2jevDk///nPmTNnDnXr1qVLly4HrgImIlVXUfVPbZwwVunUysmmqZWzg35nIompUyf08MsyC98cEpXo1Moa3hERSaFkV/8o6YuIpFCyZ/nMmKSfbsNQUj79rkQSl+xZPjPiQG7Dhg3ZtGkTLVu2jFvuKOnD3dm0aRMNGzZMdSgiGSOZs3xmRNJv27YtRUVFFBcXpzoUSUDDhg11wpZImsqIpF+/fn06duyY6jBERDJexozpi4hI9Snpi4jkECV9EZEcknZn5JpZMVDxBDap1Qr4PNVBVEDxVY/iqx7FVz3Via+9u7eurFHaJf10Z2YFiZzqnCqKr3oUX/UovupJRnwa3hERySFK+iIiOURJv+qmpDqASii+6lF81aP4qqfW49OYvohIDlFPX0Qkhyjpi4jkECX9MszsJDObY2bLzWypmd0cp00/M9tiZoui2y9SEOcaM1scvX5BnPVmZpPMbJWZFZpZXhJjOzVm3ywys61mdkuZNkndh2Y21cw2mtmSmGXHmtnrZrYy+tminOdeHbVZaWZXJzG+iWb2YfT7e8HMmpfz3Ar/FmoxvjvN7LOY3+GF5Tx3gJmtiP4WxyYxvj/FxLbGzBaV89xk7L+4eSUlf4PurlvMDTgByIvuNwU+ArqUadMPeCnFca4BWlWw/kLgFcCAM4D5KYqzLvAPwokjKduHwDlAHrAkZtl9wNjo/ljg3jjPOxZYHf1sEd1vkaT4LgDqRffvjRdfIn8LtRjfncBtCfz+PwY6AQ2AD8r+P9VWfGXW/wb4RQr3X9y8koq/QfX0y3D39e6+MLq/DVgOtEltVEdkCPCUB/OA5mZ2QgriOB/42N1Tepa1u88FviizeAjwZHT/SWBonKf+E/C6u3/h7l8CrwMDkhGfu7/m7iXRw3lAyuarLmf/JaI3sMrdV7v7HmA6Yb/XqIris3ARju8Dz9T06yaqgryS9L9BJf0KmFkHoBcwP87qM83sAzN7xcy6JjWwwIHXzGyBmY2Ks74N8GnM4yJS8+F1BeX/s6V6Hx7n7ush/FMCX4vTJl324w8J39ziqexvoTbdFA0/TS1naCId9t/ZwAZ3X1nO+qTuvzJ5Jel/g0r65TCzJsDzwC3uvrXM6oWE4YrTgIeAvyQ7PuAsd88DBgI3mtk5ZdbHu8RYUutzzawBMBj4c5zV6bAPE5EO+3EcUAJMK6dJZX8LteVh4GSgJ7CeMIRSVsr3HzCcinv5Sdt/leSVcp8WZ9kR70Ml/TjMrD7hFzPN3f+77Hp33+ru26P7LwP1zaxVMmN093XRz43AC4Sv0bGKgJNiHrcF1iUnugMGAgvdfUPZFemwD4ENpUNe0c+NcdqkdD9GB+0uBkZ4NMBbVgJ/C7XC3Te4+z533w88Vs7rpnr/1QMuAf5UXptk7b9y8krS/waV9MuIxv/+C1ju7r8tp83xUTvMrDdhP25KYoyNzaxp6X3CAb8lZZrNAK6KqnjOALaUfo1MonJ7WKneh5EZQGklxNXAX+O0mQlcYGYtouGLC6Jltc7MBgC3A4PdfWc5bRL5W6it+GKPEX2vnNd9DzjFzDpG3/yuIOz3ZOkPfOjuRfFWJmv/VZBXkv83WJtHrDPxBvQhfHUqBBZFtwuB0cDoqM1NwFJCJcI84DtJjrFT9NofRHGMi5bHxmjAZELlxGIgP8kxNiIk8WYxy1K2DwkfPuuBvYSe03VAS2AWsDL6eWzUNh94POa5PwRWRbdrkxjfKsJYbunf4SNR2xOBlyv6W0hSfH+I/rYKCcnrhLLxRY8vJFSrfJzM+KLlvy/9m4tpm4r9V15eSfrfoKZhEBHJIRreERHJIUr6IiI5RElfRCSHKOmLiOQQJX0RkRyipC8ikkOU9EVEcsj/B+9M3ub4HHM+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss) + 1)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "훨씬 좋군요! 상식 수준의 모델을 크게 앞질렀습니다. 이 그림은 시퀀스를 펼쳐서 처리하는 완전 연결 네트워크에 비해서 순환 네트워크가 이런 종류의 작업에 훨씬 뛰어나다는 것과 머신 러닝의 가치를 보여줍니다.\n",
    "\n",
    "새로운 검증 MAE는 0.265 이하(크게 과대적합되기 시작하는 곳)이고 정규화되기 전인 섭씨로 복원하면 MAE는 2.35°C입니다. 초기 에러 2.57°C보다는 확실히 낫지만 더 개선할 수 있을 것 같습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 과대적합을 감소하기 위해 순환 드롭아웃 사용하기\n",
    "\n",
    "훈련 손실과 검증 손실 곡선을 보면 모델이 과대적합인지 알 수 있습니다. 몇 번의 에포크 이후에 훈련 손실과 검증 손실이 현저하게 벌어지기 시작합니다. 이런 현상을 해결하기 위해 잘 알려진 드롭아웃 기법을 이미 보았습니다. 훈련 데이터를 층에 주입할 때 데이터에 있는 우연한 상관 관계를 깨뜨리기 위해 입력 층의 유닛을 랜덤하게 끄는 기법입니다. 순환 신경망에 드롭아웃을 올바르게 적용하는 방법은 간단하지 않습니다. 순환 층 이전에 드롭아웃을 적용하면 규제에 도움이 되는 것보다 학습에 더 방해되는 것으로 오랫동안 알려졌습니다. 2015년 야린 갈이 베이지안 딥러닝에 관한 박사 논문에서 순환 네트워크에 적절하게 드롭아웃을 사용하는 방법을 알아 내었습니다. 타임스텝마다 랜덤하게 드롭아웃 마스크를 바꾸는 것이 아니라 동일한 드롭아웃 마스크(동일한 유닛의 드롭 패턴)를 모든 타임스텝에 적용해야 합니다. GRU이나 LSTM 같은 순환 게이트에 의해 만들어지는 표현을 규제하려면 순환 층 내부 계산에 사용된 활성화 함수에 타임스텝마다 동일한 드롭아웃 마스크를 적용해야 합니다(순환 드롭 아웃 마스크). 모든 타임스텝에 동일한 드롭아웃 마스크를 적용하면 네트워크가 학습 오차를 타임스템에 걸쳐 적절하게 전파시킬 것입니다. 타임스텝마다 랜덤한 드롭아웃 마스크를 적용하면 오차 신호가 전파되는 것을 방해하고 학습 과정에 해를 끼칩니다.\n",
    "\n",
    "야린 갈은 케라스를 사용해 연구를 하였고 케라스 순환 층에 이 기능을 구현하는 데 도움을 주었습니다. 케라스에 있는 모든 순환 층은 두 개의 드롭아웃 매개변수를 가지고 있습니다. `dropout`은 층의 입력에 대한 드롭아웃 비율을 정하는 부동 소수 값입니다. `recurrent_dropout`은 순환 상태의 드롭아웃 비율을 정합니다. GRU 층에 드롭아웃과 순환 드롭아웃을 적용해 과대적합에 어떤 영향을 미치는지 살펴보겠습니다. 드롭아웃으로 규제된 네트워크는 언제나 완전히 수렴하는 데 더 오래 걸립니다. 에포크를 두 배 더 늘려 네트워크를 훈련하겠습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/40\n",
      "500/500 [==============================] - 124s 248ms/step - loss: 0.3395 - val_loss: 0.2812\n",
      "Epoch 2/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.3138 - val_loss: 0.2721\n",
      "Epoch 3/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.3080 - val_loss: 0.2682\n",
      "Epoch 4/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.3045 - val_loss: 0.2670\n",
      "Epoch 5/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.3012 - val_loss: 0.2713\n",
      "Epoch 6/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.3007 - val_loss: 0.2689\n",
      "Epoch 7/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.2968 - val_loss: 0.2657\n",
      "Epoch 8/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.2953 - val_loss: 0.2677\n",
      "Epoch 9/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.2909 - val_loss: 0.2628\n",
      "Epoch 10/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.2918 - val_loss: 0.2626\n",
      "Epoch 11/40\n",
      "500/500 [==============================] - 124s 247ms/step - loss: 0.2885 - val_loss: 0.2714\n",
      "Epoch 12/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.2882 - val_loss: 0.2623\n",
      "Epoch 13/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.2874 - val_loss: 0.2679\n",
      "Epoch 14/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.2872 - val_loss: 0.2615\n",
      "Epoch 15/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.2853 - val_loss: 0.2636\n",
      "Epoch 16/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.2845 - val_loss: 0.2660\n",
      "Epoch 17/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.2840 - val_loss: 0.2667\n",
      "Epoch 18/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.2825 - val_loss: 0.2617\n",
      "Epoch 19/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.2809 - val_loss: 0.2633\n",
      "Epoch 20/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.2808 - val_loss: 0.2666\n",
      "Epoch 21/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.2793 - val_loss: 0.2652\n",
      "Epoch 22/40\n",
      "500/500 [==============================] - 124s 247ms/step - loss: 0.2788 - val_loss: 0.2660\n",
      "Epoch 23/40\n",
      "500/500 [==============================] - 124s 247ms/step - loss: 0.2770 - val_loss: 0.2661\n",
      "Epoch 24/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.2768 - val_loss: 0.2635\n",
      "Epoch 25/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.2769 - val_loss: 0.2649\n",
      "Epoch 26/40\n",
      "500/500 [==============================] - 124s 247ms/step - loss: 0.2772 - val_loss: 0.2647\n",
      "Epoch 27/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.2753 - val_loss: 0.2647\n",
      "Epoch 28/40\n",
      "500/500 [==============================] - 124s 247ms/step - loss: 0.2755 - val_loss: 0.2648\n",
      "Epoch 29/40\n",
      "500/500 [==============================] - 124s 247ms/step - loss: 0.2733 - val_loss: 0.2642\n",
      "Epoch 30/40\n",
      "500/500 [==============================] - 124s 247ms/step - loss: 0.2743 - val_loss: 0.2656\n",
      "Epoch 31/40\n",
      "500/500 [==============================] - 124s 247ms/step - loss: 0.2734 - val_loss: 0.2666\n",
      "Epoch 32/40\n",
      "500/500 [==============================] - 123s 247ms/step - loss: 0.2720 - val_loss: 0.2674\n",
      "Epoch 33/40\n",
      "500/500 [==============================] - 124s 247ms/step - loss: 0.2722 - val_loss: 0.2670\n",
      "Epoch 34/40\n",
      "500/500 [==============================] - 124s 247ms/step - loss: 0.2742 - val_loss: 0.2648\n",
      "Epoch 35/40\n",
      "500/500 [==============================] - 124s 247ms/step - loss: 0.2711 - val_loss: 0.2656\n",
      "Epoch 36/40\n",
      "500/500 [==============================] - 124s 247ms/step - loss: 0.2689 - val_loss: 0.2613\n",
      "Epoch 37/40\n",
      "500/500 [==============================] - 124s 247ms/step - loss: 0.2712 - val_loss: 0.2678\n",
      "Epoch 38/40\n",
      "500/500 [==============================] - 124s 247ms/step - loss: 0.2703 - val_loss: 0.2667\n",
      "Epoch 39/40\n",
      "500/500 [==============================] - 124s 247ms/step - loss: 0.2696 - val_loss: 0.2691\n",
      "Epoch 40/40\n",
      "500/500 [==============================] - 124s 247ms/step - loss: 0.2689 - val_loss: 0.2658\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.GRU(32,\n",
    "                     dropout=0.2,\n",
    "                     recurrent_dropout=0.2,\n",
    "                     input_shape=(None, float_data.shape[-1])))\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=40,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xl8VNXZwPHfQ1gii6AQNxCDSqthEWJELCiC6IuioBYRBAWrRVDUFm3FpRaxvFr0RapSlbaiVZZSqUrd0CotpSoQkMWAFNQAAYQAsq9JnvePcycZwmTmJjPJTDLP9/OZT+buZ+5knnvuuWcRVcUYY0xyqBXvBBhjjKk6FvSNMSaJWNA3xpgkYkHfGGOSiAV9Y4xJIhb0jTEmiVjQN+UiIikisldEWsZy3XgSkbNFJOZ1l0Wkp4jkBk2vFpGL/axbgWP9UUQequj2Yfb7GxF5Jdb7NfFTO94JMJVLRPYGTdYHDgGF3vQdqjq1PPtT1UKgYazXTQaq+sNY7EdEbgcGq+qlQfu+PRb7NjWfBf0aTlWLg66Xk7xdVf9R1voiUltVC6oibcaYqmfFO0nOu33/i4hMF5E9wGARuUhEPheRnSKyWUSeFZE63vq1RURFJN2bft1b/r6I7BGRz0SkVXnX9ZZfKSL/FZFdIvKciPxHRIaWkW4/abxDRNaKyPci8mzQtiki8oyIbBeRr4FeYc7PIyIyo9S8SSIywXt/u4is8j7P114uvKx95YnIpd77+iLympe2HOD8EMf9xttvjoj08ea3A54HLvaKzrYFndsxQdsP9z77dhF5S0RO9XNuIhGRa7307BSRT0Tkh0HLHhKRTSKyW0S+CvqsnUVkiTd/i4g85fd4phKoqr2S5AXkAj1LzfsNcBi4BpcJOA64ALgQdyd4JvBfYKS3fm1AgXRv+nVgG5AF1AH+ArxegXVPAvYAfb1lo4AjwNAyPoufNL4NNAbSgR2Bzw6MBHKAFkBTYJ77KYQ8zpnAXqBB0L63Alne9DXeOgL0AA4A7b1lPYHcoH3lAZd6758G/gmcAJwBrCy1bn/gVO87uclLw8nestuBf5ZK5+vAGO/9FV4aOwCpwO+BT/ycmxCf/zfAK977c7109PC+o4e8814HaAOsA07x1m0FnOm9XwQM9N43Ai6M928hmV+W0zcA81X176papKoHVHWRqi5Q1QJV/QaYDHQLs/0bqpqtqkeAqbhgU951rwaWqurb3rJncBeIkHym8QlV3aWqubgAGzhWf+AZVc1T1e3Ak2GO8w3wJe5iBHA5sFNVs73lf1fVb9T5BPgYCPmwtpT+wG9U9XtVXYfLvQcfd6aqbva+k2m4C3aWj/0CDAL+qKpLVfUgMBroJiItgtYp69yEMwCYraqfeN/Rk8DxuItvAe4C08YrIvzWO3fgLt6tRaSpqu5R1QU+P4epBBb0DcCG4AkROUdE3hWR70RkNzAWaBZm+++C3u8n/MPbstY9LTgdqqq4nHFIPtPo61i4HGo404CB3vubcBerQDquFpEFIrJDRHbictnhzlXAqeHSICJDRWSZV4yyEzjH537Bfb7i/anqbuB7oHnQOuX5zsrabxHuO2quqquB+3Dfw1avuPAUb9VbgQxgtYgsFJGrfH4OUwks6Btwt/vBXsLlbs9W1eOBR3HFF5VpM664BQAREY4OUqVFk8bNwOlB05GqlP4F6OnllPviLgKIyHHAG8ATuKKXJsCHPtPxXVlpEJEzgReAEUBTb79fBe03UvXSTbgio8D+GuGKkTb6SFd59lsL951tBFDV11W1C65oJwV3XlDV1ao6AFeE93/ALBFJjTItpoIs6JtQGgG7gH0ici5wRxUc8x0gU0SuEZHawL1AWiWlcSbwMxFpLiJNgQfCrayqW4D5wBRgtaqu8RbVA+oC+UChiFwNXFaONDwkIk3EtWMYGbSsIS6w5+Ouf7fjcvoBW4AWgQfXIUwHbhOR9iJSDxd8/62qZd45lSPNfUTkUu/Yv8A9h1kgIueKSHfveAe8VyHuA9wsIs28O4Nd3mcrijItpoIs6JtQ7gOG4H7QL+FyupXKC6w3AhOA7cBZwBe4dgWxTuMLuLL3FbiHjG/42GYa7sHstKA07wR+DryJexjaD3fx8uPXuDuOXOB94M9B+10OPAss9NY5BwguB/8IWANsEZHgYprA9h/gilne9LZviSvnj4qq5uDO+Qu4C1IvoI9Xvl8PGI97DvMd7s7iEW/Tq4BV4mqHPQ3cqKqHo02PqRhxRafGJBYRScEVJ/RT1X/HOz3G1BSW0zcJQ0R6iUhjr4jgV7gaIQvjnCxjahQL+iaRdAW+wRUR9AKuVdWyineMMRVgxTvGGJNELKdvjDFJJOE6XGvWrJmmp6fHOxnGGFOtLF68eJuqhqvmDCRg0E9PTyc7OzveyTDGmGpFRCK1LAeseMcYY5KKBX1jjEkiFvSNMSaJJFyZvjGmah05coS8vDwOHjwY76QYH1JTU2nRogV16pTV9VJ4voK+iPQCfofrOe+PqvpkqeXDgbtwHSztBYap6sqg5S1xg0SMUdWnK5RSY0ylyMvLo1GjRqSnp+M6NzWJSlXZvn07eXl5tGrVKvIGIUQs3vH6QJkEXInrE3ugiGSUWm2aqrZT1Q64TpcmlFr+DK5TqUozdSqkp0OtWu7v1HIN921M8jp48CBNmza1gF8NiAhNmzaN6q7MT06/E7A2MAqON15oX1zOHSgepCGgAUH9fYvItbim9fsqnMoIpk6FYcNg/343vW6dmwYYFHXfgsbUfBbwq49ovys/D3Kbc/QIP3mEGNxCRO7yBpkeD9zjzWuA66v8sXAHEJFhIpItItn5+fl+017s4YdLAn7A/v1uvjHGmBJ+gn6oy8oxHfao6iRVPQsX5AP9aD+GG4t0b7gDqOpkVc1S1ay0tIgNyo6xfn355htjEsf27dvp0KEDHTp04JRTTqF58+bF04cP++t2/9Zbb2X16tVh15k0aRJTY1Tu27VrV5YuXRqTfVU1P8U7eRw9rFsLXD/nZZmBG2QB3IDJ/URkPNAEKBKRg6r6fJlbV0DLlq5IJ9R8Y0xsTZ3q7qLXr3e/sXHjoitGbdq0aXEAHTNmDA0bNuT+++8/ah1VRVWpVSt0PnXKlCkRj3PXXXdVPJE1iJ+c/iLcSPatRKQuMACYHbyCiLQOmuyNG9UHVb1YVdNVNR2YCPxvrAM+uH+6+vWPnle/vptvjImdwPOzdetAteT5WWVUnFi7di1t27Zl+PDhZGZmsnnzZoYNG0ZWVhZt2rRh7NixxesGct4FBQU0adKE0aNHc95553HRRRexdetWAB555BEmTpxYvP7o0aPp1KkTP/zhD/n0008B2LdvHz/+8Y8577zzGDhwIFlZWRFz9K+//jrt2rWjbdu2PPTQQwAUFBRw8803F89/9tlnAXjmmWfIyMjgvPPOY/DgwTE/Z35EzOmraoGIjATm4KpsvqyqOSIyFshW1dnASBHpCRwBvscNqVZlArmMWOY+jDHHCvf8rDJ+bytXrmTKlCm8+OKLADz55JOceOKJFBQU0L17d/r160dGxtGVCXft2kW3bt148sknGTVqFC+//DKjR48+Zt+qysKFC5k9ezZjx47lgw8+4LnnnuOUU05h1qxZLFu2jMzMzLDpy8vL45FHHiE7O5vGjRvTs2dP3nnnHdLS0ti2bRsrVqwAYOfOnQCMHz+edevWUbdu3eJ5Vc1Xi1xVfU9Vf6CqZ6nqOG/eo17AR1XvVdU2qtpBVbt7Y2mW3kel1tEfNAhyc6GoyP21gG9M7FX187OzzjqLCy64oHh6+vTpZGZmkpmZyapVq1i5cuUx2xx33HFceeWVAJx//vnk5uaG3Pf1119/zDrz589nwIABAJx33nm0adMmbPoWLFhAjx49aNasGXXq1OGmm25i3rx5nH322axevZp7772XOXPm0LhxYwDatGnD4MGDmTp1aoUbV0XLumEwxvhW1nOyynp+1qBBg+L3a9as4Xe/+x2ffPIJy5cvp1evXiHrq9etW7f4fUpKCgUFBSH3Xa9evWPWKe+gUmWt37RpU5YvX07Xrl159tlnueOOOwCYM2cOw4cPZ+HChWRlZVFYWFiu48WCBX1jjG/xfH62e/duGjVqxPHHH8/mzZuZM2dOzI/RtWtXZs6cCcCKFStC3kkE69y5M3PnzmX79u0UFBQwY8YMunXrRn5+PqrKDTfcwGOPPcaSJUsoLCwkLy+PHj168NRTT5Gfn8/+0mVlVcD63jHG+BbP52eZmZlkZGTQtm1bzjzzTLp06RLzY9x9993ccssttG/fnszMTNq2bVtcNBNKixYtGDt2LJdeeimqyjXXXEPv3r1ZsmQJt912G6qKiPDb3/6WgoICbrrpJvbs2UNRUREPPPAAjRo1ivlniCThxsjNyspSG0TFmKqzatUqzj333HgnIyEUFBRQUFBAamoqa9as4YorrmDNmjXUrp1Y+eNQ35mILFbVrEjbJtYnMcaYONq7dy+XXXYZBQUFqCovvfRSwgX8aNWsT2OMMVFo0qQJixcvjncyKpU9yDXGmCRiQd8YY5KIBX1jjEkiFvSNMSaJWNA3xsTVpZdeekxDq4kTJ3LnnXeG3a5hw4YAbNq0iX79+pW570hVwCdOnHhUI6mrrroqJv3ijBkzhqefTrzRYS3oG2PiauDAgcyYMeOoeTNmzGDgwIG+tj/ttNN44403Knz80kH/vffeo0mTJhXeX6KzoG+Miat+/frxzjvvcOjQIQByc3PZtGkTXbt2La43n5mZSbt27Xj77beP2T43N5e2bdsCcODAAQYMGED79u258cYbOXDgQPF6I0aMKO6W+de//jUAzz77LJs2baJ79+50794dgPT0dLZt2wbAhAkTaNu2LW3bti3uljk3N5dzzz2Xn/70p7Rp04YrrrjiqOOEsnTpUjp37kz79u257rrr+P7774uPn5GRQfv27Ys7evvXv/5VPIhMx44d2bNnT4XPbShWT98YU+xnP4NYDwjVoQN48TKkpk2b0qlTJz744AP69u3LjBkzuPHGGxERUlNTefPNNzn++OPZtm0bnTt3pk+fPmWOE/vCCy9Qv359li9fzvLly4/qGnncuHGceOKJFBYWctlll7F8+XLuueceJkyYwNy5c2nWrNlR+1q8eDFTpkxhwYIFqCoXXngh3bp144QTTmDNmjVMnz6dP/zhD/Tv359Zs2aF7R//lltu4bnnnqNbt248+uijPPbYY0ycOJEnn3ySb7/9lnr16hUXKT399NNMmjSJLl26sHfvXlJTU8txtiOznL4xJu6Ci3iCi3ZUlYceeoj27dvTs2dPNm7cyJYtW8rcz7x584qDb/v27Wnfvn3xspkzZ5KZmUnHjh3JycmJ2Jna/Pnzue6662jQoAENGzbk+uuv59///jcArVq1okOHDkD47pvB9e+/c+dOunXrBsCQIUOYN29ecRoHDRrE66+/Xtzyt0uXLowaNYpnn32WnTt3xrxFsOX0jTHFwuXIK9O1117LqFGjWLJkCQcOHCjOoU+dOpX8/HwWL15MnTp1SE9PD9mdcrBQdwHffvstTz/9NIsWLeKEE05g6NChEfcTrl+yQLfM4LpmjlS8U5Z3332XefPmMXv2bB5//HFycnIYPXo0vXv35r333qNz58784x//4JxzzqnQ/kPxldMXkV4islpE1orIMUPQiMhwEVkhIktFZL6IZHjzO3nzlorIMhG5LmYpN8bUGA0bNuTSSy/lJz/5yVEPcHft2sVJJ51EnTp1mDt3LutCDYYd5JJLLike/PzLL79k+fLlgOuWuUGDBjRu3JgtW7bw/vvvF2/TqFGjkOXml1xyCW+99Rb79+9n3759vPnmm1x88cXl/myNGzfmhBNOKL5LeO211+jWrRtFRUVs2LCB7t27M378eHbu3MnevXv5+uuvadeuHQ888ABZWVl89dVX5T5mOBFz+iKSAkwCLscNkr5IRGaravC90TRVfdFbvw8wAegFfAlkeUMungosE5G/q2roUQ2MMUlr4MCBXH/99UfV5Bk0aBDXXHMNWVlZdOjQIWKOd8SIEdx66620b9+eDh060KlTJ8CNgtWxY0fatGlzTLfMw4YN48orr+TUU09l7ty5xfMzMzMZOnRo8T5uv/12OnbsGLYopyyvvvoqw4cPZ//+/Zx55plMmTKFwsJCBg8ezK5du1BVfv7zn9OkSRN+9atfMXfuXFJSUsjIyCgeBSxWInatLCIXAWNU9X+86QcBVPWJMtYfCNyiqleWmt8K+BxoHi7oW9fKxlQt61q5+qnsrpWbAxuCpvOAC0uvJCJ3AaOAukCPoPkXAi8DZwA3hwr4IjIMGAbQsrLGXTPGGOOrTD9U3ahjbg9UdZKqngU8ADwSNH+BqrYBLgAeFJFj6h+p6mRVzVLVrLS0NP+pN8YYUy5+gn4ecHrQdAtgU5j1ZwDXlp6pqquAfUDb8iTQGFP5Em0EPVO2aL8rP0F/EdBaRFqJSF1gADA7eAURaR002RtY481vJSK1vfdnAD8EcqNKsTEmplJTU9m+fbsF/mpAVdm+fXtUDbYilul7NW9GAnOAFOBlVc0RkbFAtqrOBkaKSE/gCPA9MMTbvCswWkSOAEXAnaq6rcKpNcbEXIsWLcjLyyM/Pz/eSTE+pKam0qJFiwpvbwOjG2NMDeC39o51w2CMMUnEgr4xxiQRC/rGGJNELOgbY0wSsaBvjDFJxIK+McYkEQv6xhiTRCzoG2NMErGgb4wxScSCvjHGJBEL+sYYk0Qs6BtjTBKxoG+MMUnEgr4xxiQRC/rGGJNELOgbY0wS8RX0RaSXiKwWkbUiMjrE8uEiskJElorIfBHJ8OZfLiKLvWWLRaRHrD+AMcYY/yIGfRFJASYBVwIZwMBAUA8yTVXbqWoHYDwwwZu/DbhGVdvhhlB8LWYpN8YYU25+cvqdgLWq+o2qHgZmAH2DV1DV3UGTDQD15n+hqpu8+TlAqojUiz7ZxhhjKiLiwOhAc2BD0HQecGHplUTkLmAUUBcIVYzzY+ALVT0UYtthwDCAli1b+kiSMcaYivCT05cQ844ZTV1VJ6nqWcADwCNH7UCkDfBb4I5QB1DVyaqapapZaWlpPpJkjDGmIvwE/Tzg9KDpFsCmMtYFV/xzbWBCRFoAbwK3qOrXFUmkMcaY2PAT9BcBrUWklYjUBQYAs4NXEJHWQZO9gTXe/CbAu8CDqvqf2CTZGGNMRUUM+qpaAIwE5gCrgJmqmiMiY0Wkj7faSBHJEZGluHL9IYH5wNnAr7zqnEtF5KTYfwxjjDF+iOoxxfNxlZWVpdnZ2fFOhjHGVCsislhVsyKtZy1yjTEmiVjQN8aYJGJB3xhjkogFfWOMSSIW9I0xJolY0DfGmCRiQd8YY5JIUgT9qVMhPR1q1XJ/p06Nd4qMMSY+/PSyWa1NnQrDhsH+/W563To3DTBoUPzSZYwx8VDjc/oPP1wS8AP273fzjTEm2dT4oL9+ffnmG2NMTVbjg35ZY7LYWC3GmGRU44P+uHFQv/7R8+rXd/ONMSbZ1PigP2gQTJ4MZ5wBIu7v5Mn2ENcYk5xqfO0dcAHegrwxxiRBTt8YY0wJC/rGGJNEfAV9EeklIqtFZK2IjA6xfLiIrPCGQ5wvIhne/KYiMldE9orI87FOvDHGmPKJGPRFJAWYBFwJZAADA0E9yDRVbaeqHYDxwARv/kHgV8D9sUuyMcaYivKT0+8ErFXVb1T1MDAD6Bu8gqruDppsAKg3f5+qzscFf2OMMXHmp/ZOc2BD0HQecGHplUTkLmAUUBfoUZ5EiMgwYBhAS2s1ZYwxlcZPTl9CzNNjZqhOUtWzgAeAR8qTCFWdrKpZqpqVlpZWnk2NMcaUg5+gnwecHjTdAtgUZv0ZwLXRJKqqWdfLxphk4SfoLwJai0grEakLDABmB68gIq2DJnsDa2KXxMoV6Hp53TpQLel62QK/MaYmihj0VbUAGAnMAVYBM1U1R0TGikgfb7WRIpIjIktx5fpDAtuLSC6uNs9QEckLUfMnrqzrZWNMMhHVY4rn4yorK0uzs7Or7Hi1arkcfmkiUFRUZckwxpioiMhiVc2KtF7St8i1rpeNMckk6YO+db1sjEkmSR/0retlY0wySYqulSOxrpeNMcki6XP6xhiTTCzo+2CNt4wxNYUV70QQaLwVqMsfaLwFViRkjKl+LKcfgTXeMsbUJBb0I1i/vnzzjTEmkVnQj8AabxljahIL+hFY4y1jTE1iQT8Ca7xljKlJrPaOD9Z4yxhTU1hOPwasHr8xprqwoB8lP4Ow2EXBGJMoLOhHKVI9fhuZyxiTSHwFfRHpJSKrRWStiIwOsXy4iKwQkaUiMj94dCwRedDbbrWI/E8sE58IItXjt8ZdxphEEjHoi0gKMAm4EsgABoYY8nCaqrZT1Q7AeNzwiHjrDQDaAL2A33v7qzEi1eO3xl3GmETiJ6ffCVirqt+o6mFgBtA3eAVV3R002QAIDEDYF5ihqodU9Vtgrbe/GiNSPX5r3GWMSSR+gn5zYEPQdJ437ygicpeIfI3L6d9Tzm2HiUi2iGTn5+f7TXtCiFSP3xp3GWMSiZ+gLyHmHTOUuKpOUtWzgAeAR8q57WRVzVLVrLS0NB9JSiyDBkFurhtIPTf36Dr91rjLGJNI/DTOygNOD5puAWwKs/4M4IUKblsjWeMuY0yi8JPTXwS0FpFWIlIX92B2dvAKItI6aLI3sMZ7PxsYICL1RKQV0BpYGH2yjTHGVETEnL6qFojISGAOkAK8rKo5IjIWyFbV2cBIEekJHAG+B4Z42+aIyExgJVAA3KWqhZX0WYwxxkQgqscUscdVVlaWZmdnxzsZVWbqVFdnf/16V6Nn3DgrCjLGlJ+ILFbVrEjrWYdrcWRDMRpjqpp1wxBH1lrXGFPVLOjHkbXWNcZUNQv6ceSnta710GmMiSUL+nEUqbWu9dBpjIk1C/pxFKm1rpX5G2NizapsJrBatVwOvzQR1+WDMcYE+K2yaTn9BGY9dBpjYs2CfgKzHjqNMbFmQT+BWQ+dxphYsxa5Cc566DTGxJLl9I0xJolY0K/hrHGXMSaYFe/UYNahmzGmNMvpV3PhcvLWuMsYU5rl9KuxSDl569DNGFOar5y+iPQSkdUislZERodYPkpEVorIchH5WETOCFr2WxH50nvdGMvEJ7tIOXlr3GWMKS1i0BeRFGAScCWQAQwUkYxSq30BZKlqe+ANYLy3bW8gE+gAXAj8QkSOj13yk1uknLw17jLGlOYnp98JWKuq36jqYWAG0Dd4BVWdq6qBPOfnQAvvfQbwL1UtUNV9wDKgV2ySbiLl5K1xlzGmND9BvzmwIWg6z5tXltuA9733y4ArRaS+iDQDugOnl95ARIaJSLaIZOfn5/tLufGVkx80CHJzXQdtubkW8I1Jdn6CvoSYF7JrThEZDGQBTwGo6ofAe8CnwHTgM6DgmJ2pTlbVLFXNSktL85l0E4ucvNXjNya5+An6eRydO28BbCq9koj0BB4G+qjqocB8VR2nqh1U9XLcBWRNdEk2waLJyfsZpMUuCsbULH6C/iKgtYi0EpG6wABgdvAKItIReAkX8LcGzU8Rkabe+/ZAe+DDWCXeRCdS7R8bucuYmidi0FfVAmAkMAdYBcxU1RwRGSsifbzVngIaAn8VkaUiErgo1AH+LSIrgcnAYG9/JgFEqv3j56IQ7i7A7hKMSTw2clYSS093uffSzjjDFRWFG7nrtdeObhgG7iFy4JlC6YZjpZcbY2LLRs4yEUWq/ROuSmikuwDrAsKYxGRBP4lFqv0T7qIQqWjIuoAwJjFZ0E9y4Wr/hLsoRGoYZl1AGJOYLOibsMq6KEQqGvLTcMwe9BpT9SzomwqJVDQUablVBzUmPqz2jomLSDWHwF0AHn7YPQdo2dLdJVjNH2NCs9o7JqFFetBrrYWNqRwW9E1cRHrQa62FjakcFvRNXER60Btta2FjTGgW9E1cRHrQG+lOwNoBGFMxFvRN3IRrIxBNa2Gw8n5jymJB3ySkaFoLW3m/MWWzKpum2iqrSqef6qDG1DR+q2xa0Dc1TrjeQYuKqj49xlQFq6dvkpaffn+szN8kqxoT9AsK4B//gI0b450SE2+RHgJbwy+TzHwFfRHpJSKrRWStiIwOsXyUiKwUkeUi8rGInBG0bLyI5IjIKhF5VkRCDbQetY0b4fLL3eAeJrlFeghsDb9MMotYpi8iKcB/gctxg6QvAgaq6sqgdboDC1R1v4iMAC5V1RtF5Ee4oRQv8VadDzyoqv8s63jRlOlfdBEcOgRLllRoc5MkIpX524NgUx3Fsky/E7BWVb9R1cPADKBv8AqqOldVA3mnz4EWgUVAKlAXqIcbM3eLv49Qfv37wxdfwJo1lXUEUxPEouGXFf+Y6spP0G8ObAiazvPmleU24H0AVf0MmAts9l5zVHVVxZIa2Q03uL8zZ1bWEUxNEIuGX1b8Y6orP0E/VBl8yDIhERkMZOGKdBCRs4FzcTn/5kAPEbkkxHbDRCRbRLLz8/P9pv0YLVpAly4W9E140TT8An/9/tidgElUfoJ+HnB60HQLYFPplUSkJ/Aw0EdVD3mzrwM+V9W9qroXdwfQufS2qjpZVbNUNSstLa28n+Eo/fvD8uXw1VdR7cbUcBUdJhJi0y20MfHiJ+gvAlqLSCsRqQsMAGYHryAiHYGXcAF/a9Ci9UA3EaktInWAbkClFe8A9OvnfqiW2zfRCHdRiLZbaGPiKWLQV9UCYCQwBxewZ6pqjoiMFZE+3mpPAQ2Bv4rIUhEJXBTeAL4GVgDLgGWq+vdYf4hgp50GF19sQd9Unmi7hTYmnnzV01fV91T1B6p6lqqO8+Y9qqqzvfc9VfVkVe3gvfp48wtV9Q5VPVdVM1R1VOV9lBL9+0NOjnsZE2vRdgtd2eX99jzBhKWqCfU6//zzNVqbN6vWqqX66KNR78qYcnv9ddX69VVdib571a/v5odbFrz9GWeoiri/wcsiLfezf1MzAdnqI8bGPciXfsUi6Kuqdu+ues45qkVFMdmdMeVSVmA+44yjA3LgdcYZJduFC9qRlkfav6m5/AYUbneIAAAWM0lEQVT9GtvL5osvwogRsGwZtG8fg4QZEwPRtgaOtNx6GE1eSd/L5o9/7H4A9kDXJJJoWwNHWh5tD6P2PKDmq7FBPy0NevRwQT/BbmZMEou2NXCk5dH0MGrtC5KEnzKgqnzFqkxfVXXyZFeeuWRJzHZpTNSieRAb7YPgcGX+9jygeiPZH+Sqqm7bppqSojp6dMx2aUyli6b2TiQioQO7SPhlsUp7ZYrnsROB36BfYx/kBvTq5XrdXLvWPcwyJpmFexAM0XUpHSgeCm6NXL/+0W0YKks8j50okv5BbkD//vDNN7B4cbxTYkz8hSvzj/Q8IJJ4dkRnXV/4V+OD/nXXQZ06VovHGAjfmjhSS2MIH7Tj2RGddX3hX40v3gHo3dt1yfDtt1bEY0xFRSpCibaNQTRstDMr3jlK//7uH2LhwninxJjqK1IRSmV3RBfuLiPaoqlkkhRBv29fqFvXiniMiUakoB1tR3RQdmCPVDTkp2jKOElRvAPQpw8sXVrSVN0YUz7RFqFEKh4Kt/zhh634JhIr3inlxhthwwaYPj3eKTGmeoq2CCVSbjxc8VEsHtRGW3OoxnRR4acyf1W+Ytk4K9jhw6pdurjWi8uXx37/+fmq+/bFfr+VrahI9dVXVTdujHdKTHVQmQ2gwjUOi7a1cLRdTleHLquxFrnH2rRJ9ZRTVM8+W/X772O737Q01fR01cWLY7ffqrB0qfsvuOOOeKfEJLtwgT3aoBvtRcNPl9jxbg0c06AP9AJWA2uB0SGWjwJWAsuBj4EzvPndgaVBr4PAteGOVZlBX1V1/nzV2rVVr75atbAw+v0VFqpefrnqccepnn66ar16qn/8Y/T7rSoPPOD+C5o1Uz1yJN6pMcnMT79DldH9RLTbJ8pdQMyCPpCCG+f2TKAubqzbjFLrdAfqe+9HAH8JsZ8TgR2B9cp6VXbQV1V9/nn3yR97LPp9/d//uX29+KIr4rn8cjf9k5+o7t8f/f4rU1GR+/GccIJL84cfxjtFJtlVVo65MnP6fvZdFXcCsQz6FwFzgqYfBB4Ms35H4D8h5g8DpkY6XlUE/aIi1VtucV/AO+9UfD9ffKFat65q374lI3QVFLhhGkG1QwfVr7+OTZorw3/+49I5ebJqw4aqt90W7xQZUzkqs0w/0l1EVd0JxDLo9wP+GDR9M/B8mPWfBx4JMf8T4OoythkGZAPZLVu2jO2ZKMP+/S4oN2miumZN+bfft88Nx3jqqS6HX9q777ocdJMmqn//e/TprQx33aWamqq6e7fqoEEuvYcPxztVxlSOaHsvregQmFV1JxDLoH9DiKD/XBnrDgY+B+qVmn8qkA/UiXS8qsjpB3zzjQt07dqp7t1bvm2HD3dn76OPwu+/Y0e33sMPu7uARHHkiOpJJ6n26+emZ8926Xzvvfimy5h4iCY3HmnbqroTqPLiHaAnsAo4KcSye4HJfhJUlUFfVfWDD9zJv+km/4Oov/WWO3P33x953QMHXLEJuGMkSuCfM8eladYsN33woGrjxqpDhsQ1WcbERSyqhFZk4JpYHDsglkG/NvAN0CroQW6bUut09B72ti5jH58D3f0kqKqDvqrquHHuTDz9dOTAv2mTatOmLgd/8GD5j/HTn/q/uFSmoUNVjz/eXZQChgxxgb88n8uYmiAWA8iEcuCAao8ex+63Th3/dwJ++Q36EVvkqmoBMBKY4+XkZ6pqjoiMFZE+3mpPAQ2Bv4rIUhGZHdheRNKB04F/RTpWvIweDddeC/ffD23bwtNPw3ffHbteUREMGeJaCU6bBvXq+T/GQw/Bgw/CH/4A993nvtZ4OXgQ/vY3uP56SE0tmX/jjbBrF3z4of99bdgAzz8f389jTLT89AukCm+/7QZk8iM3Fy6+GD75BK6+umRfxx0HR45AYaH/Y8eUnytDVb7ikdNXdVfkyZNVL7rIXWVTUlxd/lmzVA8dcusEV8+siKIi1bvvdvt49FF/23z/veqwYapt2qhu3Vqx45b2t7+5NMyZc/T8Q4fcM47Bg/3vq3dvt69oakEZE7B5s+qbb6r+85+qublV13bET7n6u++WLLv4YtWXX3aVIEJ5/33VE090d9Nvvnn0soMHVXv2VK1Vy/0WE65Mv6pf8Qr6wVatcuPqnnaaO0PNmrnAW7p6ZkUUFro6/KA6fnz4dWfNci2Ia9VyDcrKE4zDueEG9xA31A/qtttUGzU6utinLP/4h/sctWq5i2UiFFuZ6mXPHhdMf/5zV6GidBFH7dqqrVqpdu/ufjdjx6q+8UblPBuLVIOma1fXAPOJJ1R/8AOXvgYNXLHo3Lnut11QoPrrX7t9tG9fds3APXtUO3d2MeWjjxKs9k5VvxIh6AccOeJqs9xwg/tyTjstdPXM8iooUL3xRnf2f//7Y5dv3Kh63XVueYcOqtnZqr/6lZuOtgHVrl2umuZdd4Ve/uGH7jh/+1v4/RQWurSlp5fcAX3ySXRpK4/vv1f9+GN3d1ZZjeDWrVNdv75y9p2sCgtVFyxQffxx1UsucWXb4FqyX3aZ6pNPuvYjH33kvtuHHnIVIC66yFWPDlwMunWr2u/m3/92x/3d79x0UZHqp5+6Z3SNGrllrVqp/uhH7v2QIZH74tqxw13oGjRw+4qWBf0Y27FDddu22O3v8GHVa65x38Crr7p5hYWqL73kbglTU1V/+9uSevMHDrjcxZlnRtex25//7I45f37o5UeOuDubAQPC7+fVV91+pk1zQffkk90ta2XYs0d13jzVCRNUBw5Ubd366NzgfffF/pjLlrk2Fikp7phLlsT+GMni0CFXlDhiRMnds4hqZqbqL3/pArzfC/f+/ap/+pNrTNikieqMGZWb9oDevd3vItRvb98+lzO/7DJXyWPyZP93vZs3u77AmjRx/3PRsKBfDRw44P5RatVyOYhLLnHfSPfuoW8LP/nELR89uuLHvPJK1ZYtw/c7dMcdLvdR1sVl3z7VFi1UL7igZD/jx7u0LVhQ8bSVtmqVy+HVqlUS4E8/3d0FjRvnAsnQoS4wxzIor1njLmLNm7tih+OPd8e+7DJXVhvuB717t2vzcPfdLqj94AcuB9i8uStSa9LEndu6dd30++/HLt2JZM8e1b/+1eXSGzfW4nLq669Xfe216O+Y165VvfBCt99bbnF3sJVl2TJ3nLFjK2f/337r/j9OPln1v/+t+H4s6FcTe/eW3BI2aeJyMeGCytChrpyzIt1D5+e7bR94IPx6gYvLzJmhlweqn/7rXyXzdu92D4H79Cl/ukKZNcvdNqeluTLSd95xuaLSduxwwfOCC2JTzrtxoyuyatpUdeVKN2/nTndRC+RS27VzdzqHDrk7o88+cwGha1d3fsF1wNezp7tjuvlmVx49fLi7GNx3n7twn3eeu6A980zNeR6ycKFq//6uuAbcebz1VnchjHUx3OHDrkJErVruwhquiGTHDldU+8gj5b87uOkmd2exY0d06Q1n5Up3J9GmTcX/jy3oVyM7d7py8U2bIq+7bZv75+jcufy9hL7wgvvGly4Nv15Bgct1BFrrBtuyxQXja689dtmvf+32H814BUeOuFt+UO3USXXDhsjbTJvm1n/uuYofV1V1+3b3o2vYUHXRomOXHzqk+sorqm3buuOdckpJLlZE9fzzXTD/+GN/bR327nU5X3AP0AO1xKqboiJ3x9K9u/ssjRur3nOPq4FTFbVv5s93F+qUFPc/ePiwyzFPmeLK3Nu0KblTDNTM++wzf/v++mt3UamMIsTSFi/2n65Q/Ab9pBkusSZ5/XW4+WaYNAnuvNP/dt26QX4+5OS4kYvCGTkSXn4Ztm6Fhg1L5t91F7z0ktvHD3949DY7drjRkK65xrVjKK+tW2HgQFevefhwmDjRX1sIVejVCz77DFatgubNy3/svXuhZ083pOb770P37uGP98EHbtSnpk3hiiugRw9o1qz8xy0qgjFj4PHHXZ3uWbMgLa3s9Q8fhjfecN/9+vXQuDE0aVLyN/C+cWN37lJSQr9EYN8+2LMHdu8ueQWmCwtdm5WsLDj/fPddp6QcnZYjR9y40+PHw/Ll7rz//Ofw05/C8ceX/1xEY/du9z/72muuHvyBA25+kyZw0UXQpQv86Efuc/zoR270qy++cOcpnDvvhD/9Cb79Fk47rfI/RzT8DpcY95x96Vcy5vTLq6jIdeF8/PGqeXn+tlm/XstVLjlvnlt/+vSSeatWuVxSWTV/VFV/8QuXMypvJ3aff+6eE6Smuhxaea1d67b98Y/Lv22g3nRKiutiIx6mT3fpT08PfaeUn++K1QJFTK1bu6K+6693LT4zM1XPOssVpwSKmPy+RFzu/PTTVTMy3F1kp05H1x1v0MAVX917ryuTnzixpPuAjAz3nSXCncrMme6Z1OTJqjk5oe+G//Mf911H6nrlu+9cMdXtt1deemMJK96p2QJB7vrr/a3/1FPu2/YbjAsLXYAJLsbp08cV7YRrJLZpk/uh+O2muajINXarW9cFvGgeyP7v/7rPOHu2/22OHCkpYgnUooqXhQtdtcSGDUs+w4oVLuikpro0Xn65q9cermivqMgVHe3Y4S4W333nnlWsX+8eGq5d64o/Nm1y65UV+I4cccd/5RX3LOKii9yzisCFoGtXl85YDEZU1R5/PPJ3Pnq0uyBG83C1KlnQTwKBIPf225HXzcxUzcoq3/7vvdcF8F27XOMTcA1TIrnzTlf/OlI96t27Xc0LUO3Vy5WpR+PQIVd+27Klqz0SSVFRSUO5iROjO3as5OW570nE5bbBBfxhw1S//DLeqSu5EER6LpToCgpcXf8GDUIH9Z073Z30DTdUedIqzIJ+Ejh82D1UbNGi7ObgqqqrV7tvesKE8u3/009LckPnn+9u//3UwMjNdUUM99xT9jrz57saF7VquRoYsWphOX+++qq7n5NT0gDOb5cYVWXfPje+QcuW7sIeiwaB5lgbNriuEs4//9iiqSeecP8b1al9hgX9JPHZZy5XeMUVqmPGuOp/L7/sqjx+/LFrzTtqlFtn48by7buw0AX6tDT3n/LnP/vfduhQVxSwZcvR8w8fdtXmAtXsymokFo1hw8quu5+T46pRirhilHHjak51SVN+b77p/rd/8YuSeYHGhv/zP/FLV0VY0E8iY8a429RwD+suvbRi+77vPrd9x47lK7v96isXWIMbkq1e7YouwF0UKqtBTai6+ytXupa1Iu5cPfig5aCNM2KE+58MdED4+9+76blz45qscvMb9K3KZg1SUOCqru3adezr4oshPb38+1yxArp2hb//HS65pHzb9u/vqjauWwd/+YvrUjo11VX57Nev/Gkpj+nT4aabXLfZ69bBjBlQvz7cfbdLR0WqV5qa6cABuOAC2LYNlixx/+8nnwyffhq5anMi8Vtl04K+iUi1Yv/8S5dCx46uX/D16+Hyy2HKlIrVoy8v9eruf/ghNGhgwd6Et2KFC/wnneTGiHjrLejbN96pKh+/QT/iICrGVDS306ED9OkDW7a4hlYffFA1AR9cml95BZ55xjWseeIJC/imbO3awYQJLuBnZLgGhjVVbT8riUgv4HdACm6Q9CdLLR8F3A4U4AZA/4mqrvOWtQT+iBs9S4GrVDU3Vh/AJLZp01xr15NPrvpjn3oq/OxnVX9cUz2NGOH+V7t1cy12a6qIQV9EUoBJwOVAHrBIRGar6sqg1b4AslR1v4iMAMYDN3rL/gyMU9WPRKQhUBTTT2ASWoMG7mVMohOBX/4y3qmofH6uZ52Atar6jaoeBmYAR5V2qepcVd3vTX4OtAAQkQygtqp+5K23N2g9Y4wxVcxP0G8ObAiazvPmleU24H3v/Q+AnSLyNxH5QkSe8u4cjiIiw0QkW0Sy8/Pz/abdGGNMOfkJ+qEe44Ws8iMig4Es4ClvVm3gYuB+4ALgTGDoMTtTnayqWaqalRaui0FjjDFR8RP083APYQNaAJtKryQiPYGHgT6qeiho2y+8oqEC4C0gM7okG2OMqSg/QX8R0FpEWolIXWAAMDt4BRHpCLyEC/hbS217gogEsu89gOAHwMYYY6pQxKDv5dBHAnOAVcBMVc0RkbEi0sdb7SmgIfBXEVkqIrO9bQtxRTsfi8gKXFHRHyrhcxhjjPHBWuQaY0wNYC1yjTHGHCPhcvoikg+sC7NKM2BbFSWnvCxtFWNpqxhLW8XU1LSdoaoRqz8mXNCPRESy/dzCxIOlrWIsbRVjaauYZE+bFe8YY0wSsaBvjDFJpDoG/cnxTkAYlraKsbRVjKWtYpI6bdWuTN8YY0zFVcecvjHGmAqyoG+MMUmk2gR9EeklIqtFZK2IjI53ekoTkVwRWeF1QxHXJsUi8rKIbBWRL4PmnSgiH4nIGu/vCQmUtjEistE7d0tF5Ko4pOt0EZkrIqtEJEdE7vXmx/28hUlbIpy3VBFZKCLLvLQ95s1vJSILvPP2F6/frkRJ2ysi8m3QeetQ1WkLSmOK1+38O9505Z83VU34F26Yxq9xXTPXBZYBGfFOV6k05gLN4p0OLy2X4Hoz/TJo3nhgtPd+NPDbBErbGOD+OJ+zU4FM730j4L9ARiKctzBpS4TzJkBD730dYAHQGZgJDPDmvwiMSKC0vQL0i+d5C0rjKGAa8I43Xennrbrk9COO3mVKqOo8YEep2X2BV733rwLXVmmiPGWkLe5UdbOqLvHe78F1LticBDhvYdIWd+rs9SbreC/F9aj7hjc/XuetrLQlBBFpAfTGjSGOiAhVcN6qS9Av7+hd8aDAhyKyWESGxTsxIZysqpvBBRHgpDinp7SRIrLcK/6JS9FTgIikAx1xOcOEOm+l0gYJcN68IoqlwFbgI9xd+U51PfRCHH+vpdOmqoHzNs47b8+ISL14pA2YCPySknHDm1IF5626BH3fo3fFURdVzQSuBO4SkUvinaBq5AXgLKADsBn4v3glREQaArOAn6nq7nilI5QQaUuI86aqharaATfAUifg3FCrVW2qvIOWSpuItAUeBM7BjeZ3IvBAVadLRK4Gtqrq4uDZIVaN+XmrLkHf1+hd8aSqm7y/W4E3cf/8iWSLiJwK4P3dGmH9KqOqW7wfZxFuvIW4nDsRqYMLqlNV9W/e7IQ4b6HSlijnLUBVdwL/xJWbNxGR2t6iuP9eg9LWyysuU3Uj/E0hPuetC9BHRHJxxdU9cDn/Sj9v1SXoRxy9K55EpIGINAq8B64Avgy/VZWbDQzx3g8B3o5jWo4SCKqe64jDufPKU/8ErFLVCUGL4n7eykpbgpy3NBFp4r0/DuiJe+YwF+jnrRav8xYqbV8FXcQFV2Ze5edNVR9U1Raqmo6LZ5+o6iCq4rzF++l1OZ5yX4WrtfA18HC801MqbWfiahQtA3LinT5gOu52/wjuLuk2XHnhx8Aa7++JCZS214AVwHJckD01DunqiruVXg4s9V5XJcJ5C5O2RDhv7YEvvDR8CTzqzT8TWAisBf4K1EugtH3inbcvgdfxavjE6wVcSkntnUo/b9YNgzHGJJHqUrxjjDEmBizoG2NMErGgb4wxScSCvjHGJBEL+sYYk0Qs6BtjTBKxoG+MMUnk/wEtYRw/okWkwQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss) + 1)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "성공이네요! 30번째 에포크까지 과대적합이 일어나지 않았습니다. 평가 점수는 안정적이지만 이전보다 더 나아지진 않았습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 스태킹 순환 층\n",
    "\n",
    "과대적합은 더 이상 없지만 성능상의 병목이 있는 것 같으므로 네트워크의 용량을 늘려야 합니다. 일반적인 머신 러닝 작업 흐름을 기억하세요. (드롭아웃 등을 사용하여 과대적합을 줄이는 기본 단계를 거쳤다고 가정하고) 과대적합이 일어날 때까지 네트워크의 용량을 늘리는 것이 좋습니다. 너무 많이 과대적합되지 않는 한 아직 충분한 용량에 도달한 것이 아닙니다.\n",
    "\n",
    "네트워크의 용량을 늘리려면 일반적으로 층에 있는 유닛의 수를 늘리거나 층을 더 많이 추가합니다. 순환 층 스태킹은 더 강력한 순환 네트워크를 만드는 고전적인 방법입니다. 예를 들어 구글 번역 알고리즘의 현재 성능은 7개의 대규모 LSTM 층을 쌓은 대규모 모델에서 나온 것입니다.\n",
    "\n",
    "케라스에서 순환 층을 차례대로 쌓으려면 모든 중간 층은 마지막 타임스텝 출력만이 아니고 전체 시퀀스(3D 텐서)를 출력해야 합니다. `return_sequences=True`로 지정하면 됩니다: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/40\n",
      "500/500 [==============================] - 250s 499ms/step - loss: 0.3360 - val_loss: 0.2779\n",
      "Epoch 2/40\n",
      "500/500 [==============================] - 248s 497ms/step - loss: 0.3140 - val_loss: 0.2765\n",
      "Epoch 3/40\n",
      "500/500 [==============================] - 248s 497ms/step - loss: 0.3067 - val_loss: 0.2697\n",
      "Epoch 4/40\n",
      "500/500 [==============================] - 248s 497ms/step - loss: 0.3032 - val_loss: 0.2728\n",
      "Epoch 5/40\n",
      "500/500 [==============================] - 248s 497ms/step - loss: 0.2981 - val_loss: 0.2650\n",
      "Epoch 6/40\n",
      "500/500 [==============================] - 248s 497ms/step - loss: 0.2964 - val_loss: 0.2705\n",
      "Epoch 7/40\n",
      "500/500 [==============================] - 249s 497ms/step - loss: 0.2911 - val_loss: 0.2655\n",
      "Epoch 8/40\n",
      "500/500 [==============================] - 249s 497ms/step - loss: 0.2896 - val_loss: 0.2690\n",
      "Epoch 9/40\n",
      "500/500 [==============================] - 249s 498ms/step - loss: 0.2866 - val_loss: 0.2664\n",
      "Epoch 10/40\n",
      "500/500 [==============================] - 249s 497ms/step - loss: 0.2850 - val_loss: 0.2669\n",
      "Epoch 11/40\n",
      "500/500 [==============================] - 249s 497ms/step - loss: 0.2827 - val_loss: 0.2700\n",
      "Epoch 12/40\n",
      "500/500 [==============================] - 249s 497ms/step - loss: 0.2803 - val_loss: 0.2678\n",
      "Epoch 13/40\n",
      "500/500 [==============================] - 249s 497ms/step - loss: 0.2803 - val_loss: 0.2651\n",
      "Epoch 14/40\n",
      "500/500 [==============================] - 249s 497ms/step - loss: 0.2789 - val_loss: 0.2651\n",
      "Epoch 15/40\n",
      "500/500 [==============================] - 249s 498ms/step - loss: 0.2782 - val_loss: 0.2730\n",
      "Epoch 16/40\n",
      "500/500 [==============================] - 249s 497ms/step - loss: 0.2757 - val_loss: 0.2692\n",
      "Epoch 17/40\n",
      "500/500 [==============================] - 249s 498ms/step - loss: 0.2749 - val_loss: 0.2660\n",
      "Epoch 18/40\n",
      "500/500 [==============================] - 249s 497ms/step - loss: 0.2739 - val_loss: 0.2608\n",
      "Epoch 19/40\n",
      "500/500 [==============================] - 249s 497ms/step - loss: 0.2731 - val_loss: 0.2681\n",
      "Epoch 20/40\n",
      "500/500 [==============================] - 249s 498ms/step - loss: 0.2710 - val_loss: 0.2643\n",
      "Epoch 21/40\n",
      "500/500 [==============================] - 249s 498ms/step - loss: 0.2701 - val_loss: 0.2699\n",
      "Epoch 22/40\n",
      "500/500 [==============================] - 249s 497ms/step - loss: 0.2707 - val_loss: 0.2651\n",
      "Epoch 23/40\n",
      "500/500 [==============================] - 249s 497ms/step - loss: 0.2692 - val_loss: 0.2672\n",
      "Epoch 24/40\n",
      "500/500 [==============================] - 249s 497ms/step - loss: 0.2684 - val_loss: 0.2654\n",
      "Epoch 25/40\n",
      "500/500 [==============================] - 249s 498ms/step - loss: 0.2662 - val_loss: 0.2689\n",
      "Epoch 26/40\n",
      "500/500 [==============================] - 249s 497ms/step - loss: 0.2650 - val_loss: 0.2734\n",
      "Epoch 27/40\n",
      "500/500 [==============================] - 249s 498ms/step - loss: 0.2646 - val_loss: 0.2661\n",
      "Epoch 28/40\n",
      "500/500 [==============================] - 249s 497ms/step - loss: 0.2642 - val_loss: 0.2642\n",
      "Epoch 29/40\n",
      "500/500 [==============================] - 249s 498ms/step - loss: 0.2628 - val_loss: 0.2667\n",
      "Epoch 30/40\n",
      "500/500 [==============================] - 249s 497ms/step - loss: 0.2621 - val_loss: 0.2818\n",
      "Epoch 31/40\n",
      "500/500 [==============================] - 249s 497ms/step - loss: 0.2625 - val_loss: 0.2683\n",
      "Epoch 32/40\n",
      "500/500 [==============================] - 249s 498ms/step - loss: 0.2615 - val_loss: 0.2690\n",
      "Epoch 33/40\n",
      "500/500 [==============================] - 249s 498ms/step - loss: 0.2601 - val_loss: 0.2710\n",
      "Epoch 34/40\n",
      "500/500 [==============================] - 249s 498ms/step - loss: 0.2605 - val_loss: 0.2693\n",
      "Epoch 35/40\n",
      "500/500 [==============================] - 249s 498ms/step - loss: 0.2577 - val_loss: 0.2735\n",
      "Epoch 36/40\n",
      "500/500 [==============================] - 249s 498ms/step - loss: 0.2577 - val_loss: 0.2720\n",
      "Epoch 37/40\n",
      "500/500 [==============================] - 249s 498ms/step - loss: 0.2580 - val_loss: 0.2706\n",
      "Epoch 38/40\n",
      "500/500 [==============================] - 249s 498ms/step - loss: 0.2575 - val_loss: 0.2720\n",
      "Epoch 39/40\n",
      "500/500 [==============================] - 249s 497ms/step - loss: 0.2556 - val_loss: 0.2730\n",
      "Epoch 40/40\n",
      "500/500 [==============================] - 249s 498ms/step - loss: 0.2560 - val_loss: 0.2739\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.GRU(32,\n",
    "                     dropout=0.1,\n",
    "                     recurrent_dropout=0.5,\n",
    "                     return_sequences=True,\n",
    "                     input_shape=(None, float_data.shape[-1])))\n",
    "model.add(layers.GRU(64, activation='relu',\n",
    "                     dropout=0.1, \n",
    "                     recurrent_dropout=0.5))\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=40,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "결과를 확인해 보죠:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xl8VNX5+PHPw74KCFiVyKZUDHsMiCWKIFpwAbWoIChaEVG0Wr62orhibRWtIopW6vqTKCKtlbpRF1pEZQmyi8higABqQECQzZDn98e5kwzJZOYmmcxMMs/79ZpX5t577r1n7mSee+45554rqooxxpjkUC3eGTDGGBM7FvSNMSaJWNA3xpgkYkHfGGOSiAV9Y4xJIhb0jTEmiVjQN6UiItVFZK+ItIxm2ngSkZNEJOp9l0Wkn4hkB02vEZEz/KQtw76eE5E7y7p+mO3+SUReivZ2TfzUiHcGTMUSkb1Bk/WAg8Bhb/p6Vc0szfZU9TDQINppk4GqnhyN7YjISGC4qp4VtO2R0di2qfos6FdxqloQdL2S5EhV/bCk9CJSQ1XzYpE3Y0zsWfVOkvMu318XkddEZA8wXEROF5H5IrJLRLaJyGQRqemlryEiKiKtvelp3vL3RGSPiHwuIm1Km9ZbPkBEvhaR3SLypIh8KiJXl5BvP3m8XkTWichOEZkctG51EXlcRHaIyHqgf5jjc5eITC8yb4qIPOa9Hykiq73Ps94rhZe0rRwROct7X09EXvHytgo4NcR+N3jbXSUiA735nYCngDO8qrPtQcf2vqD1R3uffYeI/EtEjvNzbCIRkYu8/OwSkY9F5OSgZXeKyFYR+VFEvgr6rD1F5Atv/nci8ojf/ZkKoKr2SpIXkA30KzLvT8Ah4EJcIaAu0B04DXcl2Bb4GrjJS18DUKC1Nz0N2A6kAzWB14FpZUh7DLAHGOQtGwv8DFxdwmfxk8e3gEZAa+CHwGcHbgJWASlAU2Cu+ymE3E9bYC9QP2jb3wPp3vSFXhoB+gL7gc7esn5AdtC2coCzvPePAv8FmgCtgC+LpL0MOM77Tq7w8vALb9lI4L9F8jkNuM97f66Xx65AHeBp4GM/xybE5/8T8JL3/hQvH3297+hO77jXBDoAG4FjvbRtgLbe+0XAUO99Q+C0eP8WkvllJX0DME9V/62q+aq6X1UXqeoCVc1T1Q3AVKB3mPVnqmqWqv4MZOKCTWnTXgAsVdW3vGWP404QIfnM419UdbeqZuMCbGBflwGPq2qOqu4AHgqznw3AStzJCOAcYJeqZnnL/62qG9T5GPgICNlYW8RlwJ9UdaeqbsSV3oP3O0NVt3nfyau4E3a6j+0CDAOeU9WlqnoAGAf0FpGUoDQlHZtwhgCzVPVj7zt6CDgKd/LNw51gOnhVhN94xw7cybudiDRV1T2qusDn5zAVwIK+AdgcPCEi7UXkHRH5VkR+BCYAzcKs/23Q+32Eb7wtKe3xwflQVcWVjEPymUdf+8KVUMN5FRjqvb8Cd7IK5OMCEVkgIj+IyC5cKTvcsQo4LlweRORqEVnmVaPsAtr73C64z1ewPVX9EdgJtAhKU5rvrKTt5uO+oxaqugb4P9z38L1XXXisl/QaIBVYIyILReQ8n5/DVAAL+gbc5X6wZ3Gl25NU9SjgHlz1RUXahqtuAUBEhCODVFHlyeM24ISg6UhdSl8H+nkl5UG4kwAiUheYCfwFV/XSGPiPz3x8W1IeRKQt8AxwA9DU2+5XQduN1L10K67KKLC9hrhqpC0+8lWa7VbDfWdbAFR1mqr2wlXtVMcdF1R1jaoOwVXh/RX4h4jUKWdeTBlZ0DehNAR2Az+JyCnA9THY59tAmohcKCI1gFuA5hWUxxnArSLSQkSaAreHS6yq3wHzgBeBNaq61ltUG6gF5AKHReQC4OxS5OFOEWks7j6Gm4KWNcAF9lzc+W8krqQf8B2QEmi4DuE14FoR6SwitXHB9xNVLfHKqRR5HigiZ3n7/gOuHWaBiJwiIn28/e33XodxH+BKEWnmXRns9j5bfjnzYsrIgr4J5f+AEbgf9LO4km6F8gLr5cBjwA7gRGAJ7r6CaOfxGVzd+wpcI+NMH+u8imuYfTUoz7uA3wNv4hpDB+NOXn7ci7viyAbeA/5f0HaXA5OBhV6a9kBwPfgHwFrgOxEJrqYJrP8+rprlTW/9lrh6/nJR1VW4Y/4M7oTUHxjo1e/XBibi2mG+xV1Z3OWteh6wWlzvsEeBy1X1UHnzY8pGXNWpMYlFRKrjqhMGq+on8c6PMVWFlfRNwhCR/iLSyKsiuBvXI2RhnLNlTJXiK+h7P8Y13s0c40IsHy0iK0RkqYjME5HUIstbejeS3BatjJsqKQPYgKsi6A9cpKolVe8YY8ogYvWOd5n9Na5/cg6FN1p8GZTmKK9bGN6dgzeqav+g5f/ANdwsUNVHo/4pjDHG+OKnpN8DWOfdgHIImE7hjSpAQT/ggPoEdSkTkYtwpbdV5c+uMcaY8vAz4FoLjryJJAd3B94RRGQM7tb5WrjbtBGR+rjucOcAJVbtiMgoYBRA/fr1T23fvn1JSY0xxoSwePHi7aoarpsz4C/oh7rRpFidkKpOAaaIyBW4rlojgPtxt7vvdffahKaqU3G30ZOenq5ZWVk+smWMMSZARCLdWQ74C/o5HHnnYAquK11JpuP68YK7IhgsIhOBxkC+iBxQ1adKXNsYY0yF8RP0F+EGS2qDu916CG78kQIi0i7oLsXzcTeOoKpnBKW5D9hrAd8YY+InYtBX1TwRuQmYjRtP4wVVXSUiE4AsVZ0F3CQi/XCj6e3EVe0YY4xJMAl3R67V6RsTWz///DM5OTkcOHAg3lkxPtSpU4eUlBRq1jxy6CURWayqEYfftsclGpPkcnJyaNiwIa1btyZchwsTf6rKjh07yMnJoU2bNpFXCKHKDMOQmQmtW0O1au5vZqke921M8jpw4ABNmza1gF8JiAhNmzYt11VZlSjpZ2bCqFGwb5+b3rjRTQMMK/fYgsZUfRbwK4/yfldVoqQ/fnxhwA/Yt8/NN8YYU6hKBP1Nm0o33xiTOHbs2EHXrl3p2rUrxx57LC1atCiYPnTI37D711xzDWvWrAmbZsqUKWRGqd43IyODpUuXRmVbsVYlqndatnRVOqHmG2OiKzPTXUVv2uR+Yw8+WL5q1KZNmxYE0Pvuu48GDRpw221Hjtqiqqgq1aqFLqe++OKLEfczZsyYsmeyCqkSJf0HH4R69Y6cV6+em2+MiZ5A+9nGjaBa2H5WER0n1q1bR8eOHRk9ejRpaWls27aNUaNGkZ6eTocOHZgwYUJB2kDJOy8vj8aNGzNu3Di6dOnC6aefzvfffw/AXXfdxaRJkwrSjxs3jh49enDyySfz2WefAfDTTz/xm9/8hi5dujB06FDS09MjluinTZtGp06d6NixI3feeScAeXl5XHnllQXzJ0+eDMDjjz9OamoqXbp0Yfjw4VE/Zn5UiaA/bBhMnQqtWoGI+zt1qjXiGhNtsW4/+/LLL7n22mtZsmQJLVq04KGHHiIrK4tly5bxwQcf8OWXXxZbZ/fu3fTu3Ztly5Zx+umn88ILL4TctqqycOFCHnnkkYITyJNPPsmxxx7LsmXLGDduHEuWLAmbv5ycHO666y7mzJnDkiVL+PTTT3n77bdZvHgx27dvZ8WKFaxcuZKrrroKgIkTJ7J06VKWLVvGU0/FZ3CCKhH0wQX47GzIz3d/LeAbE32xbj878cQT6d69e8H0a6+9RlpaGmlpaaxevTpk0K9bty4DBgwA4NRTTyU7Ozvkti+55JJiaebNm8eQIUMA6NKlCx06dAibvwULFtC3b1+aNWtGzZo1ueKKK5g7dy4nnXQSa9as4ZZbbmH27Nk0atQIgA4dOjB8+HAyMzOL3VwVK1Um6BtjKl5J7WQV1X5Wv379gvdr167liSee4OOPP2b58uX0798/ZH/1WrVqFbyvXr06eXl5Ibddu3btYmlKO0JBSembNm3K8uXLycjIYPLkyVx//fUAzJ49m9GjR7Nw4ULS09M5fPhwqfYXDRb0jTG+xbP97Mcff6Rhw4YcddRRbNu2jdmzZ0d9HxkZGcyYMQOAFStWhLySCNazZ0/mzJnDjh07yMvLY/r06fTu3Zvc3FxUlUsvvZT777+fL774gsOHD5OTk0Pfvn155JFHyM3NZV/RurIYqBK9d4wxsRGoNo1m7x2/0tLSSE1NpWPHjrRt25ZevXpFfR8333wzV111FZ07dyYtLY2OHTsWVM2EkpKSwoQJEzjrrLNQVS688ELOP/98vvjiC6699lpUFRHh4YcfJi8vjyuuuII9e/aQn5/P7bffTsOGDaP+GSKxAdeMSXKrV6/mlFNOiXc2EkJeXh55eXnUqVOHtWvXcu6557J27Vpq1Eis8nGo78wGXDPGmFLau3cvZ599Nnl5eagqzz77bMIF/PKqWp/GGGPKoXHjxixevDje2ahQ1pBrjDFJxFfQF5H+IrJGRNaJyLgQy0eLyAoRWSoi80Qk1Zvfw5u3VESWicjF0f4Axhhj/IsY9EWkOjAFGACkAkMDQT3Iq6raSVW7AhOBx7z5K4F0b35/4FkRsSolY4yJEz8l/R7AOlXdoKqHgOnAoOAEqvpj0GR9QL35+1Q1cGdEncB8Y4wx8eEn6LcANgdN53jzjiAiY0RkPa6k/7ug+aeJyCpgBTA66CQQvO4oEckSkazc3NzSfgZjTCV21llnFbvRatKkSdx4441h12vQoAEAW7duZfDgwSVuO1IX8EmTJh1xk9R5553Hrl27/GQ9rPvuu49HH3203NuJNj9BP9RjWoqV2FV1iqqeCNwO3BU0f4GqdgC6A3eISJ0Q605V1XRVTW/evLn/3BtjKr2hQ4cyffr0I+ZNnz6doUOH+lr/+OOPZ+bMmWXef9Gg/+6779K4ceMyby/R+Qn6OcAJQdMpwNYw6acDFxWdqaqrgZ+AjqXJoDGmahs8eDBvv/02Bw8eBCA7O5utW7eSkZFR0G8+LS2NTp068dZbbxVbPzs7m44dXVjZv38/Q4YMoXPnzlx++eXs37+/IN0NN9xQMCzzvffeC8DkyZPZunUrffr0oU+fPgC0bt2a7du3A/DYY4/RsWNHOnbsWDAsc3Z2NqeccgrXXXcdHTp04Nxzzz1iP6EsXbqUnj170rlzZy6++GJ27txZsP/U1FQ6d+5cMNDb//73v4KHyHTr1o09e/aU+diG4qdRdRHQTkTaAFuAIcAVwQlEpJ2qrvUmzwfWevPbAJtVNU9EWgEnA9lRyrsxJspuvRWi/UCorl3Bi5chNW3alB49evD+++8zaNAgpk+fzuWXX46IUKdOHd58802OOuootm/fTs+ePRk4cGCJz4l95plnqFevHsuXL2f58uWkpaUVLHvwwQc5+uijOXz4MGeffTbLly/nd7/7HY899hhz5syhWbNmR2xr8eLFvPjiiyxYsABV5bTTTqN37940adKEtWvX8tprr/H3v/+dyy67jH/84x9hx8e/6qqrePLJJ+nduzf33HMP999/P5MmTeKhhx7im2++oXbt2gVVSo8++ihTpkyhV69e7N27lzp1ilWOlEvEkr5XB38TMBtYDcxQ1VUiMkFEBnrJbhKRVSKyFBgLjPDmZwDLvPlvAjeq6vaofgJjTKUXXMUTXLWjqtx555107tyZfv36sWXLFr777rsStzN37tyC4Nu5c2c6d+5csGzGjBmkpaXRrVs3Vq1aFXEwtXnz5nHxxRdTv359GjRowCWXXMInn3wCQJs2bejatSsQfvhmcOP779q1i969ewMwYsQI5s6dW5DHYcOGMW3atII7f3v16sXYsWOZPHkyu3btivodwb62pqrvAu8WmXdP0PtbSljvFeCV8mTQGBM74UrkFemiiy5i7NixfPHFF+zfv7+ghJ6ZmUlubi6LFy+mZs2atG7dOuRwysFCXQV88803PProoyxatIgmTZpw9dVXR9xOuHHJAsMygxuaOVL1Tkneeecd5s6dy6xZs3jggQdYtWoV48aN4/zzz+fdd9+lZ8+efPjhh7Rv375M2w/F7sg1xsRdgwYNOOuss/jtb397RAPu7t27OeaYY6hZsyZz5sxhY6iHYQc588wzCx5+vnLlSpYvXw64YZnr169Po0aN+O6773jvvfcK1mnYsGHIevMzzzyTf/3rX+zbt4+ffvqJN998kzPOOKPUn61Ro0Y0adKk4CrhlVdeoXfv3uTn57N582b69OnDxIkT2bVrF3v37mX9+vV06tSJ22+/nfT0dL766qtS7zMcu1HKGJMQhg4dyiWXXHJET55hw4Zx4YUXkp6eTteuXSOWeG+44QauueYaOnfuTNeuXenRowfgnoLVrVs3OnToUGxY5lGjRjFgwACOO+445syZUzA/LS2Nq6++umAbI0eOpFu3bmGrckry8ssvM3r0aPbt20fbtm158cUXOXz4MMOHD2f37t2oKr///e9p3Lgxd999N3PmzKF69eqkpqYWPAUsWmxoZWOSnA2tXPmUZ2hlq94xxpgkYkHfGGOSiAV9Y0ypHwhu4qe835UFfWOSXJ06ddixY4cF/kpAVdmxY0e5btiy3jvGJLmUlBRycnKwwQ4rhzp16pCSklLm9S3oG5PkatasSZs2beKdDRMjVr1jjDFJxIK+McYkEQv6xhiTRCzoG2NMErGgb4wxScSCvjHGJBFfQV9E+ovIGhFZJyLjQiwfLSIrRGSpiMwTkVRv/jkisthbtlhE+kb7AxhjjPEvYtAXkerAFGAAkAoMDQT1IK+qaidV7QpMBB7z5m8HLlTVTrinadkDVYwxJo78lPR7AOtUdYOqHsI9+HxQcAJV/TFosj6g3vwlqhp4iPoqoI6I1MYYY0xc+LkjtwWwOWg6BzitaCIRGYN7Pm4tIFQ1zm+AJap6sAz5NMYYEwV+SvqhHjtfbGQmVZ2iqicCtwN3HbEBkQ7Aw8D1IXcgMkpEskQky8b/MMaYiuMn6OcAJwRNpwBbS0gLrvrnosCEiKQAbwJXqer6UCuo6lRVTVfV9ObNm/vIkjHGmLLwE/QXAe1EpI2I1AKGALOCE4hIu6DJ84G13vzGwDvAHar6aXSybIwxpqwiBn1VzQNuAmYDq4EZqrpKRCaIyEAv2U0iskpEluLq9UcE5gMnAXd73TmXisgx0f8Yxhhj/LAHoxtjTBVgD0Y3xhhTjAV9Y4xJIhb0jTEmiVjQN8aYJGJB3xhjkogFfWOMSSJJEfQzM6F1a6hWzf3NzIx3jowxJj78DLhWqWVmwqhRsG+fm9640U0DDBsWv3wZY0w8VPmS/vjxhQE/YN8+N98YY5JNlQ/6mzaVbr4xxlRlVT7ot2xZuvnGGFOVVfmg/+CDUK/ekfPq1XPzjTEm2VT5oD9sGEydCq1agYj7O3WqNeIaY5JTle+9Ay7AW5A3xpgkKOkbY4wpZEHfGGOSiK+gLyL9RWSNiKwTkXEhlo8WkRXek7HmiUiqN7+piMwRkb0i8lS0M2+MMaZ0IgZ9EakOTAEGAKnA0EBQD/KqqnZS1a7AROAxb/4B4G7gtuhl2RhjTFn5Ken3ANap6gZVPQRMBwYFJ1DVH4Mm6wPqzf9JVefhgr8xxpg489N7pwWwOWg6BzitaCIRGYN7KHotoG9pMiEio4BRAC3triljjKkwfkr6EmJesaepq+oUVT0RuB24qzSZUNWpqpququnNmzcvzarGGGNKwU/QzwFOCJpOAbaGST8duKg8mTLGGFMx/AT9RUA7EWkjIrWAIcCs4AQi0i5o8nxgbfSyaIwxJloi1umrap6I3ATMBqoDL6jqKhGZAGSp6izgJhHpB/wM7ARGBNYXkWzgKKCWiFwEnKuqX0b/oxhjjIlEVItVz8dVenq6ZmVlxXSfmZlufP1Nm9zomw8+aMM2GGMqFxFZrKrpkdIlxdg74diTtYwxySTph2GwJ2sZY5JJ0gd9e7KWMSaZJH3QtydrGWOSSdIHfXuyljEmmSR90LcnaxljkknS994Be7KWMSZ5JH1J3xhjkokFfR8yM6F1a6hWzf3NzIx3jowxpmyseicCu3nLGFOVWEk/Art5yxhTlVjQj8Bu3jLGVCUW9COwm7eMMVWJBf0I7OYtY0xVYkE/Art5yxhTlVjvHR/s5i1jTFXhq6QvIv1FZI2IrBORcSGWjxaRFSKyVETmiUhq0LI7vPXWiMivo5l5Y4wxpRMx6ItIdWAKMABIBYYGB3XPq6raSVW7AhOBx7x1U3HP1O0A9Aee9rZnjDEmDvyU9HsA61R1g6oeAqYDg4ITqOqPQZP1gcAzGAcB01X1oKp+A6zztmeMMSYO/NTptwA2B03nAKcVTSQiY4CxQC2gb9C684us2yLEuqOAUQAtrS+kMcZUGD8lfQkxr9jT1FV1iqqeCNwO3FXKdaeqarqqpjdv3txHlhKLjc1jjKks/JT0c4ATgqZTgK1h0k8HninjupWOjc1jjKlM/JT0FwHtRKSNiNTCNczOCk4gIu2CJs8H1nrvZwFDRKS2iLQB2gELy5/txGFj8xhjKpOIJX1VzRORm4DZQHXgBVVdJSITgCxVnQXcJCL9gJ+BncAIb91VIjID+BLIA8ao6uEK+ixxYWPzGGMqE1EtVsUeV+np6ZqVlRXvbPjWurWr0imqVSvIzo51bowxyUpEFqtqeqR0NgxDOfkZm8caeo0xicKCfjlFGpsn0NC7cSOoFjb0WuA3xsSDVe9UMKv+McbEglXvJAhr6DXGJBIL+hXMHsJijEkkFvQrmD2ExRiTSCzoVzB7CIsxJpHYQ1RiwB7CYoxJFFbSN8aYJGJB3xhjkogFfWOMSSIW9I0xJolY0DfGmCRiQT/ObDA2Y0wsWZfNOLKnbhljYs1XSV9E+ovIGhFZJyLjQiwfKyJfishyEflIRFoFLXtYRFZ6r8ujmfnKzp66ZYyJtYhBX0SqA1OAAUAqMFREUoskWwKkq2pnYCYw0Vv3fCAN6AqcBvxBRI6KXvYrNz+DsVn1jzEmmvyU9HsA61R1g6oewj34fFBwAlWdo6qBMut83APQwZ0k/qeqear6E7AM6B+drFd+kQZjs7H4jTHR5ifotwA2B03nePNKci3wnvd+GTBAROqJSDOgD3BC0RVEZJSIZIlIVm5urr+cVwGRBmOz6h9jTLT5CfoSYl7IJ6+IyHAgHXgEQFX/A7wLfAa8BnyOe0D6kRtTnaqq6aqa3rx5c59Zr/wiDcZmY/EbY6LNT9DP4cjSeQqwtWgiEekHjAcGqurBwHxVfVBVu6rqObgTyNryZblqGTbMPUErP9/9De61Y2PxG2OizU/QXwS0E5E2IlILGALMCk4gIt2AZ3EB//ug+dVFpKn3vjPQGfhPtDJf1dlY/MaYaIvYT19V80TkJmA2UB14QVVXicgEIEtVZ+GqcxoAb4gIwCZVHQjUBD7x5v0IDFfVYtU7JrRAqX/8eFel07KlC/jWh98YU1b2YHRjjKkC7MHoScL68RtjSsOGYajEbBgHY0xpWUm/ErN+/MaY0rKgX4lZP35jTGlZ0K/E/PTjtzp/Y0wwC/qVWKR+/DZ2jzGmKAv6lVikYRyszt8YU5T106/CqlVzJfyiRNywD8aYqsP66Rsbu8cYU4wF/SrMz9g91tBrTHKxoF+FRarzt4ZeY5KP1eknsdatXaAvqlUrN8yzMabysDp9E5Hd3GVM8rGgn8SsoddUhG+/he++i3cuTEks6Ccxe0iLqQiXXgpDhsQ7F6YkFvSTWKSGXgjfu8d6/piiDh6EhQvhs8/gwIF458aE4mtoZRHpDzyBe3LWc6r6UJHlY4GRuIee5wK/VdWN3rKJwPm4E8wHwC2aaK3HSWzYsJKHYQ43dDPYsM6muOXL4dAh9z4rCzIy4psfU1zEkr6IVAemAAOAVGCoiKQWSbYESFfVzsBMYKK37q+AXrhn43YEugO9o5Z7U6HCDeNgQzyYUBYuLHw/b1788mFK5qek3wNYp6obAERkOjAI+DKQQFXnBKWfDwwPLALqALUAwT0z15p4Komy9O6xnj/JbeFCOPZYaNzYgn6i8lOn3wLYHDSd480rybXAewCq+jkwB9jmvWar6uqiK4jIKBHJEpGs3Nxcv3k3FSxc7x4b1tmEsmgR9OgBZ5wBn35qYzwlIj9BX0LMC1knLyLDgXTgEW/6JOAUIAV3ougrImcW25jqVFVNV9X05s2b+827qWDhevdEY1hnOylULbt3w1dfuaDfqxfs2gVffhl5PRNbfoJ+DnBC0HQKsLVoIhHpB4wHBqrqQW/2xcB8Vd2rqntxVwA9y5dlEyvheveUd1hnGwKi6lm82H2XPXoUNuBaFU/iiTgMg4jUAL4Gzga2AIuAK1R1VVCabrgG3P6qujZo/uXAdUB/3BXD+8AkVf13SfuzYRiqhkjDOtsQEFXPQw/BHXfADz+4Ov3jj4ezz4Zp0+Kds+QQtWEYVDUPuAmYDawGZqjqKhGZICIDvWSPAA2AN0RkqYjM8ubPBNYDK4BlwLJwAb88Dh+Gq66Cf//b6hETQaQ6fxsCoupZuBDatYMmTdzJPSPDSvqJyNfNWar6rqr+UlVPVNUHvXn3qOos730/Vf2Fqnb1XgO9+YdV9XpVPUVVU1V1bEV9kI0bYc4cGDgQTj4ZnnoK9u6tqL2ZSCLV+dsQEFXPwoWuaicgI8P9LjdvLnkdE3tV5o7ctm1hwwZ4/XVo1gxuvhlSUuC220JXI5iKFanO34aAqFq2bHGvokEfXC8ekziqTNAHqFkTLrsMPv/cvQYMgEmT3Anh0kvdPBM7w4a5+vn8fPc3+E7d8g4BYRLLokXub3DQ79IF6te3Kp5EU+XH09+8GaZMcQFl5074+GPo0ydqmzcVpOgQEOCuBIqeGExiGD8eJk6EPXugTp3C+eecA7m5sHRp/PKWLGw8fc8JJ7heBRs3wnHHwZ//HO8cGT/8DPNgVwKJY+FCV7IPDvjgqnhWrHB9+E1iqPJBP6BhQ7j1VvjwQ9ef2CS2SL17rJ9/4sjPL7wTt6iMDLd8/vzY58uEljRBH+D66+Goo9xlqElskXr32IBviWPtWleS7969+LLTToPq1a1eP5EkVdDlEiKQAAAYUElEQVRv1AhuvBFmzoR16+KdGxNOpN49fq4ErOonNgIja4Yq6TdoAN26WdBPJEkV9AFuucX18nn00XjnxIQTqXdPuCsBq/qJrYULXXBv3z708owMWLCgcJx9E19JF/SPPRauvhpeesk9y9MkrnBdPsNdCVgjcGwtXAjp6a4aJ5SMDNi/H5YsiW2+KouNG+H552HoUFcFXdGSLuiDu2Hr55/hiSfinRNTVuGuBJK9ETg/H+691z25qqIdPOi6Y4aq2gno1cv9jXcVj6oLqv36wezZoceGioUdO+CNN2D0aDjpJFfoGDkS/vtfVwtR4VQ1oV6nnnqqxsKll6oedZTqrl3+18nPr7j8JKu33lL973+ju81WrVTdT/rIV6tW/pZXdo884j5Pz54Vv6+FC92+Zs4Mn+6kk1Qvuqji8xPOU0+5vDZq5P5266Y6Y4ZqXl7F7jc/X/WLL1Tvvls1LU1VxO2/YUPVgQNVn3hCddWq8scXIEt9xNi4B/mir1gF/aws9+kffjhy2p9+cl/Or39tgT+adu5UrVtXtXZt1fnzo7fdadNU69U7MqDXq+fmqxb+6Iq+RKKXh3hZulS1Zk3VX/zCfaYFCyp2f1OmuP1s2hQ+3dVXqzZrFr/fT1aWaq1aqhdcoHrggOrzz6v+8pcu77/8pZs+eDB6+8vLU/3kE9Xf/161dWu3n2rVVM88U/WBB1Q//1z155+jtz9VC/q+9Ouneuyxqvv3l5xmzx7V3r0LA8PChTHLXpX35JPumB5zjPseIgWO0pg2zZXcRdzfQMBXrbol/X37VDt0cMdywwZXkhw2rGL3OWKE21+kYP73v7tj/NVXFZufUHbtUm3bVvWEE1S3by+cn5fnSvrdurm8paSoTpqkundv2faTl6f6/vuqo0a5/2koPNE8/7xqbm50Pk9JLOj78OGH7ghMnRp6+a5dqr/6lWr16qp/+5tq/fqqv/1tzLJXpeXnq3bs6C53V650Aapr17L/4Eoj0pVAIE1JJ41Edcst7rO8/37hdM2aqlu3Vtw+TzlF9cILI6dbvdrl7bnnKi4voeTnqw4e7H7Dn35acpr333elcFBt0kR13DjVnBx/+9i9W/XxxwtL9A0aqF5+uerrr6v++GP0PkskFvR9yM9XPfVU1Xbtitfr/fCDavfuqjVqFNZXXnedq47YuTNmWayyPv3U/fc9+6ybfucdd/l7ySWqhw9X/P7DBXU/J4XybL8izJ7t8nnzzYXz1q51+7/nnorZ565dbvsPPBA5bX6+atOmrponlgL1+BMn+kv/6aeqv/mN+1+sUUP1iitUFy0KnTY7W3XsWNc2CKoZGS5WhKs5qEhRDfq4J1+tAdYB40IsHwt8CSwHPgJaefP7AEuDXgeAi8LtK5ZBX9Vd3oHqG28UzsvNdZd8tWqpzppVOD/QDjB5ckyzWCWNGOFKRMElob/+1R3fu+6KW7ZU1V/1T0WfNEpj+3bV445zpe59+45cdsEFrqrhwIHo7/ejj9xnmz3bX/pBg1yDbnm8+67q9derfvZZ5LSBevzzzy99QWLDBlcf37BhYUD/5z9d4XD+fNXLLnNXD9Wrqw4dmhjVvlEL+kB13NOv2gK1cE/ASi2Spg9Qz3t/A/B6iO0cDfwQSFfSK9ZBPy/P/SOmp7vSyLffumqHOnVU33uvePru3VVTU0vXIJWfr/rqq27bxl1F1anjfrzB8vNVr73W/Ve++mp88qYauaE3UlCPZZtBfr4rmdas6XqIFPXBB27fL78c/X3/5S9u2zt2+Esf6FW0bVvp97V1qwu0gQZRUL34YldtFEqgHj8l5ch6/NLavVv1sccKq26aNNGCHkB//GN026HKK5pB/3RgdtD0HcAdYdJ3Az4NMX8UkBlpf7EO+qquigFUX3lFtX179wP+6KPQaZ9/3qWdO9f/9mfOdOv07ZuYvX8++yy2VVZPPOGOR6ggdfCgq1utXbvie56UpLxdPv2cNKJV9fPSSxq2F1p+viukpKVF/3/v4otd1ahfn3+uvrp3Bjt8WPXpp10VSu3arippxw73t2FDV9K+7jrVLVsK1/FTj19aeXku35dd5q709+yJznajKZpBfzDwXND0lcBTYdI/BdwVYv7HwAWR9hePoL9/f2EXtwYNwgf0vXvdWf6KK/xvu02bwnq/F16ITp6j5aGHtKDnQrT7y4cSCELdu5ecJjfXHbNjj1XdvLni81RUebt8hjspRLPqZ/169//au3f4vuZ/+5vbzyeflH4f4bRoUbreQQcPuiu8W2/1l37pUtXTTnN5P/ts1a+/PnL5998XNlbXrat6xx2u8FLaevyqIppB/9IQQf/JEtIOB+YDtYvMPw7IBWqWsN4oIAvIatmyZcUfnRCeflq1eXN/dYU33+zqCr//PnLaQFD94APVM85wl4eJUs3zwAMubwMHuhKbiPvhHDpUcfv85BO3z7//PXy6QI+ebt1i06OnqPJ0+QwX2MvbXhDw88+qvXq5wkR2dvjPsnev+7+79NKyH4+itmxx+X7iidKt17u3q0oNZ+9e1dtucyX15s3d5w93lbJ+vSuEgerRR5e9Hr+yi3n1DtAPWA0cE2LZLcBUPxmKR0k/wO8/ycqVGvaSOmDbNhe4Bg1y06tXu3/Iyy8vXb7efNMF5m++Kd16JcnPdz06QPXKK10pcc+ewvr07t2Ll6qiZfhwd0z8XB6//bYLfOecU7yBMp4mTiwetOvW9dfls7ztBQETJrhlfq8Q/vAHF0SjVQf9r3+5/fspJAUbP97lo+j3n53tqlevu071+OPdtkeO9N9eoOqqC8891zVol6cev7KKZtCvAWwA2gQ15HYokqab19jbroRtzAf6+MlQPIN+aZx5pmsoCneiGDnSXXoGB9DAj/Xtt/3t58MP3TbAlXrKe4men+9K86B6zTXFqwVmznSlwvr1XftFNOuBd+xw9bI33OB/nRdfdAGxXz93Z3QiuOACF+RTUgoDs99SdDSGiHjtNTdv2DD/3092tmsAHTfO/+cM5847XZfG0p6M33uv8Erv2WddIaBly8LP2aiRO76laTMzTrS7bJ4HfO0F9vHevAnAQO/9h8B3QV0zZwWt2xrYAlTzs6/KEvRffdUdvZK6qy1Z4oLV//3fkfMPHnR3TZ5wQuQbN7KyXJ1tx46uEaxdO3cCePHFsuU5P99dNoO7a7CkE9amTapnnVUYzH74oWz7K+rxx902ly4t3Xovv+yOZd++0Q388+a5RrnSnNgCvWGCr/Iuu8xdwa1aFXn98rYXfPSR+x8444zS9we/5BJX/RGNY9ivn2scLq1A3/7A5/rFL9z/2OTJ7v+iosfBqcqiGvRj+aosQf/AATeWyMUXF1+Wn+/qLps1C90r5rPP3D/+735X8va//tqV7Fu1KuyZsGOHa9ACdzIpzQ8kP7/wjs0xYyIHurw81yWvRg1Xoi3pBpXS7L99e9cwVxavvOJKqn36RKeO//Bh1ZNPdsfjr3/1t05enmqnTq6ROTjgfvedu/HotNP8fSdlbS9YutTV4XfoULYT8f/+V1jKLo/Dh12JfPTosq0/bZrLw5o1idmbrbKyoB8Df/yjq58serv2P/7hjuwzz5S87pgx7kcfaqCxLVtcv+BmzdwPI9ihQ25dcI1Vu3dHzufhw6o33ujWufXW0v3QFi1yAaduXfe5yioQcMrTe2naNBf4e/cuf+APVDOceKLbpp/qtqlT3TozZhRflpnplj32mL/9Hzigunhx8e+ipCuBSZPcDVgtWpS9Xj4/X7VLF3flWJ5g+9VX5f8uTfRZ0I+BdevcEbz//sJ5Bw64kmDHjuFH0du92/2AO3U6cnS/nTvdvAYNwpeun37anXA6dHC9F4Ll57vA8M47rvfQr3/t8vmHP5Ttx/7tt4Vd5x5+uGzbuOIKVzosb9XCq6+6IH3GGeXrK92/v+sS+sMProdQw4augb4ku3e7O1szMkJ//vx8NwZN3bpu+INw5s93jY3grlyKVncVvRL4299c+kaNVFesKF8//xdecPv9+GP/6xT1//6f20a442Viz4J+jJx7rqv+CAT4hx92R/XDDyOv+9ZbLu2f/uSm9+1zQaVmTVd3HMlHH7lG16ZN3X5vuMGtHxgvPPBKSXH7KE/pbt++wjsiR44sXbfO3FxX5z1mTNn3H2z6dHfCy8go24BWgZJq4GS9ebM7AbRpU/JIiOPGuXXC3W6fk+OqX3r3Dt1esm+fa1OpVs19J3ff7b47EXdMQ3Xl3bfPdc2sVcvdR1Hefv7797sryECPstJau9bdZ9G4sdW/JxoL+jHy5pvuKL71lvvRBh6M4Nfgwa5Hy6pVrqQo4kbn8+vrrwvrphs1coHwhhvclcDcudFrhFV1gWz8eLevs8/2fxdvYEyd5cujl5cZM1zg/9WvSl/iHzPGBdHgIDt/vvsezjyz+Ljq33zjll15ZeRtP/ec+6xPP33k/HnzCsdvHzWqsFruhx/coF01arj/nb/8pbC9IC/PPXhEpLBKKRpDPIwf77ZZ2rtV33vPBfujj/ZXqDGxZUE/Rn7+2fUrHjDA9TEu2kUzkq1bXbBu0CB0sPDj0CHXDhCrRrGXXnKfs3374lVLReXnu5PS6adHPx9vvOGC19ix/tfZudN1Rx0xoviyQL38tdceeSwvu8xV2/i5Ozg/3/VsadBAdeNG1/Zwyy0un61blxws16xxhQVw6WbMKGyHCb4Bys8DYCJV/2zd6v5nRdwJKFKf9vx81T//2aXv0sUNRmYSjwX9GLr3XveDKG0ACgg8YOLee6Ods4ozZ46rWmrWzFVFZWe7YLB+vWvr+PprF8imT3ef7aWXKiYf113nSvx+ryICVx2LF4deHriSCTTIzptX+u9mwwZ3YunZ093LEegx5eeK5MMPXZtOIJj/8Y9HLi/P3cAB06a5LsOB5Q0auE4Hoapr9uxxV6PgRpNMlHslTHEW9GNo82ZXT9u0adkHLsvOrnzd19ascSOUhgpCwa8mTSrujtrt291xP+MMf91Q27RxVWAlOXzYdcMN9Ojp3t2VikvbWyjwVLATTyz9mEZ5ea6a6IEHircNlHeEz1DrB0atTEtzJ7fAVcLxx7u2h2rV3Mmysv1/Jhu/QV9c2sSRnp6uWVlZ8c5GqT31FLRtC+edF++cxNYPP8Dbb8PhwyAC1aq5v8HvO3SAzp0rLg/PPQfXXQcvvwxXXVVyurfegosugjfegMGDS063dy9kZMCqVZCXBy+9BCNGlC5P+fnw0UfQqxfUq1e6dSPJzITx42HTJmjZEh58EIYNc8uqVXOhvCgRl6fWrWHjxuLLmzVz3+HOncWXjRsHf/lLVD+CqQAislhV0yOms6BvKrv8fBdc16+HNWugSZPQ6fr2dWnWr4caNcJvc9Mm6NHDBdX5810wrQxKCuqtWkF2dviTQkoKbN5c8romsfkN+pXkX9mYklWrBs88Azt2wF13hU6zfDnMmQNjxkQO+OCC/erV8PHHlSfggyv1F72yqFfPzQf3uUJp2RJyckIv27Sp8H1mpjuxVKvm/mZmljfHJtYq0b+zMSXr2tUF9GeegcWLiy+fPBnq1oWRI/1vs0kTaNAgenmMhWHDYOpUVzoXcX+nTi2s/gl3Ugh3QgAX4EeNclcSqu7vqFGlC/x20kgAfir+Y/mqjA25JjHs2uUG8OrR48gG0Nxc9/COUaPil7dEUlKXzmg8BjKRnh2cbLDeOyYZTZvm/quffbZw3p//rDZsgE/hgnZlenZwMvIb9K0h11Qpqq7Bdtky16jbuDG0aQOnnAIffBDv3FVukRqJy9OInJ/v3ofrmWTCs4Zck5REYMoU2LPHdTX85z9hyxa45ZZ456zyi9RIHNzgGywwPxZtBiYyC/qmyklNhbFj4YUXXOA/8cTku3+iIkRqJI4U1COdNMaPh337jly+b5+b74c1Evvkpw4I6A+sAdYB40IsHwt8CSwHPgJaBS1rCfwH9/zcL4HW4fZldfomGvbsKXyc4aRJ8c5NcvA7BER52gyskbhkRPEZudVxj0lsS+EzclOLpOkD1PPe3wC8HrTsv8A53vsGgXQlvSzom2h57z03CqefB82Y6CjPWP/hGnqtkTgyv0E/YkOuiJwO3Keqv/am7/CuEELemC0i3YCnVLWXiKQCU1U1w+eFhzXkGpOkAnX6wVU89eq5KqTx48vfSFzVRbMhtwUQfHN2jjevJNcC73nvfwnsEpF/isgSEXlERKqHyOwoEckSkazc3FwfWTLGVDXh2gzK20hsCvkJ+hJiXsjLAxEZDqQDj3izagBnALcB3XFVRFcX25jqVFVNV9X05s2b+8iSMaYqGjbMldzz893faDUSm0J+gn4OcELQdAqwtWgiEekHjAcGqurBoHWXqOoGVc0D/gWklS/LxphkEymoR+pZFA1VpndQpEp/XGl9A9CGwobcDkXSdMM19rYrMr+6l765N/0iMCbc/qwh1xgTSnkaif2sX57eQeXNWzQQzWEYgPOAr73APt6bNwFXqgf4EPgOWOq9ZgWtew6uK+cK4CWgVrh9WdA3xkSbn6Bd1t5BidJd1G/Qt2EYjDFVXkUOIdGyZfh1Y8WGYTDGGE+k3j/l6R0Uad1EY0HfGFPlRer9U57eQdHoLhrLRmIL+saYKi9S75/y9A7y0100XFCP+UBzfir+Y/myhlxjTEUoT++d8mw7VkNIYA25xhgTf9F4zoAf1pBrjDEJINGGkLCgb4wxFSjRhpCwoG+MMRUoEYaQCFajYjZrjDEGCoN3uGf/DhsWu2cBW9A3xpgKFsugHolV7xhjTBKxoG+MMUnEgr4xxiQRC/rGGJNELOgbY0wSSbhhGEQkFwhx03KBZsD2GGWntCxvZWN5KxvLW9lU1by1UtWIDxlPuKAfiYhk+RlfIh4sb2VjeSsby1vZJHverHrHGGOSiAV9Y4xJIpUx6E+NdwbCsLyVjeWtbCxvZZPUeat0dfrGGGPKrjKW9I0xxpSRBX1jjEkilSboi0h/EVkjIutEZFy881OUiGSLyAoRWSoicX3eo4i8ICLfi8jKoHlHi8gHIrLW+9skgfJ2n4hs8Y7dUhE5Lw75OkFE5ojIahFZJSK3ePPjftzC5C0RjlsdEVkoIsu8vN3vzW8jIgu84/a6iNRKoLy9JCLfBB23rrHOW1Aeq4vIEhF525uu+OPm50G68X4B1YH1QFugFrAMSI13vorkMRtoFu98eHk5E0gDVgbNmwiM896PAx5OoLzdB9wW52N2HJDmvW8IfA2kJsJxC5O3RDhuAjTw3tcEFgA9gRnAEG/+34AbEihvLwGD43ncgvI4FngVeNubrvDjVllK+j2Adaq6QVUPAdOBQXHOU8JS1bnAD0VmDwJe9t6/DFwU00x5Sshb3KnqNlX9wnu/B1gNtCABjluYvMWdOnu9yZreS4G+wExvfryOW0l5SwgikgKcDzznTQsxOG6VJei3ADYHTeeQIP/0QRT4j4gsFpFR8c5MCL9Q1W3ggghwTJzzU9RNIrLcq/6JS9VTgIi0BrrhSoYJddyK5A0S4Lh5VRRLge+BD3BX5btUNc9LErffa9G8qWrguD3oHbfHRaR2PPIGTAL+COR7002JwXGrLEFfQsxLmDO2p5eqpgEDgDEicma8M1SJPAOcCHQFtgF/jVdGRKQB8A/gVlX9MV75CCVE3hLiuKnqYVXtCqTgrspPCZUstrnydlokbyLSEbgDaA90B44Gbo91vkTkAuB7VV0cPDtE0qgft8oS9HOAE4KmU4CtccpLSKq61fv7PfAm7p8/kXwnIscBeH+/j3N+Cqjqd96PMx/4O3E6diJSExdUM1X1n97shDhuofKWKMctQFV3Af/F1Zs3FpHA41jj/nsNylt/r7pMVfUg8CLxOW69gIEiko2rru6LK/lX+HGrLEF/EdDOa9muBQwBZsU5TwVEpL6INAy8B84FVoZfK+ZmASO89yOAt+KYlyMEgqrnYuJw7Lz61OeB1ar6WNCiuB+3kvKWIMetuYg09t7XBfrh2hzmAIO9ZPE6bqHy9lXQSVxwdeYxP26qeoeqpqhqa1w8+1hVhxGL4xbv1utStHKfh+u1sB4YH+/8FMlbW1yPomXAqnjnD3gNd7n/M+4q6VpcfeFHwFrv79EJlLdXgBXAclyQPS4O+crAXUovB5Z6r/MS4biFyVsiHLfOwBIvDyuBe7z5bYGFwDrgDaB2AuXtY++4rQSm4fXwidcLOIvC3jsVftxsGAZjjEkilaV6xxhjTBRY0DfGmCRiQd8YY5KIBX1jjEkiFvSNMSaJWNA3xpgkYkHfGGOSyP8H14lDpgKlw2UAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss) + 1)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "층을 추가하여 성능을 조금 향상시켰지만 크지는 않습니다. 여기서 두 가지 결론을 얻을 수 있습니다:\n",
    "\n",
    "* 아직 충분히 과대적합을 만들지 못했기 때문에 검증 손실을 향상하기 위해서 층의 크기를 늘릴 수 있습니다. 하지만 적지 않은 계산 비용이 추가됩니다.\n",
    "* 층을 추가한 만큼 도움이 되지 않았으므로 여기서는 네트워크의 용량을 늘리는 것이 도움이 되지 않는다고 볼 수 있습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 양방향 RNN 사용하기\n",
    "\n",
    "이 절에서 소개할 마지막 기법은 양방향 RNN입니다. 양방향 RNN은 RNN의 한 변종이고 특정 작업에서 기본 RNN 보다 훨씬 좋은 성능을 냅니다. 자연어 처리에서는 맥가이버 칼이라고 할 정도로 즐겨 사용됩니다.\n",
    "\n",
    "RNN은 특히 순서 또는 시간에 민감합니다. 즉, 입력 시퀀스의 타입스텝 순서대로 처리합니다. 타입스텝을 섞거나 거꾸로 하면 RNN이 시퀀스에서 학습하는 표현을 완전히 바꾸어 버립니다. 이는 온도 예측과 같이 순서에 의미가 있는 문제에 잘 맞는 이유이기도 합니다. 양방향 RNN은 RNN이 순서에 민감하다는 성질을 사용합니다. 앞서 보았던 GRU나 LSTM 같은 RNN 두 개를 사용합니다. 각 RNN은 입력 시퀀스를 한 방향(시간의 순서나 반대 순서)으로 처리한 다음 각 표현을 합칩니다. 시퀀스를 양쪽 방향으로 처리하기 때문에 양방향 RNN은 단방향 RNN이 놓치기 쉬운 패턴을 감지할 수 있습니다.\n",
    "\n",
    "놀랍게도 이 절에 있는 RNN 층이 시간의 순서대로 (오래된 타임스텝이 먼저 나오도록) 시퀀스를 처리하는 것은 근거 없는 결정입니다. 적어도 이 결정을 궁금해하지 않았습니다. 시간의 반대 방향으로 (최근 타임스텝이 먼저 나오도록) 입력 시퀀스를 처리하면 만족할만한 RNN 성능을 낼 수 있을까요? 실제 이렇게 해 보고 결과가 어떤지 확인해 보죠. 해야 할 일은 입력 시퀀스를 시간 차원을 따라 거꾸로 생성하는 데이터 제너레이터 만드는 것 뿐입니다(제너레이터 함수의 마지막 줄을 `yield samples[:, ::-1, :], targets`로 바꿉니다). 이 절의 첫 번째 예제와 동일하게 하나의 GRU 층을 가진 네트워크를 훈련합니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "def reverse_order_generator(data, lookback, delay, min_index, max_index,\n",
    "                            shuffle=False, batch_size=128, step=6):\n",
    "    if max_index is None:\n",
    "        max_index = len(data) - delay - 1\n",
    "    i = min_index + lookback\n",
    "    while 1:\n",
    "        if shuffle:\n",
    "            rows = np.random.randint(\n",
    "                min_index + lookback, max_index, size=batch_size)\n",
    "        else:\n",
    "            if i + batch_size >= max_index:\n",
    "                i = min_index + lookback\n",
    "            rows = np.arange(i, min(i + batch_size, max_index))\n",
    "            i += len(rows)\n",
    "\n",
    "        samples = np.zeros((len(rows),\n",
    "                           lookback // step,\n",
    "                           data.shape[-1]))\n",
    "        targets = np.zeros((len(rows),))\n",
    "        for j, row in enumerate(rows):\n",
    "            indices = range(rows[j] - lookback, rows[j], step)\n",
    "            samples[j] = data[indices]\n",
    "            targets[j] = data[rows[j] + delay][1]\n",
    "        yield samples[:, ::-1, :], targets\n",
    "        \n",
    "train_gen_reverse = reverse_order_generator(\n",
    "    float_data,\n",
    "    lookback=lookback,\n",
    "    delay=delay,\n",
    "    min_index=0,\n",
    "    max_index=200000,\n",
    "    shuffle=True,\n",
    "    step=step, \n",
    "    batch_size=batch_size)\n",
    "val_gen_reverse = reverse_order_generator(\n",
    "    float_data,\n",
    "    lookback=lookback,\n",
    "    delay=delay,\n",
    "    min_index=200001,\n",
    "    max_index=300000,\n",
    "    step=step,\n",
    "    batch_size=batch_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "500/500 [==============================] - 112s 224ms/step - loss: 0.4750 - val_loss: 0.4889\n",
      "Epoch 2/20\n",
      "500/500 [==============================] - 111s 223ms/step - loss: 0.4524 - val_loss: 0.4769\n",
      "Epoch 3/20\n",
      "500/500 [==============================] - 111s 223ms/step - loss: 0.4186 - val_loss: 0.4593\n",
      "Epoch 4/20\n",
      "500/500 [==============================] - 111s 223ms/step - loss: 0.3778 - val_loss: 0.4280\n",
      "Epoch 5/20\n",
      "500/500 [==============================] - 111s 223ms/step - loss: 0.3553 - val_loss: 0.4672\n",
      "Epoch 6/20\n",
      "500/500 [==============================] - 111s 223ms/step - loss: 0.3386 - val_loss: 0.4349\n",
      "Epoch 7/20\n",
      "500/500 [==============================] - 111s 223ms/step - loss: 0.3224 - val_loss: 0.4291\n",
      "Epoch 8/20\n",
      "500/500 [==============================] - 111s 223ms/step - loss: 0.3109 - val_loss: 0.4187\n",
      "Epoch 9/20\n",
      "500/500 [==============================] - 111s 223ms/step - loss: 0.2995 - val_loss: 0.4242\n",
      "Epoch 10/20\n",
      "500/500 [==============================] - 111s 223ms/step - loss: 0.2899 - val_loss: 0.4294\n",
      "Epoch 11/20\n",
      "500/500 [==============================] - 111s 223ms/step - loss: 0.2809 - val_loss: 0.4371\n",
      "Epoch 12/20\n",
      "500/500 [==============================] - 111s 223ms/step - loss: 0.2734 - val_loss: 0.4408\n",
      "Epoch 13/20\n",
      "500/500 [==============================] - 111s 223ms/step - loss: 0.2665 - val_loss: 0.4509\n",
      "Epoch 14/20\n",
      "500/500 [==============================] - 111s 223ms/step - loss: 0.2598 - val_loss: 0.4555\n",
      "Epoch 15/20\n",
      "500/500 [==============================] - 111s 223ms/step - loss: 0.2554 - val_loss: 0.4587\n",
      "Epoch 16/20\n",
      "500/500 [==============================] - 111s 223ms/step - loss: 0.2503 - val_loss: 0.4498\n",
      "Epoch 17/20\n",
      "500/500 [==============================] - 111s 223ms/step - loss: 0.2465 - val_loss: 0.4415\n",
      "Epoch 18/20\n",
      "500/500 [==============================] - 111s 223ms/step - loss: 0.2408 - val_loss: 0.4556\n",
      "Epoch 19/20\n",
      "500/500 [==============================] - 111s 223ms/step - loss: 0.2374 - val_loss: 0.4519\n",
      "Epoch 20/20\n",
      "500/500 [==============================] - 111s 223ms/step - loss: 0.2341 - val_loss: 0.4727\n"
     ]
    }
   ],
   "source": [
    "model = Sequential()\n",
    "model.add(layers.GRU(32, input_shape=(None, float_data.shape[-1])))\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen_reverse,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=20,\n",
    "                              validation_data=val_gen_reverse,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xl8FFW2wPHfYV9kACGjCELABWVfIuKAgjuKLCqMYBBcGETFZcY3T1RGkRFFZVxQRkXFpxJBRkfFlWEUBcZBCRDCJrJrBAHZETAkOe+PW4EmdJJKekunz/fz6U+6q29Vna7unL59695boqoYY4xJDBViHYAxxpjosaRvjDEJxJK+McYkEEv6xhiTQCzpG2NMArGkb4wxCcSSvikREakoIvtEpHE4y8aSiJwqImHvuywiF4nIhoDHq0TkXD9lS7Gvl0XkvtKuX8R2HxaR/wv3dk3sVIp1ACayRGRfwMMawK9Arvf4ZlVNK8n2VDUXOC7cZROBqjYPx3ZEZCgwSFW7B2x7aDi2bco/S/rlnKoeTrpeTXKoqv67sPIiUklVc6IRmzEm+qx5J8F5P9/fEpGpIrIXGCQi54jIfBHZJSKbRWSCiFT2ylcSERWRZO/xFO/5T0Rkr4j8V0SalrSs9/xlIvKdiOwWkWdF5D8icn0hcfuJ8WYRWSMiO0VkQsC6FUXkKRHZLiJrgR5FHJ9RIjKtwLKJIvKkd3+oiKz0Xs9arxZe2LayRKS7d7+GiLzhxbYc6Bhkv+u87S4Xkd7e8tbAc8C5XtPZzwHHdnTA+sO9175dRN4TkQZ+jk1xRKSvF88uEflcRJoHPHefiGwSkT0i8m3Aa+0sIou85VtE5Am/+zMRoKp2S5AbsAG4qMCyh4FsoBeuElAdOAs4G/dLsBnwHTDCK18JUCDZezwF+BlIASoDbwFTSlH2t8BeoI/33J+AQ8D1hbwWPzG+D9QGkoEd+a8dGAEsBxoB9YA57l8h6H6aAfuAmgHb3gqkeI97eWUEuAA4ALTxnrsI2BCwrSygu3d/PPAFUBdoAqwoUPb3QAPvPbnWi+EE77mhwBcF4pwCjPbuX+LF2A6oBvwd+NzPsQny+h8G/s+7f6YXxwXee3Sfd9wrAy2BjcCJXtmmQDPv/gJgoHe/FnB2rP8XEvlmNX0DME9VP1DVPFU9oKoLVPVrVc1R1XXAJKBbEeu/rarpqnoISMMlm5KWvQLIUNX3veeewn1BBOUzxkdVdbeqbsAl2Px9/R54SlWzVHU7MK6I/awDluG+jAAuBnaparr3/Aequk6dz4HPgKAnawv4PfCwqu5U1Y242nvgfqer6mbvPXkT94Wd4mO7AKnAy6qaoaoHgZFANxFpFFCmsGNTlAHADFX93HuPxgG/wX355uC+YFp6TYTrvWMH7sv7NBGpp6p7VfVrn6/DRIAlfQPwQ+ADETlDRD4SkZ9EZA8wBqhfxPo/BdzfT9Enbwsre1JgHKqquJpxUD5j9LUvXA21KG8CA7371+K+rPLjuEJEvhaRHSKyC1fLLupY5WtQVAwicr2ILPGaUXYBZ/jcLrjXd3h7qroH2Ak0DChTkvessO3m4d6jhqq6Crgb9z5s9ZoLT/SK3gC0AFaJyDcicrnP12EiwJK+AfdzP9CLuNrtqar6G+ABXPNFJG3GNbcAICLC0UmqoFBi3AycHPC4uC6lbwEXeTXlPrgvAUSkOvA28Ciu6aUO8C+fcfxUWAwi0gx4HrgFqOdt99uA7RbXvXQTrskof3u1cM1IP/qIqyTbrYB7z34EUNUpqtoF17RTEXdcUNVVqjoA14T3N+AdEakWYiymlCzpm2BqAbuBX0TkTODmKOzzQ6CDiPQSkUrAnUBShGKcDtwlIg1FpB5wT1GFVXULMA94FVilqqu9p6oCVYBtQK6IXAFcWIIY7hOROuLGMYwIeO44XGLfhvv+G4qr6efbAjTKP3EdxFTgJhFpIyJVccl3rqoW+supBDH3FpHu3r7/jDsP87WInCki53v7O+DdcnEv4DoRqe/9Mtjtvba8EGMxpWRJ3wRzNzAE9w/9Iq6mG1FeYr0GeBLYDpwCLMaNKwh3jM/j2t6X4k4yvu1jnTdxJ2bfDIh5F/BH4F3cydB+uC8vPx7E/eLYAHwCvB6w3UxgAvCNV+YMILAdfBawGtgiIoHNNPnrf4prZnnXW78xrp0/JKq6HHfMn8d9IfUAenvt+1WBx3HnYX7C/bIY5a16ObBSXO+w8cA1qpodajymdMQ1nRpTtohIRVxzQj9VnRvreIwpL6ymb8oMEekhIrW9JoK/4HqEfBPjsIwpVyzpm7KkK7AO10TQA+irqoU17xhjSsFX0vdqYKu8EXwjgzx/vYhsE5EM7zY04LkhIrLauw0JZ/CmfFHVUapaT1VrqWpnVV0Q65iMKW+KbdP32la/ww1KyeLI6LoVAWWux41QHFFg3eOBdNygEgUWAh1VdWcYX4Mxxhif/Ey41glYkz+6zpuHpA9u2HhxLgVmqeoOb91ZuJ/tUwtboX79+pqcnOxj08YYY/ItXLjwZ1Utqpsz4C/pN+TokYNZuGHXBV0tIufhfhX8UVV/KGTdogbckJycTHp6uo+wjDHG5BOR4kaWA/7a9IONLizYJvQBblKtNsC/gddKsC4iMkxE0kUkfdu2bT5CMsYYUxp+kn4WRw8Xb4TrP32Yqm4P6GXxEkemiS12XW/9SaqaoqopSUnF/joxxhhTSn6S/gLcDHlNRaQK3kx7gQXy5+r29AZWevdnApeISF0RqYubjGpm6GEbY4wpjWLb9FU1R0RG4JJ1RWCyqi4XkTFAuqrOAO7wLvKQgxuOfr237g4R+SvuiwNgTP5JXWOMMdFX5qZhSElJUTuRa4wxJSMiC1W12Gsu2IhcY4xJIJb0jTEmgZSbpK8Kf/4zrFxZfFljjElU5Sbpr1kDL78MrVvDXXfBrl2xjsgYY8qecpP0TzsNvvsOhg6FCRPc40mTIDc31pEZY0zZUW6SPkBSErzwAixaBGeeCTffDCkpMNcuwWGMMUA5S/r52rWDL7+Et96C7dvhvPNgwAD44Yfi1zXGmPKsXCZ9ABH4/e/h22/hwQfh/feheXMYMwYOHIh1dMYYExvlNunnq1EDRo+GVaugVy/3BXDGGfCPf7geP8YYk0jKfdLP17ixa+754guoW9f9Cjj/fMjMjHVkxhgTPQmT9PN16wYLF7oTvsuWQfv2cMst8PPPsY7MGGMir9wk/bQ0SE6GChXc37S0wstWrOh69qxeDbffDi+95Lp4Pvss5OREK2JjjIm+cpH009Jg2DDYuNG102/c6B4XlfjBNfM8/bRr4jnrLLjjDrj8cti9OzpxG2NMtJWLpH///bB//9HL9u93y/1o0QJmzoRXXoHZs+F3v4MNG8IepjHGxFy5SPrff1+y5cGIwI03wr/+BZs2wdlnw/z54YnPGGOKc/Ag7NsX+f2Ui6TfuHHJlhfl/PNdsq9VC7p3dz1+jDEmkg4ehCuvhJ49Iz91TLlI+mPHuv74gWrUcMtLo3lzl/hTUtxI3rFjrU+/MSYyfv0Vrr4aPv0UBg92HU0iqVwk/dRUN7lakyaumaZJE/c4NbX026xfHz77zG1j1Ci44QbIzg5fzMYYk5/wP/7Y5aybbor8Pou9Rm68SE0NLckHU7UqvPEGnH66G8m7YQP8859w/PHh3Y8xJvFkZ0P//vDRR27c0B/+EJ39louafiSJwAMPuO6f//0vdO7s+veXNarw5JNuwJkxpmzLznazAnzwAUyc6MYNRYslfZ+uvRY+/xx27nSJv6xN1zxjBtx9t/upePBgrKMxxhTm0CF3rvD9992A0Ftvje7+LemXQJcu7gRvUhJceKFr+ikLcnPdmISkJHchmYcfjnVExphgDh2CgQPh3XfhmWdgxIjox2BJv4ROOcU183Tt6s60P/BA7Hv2vPkmLF8Ozz0H118Pjz1mE8kZU9bk5Ljzju+845pi77gjNnGIxjpjFZCSkqLp6emxDqNY2dluorbJk9039+TJUK1abOI44wyoUwfS013zU4sWrgfTf/8b+e5fxpji5eTAoEFu3M/48a4pNtxEZKGqphRXzmr6pVSlirsQ+7hxMHWqa+7Zti36cbz0EqxfD4884iabq1fPXSN4wQLXXmiMia2cHNcq8NZb8PjjkUn4JWE1/TB4+2247jpo0ABmzXJNQNHwyy9uX2ec4eYMEnHLVaF3b3fiedkyaNo0OvGYxJaT435Z5n8OjTvfNmSI6/336KMwcmTk9mU1/Sjq189dk3fXLtdmF+lh1PmeeQa2bHEfpsB/NBH4+99dzX/48NifczDlh6r7RTtvnpug8H//11UwmjeH6tWhWTPXfLFzZ6wjjb3cXDeoMy3NjeqPZMIvCavph9HUqa5r5xNPwP/8T2T3tWOH+wc77zzXXTOYiRNd74DXX3e/RIzxKzsb1q51lxn99lv3N/+2Y8eRclWquMGLzZu7a1J89RXMmeOmQbnuOneyskWL2L2OWMnNdaNrX3sN/vpXN6o/0vzW9C3ph5GqmzTp009hyRL3jxApI0e69sElS6B16+Bl8vJcL6NVq2DlSvjtbyMXj4lfW7e6k/7z57vmwG+/deeJAn+xNmjgPs/Nm7vmxPz7TZoc21kgI8OdT0pLc9MMXHTRkWtVlNWOBb/+6n6hrF4NbdpA27bub1JSybeVlwdDh8Krr7rrcz/4YNjDDcpv0kdVy9StY8eOGgtTpqg2aaIq4v5OmVK67WzapFq3rurvfqeakxPOCI/48UfVatVUU1OLL7t8uWrlyqrXXhuZWEx8yc5WXbBA9dln3WeiWTNVV11RrVRJtXVr1f79VUeNUn3jDVd29+7S7WvbNtVHHlFt1Mhtv1kz1SefVN25M7yvKVRLlqi2aeNiTEo6cjxAtUED1UsvVf3f/3U5YelSdwwLk5uretNNbt0HHojea1BVBdLVR471lYiBHsAqYA0wsohy/QAFUrzHycABIMO7vVDcvmKR9KdMUa1R4+g3u0aN0if+119323jqqfDGmW/4cPcPumaNv/KjR7t4PvooMvGYsmvTJtV//lP1z39WPfdc1erVj3zGTzpJ9eqrVcePV503T3X//sjEkJ2tOn26ateubr81a6reeqvqihWR2Z9fOTmq48a5StEJJ6h+8IFbvmWL6qxZqn/7m+rgwart2qlWqXLkuFWp4pYNHuzKzJqlunWrS/h/+IMrc//9qnl50X09YUv6QEVgLdAMqAIsAVoEKVcLmAPML5D0l/kJJP8Wi6TfpMnRCT//1qRJ6baXl6fas6f7B1u9OpyRukRfqZL7p/Hr4EHVFi1UTz5Zdc+e8MZjyo5ff1X9+mvVp59WHTDg6M91lSqqnTur3nWX6ltvqX7/ffSTkqrqokWqN9ygWrWqi+uSS1Q//NAlzGhas8b9GgfVfv3cr5KiZGe7Wv6UKa7Wf+ml7ldAYL6oU8f9vffe2BzbcCb9c4CZAY/vBe4NUu5p4Argi3hL+iLBk75I6beZlaVau7bqeeeF9wN97bXuy2TTppKt99VX7vXccUf4YjFlw549qo89dnTTRKNGrpnmySfde3/wYKyjPNrWrapjx6o2bOjiPfVU92W1a1dk95uXp/r88+7XRp06qmlpoSXorVtV//1vV+MfMsS9hlgkfNXwJv1+wMsBj68DnitQpj3wjne/YNL/BVgMfAmcW8g+hgHpQHrjxo2jcoAChbumn++VV9x2nnsuHFGqZmS4xD1yZOnWHzHCrf/f/4YnHhNbu3apPvyw6vHHH6k1T5+u+sMPsY7Mv+xs98ujSxc93Kw6dKg7lxBuWVmqPXq4/Vx8cXwdJz/CmfT7B0n6zwY8ruAl+mQ9NulXBep59zsCPwC/KWp/5aFNP19envsZWLOm6rp1ocfZs6ernezYUbr19+xxNcCWLV1TgIlPO3aoPvjgkeaEnj1V58+PdVShS093CT//f7FDB9VJk1T37g1921Onug4W1aurTpwYu9p4JEWteQeoDfwMbPBuB4FN+Ym/wLa+CLY88BbvvXcK2rhRtVYt1QsuCO2DNm+ee7ceeSS0eD74wG1nzJjQtmOib9s21fvuc58nUO3b1yXK8mbXLpeYW7d2r7NWLdd5ISOj5Nv6+WfVa65x2+ncWfW778Ifb1kRzqRfCVgHNA04kduyiPKBNf0koKJ3vxnwI3B8UfuLVdKPpBdfdEf6hRdKt35enut5ccIJqvv2hR7PgAHuxF6se08Yf376yfW+qVnTVUr693fdDMu7vDx3PmLIENdFGVTPPlt18mTVX34pfv2PP3YnWytXdpWlQ4ciHnJMhS3pu21xOfCd14vnfm/ZGKB3kLKBSf9qYLn3RbEI6FXcvspj0s/LU73wQldj2bix5Ot//LF7pyZODE88W7a4duAuXaLfa8L49+OPrrdN9eqqFSq4k/jLl8c6qtjYvt2dJD3jDPe/ULu26u23qy5bdmzZvXtVhw1z5Vq1Ul28OPrxxkJYk340b+Ux6auqrl/vamqXXFKyZp7cXNcnuGnT8LbDv/aae/f//vfwbdOEx/ffq952m+vWWLGiq+muWhXrqMqGvDzVL79UHTjwSN/5rl3dQLIDB1TnznWDwERc18qy1mspkizpl0HPPeeO+Cuv+F9n6lS3zhtvhDeWvDzXg6FWrfLXiyEe5eS4fuDDhrnmiMqV3UnNtWtjHVnZtXWr6uOPu+6e+f3kRVzSnzs31tFFn9+kb3PvRFFeHlxwASxe7K501ahR0eUPHXKTVVWr5uYzCfe8JevXQ6tW7loA779vU+JGy88/w9Kl7upm+bdly9y1jatUcfO23HMPNG4c60jjQ16em1p88mQ44QQYMwaOOy7WUUWf37l3KkUjGONUqOCmo23TBm6+GT78sOhE++qrsGaNS8iRmKiqaVM3A+Ddd7trAvTvH/59JLLsbDd5WWByX7oUNm06UiYpyU3udeut7nNx8cVw0kmxizkeVajgKi4XXhjrSOKD1fRj4Jln4K673LSrgwcHL3PgAJx6qpvF8D//iVwtPCcHzjkHvv/ezcR5/PGR2U95l5fn5pifP/9Igl+50h1fcDX4Fi1cYg+8nXBCbOM25YfV9Muw22+Hf/wD7rzT1ewaNDi2zMSJrkb45puRbXapVMld9rFjR/jzn90vEePfmjXuegWvvw4bN7plJ5/sEvoVVxxJ7qedBpUrxzZWY8CSfkxUqODaH9u2dRdXf/fdoxP77t3ualiXXgrdukU+nrZt3RWQHn3U/bK48krX1m9t/MHt3u2+tF97zdXuRdyX9yOPQI8e9mvJlG12ucQYOf10157+/vswbdrRz40f765O9Mgj0YvnL3+B7t3dBR/atHEnmW+6CaZPP/pKSYkqNxf+9S93ZbQTT4Q//MGdkH30Udc0NnOme84SvinrrE0/hnJz3ZWtvvsOVqxw7btbtriLnV9+uUu40ZaV5RLYzJnuIu+7drlfJp06uVrspZfCWWeV3SsghdvKla5GP2UK/Pgj1K0LAwe6i12fdZb9GjJlh10uMU6sXAnt27v237ffdu38Eye6Lp2RvNyiHzk5sGCBu/zjp5+6+6quNnvxxe4L4NJLy19vkx073K+v116Db75xX3CXXeYSfa9eULVqrCM05liW9OPIuHFw773ugur33+969Lz0UqyjOtb27a72P3Om+xL46Se3vE0bl/x79HAXaq8Uh2eKdu50fb2nTnUXms/Odq9ryBBITbVeNqbss6QfR/K7Taanu1rkmjXFD9yKNVXXLTH/C2DePDeY7JRT4IEHXPt2WU7+O3fC3LnwxRcu2S9Z4l5TUpKL/frroV27WEdpjH+W9OPMsmWujfjOO13NP97s2weffOJOPmdkuBPVDz4I11xTNtr/d+06kuS/+MKNilZ1X7K/+x2cf747kd25s3WtNPHJkn4c2r7dtZfH88lBVXjvPRg92v0SOPNMl/z793cnhKNl9+5jk3xenkvy55xzJMl36uSmuTAm3lnSNzGVlwf//KdL+CtWuH7/o0e7MQCRSP6//AJz5sDnn7skv2iRi6FKFZfku3d3if7ssy3Jm/LJkr4pE3Jz3UCm0aNh1So3EOyhh6B379B+0eTmwsKF7sTyrFnw1VfunEKVKq6JJjDJV68erldjTNllSd+UKbm5rmfMQw+5E9UdOrj7PXv6T/7r1x9J8p995k7Ggjvhesklrhtply6W5E1i8pv0bURumKSlQXKya7pITnaPzREVK8KgQW5cwquvuoTdq5eriX/6qTsXUNCuXa6J6JZb3ORzzZq52Unnz4e+fd28RFu2uPb6xx6Diy6yhG9McaymHwZpaTBsGOzff2RZjRowaZLr422OdeiQm6Tsr391E5Wdc45rAqpW7UhtfsEC1y5/3HGuqebii92tefP4PtltTCRY804UJScfmWExUJMmsGFDtKOJL9nZrub/8MNuCgg4Mu3DxRe7Zpuzz7ZulMYUx5J+FFWoELx5QsTVVE3xfv3VnfCtWdPV6uvUiXVExsQXm08/iho3Dl7Tt8vd+Ve1qmvzN8ZElp3IDYOxY10bfqAaNdxyY4wpSyzph0Fqqjtp26SJa9Jp0sRO4hpjyiZr3gmT1FRL8saYss9q+sYYk0As6RtjTAKxpG+MMQnEkr4xxiQQS/rGGJNALOkbY0wC8ZX0RaSHiKwSkTUiMrKIcv1EREUkJWDZvd56q0Tk0nAEbYwxpnSK7acvIhWBicDFQBawQERmqOqKAuVqAXcAXwcsawEMAFoCJwH/FpHTVTU3fC/BGGOMX35q+p2ANaq6TlWzgWlAnyDl/go8DhwMWNYHmKaqv6rqemCNtz1jjDEx4CfpNwR+CHic5S07TETaAyer6oclXddbf5iIpItI+rZt23wFbowxpuT8JP1gl6s4PJGwiFQAngLuLum6hxeoTlLVFFVNSUpK8hGSMcaY0vAz904WcHLA40bApoDHtYBWwBfiLmd0IjBDRHr7WNcYY0wU+anpLwBOE5GmIlIFd2J2Rv6TqrpbVeurarKqJgPzgd6qmu6VGyAiVUWkKXAa8E3YX4Uxxhhfiq3pq2qOiIwAZgIVgcmqulxExgDpqjqjiHWXi8h0YAWQA9xmPXeMMSZ27HKJxhhTDvi9XKKNyDXGmARiSd8YYxKIJX1jjEkglvSNMSaBWNI3xpgEYknfGGMSiCV9Y4xJIJb0jTEmgVjSLyPS0iA5GSpUcH/T0mIdkTGmPPIz4ZqJsLQ0GDYM9u93jzdudI8BUlNjF5cxpvyxmn4ZcP/9RxJ+vv373XJjjAknS/plwPffl2y5McaUliX9MqBx45ItN8aY0rKkXwaMHQs1ahy9rEYNt9wYY8LJkn4ZkJoKkyZBkyYg4v5OmmQncY0x4We9d8qI1FRL8saYyLOavjHGJBBL+sYYk0As6RtjTAKxpG+MMQnEkr4xxiQQS/rGGJNALOkbY0wCsaRvjDEJxJK+McYkEEv6xhiTQCzplxN25S1jjB829045YFfeMsb4ZTX9csCuvGWM8cuSfjlgV94yxvjlK+mLSA8RWSUia0RkZJDnh4vIUhHJEJF5ItLCW54sIge85Rki8kK4X4CxK28ZY/wrNumLSEVgInAZ0AIYmJ/UA7ypqq1VtR3wOPBkwHNrVbWddxsersDNEXblLWOMX35q+p2ANaq6TlWzgWlAn8ACqron4GFNQMMXoimOXXnLGOOXn947DYEfAh5nAWcXLCQitwF/AqoAFwQ81VREFgN7gFGqOjfIusOAYQCNrU2iVOzKW8YYP/zU9CXIsmNq8qo6UVVPAe4BRnmLNwONVbU97gvhTRH5TZB1J6lqiqqmJCUl+Y/eGGNMifhJ+lnAyQGPGwGbiig/DegLoKq/qup27/5CYC1weulCNcYYEyo/SX8BcJqINBWRKsAAYEZgARE5LeBhT2C1tzzJOxGMiDQDTgPWhSNwY4wxJVdsm76q5ojICGAmUBGYrKrLRWQMkK6qM4ARInIRcAjYCQzxVj8PGCMiOUAuMFxVd0TihRhjjCmeqJatjjYpKSmanp4e6zCMMSauiMhCVU0prpyNyDWATdhmTKKwCdeMTdhmTAKxmr6xCduMSSCW9I1N2GZMArGkb2zCNmMSiCV9YxO2GZNALOkbm7DNmARivXcMYBO2GZMorKZvjDEJxJK+McYkEEv6xhiTQCzpG2NMArGkb8LC5u4xJj5Y7x0TMpu7x5j4YTV9EzKbu8eY+GFJ34TM5u4xJn5Y0jchs7l7jIkflvRNyGzuHmPihyV9EzKbu8eY+GG9d0xY2Nw9xsQHq+kbY0wCsaRvygQb3GVMdFjzjok5G9xlTPRYTd/EnA3uMiZ6LOmbmLPBXcZEjyV9E3M2uMuY6LGkb2LOBncZEz2W9E3M2eAuY6LHeu+YMsEGdxkTHVbTN8aYBOIr6YtIDxFZJSJrRGRkkOeHi8hSEckQkXki0iLguXu99VaJyKXhDN6YfDa4yxh/im3eEZGKwETgYiALWCAiM1R1RUCxN1X1Ba98b+BJoIeX/AcALYGTgH+LyOmqmhvm12ESmA3uMsY/PzX9TsAaVV2nqtnANKBPYAFV3RPwsCag3v0+wDRV/VVV1wNrvO0ZEzY2uMsY//ycyG0I/BDwOAs4u2AhEbkN+BNQBbggYN35BdZtGGTdYcAwgMbWOduUkA3uMsY/PzV9CbJMj1mgOlFVTwHuAUaVcN1JqpqiqilJSUk+QjLmCBvcZYx/fpJ+FnBywONGwKYiyk8D+pZyXWNKzAZ3GeOfn6S/ADhNRJqKSBXcidkZgQVE5LSAhz2B1d79GcAAEakqIk2B04BvQg/bmCNscJcx/hXbpq+qOSIyApgJVAQmq+pyERkDpKvqDGCEiFwEHAJ2AkO8dZeLyHRgBZAD3GY9d0wk2OAuY/wR1WOa2GMqJSVF09PTYx2GSTBpaa63z/ffu3MBY8fal4iJLyKyUFVTiitn0zCYhGf9/E0isWkYTMKzfv4mkVjSNwnP+vmbRGJJ3yQ86+dvEoklfZPwrJ+/SSSW9E3Cs37+JpFY0jcGl+A3bIC8PPe3pAnfpnY28cK6bBoTIuvyaeKJ1fSNCZF1+TTxxJIvOKbEAAARPUlEQVS+MSGyLp8mnljSNyZE1uXTxBNL+saEKBxdPu1EsIkWS/rGhCjULp/5J4I3bgTVIyeCLfGbSLBZNo2JseRkl+gLatLEdR81xg+/s2xaTd+YGLMTwSaaLOkbE2N2IthEkyV9Y2LM5v4x0RQXI3IPHTpEVlYWBw8ejHUoxodq1arRqFEjKleuHOtQ4kL+Cd9QrtxlV/4yfsXFidz169dTq1Yt6tWrh4jEKDLjh6qyfft29u7dS9OmTWMdTkIoOA0EuF8KNmlcYilXJ3IPHjxoCT9OiAj16tWzX2VRZNNAmJKIi6QPWMKPI/ZeRZf1/jElETdJ3xgTnPX+MSVRLpN+uIe0b9++nXbt2tGuXTtOPPFEGjZsePhxdna2r23ccMMNrFq1qsgyEydOJC1MwzC7du1KRkZGWLZlyjabBsKUiKqWqVvHjh21oBUrVhyzrDBTpqjWqKHqBrS7W40abnk4PPjgg/rEE08cszwvL09zc3PDs5Mw6NKliy5evDhm+y/Je2ZCN2WKapMmqiLub0k+75H+nzHRAaSrjxxb7mr60TyptWbNGlq1asXw4cPp0KEDmzdvZtiwYaSkpNCyZUvGjBlzuGx+zTsnJ4c6deowcuRI2rZtyznnnMPWrVsBGDVqFE8//fTh8iNHjqRTp040b96cr776CoBffvmFq6++mrZt2zJw4EBSUlKKrdFPmTKF1q1b06pVK+677z4AcnJyuO666w4vnzBhAgBPPfUULVq0oG3btgwaNCjsx8xERihX/rITwYml3CX9aJ/UWrFiBTfddBOLFy+mYcOGjBs3jvT0dJYsWcKsWbNYsWLFMevs3r2bbt26sWTJEs455xwmT54cdNuqyjfffMMTTzxx+Avk2Wef5cQTT2TJkiWMHDmSxYsXFxlfVlYWo0aNYvbs2SxevJj//Oc/fPjhhyxcuJCff/6ZpUuXsmzZMgYPHgzA448/TkZGBkuWLOG5554L8eiYeBCO/xlrHoof5S7pR/uk1imnnMJZZ511+PHUqVPp0KEDHTp0YOXKlUGTfvXq1bnssssA6NixIxsKmVXrqquuOqbMvHnzGDBgAABt27alZcuWRcb39ddfc8EFF1C/fn0qV67Mtddey5w5czj11FNZtWoVd955JzNnzqR27doAtGzZkkGDBpGWlmaDqxJEqP8zNktofCl3ST/aQ9pr1qx5+P7q1at55pln+Pzzz8nMzKRHjx5B+6tXqVLl8P2KFSuSk5MTdNtVq1Y9poyWcDBdYeXr1atHZmYmXbt2ZcKECdx8880AzJw5k+HDh/PNN9+QkpJCbm5uifZn4k+o/zPWPBRfyl3SD3Vu81Ds2bOHWrVq8Zvf/IbNmzczc+bMsO+ja9euTJ8+HYClS5cG/SURqHPnzsyePZvt27eTk5PDtGnT6NatG9u2bUNV6d+/Pw899BCLFi0iNzeXrKwsLrjgAp544gm2bdvG/oL/zabcCfV/xsYJxBdfc++ISA/gGaAi8LKqjivw/J+AoUAOsA24UVU3es/lAku9ot+rau8wxV6o1NTYDD/v0KEDLVq0oFWrVjRr1owuXbqEfR+33347gwcPpk2bNnTo0IFWrVodbpoJplGjRowZM4bu3bujqvTq1YuePXuyaNEibrrpJlQVEeGxxx4jJyeHa6+9lr1795KXl8c999xDrVq1wv4aTNkTyv9M48bBrwdQkiZVmzsoiorr3oNL9GuBZkAVYAnQokCZ84Ea3v1bgLcCntvnpxtR/i3ULpvl3aFDh/TAgQOqqvrdd99pcnKyHjp0KMZRHcves8QRapdP6zIaHvjssumnpt8JWKOq6wBEZBrQBzjcrqCqswPKzwesr1+E7Nu3jwsvvJCcnBxUlRdffJFKleJislRTToU6S2hR5wSsth9+frJFQ+CHgMdZwNlFlL8J+CTgcTURScc1/YxT1fdKHKU5rE6dOixcuDDWYRhzlFCah8LVZdSah/zxk/SDzZ4VtEuIiAwCUoBuAYsbq+omEWkGfC4iS1V1bYH1hgHDABrbhCHGJJRQzwkUnFo6v8soWOIPxk/vnSzg5IDHjYBNBQuJyEXA/UBvVf01f7mqbvL+rgO+ANoXXFdVJ6lqiqqmJCUllegFGGPim3UZjS4/SX8BcJqINBWRKsAAYEZgARFpD7yIS/hbA5bXFZGq3v36QBcCzgUYY4x1GY2uYpt3VDVHREYAM3E9eSar6nIRGYM7WzwDeAI4DviHN5d6ftfMM4EXRSQP9wUzTlUt6RtjjhLrLqOJxNfgLFX9WFVPV9VTVHWst+wBL+Gjqhep6gmq2s679faWf6WqrVW1rff3lci9lMjp3r37MQOtnn76aW699dYi1zvuuOMA2LRpE/369St02wUvD1nQ008/fdQgqcsvv5xdu3b5Cb1Io0ePZvz48SFvx5hYsqmlS6bcjciNhIEDBzJt2rSjlk2bNo2BAwf6Wv+kk07i7bffLvX+Cyb9jz/+mDp16pR6e8aUJ6E2DyXa3EFx18H7rrsg3NcGadcOvBmNg+rXrx+jRo3i119/pWrVqmzYsIFNmzbRtWtX9u3bR58+fdi5cyeHDh3i4Ycfpk+fPketv2HDBq644gqWLVvGgQMHuOGGG1ixYgVnnnkmBw4cOFzulltuYcGCBRw4cIB+/frx0EMPMWHCBDZt2sT5559P/fr1mT17NsnJyaSnp1O/fn2efPLJw7N0Dh06lLvuuosNGzZw2WWX0bVrV7766isaNmzI+++/T/Xq1Qt9jRkZGQwfPpz9+/dzyimnMHnyZOrWrcuECRN44YUXqFSpEi1atGDatGl8+eWX3HnnnYC7NOKcOXNs5K6JqVCah8IxTiCeuoxaTd+HevXq0alTJz799FPA1fKvueYaRIRq1arx7rvvsmjRImbPns3dd99d5KRozz//PDVq1CAzM5P777//qD73Y8eOJT09nczMTL788ksyMzO54447OOmkk5g9ezazZ88+alsLFy7k1Vdf5euvv2b+/Pm89NJLh6daXr16NbfddhvLly+nTp06vPPOO0W+xsGDB/PYY4+RmZlJ69ateeihhwAYN24cixcvJjMzkxdeeAGA8ePHM3HiRDIyMpg7d26RXybGlHWhngiOt18KcVfTL6pGHkn5TTx9+vRh2rRph2vXqsp9993HnDlzqFChAj/++CNbtmzhxBNPDLqdOXPmcMcddwDQpk0b2rRpc/i56dOnM2nSJHJycti8eTMrVqw46vmC5s2bx5VXXnl4ps+rrrqKuXPn0rt3b5o2bUq7du2AoqdvBje//65du+jWzQ2vGDJkCP379z8cY2pqKn379qVv374AdOnShT/96U+kpqZy1VVX0ahRIz+H0JgyKdQTwfE2othq+j717duXzz77jEWLFnHgwAE6dOgAQFpaGtu2bWPhwoVkZGRwwgknBJ1OOZDXw+ko69evZ/z48Xz22WdkZmbSs2fPYrdT1C+K/GmZoejpm4vz0Ucfcdttt7Fw4UI6duxITk4OI0eO5OWXX+bAgQN07tyZb7/9tlTbNqYsCPVEcLxdhMaSvk/HHXcc3bt358YbbzzqBO7u3bv57W9/S+XKlZk9ezYbg1UZApx33nmHL36+bNkyMjMzATctc82aNalduzZbtmzhk0+OzGRRq1Yt9u7dG3Rb7733Hvv37+eXX37h3Xff5dxzzy3xa6tduzZ169Zl7ty5ALzxxht069aNvLw8fvjhB84//3wef/xxdu3axb59+1i7di2tW7fmnnvuISUlxZK+iWuhngiOt4vQxF3zTiwNHDiQq6666qiePKmpqfTq1YuUlBTatWvHGWecUeQ2brnlFm644QbatGlDu3bt6NSpE+CugtW+fXtatmx5zLTMw4YN47LLLqNBgwZHtet36NCB66+//vA2hg4dSvv27YtsyinMa6+9dvhEbrNmzXj11VfJzc1l0KBB7N69G1Xlj3/8I3Xq1OEvf/kLs2fPpmLFirRo0eLwVcCMiVehnAgeO/boaSAgfCOKI9E8JEU1EcRCSkqKFuy3vnLlSs4888wYRWRKw94zk0hC6b1ToYKr4Rck4i5075eILFTVlOLKWU3fGGNCFE8jiq1N3xhjYija1/WOm6Rf1pqhTOHsvTLGv2hf1zsumneqVavG9u3bqVevXtDujqbsUFW2b99OtWrVYh2KMXEjmtf1jouk36hRI7Kysti2bVusQzE+VKtWzQZsGVNGxUXSr1y5Mk2bNo11GMYYE/fipk3fGGNM6CzpG2NMArGkb4wxCaTMjcgVkW1A0RPYxFZ94OdYB1EEiy80Fl9oLL7QhBJfE1VNKq5QmUv6ZZ2IpPsZ6hwrFl9oLL7QWHyhiUZ81rxjjDEJxJK+McYkEEv6JTcp1gEUw+ILjcUXGosvNBGPz9r0jTEmgVhN3xhjEoglfWOMSSCW9AsQkZNFZLaIrBSR5SJyZ5Ay3UVkt4hkeLcHYhDnBhFZ6u0/PcjzIiITRGSNiGSKSIcoxtY84NhkiMgeEbmrQJmoHkMRmSwiW0VkWcCy40Vklois9v7WLWTdIV6Z1SIyJIrxPSEi33rv37siUqeQdYv8LEQwvtEi8mPAe3h5Iev2EJFV3mdxZBTjeysgtg0iklHIutE4fkHzSkw+g6pqt4Ab0ADo4N2vBXwHtChQpjvwYYzj3ADUL+L5y4FPAAE6A1/HKM6KwE+4gSMxO4bAeUAHYFnAsseBkd79kcBjQdY7Hljn/a3r3a8bpfguASp59x8LFp+fz0IE4xsN/I+P938t0AyoAiwp+P8UqfgKPP834IEYHr+geSUWn0Gr6RegqptVdZF3fy+wEmgY26hKpQ/wujrzgToi0iAGcVwIrFXVmI6yVtU5wI4Ci/sAr3n3XwP6Bln1UmCWqu5Q1Z3ALKBHNOJT1X+pao73cD4Qs/mqCzl+fnQC1qjqOlXNBqbhjntYFRWfuItw/B6YGu79+lVEXon6Z9CSfhFEJBloD3wd5OlzRGSJiHwiIi2jGpijwL9EZKGIDAvyfEPgh4DHWcTmy2sAhf+zxfoYnqCqm8H9UwK/DVKmrBzHG3G/3IIp7rMQSSO85qfJhTRNlIXjdy6wRVVXF/J8VI9fgbwS9c+gJf1CiMhxwDvAXaq6p8DTi3DNFW2BZ4H3oh0f0EVVOwCXAbeJyHkFng92ibGo9s8VkSpAb+AfQZ4uC8fQj7JwHO8HcoC0QooU91mIlOeBU4B2wGZcE0pBMT9+wECKruVH7fgVk1cKXS3IslIfQ0v6QYhIZdwbk6aq/yz4vKruUdV93v2PgcoiUj+aMarqJu/vVuBd3M/oQFnAyQGPGwGbohPdYZcBi1R1S8EnysIxBLbkN3l5f7cGKRPT4+idtLsCSFWvgbcgH5+FiFDVLaqaq6p5wEuF7DfWx68ScBXwVmFlonX8CskrUf8MWtIvwGv/ewVYqapPFlLmRK8cItIJdxy3RzHGmiJSK/8+7oTfsgLFZgCDvV48nYHd+T8jo6jQGlasj6FnBpDfE2II8H6QMjOBS0Skrtd8cYm3LOJEpAdwD9BbVfcXUsbPZyFS8QWeI7qykP0uAE4TkabeL78BuOMeLRcB36pqVrAno3X8isgr0f8MRvKMdTzegK64n06ZQIZ3uxwYDgz3yowAluN6IswHfhflGJt5+17ixXG/tzwwRgEm4npOLAVSohxjDVwSrx2wLGbHEPflsxk4hKs53QTUAz4DVnt/j/fKpgAvB6x7I7DGu90QxfjW4Npy8z+HL3hlTwI+LuqzEKX43vA+W5m45NWgYHze48txvVXWRjM+b/n/5X/mAsrG4vgVllei/hm0aRiMMSaBWPOOMcYkEEv6xhiTQCzpG2NMArGkb4wxCcSSvjHGJBBL+sYYk0As6RtjTAL5f7ah+Tv21hMQAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss) + 1)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "순서를 뒤집은 GRU는 상식 수준의 기준점보다도 성능이 낮습니다. 이 경우에는 시간 순서대로 처리하는 것이 중요한 역할을 합니다. 사실 이는 당연합니다. 기본적인 GRU 층은 먼 과거보다 최근 내용을 잘 기억합니다. 또한, 최근에 가까운 날씨 데이터 포인트일수록 오래된 데이터 포인트보다 예측에 유용합니다(상식 수준의 기준점이 꽤 강력한 이유입니다). 시간 순서대로 처리하는 네트워크가 거꾸로 처리하는 것보다 성능이 높아야만 합니다. 하지만 자연어 처리를 포함하여 다른 많은 문제에서는 그렇지 않습니다. 문장을 이해하는데 있어서 단어의 중요성은 단어가 문장 어디에 놓여 있는지에 따라 결정되지 않습니다. 같은 기법을 이전 절의 LSTM IMDB 예제에 적용해 보죠:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 20000 samples, validate on 5000 samples\n",
      "Epoch 1/10\n",
      "20000/20000 [==============================] - 76s 4ms/step - loss: 0.4841 - acc: 0.7701 - val_loss: 0.3777 - val_acc: 0.8588\n",
      "Epoch 2/10\n",
      "20000/20000 [==============================] - 75s 4ms/step - loss: 0.3136 - acc: 0.8791 - val_loss: 0.6957 - val_acc: 0.7904\n",
      "Epoch 3/10\n",
      "20000/20000 [==============================] - 75s 4ms/step - loss: 0.2685 - acc: 0.9016 - val_loss: 0.3353 - val_acc: 0.8630\n",
      "Epoch 4/10\n",
      "20000/20000 [==============================] - 75s 4ms/step - loss: 0.2195 - acc: 0.9195 - val_loss: 0.5092 - val_acc: 0.8182\n",
      "Epoch 5/10\n",
      "20000/20000 [==============================] - 75s 4ms/step - loss: 0.1973 - acc: 0.9317 - val_loss: 0.3785 - val_acc: 0.8692\n",
      "Epoch 6/10\n",
      "20000/20000 [==============================] - 75s 4ms/step - loss: 0.1733 - acc: 0.9383 - val_loss: 0.3698 - val_acc: 0.8780\n",
      "Epoch 7/10\n",
      "20000/20000 [==============================] - 75s 4ms/step - loss: 0.1595 - acc: 0.9451 - val_loss: 0.4101 - val_acc: 0.8754\n",
      "Epoch 8/10\n",
      "20000/20000 [==============================] - 75s 4ms/step - loss: 0.1450 - acc: 0.9508 - val_loss: 0.3898 - val_acc: 0.8564\n",
      "Epoch 9/10\n",
      "20000/20000 [==============================] - 75s 4ms/step - loss: 0.1362 - acc: 0.9556 - val_loss: 0.3874 - val_acc: 0.8628\n",
      "Epoch 10/10\n",
      "20000/20000 [==============================] - 75s 4ms/step - loss: 0.1230 - acc: 0.9575 - val_loss: 0.4713 - val_acc: 0.8704\n"
     ]
    }
   ],
   "source": [
    "from keras.datasets import imdb\n",
    "from keras.preprocessing import sequence\n",
    "from keras import layers\n",
    "from keras.models import Sequential\n",
    "\n",
    "# 특성으로 사용할 단어의 수\n",
    "max_features = 10000\n",
    "# 사용할 텍스트의 길이(가장 빈번한 max_features 개의 단어만 사용합니다)\n",
    "maxlen = 500\n",
    "\n",
    "# 데이터 로드\n",
    "(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)\n",
    "\n",
    "# 시퀀스를 뒤집습니다\n",
    "x_train = [x[::-1] for x in x_train]\n",
    "x_test = [x[::-1] for x in x_test]\n",
    "\n",
    "# 시퀀스에 패딩을 추가합니다\n",
    "x_train = sequence.pad_sequences(x_train, maxlen=maxlen)\n",
    "x_test = sequence.pad_sequences(x_test, maxlen=maxlen)\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.Embedding(max_features, 128))\n",
    "model.add(layers.LSTM(32))\n",
    "model.add(layers.Dense(1, activation='sigmoid'))\n",
    "\n",
    "model.compile(optimizer='rmsprop',\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=['acc'])\n",
    "history = model.fit(x_train, y_train,\n",
    "                    epochs=10,\n",
    "                    batch_size=128,\n",
    "                    validation_split=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "시간 순서로 훈련한 LSTM과 거의 동일한 성능을 얻을 수 있습니다. 놀랍게도 이런 텍스트 데이터셋에는 순서를 뒤집어 처리하는 것이 시간 순서대로 처리하는 것과 거의 동일하게 잘 작동합니다. 이는 언어를 이해하는 데 단어의 순서가 중요하지만 결정적이지는 않다는 가정을 뒷받침합니다. 거꾸로 된 시퀀스에서 훈련한 RNN은 원래 시퀀스에서 훈련한 것과는 다른 표현을 학습합니다. 이와 비슷하게 시작할 때 죽고 마지막 날 태어나는 삶처럼 실제 세상의 시간이 거꾸로 흘러간다면 우리의 정신 세계가 달라질 것입니다. 머신 러닝에서 다른 표현이 유용하다면 항상 사용할 가치가 있습니다. 이 표현이 많이 다를수록 더 좋습니다. 이 표현이 데이터를 바라보는 새로운 시각을 제공하고 다른 방식에서는 놓칠 수 있는 데이터의 특징을 잡아냅니다. 이런 표현은 작업의 성능을 올리는 데 도움을 줍니다. 이것이 다음 장에서 살펴볼 앙상블의 개념입니다.\n",
    "\n",
    "양방향 RNN은 이 아이디어를 사용하여 시간 순서대로 처리하는 RNN의 성능을 향상시킵니다. 입력 시퀀스를 양쪽 방향으로 바라보기 때문에, 드러나지 않은 다양한 표현을 얻어 시간 순서대로 처리할 때 놓칠 수 있는 패턴을 잡아 냅니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![bidirectional rnn](https://s3.amazonaws.com/book.keras.io/img/ch6/bidirectional_rnn.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "케라스에서는 `Bidirectional` 층을 사용하여 양방향 RNN을 만듭니다. 이 클래스는 첫 번째 매개변수로 순환 층의 객체를 전달받습니다. `Bidirectional` 클래스는 전달받은 순환 층으로 새로운 두 번째 객체를 만듭니다. 하나는 시간 순서대로 입력 시퀀스를 처리하고 다른 하나는 반대 순서로 입력 시퀀스를 처리합니다. IMDB 감성 분석 문제에 이를 적용해 보죠:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras import backend as K\n",
    "K.clear_session()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 20000 samples, validate on 5000 samples\n",
      "Epoch 1/10\n",
      "20000/20000 [==============================] - 137s 7ms/step - loss: 0.5601 - acc: 0.7167 - val_loss: 0.3782 - val_acc: 0.8538\n",
      "Epoch 2/10\n",
      "20000/20000 [==============================] - 136s 7ms/step - loss: 0.3297 - acc: 0.8728 - val_loss: 0.4579 - val_acc: 0.8104\n",
      "Epoch 3/10\n",
      "20000/20000 [==============================] - 136s 7ms/step - loss: 0.2649 - acc: 0.9032 - val_loss: 0.5085 - val_acc: 0.8412\n",
      "Epoch 4/10\n",
      "20000/20000 [==============================] - 136s 7ms/step - loss: 0.2348 - acc: 0.9154 - val_loss: 0.3106 - val_acc: 0.8798\n",
      "Epoch 5/10\n",
      "20000/20000 [==============================] - 136s 7ms/step - loss: 0.2058 - acc: 0.9269 - val_loss: 0.3287 - val_acc: 0.8838\n",
      "Epoch 6/10\n",
      "20000/20000 [==============================] - 136s 7ms/step - loss: 0.1828 - acc: 0.9372 - val_loss: 0.3816 - val_acc: 0.8766\n",
      "Epoch 7/10\n",
      "20000/20000 [==============================] - 137s 7ms/step - loss: 0.1686 - acc: 0.9405 - val_loss: 0.3666 - val_acc: 0.8774\n",
      "Epoch 8/10\n",
      "20000/20000 [==============================] - 136s 7ms/step - loss: 0.1593 - acc: 0.9461 - val_loss: 0.3750 - val_acc: 0.8418\n",
      "Epoch 9/10\n",
      "20000/20000 [==============================] - 136s 7ms/step - loss: 0.1416 - acc: 0.9536 - val_loss: 0.3840 - val_acc: 0.8548\n",
      "Epoch 10/10\n",
      "20000/20000 [==============================] - 136s 7ms/step - loss: 0.1269 - acc: 0.9592 - val_loss: 0.3876 - val_acc: 0.8784\n"
     ]
    }
   ],
   "source": [
    "model = Sequential()\n",
    "model.add(layers.Embedding(max_features, 32))\n",
    "model.add(layers.Bidirectional(layers.LSTM(32)))\n",
    "model.add(layers.Dense(1, activation='sigmoid'))\n",
    "\n",
    "model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc'])\n",
    "history = model.fit(x_train, y_train, epochs=10, batch_size=128, validation_split=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "여기서 얻은 검증 정확도는 88% 정도로 이전 절에서 사용했던 일반 LSTM보다 조금 더 성능이 높습니다. 이 모델은 조금 더 일찍 과대적합되는 것 같습니다. 양방향 순환 층이 단방향 LSTM보다 모델 파라미터가 두 배 많기 때문에 놀라운 일은 아닙니다. 규제를 조금 추가한다면 양항뱡 순환 층을 사용하는 것이 이 작업에 더 적합해 보입니다.\n",
    "\n",
    "이제 동일한 방식을 온도 예측 문제에 적용해 보죠:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/40\n",
      "500/500 [==============================] - 205s 411ms/step - loss: 0.2930 - val_loss: 0.2739\n",
      "Epoch 2/40\n",
      "500/500 [==============================] - 205s 409ms/step - loss: 0.2743 - val_loss: 0.2716\n",
      "Epoch 3/40\n",
      "500/500 [==============================] - 205s 409ms/step - loss: 0.2666 - val_loss: 0.2689\n",
      "Epoch 4/40\n",
      "500/500 [==============================] - 204s 409ms/step - loss: 0.2628 - val_loss: 0.2705\n",
      "Epoch 5/40\n",
      "500/500 [==============================] - 204s 409ms/step - loss: 0.2553 - val_loss: 0.2730\n",
      "Epoch 6/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.2488 - val_loss: 0.2715\n",
      "Epoch 7/40\n",
      "500/500 [==============================] - 205s 409ms/step - loss: 0.2448 - val_loss: 0.2799\n",
      "Epoch 8/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.2399 - val_loss: 0.2820\n",
      "Epoch 9/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.2334 - val_loss: 0.2822\n",
      "Epoch 10/40\n",
      "500/500 [==============================] - 205s 409ms/step - loss: 0.2282 - val_loss: 0.2832\n",
      "Epoch 11/40\n",
      "500/500 [==============================] - 205s 409ms/step - loss: 0.2226 - val_loss: 0.2878\n",
      "Epoch 12/40\n",
      "500/500 [==============================] - 204s 409ms/step - loss: 0.2163 - val_loss: 0.2893\n",
      "Epoch 13/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.2099 - val_loss: 0.2917\n",
      "Epoch 14/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.2065 - val_loss: 0.2999\n",
      "Epoch 15/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.2009 - val_loss: 0.3017\n",
      "Epoch 16/40\n",
      "500/500 [==============================] - 204s 409ms/step - loss: 0.1969 - val_loss: 0.3040\n",
      "Epoch 17/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.1914 - val_loss: 0.3053\n",
      "Epoch 18/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.1899 - val_loss: 0.3080\n",
      "Epoch 19/40\n",
      "500/500 [==============================] - 205s 409ms/step - loss: 0.1850 - val_loss: 0.3129\n",
      "Epoch 20/40\n",
      "500/500 [==============================] - 205s 409ms/step - loss: 0.1807 - val_loss: 0.3135\n",
      "Epoch 21/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.1766 - val_loss: 0.3119\n",
      "Epoch 22/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.1734 - val_loss: 0.3178\n",
      "Epoch 23/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.1694 - val_loss: 0.3176\n",
      "Epoch 24/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.1659 - val_loss: 0.3228\n",
      "Epoch 25/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.1640 - val_loss: 0.3193\n",
      "Epoch 26/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.1612 - val_loss: 0.3236\n",
      "Epoch 27/40\n",
      "500/500 [==============================] - 205s 409ms/step - loss: 0.1587 - val_loss: 0.3241\n",
      "Epoch 28/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.1548 - val_loss: 0.3214\n",
      "Epoch 29/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.1533 - val_loss: 0.3263\n",
      "Epoch 30/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.1523 - val_loss: 0.3303\n",
      "Epoch 31/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.1501 - val_loss: 0.3229\n",
      "Epoch 32/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.1482 - val_loss: 0.3286\n",
      "Epoch 33/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.1451 - val_loss: 0.3310\n",
      "Epoch 34/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.1445 - val_loss: 0.3332\n",
      "Epoch 35/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.1429 - val_loss: 0.3343\n",
      "Epoch 36/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.1422 - val_loss: 0.3372\n",
      "Epoch 37/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.1407 - val_loss: 0.3314\n",
      "Epoch 38/40\n",
      "500/500 [==============================] - 205s 409ms/step - loss: 0.1394 - val_loss: 0.3357\n",
      "Epoch 39/40\n",
      "500/500 [==============================] - 205s 410ms/step - loss: 0.1371 - val_loss: 0.3326\n",
      "Epoch 40/40\n",
      "500/500 [==============================] - 205s 409ms/step - loss: 0.1363 - val_loss: 0.3369\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.Bidirectional(\n",
    "    layers.GRU(32), input_shape=(None, float_data.shape[-1])))\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=40,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 네트워크는 일반 GRU 층과 비슷한 성능을 냅니다. 이유는 쉽게 이해할 수 있습니다. 모든 예측 성능은 시간 순서대로 처리하는 네트워크의 절반에서 옵니다. 시간 반대 순서로 처리하는 절반은 이런 작업에 성능이 매우 좋지 않기 때문입니다(최근의 정보가 오래 전의 정보보다 훨씬 더 중요합니다)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 더 나아가서\n",
    "\n",
    "온도 예측 문제의 성능을 향상하기 위해 시도해 볼 수 있는 것들이 많이 있습니다.\n",
    "\n",
    "* 스태킹한 각 순환 층의 유닛 수를 조정합니다. 지금 설정은 대부분 임의로 한 것이라 최적화가 덜 되었을 것입니다.\n",
    "* `RMSprop` 옵티마이저가 사용한 학습률을 조정합니다.\n",
    "* `GRU` 대신 `LSTM` 층을 사용합니다.\n",
    "* 순환 층 위에 용량이 큰 완전 연결된 회귀 층을 사용합니다. 즉 유닛 수가 많은 `Dense` 층이나 `Dense` 층을 스태킹합니다.\n",
    "* 최종적으로 (검증 MAE 기준으로 봤을 때) 최선의 모델을 테스트 세트에서 확인해야 합니다. 이를 잊으면 검증 세트에 과대적합된 네트워크 구조를 만들게 될 것입니다.\n",
    "\n",
    "늘 그렇듯이 딥러닝은 과학보다는 예술에 가깝습니다. 어떤 문제에 적합하거나 그렇지 않은 가이드라인은 제시할 수 있지만 결국 모든 문제는 다릅니다. 경험을 바탕으로 다른 전략들을 시도해 봐야 합니다. 현재는 문제를 해결하는 최선의 방법을 미리 알 수 있는 이론이 없습니다. 반복해서 시도해야 합니다.\n",
    "\n",
    "## 요약\n",
    "\n",
    "다음은 이번 절에서 배운 것들입니다.\n",
    "\n",
    "* 4장에서 처음 배웠던 것처럼 새로운 문제를 해결할 때는 선택한 지표에서 상식 수준의 기준점을 설정하는 것이 좋습니다. 기준점을 가지고 있지 않으면 실제 향상이 되었는지 알 수 없습니다.\n",
    "* 계산 비용을 추가할지 판단하기 위해서 비용이 비싼 모델 전에 간단한 모델을 시도합니다.\n",
    "* 시간 순서가 중요한 데이터가 있다면 순환 층이 적합합니다. 시계열 데이터를 펼쳐서 처리하는 모델의 성능을 쉽게 앞지를 것입니다.\n",
    "* 순환 네트워크에 드롭아웃을 사용하려면 타임스텝 동안 일정한 드롭아웃 마스크와 순환 드롭아웃 마스크를 사용해야 합니다. 둘 다 케라스 순환 층에 포함되어 있습니다. 순환 층에 있는 `dropout`과 `recurrent_dropout` 매개변수를 사용하면 됩니다.\n",
    "* 스태킹 RNN은 단일 RNN 층보다 더 강력한 표현 능력을 제공합니다. 하지만 계산 비용이 많이 들기 때문에 항상 시도할 가치가 있지는 않습니다. (기계 번역 같은) 복잡한 문제에서 확실히 도움이 되지만 작고 간단한 문제에서는 항상 그렇지 않습니다.\n",
    "* 양쪽 방향으로 시퀀스를 바라보는 양방향 RNN은 자연어 처리 문제에 유용합니다. 하지만 최근의 정보가 오래된 것보다 훨씬 의미 있는 시퀀스 데이터에는 잘 작동하지 않습니다.\n",
    "\n",
    "여기서 자세히 다루지 않은 두 가지 중요한 개념이 있습니다. 순환 어텐션과 시퀀스 마스킹입니다. 둘 다 자연어 처리에 깊게 관련되어 있고 온도 예측 문제에는 적합하지 않습니다. 이 책을 끝내고 앞으로 공부할 목록으로 남겨 두겠습니다."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}