Continuous_Control.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Continuous Control\n",
    "\n",
    "---\n",
    "\n",
    "You are welcome to use this coding environment to train your agent for the project.  Follow the instructions below to get started!\n",
    "\n",
    "### 1. Start the Environment\n",
    "\n",
    "Run the next code cell to install a few packages.  This line will take a few minutes to run!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[31mtensorflow 1.7.1 has requirement numpy>=1.13.3, but you'll have numpy 1.12.1 which is incompatible.\u001b[0m\r\n",
      "\u001b[31mipython 6.5.0 has requirement prompt-toolkit<2.0.0,>=1.0.15, but you'll have prompt-toolkit 3.0.5 which is incompatible.\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!pip -q install ./python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The environments corresponding to both versions of the environment are already saved in the Workspace and can be accessed at the file paths provided below.  \n",
    "\n",
    "Please select one of the two options below for loading the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:unityagents:\n",
      "'Academy' started successfully!\n",
      "Unity Academy name: Academy\n",
      "        Number of Brains: 1\n",
      "        Number of External Brains : 1\n",
      "        Lesson number : 0\n",
      "        Reset Parameters :\n",
      "\t\tgoal_speed -> 1.0\n",
      "\t\tgoal_size -> 5.0\n",
      "Unity brain name: ReacherBrain\n",
      "        Number of Visual Observations (per agent): 0\n",
      "        Vector Observation space type: continuous\n",
      "        Vector Observation space size (per agent): 33\n",
      "        Number of stacked Vector Observation: 1\n",
      "        Vector Action space type: continuous\n",
      "        Vector Action space size (per agent): 4\n",
      "        Vector Action descriptions: , , , \n"
     ]
    }
   ],
   "source": [
    "from unityagents import UnityEnvironment\n",
    "import numpy as np\n",
    "import torch\n",
    "import numpy as np\n",
    "from collections import deque\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "# select this option to load version 1 (with a single agent) of the environment\n",
    "env = UnityEnvironment(file_name='/data/Reacher_One_Linux_NoVis/Reacher_One_Linux_NoVis.x86_64')\n",
    "\n",
    "# select this option to load version 2 (with 20 agents) of the environment\n",
    "# env = UnityEnvironment(file_name='/data/Reacher_Linux_NoVis/Reacher.x86_64')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Environments contain **_brains_** which are responsible for deciding the actions of their associated agents. Here we check for the first brain available, and set it as the default brain we will be controlling from Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the default brain\n",
    "brain_name = env.brain_names[0]\n",
    "brain = env.brains[brain_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Examine the State and Action Spaces\n",
    "\n",
    "Run the code cell below to print some information about the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of agents: 1\n",
      "Size of each action: 4\n",
      "There are 1 agents. Each observes a state with length: 33\n",
      "The state for the first agent looks like: [  0.00000000e+00  -4.00000000e+00   0.00000000e+00   1.00000000e+00\n",
      "  -0.00000000e+00  -0.00000000e+00  -4.37113883e-08   0.00000000e+00\n",
      "   0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00\n",
      "   0.00000000e+00   0.00000000e+00  -1.00000000e+01   0.00000000e+00\n",
      "   1.00000000e+00  -0.00000000e+00  -0.00000000e+00  -4.37113883e-08\n",
      "   0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00\n",
      "   0.00000000e+00   0.00000000e+00   5.75471878e+00  -1.00000000e+00\n",
      "   5.55726671e+00   0.00000000e+00   1.00000000e+00   0.00000000e+00\n",
      "  -1.68164849e-01]\n"
     ]
    }
   ],
   "source": [
    "# reset the environment\n",
    "env_info = env.reset(train_mode=True)[brain_name]\n",
    "\n",
    "# number of agents\n",
    "num_agents = len(env_info.agents)\n",
    "print('Number of agents:', num_agents)\n",
    "\n",
    "# size of each action\n",
    "action_size = brain.vector_action_space_size\n",
    "print('Size of each action:', action_size)\n",
    "\n",
    "# examine the state space \n",
    "states = env_info.vector_observations\n",
    "state_size = states.shape[1]\n",
    "print('There are {} agents. Each observes a state with length: {}'.format(states.shape[0], state_size))\n",
    "print('The state for the first agent looks like:', states[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Take Random Actions in the Environment\n",
    "\n",
    "In the next code cell, you will learn how to use the Python API to control the agent and receive feedback from the environment.\n",
    "\n",
    "Note that **in this coding environment, you will not be able to watch the agents while they are training**, and you should set `train_mode=True` to restart the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total score (averaged over agents) this episode: 0.18999999575316906\n"
     ]
    }
   ],
   "source": [
    "env_info = env.reset(train_mode=True)[brain_name]      # reset the environment    \n",
    "states = env_info.vector_observations                  # get the current state (for each agent)\n",
    "scores = np.zeros(num_agents)                          # initialize the score (for each agent)\n",
    "while True:\n",
    "    actions = np.random.randn(num_agents, action_size) # select an action (for each agent)\n",
    "    actions = np.clip(actions, -1, 1)                  # all actions between -1 and 1\n",
    "    env_info = env.step(actions)[brain_name]           # send all actions to tne environment\n",
    "    next_states = env_info.vector_observations         # get next state (for each agent)\n",
    "    rewards = env_info.rewards                         # get reward (for each agent)\n",
    "    dones = env_info.local_done                        # see if episode finished\n",
    "    scores += env_info.rewards                         # update the score (for each agent)\n",
    "    states = next_states                               # roll over states to next time step\n",
    "    if np.any(dones):                                  # exit loop if episode finished\n",
    "        break\n",
    "print('Total score (averaged over agents) this episode: {}'.format(np.mean(scores)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When finished, you can close the environment."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. It's Your Turn!\n",
    "\n",
    "Now it's your turn to train your own agent to solve the environment!  A few **important notes**:\n",
    "- When training the environment, set `train_mode=True`, so that the line for resetting the environment looks like the following:\n",
    "```python\n",
    "env_info = env.reset(train_mode=True)[brain_name]\n",
    "```\n",
    "- To structure your work, you're welcome to work directly in this Jupyter notebook, or you might like to start over with a new file!  You can see the list of files in the workspace by clicking on **_Jupyter_** in the top left corner of the notebook.\n",
    "- In this coding environment, you will not be able to watch the agents while they are training.  However, **_after training the agents_**, you can download the saved model weights to watch the agents on your own machine! "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 100\tAverage Score: 5.44\n",
      "Episode 187\tAverage Score: 30.21\n",
      "Environment solved in 187 episodes!\tAverage Score: 30.21\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzsnXd4XNWZ8H9n+mjUJVvuLTYGY2PjAhhTRCchJAGSJWUTSCObnl2WkF2+JCSBJclH2rIEQhYCX5alJYEQA6FaNjYGbGMw2Ma9yrJl9RlNnznfH+feOzPSSBrJmlE7v+fR45lb39HI571vF1JKNBqNRjN2sQ21ABqNRqMZWrQi0Gg0mjGOVgQajUYzxtGKQKPRaMY4WhFoNBrNGEcrAo1GoxnjaEWg0Wg0YxytCDQajWaMoxWBRqPRjHEcQy1ALlRXV8sZM2YM6NzOzk58Pt/gCjTIaBkHBy3j4KBlHByGg4ybNm1qklKO6/NAKeWw/1myZIkcKKtWrRrwuYVCyzg4aBkHBy3j4DAcZAQ2yhzWWO0a0mg0mjGOVgQajUYzxsm7IhBC2IUQm4UQK433M4UQbwghdgkhHhNCuPItg0aj0Wh6phDB4m8D24FS4/3PgF9JKR8VQtwLfBG4p78XjcViHD58mHA43OtxZWVlbN++vb+XLyiFkNHj8TBlyhScTmde76PRaEYeeVUEQogpwBXA7cC/CCEEcCHwaeOQh4BbGYAiOHz4MCUlJcyYMQN12ez4/X5KSkr6e/mCkm8ZpZQ0Nzdz+PBhZs6cmbf7aDSakUm+XUO/Br4LJI33VUCblDJuvD8MTB7IhcPhMFVVVb0qAY1CCEFVVVWf1pNGoxmb5M0iEEJ8GGiUUm4SQtSam7McmnVEmhDiBuAGgJqaGurq6jL2l5WVEQgE+pQjkUjg9/tzF3wIKJSM4XC42+8xVwKBwIDPLRRaxsFByzg4jAQZLXLJMR3ID3AH6ol/P3AUCAIPA02AwzhmOfB8X9fKVkewbdu2nPJoOzo6cjpuKCmUjLn+zrIxHHKi+0LLODgUUsY39zXLZDLZbfvW+na5fk9Tj+eZMm470i7rW4P5Eq9fdP0cw+G7ZqjrCKSU/yalnCKlnAF8EnhFSvkZYBXwceOw64C/5kuGQnLrrbdy5513Zmzbv38/8+fPHyKJNJrhzXv17Xzi3vW8tqe5274fr9zKvz7xDqAeVv3hWNZrfPORzdz+7NAngzy24SBn//QV2oPZ5eyLQy1BLv7lajYdaBlkyXJjKOoIbkYFjnejYgb3D4EMGo1miDkeiADQ0J4Zu0okJVsOt1PfFiIcS/DMuw2ccfvLNBvHp+MPx9h1TLlVG/1hthxuO2G5kknJYxsOcsbtL/HH9ftzOmfNriYa2sPcv3bvgO5305/eYXdjgA37W/t9/mBQEEUgpayTUn7YeL1XSnmGlHK2lPITUsru3+4I4fbbb2fu3LlcfPHF7NixA4BNmzaxcOFCli9fzt13320d++CDD/LRj36Uyy+/nLlz5/KjH/3I2vezn/2Mk08+mUsuuYRPfepT3SwLjWY0EgirnJGWzswlYOcxP8FoAilhX1MnG/a1EIol2FLf3u0akXiS/c1BEknJnc/v4EsPbcx6r7cOtvZoVZhIKXnotf1c/KvV3Pznd2n0R9jWkFvsbqsh2wPr9tPaGc3pHJOH3zjA63uVJXCkLdSvcweLEdF0ri9+9LetbDvSkXVfIpHAbrf3+5rzJpXywytP7XH/pk2bePTRR9m8eTPxeJzFixezZMkSPv/5z3PXXXdx/vnnc9NNN2Wc8+abb/Lee+9RVFTEsmXLuOKKKwB4+umnu11HoxntBCKmIshcoDcfTD3V72vqZIfxxL+1vp0L5o7PODYSSxKNJ6lvDfFufQftoe6LfSia4BP3rufUSaX8z5fOpNSTvZbmzX0t/PDprSycWs5vPrmIX76405KxN/zhGPubg3xk4ST+tuUID6zbx42Xzu3zPIDOSJxfvLiTFbOraA5EqW8dGkWgW0wMkFdffZWrrrqKoqIiSktL+chHPkJnZydtbW2cf/75AHz2s5/NOOeSSy6hqqoKr9fL1Vdfzdq1a1m7di0f+tCH8Hq9lJSUcOWVVw7Fx9FoCo75hN71CfrtQ62UeNQz6t7jAXYcNRRBl4c9KSWReAKA9492sLvRTySeJJnMTERsDUYtd9OXHtzYbb/J9gZ1/d9/bgkfXTSZYreDzhwUwXbDarjq9MksnlbBG3tz9/P/7xsHaQvGuPHSuUyp8FKvLYKB09uTez6LtbrWMPh8vl7rGrruE0KQTCZ7OFqjGd2YrqHmLopg88E2lk6v4P2jft7Y10JrMIYQ3RVBPCkx1/QXth0jllBvQrEEPndqaTOthMXTynlzfwv1bSGmVhZ1k2fHsQDlRU7GFbsBKHY7crIIth5RbqFTJ5VyUk0Jz77bYGZO9ko4luC+V/dy9geqWDytgknlXt7YN3aCxaOC8847jyeffJJQKITf7+dvf/sboOob1q5dC8DDDz+ccc6LL75IS0sLoVCIp556ihUrVnDOOefw97//nXA4TCAQ4Jlnnin4Z9FohgK/sci2BlOKwB+Osft4gEVTK5g1zsd6I6Po7A9UcbAlSEeanz8STz1EPb/1qPU6FEtk3KfDUAQLJpcBZHUfAew65uekmhLrga3Y7bCUVW+8V99BdbGb8aUe5tYU0x6Kcdzfd+jz+a1HOe6P8PULZgMwudyLPxy3PmMskWTTgRZiifw/LGpFMEAWL17Mtddey6JFi7jmmms499xzAfjDH/7A17/+dZYvX47X680455xzzuGzn/2sdc7SpUtZtmwZH/zgB1m4cCFXX301S5cupaysbCg+kkZTUMxFNt019O7hdqSERdPKmVVdTNx45L/69CkAGbHASNqC709bsEPRTEVgLvymFZBNEUgp2XHMz0k1xda2Yo+DzmhuFsGpk1QrtZNqlPdh57G+i123N/hx2W2cNasKgEnlar1oaFNZVO8cauOae9bz8vZjfV7rRBkVrqGh4pZbbuGWW27ptv2dd96xXt96663W6/Hjx/Nf//Vf3Y7/1re+xR133EEwGOS8887jxhtvzIu8Gs1wwnS7pLuGjnaoRXBqhZdZ49R0r+piF+edpIZsbT3SYS2cpkVgtwkSaX7/bhaBoSSmpSkCKSX3r93Hx5dMobzIxdGOMP5wnLk1KTeyLweLIBJPsLsxwIUnqyD2SRPU+TuO+flAH59/X1OAaVVF2G3KAplcoRRBfVuQuRNKWLe7GSGwPm8+0RbBMOBb3/oWixYtYvHixVxzzTUsXrx4qEXSaPKO+RTfHooRN9wf5tN6mdfJrHHq6fykmhLGlbgZX+K20jRB+dgBZlUrhTGh1APkZhHsOd7Jbc9s58Vt6mnbfIKfk6YISnKIETy24RDxpORMY7GuLnZT6XNZtQ29sb8pyIyq1CjLyeWmIlDKcN2eJuZPKqO8KP+d+rVFUCCuv/56rr/++qz7HnjggWHfIVWjGWz8aYtsWyhGdbHbWrRLvU5rgZ9rPGV/YFwxB1uC1jmmRXDKxFJ2NQZYMr2CZ95tINiDIphiPHG3h2K0GFZI1FBAO43MpJO6WASReJJYIonT3v2ZuaUzyi9eUKmf582ptrbPGV/MjmN+Lu/lQT6ZlOxv7uS8k1LnjSt247QL6ltDBKNxNh9s5QsrCtMtWFsEGo1mUMgl1TKdQDiG4RWxFub2UAyfy47TbmNyuZfPnDmNq05XDYorfM6MwLKpCM6ZU80ZMyv54IIJQMpSMOkIxSjxOCh2O3DYhKEIVDA3alxj5zG/9TRvUmxkHvX0uf7v8zsIROL88MpTMzIC504oYfexQK+ZQw0dYSLxJDOqUxaBzSaYWOblSFuIDftbiSUkZ8+u7vEag4lWBBqN5oR5dddxFv7oBbYcbiORlNy2chs/fe59Xt/bvY+QSSAStwKk6YqgzKsKvmw2we1XLeC0KeUAlBe5aEvr5WMGiyeXe3n8K8uZM149zXe1CDqMawohKPM6DUWQyswBpQjmTijOOM9UBP4scYL36tt5dMNBrls+I8OKAOVe8kfitIS7K4Lj/giHWoLsb+oEYGaaIgCYVO7hSFuI13Y34bQLls2oyPKbG3y0ItBoNCfM028fIZ6U/G71Xp5+p57/XruP+9bs4ZP3vW4VarV2RjOekv3hONOriqx9oBbtUm/2yt+KIidtRqAXUhaB26GWMa9TdRDoGixuD8WsamJTEZiWhWkRNAWi1JR4Ms4rNoraumYOSSn54dNbqfK5+M4lc7rJaQac6wPd0z5vf2Yb//C79exuVDGJropgcnkRO475+cvmek6fVkGRqzDee60INBrNCRFPJHlp+zFcdhvPvdfAz57bwSkTS3nj3y/GZbfx+MZDbD3Szhn/8RLPb1XB2URSEowmmFapFsKWYHeLoCsVRS4SSWllAaUUgVIAXld2RdARTl2z1OukIxSjOWDGCJRSiSWSuByZy6GvB9fQC9uOselAK9+9/OSs7SrMFNT6QHeLoCUYo6E9zCNvHsTjtHVTPrPG+fCH41QWubj58pOz/h7ygQ4WazSaE2LTgVZagzF+eOU8/uPZ7RztCHPbx+YzrsTNxfPG89TmevYc7ySWkGxv6ODy+ROsbJyplYZrKJBSBOmZNOmY2TNthtIw20u4nYZFYCqCLk/w7aEYs6rV4lzmVXEG0yIwXUPRLIqgJ9fQ+j3NeJ12rj49+3DF8iIX40vc1AcS3faZsr1/1M/JE0qw2TK7DXx+xQzOmV3NaVPKCjp9USuCYcZAm+RpNEPFi9uUNfCJpVOpbw2xt6mTi05RefWfWDqVZ989ypqdxwHVdx9SNQRVPhfFbkeGRVBe1LNrCKDViBNEYj24hqKZLpn2UIxSr1rqyrxO9jd3prKGDKsiGk/ismdXBJ2RzAX93fp25k8uxZElk8jkpJoS6hu7t4tIj190dQsBFLkcLJxa3uN184V2DZ0gH/vYx1iyZAmnnnoq9913H/fccw/f/e53rf0PPvgg3/zmNwH4n//5H8444wwWLVrEV77yFRIJ9UcxceJEfvCDH3DmmWeyfv16fvzjH7Ns2TLmz5/PDTfcYPlEN2zYwGmnncby5cu56aabrKE3iUSCm266iWXLlnHaaafxu9/9rsC/Bc1Y5qXtx1gxu4pit4P/8+F5PHD9Mutp9rw546gpdVNd7OK0KWVW+qdZqFXsdlLpc1kxgt5cQ6ZF0GpZBGoR9xgKwG4TuBw2grHMJ/iOUNy6ZipY3MUiiCdxdrUIjBhBIJIKUMcTSbYeaWfB5N4X6zk1xdR3dm+AF4olrHqBGVkUwVAxOiyC73wH3n476y5vIgEDecJetAh+/es+D3vggQeorKwkFAqxbNkyXn75ZVasWMHPf/5zAB577DFuueUWtm/fzmOPPca6detwOp187Wtf4+GHH+Zzn/scnZ2dzJ8/nx//+McAzJs3jx/84AeA6mC6cuVKrrzySj7/+c9z3333cfbZZ/O9733PkuH++++nrKyMDRs2EIlEWLFiBZdeeikzZxYmB1kztjnSHuayUydk3We3Ce75xyXYhODh1w+wZpeyDMzOo8UeBxU+F82dUSLxBOFYspcYgdreFoxSQZprKG0B9zrthNOeuqPxJKFYIkMRdIRiNDtT6aPJpCSelN0tApepCFLX2308QDiWZMGU0l5/J3NrSogm4HBriGlVqQZ3oWiCFbOrmTXOx+U9/M6GgtGhCIaQ//zP/+TJJ58E4NChQ+zbt49Zs2bx+uuvM2fOHHbs2MGKFSu4++672bRpE8uWLQMgFAoxfrwyn+12O9dcc411zVWrVvHzn/+cYDBIS0sLp556Kueeey5+v5+zzz4bgE9/+tOsXLkSgBdeeIEtW7bwpz/9CYD29nZ27dqlFYEm70gpicaTuJ09P2wtnqZSINfsPM6xjgjhWMIqJivxOKgscnI8EMmoKs5GhWkRdMYMRZAZLAYoctkzgsXpBWrmtZMy1coimkhaRWXdg8XquoFwnMc2HGTP8U7mjFexhr4tArPnkD9TEcQSFLnsfK12dq/nF5q8KQIhhAdYA7iN+/xJSvlDIcSDwPmAWSt+vZQy++N8rvTy5B7KYxvquro6XnrpJdavX09RURG1tbWEw2GuvfZaHn/8cU4++WSuuuoqhBBIKbnuuuu44447ul3H4/FYcYFwOMzXvvY1Nm7cyNSpU7n11lsJh8O9FqdIKbnrrru47LLL8vI5NZqeMBdRt6NvL7PZ6+dwa9ByDZW4HVT63Ow46re6hPaUPlrqdSKEESx2pWIEri4WQbof3uzkmW4RAFb76lhC9vgZHHYbHqeNzmicP7/VxJv7Wjh1Uik+l92qeu4JM3NoZ6Ofi+fVWNuD0YQV1B5O5DNGEAEulFIuBBYBlwshzjL23SSlXGT8nJgSGELa29upqKigqKiI999/n9dffx2Aq6++mqeeeopHHnmEa6+9FoCLLrqIP/3pTzQ2NgLQ0tLCgQMHul0zHFZPKtXV1QQCAespv6KigpKSEusejz76qHXOZZddxj333EMspv7od+7cSWdnZ54+tWasE08keWDtPsKxRLdc/t4we/0caglZweJij4MJZW6O+SNWSmdPFoHdJij1qFoCUK4hp11YTdtAZQ6Fe7MIugSio/GEFTDuahFAaibBYSO2sfVIB/Mnl3XL9ulKicdJpUdYrStApcxG40krqD2cyJtFINUjrNmL1Wn89D2tYQRx+eWXc++993Laaacxd+5czjpL6bmKigrmzZvHtm3bOOOMMwDl97/tttu49NJLSSaTOJ1O7r77bqZPn55xzfLycr785S+zYMECZsyYYbmSQMUCvvzlL+Pz+aitrbXaVX/pS19i//79LF68GCkl48aN46mnnirQb0Ez1tiwv5Ufr9zGzHE+5k9Sf4O5KQIVJD3YErT8+8VuBzOqfCSS0ho805MiABUnaA3GoEy5htLdQpDFIjAVgSfTIjCJJaSlCLL1Eyp2O2gLRjnaEWZuTQk7jvmtuQZ9MaXYltGO2nRZFQ1Di0DkMklnwBcXwg5sAmYDd0spbzZcQ8tRFsPLwPeyDbAXQtwA3ABQU1OzJP0JGNQAmNmz+/azjYR0zFxlDAQCFBcrk/OXv/wlR48etYLSubB7927a27sPAM+F9HsPV7SMg0NfMr52JM59WyJ8fZGbWWU2blwd4gvzXZw3pecFHJQL8ysvBrlgqgO3Q/C3PTHuv6yIPW1Jbn8jzFkT7bzekOCn53qZ4MuuWH68PoTXAV89JcFfDjjZcCzOXRem3DT/d0OIcBy+v1wpndePxLl3S4T/OMfLpGIbh/xJvr9OjYMUwLwqG5+b5+bmV0N8eYGLFZMzP8MPXwsRS0iOdEq+MN9FMAaLa+yML+pb8T20JcC6Y4LfXVyEEIL2iOTbq4J8dp6Li6b1/rsaLC644IJNUsqlfR2X12CxlDIBLBJClANPCiHmA/8GHAVcwH3AzcCPs5x7n7GfpUuXytra2oz927dvz8n3n89RlYNFrjI+++yz3HHHHcTjcaZPn86DDz7Yr8/m8Xg4/fTTByRjXV0dXb+D4YaWcXDoS8ada/bAlveZM/cUTptSBqtXs3D+PGoXZS+wSmfG26tJ+nxUVXgpPnyYCy+4gAX+CLe/8RJHo24gyCXnr6DKGBfZlYf2vcnxQITi4gRV4yso6WjKkPV/D27kYEuQ888/lwPNQQ65j8OWrVxy/grGlbg50hbi++teAWBciRtfqY/FS+fDq2s4bf6p1C6clHG/CTvWs/FAKwCXLF/M8g/kPhvg7/teJHokyuKzzqHM6+RgcxBWrWLhqadQu2RKztcpBAXJGpJStgkh6oDLpZR3GpsjQog/AP9aCBlGA9dee60Vc9Bo+sNre5qoLnZ3a5A2EBo7lAGfHiPomnrZE9MqizjUEqTc67Ty9KuLVVHZ/mblh+8pWAwqc0i5W2yEs2QreY2soVd3NfG5B960mralF5SZTCjzEI0nU5+hhxiBOfTGbGOdK5UeFUc42h6mzOu0XEPDMUaQt2CxEGKcYQkghPACFwPvCyEmGtsE8DHgvYHeI59urdGG/l2Nbf7tL+/yg78O+L9aBo3GPN5I2iJqtnnoiykVRRxsCdIWilmVu0IIZlSrQLLZgronVAdSo6AslugWmyhy2QlFExwwgrsb9rficdqsWEKRy47DJrAJVdUc6yV9FFJFZXabYGKZp9v+3qgwFEFDu3JFBY32EsMxRpDPrKGJwCohxBZgA/CilHIl8LAQ4l3gXaAauG0gF/d4PDQ3N+sFLgeklDQ3N+Px9O8PWTN68IfjbNzfahVynQiNfpXZFo6lMm66Bm17YsXsaoLRBHU7Gq1FFrD6C/UWKAYVLO6MJognpREszlzCPE6lCI4bdQJdr2m2oq4ocuFx2pUi6MWqMRvPTSzz9NpSIhvpFgGkgsWeYWgR5DNraAvQzSEtpbxwMK4/ZcoUDh8+zPHjx3s9LhwOD/sFsBAyejwepkwZXn5JTeEIRuPEk5LX9jT3WAWcK5kWQffq3t64+JTxnDmzkjf2tVCS1rnT7LvTm1sIoNwYHBOISiLxRDcFZBaUNfojVBe7mFjW3Z1T5nVitwmcdhtRYwIZZLcISgxFMLWiqNu+vihzC4SABlMRRIdv1tCIrSx2Op05Vc7W1dUNOEBaKEaCjJqRSyIpCRvFV3U7jveoCJJJyTcf3cxpngS1vVzveHqMIEtRV28IIfjBlfP48F1rrUUW+mcRAARiShEVuzOXMK/TTjwpqW8LMb7Ew0NfOKNbW+pKnwun3YbLYctIH+3NIuhvfADAYROMK3ZbFoGZ1jocC8pGrCLQaDS5EUxry7x6RyNSyqwtjv2ROM9saSA63cFXerhWKJpqD5ERI8jRNQRw6qQy7vz4QmsoDaQasPWtCAyLICaJxJJU+bq7hkDVKsys9mWMnjT5ycfmY7cJ/rBuP5F4ss+CMkgVw/WXCWUeq53FmAwWazSa4YHpkpg/uZQj7WFrOlZPxx3t7DnuZsYHQFX2RhP9cw2ZXLNkCktnVFrvZxhKoS9FYLao7tk1pBbu+tYQ43pIQT1lYikn1ZTgsou+g8UnYBEATCj1pGIEw9g1pBWBRjPK6TQWoKXT1cJrtoLuivnEerSz+4hFk2MdqdrPcCyZmgmQY9ZQT1T6XMyoKmJOTe/FdhkWQTzZ7b5el3ofT0rGl2ZXBCbKNZTsNQW2xHNiFsHEMo+VNWRZBFoRaDSawUZKyS1PvsuWw21Z93emDYEBrHjBf768izf3pYanmE+sTSGZ0a8nnUyLoPdFtD8IIXjpX87ny+fO6vU409Xjj8oeWkykvN09WQQmuQSLa+eO5/sfnmd1UO0vE8q8dITjdEbiVozA0w83WqHQikCjGeF0hOM8/MZBVu/InkFnLkCVxaYiUO//a9Vu/vp2vXVcyBjoIoEDzdmtBrOYbEKpxygoM8dFnvji5rDb+hzP6HHa8bnsShFkqSNIf9oeX9p7Jp7LYVNpqLGelZnXZeeL58zMaGzXH8zag6MdYcKxBB6nrc+GdUOBVgQazQjHXNi7ZseYmMHiSsOtEjGGsUTjSas7J2SOeNx7PHscodEfwWkX1JR5MgKt/Y0RnAiVxa6UReDsXlBmMr6kb4sAUhZTrplP/WGCqQjawwSjcSuGMdzQikCjGeGYT/ymy6en/RW+lEVgunQyFEGaItnblL2NeaM/zLhiNx6HzbqOTahUyUJRWeTCH+25+6jJuD4UgWkBBPKoCEyLoKE9TCg6PFtQg1YEGs2Ix3zi78ki6BYjiCcsK6IjnEotTU8z3Xs8uyI47o8wrtSDx2m3YgRuh71Pl85gUulz0RrJbol4+qMIHClFIPKkzGpKTYsgRCgWH5aBYtCKQKMZVkgpeXzjoX61gjAX9Yjx709WbuNnf3/f2m8qCHP4eySWJGz49jvSLALzOuOLBHubAtxTt4cnNh7KuFdjR4TxJW7cDhuRWIJILJGXJ+neqPS5aQmrFNdsvYZApX325YZJdw05c4hPDASP006lz2VYBIlhaxEMT4eVRjNG2dvUyXf/tAWbEHw8x1bFpuvHXPDf3NeSsUB2RlJDYFwOm2ERZHENGdeZXmpjw8E2Nh9sY+GUMj6xdKp1THNnlMXTywlElFsomuje7yffVBW7CBnGS7fuo8b7vuIDkG4RJHCfYNZTb5i1BMN1TCVoi0CjGVaYWTk9uXmykYoRJIz38Yzzg1Hl+vA4bXgcNmURGPvbQzGrcWPQ2DajVC0LPpfd6isEylppC0YpL3IZ11EtJk60hqC/pFcL95Q1VJ2DInDalQXQGYnn1apRtQQqa2i4WgRaEWg0w4imgNHQrR+KoGvWUDiWtJ7uQSkKn8uBEAK3004kLUaQSEqr4Cxs/HvBVCf3/uNi/nH5dI77IySNfvyBiGpcV1HkxO1U8wCyBWzzTW+KwO2wIUSOFkFasDifiqDGaDMRjCaGZVUxaEWg0Qwrjqd19syVlGsoabyPZ8ztDUZTQUqP00Y4lszIMDLdQ0HDh13kFFw+fyI1JR7iSUmr0f+/LaiOUxaBXVkE8cQJF5P1l6oMRZC5sAqhGr2ZTex6w1z8824RlHpo6YzSFooNW4tAxwg0mmGEaRFEB6AIIpZrKEEyrV1QZySBz1QEDjvhWMIKFoMKGE8u9xKKZT6xmhkvjf4IVcVuWjqVQqgocuF22lJZQ0PpGspy77987eyszea64kyzCPpqf30imLUEx/0RHSPQaDR9Y7mG+qEI0l1DCWNgS1fXkJlBYy3gaa4n0yIIxRIZ6Zdmrx4zTmBaBhVFTtwO1e45GO1e3Ztvqnwpt0+2e0+pKMqpcCvDIsijVZM+E2G4WgT5HFXpEUK8KYR4RwixVQjxI2P7TCHEG0KIXUKIx4QQfatujWaMkHIN9SdYrFJowrGU7z+aSBJPpFxFRV0tgiyuoVAXH7bpZ2802ihnuIaMJ/GOUKzwMYLinl1D/cFKH43mNwV2QtqIy7EYI4gAF0opFwKLgMuFEGcBPwN+JaWcA7QCX8yjDBrNiKIpYMzjHUiMIJrIiA2YwePOaIIio52a/RFNAAAgAElEQVSyx2nPUBiQaRFk9OopSbmGoLtFYJ5b6DoCn8uOw0j5PxFrJN0KKJQi8PRHETQ1wQ03QGtrHqTKJG+fXirMhiVO40cCFwJ/MrY/hBpgr9FoSM8a6r9rKNzFJRSyFETcihG4Hco1lK4IOtIsgnTXkNdlp8TtsCyC1mAMIdTMAMsiCMcK7hoSQlDiUprgROb/pi/++fwMxW6H1c66KFd5X34ZFi6Ehx6C11/Pm2wmeQ0WCyHswCZgNnA3sAdok1KateyHgck9nHsDcANATU0NdXV1A5IhEAgM+NxCoWUcHEa6jFJKa9E9dKSBurrcngT3HVLnRONJ6tatt7bXrX2N8UU2mtuDjLOHqauro6MtTIs/ybYdu6zj3nl/N3WJgzQ2hyhxCQKBuCWjz5Fg697D1NU1sXVXhCIHrH11DXuOmO6oJK1Nxwv+e/c5krRGBJs3vsnhooEt4ulzFzraWgf9M6R/16WOBH7gwL7d1MUO9HiO++hR5tx1F9WvvUZw6lS23X03Aa8X8vz7zasikFImgEVCiHLgSeCUbIf1cO59wH0AS5culbW1tQOSoa6ujoGeWyi0jIPDSJexLRgl8fyLAJRVVlNbuzSna/7vwY3QcAyA2fMWwmtvALDg9KWcMrGU5JoXmDVtErW183m68W2O7G1h0rQpsHMXJR4HlePVvtvfWs3k8cUUF/stGWfsXE8sIamtPZs/N2xmXGcbtbW1hN9rgC1vATB96iRqaxecwG+l/5RteI7DnUnOP/dsy4XVXw63BuHVVQBMHD+O2tolgylixnf9gb1vUr/zOKcvOJXahZOyn7BlC3z60+D3w+23U/Sd77C0aGADcfpLQdJHpZRtQog64CygXAjhMKyCKcCRQsig0Qx3TLcQ9C9GkF5F3GykeKZvVzEC0zVkN1xDSVx2GxVFrswYQRfXxfgSD5sPKcvErCo2r2NS6DoCwHINnUiwuFAxAlC1BNBL1lBdHXz0o1BaCm+8AfPm5VWeruQza2icYQkghPACFwPbgVXAx43DrgP+mi8ZNJqRhBmUtYn+VRanB4jNgC4on388oWYG+FxmsFi1hgjHEridNkq9joysoa557uNL3DR2RJBSFZZVGDOD033qha4jgHRFMPB7OwuoCMyAcdasoT//GS67DCZPhtdeK7gSgPxmDU0EVgkhtgAbgBellCuBm4F/EULsBqqA+/Mog0YzYjAzhiaUevpnEaQpguZAShEEowmrf5CVPmq1j1aB4TKvs1eLoMaQpSMcp7UzZs0MTm/2Vuj0UYCZZXZmVftOLGvIUXhF0C24fc898IlPwJIlsHYtTJ2a5ez8kzfXkJRyC3B6lu17gTPydV+NZqRi1hBMrvBaHUNzIWSMQAzHkpkWQSxBMGIqAqOgzGEjmkgSjKpzyrxOjraHkVJ2Sx+FVFHZcX+Y1gzXUGEybnri7EkO/v3TtSd0jQyLIM/urWUzKpgzvpiZ1Ubri3gc/u3f4M474cMfhscegwLFA7KhK4s1mmFCU0CNgRxf4skoKOuMxLnpiXdoTfP/pxOMxq0xlC3pMYJo3Co287lTFgGo/H+Pw7QI4kTiSaSkmyIwh7scag0RjCYs15AnwyIYmcuI2X0U8m8RzB5fwov/cr5qfeH3w4c+pJTAV78KTz45pEoAtCLQaIYNTf4IVT635b4xeedQG09sOsyG/S1ZzwtFE1a1bdcYgRk/MF0+HmPBawvG8DjtlHqddIRjlnspm2sIYGt9OwDlvuFhEQwGQgjLEihYwLutDS69FF55BX7/e/jtb8Ex9C3fRuY3qNGMQo4HIlSXpBq6mbQZPnxztm5XQrGE5bvPiBHEEtaYSp/b7DWkFvqOUMxyDUXjKZdS12DmjCof1cVunnpbJfdZwWJnuiIYnm0TcsG0CgpSHX3oEJx/PmzaBE88AV/6Uv7vmSNaEWg0w4TWoArGuuy2jKwhs8dPZxZFEEskiSWk1W2zNRjF57IjhGERdAsW26zjzGAxwNH2sLE/c1G32wSXnlrD7kbVJMB0QWW4hoYga2iwMBVA3hXBK6/AmWfC/v3wzDNw1VX5vV8/GbnfoEYzygiEY5R6nN0sAvNp3Z9FEZi1AqZF0NoZw+ty4HXalSLoEiz2pPUIcjtSiqDBUATZ8twvP3WC9TpbsHgo6ggGCzNg7MzXZwiHVRzgoovA54N16+CSS/JzrxNg5H6DGs0oozOSwOe2W0Vf5ghJM70zEM6iCAzfvjmsJZpI4nXZKHLZlWvICBabFoH59J6UyjqYYMQA9hwPGMd191efNauKUqNXToVPKQ6XXU0CS7/mSCSfFoGrqQlqa+Hee+HGG1Xl8Pz5g36fwWDkfoMazSgjEIlT7HZaT9tRo410m2ERmK6hz97/Bk9sPASkiskq0gaxFDkdeF12wtGEpSjMGIEnzZ/vcdqZYaQzbm/oAMDr6r4kuBw2LjqlRt3HsAiEEJacIzlGYFozgz68fvNmlnz1q/Dee6pg7M47wevt+7whQisCjWYYkExKOqNxit12a4E13UNmjMAfiZNMStbubmLzoTYgZRGkT+Tyuux4nXaC0WwWQboisFHlc1HidrC9wW9sy76o//PFJ/HTqxdk7Ddfj9SsIUi5hAbVInj6aTjnHKQQyhV09dWDd+08MXK/QY1mFBGMJZASij0Oa7E2W1GbiiAQjuMPx5EyZR2EYqmsIPPptshlx+tyEIwlaAuqNtGpp/fUf3mPw44QgunVRRw1up72NNlrWlURnzxjWsY2d6ECrXlk0F1Dv/kNfOxjMG8eb91zj2olPQIYud+gRjOKMBf2rK6hkOEaisateIGlCKLqGK/TbmUEqQH0yjV0rCNMTakHIbr37zdfpw96788oRdMlNJJdQ2b66KAEi3/7W/jOd5QiWL2aaFXViV+zQGhFoNEMA/zhVAWw5RoyMoLSLYKOcGZNQTDN9WMu7F6XHa/LTjAWNxRBasavJy2wa74eqCIwzx/JrqFBswj++Ef4+tfhyiuHvF3EQBj6kjaNRmMt7CUeB1EjNmBmDqXHCFIWQWpgPaQWf1BKQYKVPnrKxFLrPm5HFougOk0R9GOUomURjOCsIedgVBY//DBcfz1ceCE8/jg4nYMjXAEZud+gRjOKsCqAXQ5rgY3Ek4RiCctF1BnJ5hpKtYYwM4KK0uoIGv0Rq3EcZFoEbss1pJ5e7TaR0X+nL8xrjeQ6AteJBosfeQQ+9zlVMfy3v4FnYENyhhptEWg0wwDTNVTscZBIqvqBiBHsBTX3NhBOKYKUayhVOexJazWdlJLmziiReNLqF2Tus14bi59pERQ57VYsIRdSFsHIjRG4sgTRc+axx+Af/xHOPVcpgRHmDkpn5KpyjWYUYT7hl7idlqslktYDaEqFl85ownqfyhpKcw0507OGUo3r0mMEDpvAZqz1phuoyuei2O2wFEmuZMtEGmkMuLL4iSfgM5+BFStg5UpVNTyCGbnfoEYzighE0oPFKddQu2ERTKlQxUgNbSrNszOaIJmUhKIJ7DbVRdN82i8y6ghM0mf6CiGs40xXkhCCGdVF/QoUg7IubEIpl5HKgILFjz4Kn/oULF8Ozz4LxcV5kq5w5HNU5VQhxCohxHYhxFYhxLeN7bcKIeqFEG8bPx/KlwwazUjBVATFHkdaQVnC6jw6pUK5HY60haxzOqNxglE1VUwIYS3kXpc9o4toukUAKfdQupto6fTK1NCUHHE7bLgctn65k4Yb/S4o+81vlBJYsWLUKAHIb4wgDtwopXxLCFECbBJCvGjs+5WU8s483lujGVEEInGcdoHbkWYRxJKEjQE1pkVQn64IIglCsbjl4rHSR512jDADAONLMwOYpqJJDxz/8Mr+z8l1O+0juoYAwGW2oe7LNRSNwre+Bb/7neoc+r//O2IDw9nI56jKBqDBeO0XQmwHJufrfhrNSCYQjlNszQxIxQjauriG0hVBIBInFE1kzCMG5RoyFYHXaafEnfnfPJtFMJCn+k+dMZWFU8r6fd5wIifXUCCgisRefhm+9z247Tawj2wF2JWCZA0JIWag5he/AawAviGE+BywEWU1tGY55wbgBoCamhrq6uoGdO9AIDDgcwuFlnFwGMky7jkYwS4T1NXVEYiqVXzr+ztoDklcNji4axugsosEIIE169/gwJEYyaikrq6OpmNq5vGu7VsJG+MMSp1JVq9enXGvWDgIwJbNm2ja1X0B7M/vcQJQV7c3p2MHk8H6rhvqVfD99dfW4s6SOmsPBDjte9+jdPt23r/5Zo5ddhm8+mpBZSwEeVcEQohi4M/Ad6SUHUKIe4CfoP6WfwL8AvhC1/OklPcB9wEsXbpU1tbWDuj+dXV1DPTcQqFlHBxGsowPH9xIdTJIbe15qlr4leeZNmMW0cYAlS1NnHvWMnhDLUDjStw0+iOcfOpCVtbvYHqpg9raM3kz/D4vHtjDWcsWq5bVb29g+vgKamuXZ9yraus6DgfaOPfss5ha2T3lcST/HvvLDrGHlw7t5OILarF3DXq3tqqxkjt3wuOPc8o113DKEMhYCPKqCIQQTpQSeFhK+RcAKeWxtP2/B1bmUwaNZiTQGYlTYvT8N/3VkXiStlCM8iKn5TYCmFjupdEfIRCJ09wZtQLJ6TECsxZhfJdAMaTqB3rqNDqW+MxZ0znvpHHdlUBTkxogs20b/OUv8OEPD42ABSKfWUMCuB/YLqX8Zdr2iWmHXQW8ly8ZNJqRQiASt2YGOOw2HDZBJJ6gPdhdEUwuV0HKzmiclkCUKmNwfbasoZrS7gFNtxUj0NnjxW5HRgsOABobVbuI999XLaVHuRKA/NYRrAA+C1zYJVX050KId4UQW4ALgH/OowwazYB5+p0j3PynLQW5lxpKk1rs3Q4bkZgqKCv3uiwlATCpTAWOWzpj+CNxazqZeUyJ22Epha6po6Atgl5paFBTxXbvVoVil1021BIVhHxmDa0FsqUiPJuve2o0g8n6PU08914DP/v4aXm/V3rWEKin9kg8SaM/wlmzqnAZMwUi8SSTypUiONSigr5VxWqxv+K0ifjcdsaXevC47FT6XCyYXN7tXh6n3egrpC2CDOrrlSVQXw/PPaf6B40RdK8hjaYHIvEksYTs+8BBIJtF0BKM0h6KMdFwBRW7HUTiUSaUebAJOGgoAnM6WZnXyUcXqQztUo+Tt76ffUi622GzrAKNwcGDSgk0NsLzz6uCsTGE/mvQaHogGk8SMzp/5pNEUhKMJjLcP26Hjf1NnQBMNiyAYiOYXO514nM5LEVQXeyiP8yo9jG9amT3xhlU9u1TT/9NTfDii2NOCYC2CDSaHonGk8STkmRSYstjPx1zrrCZNQSqs6epCCYaMQGfMUay1OvE53akWQTd4wC98dXzP8A/nf+BE5Z7VPD22/CRj0BnpyoYW7JkqCUaErRFoNH0gGkNxJL5swrCsYTK+YcM15DLYaPTaDE9scxwDRmKoszrxOe2WwNsqvppEdhsonu65FijsxOuuw4WL4ZwGF55ZcwqAdAWgUbTI+ZAmFhC4s7D/5RoPMlZd7zMh09TGdVdXUMAQsAEQxGYrSJKval0UqdddGshoemDQACuuALWroXvfhduvhkqKoZaqiFF/wVpND1gPnHH4knon/clJzojcdqCMR7fcBhIPfFDqt/Q+BK3ld3jczsQQikEU2lU+dwjuvtnwTl6VDWN27BBNY679tqhlmhYoBWBRtMDUSNjKF8B46AxVMa0PDKzhlSOvxkfAOUCGlfsxmYTliIwM4Y0OfDuu/DBD6rWEU88oRSCBtCKQKPpEdMiiOZJEZjzhk26po8CTCpPVQZ/44LZXLtsasax/Y0PjFm2bFHpoR4PrFsHixYNtUTDipyDxUKIc4QQnzdejxNCzMyfWBrN0BM1ZgGYCmGwMRVBeZETyK4IMi0CNydPUO0QfO7UmElNH7z1VkoJ1NVpJZCFnCwCIcQPgaXAXOAPgBP4H1QbCY1mVJIeLM4H5rzhb1wwm/fq263sIEh3DWUffmLFCIrzELwYTaxfr9xBZWUqPXT27KGWaFiSq2voKtQ8gbcApJRHjKljGs2oJRbPc4zAqB84fVoFXzp3VsY+M1hsFpN1pdilYwR9smoVXHklTJyolMC0aUMt0bAlV9dQVEopUTMEEELoskTNqMe0CPIVIwgbFkH6fGETyzXUgyIwLYL+VhWPGZ57Dj70IZgxA9as0UqgD3JVBI8LIX4HlAshvgy8BPw+f2JpNENPRvpoHjBdQ94sXUDN0YmTenANFVtZQ9o11I0//xk++lGYN0/FBCZO7POUsU5OriEp5Z1CiEuADlSc4AdSyhf7OE2jGdFYiiBPMYJgtGeLYP6kMhZOKaO6hxiAWXOgs4bSiMfhhz+E//gPWL4cnn0Wyrt3X9V0p09FIISwA89LKS8G9OKvGRNIKdOCxfnNGvJkUQQfXDCRDy7o+Un2/JPGcfPlJ7Nwil7oANUm4uMfh2eegS99CX7zGyjqPoZTk50+FYGUMiGECAohyqSU7YUQSqMZatKtgHzXEWRzDfWFz+3gq7W6cRwAwSB87GPw0ktwzz3wT/801BKNOHLNGgoD7wohXgQ6zY1Sym/1dIIQYirw/4AJQBK4T0r5GyFEJfAYMAPYD/yDlLJ1QNJrNHkiffHPm0UQS+C06wExJ0QgoDKDVq+GBx6A668faolGJLkqgmeMn/4QB26UUr5lpJpuMhTJ9cDLUsqfCiG+B3wPuLmf19Zo8kp6EVm+CsqC0YQeF3kihEJqnvCrr8If/wif+cxQSzRiyTVY/JAQwgWcZGzaIaWM9XFOA9BgvPYLIbYDk4GPArXGYQ8BdWhFoBlmxApgEYRjiayBYk0OJBJq4V+zBh5+GD71qaGWaEQjVHlAHwcJUYtatPej5hBPBa6TUq7J6SZCzADWAPOBg1LK8rR9rVLKbj1ghRA3ADcA1NTULHn00UdzuVU3AoEAxcXFAzq3UGgZB4fBlPF4MMlNa0IAfG6eiwunOQfluuky3vtOmH3tSX523vAKag7379oWDjP7ttuYtG4du775TeqvvnqoRcrKcPg9XnDBBZuklEv7PFBK2ecPsAmYm/b+JGBTjucWG+dfbbxv67K/ta9rLFmyRA6UVatWDfjcQqFlHBwGU8Zdx/xy+s0r5fSbV8r7X93bbf/W+nYZjMT7fd10Gb/00AZ5+a/XnIiYeWFYf9f19VIuXSqTQkh5551DLU2vDIffI7BR5rBO5xqlckopd6Qpj52ofkO9IoRwAn8GHpZS/sXYfEwIMdHYPxFozFEGjaZgpMcFurqGwrEEH/vtOh7bcPCE7hGKJvA6daA4Z956C5Ytg/ff573bboMbbxxqiUYNuf4VbhRC3C+EqDV+fo96yu8RoaZl3A9sl1L+Mm3X08B1xuvrgL/2V2iNJt/0FiMIxxJE40naQr2GybIipWTXMT+gsoaKXLoTfE5s3AgXXAAOB6xbR/PZZw+1RKOKXBXBV4GtwLeAbwPbgL6SdVcAnwUuFEK8bfx8CPgpcIkQYhdwifFeoxlWpKePRrtUFltzCgaQTbStOcklv1rDnuMBnTWUK++8A5deCpWVarzkaacNtUSjjlwfRxzAb8wne6PauNcmJ1LKtajAcjYuyllCjaYA/GnTYX75wg7W3nwhNpvo1TUUOQFF0BJW5xxpC+msoVzYtg0uuQR8PjVgfurUoZZoVJKrRfAykN4G0YtqPKfRjAr2NQU40h7GH1atoTMKyros+CfSlTSoLk9LZ9SIEWhF0CO7dsFFF4HdrpTATD0LK1/kqgg8UsqA+cZ4Pbxy3jSaE8B8um8LRTPeQ/cF39wXifVfEYSMGQetnVGC0ThebRFkZ/9+pQTicTVLYM6coZZoVJOrIugUQiw23wghlgKh/Iik0RQeSxEEYxnvobtr6ERmGQdjShG0BGOEY0mtCLJx+LAaLRkIqP5B8+YNtUSjnlxjBN8BnhBCHEENp5kEXJs3qTSaAmMu6mYmUGaLCZn12IHECEKGa+i4P0I0kdSuoa40NChLoLlZKYGFC4daojFBrxaBEGKZEGKClHIDcDKqWVwc+DuwrwDyaTQFIWJZBMo1ZFoBboetR4sgMgBFEDSUSkO7Mqh1sDiNw4fh/POhvl7NEli2bKglGjP05Rr6HRA1Xi8H/h24G2gF7sujXBpNQTHbTrebFoGx+Be7HXlxDdW3KkWg00cNDhxQSuDoUXj+eVixYqglGlP05RqySylbjNfXolpJ/xn4sxDi7fyKptEUjmhczQboGiPwZVEEEStYnOj3fUzX0JE2bRFY7N2risU6OpQ76IwzhlqiMUdfFoFdCGEqi4uAV9L26ZJIzaiha7A4kqYIuhWUnVD6qLpW5wkMpRlV7NwJ552nAsMvv6yVwBDR12L+CLBaCNGEyhJ6FUAIMRvQ08o0o4ZUsDgzRlDstnevIziBgrJQLFOpZBtTOWbYvVu5gxIJNWR+wYKhlmjM0qsikFLeLoR4GZgIvGB0swNlSXwz38JpNIXCXNTb01xDDpvA7bAT7uICGqgikFISjIPTLqyYRNFYtQiamuBDH4JYTM0U0CmiQ0ouM4tfz7JtZ37E0WiGhlRBWUoRuBw2nHZBR7irRaAUQ39dQ+FYkoSEaRVF7GtSE1/HZB1Ba6uaLHbwoKoY1kpgyNE9cDUauqePRhOmIrARjSfZeqSd0259nsaOsKUA+ltZ7A8rJTOtMlWUP+aCxUeOqJjA5s3w6KOgu4gOC3TAV6Mh9XTfbqT1xBJJnHYbTqOOYNexAB3hOIdaQwNOH+0wFMH0qpQiGFPpo7t3qwZyTU3w3HOqelgzLNCKQKMhLUYQiiKlJBJP4rLbcNltxBISf0QpiIgxiyD9nFzpMBraZVoEY+S/4ObNcPnlkEzCqlWwtO/piZrCoV1DGg2pRT2WkASjarF3GzGCWCJJwFjEQ7EEkQG2mOgw4g8zqnzWtjGRPrpmDdTWgtsNr76qlcAwRCsCjQbl5nE71H+HtlAsLVisXEOdhkUQjiUzXEOpRLq+MVtcT6n0IoxJHZ7RPqpy9WplCUyaBOvWwcknD7VEmizk7a9QCPGAEKJRCPFe2rZbhRD1XSaWaTRDTjSeZFyJmrXUFoymYgRGsDhgKYJEhiXQU7+hrz/8Fne9vCtjm6kIyrxOyr1OvE47QvQ0u2kU8OqrKkV0xgxVJ6CHygxb8vk48iBweZbtv5JSLjJ+ns3j/TWanIklkow3FEF7MGZlDbkcKkZgKoJQF0XQU8B444EWXtnRmLHNDBaXepxU+FyjO3V03Tr44Adh2jSVIlpTM9QSaXohb4pASrkGaOnzQI1miEkmJbGEZHyJB0hzDVnB4lSMIBxLZM4z7sEi6Iwk2H0skOE68odj2IRKGa0oco3e+MD69codNHmyUgITJgy1RJo+GIqUhW8IIT4HbARulFK2ZjtICHEDcANATU0NdXV1A7pZIBAY8LmFQss4OAxURrOXUMzfBMCbb79Hc2scn1NQf6iDeFJyoEE93W/buZv6jtTiv/rVdVR5M5+npJR0RuJI4KnnV1HhUfu374ngsUtWr16NCIexJ+Sw/J2eyHddsm0bC2+6iWhFBW/fdhvRHTtgx47BFZDR/fc4FBRaEdwD/AQ13OYnwC+AL2Q7UEp5H0ar66VLl8ra2toB3bCuro6BnlsotIyDw0Bl7AjH4MUXOP3kWbx8cCfjp87E3dLAhAovc6aVw+4d4C4GOpg0ZTrtDR1wTCmGJcvO5N36dhw2wQcXTAQgGI0jn38egHEfOI0l0yvwh2OUHt2O73gDtbW1zF4YpDOSYO6EksH6+IPGgL/rt96Cf/93mDABx+rVnD1lyqDLZjKa/x6HgoIqAinlMfO1EOL3wMpC3l+jyYbp3in1OnE7bCpGEE9YriFQM4bBiBEkMmME//3qXmxpiqAzkupNtKvRz7PvNfD3945yysQSvA4VHJ5SMcpGfr/7Llx6KZSVKXdQHpWAZvApaO6aEGJi2turgPd6OlajKRSmInDZbZQXOWkNRjNaTAC0GK0nwrFERqZQJJakM5qgpTNqbQtG49brncf8/P29o7R0Rlm/p5kiZyE+UYF5/324+GJVJ/DKKzB9+lBLpOknebMIhBCPALVAtRDiMPBDoFYIsQjlGtoPfCVf99docsVSBA4b40rcNPojxOISlz2lCMxjzKwhIUBKiCYSdEbiVmooYGUYAazc0oA/HMduEySS0rIIRg179qgZw0IoJfCBDwy1RJoBkDdFIKX8VJbN9+frfhrNQDFdPS6HjSnlRexq9KdZBJkLd8QoKCt2O/CH40TiqtgsEIkTiSdwO+wEjaEzUyq8HG4N4bLbuOG8WfzXqt0UjSZFsH+/6hcUiag6gblzh1oizQAZ5WWNGk3fpLuGzMU7EkvgtKs6gnTM9NESt8M615w2ZrqHTItg4dRyAM6eXcXnzp6OwyYodhXkI+WfJ56AxYvVeMkXXoD584daIs0JoBWBZswTSXMNTa0sUk/50URGjMDEdA0Ve5Qi6AjHSSRV+mlzQCmCoBEsPt1QBJfMq2F8iYdHbziLD84Y4UECKeEHP4B/+AeYMwfefFMpBM2IZoy0PtRoeiY9RjClwmttz6YIzBYT1cajfWtakLjZeN1pBIsvmVeD22nnmsUqg2bpjEoC+0fws1cyCd/8Jvz2t/DFL8I994BzhCs2DaAVgUaTihHYbVRXuq3tbkema8huE4RiSeUa8qgFsDWYpggCEQCrQV2px8lnzxolGTTRKFx3nRomc9NN8LOfwWjukzTG0IpAM+aJpVkEk8tTFoHTLjKCxVU+lzWPwHQNpVsEZozADBYXuUdJC4lDh+CTn4TXXlMK4LvfHWqJNIOMVgSaMU961pDP7aDK56K5M5pRUAZQXexWNQbxVLC41Rh2D9AUSAWLHTaRce6I5dVX4aqrVGbQo/rHW2AAACAASURBVI/CtdcOtUSaPDAK/lI1mhMjPWsIsOIELoc9I0ZQXeK2KouLLUWQbhEo11AwEsfndoz8FtOPPKIKxaqrYdMmrQRGMVoRaMYUuxv9hGOJjG3pwWKAKcYoya7B4upil1U4ZsYITHeQ0y6srKHOaALfSG4xLSXccQd8+tNw1lnKJXTSSUMtlSaPaEWgGTOEYwmu+M+1PPzGwYztkUQXRWBYBE67wJVWADau2G2livoM/78ZI5hSUZTKGjIsghFJLMZJv/iFah736U+rGoHKyqGWSpNntCLQjBlU9W+Sg82dGdtNi8BtV4u72RDOnWYROO2CUm8qVdIcWmPGCKZWFtFsuIY6owmKRqIiCAbhIx9h0jPPwC23wB//qPoHaUY9WhFoxgwhI5vnWEckY3tX19BUK0aQUgTFbkfGIBmX3YbbbiNkuJmmVXppCaRZBCPNNRQIwBVXwAsvsOPGG+G228Cml4exwgh8bNFoBoaZ1tnoD2ds76oIzpxZxfVnz2DpjEoiMbXP53bgSVcEDhtupw1/BBw2wcQyL53RBOGYakJX6RtBbab374ePfxw2b4Y//pGGSZPQXYPGFlrla8YMZnvobhZBIoHdJrDbVDzA67Jz60dOpdTjtDKJit0OvK7UfxeXI5VaWuSyW5XGzZ1RgiMpWLxypWoRsXs3PPWUigtoxhxaEWjGDKE0iyB9lrA5nzgbppVQ4nHgcWS6hsx9xW4HlT7lS28OREZGsFhK+P734cor1fyATZvUa82YRCsCzZjBdA3FEjKjECwaT3ZrN21ibs/mGjIVQZHbQVWaRdAZHeaKIJmEr31NxQG+8AWVHqrnCIxphvFfq0YzuATT6gca/WEeefMgbofNmD2Q3ZVjtwmEUE/93WIExjk+t4NqwyI42h4mHEtSNFxdQ42NcMMN8Ne/ws03q3qBkV74pjlh8mYRCCEeEEI0CiHeS9tWKYR4UQixy/i3Il/312i6EkobIXmsI8If1x/gb+8cIRqXuB3Z/ysIIXDabYYiSB2T3pDO57IzqdyDy27j3fp2AKvyeFjx+utw6qnw97/DL3+plYDGIp+uoQeBy7ts+x7wspRyDvCy8V6jKQimawhge0MHRzvCNAVS84l7YnK5l+lVPryu9BiB3Yor+NwOHHYbs8b5eOdQGwBFrmGmCNasgUsugfJyeOst+Od/1kpAY5E3RSClXAO0dNn8UeAh4/VDwMfydX+NpivpiqBuRyOgWkRE44leG8Q99+1zueG8WZnBYkdmsBhg9vhi3j/qB1KVx8OCF1+Eyy+HqVNh9WqYN2+oJdIMM0R69sSgX1yIGcBKKeV8432blLI8bX+rlDKre0gIcQNwA0BNTc2SRx99dEAyBAIBiouLB3RuodAyDg59yfjnnVFW7o1R5IRwHBLGn/7JlTbCcbj1bG+P5wK0hpP8c10IgJ+e6+WxHVE2Nya4YKqD605189fdUZ7crYLQ317s5vTx3a2CQv8ex61ezSm3305w6lTeufNOYhV9e2NHw3c9HBgOMl5wwQWbpJRL+zpumNmvKaSU9wH3ASxdulTW1tYO6Dp1dXUM9NxCoWUcHPqScY1/G776Q0wu97LjmN/aHhIeqivc1Nae3ev124MxqHsBgHNXLGdt+3Y2NzZw0sxp1NaeQrCqgSd3vwXAWUtOZ/kHqvot46ARi8H/+T/w85/D8uUUr1zJihx7Bo2G73o4MBJkNCl0+ugxIcREAOPfxgLfXzOGCcXieJx2xpeqDB/TtdPQHs5pdoA7LVicXkdgxgPmjE89/Q2pa2j3bjjnHKUEvvIVWLVKN47T9EqhFcHTwHXG6+uAvxb4/poxTDCaoMhlZ3yJB4AzZ6rFMRrvPVhs4nbYrPiqSh81g8Vq0Z9e5cNhVCcPWbB440Y480zYtQsefxzuvVc3jtP0ST7TRx8B1gNzhRCHhRBfBH4KXCKE2AVcYrzXaPKKOX/AVAQ1hkVw7pxq65hcFIEQwgoYu9NaTJjFYy6HjRnVPmCI0keffRYuvBDKypRC+MQnCi+DZkSSz6yhT0kpJ0opnVLKKVLK+6WUzVLKi6SUc4x/u2YVaTSDyvo9zSy49XkOtwYJRRN4XXYmGXOJzztpnHVcLooAsFJIXXYbbmeqoMzEdA8VdF5xIqHSQa+4AmbOVKmis2YV7v6aEY9uMaEZVdS3hTL6CL20/RixhORgc5BgNE6Ry87Viyfz/75wBidPKLVaS7tznC/scdhw2AS2tJnE6Q3mFkwpo9jtwFco11AsBp/5DPz61/CNb8Abb8CUKYW5t2bUoBWBZkj58d+28fAbB3rcX7ejkcc3HMrpWluOx1nx01fYeKDV2vb63mZADZkPRhN4nQ6KXA7LGjB7BDlzVQQuu2U9WJXFaRbBF8+ZyXPfPtfqZJpX9u+Hyy6Dxx5TgeG77gKPJ//31Yw6tCLQDClPv1PPExsP97j/D+v285OV26wRkT0hpeTJXSqHf99xNYGsLRhlW0MHoIbMh2KJbj2AqnxKEeTqGvI4siiCtKd/t8PO1Mo8zyJIJuHuu2H+fNiwAR54AG66Kb/31IxqtCLQDBnJpOoCur2hg7gxN7grLZ1R/JE424509Hqtup3H2dehrnGsQw2eeXNfC6aXqC0YtYLF6VQVZ6aS9oXXlWotkQoWFzAe0NgIF1+s3EArVsDWrfD5zxfu/ppRiVYEmiGjPRQjkZRE4kl2Hw9kPabFGAhvunh64ner91DlEZR6HBwzJpC9vrcFt8OGx6lmC5vB4nT6bRE4U/UDVcUunHZBRZErp3NPmDffhKVLVfO4//5v1Txu2rTC3FszqtGKQDNkNBuLPMC7h9uzHmMqgjf29a4Idh0LsGCcnYllXmsC2et7m1kyvYIqn5vWYNQKFqdTacQIcikog0zX0BULJvLCP59PhS/PiiASgX/9V1i+XDWKW7sWvvhF3TROM2hoRaAZMlrSFMHWLK6fUDRBKJbAJpSbp6c4QSIpaQlGKXEJxpe6afRHiMaTvH+0g8XTKqjwOTnWESYpuxd6mXMEcrUILps/gY8unAyAw25jplE3kDeampQr6Be/gC9/GbZsUaMlNZpBRCsCTUHpCMe47Fdr2HywlZZO9eRe4nHwXn13i6DZ2H/GzEo6wnG2N2SPE7QFo0gJpU5BTamHxo4wh1uDJCXMrPZRUeTiSJtyF3mdXWME6mm+p3kEXfmHpVP59sVzcvuwJ8r69bBsmQoIP/qoqhIuKyvMvTVjCq0INAXl3cPt7Djm5819LZZr6JzZ1Ww90tHtid+0GD60YCIAG/dnrz80jytxCWoMi2Bfk8ocmlFdRHmRi/o21TW0m2uonzGCghAKwS23wLnnqverV8O11w6tTJpRzTD669eMVt6r///tnXl4W8W58H8j27Il7/sa27ETO07IvgAFgiFtINCSQKGsLS20AdrS297b2wtt6aXl68bWvV/L7ceFtinQAg2hAcKWkISwZY9jx06cxfG+y5atxZLm+2OOZNmxnQXHcuz5Pc95dDRnjub1SJ73zDvvvK+Ne/66A5fHS5UR9bO+00G7XQ3gS4tScfR5OdI6cMHYryhmZcURGxUeGNwH468XF6lmBF6fZGeN2kuQlxxNojUCt0d5FA1eLE7xew2d4hrBWefQIZg7F37yE7jtNti9W8UO0mjOIuPk16851/AFPb0fae1hy8GWYes++e4RXi1rZG+tLaAI6jqdtPW4iY0MZ1ZWHACHWwYO9B3GAJ8UHUlukpWa9t4hPz94RuAPKPfhkXZiIsNJjjaTEOTVM3iNIC/ZSkFKNEUZsaf0d59VDhyApUuhvR3efBOeekqbgjRjwrjNR6AZ39zyp/fpdnq469JCHlhbhs8n2fvgcoThydLl7MPV5yPOEs4b+5sA2FXTQVWTeuqv73RgNYeRFGMOPJUHLx4Hv0+KNpObZA0okcH4ZwSxEQQCyu05bmN6egxCCBKtEYG6g01DsVERvP3t0o/TFaPDli2wahVERMCmTWqzmEYzRugZgeaMKKvrYn99F994Zhddzj66XR5sjr7A9R+sLWPFr7bw0u56ul0eTAJ2HusMDOYNNgftPW6Sos0BO33bIEXQ1uMm3KT2BkxJsnK8wzFgJuLHb2KKMSvTEIDb6wtEAg328x9sGgo5Xi/84hewbBmkpCjXUK0ENGOMnhFoTptetwe7y8MXP5FPbJQyvzz4cjm1HY6AGeZwaw+tdhf3v7iPuKhwLp6ewjuVLfS4vWTGR9Fgc1LX6aAgJZqoiDCizWG02QfNCOxuEqPNCCGYkmTF7fHR3O0iI35gPJ22HhdxUeGEm0RgdgGQn6xCPSSMMCMIKRUVcMcdaoPY1VfDX/4Cp5BKUqMZbfSMYAKz5oNj/H37qQVsOx2ajQ1bs7Pj+Y/lxczPVYOX3zMHoL7TSYI1Aq9PsnxWBkvyk+gxkseXFquAb0daewKzgeSYyIA7qZ/2Xndg52+uEb9nqHWCth73gFAR/nvykk+cEVgjxsGzT10d3HcfzJsHVVXw17/Cyy9rJaAJGVoRTFB8Psljr1fxpy2HR/2zm7vVgO1P+ZidqOL71xuKwO3x0Wp38YUL8/nOlcV8tbSQebn9g9ylRWmBc//O3qRo8wmmIb/pCPoVwXFDEdR29HLxz9/m/cNttNv7FYaSS80Y8odQBCE1DTU0MP2Xv1Q5Ax55BK6/HsrLVRhpvUtYE0JC8ngkhDgKdANewCOlXBQKOSYyBxq7VcA2Zx99Xt8ph1k+FfxB3fweOsnRZqIiTNR1OAZcz0mw8LnFU4D+dJBxUeGclx0X+Cz/AJ4cbabRuM9Pe4874FGUnWBBiP4ZwWOvV1Hb4WDLwRbae9zkJVsBpUjS4yKpaFB7CAASokNoGurtVYu/a9bA88+T6fXC6tUqWujUqWMri0YzDKGcJ18mpWwNYfsTmm3Vqmv7vJJjbT1MSztz98ifvXqA5no3paXqfWBGEKtmBEIIshIsAdOQf2YQbMs3h5tYnJ+IJSKM9LgoTAJ8UrmGgpoRDA4z0WZ3DQgKlxVv4Xh7L2V1Nv65qw6A8vou2nrcLMhLCNyXGW8hJjKcVMNcFBup1g88PnnCzuJRR0qoroZXX1WpIzduVLGC4uLgrrv48PzzueDWW8+uDBrNaTIODKaas8G26jbM4SbcHh8Hm+ynrAjW7qpjY2Uzi/KTuGFhDq4+H09uPUJ6UIj95m4n5jDTgEXY7CBF4H+yz0oYuKj7h9sWIoQgIsxEWmwUjV3O/hlBTCTtPW6klDyyoZJF+Yl0OT0DArrlJFo42tbDj/5VTqI1goV5ieyttdHR229CAvj65dNYOS8r4MoqhCDBGoHd5cE0mgljpIRjx2DdOpUjuLJS2fw7O9X14mK45x646iq1SzgqCuemTaPXvkYzSoRKEUjgdSGEBP4opXwiRHKc83i8Pt4/3M5F05IDA1+f18cHh9v4zJwsXthZS1WTnRWzT7z3tbIGdtZ08t2rSgKf9eNXKujocfPS7nrKam3My03A7fXREWS1aelykRobGWgP1CDtzxngj+uTEW8Z0F5sVL/iyEpQiiApyDTk9vposbv4/abqgO9/sO0/N8nKP3aoJDYPXz+Hzl43b1Y0A8bMwqPqZSdYyE4Y2Hai1cxJctucGk6nygj2q1+pgb/XWLzOzoYZM+Dmm5X75xVXQGHhKDSo0Zx9QqUILpJS1gsh0oA3hBAHpJSbgysIIVYDqwHS09PZdIZPUna7/YzvHSs+jow7mjz8ZpeL/1ocRUmyMnsc7PDS4/aS4Wsh1SJ4t6yaPM9xqm0+Fqarr9zjk3x3swObS3KBpRGTEOxu9tDS7eLe+ZEcaPfy9+3HeadCDbz2Ptjw1kYiwwSVNQ6iJANkdnW4aevpY8NbG9le6cYSDtvf2zqs3OFupSyq9u2ko9pEc53ag/Dn9eqejl71vuHYITa5jgLg7VJrAIvSw0jtPkRLW38ym6aaatLjnMP2o+hzILzytPvZ5HYTU1lJxuuvk7J1K2bjad9eUEDH1VfjzMigfckSHIPzBB8/ro5BTPTf41ihZRxdQqIIpJT1xmuzEOKfwBJg86A6TwBPACxatEiW+g3Up8mmTZs403vHilOR8TvP76E4I447Lx64wHhw82HYVUGnNYvS0pL+Miq449OXUOHcS12Hgw2t0by2v5GyB0uJjgxn7a462p27AShZcAGZ8Rae/csOkqPb+cb1l9Pt9LD04Y009ngoSInmcGsPRXOXMDUlmp/seodpGdGUlvav8bfH1fLiwT1Mm7MYU90BcpN7KS1dOuzf815vBR82HubqTy7Fag5HVjbzP/s+whmTBRwhLVYFj7tk8XwuLEwGIGW6ja7wKh69YS4JVjNz7C4e2f4mAJcsnou3bv+w/bi2cRdHWnsoLb14xH4O0NEB3/qWWuT1eMBigWuvhZISWLiQmCuvJOYMPH0myu8x1GgZR5cxdx8VQkQLIWL958ByoGys5TiXkFLyr70NbNjfeMK1epuyy79T2R/rp7rFTnK0meSYSKanx3Coxc5r+xuRUi3kSil5YvNhwg17eV2H2uX71oEmVs3PJiLMRFK0mXtKCwk3CVYvLQCgwVgDaO52BTyG/PhNMXUdDhpsjhM2fQ3mpiW5/GjlrEDsH39egI+MxPOP3jCXvGQr09JiAveclx3Pn25fHNi0lhwTGViwTjpJcpgHPj2T39+2cMQ6uN2wfj3cey/MnKn8++++G154Qfn+r1kD3/8+rFih3T01E4pQzAjSgX8a9uVw4G9SytdCIMc5Q4vdRa/bO2T0zQbDHn+gsZvmLidpcVFUt9gpTFUD6PS02AHhnWs7HDj7fJQ3dHHHRVN58t0j1HU6qOt00OeVrJyXFah7z6WFrJqfTZ8RubPe5sTl8dLZ2xcYgP349xLUdTpotDmZnT1ysLSpKdEDkrr49xPsr7ORGR/F0qJU3vnPy07aNyWZcTR3t5AcHcnwYe/6cxMPSVeXSvzyu99BWxtYrXDppfDQQ7DwJMpDo5kAjLkikFIeBuaOdbvnMv5NVC3dLrqdfQMWXRtsDtLjImnqcrH5YCvXL8zhULOdK89TMfyL0pVCuLQolXeqWqjtdASCuV23IJsn3z1iKAeVCaw4KAqnySTITrDg7FM7ghttDloGbSbzkxEXRZhJsK/ORqvdTUbcwMXak+FfFPb45Gll/ZqTE8971W0kBu0VGBGXS23i2rOn/9ixQymDVatUCsjly8E8RnmINZpxgHYfPQc41tYfVuFIaw9zcvp95uttTi4rTuXtAy28U9XC5TPS6OjtozBVDaazsuL5t2XTuXHxFEof2URtRy8RJhNhJkFxRiyJ1giVG6DHTV5yNJHhJ/rZR0WEERuh2urfQzDQ9BMeZmLFeRn87YMaADITRjYNDdVGtDmMHreXgtRTVwSrlxawrCT9RLmlhNpaNdBXVoLdDrt2wYYNyvMHlN1/9my46Sb48pdVNjCNZhKiFcE5wHCKwOXx0tLtIivBwqVFqbxZ0URlo4ru6beth5kE3/pUEaBcNus6HPikJDfJSkSYiexE5f9f1+EImJOGIslioqHTEYgzlBp7oqnlx6tms/NYB/U2J1nxpzcjAGUe6ml3MDVleDkGICWxbgfzuuthzXoKX3oJfvpTaG6GmhoV1z+YnBw14F9yiUr+Mm0ahI2jIHQaTYjQimAc4PVJ7njqI+68eCoXTUs54XpNey8pRlC26qDkLU02NShnJVgoTo/lhZ21gSBzQw3qOYnWgBnIb37JirdwqNnO8Y5elpWkDytjYqSgweakpdsILxF3oiKIt0bwm1sW8OP15YHQEKdDUnQkx9sd/TOC7m71FL99u8rU1diovHk6O/tfvd7A/Vlms3rCz8mBJUvUYD93rlr4jYvTg75GMwxaEYwDGnskbx9oJjoyPKAIpJQ899FxVszOpKa9l2lp0dR3hg1YMPZ7DGXFW5iXm4A5zMS6PfVERZhO2FAFyrPnrQNN2F2eQDvZiRZeL1eJY4I9dAaTZBHsaHGyp9ZGtDmM5OihF18X5iXy4lcvOvkf3denQjG8/bYa6KuqeKzRhsPuoORvfdDRrsw5AeGzYcoUFbN/+nRISFDROhMTIT0d5s9na1MTly5bdvK2NRrNALQiGAfUdCuvnHcPteLzSUwmQXlDF/e9uI+6TgfH2nq5fEYqkeFhA/L6NhiKIDMhipjIcC4oTGZzVQvF6bFDhlLISbTQasT8988IghXGiIogSmBz9PGvvfWsmpdN2KmGaujqgr17oawMjhxRNvvt25V3jp+0NCgpwRsTSxsWTBfOgJRkVT53rvLcSR9+tuJHturQVRrNmaAVwWnS4/LwzIc13P6J/FGL6OlXBO09bsobujgvOz4QruGFHbW02l3kJUdjNYez/Wg7UkqEEIFQDn57/CdL0thc1ULhMAN6TlL/oO83v+Qk9pcVjrBImxSl/lZnn4+bluT2X5BSDfCVlWCzqYG/uVkN+Lt3q2TsfsxmFX/n2mvV031ODpSWQoHap+Cq7aSxvgtT8OdrNJqzjlYEp8mG/Y38n/UVFKRGc/mMkz+ldjn7iAoPwxw+vNKo6fKSFR9Fvc3Ju4daOS87nooGtehbb1ODfW6SlbiocHrcXpq7XaTHRVHf6SDBGhGIsb+sJJ3/Xref4vShFUF2Qn/kuAJjQTbLmBFkxEUNcEsdQF8fWa5OMrvszEiKZO7RffDs+/Dee7BtGzQ1nXhPYaFKvHL77TB/fr/t3jR8P8zJSRjgEaXRaMYGrQhOE7+N/r3qthEVwaFmOw+u28+26lZWzc/m8c/NG7KelJKabh8r5qSw+3gnWw+1ctelhZQ32JiWFkNNWy9ur4/cJGsgwUplYzfpcSrdY2aQd052goW/33UhMzKGjjTqf/pXoaAjA/cAFKZFg8OhnuzLy9VRUaFeDx3iix4PX/R/0I+N18JC5XP/iU+ogT4xEeLj1avVikajOTfQiuA0CSiCw20j1vvx+nL21HZyXnY86/c28MNrZg35xN3S7aLbDTMz44iODOdvH9Tg7PNS0dDNVbMzKUyNZsP+JvKSrZjDTVjNYbyyr4GlRanUdzoGmHYAFucnDS2Q3U56/XGK24+z2NSDeHw/7NtHUl0df69uYZqzHVbXKlMPKA+badOUx81111Flt5M8tZikpBhEUpLyyklLG7otjUZzTjEpFEGDzUGCxTwqaQqPtilFsL++C1tvH/HWEwf3mrZeNlW1cO/l0yktTuW632/jtbJGblg0JVDH4fay5WBLIJRzSWYcRRmx/O+7R3ny3SPYHH3MzIxlydR85k1JDMTXuXJWBuv3NfDvy4uoae9lydRhBv6qKli7Vplv9u2Dw4cJk5INwXUyMxF5ecxJjyYspwjOm6UG/pISKCoasLu2ftMmis6RAFoajeb0mPCKwOOTrPjVFj67IIcHPj3zY32WlJKjrb3MyIjlQGM3HxxpY/msjBPq/fWDY5iE4JYluaTHRZKXbGXt7rqAImizu7jz6e3sPt5JbKT6Ckqy4ogxh5OfbOW3b6sF1plZcRRnxA4I+3Dtgmxe3FXHzU+8j7PPy03TYlVilLIylSSluVl55dSq8NEUFcGCBcpWX1BAu9OLOTuLmEXzlCsmcHp7gDUazURjwiuCY10+Onv72FTZzAOfnsm26lZaul2snJd90ntdHi/r9zZwxawMoiPDabW7sbs8XLcgm8der+L9w+0nKIIel4fnPjrOFbPSAxE4V87L5jdvH6SqqZvIcBO3P/khDTYndy0t4KltR0m3CuIMs9FNS3L52asHAChOsaqF2PJyFRXz4EEubmtj/dFWpMdDhlmS8vBR8Blx+dPSIDkZLrpI7Z79zGcgd6AHzjDzB41GM4mZ8IqgqkMNktUtPTR1Ofn+2jJq2nqZPyWR3OSRFzR/t7GaX791kOsWtPL45+YFzEJF6bEsyk/knapmpCwZkKnrkQ2VdDn7+MolBYGyGxbm8PS2o1zz261YIsIQPi/P3ljC/MRwPp+QRvX6V+A7m6CmhjuaWznvSBvZjg5iHq3t3zlrNsOMGYjkZGKy02ns9ZBYmAZfvFUt2M6dC7FnnpdYo9FMXia8IjjY6uaPL/2U+ugknp1hYfGba7nO1sS2tvfJve9LMHXqkPcdbrHzh03VpMZG8uLOOi4rTsPhVoPy1GQr1xbE8tBLR9lxpI1FBSm4PF62Vbfx7OZKvjndwvwdm+CJHbBrF1MaGthh66KntQOzoxdLnxN+qNrJMQ7MZsjLw5yURIEF3FmFcOetkJWlnuovuwxilMtnnnFoNBrNaDChFYHPJ8l7fytXHHhXFXzpZVUuTJje88ETD8GFF8Itt8DSpQH3R4cpgh89vZXFTZX8fp6F53eVU37fevKLc/nlm6+T+9sbyevp4XrA9xuBzxRGmM/HUuCA7E+fSFiYWnjNzyd8xgziY2KQMTEq7k1sbODY1drK/NWrIVK5dGah0Wg0Y8eEVgSVjV18ZetzdOcV8Os7f0Tcqy9jv/xT3POdm/n8957h5rqd3Fy9FXHvvQPuswBPBb2/03/yGtijohG33QzFxayvaOZo5XFMPi9ZCRaKMuKYkpNKTH4OzJoFc+aoUMdBDBWYwbZpU0AJaDQazVgTEkUghLgS+BUQBvxJSvmzs9FO/ZrnWdZ8hLbf/pEpCy7iBz0J/PqG+STEWbnx1k/y3bUZWB64n2vNNnZseI/n3thHnMuOxeNm+ZJpzL5ojhrMk5N5aO0etmwpo2DJbP5wl8rFO7W+i6/9egszMmJ5/p5PEBM5ofWqRqOZoIz5yCWECAN+B3wKqAU+EkKsk1KWj3ZbMW9toC4hnayvfJHrpAlnn5crDS+fW5bk8vyOWh76VwWRq87jgZ48sq6awQ9XziLRaj4hS9bdn72Avx91sjy/fzfxzKw4nvrSYmZlxWsloNFozllCMXotAQ4ZRGBawAAACIZJREFUKSsRQjwLrARGXRGc/+pzvPGPF8k2m4kBVi8tDFwzmQSP3jCHO5/ezlfX7CQiTLDmhvOZkTF0HP3U2Eg2fruU2KiBXVZarHfXajSac5tQKIJs4HjQ+1rg/LPSkhBEpCUPe3laWiwbvrlU+fLHRQ6rBPykjJQAXaPRaM5RhPTHlhmrBoW4AbhCSvll4/3ngSVSynsH1VsNrAZIT09f+Oyzz55Re3a7nZiYU0x9GCK0jKODlnF00DKODuNBxssuu2yHlHLRSStKKcf0AC4ENgS9vx+4f6R7Fi5cKM+UjRs3nvG9Y4WWcXTQMo4OWsbRYTzICGyXpzAuj05mldPjI2C6EGKqEMIM3ASsC4EcGo1GoyEEawRSSo8Q4uvABpT76JNSyv1jLYdGo9FoFCHxeZRSvgK8Eoq2NRqNRjOQUJiGNBqNRjOO0IpAo9FoJjlaEWg0Gs0kRysCjUajmeSM+YayM0EI0QIcO8PbU4DWURTnbKBlHB20jKODlnF0GA8y5kkpU09W6ZxQBB8HIcR2eSo760KIlnF00DKODlrG0eFckNGPNg1pNBrNJEcrAo1Go5nkTAZF8ESoBTgFtIyjg5ZxdNAyjg7ngozAJFgj0Gg0Gs3ITIYZgUaj0WhGYEIrAiHElUKISiHEISHEfeNAnilCiI1CiAohxH4hxL8Z5Q8KIeqEELuN46oQy3lUCLHPkGW7UZYkhHhDCHHQeE0MoXzFQX21WwjRJYT4Zqj7UQjxpBCiWQhRFlQ2ZL8Jxa+N3+ZeIcSCEMr4iBDigCHHP4UQCUZ5vhDCEdSffwihjMN+t0KI+41+rBRCXBFCGZ8Lku+oEGK3UR6SfjwtTiVW9bl4oCKbVgMFgBnYA8wMsUyZwALjPBaoAmYCDwLfDnWfBcl5FEgZVPYwcJ9xfh/w81DLGfQ9NwJ5oe5HYCmwACg7Wb8BVwGvAgK4APgghDIuB8KN858HyZgfXC/E/Tjkd2v8/+wBIoGpxv98WChkHHT9MeAHoezH0zkm8owgkBtZSukG/LmRQ4aUskFKudM47wYqUKk7zwVWAk8b508Dq0IoSzDLgGop5ZluOBw1pJSbgfZBxcP120rgz1LxPpAghMgMhYxSytellB7j7ftAztmWYySG6cfhWAk8K6V0SSmPAIdQ//tnlZFkFEII4HPAM2dbjtFiIiuCoXIjj5tBVwiRD8wHPjCKvm5MzZ8MpdnFQAKvCyF2GClDAdKllA2gFBqQFjLpBnITA//hxlM/wvD9Nl5/n3egZip+pgohdgkh3hFCXBIqoQyG+m7HYz9eAjRJKQ8GlY2nfjyBiawIxBBl48JFSggRA7wAfFNK2QX8X6AQmAc0oKaVoeQiKeUCYAXwNSHE0hDLMyRGhrtrgH8YReOtH0di3P0+hRDfAzzAGqOoAciVUs4H/h34mxAiLkTiDffdjrt+BG5m4MPJeOrHIZnIiqAWmBL0PgeoD5EsAYQQESglsEZK+SKAlLJJSumVUvqA/2EMprYjIaWsN16bgX8a8jT5TRfGa3PoJAywAtgppWyC8dePBsP127j6fQohbgc+DdwqDcO2YW5pM853oOzvRaGQb4Tvdrz1YzhwHfCcv2w89eNwTGRFMO5yIxu2w/8HVEgpHw8qD7YNXwuUDb53rBBCRAshYv3nqIXEMlTf3W5Uux14KTQSDmDAk9d46scghuu3dcAXDO+hCwCb34Q01gghrgT+C7hGStkbVJ4qhAgzzguA6cDhEMk43He7DrhJCBEphJiKkvHDsZYviE8CB6SUtf6C8dSPwxLq1eqzeaA8M6pQGvh740Cei1HT1r3AbuO4CvgLsM8oXwdkhlDGApQXxh5gv7/fgGTgLeCg8ZoU4r60Am1AfFBZSPsRpZQagD7Uk+qdw/UbyqTxO+O3uQ9YFEIZD6Hs7P7f5B+Mup81fgN7gJ3AZ0Io47DfLfA9ox8rgRWhktEofwq4e1DdkPTj6Rx6Z7FGo9FMciayaUij0Wg0p4BWBBqNRjPJ0YpAo9FoJjlaEWg0Gs0kRysCjUajmeRoRaCZ0AghvGJgpNIRo9AKIe4WQnxhFNo9KoRIOYP7rjAibSYKIV75uHJoNKdCeKgF0GjOMg4p5bxTrSylDHWI4EuAjajolu+GWBbNJEErAs2kRAhxFBUG4DKj6BYp5SEhxIOAXUr5qBDiG8DdqPg75VLKm4QQScCTqI13vcBqKeVeIUQyapNRKmpnqwhq6zbgG6hw6B8AX5VSegfJcyNwv/G5K4F0oEsIcb6U8pqz0QcajR9tGtJMdCyDTEM3Bl3rklIuAX4L/HKIe+8D5ksp56AUAsAPgV1G2XeBPxvl/w1slSqw2DogF0AIUQLciArkNw/wArcObkhK+Rz98e1no0IozNdKQDMW6BmBZqIzkmnomaDXXwxxfS+wRgixFlhrlF2MChmAlPJtIUSyECIeZcq5zihfL4ToMOovAxYCH6lQU1gYPmDfdFSoBACrVDkrNJqzjlYEmsmMHObcz9WoAf4a4AEhxCxGDns81GcI4Gkp5f0jCSJUStAUIFwIUQ5kGqkO75VSbhn5z9BoPh7aNKSZzNwY9Ppe8AUhhAmYIqXcCHwHSABigM0Yph0hRCnQKlVOieDyFYA/ccpbwPVCiDTjWpIQIm+wIFLKRcB61PrAw6hgf/O0EtCMBXpGoJnoWPxJxA1ek1L6XUgjhRAfoB6Ibh50XxjwV8PsI4BfSCk7jcXk/xVC7EUtFvtDTP8QeEYIsRN4B6gBkFKWCyG+j8r4ZkJFq/waMFRqzQWoReWvAo8PcV2jOSvo6KOaSYnhNbRIStkaalk0mlCjTUMajUYzydEzAo1Go5nk6BmBRqPRTHK0ItBoNJpJjlYEGo1GM8nRikCj0WgmOVoRaDQazSRHKwKNRqOZ5Px/mrREIn2zhfIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd9ac690e48>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from agents.ddpg_agent import Agent\n",
    "\n",
    "def plot_scores(agent_type, scores, average_scores):\n",
    "    fig = plt.figure()\n",
    "    ax = fig.add_subplot(111)\n",
    "    plt.plot(np.arange(len(scores)), scores, label=agent_type)\n",
    "    plt.plot(np.arange(len(scores)), average_scores, c=\"r\", label='average')\n",
    "    plt.ylabel('Score')\n",
    "    plt.xlabel('Episode #')\n",
    "    plt.legend(loc='upper left')\n",
    "    plt.grid(which='major')\n",
    "    plt.show()\n",
    "\n",
    "def ddpg(n_episodes=500, max_t=10000, print_every=100):\n",
    "    mean_scores_deque = deque(maxlen=print_every)\n",
    "    mean_scores = []\n",
    "    average_scores = []                                    # moving average based on scores\n",
    "\n",
    "    for i_episode in range(1, n_episodes+1):\n",
    "        env_info = env.reset(train_mode=True)[brain_name]\n",
    "        agent.reset()\n",
    "        states = env_info.vector_observations[0]\n",
    "        score = 0\n",
    "\n",
    "        for t in range(max_t):\n",
    "            actions = agent.act(states)\n",
    "            env_info = env.step(actions)[brain_name]   \n",
    "            next_states = env_info.vector_observations[0]      # get the next state\n",
    "            rewards = env_info.rewards[0]                      # get the reward\n",
    "            dones = env_info.local_done[0]                     # see if episode has finished\n",
    "            \n",
    "            agent.step(states, actions, rewards, next_states, dones)\n",
    "            \n",
    "            score += rewards\n",
    "            states = next_states                               # roll over the state to next time step\n",
    "            \n",
    "            if dones:                                          # exit loop if episode finished\n",
    "                break\n",
    "\n",
    "        mean_scores_deque.append(score)\n",
    "        mean_scores.append(score)\n",
    "        average_scores.append(np.mean(mean_scores_deque))      # save most recent average score\n",
    "        \n",
    "        print('\\rEpisode {}\\tAverage Score: {:.2f}'.format(i_episode, np.mean(mean_scores_deque)), end=\"\")\n",
    "\n",
    "        if i_episode % print_every == 0:\n",
    "            print('\\rEpisode {}\\tAverage Score: {:.2f}'.format(i_episode, np.mean(mean_scores_deque)))\n",
    "            torch.save(agent.actor_local.state_dict(), './models/checkpoint_ddpg_actor.pth')\n",
    "            torch.save(agent.critic_local.state_dict(), './models/checkpoint_dppg_critic.pth')\n",
    "\n",
    "        if np.mean(mean_scores_deque)>=30.0:\n",
    "            print('\\nEnvironment solved in {:d} episodes!\\tAverage Score: {:.2f}'.format(i_episode, np.mean(mean_scores_deque)))\n",
    "            torch.save(agent.actor_local.state_dict(), './models/checkpoint_ddpg_actor.pth')\n",
    "            torch.save(agent.critic_local.state_dict(), './models/checkpoint_dppg_critic.pth')\n",
    "            break\n",
    "            \n",
    "    return mean_scores, average_scores\n",
    "    \n",
    "agent = Agent(state_size=state_size, action_size=action_size, random_seed=10)\n",
    "scores, average_scores = ddpg()\n",
    "plot_scores('ddpg', scores, average_scores)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "env.close()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "drlnd",
   "language": "python",
   "name": "drlnd"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}