Lab1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Problem 1 (Value iteration)\n",
    "\n",
    "*OpenAI gym FrozenLake environment*\n",
    "\n",
    "Winter is here. You and your friends were tossing around a frisbee at the park\n",
    "    when you made a wild throw that left the frisbee out in the middle of the lake.\n",
    "    The water is mostly frozen, but there are a few holes where the ice has melted.\n",
    "    If you step into one of those holes, you'll fall into the freezing water.\n",
    "    At this time, there's an international frisbee shortage, so it's absolutely imperative that\n",
    "    you navigate across the lake and retrieve the disc.\n",
    "    However, the ice is slippery, so you won't always move in the direction you intend.\n",
    "    The surface is described using a grid like the following\n",
    "\n",
    "        SFFF\n",
    "        FHFH\n",
    "        FFFH\n",
    "        HFFG\n",
    "\n",
    "    S : starting point, safe\n",
    "    F : frozen surface, safe\n",
    "    H : hole, fall to your doom\n",
    "    G : goal, where the frisbee is located\n",
    "\n",
    "    The episode ends when you reach the goal or fall in a hole.\n",
    "    You receive a reward of 1 if you reach the goal, and zero otherwise."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this assignment, you will be implementing a value iteration algorithm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "numpy version 1.25.2\n",
      "gym version 0.25.0\n"
     ]
    }
   ],
   "source": [
    "import gym\n",
    "import numpy as np\n",
    "print(\"numpy version\", np.__version__)\n",
    "print(\"gym version\", gym.__version__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/wooseong/miniforge3/lib/python3.9/site-packages/gym/core.py:329: DeprecationWarning: \u001b[33mWARN: Initializing wrapper in old step API which returns one bool instead of two. It is recommended to set `new_step_api=True` to use new step API. This will be the default behaviour in future.\u001b[0m\n",
      "  deprecation(\n",
      "/Users/wooseong/miniforge3/lib/python3.9/site-packages/gym/wrappers/step_api_compatibility.py:39: DeprecationWarning: \u001b[33mWARN: Initializing environment in old step API which returns one bool instead of two. It is recommended to set `new_step_api=True` to use new step API. This will be the default behaviour in future.\u001b[0m\n",
      "  deprecation(\n"
     ]
    }
   ],
   "source": [
    "env=gym.make('FrozenLake-v1')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "## DO NOT CHANGE THIS CELL\n",
    "\n",
    "def argmax(env, V, pi, action, s, gamma):\n",
    "    e = np.zeros(env.action_space.n)\n",
    "    for a in range(env.action_space.n): # iterate for every action possible\n",
    "        q = 0\n",
    "        P = env.P[s][a]\n",
    "        for prob, next_state, reward, done in P:    # iterate for every possible states\n",
    "            q += prob * (reward + gamma * V[next_state])    # calculate action_ value q(s|a)\n",
    "        e[a] = q\n",
    "\n",
    "    m = np.argmax(e)\n",
    "    action[s] = m   # Take index which has maximum value \n",
    "    pi[s][m] = 1    # update pi(a|s) \n",
    "\n",
    "    return pi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Your task is to complete the \"bellman_opt_update\" function below, which servces as a subroutine for the value iteration method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def bellman_opt_update(env, V, s, gamma): # update the state_value V[s]\n",
    "    ####Your code here####\n",
    "    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def value_iteration(env, gamma, theta):\n",
    "    V = np.zeros(env.observation_space.n)   # initialize v(0) to arbitory value, my case \"zeros\"\n",
    "    while True:\n",
    "        delta = 0\n",
    "        for s in range(env.observation_space.n):    # iterate for all states\n",
    "            v = V[s]\n",
    "            bellman_opt_update(env, V, s, gamma)    # update state_value with bellman optimality update\n",
    "            delta = max(delta, abs(v - V[s]))   # assign the change in value per iteration to delta  \n",
    "        if delta < theta:\n",
    "            break   # if change gets to negligible --> converged to optimal value\n",
    "\n",
    "    pi = np.zeros((env.observation_space.n, env.action_space.n))\n",
    "    action = np.zeros((env.observation_space.n))\n",
    "    for s in range(env.observation_space.n):\n",
    "        pi = argmax(env, V, pi, action, s, gamma)   # extract optimal policy using action value \n",
    "\n",
    "    return V, pi, action     # optimal value funtion, optimal policy with one-hot encoding (pi), optimal policy with discrete number (action)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/wooseong/miniforge3/lib/python3.9/site-packages/gym/core.py:268: DeprecationWarning: \u001b[33mWARN: Function `env.seed(seed)` is marked as deprecated and will be removed in the future. Please use `env.reset(seed=seed)` instead.\u001b[0m\n",
      "  deprecation(\n",
      "/Users/wooseong/miniforge3/lib/python3.9/site-packages/gym/utils/passive_env_checker.py:241: DeprecationWarning: `np.bool8` is a deprecated alias for `np.bool_`.  (Deprecated NumPy 1.24)\n",
      "  if not isinstance(terminated, (bool, np.bool8)):\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " agent succeeded to reach goal 741 out of 1000 Episodes using this policy \n",
      " success rate: 0.741\n"
     ]
    }
   ],
   "source": [
    "## DO NOT CHANGE THIS CELL\n",
    "\n",
    "gamma = 0.99\n",
    "theta = 0.000001\n",
    "\n",
    "env.seed(1)\n",
    "V, pi, action = value_iteration(env, gamma, theta) # note \"action\" is optimal policy with discrete action number\n",
    "\n",
    "#initialize episodic structure\n",
    "num_episodes=1000;\n",
    "episode_max_length=10000;\n",
    "\n",
    "e=0\n",
    "for i_episode in range(num_episodes):\n",
    "    s = env.reset()\n",
    "    for t in range(episode_max_length):\n",
    "        s, reward, done, _ = env.step(action[s])\n",
    "        if done:\n",
    "            if reward == 1:\n",
    "                e +=1\n",
    "            break\n",
    "print(\" agent succeeded to reach goal {} out of {} Episodes using this policy \".format(e+1, num_episodes))\n",
    "print(\" success rate:\", (e+1)/num_episodes)\n",
    "env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Problem 2 (Q-learning)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For proper accounting rewards while you learn, we build a wrapper around env.step() and env.reset(). In an episode, every time you take an action the reward will be appended to the reward of the episode, and when ever the environment is reset (at the end of an epsiode), the episode reward is reset to 0. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "## DO NOT CHANGE THIS CELL\n",
    "#wrapper for accounting rewards\n",
    "rEpisode=0\n",
    "rList=[]\n",
    "\n",
    "def reset_decorate(func):\n",
    "    def func_wrapper():\n",
    "        global rList\n",
    "        global rEpisode\n",
    "        rList.append(rEpisode)\n",
    "        rEpisode=0\n",
    "        return(func())\n",
    "    return func_wrapper\n",
    "\n",
    "env.reset = reset_decorate(env.reset)\n",
    "\n",
    "def step_decorate(func):\n",
    "    def func_wrapper(action):\n",
    "        global rEpisode\n",
    "        s1, r, d, other = func(action)\n",
    "        rEpisode+=r\n",
    "        return(s1, r, d, other)\n",
    "    return func_wrapper\n",
    "\n",
    "env.step = step_decorate(env.step)\n",
    "\n",
    "def init():\n",
    "    rEpisode=0\n",
    "    rList=[]\n",
    "    return;"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "## DO NOT CHANGE THIS CELL\n",
    "# PLOTTING functions\n",
    "from numpy import convolve, ones\n",
    "def movingaverage(interval, window_size):\n",
    "    window= np.ones(int(window_size))/float(window_size)\n",
    "    return np.convolve(interval, window, 'valid')\n",
    "from pylab import plot\n",
    "%matplotlib inline "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below we illustrate the execution of the Open AI gym enviornment using the policy of chosing random action in every state. Every time an action is taken the enviorment returns a tuple containing next state, reward, and the status (whether terminal state is reached or not). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD4CAYAAAAAczaOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAABFxElEQVR4nO29ebwdVZXo/113zM08h4yEIQwBZAqTAk9FEJyiNNhgq7RPxefQ7dDdNtqf5iGtP9t+iv17LQ68Rn8+HEBB27RC4wDS2kAgzAQIJExJIPM83OQO+/fHqXNOVZ1dVbvqVJ1z7j3ryyfcOlV7WFW196q119q1S4wxKIqiKO1BR7MFUBRFURqHKn1FUZQ2QpW+oihKG6FKX1EUpY1Qpa8oitJGdDVbgDDTp083CxcubLYYiqIoI4qHHnpoizFmRlK6llP6CxcuZMWKFc0WQ1EUZUQhIi+5pFP3jqIoShuhSl9RFKWNUKWvKIrSRqjSVxRFaSNU6SuKorQRTkpfRC4UkVUislpErrIc7xWRW7zjy0Vkobd/oYjsF5FHvX/fzll+RVEUJQWJUzZFpBO4HjgfWAc8KCLLjDFP+ZJ9ENhujDlSRC4DvgL8qXdsjTHmpHzFVhRFUbLgYumfDqw2xjxvjDkI3AwsDaVZCnzf274VOE9EJD8xRy97Dwzyzd+v5t8eWd9sURRFaQNclP5cYK3v9zpvnzWNMWYQ2AlM844dJiKPiMg9InKOrQIRuVJEVojIis2bN6c6gZHOvWu28k//sYpP3fJos0VRFKUNKDqQ+yqwwBhzMvAZ4EciMjGcyBhzgzFmiTFmyYwZiW8RjyqGhoebLYKiKG2Ei9JfD8z3/Z7n7bOmEZEuYBKw1RhzwBizFcAY8xCwBjiqXqFHE/rhMkVRGomL0n8QWCQih4lID3AZsCyUZhlwhbd9CXCXMcaIyAwvEIyIHA4sAp7PR/TRgV/n66crFUUpmsTZO8aYQRH5BHAn0Al81xizUkSuBVYYY5YBNwI3ichqYBulBwPAucC1IjIADAP/wxizrYgTURRFUZJxWmXTGHM7cHto39W+7X7gUku+24Db6pRxVOM37o0BnfOkKEqR6Bu5Tcb4HDzq3FEUpWhU6TcZdeMritJIVOm3EBrIVRSlaFTpNxlV84qiNBJV+k3Gb93rA0BRlKJRpa8oitJGqNJvMuEpm4qiKEWiSr+FMOrgURSlYFTpNxlV9IqiNBJV+k1G3TuKojQSVfpNRhW9oiiNRJV+k1GdryhKI1Gl30Ko1a8oStGo0m8yuvSCoiiNRJV+kwl8REWdPYqiFIwq/Wajel5RlAaiSr/JBNbT1weAoigFo0q/hVCdryhK0ajSbzJq3SuK0khU6TeZQCBXnwCKohSMKv0mo3peUZRGokq/yeiH0RVFaSSq9FsItfoVRSkaVfpNRhW9oiiNRJV+kzGRPxRFUfJHlX6zUVNfUZQGokq/yejaO4qiNBJV+i2EGv2KohSNKv0mo4peUZRGokq/yfjfwlX9ryhK0ajSbzKq6BVFaSROSl9ELhSRVSKyWkSushzvFZFbvOPLRWRh6PgCEdkjIn+dk9yjBr97R9feURSlaBKVvoh0AtcDFwGLgctFZHEo2QeB7caYI4GvA18JHb8OuKN+cUc3qvIVRSmaLoc0pwOrjTHPA4jIzcBS4ClfmqXANd72rcA3RESMMUZE3gm8AOzNS+iRyupNe/iTb93L/oNDiMDRh0xg5Su7mi2W0mRu/OMLfPXOVTz9Dxfy/Xtf5Eu3P83BwWEADpk4ht985lwmjOluspTKaMHFvTMXWOv7vc7bZ01jjBkEdgLTRGQ88LfAF+IqEJErRWSFiKzYvHmzq+wjjrXb97Fz/wBvO3E2B4eGeXzdTvq6OznmkAmAzuRpV/7hl0+xf2AIgGc27K4ofIANu/rZtvdgs0RTRiFFB3KvAb5ujNkTl8gYc4MxZokxZsmMGTMKFqmJeEr9/WctpFMEgEl93fzZmYc2USiltag++Xu7St1TjQElT1zcO+uB+b7f87x9tjTrRKQLmARsBc4ALhGRfwImA8Mi0m+M+Ua9go9Eym/cCuDp/Mpf/3GlffEr+HLb0Fah5ImL0n8QWCQih1FS7pcB7wmlWQZcAdwHXALcZUpTUc4pJxCRa4A97arwIdpiq+h97d1tj62N6KwuJU8Slb4xZlBEPgHcCXQC3zXGrBSRa4EVxphlwI3ATSKyGthG6cGghCj3XREQBDClbYnNprQJxhjraE9VvpInLpY+xpjbgdtD+672bfcDlyaUcU0G+UYV5c4rSMW8l6qdr51bCbp3UGtAyR99I7cJSFXn+6x+pd0Je3E6xL5fUepBlX4D8ftmK4HcwPHGyqO0FobgaE8qfj9tGEp+qNJvIFFdV336Cng+fWsgt/GyKKMXVfoNpDaQ67fmdMpmu1Oy9DWQqxSLKv2GUp6nLwH3TmUQr71bCQRyFSV/VOk3AX8gF52yqXiEH/qigVylAFTpN5CAe8fr0YFAbuNFUlqI8Cz9chtRt5+SJ6r0G0hkIFcH8golo8D29q1a+kqeqNJvIBVL36fmA4Fc7d1tT9DS9/Zps1ByRJV+A6ksuOaL3grVbe3cirYBpWhU6TcB/4ydQFBXaWtq38hVn76SP6r0G4g9kKsqXylRE8gt71edr+SIKv0GEv1Grip+JTqQqyh5okq/gVQ7tNg/oqL9ve3RQK5SNKr0m0DYj19dVkt7dztjKv8royNAJX9U6TeB0ucSq2vvqHdHgVrXTvVziWoMKPmhSr+BVAO5vnn6luNKexJecE0DuUoRqNJvIFEWm1r67U1SXEd1vpInqvQbSPWNXOyB3IZLpLQSpdk71d/VQK62DCU/VOk3EP88fSrr6Vfn6mvnbnNM2L2jQ0Alf1TpNxD/h9Gr6+lrILfd8c/eslr6DZdIGc2o0m8C4Q+jl9HO3d5EDfR0AKjkiSr9BqLuGyUJ2zIMag4oeaJKv4HY3rbUKZtKGUM4kFuO9TRHHmV0okq/kVg+jI6Ib+0d7d3tSFW5l2bqK0qRqNJvINX19O0fRlfaE/8jXwO5StGo0m8C4fX0y+gwvr3RQK7SCFTpN5CozqtTNpUy9lU2Vesr+aFKv4FU5ukHPqJSe1xpT0rz9GtfztJ2oeSJKv0G4v8wepnS4ms6S6Od8cfxbZa+ouSJk9IXkQtFZJWIrBaRqyzHe0XkFu/4chFZ6O0/XUQe9f49JiLvyln+EYX/w+iBQK527rbGb9EHArneXzUGlDxJVPoi0glcD1wELAYuF5HFoWQfBLYbY44Evg58xdv/JLDEGHMScCHwHRHpykn2EUv0gmvau5Uqlamc2i6UHHFRwKcDq40xzwOIyM3AUuApX5qlwDXe9q3AN0REjDH7fGnGMIrck394bjMbdvZjgN39g5X9C6aO5fzFs6x5Nu06YN1f1vs/f2Q9MydsBWDOpDH0Dw7x1hPm0NOVjxfujide5czDpzFlXE8u5fn53dMbeXFr6Xa/7shpHHPIxNzrCNM/MMRtD69DEC4+ZS5jujsBuHvVJjbu7OfiU+bldu2KYPPuA/zq8Vc4ODQMwL/+4XnueXZzTbo7ntjArx5/lTcdO4s3HDOz0WIGeGbDLv5r9dbKbwEuPP4Q5kzua55QSipclP5cYK3v9zrgjKg0xphBEdkJTAO2iMgZwHeBQ4H3GWMGQ3kRkSuBKwEWLFiQ9hyawvtufMC6v0Pg2S9eRFdnrbLZ1T8AwPgxXcybPJa12/Yzb8pYhr1H4Xfueb4mT2dHB+84cU7d8q7fsZ+P/vBhLj5lLte9+6S6y/MzPGy48qaHGPJO5PVHz+D/+8DpudZh4w/PbeHvfv4kAHMmj+H1R5cU4ge+9yAAC6eP48zDpxUuR1ZufuBlvvabZyu//88fXgBg5oReNu0+wCWnzuOff/ssN93/EgA/XP4yL/7jW5sia5mv3PEMd68KPpg27Orn8285tkkSKWkp3NVijFkOHCcixwLfF5E7jDH9oTQ3ADcALFmyZMSNBr7xnpM5Z9EM/vUPz/Mvd62uKPEwPZ0ddHcKY3u6+MGHzmDPgUEm9Hbx1Ku7Kmk+d9ExbNx1gO/+V0kB7Nh3MBcZ+weGAHj05R25lOfHAEPDho+9/gjuXbOVAc9yLRp/PQNDtRe9UXJkpSzfY//zAu544lWu+tkTANzzN2+gs0Po6ergv7/uMG59aC1//4uVzRS1wsCQ4cR5k/i/HyzZfWf/410cHGzt66wEcRn7rgfm+37P8/ZZ03g++0nAVn8CY8zTwB7g+KzCtirjerqY1NddcS/E+WA7O6Tyd1JfNx0dwShuX08nvd2t65KwUZ5mOKa7k64OaVjg0V+PbS57qwdADaWYzqS+bib2dVf2d3RQcUv19XRW2lUrYDB0dXYwqa+bSX3dOglhBOKiXR4EFonIYSLSA1wGLAulWQZc4W1fAtxljDFeni4AETkUOAZ4MRfJWwnHhh+lg/wdZyQvy1AOULeKsm0RMWKxfSs5/PEUaSHNakxI1haSTXEj0b3j+eg/AdwJdALfNcasFJFrgRXGmGXAjcBNIrIa2EbpwQBwNnCViAwAw8DHjDFbijiRZhJeUiFO6SV+DUlG3kdVAi+dNfCR5R9RjQQFH8YYN6UZXInVNF3RBowU0TeGRxpOPn1jzO3A7aF9V/u2+4FLLfluAm6qU8aWp/p2bXxnjOocgZe1HMppNaqfgSx9HaZRUwyD7h3b8dZWRgZjXYMprNPD6zM1U+eXLP1ge23tq6yEGVnO4xFC3MJZtg4b1+FHAmEl3zCffswv+57WInidxLJlyVOUMI7YHugt/mxVQqjSz4Ea907Krhln2Y0Ewm+RNkoH+C358mbAum9xZVQO5EL4wR/26fvyNFnDGkPgqSQi+vLYCEOVfg6Ev4IVaeljt+KCw+WR5typUlleook6IKjzW18Zle92MJBrT9MKhNuwoJb+SEOVfg5UOq5D37QF4WrcOyPN1PeQJj6yRqLe8VvNccHZ4FIdzWekuyPbHVX6OVC19OOXwnWxiEbilM1qINebzdHEQK6JON6K+AO5fuIUadPPyYRHHtISDyLFHVX6BRDld43s5P7tkabxaWYg19Rs2/z8LYtlRU2It/qb7bLSQO7IR5V+DtQGclPmD7ycNQLn6Ve+E1C29Btbr387YOk3SI6sRAVyw/gfAs1WsOEZaKXtVr/Sih9V+nkQ6rCx3zq1du7QbI0R5+ApUX45q5kzTJKWZmg1XOJBrdQa/A8q0EDuSESVfg5UO266tysr+0LTIUacpe/9LY9SmmLpV/6OnLd0S2/XlrbjHvSt1B6MMaEvvzVRGCUTqvRzIDxlsx5tU1Qgt0hrrGxRN1oBBIO2ra7iawmvYxOFX8m2wmmG3ZGtIJPijir9AogKtkWtm9KYBayK65nhkhu3ymZ8ULHVlVHEC7kJeZodyLXta/ELrQRQpZ8DaRZcs+b3KfqipukXa+lXt0tvaDaG5OmZra2M/Auuxd3y8No7zSRcfyutqqq4oUo/A2ELM9xxI+fpE7H2Tvj3CHWUiniOiJYJ5DZNDGeqBkOMT78xojhRasO64NpIRpV+BmzWTulv/YHcovR9oR2zSVM2/RVV5umPpEAuVae+s6VfqEQOGNMgd6RSFKr0MxDueLXunSiffnLZlfVrcqZQ9w7VQG4jp/AFFLwJ/g1vtyKugVwCgdzm+/TD7bPVr7MSRJV+BpI6XrR7JyqQK9btPCky2NasTj+SFHwSrg/6VjxNDeSOLFTp54DrKptJ+cvbRSj+Yi39EiX3TuOW2rW9fRvc19rKyD+by3WefrMfbrWfS6Q1n0RKJKr0M1DbxoPv0kdP2XSclz1C3aTlQG4zFdOIWnsHx2UYGiOKE+HRakNjOEouqNLPQGQg1yFv0pezXMtJS6NezmrkFD7bkgsjbu0dbzs+kNs6kdwaS7+lHkmKC6r0MxC25MOB3KiO6dJfpaAPoxfq0/f+lh0VjZunXztTZyStvZPpw+jNfjnLsuBaq19nJYgq/QKIDOQasNl04XnPRdCol7NKvxvk009y6rc4AQXuGshttqXvuE9pXVTpZ6DWvRMMxqUO5AbKGnlD5oryKmiUEl2vRYYRhN9V4hzILVYkR0IvZ7WGUIojqvRzoGaefsykzSSf/khcT79MUYvFpcE2d7+VcVtPvzGyuOBfGRQau+yGkg+q9DOQFMiNUzbWN3JjlqrNq79XZCokSuwV3eBALpaZOiPpw+he+Nv3fzvBVTabf04S2m4FmRR3VOlnoDaQ63XcBIUa1TdqFL3/S0mppYuou1xSAf2z+jwp/dcy8/RbXBf5g6KxAd0Wcu+EA7lNH9opqVGlXwBxgdykB8PoCOQWV1dUPVVLfyS5d9IL2Oxzsn4jtwlyKNlRpZ+BaPdOOZCbrhvUBnLzp8iOGVh7p5GBXL+CH4GqJxDIdXw5qxXOM7hsCKr1Rxiq9DMQ2cYrC65F5Qt+ai6cr/yjlQJ3aWj4KpsWRtLLWeD2Yl8rrWRZO0+/ce48JR9U6Wegdj19769DXvt6+sFAbhEu04rMRbz45Q/k0rgPo9v89yPu5SyHeFDgUNPdO+HZZs13OSnpUKWfA64fRm9uIDfnAi1lC6UnVhMm7/gCubVv6bYqJjCFd2TM09cPo498VOlnIKnjRbt3kg3tkRnIDUdyi6srspqKqd94ObKS5Z4026q2vpHb4tdZCeKk9EXkQhFZJSKrReQqy/FeEbnFO75cRBZ6+88XkYdE5Anv7xtzlr8pJM7TT6ltgoHconz6xfVM/zsAUmhN4XprrfqR9Jau3wiId+/4R34toPVDLxM2XSYlFYlKX0Q6geuBi4DFwOUisjiU7IPAdmPMkcDXga94+7cAbzfGnABcAdyUl+BNJUrpJwVyjd0FVPNh9BxEbAalQO5Ilb45pP0weisQnm2mlv7IosshzenAamPM8wAicjOwFHjKl2YpcI23fSvwDRERY8wjvjQrgT4R6TXGHKhb8hz4+I8e5oEXtgX2CfB3bz2WpSfNjcy37PFXAr87vF5Z/nvLirX87YXHOMvR4etFHR1BxfmlXz3NN+9ew7VLj+OC4w4J5Pvz7z3Ayld2BfZNHdvD4PAw714yn589vJ5t+w4CMDA0DMDzW/Zy1pd/x9cuPZHXHjk9Vq7H1u7gYz98mINe3o+cezgfOufwyvH+gSHe9c172birv3L+HQIvbS3V8f6zFvKD+1/ik29axLuXzK/k++mKtVx/92pu+chZ3LNqM1/99SoMMDRs2Lb3INPH9yICR8+awA8+dEZApo/98CEefHE7APsODFb2X/2LlVz9i5VMG9dT2ffFXz7NV3/9LON7u/jhh85gzuS+2PN14d7VW/irnz7G4LDxzhm27DnI1HE9TB3bw20fey3je6O71QMvbONTNz/CwLBh1/4BZkzorVy7KPxH3v4vf+SvLziay05fUPe52PjsrY9x96rNld/Tx/dy20fPYvHVd3pywvFzJ1WOGwO/fmoj+w8O8dK2vXzo+ys4MDgcKHPz7lJ3/8+/eQMLpo2NrPsXj67nS796umbcsHn3AU6YO4lffPx1dHTEPwH/+bfP8sPlLwf29XV38r0PnMYRM8bH5s2L/3XnM/xkxTqgeu7l+3zB4ll86V0n1OQ5MDjEJd+6j+PmTOQf/+Q1hcrnovTnAmt9v9cBZ0SlMcYMishOYBolS7/MnwAP2xS+iFwJXAmwYEExjdnG/Wu2MmVcD6ctnFrZd+tDa3ls7c5Ypb9m0x4A3vaa2cyfOpbDp48D4OxFJSX66o791nxRw+BJfd381flHsX3fAKceOpVjZw/y7MZDufWhdSw9aS4/fuBlnly/s0bp37t6K0fMHM9J8ycDsGrDLh5+eQcAX77jGQDOOnwaCz35fvnYK5y9aDp3PLmBp17dlaj0V23czfod+3nHiXP4/apNPPTSdj50TvX4lj0HePrVXZx+2FQWz57IOUdNZ+H0sYzp6uSWFWv5t0fWs37Hfv79sVcCSv9b96zhxa37eHnbPh5Zu51d/QO86+R53LdmC9v2HmTulD4Gh4a5d82WGpnuXbOVGeN7WeLds3lT+hgeNnztN88CcObh05g+vocDg8OICJt29fO7Zzbx8rZ9uSj9p17dxas7+7nk1Hl0d3bwkxVrGRo27Nh3kM27D7B594FYpf/0q7t4ZWc/F58yl96uTpYcOgWAY2ZP4CPnHs7hM8bV5DlpwWQuPnkuW/Ye5MEXtvHo2h2FKf37n9/GuJ5OzjpiOms27+GBF7axcVe1y1566nwuXTKv8nvh9LGs2ribrXsP8NzGPazbvp+3njCbiX3dlTQ/fqCkhH/2yDo+9aajIut+dO0Otu87yCWnVtvK6k272bz7AE+s38nA8DC9HZ2x8j/44jaMgfMXzwJg294D3LlyI89v3tswpf/gC9sR4LxjZ1XO/fSFU3l8/Y4aI7PMzv0DPLF+J71dxYdZXZR+3YjIcZRcPhfYjhtjbgBuAFiyZEnDBouGklL8h3ceX9n3y8deSfRRGmOY1NfNN95zSmD/9PG9HDptbHTuiDdyRYS/OG9R5fekvm6uXXo81y4tyXXLgy9HLGlreP3RMyqjiu/+8YWK0i9z2enzKw+wL198Arv7B7jjyQ1uQ3Ivzd9edAzPbNgV+ebtJafOqyj1mRPGcPzcSdyyYm30dfRNrzQGJo7p5ssXn8CXfvUU/+cPL/D218xm1/4Bnnp1V21WA689YhpfWHp8YH9Z6X/6/KM4cma1c9+3Ziu/e2ZTbi6IcjnXvOM4xvd28fNH1jE0bOjr7mRgaDD5+8ne8b9/62Km+EYlvV2dfO4tx1rzjO3p4ro/PQmAM/+f3xX+kftTFkzlyxefwE9WrOWBF7YFzukLS49jTHdV8Z537CzuXLmxdC+9fZ8+fxFHzpxQSVNWfElyG1Oyyr98cdUS/uHylyojO5fzNgYOmz62UsaT63d68jXOB2UwHDlzPF+++AT+8Nxm1m3fz6fPP4qv/+ZZVm3cHZUJgHeeHG1s5oXLY2U9MN/3e563z5pGRLqAScBW7/c84OfA+40xa+oVOE/CKwYCpSmHSY2TaD9rEfOWRcRapvV7pQ5lgVtAsPKmLfEBu9pLWNoznNjJTeBln8Bc+8hztn9cvlJ3zfRXL19OwUb/NSn9DU7XTZzZFZIrLaWX3woOyocmJvjvY9yCgFXFmj0IEb63aZcZ97/7ENifWaL02Nq0eMG6qIdPve0iDS5K/0FgkYgcJiI9wGXAslCaZZQCtQCXAHcZY4yITAZ+BVxljPmvnGTOjawNIe5bt3FLzcY9LBLrzHnNE1erKc3vMMNJVi/pFVjmc87Z0g+TFMRPyp+HDHkQ9aCNovKwc2lPiXUnjZJc6ohQqg3U+nFVReqG8sOhAdM4EpW+MWYQ+ARwJ/A08BNjzEoRuVZE3uEluxGYJiKrgc8A5WmdnwCOBK4WkUe9fzNzP4uM2JR3yVJPUlbR1qZL/rREjR5q1ja35a2xnLy8DvX6rQ/b8grVNXdCdaRQgFGWWUXOlO8A2O6nQzZnwhZZ+W81EJv8oCvJla1zN2JKbHj0ErD0iWpPJmjV2nAwAqJGauU6kvCPVIL5G+jeMbXLrVRm5UV6PMt9qVDRAEefvjHmduD20L6rfdv9wKWWfF8EvlinjIVSq7Dcrnpkqpg3Um2Nwakuxznc1umgKcpyqcN+3P47ydKPk8n/4IhTBLX57Pczd5dbWTF6vxMmldgKyFZvA7RC3Hsnke6zwIOhjroTfqcto1lfobO9ZR937wpcJaWGtn4j12aRuywYllWBZHXvlPzpyeUlfZWrXBakc++ULP1aH3uUZeeqaI2nTir+cZ/1HNVZDfEdOY+HXBzhc67Ms3c9Z1O/RVf029Xh++Gq0KNGfu51R48aw3JEy2DvE81275Qt/UbGFqJob6WPPTBVbyA39zsrUe6dsFVjyxrhekkdyC3tCR4PlhmWI7EOE+zo/kBuVc5QFlvw3V93Vlkcqf2ATrgeN7Lq/MIDudS6rvwjtkhXHvVbq6VVaIME30Z2KsS6NlDjA7m116n0ItvICOSOXizX32Wp2NJ9s9+duPxxAeAknDp6ihaTNpBre/MyKXYxPBx72CmQG64j5LJ1Ju8pm2EaGcgtUoNFGRdR+L8h4WIspa07eNzdUElbdp74q7K1X2ue8giwFQK5o5koV0FyA4m2Nl1GCmmxjR6qSyUn+NvrcSMEBLAFcsuHI6w/l9kYUUo0LIM/TwzRI5t8qVxX7285kJv4EKvTBeLifqwH/zcfqlNvfZa+RZ5Svtp9NWXXMYIO1xGZJhQDqjyUGmnrG/uIJU435DDb1Zn2VvoWV0G9fre4tUhKjTr9XY0P5Nq3o/ZljSmkcqlYZn3Elh8Ti7BdyzSy4LNE86Q2kOt2Yet1gTTCEgy7d8IjPhv+l7OyymgdQWdprwlxrkZgC3i79P1GiNvWSh+yKcXYefrEuXdqLQAXbIFcm6GfLpDr7t8pB1ZrhqoOgdqk41HWXdRLZHHTZW3kbemHA7HVQG75eDq50hLnF84D22ypoBUfHkn5oiYJQWoXazvW0ncN5Pp9+iny5oXfBRm4diS7nxoxO6utlb5d4djfBA3ks3SMIrF1dJtVZbewolwvyfX6R5x2t0K0q0LE5Y3c8oMwXSwiLnVUIDcvwpZ6nFVszV+RK6N7h/weYDbKnmXwj9jcaqxfLotRZCK2o0oIv7vSQoHcqAkZjaa9lX6k6yDZQo3qtEnunaxayCWImuZB5OofrZRtlcE9f5QMaYN7ma9hXoHciP2uvuM8On2xUzZt+6IrDFjSdcpV78iRGBEavfZOtd6wHPF51L1TMDZXgUsgNsnSzz1oaCmzYjFKMF1N3gjL182747PkxeJiiquX5I4WXnsnkD/q+pp4Kzny5ay8pmyGhuFVn37weGT+yugoW/1xy3zkQ9VSLosYN2LLNZCb1K/cPZJ+aZIz5YxtNCoise3WcV5GLrS30o9wFdRjFEQtjlYqOFsTjF1gLClvirIy1xHhk3d+I9e6rxx7cCoisqyifLpV905pK20gt956iyTsunKbsul/oGUM5NpG0JkCuT6XZ6sEci37/NiMuKJoa6UP1E4WcAnk1mYLFRcRrEkZhPSXGTVMTA7k1o5kyrIk4W+INsu96t+2+PSJugrB8o1FxnKdNjmDHxOvxTZrwi9rvYQt9bCCdCVz57bEd/LEb227BP399ylpZpKL1HVP2QzVX9RDP1YGnw/S317iRr86T79BhCP9eL9d/LIN/Syg2JUfhKyaCOUb+J1CCfqVuu1041wVIjCcGMktB3It+WNkirvytvuZJzXuHQn+dp6908KB3OqCa6V9rlNv67VWrffW+I+7NdqgIZSve8+FkiFTu98+GaLxtLXSt92BuEBsNVu85RP3AkZWFeQUEEpRuKvV5C86dSA3sfzkrmgL5GZRKnl1tuhAbvl4chyjMCFywCafayC33lNzGRlmLaOxln6WQG4Jde8UjM1V4GRJxQSciniau7YDq8Vc4+5wN/Wrb/2WA4hR7h2bLG5TX6N8ZVFT7ZKmeEa7d/Ly74TLCfr0XWc01RfILdC9Q63ryimQ60sTuVieUyA3OdiZWIZfvhR586Q2tiSxXoRGytfeSj8iyl5XIJfal5gC9dUZmKo9Fp8uD8uh7I90kaG6M8XSytZ90f7kWJ9+xP48+5QtjuJ6nSsWXda6M+bLUkdUXMWW2vjGbZndO9ahd/pyWjGQG7WvSn1B8DS0tdKH7K6CLJZ+3Pz+OKwvZ8UEUQN5bX7+lKORKGvJxKgwmzsoTHnGR3wgN1yno7DhcnIL5NotyeraO27UtfZOoe4dv2zJoxf/9U0O5CYLHh/IdRidYkL3J919yYOSMRms15v1HOv6hcY81Nta6dsCuaX9yX5Zlzm3eWFzOdmsKqtENouDlIFcEWuDjXNViIOlb0iOc9jqzBbIzeemhF0QYUvf9ZN/2S39+kaiSfjlrwZy3SqspMo63LKNhE3EdlQRoTJyd+85ED3DLP5Tqo2ivZV+lOvAIUAZaenH1lffzIa438TIZC3P0WqqlO0wq6kmf6Kln7wcr5VmjdmJvm5RIxPX/HnIkAe2kuPn6Rdbd5rjkBwobQRx1yupvWsgt2BsytvF9RF74yT/NyatQdTysWDVtXmjyktj6Xv/i7T0bXW4TH0l2ioSixatzmWOpvB5+jWBwkYHcgsO+vkMk2ogN7rCwFTVOueaW0fQ4j/uUEY4U/BAQ7Dqlcq++ECuztMvGGsgNyYQG0wXvT/p6zhpcVVydv+93d+eqv7EQK7dJ++8tHJETADsVm2aQK6rBZ4GeyDX7apWfbzNG60kEf4wuoul759+G3VqLpZ8XhMPbNuNxPZGeawXoM4geBraWukDma5yqXHa88UGoky2zm73p7tZVXZL33GoXCnDHgyLWyQq7uFXyW+iXV42C93F0qt9A9nNAnclHAeqBnL9KYqj6LV3/IrXJRqSJpDrgotijCP8sl9TXs4KWJPVi6KB3BbBZhkmu3fi18XPf/gds9hZglVjd/mkdO+IPdYRH8gVB0s/uivaFE74IRSXrypHUk3pMCZYSXjhtcTrauKXkUhCXOqoA9uH0V0CuQafIRJxgi5GQE3elA/9Uv2+7ZR58yDURHyyxARyGyhf2yr9qAbo0qlMOWFU/pgYfdb+7vpmn3N5OQRy622nfuswOo2p2c40zTY3Sz8qkGsfDdXmz0eKorAHcuN8+slp6qk7zfGSHNnz5kZsIDfK9avuncKJslKdhs9xPs4C7po1AGSRwe6/j4juprD0yzJEupgcA8g15Xv/t8cEavdVLf1obAG0XLHEgSDF0soR+V1pzDz9al2Q8Eau72xc7k983RajSILHE8sgGAzOO5Dvgs39W16/Knu0MD/aV+l7f2vndbs1rvhAbkSdEf7rrHVBsHFF+e/TlGevIyl4Wl8lcXLbLmUaWYoO5Fb3uQZys620Gld33lRdVg7xEN/1TZqZVM8IOg1Jkxsagc2t5OISVku/QCKtVBeffkzHjQ3W2OpzwGplW4Ko7lM20ynB6rohYRmiK4k9zVDwL05uWyA3VmlGmIp5vZwTDuSWqVr6Dn7rOuqP8wvnQeCNVod4iP8+ucRcEiqvO1YW+XJWgwO5YWOjZDglx9I0kNsAslzkuI6b5SWmJGyBV2fLIDKQ6zYTwl9H9FIQljpipzGV/0R/GL28M/DpOZfBcY27LjFLKsLfYK3UEzHDyUZdgVzJ7wFmwx+oLosZv+BaRe37dkaU7VB/ve7R8EO5lQK5JTkifPouBk1OtK3Sj2oDNvd5Td6Mx9N+BDyQ1yGImvYD42nS2EcH9fWkkqXvMuavlSdLXXkQVY6r7zgPMYq19C37nAwEt1FOfN315S+liQqUNo44OaMONXIk0r5KPzaQm9D4iJl2l9J14kL8PP1gulpxbEFSx5kQlTLKdYZlKJdnr8Ol/Ch3icV+dCrb4q3zysnTvVNLh2VkYs1v6vMzx8WMcsEnX/m+xr6RW81W3ZfVu5Pk3nGacUaoU/gKbxBR3962TXuu5IkZNedN+yr9yhQpWyA3OX9Ux41vtCM1kCuxw05rvanKd9tXLTtGlvD9LGQ2VbaHHHjtri6nfvFqoeblrFj3TnW7XsWVdaJDmKDOb3Yg1/j2JcvSMoFcEblQRFaJyGoRucpyvFdEbvGOLxeRhd7+aSJyt4jsEZFv5Cx7XcQN052m3UXcHJeneVps00j9ASIs25V9UeWlcO+I989FBn8dkfhiBFHWnW3miEscI9LSz9G9E2fpu0w2ry+QW7R7p/blrFil77tPhXwY3X/ccXhqWwW14e4diwchboQd15fyJlHpi0gncD1wEbAYuFxEFoeSfRDYbow5Evg68BVvfz/w98Bf5yZxztTO606+6nENKCmQW48VZPtdO04J1Wedu+86VK4GcsUyBKrKEO2eiSg4uBljOdsCufEjn4hy8lL6EVo7jXJp9UBu1dJ3cO/4HuCVfVFlZwjEB/MnE9a3TQnkRrjwSl0oKpBbbtvFa/0uhzSnA6uNMc8DiMjNwFLgKV+apcA13vatwDdERIwxe4E/isiR+YlsZ9veg9z9zCYAFs+ZyL6Dg7y4ZZ817TmLpvP0ht3WYwbDf6zcwPod+5k7uY99Bwf57dObGBgcrqR5Zcf+WFm27DnIssde4YLFs3jope1s2NkPwMZd/fR2d2Y5PV7aupfbHloHwNCwYf/AUKZyyqzetKdSXmeH8MZjZzJxTDePr9vBcxv3APDUK7sCebbuPVjJs37HfvodZXhlx/5KPoBXveux/IVtbNzZz5ie6Gvyq8dfZcrYHoaNYd/B7Oe84qVtAMyd0sf67fb795p5k1g0a4L12L1rtvDqjn5Wb9oT2B+Orfznc5sr9xvgzCOmMXdyHwCbdvfzi0dfYXCoPg20cVd/4HqefthU5k8dC8DvV21i656DNXm6OoU3HjOTCWO6a47t7h/grmc2MThkrAp+xYvbE2W659nN/H7V5tg0K17cHpAboLurg/OPncXBoWF+/dRGjpppv/4Ad67cwIzxvYF95yyazsyJYxgcGuY3T21kd/+gNe+ja3dw2iu7WDxnIpt29/PH57ZYHwSHThvLkoVTa/bfu3pLpd366eiANxw9k4ODw/zhuS0A7DtYlSFcxcCQqVyDcb1dvGbeJO5bs5WXttl1VRG4KP25wFrf73XAGVFpjDGDIrITmAZscRFCRK4ErgRYsGCBS5Ya1m7bx1/99DEAjjlkAmu37WNvhJK4/PT5/PiB0ilNDzWiGRN6eXbjHr5yxzP878tP5pePvcpnb3u8poxzFk23lj1jQi/3Pb+Vv/zxI3zxncfz9794MtC4zjtmZupzmz6hlxUvbWfFS7Wdzy//jAk9gWMiMHVcTzgLMyb08ofntlQaKcDn33IMV557BB+56aFA4542rgcRYfr4Xn6/anPlGvuJqmPr3pLyWbN5rzXfj5a/DMCbji1dk5MWTAbgqEMmVB6yX/zV07HnXOa8Y2byu2c20d0ZHLxO6uump7ODHz+wtnLPozhx/mR+8fHX1ezff3CI9/7r8srUxcOnj6scmzO5j1d29nPUIRP4/bOb+c49zwfyvv3EOfzL5ScD8M2717BhVz/zpvTFyhHHjPG9PPLyjsD1fPNxs/jO+5awYWc/f/69ByPz/s+3L+YDrzusZv8tD64NXOfy9Z02vnRfb1kRfd3Kab75+zWVfT1dwXtw0fGHcMeTG3hi/U5rO/jqpSeyefcBjIHpoTZ8mO9a/+Mdz9Tkfe+ZC/jiO0/ggRe38dEfPhyQH2BsTxdjezr5+SPrWfnKTn796f/G9Xet5vv3vWQ9n77uTp7+hwsD+/YdHOTPblweOVr45HmL2Lb3IDfdXy2zfB6Xn76Ab/1+DWN7Opk+oZeDQ8OBa3D0rAms2lg1QMvXs0hclH7hGGNuAG4AWLJkSSYz6OhDJvCff/MGvvDvK3l+y176B4d535mH8uFzDg+ku/hb91YsxveeuYBLT50XOH7jFadx2pd+y4HBUpp+7++/ffx1TB1bvSEzJ9YqHoCvvftE3nfWoVz67fvYc2AQY+Avz1vEJaeU6pk1yZ4vjps/fCabdx8AShbrZ35SajT3fe6NzJ5UVSCnHjqVBz5/HkPGMDBoGNvbaVWQ//4XZ1eswYHhYc772j0cGCgp2f6BId518lw+/aajAJgyrmQZfvniE/jLNy4C4P7nt/LZ2x5n+vhe7vjkOcyYUFvHzz72WrbsPsjY3k72Hah9+PZ2d1TqLF+Tt71mDmcfOZ3J3nW+73NvZGCw1BzO/V93A3DjFUt4o+XB+e33ncq+g0M1Sn/y2B6Wf/48dvcPcvG37mXLngOcvnAqX730xEC6z//8CbbsOVBTbvkaDRv42OuP4LLTFgQ65g8+dAZb9x5kzqQxfPicwwPn+uffe4ADvtFQeWT0H58611qPC//ynpPZuLMq54f/7woOeA/Icpv9/FuO4cLjZlfS7B8Y4s3//J+VdGHK+3/7mXPp7eqsPJSOnT2RB//uTew/OERXpzB5bO0o4ZhDSmk+//Mn+M1TGzln0fSae3D9e07hxa176eoI7t+y9wAXf/NeDgwOVa7NjVecFkhz2sKpPHr1+RwYHK60lzKXfPveyr7y32+/91TOXzyrkqavp5P7rjqPv/rpYzy3qaRc+weGmT6+h599NPiA/9c/Ps8P7q99GAwMlmJPn3jDkbx7yfzAsfOu+z0HBofpHxhixoRebvsfrwWoXMPPvvloPv2mo+jp6uAj5x7OW0+YjTHwyNrtfPLmR9m5f4C5k/v48YfPZExPBzMnjKmpP29clP56wH+m87x9tjTrRKQLmARszUVCR8Z0d7Jg2ljG9nZ5AULDpL5uFkwbG0jX1VFd/XHu5LE1Pu8x3Z3MndxXeaqX/86b0mdVoGG6OzuY4w3ny0PlqWNr5UhDX09nJf/a7dVhoF/hl5k5MbnRjOvtYlxv6dYPDJU6S/lJa4AJY7pq5O3u7KjsW7Ol5OIY39tpVfhQsrAWTPOa1/hEkSpM9j1Ybec3fXyvNU7R3dnBpD57iGrKuB6mjOuhy3ttdozvepbpi3ExldvA1HE9NfnK7aUsm/9ce7o6AsN7Y2DWxF7G92a3tXq7grL3dnfUtNXp43sDacruhrj3RwDmTx1Lb1fwOkTd33Ca8jlN7Kt9MHR0CIfPqG0EY7o7KnKZyr7a++BvE366OsTXbktbh0waQ2dHsH1MGtvN+N7O6nXC0NXRUXMvJ/V1299T8PZOsdz/8hRvA3R3iPV4T1d1CmzZDbfecxEPG8OYzlpZisRl9s6DwCIROUxEeoDLgGWhNMuAK7ztS4C7TCM/SukjEMSJCLhlWa0x1RRE728Rb9nlHebJMt2yOZPgvLrrDIJC/VNM6yHvQJ3LNF3XOut9fyBKnsRMWeuMmKTgkjbLVOakdpOmn/sD/43+oE6iyeH56D8B3Al0At81xqwUkWuBFcaYZcCNwE0ishrYRunBAICIvAhMBHpE5J3ABcaYpyiI8rSoqJdoXKe8VSyIygMi/Q2N+l0XObeP8NeRjEk+18qCXA1urJCTYoo0BiIyZnx4h6fGFvLWpUhglAa11yhp/ZlcFvvKkDewdEVGGzE8yomcSk31/CNfAouYyhxXdnmKdlrx/YZho3uR0zjTGHM7cHto39W+7X7g0oi8C+uQLzXlmYWlubL2KYxJ31r1d9ZqR0ojgzfVbTh5imFairIUqx3CYVpdOU8TBnP1Wfrew8p2LGaqrctUUWt9Xu5KOaZOxRpRR/k+JI1gI907vrKyy5HeEBCf5ovoron5/YrcL4dFwECfTjNKiLs+5QXwYt/St+WrGFuN1/qj8o3cJItq2IsHxVkFhBpTFioLVTXBInYlPI+9KT65FiDO0q/HEg5a+sX276z3Lk83ZJYSDHVYvCnabXhUEJ3OxP62ps948V2+SpY3o07p+630qEY0XLH07SlEapVguqd4sJ48Ker5YXwbSXW08DPMicjP+UWkz3oXw29gurjOMtWRoMxsL1D5ycPtVImXZIiTld/mTe8+s7lho+qqHoj++llVHj9VHWDzHFQD0dl0ROPdO6NP6ZPsg0tS5PYgT4phq/c3j2FzVNm5lWcd5ib49JsYyi0skOtQbmr3TihDET79YBX2+JN7ILd+OVK5Qf0ujgz1+5cIT7qyLnG2xHZvzVMeqaRbQbecctjU91GdLIw6pe+/7pErTDr0vbAFUc/0nTzvaWENxNd5XC39ERfIjbFG474xkCWYn1RmXthWhQ1LmbQURR6B3Cx5/VmyXKbIwGpEXcY3fI/LGxYlTgXY1vlxoRrPUEu/bgSpum9Sdu64NOmGrV4gNym4lIEi9Gx4mJxURfWZNsICuXH2qERbi1lDM2H3jovrLC3+kW2U8g6PPMPEuS/c5ZDUZfiVbKZALv52G5TDVlclbUS6qHZdeahGBH8r7p0UspdTZznvehl1Sh+SF4gaTrDcgkqwfjla3QceUBxNlaR5CESefJbBXjhvufgi2oLrvYuekprfXc96jaIWKXPJ6205p00MzCbuCB82mS+hBnJzoKTUve2INEnHg0owvb/RH6TJm6KeH4E5zK6mfhPIxacfUUa0JZztRvotUUjv93WqQ5KnLVZ855FTUvORw6vcPY/P2s3ynQH/kuPJ8/SrbrCoh29iINdesC+Qm36UM1xAm0hi9Cl9EgK5OAzXI+b3p5EBsj0wEssuxL1jVxKR6ZsZyK3Hpx/6GzjmFMnN4N/xUYSlb5uVEuXeiS2nTrmqQfJUWh+oJ5BbzZt2ckaWNhClT8DNLWrLN1yAyy+J0af0feN0+xQrh5ezoNaCSCVDMCKU703Nv4WERzaJhv6ItfTLfmfLMWKG/RndO+EyC3kPRyw+/XCSCAs2X7mir21kDn/aLMrP4oZ1moZt4hV8zfeoY8ou6xPjyeMselTUuAGMPqWPxLpV/AouDlcLIqoOSH4fIAvFB3JJbLzNDFHUU3fV0rd13mICuUUTdiHZ6q26d+xkmSNfI0eG7CGdX1f+ZDmCy1XE5a1178RMDPHty2bp65TNXIgNjohbILdMPm/kZi+jEZRmK5W22zmQmzilsa5AY3q/r3sltXUlChPanZdUWcopBXIz+rZDsbekekp/07XyxDd4fXKkpRn9bdQpfRHfmjcRw/ikWTVhd0dpX7YgTbm8vCjk+SG+TuMwi6IZ8/OrddeTOboM23z3MnGWXlJ1gc89pvT7OtXhu3fVuuy1xI1k6vbp++RxzuMLMDtNILDkrw1iR6Ut1VT5v1U3SKCsMnFXtawr0o6WgoHcxjIqlX71JkU0/sQGYh/+O8sQ0XjyoBEKN6mK5k5BzV55FsVUD+F7ldbv61SHP5AbN4Gh4HPOEsgtp8w6JTZonAXliKor6ndc3rjjlfX0U46W/DpCA7l1U9X60TeJ6AQELbRMijvCp5oHRbQPnxHkFsgtQAZXcgnk2nz6uLh3UtYXLrOAQK4/QOmvN1EWv1gOo7tEOeoI5Bqyub6Cwdn4UU7gOkUEcisPoZpAbvRov2Lpp1TegViAKv36CLx8ZTuOm0+vpiNluKGFuHcKaCDhQG4rW/o5eHciG0bC5J1MgdygTz//oV9NEJ6okWqC+6pe906G/H5rN9MyDFiWoIi09INf2Yqz9GvcOzHPk/L1L5WZ7SLqPP0ciJ29Iw4vZ1ktiCxyZPMFN5o0C1eNVuI6Xpyll0StUsq/Mbi20eg3cvMzTLKenssIs16q3x1IkmV0M+qUvhCvbJPW5imncf44Q4QM4Hu45Niai7AKwg+55Dqa9xSra32Yit/Zfix6wbVyopT1EfpyVhHuHWq/Exs5wo0oI89AbpozrLp3jNNUYVv+pHcUAmmppo3SDaXjYfdOXNmi7p1mkxTI9TeUuK/s2PK5y1AdtsbWk4FC3Dsp6xi57p2y3znan5tr3aEMRQTtbNOLswRy6/bpx7z4lkTWmAlQ+8CLidMFf9t1QxzREzzSO+4k02MyH0af0k8I5IJbpL9mVkAqGcI1tT6+GFcL2/n1Kc04Sx/yv1th67oev28cLjNYwqOOYP78zjxVP5Hw73TXJvBZU4cZeYG3zmOqCl+NxBFUxdJPP8qp/VE8o0/pS/LKdU6B3Eracrnpb2jSZxmzUEwgN9R5Eipp6jz9HJRmlCWcOHsnw+wSvwYpagpvta3GxB4k2h7NYwRSXyDXZHrweLm9MpLlSPLph0fo1XzRZVfbTcq1d/zunRT58mDUKX1IWlpZEmfV+DtrPTMumrFsahZKpzsyZC2KIj6M7s9bqaeIHu7azmJmJzU/kFu8wWtCfxMTuqYfYYw6pV8K5HrbEb7b5A+ju7/pF5WfBDmyUsj0LikPUd2U28h173h+54hyky39lPWFA7npsjvWYVFmdkM/OpCb0jVhl0MCf53ylAO5ZfdI2jr9gdyQHNa6fKPZTPP0o9xmGd4oDvj0NZBbH4HPl1mPOzYQh33RMpT+jpillVPW0epTUKMoi531jess8/T95KFc4+qIM1CSA7n5yJFO8ZWoxiLSu89qXFsxhpyt7nB5sfXFTAxJO1JR905BRAZyE8x3qwWRaVpC+ixJFKlwXWcbjcYPo0P8lMas9YVsxgKmbPpHKNEzWGK/AZxDQ5XQX6c8Ph96lnn6aT+MXk1jD+RWHkIR7p2oPAbXqc61dZXK1UBuXSQ9QeOG8X6q/ShDgMmruIiXs4qZp+9bE5zWtvTzmKdvv4TJs1vSXvuwoi0ukJvsikx0XzUlkOvVX56nn6HO0PMuYUZe9TrZR/PiL6pCnGzlSRBq6bcwgZezYtLk8YZqER9GL4Jy52nGh85bhfJ8axuuD8O4vOXtIh6Y4VGpiyxh8hIry0M5i9L05wW30YprILe2H8TnqOcbuc1g1Cn9YIDE7oNL+2H0LAE8KMjSL8inX+545d+tSj2yxQUbg26SIFk7tM26zvv+2eSOdEPEjGTq/4hK+vy18Yi0Iyl3OQIu2whXTDUWF8R1ym5W6TWQWyeBYVOMD6687VRmRhmKUKJFtI/ab+QmpS9ACEdy8elbh/Yu+es78aI+jF4t39tnVWgJCqte906GcsLulNQyBN5RCMoRVZcva22apOrsIkDZYEpxAkH3jvr060Iitqs7q2Z8pP9Pwm/vZXyGZ5zqF1t2UZa+b4jalh9GR2IG8dlmYQWW8aYo947l5axIYyfafVW3WOUHapaSjCFLkDvgpy/vi/PpU00bqRuoHRHFBnJ9rtE08kvkj+IZfUpfon5Ucfpylredbf5w0L2TL8W0kPIMCicJRqilX86cOKurZn/G6kKZC1lwTSyKL0KYvN1XweLjr21kvorSzJa3jMt6WklvnZf3pPowui/+l2WU46+3UYw6pZ+EEL/0coVcArkjwUse7HjtSu30yip5BnKLJOv9y+PD6PVQfjkra17IN5Bb+0ZuUiB3ZPUdJ6UvIheKyCoRWS0iV1mO94rILd7x5SKy0Hfsc97+VSLy5hxlj5K1um097v+4SUzQx9vOFsgtUcjSyoX0zeB0xZb26eeQN/uXs1K6+ULWtcljkZtwHVgUX5R7J2YkU69USe9ARObDcy2SPt4RcNkkuWypduooV0xyINeeJ8t7Bg4OicJIVPoi0glcD1wELAYuF5HFoWQfBLYbY44Evg58xcu7GLgMOA64EPimV15hJF1MfyeJc+8Ef6fv7PjrSZU7oewcy6qUGSq0lV/Oqqfq+EBucsFZH/5J++rB3sYtD7UWDOSW0vuMtDryhuWoTRv/u5Q3qd3H78u6ymaj+1OXQ5rTgdXGmOcBRORmYCnwlC/NUuAab/tW4BtSugJLgZuNMQeAF0RktVfeffmIbyHhYooIG3f1xxchwjMbdnP+dfewZc+B1D21fPPvW7Ml8DsPihiGC/Crx1/hgRe2enUkyVD6O3VcT+6yJJFLIDeiiF39A5x/3T01+w8MDgfyO9cnwqqNuytlrtu+n2NmT0hZSkIdCC9s2cv5193DvoNDXr02WeDfHn2Fe9dsrTm2cVc/PV312WLVB2p6a/1Hy19mcNgwoddFHQXzPvzSDs6/7h527B+o7oxIe3BomPOvu4d12/dzrOU+lEW/7Ib76eqoFlS5/xHGwt3PbGJw2HDcnIkpZM/+sKsXl6s8F1jr+70OOCMqjTFmUER2AtO8/feH8s4NVyAiVwJXAixYsMBVditvPWE267bvp6tDOGfR9Jrj7z/rUO6c2EtvVydnHj7NWsZlp81nTHdpELRo1niOPcT9Zpb5y/MWsXrTbsZ0d3LKgsmp80cxf0of7z1zAUdnkCmKj/y3I3jopW0ALJ4zifOOmRmbfuG0cbz3zAX86ZL67lUavvjO41mzeQ/Tx2d/0Fzx2oXMnrSRpSfVNEHefuIcNu8+EOm/XXLoFE5dOCVVfZefvqDSjqDUli5YfEg6oRP409Pm0+Ebr7/2iGlW5fOx1x/Bo2t3WMtYNGs8pyxId25hLjp+Nmu37ectJ8xOle8v3riIVRt3AXDawqmp8r7vrEO5c+WGyu9ZE8cwfVxvpHwvbduHMSbyPpx95HSWnjSHgaHhmmNLDp3CqYfWyvfBsw/jXs+4e/Nx7vd27pQ+3nfmoWzde4C3vWaOc748kKS3MEXkEuBCY8yHvN/vA84wxnzCl+ZJL8067/caSg+Ga4D7jTE/8PbfCNxhjLk1qr4lS5aYFStW1HVSiqIo7YaIPGSMWZKUziWQux6Y7/s9z9tnTSMiXcAkYKtjXkVRFKVBuCj9B4FFInKYiPRQCswuC6VZBlzhbV8C3GVKQ4hlwGXe7J7DgEXAA/mIriiKoqQl0afv+eg/AdwJdALfNcasFJFrgRXGmGXAjcBNXqB2G6UHA166n1AK+g4CHzfGDBV0LoqiKEoCiT79RqM+fUVRlPTk6dNXFEVRRgmq9BVFUdoIVfqKoihthCp9RVGUNqLlArkishl4qY4ipgNbchJnJKHn3V7oebcXLud9qDFmRlJBLaf060VEVrhEsEcbet7thZ53e5Hneat7R1EUpY1Qpa8oitJGjEalf0OzBWgSet7thZ53e5HbeY86n76iKIoSzWi09BVFUZQIVOkriqK0EaNG6Sd9vH2kISLfFZFN3gdqyvumishvROQ57+8Ub7+IyP/2zv1xETnFl+cKL/1zInKFra5WQkTmi8jdIvKUiKwUkU96+0f1uYvIGBF5QEQe8877C97+w0RkuXd+t3jLm+MtV36Lt3+5iCz0lfU5b/8qEXlzk04pFSLSKSKPiMgvvd+j/rxF5EUReUJEHhWRFd6+4tu5MWbE/6O05PMa4HCgB3gMWNxsueo8p3OBU4Anffv+CbjK274K+Iq3/RbgDkqfAj0TWO7tnwo87/2d4m1Pafa5JZz3bOAUb3sC8CyweLSfuyf/eG+7G1junc9PgMu8/d8GPuptfwz4trd9GXCLt73Ya/+9wGFev+hs9vk5nP9ngB8Bv/R+j/rzBl4Epof2Fd7Om37iOV28s4A7fb8/B3yu2XLlcF4LQ0p/FTDb254NrPK2vwNcHk4HXA58x7c/kG4k/AN+AZzfTucOjAUepvTJ0S1Al7e/0s4pfd/iLG+7y0sn4bbvT9eq/yh9Ue93wBuBX3rn0Q7nbVP6hbfz0eLesX28vfbr1yOfWcaYV73tDcAsbzvq/Ef0dfGG7idTsnpH/bl7Lo5HgU3AbyhZqzuMMYNeEv85VM7PO74TmMYIPG/gn4HPAuUvkk+jPc7bAL8WkYdE5EpvX+HtPPHLWUprYowxIjJq59uKyHjgNuBTxphdIlI5NlrP3ZS+KneSiEwGfg4c01yJikdE3gZsMsY8JCKvb7I4jeZsY8x6EZkJ/EZEnvEfLKqdjxZLv10+wL5RRGYDeH83efujzn9EXhcR6aak8H9ojPmZt7stzh3AGLMDuJuSW2OyiJSNM/85VM7POz4J2MrIO+/XAe8QkReBmym5eP5fRv95Y4xZ7/3dROkhfzoNaOejRem7fLx9NOD/AP0VlPzd5f3v9yL8ZwI7vSHincAFIjLFmwVwgbevZZGSSX8j8LQx5jrfoVF97iIyw7PwEZE+SnGMpykp/0u8ZOHzLl+PS4C7TMmpuwy4zJvlchiwCHigISeRAWPM54wx84wxCyn127uMMX/GKD9vERknIhPK25Ta55M0op03O5iRY1DkLZRmeqwB/q7Z8uRwPj8GXgUGKPnpPkjJd/k74Dngt8BUL60A13vn/gSwxFfOfwdWe/8+0Ozzcjjvsyn5Oh8HHvX+vWW0nzvwGuAR77yfBK729h9OSXmtBn4K9Hr7x3i/V3vHD/eV9Xfe9VgFXNTsc0txDV5PdfbOqD5v7/we8/6tLOusRrRzXYZBURSljRgt7h1FURTFAVX6iqIobYQqfUVRlDZClb6iKEoboUpfURSljVClryiK0kao0lcURWkj/n872kIuY9fH+QAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "## DO NOT CHANGE THIS CELL\n",
    "### RANDOM SAMPLING EXAMPLE\n",
    "num_episodes=5000; #number of episodes you want to try\n",
    "episode_max_length=100; #you can explicitly end the epsiode before terminal state is reached\n",
    "\n",
    "env.reset()\n",
    "#env.render()\n",
    "#execute in episodes\n",
    "for i in range(num_episodes):\n",
    "    d = False #not done\n",
    "    #rFinal=0\n",
    "    for t in range(episode_max_length):\n",
    "        \n",
    "        ################ Random action policy ###########################\n",
    "        #play random action \n",
    "        a = env.action_space.sample()\n",
    "        #get new state, reward, done\n",
    "        s, r, d, _ = env.step(a)\n",
    "        #################################################################\n",
    "        \n",
    "        \n",
    "        #break if done, reached terminal state \n",
    "        if d == True:\n",
    "            break\n",
    "    #reset the environment for the next episode\n",
    "    s = env.reset()\n",
    "\n",
    "rm=movingaverage(rList, 100)\n",
    "plot(rm)\n",
    "rEpisode=0\n",
    "rList=[]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Implement tabular Q-learning (*YOU SHOULD ONLY CHANGE THE CELL BELOW*)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "## CHANGE THIS CELL BY YOURSELF\n",
    "###ALWAYS DO RUN ALL CELLS, otherwise some structures may not get initialized properly\n",
    "rEpisode=0\n",
    "rList=[]\n",
    "\n",
    "#initialize episodic structure\n",
    "num_episodes=5000;\n",
    "episode_max_length=100;\n",
    "\n",
    "#initialize discount factor, learning rate\n",
    "########Hyperparameter tuning###########\n",
    "gamma=0\n",
    "learnRate=0\n",
    "\n",
    "## Epsilon greedy\n",
    "eps = 0.9\n",
    "eps_decay = 0.99\n",
    "##\n",
    "\n",
    "#create Q table\n",
    "Q=np.zeros([env.observation_space.n,env.action_space.n]) #matrix Q[s,a]\n",
    "\n",
    "#execute in episodes\n",
    "for i in range(num_episodes):\n",
    "    \n",
    "    #reset the environment at the beginning of an episode\n",
    "    s = env.reset()\n",
    "    d = False #not done\n",
    "    \n",
    "    for t in range(episode_max_length):\n",
    "        \n",
    "        ###########SELCT ACTION a for state s using Q-values ##################\n",
    "        # e-greedy action\n",
    "        ################# YOUR CODE HERE #####################\n",
    "        \n",
    "        \n",
    "        \n",
    "        ######################################################\n",
    "        \n",
    "        #get new state, reward, done\n",
    "        s1, r, d, _ = env.step(a)\n",
    "        \n",
    "        ##### update Q(s,a) ############\n",
    "        ## Q-learning update: Bellman Optimal Equation update\n",
    "        ################# YOUR CODE HERE #####################\n",
    "        \n",
    "        \n",
    "        ######################################################\n",
    "        \n",
    "        #break if done, reached terminal state \n",
    "        if d == True:\n",
    "            break\n",
    "        s=s1\n",
    "    \n",
    "    eps = eps * eps_decay"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "plot moving average over 100 episodes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Solved after 838 episodes.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAAA8JElEQVR4nO2deXxU5dXHf2eWTPYEkhCWBBIggIBsRkBFBVdcil2s4tLaams3rdW+bbFVq6hVu1hra19rW+1bW6t2Uyq0KigiVhGUTfZFlrCGJSQkmcksz/vHXebOnXtn7mTWzD3fz4cPd555Zua5kzvnnuesJIQAwzAMk184sr0AhmEYJvWwcGcYhslDWLgzDMPkISzcGYZh8hAW7gzDMHmIK1sfXF1dLRoaGrL18QzDMH2SDz744IgQoibevKwJ94aGBqxatSpbH88wDNMnIaLdVuaxWYZhGCYPYeHOMAyTh7BwZxiGyUNYuDMMw+QhLNwZhmHyEBbuDMMweQgLd4ZhmDyEhTvDMFH8c3ULOrz+bC+DSQIW7gzDRLC+5QRuf2Et7nrpo2wvhUkCFu6MrWjr6sGJLtZIY3GiW/p+th46meWVMMnAwp2xFZPmv46J81/L9jJymqVbDgMANh1oVwU90/dg4c4wTARuV1gsHO/syeJKmGRg4c7YhhU7j8adc9dL6/H08o8zsJrcxesPqsch7rHcZ8laVUiGyTRf/mP8KqR/em8PAODGGY3pXk7O4vWH1GN/kIV7X4U19xzkcLsXf3jnYwjWmlJKuzegHv/89a1o14X6dfeENVY7f/c+jeb++BvbsrgSJhlYuOcgDy7ahHv/tRHbD3O0Qio5dUiFevyLJdvwZ1lLV3h/1zH1+K2trRlbV67hDYSF+8J1B7K4EiYZWLjnIGv2tgGwx5a4qyeATl8AQggcOemz/LrthztwqN2b0GdVlxZEPN7RehIb97erj/e3davHJ30BJIMQAq0d1s/HCh8f6cTBE14EgqH4k5Pg6MlIJ2owlP/XYT5iSbgT0Wwi2kJE24lonsHzQ4noTSJaTUTriOjS1C/VPuw+2gUA8Kf5R5wLTJ7/Osb98FU8tWwnmh9YjL3HuuK+prXDhwseXYZpP1qS0Gd9fKQz4vHfPmjBpY+/jW2HOgAAd/5jvfpcsgLtrx+04PQHF+OjfSeSeh+FY509mPXTpZj+0BLc/fKGlLynGfq/QapvUkxmiCvcicgJ4AkAlwAYC+AaIhqrm3YXgBeFEJMBzAXw61Qv1I5ooxbyFV9AuoG9sVmKrbYi3I92hoVNIrbxQrcTQyqLosYPtUcLr2SjRP67/QgAYMvBjqTeR0G7q/nX2v0peU8zigqcOGN4FS4eVwsAHOveR7GiuU8FsF0IsVMI0QPgeQBX6OYIAOXycQWA9F59ecY/PmxBw7yFmPHIG3j435vVcW8gM5p7IBjCnF8tx5ty8ko2UMwj8RTmrp4AZj/2tvp4bYs1zfjbL67F5oMd6OyJNrcs+ugAOnVmGH8gOeHucko/rXUtbUm9j8J3/rpWPXY6KCXvaYYvEMKgikJcO20YAGD30c44r8g/9h7rQsO8hWiYtxCLNx7K9nJ6hRXhPgTAXs3jFnlMy70ArieiFgCLANxq9EZEdDMRrSKiVa2t9nVY6bnjRemH23K8G0++tUMd10ZvpJO2bj/WtZzAt19cG39ymlA0+EJ37Ety3/HuiMebDrSbzIzk7x+2AADaDEoPdHgDEfZ2APDEWUc8Rg4oBSBpPanA6k0sFfgCIXjcDlQWuQFkTsnIJbQ7rm8+vzqLK+k9qXKoXgPgD0KIOgCXAniWiKLeWwjxlBCiWQjRXFNTk6KPzl/ufjkzhZsCsuP2WGcPGuYtTJkpIR7vyKYLAOiR/QtXPvkudrYaRwm1dvhw4c+XRYzd+Y/1STs/vf6gGtt935xxAIA3Nye3i3l7m6S8pELH1pueqkoKTGaa89LqfWiYt1BdVyy8/iA8LicGlHsARN5A9xztwreeX43thzNzjWQLbcRQV4aUrFRjRbjvA1CveVwnj2m5CcCLACCEeBdAIYDqVCzQzrR2+NIeGQEgKkrl4seWmcxMLdf9boXh+Hk/e8tk/nuG4+/tiJ95qnDP5WNx9+WRLiOvP6j+mOv6STb5ZItmvbNdWlMqNHfFwZ4M33phjfT/82vizlU096oSSbif1OQHzP7FMry0Zj8ueDQz10i2yNSuOZ1YEe4rATQRUSMRFUBymC7QzdkD4HwAIKJTIAl3trtYIBTHyJysVmoFX6BvXMiKwL1l1kjsevgydbzLguN5WmN/TG3sjxtnNOKmGY2q9jusqhg+f0h1XlcUuTF73EAEQiEIIXDwRGLhlkCkpu3zJ39z1l8DrQmEjAJSUpzC0c6emE7oUEigJxBCocuJApcDQyqLsHRreBfTV7VYK3R4/WoN+w37rZn7FPYe68LhjsSvlXQSV7gLIQIAbgHwKoBNkKJiNhDRfCKaI0/7NoAvE9FaAH8B8AVh5xS/BFgSZ/ufCRNJX/vBNtWWRjz2Wlh/TzAEj6Yg1icmDgYA1JR64A0EVU2t0O1EodsBrz+Ep9/ZhekPLUnYBLFOYx9PxY3z0de3Rjzu8AYSihKaqgsZ/cv7e01mhs1jis9hX1s39h7rxj9Xt1j+vL7Kqfe+hlPvlSqG/uG/uyy/ricQwtk/fhNTH0wsNDfdWLK5CyEWCSFGCSFGCCEelMfuEUIskI83CiHOEkJMFEJMEkJwTVWLmMUQ/2LuJABAdwbCIRXBNkcWeJlkQJknamxifaXh3MbqEhQ4HfjEBGmdr91+DgBr31FPIIQCZ/hyn3fJGLx++zmoLvXIZpmwQ9fjcsIXCOJd2dyzozWxaBGtmcubAs1dCRO9urkeZ46oAhB2QMfDaGeo9XXoUXYwhS4nAKg3xDc322cjrs9xmGRyPSrk6s6XM1SzzEpNyruWxuoSAKkRDvH44QIpKeab5zepY0vTHBZ5yt3/AQDccGYDThlUHvHc2r1thglcPYEQPjFxMBxyKODQ/sUApPU3zFsYM2N1w/52uJxh92ah24mm2jLsPtaFrYdOYplcbsDjkjT3Q+0+LN4khcD96b3dCZ2b9mbznw0HE3qtlnUtbTjr4TfUxxPqK3DxuIEAgJbj1uzwx7uiS/YuXH/ANEHugGyGUsr+njasHwBgwdr9mPHIGxFztWvr62h9WyO+v0g9rix2Y83eNtNr65t/WY3bNH6MWDuqXUc60TBvIUZ8f1FGfGks3LMMyfLm2Zum4ivnDlfHSzxSwc5MaAXHOntQ4HSgsboE/Yql8Ld7F6Q3C1IRgJdPGIQaWXsfLt/QAOM64l5/MCJUstDtjNDG395mrJEqPzgHRceuKJEgiiCvKfPA43ZGzAkkWAYiVWauDfvbsU8Tojn39KGolP8+hy1mjWoTtH7zudPUY7PEJKX0QP9iySfxpOY1Lbow1H268NG+jNH3MaDMo4bOKiVB9CxYu1/dWQFhs5bZXEDaGbRlIDGMhXuW+XD3cYyuLcPZTTW485JT1PFCWcCkO0vV6w/CFwjhqtPr4HQQzh0lhajuOtqVkfIHw6pKUFMqCfevnjtCHTe6+CXhHil4Z48fqB6bxbwrP7jxmsJhetq6/CCSzBD6837XQh14LXo/SW/dT+9qooCGVBbB6SAMq5JugA8t2mzJ2a4N6VO0fgD447vGuxHlpqvsisoL3THfP18SnIzMXHOnDlWPjeoeGSleZjfNDftP4OeLw74Tff2edMDCPcsEQiJiG3/emAG4aUYjCuVtcbrNMi+vkaJaX1wpOcw+PaVOfe5ZEwGQLPps0BtnNKCkwImZo2swsU4SwEYXvzcQikpy0kazmP2wvD3Sd1ikuzEAYd8GINmZiQjPvLMr+j0SuMnq58bS5mKxQFNmQImQUc5h/b4TeMVCGQIlWmfMwLKI8ceXGJfyVa7FooLw9zytsX/EnDJPuA3EuT9ZGncNfYEeA+H+2dPqcMuskabPP718V9TYriPG5rKv/elDaO/xeyyU2UgWFu5Z5MCJbrQc78bZTeGUgKe/cDruvnysqqEaZVTqOXLSh1Umtvt4KBeZIoDOGRVOLjMLuev0BRKq4Kjg9QdxqN2rCpD5V0gJQ+MGV2DD/NkYUF6Iu+QYdK8/iAMnpG3/jtaTWLXrGIIhESWgaysK1WOzYl9K3faigmjhfsWkcLJ1rKxU/Q0pFr5ACEMqi3CPfC6piJlWhEux5hziOZJ7AiHskjXrH3361KjnjZz5ijmsqCAswL9wZgMAYERNCXY9fBnW33dxxGv6eg2kdq8/yvf13Jenob5/MW49XxLuR076sPdYV8Qu7KjBb8DrD+Jwhzfqb64X5l0GZTBSDQv3LOEPhnDGQ5JDykgmKcLdyh2++YHFMTM7Y6Fou5dPGBT1nFkRr8sefxvNDyxO+LO+/ucPMe1HS6IiMrQoNvTv/n0dznjoDXT1BHD+z97ClU++CyDaznva0Er12EzI/GudpOFWFMU2MSg3Ur2mavS5sejW+QZSWZe/XHMO8bJf73ppvVrp0ui7Pv3B6L/hhv1SGGd5YVi4K9ei2U1qjOwc76tMuPc1fOdv6yLGXA7p7+eRv7d/rz+Is3/8ZoQ5q8zAZNXtD2Lqg0twrUnCnUKHl4V73qK18U2qj7YFOx2EApcjoRonvXHS1PeTbKsPGWh2pR7jLoy7epkxqTieFCFhdG4FsjlK0Sr1PwL9D+qGMxvw9ndn4dQhFaaarGKauGhsreHz+t3Ac1+ejkc+cyoW3HKWaraxGnoISHH3RQVOTJT/rh1JJKKdN2ZAxON4NygtWkefcrN58vopMV/jdBAKnI6I71mJMhpQHt4l3XXZKVGvzVfKC12qM/n9j8Mafo1BGK9yba/e02b6fs99aZrptZhKWLinmc8//T5++uqWqHGtDa9fsXGtkNpyT9wt/f2vbFSPf65LdrHCQ3IVypICa+10rRbqisVVv5G0cKNmJG5npD6qr9mu/66ICPX9i7HpQDuWbmmNirIJhQR+IduXXU7jy/2cUZGVMpwOwtWnD8WEukoMqpDKEVjNNO30BbBk82EUupyqgPziMyvxs9eirwEt7V4/zvvpUjVZSDExTayrNH2NWRDPjtaTaJi3EEc0fgslAqiqNFogKexr68Zf3t8b5SNwylFGTQPCyWPVMd4nWyhVHGMFAihzFLRVWLUUaBLe2r2B8HeiuTy3HIz+LShlHrTozV9njqyOuFGmCxbuaWbZ1lb86s3tUeNa4V4vRyboKXI74wr33y//WD02CwW0gkNTRvbJ66Xwt6rS6JvOswnGfBtxXDZ/tBnEYMcLLLlxRoPheEAWhu/rbKcnLdg2vzd7DEo9Ljx61cSo5xSN16pdWYkTb6otQ21Z+Af8yzeirwEt+453Y+eRTvz01a0Rn1dU4MAvr5mMxXeco8791gVNMdf04sroDFTFQa//frU25NV7jhu+X3NDf9x+wSjccdEodUwbpaR/n2yg/fxE6s9rq7Aq3H7BKNWxHwvtDSAW2oiiX14z2fLaksWausbE5aQvgP/77y589dwRcDoIQoioCoYKgWAI0x8Ka6QDTe7iRW4nNhloB6niPZMQv9njB6LI7TSMEEhl4wajuPN4eAxsx1oeWLgRF55Sq96s3t8Z39E8vKYUH+mchPrPs2qWeWOzFC8/a3QNij3ma31n+xF0+gJYs7cNXzirQTUp7WvrRsO8hWpIapHbqZZKULjt/CY8tnibah/Xs9cgwUnR3PVNSLp6gmpOxaOvGe/8ClwO3HZBU8RYoduJpgGl2Cb7E076AoY26FSxdMtheFxOrGtpw6wxAzCqNjL6R7sLvO9fG9HlCyAoBG6ZNRLNDdE+lK88uyrqPQDgP986G2MGlkeNKyxcdwBPXCsdf7D7OKpLCyJ2SAolsuN7+bYjuP3FNQCAv371DJxusJZ0wcI9RTz8703403t7MKKmBLPHD0LL8W5TR9oCXQhbuYkdtd0bMHSEpYq5T5k7fTxuh6Fwb6wKJxoFQyKpxhGXnhrtxDXbxQBSyQAz5kwcjAVr92PvsW68t/MozhwpmVqWbE6u0UKimvuPFknb/G5/EG4TMxAQWREzGBIRUUpAuEG3UWw+yTdFs+Sq/rqSwM3D+qFYFu7jh1Sgrl+RakP++Ein+hk75TaE9+iqZprxvdlj8KU/rgIgOY0nD+1n6XW94QvPrFSPF60/gJdvmRHxvDaeX9upqtTjMhTur244hFc3RF8b8ZQHLUEhmeuah/XDVafX47sap+wQubro9b8P/52NQnHTCZtlUsAHu4/hT+/tAQD0BAV2H+3EMYMMSwW9Dc5MQE6oq8ha3YoCpyNKWz3pC6hhhUDscK4Orx/rWtqweOMh0xDFgRXROxZ9kpLCrocvi0hy0qN18GkdmDtbO9FQVRxRRTIRlPWYZSiaYXTO72w/Ymi++HDPccMd0czRNaYCc/LQSuw9bhzBo0R6KPzta2eqO5lSjwvLv3ce/u/GqQCMdyQ3zmg0fF89F4ytxQs3TwfQ+7LER0/6YkYitXX1RHUIM2pccswkKUi5KR/v7LHUwtGqqSUQDGHt3jbMaKrG3752Jq5qrscdF4bNVmWF7qgSEUahuOmENfckOXCiG5/533fVxx1ef9zEDqthdUVuZ9YqNnrc0cJ9zi+Xq9odADzzzq6IejRaPvHL5WpUzTdmjcD/XDQ6fYtF5O5nw74Tajbmxv3tpjsjKyiFs559bzeuPK3OtKiZnhE1pVFj1/1uBe667BR86ezhEeMrdx03vKnphbSWQFDg4yPGO0OtQlBbbuz4VM7LJwu/3trMlV3Ct15Yg09O1jdoi88Nz7yPvce6sfaHFxk+P2n+65be5zYDRyYQzgX40h9X4YPdxj4FLR6Lwl3xPS3X+Ll2aEKRvf4gLtKZZcsKMytuWXNPEr2GbsUmrTRB+MMXT8fae4wvakC60ydaFbI3P1JF+9LicUXb3LWCHTB3wAGR4ZL/+ehg1I1i0TfPjruuZrlo1WNXT4o7VyscnVqhSEBzQ+/NBdr3jVWYTI9yE3jrOzMjQuY+NPnOjDVf87/l2EHlIJNId68/BAcBr9w6AwtNvmdVuMt/FyUa5IYzhpl+phHDDW5iifDRvvZe+XH01S7XmuysFOXIimAHYgv3T0+Rbl5CCKyXdw9uzXxt8IMvEIpSzKpLMhthxMI9SfTFlH78n9ghby+v2afWmJg5egAqis21So/LiQ5vwLSCnFb4nirbTRNt5AAYR8VIZpnYN5Y3t7Qa1hbp8Eb+WHe0dkZVVtTbhY1okh1eVuZq8QaC6AmEcN7PlqLDG1DDGXuDtjCZyxnbv2BkihlWVYL6fuHPN/JjAMbJarH6uNSUedATDEXdzPe3deOfq/chJCT7ulnIor52kSLkY/k8jEjG56KUvjBjfYy+sVYd3Kv3tEX1x42Fkc1dKbeg9MXddKAD/1gtrX2KxmymvU6N/G2ONDc218NmmSSxWlwrFBJwOAg/M4lIMEIJJmn3BgwFnHbXMHdqPdb/8wT2t3kxoMxaDG1lsRttXX6MHBAdNWBkllGfc4WfW72nTS1mpbDToP75Aws3AQA+NXkIxg0uNzUXAMBL3zgLXT0BTKirxNhBZRHlGazg9QdxrLNHXUcyjiztDzLen1oRlJ/SmSe0u5hY5SSmNvRHiceJpVtbIQRQGyMWuqjAiWBIwB8UKHCF16iULo6HXnNX1q6viGmV8l6YHLQ5Gka8uMq8qUi3Pxhhwz5tWD9T7fyNBPrh6vMsAGDhN8/G+n0n1E5LL6zcoz53n1xCAwDuunwsxg0ux90vR1dUzUTSkh7W3JNkwZrYxZvulCM8hn9/EVqOd5k6F41QIlOMHJcf7TuBC38u9Rr9xdxJGClvj2M1YtDT1uXHZ0+rM3xOK8AB4IRGKFVqdhtGmlsgxjme3VSNL509XI34MGJSfSXOHFGNUo8LnzujIeZcI555Zxeu/W04EkhbBCsZOn0BPLZ4q2k1RsXZrC/Spf2b+4Mh0zo1100fime+OBXXTxtm+D5aFM1bH85qVZPWa+5KBcpCizZnLRePq+3V7kgbTTT1wcVRviilbIQR2hyJAye68cHu4xhVa2wiuuul2I3mL9NEbRlda0OrinHZhEGqkvB/mhIEQyrD561cr0bo/SyZgIV7kry20Xqo3XW/W4FE5FRhgXnZ3z+v2K2m5juIVNvfTwyyYY1QtvNmNv0ClzNCuL+706RWusFYrLBBfTp9KvnKOeEfkNY/ECsk0QozR0thii+s3IvHFm/DYyaZwL95aycA4M8r9kSMPzZ3khpSOam+0nBnA4R3GDfOaMTwmhLDUFEFZcf4+affjxhXnHYXnBL7e9Zr7krDid5o7krnqkRp19jaD3f4sGRT5G8p1i7nuOY5pSXe1kMnMVtT1vhHn4ouqaHwmSl1ajP0W88fibsuOwWX6BKz9FjtZ2wUSprpMEiAhXvK2HDfxfjVtdHZZ1qH0+6jXVE2+lgoscmtHdFhXtqkjarSgojyAfGabgOSqQcI2+r1uByEjZokGe2PKSSAt787C4CxIP9oX/h1eptvpUmphVRw56XG9U7GDjJPSrHCjz8zAUC4rrvXRJApXY/0iUKzRg/A5vsvASDZa49ptE5t9I1iZmisLsEb355pWLtEYaiJbVzZNX3nYvOcACCsuesrG/bGIa/0nAWA/24/gsMdXhw96YvbMFqvJS/edDjKXwMAWx+4RL1ZfUkO0/RprrvtcuN0j8uBJz93GnY9fBl2PXwZLh5nbArZ9fBl+NlVE7H8e+dh18OXYczAcnzp7OH43+tPM5yvYLXkwo0zGqPMVMn4JnoLC/cUUeR2GmqIRjY8qyg/7oXro7en2q1+fb9iVGucomaZp1q+8Iyk8R04YfwDfGPzYfjlmH0AanVBQIrTV4SDz0C4P6Sp15GNvqx6YglJK+gzL81CFBtkM9oFp5jbV9/fdQzf0yS7zNQkLyWyTqNQSyBc/19f916P8vfTd3Sy6q/Romjuh9u9uPZ3KzD1wSU47YHFcRtG64ugLdvaiis1YcUKbifh3NGScFdaMmp3nEqT+S+eFRmfbzVm3SpGGa1mtOsK3hkFLaQbdqgmyYAyD0YPLIPDQarAHTe4HE9cOwVVpQWWQ7CMmCDXtzBSprTbYH2Ew3ELNeCVqnUNVbGjIw6c8EY5TAeWF2oyN829jB/efSEqityYNaYGn/v9+6bzUsnaey7Cks2HcMeLawEAy783C3X9EosA0aNPPjGLm2+Sbb7XThtq+LzCQTmk8v4rxuG6acPUwmax0t71jB5YhvJCF87WZbaq5ZTjmAGUqqOKADy7qRob9rdj+vDE0+MVzT3RqqRNtaVRdvYth8JdrNxOwqcmDwER4fppQ3HJ+IFqYTij/I/vXByZS5Fq4T52cDne+s5MlBe6Mfn+2PH3NWUetHb48PevnYHBlUUxnePpgjX3JDnc4VOFhyIEZoysRkN1CcoK3Sg2qbYYT6gC0ra10O3An1fsiUiWAIBF6w/Kc6Jf943nPkTDvIVqAakHF27ERbLzVc+gytiOsIMnvPjd2zsjxko8TlV46At1acPX+pcUwOmgCKdTuqkoduOTcgOOwRWFSQt2BW2TDLO9mBLmWGDRxj92cHlS4XHlRW4sXHcAr8lNuNfubVOdh1ayISuL3OqafYEQmgaUJuy8BiQh2u0P4kE5IsoKVzzxDpZuMY/s8QWC8AeFupshIlSXelAshyUqUUFTNEJWb/qw+ndIhGFVJehnITRXCXAYWFGUVChuMrDmngSKU0v5PZw9shp3Xz4Wn5kSDoVrHtYP/3PRKPxUFwIZz76noGjGv1m2AzPkkEBt3PvS/5lp+tr7X9mIq06vx2/f/jhiXGvSiReiRQQ88p/osqiKCapYJ0T+9oF5+FqmcDgI/3vdFEzSNPJIFq0d3cwqrQp3E43x2mlD8dyKPZhYX4m1e9vUGOlvXzgKzl6Y7xT/zWOLt+GicQPx9w9b1Ofi9T5V1qkKd3+w1/4Q5Qb6lkEYphDC8IahJB01D+sHl5Pwnq7AmxKd5dSZwAZrSlYIIdRw4HGDo3c9RISqkgIc7ezBuaNqcNOMRgyuTI0G/cLN01EaI/zzieumYNnW1owqNnpYc08CZWs4vFoyW7icDtw0ozHiR+JwEG45LzpF/5QEnXwrNE0CvJooFr3JREuHL4AVBvb3pzVlguNparc9vyaq7rqyGzllUHnE9tgfDEWEiWWTS04dlFKNSWt+2n64w3COkuVpJtzPkW/O61vaMKSySP3ubz2/CV+fOTLhNdX3l85vo1xjX/GfWNVYW4534x+r9+Ffa/djbcuJuHZ6M2LZoo1yJbS5IXX9ivD8zWegny6ZT02q6hf5NyQijBxQivX7TkTU+/naTOO6Q0qLwFOHVOCcUTWGOR29YdrwKowbbF4WuH9JQa/KMaQSFu5JEK65nViY0w8/Ya3qnhZtZqORExOAYZ2Xqw0qPz64SNo+l8RY9/2fHG84fsbwKnxfjkopcjsiomW03WcGaTQsxSfwmSnGMfV9DbPKgfE097GDJGEQEtZrmMRCGyro9QfV0L6fGdSlj8Wtf1kNINyVK1EqY2RZG/Uj0F4zy2Rzo97foPiUjHwH3T1BeNzOiDDkyiLjXcdVp9dj7KByXDc9th8kH2GzTBIoWquVGNYyjwsdvgBeu/2chLzuRnhNMkfvuHCUaVd7QErCKdG0zvvBZeY3mSsmDcbduuSPiXUV+IumDk1RgVNtwAxEhtFpvxO309Hrqoy5iFlMty+OzX2oxs/S20xQLdqQ1/1t3XhrSysqitxR9d+tEs8RbEasKJ+jnT1RNmrtLkgxk2tviFsOdqj1ZoxugpPqK7FsWyuOaCJ9hpn4sGrLC7Hotvh1jPIR1tyTQKlZYSU0WCk6lGidFCPMNHcAMW18+gSnUwaZ32SKDYTPxbokj05fEAfawqGU2vD6z5hkvvZ1ClwO0wihRByqvWlmrkf7HZ/3s7ew80hnUiWie2tzj9Wi8dJfvB01ptXc554u3VAu0Ph+Ln5sGT4n10E3ugn6AiF0eAMRkTZ63w/Dwj0pFNthQ3X87ezdl4/Fiu+fn3Dvyc33z8anZdud8nmKcHncoGXXa7efg+Xfm2X4Xsvl0gQDywsxbnB5zOYKLqcDb3z7XPXx12aOwNd09dQn1VdGOBeV5J5fXzclam6+MGFIhWkGrj8YgstBliJgEmm4bcYPPzEuauzKBG6qm++fHeGg1Nu9reJ0UFSzEQV9P1YgLNy/MWuEWgP9et2uQfl+jDT3qY3R122ma6X3BVi4J0E4YST+heVyOnoV61rodqrt2pT6H4p2VuaJ1phKPC7T8D+lUl0gFLJUl3ywZhdQ3684yvlaUeRGMCTwqNz8+Ytyt5yGqpKMV8BLN6Xyd13odpoK918v3RGzrk6qMcp6TCQJqdDtxDg5Q9njcvQqDFIhkVrl18pdqE4dUqleJ0RkuKs1+m31Nyidm86OZX0VFu5JEMvpk0pWfiwlQv15hRSJotxUPDGiG7SRD7fJjlalL6fXH7L0Y9Cel1EkhfL847rmz72Nushlnr95Op7+QnNEqn1v+O3nm1O4qsgOVEDi372iGSfr4FXCc69ursc3zxuJV26V2uAZZSgrncgm6JpQKyUtjNan5UJNBvAXz2rAL+ZOyjtlIhXk368wg1jNBkwWJU1bceAqtn4r/R7X33sRbpe3vm9tbcVJX8ByASQtRk5j7Q9vmyazMBXOwlxj/JAKnDemFoVupxp62BsuTHHpV33dnERLwyjF5/Tp8onikv0MF4ytxR0Xjcb4IRUoK3Rhwdr9pn4AvSO2xGAnaiTcK4rdam7JTTMaccWk7IYc5ios3JNANcukOM1ZzzfOk2KgzxwhxUkH5V9wqcGPQeHpG07HxeNqo5xdD/9bCoP8a4LJRkY3sA6NQLhQ01JsQJK1XHKZ7p5g0lru1c31+OlnEwtXNKNJF3n1pxWJ5RkYJR71hq/PHIEzR1RhmqZ8gXJ9/P0D46YcRrWYpugSz+IpCtmotthXYOGeBErVu3Q7cxTtTAkPU5xN/UrMHWBnjqzGbz7XHLVdVcrNWt3EKjXFjUxAwZCxeSLZEru5zPghFfAFQoaVNxurSyyFIT5y5YSEHJ+x0Gu/Jn+StDNucAWe+/J0w8zYZ9/bbbmpzT++flbE43iF99iRak7+/gozgFKgK93OHEWwPvnWDgDhUMjemIP+Kztlr2qutzRfqb9u5KibUFcZNZZo16S+hiKk9NUUAenvkorkpETR1jC/Zqq1v6uCWcnnVNCktqVrx/Mre1eWwizM8jo5uoYdqeawcE8CB0mmkXQ7c/QXcKwwMTP0YZM3axpbxOJ/LhqN939wvto/UssFBvbjP9441fKa+iKj5Z1Mp0F3LF8glBXh/vg1k7H+3ovw9ndnGZa6iMXL3zgr/qRe8uMrJ6jH2w51IBQSaqLbTTMazV6GW2Uz5A1nDDO0wwPA/DnjsWn+bHakxoAzVJPA6w/1qndkoui3pl5/EESJVb3Tzy2OYa/X4nBQzPA6bfEpbb2UfEVxYhuFQ/oCobQ7141QSvfq685bIZ3CUWsy+eO7u/HCyr2qYhIr6UgJGR4ZI5Pb4SA2ycSBhXsSeP3BjPyYtQJTCCF9rsuZoCCNtBHHcsYmwtvfnYVpP5KaMoww0O7zjXAde+OaKdnQ3JPllVtnpKVTEOk8O9rErY4Y0TnXTB2KEo8TcyZyFEwyWLoSiWg2EW0hou1ENM9kzlVEtJGINhDRc6ldZm7iC4RS3hDAymf6AqGYMe6ZRBsZc06e29sBbWPpSAdhIBhCICQshafmGuOHVCRcpdQKsZyoRgXFFJwOwqcm12WlNV0+EVd9IyIngCcAXAigBcBKIloghNiomdME4E4AZwkhjhNR+rog5xCZ0twB6YIPhgR8gZCquSfCmSPDgtes4mNvyHczjJ4it7FZRkmzz5Wbbi4wemCZadJXJjN57YqVK3EqgO1CiJ1CiB4AzwO4QjfnywCeEEIcBwAhxOHULjM3yaQD7b45Uh0Rnz+IDfvbE94xlBe61cbBn5s+LB1LxPAa89ry+YJyM1eaRCj4MpTz0JdwOx1qY3A9eRwtmzNY+YqHANDGMbXIY1pGARhFRO8Q0XtENNvojYjoZiJaRUSrWltTkzyRTXwZ1NzVhtSBELYdPonOXmSZpptsOBMzjVJDpfVkZCikUjQtH7Nz08H5MZqIM6khVQ5VF4AmADMB1AFYRkSnCiHatJOEEE8BeAoAmpub+/y+rNsfxMAMbcOVHYLXH0RPIIRPTs8dZ9OImhLsaO20RbagUtUzqOtOpWjufdGhmm6uaq7Di6ukFoAf3XcxXA6yhSKQbaxcifsAaDMj6uQxLS0AFggh/EKIjwFshSTs85qth05mzOmjb0it1JvJBZReoHb4wSrmsLe3RzYsX9vSBiBcY4UJo21OU+px2eI6yQWsXIkrATQRUSMRFQCYC2CBbs5LkLR2EFE1JDPNztQtMzcpcDqiGvimC0Uj3HZIKts7tbF/rOkZZd4lY/DTz05USxXkO6UeV1QoqRL9Md6gUbPdufEs84QlJn3ElUxCiACAWwC8CmATgBeFEBuIaD4RzZGnvQrgKBFtBPAmgO8IIaI7M+cRQgj0BENorM6ME1HRdl7bcBAAMLR/7/pdpoOqUg+uPK3ONpEzYweVR4XyKTHcve1mlM9wFml2sGRzF0IsArBIN3aP5lgAuEP+ZwuUH3Om7My15ZKtVynXG6tvJZNeCgucahE3hXD5ZzbLGDGhriKiyTuTfjhDtZeEm2Nn5seslABo9wZQXODM68qLuU6R24F1LZ0RY2oDlT6YxJQJFtwyI9tLsB0sIXrJgRNSw4xMJWNoozC6YmT3MemnvTsQlcTkCwThdhJnVTI5Awv3XqII2NEZciI6HKRGapw/xhYJwDnL2MHl0N/Tvf4Qa+1MTsHCvZe8LXewMWpOkC6Uyo7lORQGaUeK3E4EgiG1fC0gae5sb2dyCb4ae4miuaWz2YGeEo+kGXKccHbxuBwIiUiTHGvuTK7Bwr2XeP1BFLmdGQ3zUhoXWG1ZxqQHpTiYtoTtqt3HODuVySn4auwl3kAw480ClP6n7+7I6xSCnEfR0H0ap6rH5YgQ9gyTbVi495IVO49FVQbMFPGaBjPpRdHQtcJ866GTmD68KltLYpgoWLj3km2HT2b8Mz85aTAA4MYY/SeZ9KM3yygVOnvYXMbkEJzE1If4+dWTMP+T4zMaocNEo5hllIzLdq+UrXrmCNbcmdzB1pr7nF8tx4R7X034dQvXHUjDauJDRCzYc4CwWUayue87LiW0cdYwk0vYWnNf13KiV697eY1U8dgmdbIYHapDVdbcu2XHqlL/h2FyAVY1ALywck9C8x2yVJ9YV5mG1TC5jkt2aD/62lY8+dYOfOv5NQCAflwRkskhWLgD+N7f1yc0f6xcs/vbF41Kx3KYHEcxw7y78yge/vdmHJWjpji5jMklWLjLbDnYgaVbrPX1VsruKh2IGHuhZArrKc5w3gPDxIKFu8zcp97FF55ZaWmuEiXBmpo9mVhfaTjOZhkml7C1cFeiHk4dUoHjXVI4m5XUfiEEygtdXN7VpgyqKDIcz3TGMsPEwrbC/b2dR9Voh/X7wlEz3f74tdK9/hD/kBmGyWlsK9zf2GxsX/daaIThCwS5AiATwSu3cqchJrewrXDffLDDcLz1pC/ua32BENfuZlQGlHkwPoOlnxnGCraVUGPkDkqnDYuMeOn0WdHcuXa33bnn8rEgAm44YxievWlatpfDMFHYNkM1GBIo87hw9en1+GD3cXXcis1dMsvY9r7IQCrexgXcmFzGthLKFwiiwOXAiJpSAEB9fykCotuKzd0f4jBIhmFyGvsKd38IHpcDpw3rhy0PzMYfb5S21l09gbiv9bLmzjBMjmNbCeULhOCRtW+Py4nyQslCtXLX8Vgvk17rD6k1vRmGYXIR20oovd28qlSq6FdiIX6dHaoMw+Q6NhbuIUPTyu+Wfxzzdf5gCHuOdbFZhmGYnMa2EkqyuSeufR/ukOLg2aHKMEwuY1/hHgj2ym7ulUMlJw+tTPGKGIZhUoeNhbuxWSYeSqgk29wZhsllbCvc27r8cDkiT/+S8QMxqrY05uuUvplcfoBhmFzGlhmqoZDAvrZu7Gvrjhj3uBxqpUgzvH7p+SK2uTMMk8PYUv3sManZ7nE54fPHE+6K5s7CnWGY3MWWwt1MgHvcDtXsYsauo10AWLgzDJPb2FK4e2UBfs3U+ojxQrdTNbuYofRe6l/CLdUYhsldbCncFc39tGH9I8Ylm3sQQgjT1ypVI8sKbemuYBimj2BP4R5QwhkjT9/jciAkgEDIXLh7/UE4KPq1DMMwuYQtJZQSERMt3J0RzwNAW1dPxJwT3X4UuBwg4ubYDMPkLpaEOxHNJqItRLSdiObFmPcZIhJE1Jy6JaaecKx6pFNUQNLYdx3pBAAs33YEk+a/jmVbW9U52w+fzNAqGYZhek9c4U5ETgBPALgEwFgA1xDRWIN5ZQBuA7Ai1YtMNYrNXa+5N9VKrfc6vFJN9w/3SOV/3//4mDqnuMCFiiJ3JpbJMAzTa6xo7lMBbBdC7BRC9AB4HsAVBvPuB/AIAG8K15cWlGgZj05zV2q6v77xEADA5ZRML3uOdalzfIEgBlcWZWKZDMMwvcaKcB8CYK/mcYs8pkJEUwDUCyEWxnojIrqZiFYR0arW1tZYU9OKmeau2Nqffkcq+1tZVBA1z+sPopDryjAMk+Mk7VAlIgeARwF8O95cIcRTQohmIURzTU1Nsh/da5SyA3rhrrfBO+Wnd2s0d68/xHVlGIbJeaxIqX0AtNk+dfKYQhmA8QCWEtEuANMBLMhlp6o/KDlOy3W285EDpKJhDVXFAMKavNsZjozx+oOcncowTM5jRbivBNBERI1EVABgLoAFypNCiBNCiGohRIMQogHAewDmCCFWpWXFKcAv15bRO0bLC90oL3Sh0O1ETyCk1pFp7w43zd52+CTHuDMMk/PElVJCiACAWwC8CmATgBeFEBuIaD4RzUn3AtOB1x+Ey0FwO6NPv90bwOaDHZj2o8VqKYJthzsAAKvl6JmX1uzP3GIZhmF6gaUceiHEIgCLdGP3mMydmfyy0otkN49tWjne5Vfj4Us9koa/ft+JtK+NYRgmFdjSvuANBC05RX+7TIqaCYQkDf6elzcAAIoL2ObOMExuY0/h7g+atsn71bWTAQDnjxkAvyzU/boGHn/76pnpXSDDMEyS2FK4bzt00lRzv3zCYAyrKkaBywGlOGRnTxA//s9mdU68VnwMwzDZxpbCvcDlwIluv+nzHpdDbcqh8OulO9Rjl4EjlmEYJpewpZTqCYQwoa7S9HmPy4lNB9oNn5va2N9wnGEYJpewpXCXEpHMT10bFVNZHBkLzzHuDMP0BWwpqbwB6/Vh/vqVMyIeF7BJhmGYPoAtJVV3TyiqIqQZ/UoK8KnJ4TppDgc36WAYJvexpXD3xTHLaKkqKcAdF45SH7fHcMQyDMPkCrYT7vvbutHhCyBGD+wIiAj1/Ytx35xxAIAh/biWO8MwuY/thPv8f20EACzdcjih14Xku4G2iBjDMEyuYjvhvuuo1B91lNxSzwhtiV+FIWr3JYsqP8MwTBaxnXCvLS8EAJwzyrxZyCXjBwEABspzAWBQhSTcmxs4zp1hmNzHUlXIfGJUbSne2tqK66YNNZ3z089OxA1nDkN9/2J17NS6Crx2+zkYXl2SiWUyDMMkhe2Euy8QQmWxG0TmIY0FLgdOGxatoccy5TAMw+QStjPLcINrhmHsgO2Euy8QgocbXDMMk+fYTsqx5s4wjB2wnXDffbSLNXeGYfIe20m5LYc60N0TzPYyGIZh0orthDsBOJ1rsjMMk+fYSrgHgiGERGRyEsMwTD5iK+HulRtdF1ks98swDNNXsU0S06OvbcHy7UcAAIUFLNwZhslvbCPcH39ju3pcUeSOMZNhGKbvYyuzjMK4weXZXgLDMExasaVwZ5s7wzD5ji2FeyELd4Zh8hybCndbnjbDMDbCFlJO6Bqmcm0ZhmHyHVsI9x2tJyMeOxzmtdwZhmHyAVsI97e3HVGPR9WWZnElDMMwmcEWce5ev5SZumn+bBRxAhPDMDbAFpq7LyBVgfS4bHG6DMMw9hDuXn8IBU4H29oZhrENthDuvkCQtXaGYWyFLSSe1x+ChxOXGIaxEZaEOxHNJqItRLSdiOYZPH8HEW0konVEtISIhqV+qb2HNXeGYexGXIlHRE4ATwC4BMBYANcQ0VjdtNUAmoUQEwD8DcCPU73QZPD5Q5yVyjCMrbAi8aYC2C6E2CmE6AHwPIArtBOEEG8KIbrkh+8BqEvtMpOjpa0bHs5KZRjGRlgR7kMA7NU8bpHHzLgJwL+NniCim4loFRGtam1ttb7KJOn0BdDh82fs8xiGYbJNSm0VRHQ9gGYAPzF6XgjxlBCiWQjRXFNTk8qPjomDgKYBZRn7PIZhmGxjJUN1H4B6zeM6eSwCIroAwA8AnCuE8KVmeanB6w+hvNAWybgMwzAArGnuKwE0EVEjERUAmAtggXYCEU0G8BsAc4QQh1O/zOTo9ge5hjvDMLYirnAXQgQA3ALgVQCbALwohNhARPOJaI487ScASgH8lYjWENECk7fLCl4W7gzD2AxLtgohxCIAi3Rj92iOL0jxulKKFArJwp1hGPuQ98HfwZBAT5Dj3BmGsRd5L/F2H+0EAPQEQlleCcMwTObIa+EeCgmc6Jbi2ycP7Zfl1TAMw2SOvI4PvO53K/DuzqMAuCk2wzD2Iq8lniLYAbBDlWEYW5HXwl0LV4VkGMZO2EbisebOMIydyFvh7g9GRsew5s4wjJ3IW4n3/sfHIh5zyV+GYexE3gr3Dm9kiV/W3BmGsRN5K/G6/UEAwIAyDwCggIU7wzA2Im/j3Lt7JJv7n740DUdO+lDiydtTZRiGiSJvJd7WQx0AgNryQoyq5UYdDMPYi7y1VbidBADcpINhGFuSt8L9vzuOorrUAyLK9lIYhmEyTt6qtRv2t6O4gMMfGYaxJ3mpuYdCAgDwxbMasrsQhmGYLJGXwt0bkMIgywrdWV4JwzBMdshL4d7VIwl3NsswDGNX8lK4/2XFHgBcLIxhGPuSl8J9hVxXZlpj/yyvhGEYJjvknXDv8PqxfPsRnDOqBsOqSrK9HIZhmKyQd8L93x8dBBCuKcMwDGNH8k64t3X1AADu+cTYLK+EYRgme+SdcP9oXzsAoLQgb/OzGIZh4pJ3wl0Jf3Q4uOwAwzD2Je+Ee7c/iIaq4mwvg2EYJqvknXDv6glyfDvDMLYn74T7mr1tKOLMVIZhbE7eCfejJ3046Q1kexkMwzBZJW9CSoIhgT3HuhASwGUTBmV7OQzDMFklb4T7I//ZjKeW7QQA9C8pyPJqGIZhskveCPeW410YWF6I7192Cs4fMyDby2EYhskqeWNzf33jIdRWFGLOxMEo8eTNPYthGKZX5I1wB4ACJycuMQzDAHki3Du8fviDAmc31WR7KQzDMDlBXgj3d7YfBQBUFHFbPYZhGKAPCvfjnT041O6NGGv3+gEAs0azI5VhGAawKNyJaDYRbSGi7UQ0z+B5DxG9ID+/gogaUr5Smafe3onpDy3B/rZudWzTAakSJGvuDMMwEnGFOxE5ATwB4BIAYwFcQ0T6Yuk3ATguhBgJ4OcAHkn1QhWqSgogBHBQo727ndJpVBSzcGcYhgGsxblPBbBdCLETAIjoeQBXANiomXMFgHvl478B+BURkRBCpHCtAICJ9ZUAgFufW62W9z3c4UMVJy4xDMOoWBHuQwDs1TxuATDNbI4QIkBEJwBUATiinURENwO4GQCGDh3aqwWPH1yBq5vr0eHzq2NNtaVoHsbNsBmGYRQymu0jhHgKwFMA0Nzc3CutvqjAiUeunJDSdTEMw+QbVhyq+wDUax7XyWOGc4jIBaACwNFULJBhGIZJHCvCfSWAJiJqJKICAHMBLNDNWQDgBvn4SgBvpMPezjAMw1gjrllGtqHfAuBVAE4ATwshNhDRfACrhBALAPwewLNEtB3AMUg3AIZhGCZLWLK5CyEWAVikG7tHc+wF8NnULo1hGIbpLX0uQ5VhGIaJDwt3hmGYPISFO8MwTB7Cwp1hGCYPoWxFLBJRK4DdvXx5NXTZrzbBrucN2Pfc+bzthZXzHiaEiNu8ImvCPRmIaJUQojnb68g0dj1vwL7nzudtL1J53myWYRiGyUNYuDMMw+QhfVW4P5XtBWQJu543YN9z5/O2Fyk77z5pc2cYhmFi01c1d4ZhGCYGLNwZhmHykD4n3OM16+5rENHTRHSYiD7SjPUnoteJaJv8fz95nIjocfnc1xHRFM1rbpDnbyOiG4w+K5cgonoiepOINhLRBiK6TR7P63MnokIiep+I1srnfZ883ig3l98uN5svkMdNm88T0Z3y+BYiujhLp5QQROQkotVE9Ir8OO/Pm4h2EdF6IlpDRKvksfRf50KIPvMPUsnhHQCGAygAsBbA2GyvK8lzOgfAFAAfacZ+DGCefDwPwCPy8aUA/g2AAEwHsEIe7w9gp/x/P/m4X7bPLc55DwIwRT4uA7AVUgP2vD53ef2l8rEbwAr5fF4EMFcefxLA1+TjrwN4Uj6eC+AF+XisfP17ADTKvwtnts/PwvnfAeA5AK/Ij/P+vAHsAlCtG0v7dZ71E0/wSzoDwKuax3cCuDPb60rBeTXohPsWAIPk40EAtsjHvwFwjX4egGsA/EYzHjGvL/wD8DKAC+107gCKAXwIqSfxEQAueVy9ziH1UThDPnbJ80h/7Wvn5eo/SF3clgA4D8Ar8nnY4byNhHvar/O+ZpYxatY9JEtrSSe1QogD8vFBALXysdn59+nvRd5yT4akxeb9ucumiTUADgN4HZL22SaECMhTtOcQ0XwegNJ8vs+dN4DHAHwXQEh+XAV7nLcA8BoRfUBEN8tjab/OM9ogm0kcIYQgoryNVyWiUgB/B/AtIUQ7EanP5eu5CyGCACYRUSWAfwIYk90VpR8iuhzAYSHEB0Q0M8vLyTQzhBD7iGgAgNeJaLP2yXRd531Nc7fSrDsfOEREgwBA/v+wPG52/n3yeyEiNyTB/mchxD/kYVucOwAIIdoAvAnJHFFJUnN5IPIczJrP97XzPgvAHCLaBeB5SKaZXyD/zxtCiH3y/4ch3cynIgPXeV8T7laadecD2objN0CyRyvjn5c96tMBnJC3dq8CuIiI+sle94vksZyFJBX99wA2CSEe1TyV1+dORDWyxg4iKoLkZ9gESchfKU/Tn7dR8/kFAObKUSWNAJoAvJ+Rk+gFQog7hRB1QogGSL/bN4QQ1yHPz5uISoioTDmGdH1+hExc59l2NvTCOXEppMiKHQB+kO31pOB8/gLgAAA/JDvaTZBsi0sAbAOwGEB/eS4BeEI+9/UAmjXvcyOA7fK/L2b7vCyc9wxItsh1ANbI/y7N93MHMAHAavm8PwJwjzw+HJKQ2g7grwA88nih/Hi7/PxwzXv9QP4+tgC4JNvnlsB3MBPhaJm8Pm/5/NbK/zYoMisT1zmXH2AYhslD+ppZhmEYhrEAC3eGYZg8hIU7wzBMHsLCnWEYJg9h4c4wDJOHsHBnGIbJQ1i4MwzD5CH/DwKqmTPymNmKAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "## DO NOT CHANGE THIS CELL\n",
    "rm=movingaverage(rList, 100)\n",
    "plot(rm)\n",
    "criterion = len(np.where(rm >= .78)[0]) >0\n",
    "if criterion:\n",
    "    print(\"Solved after %d episodes.\" % np.where(rm >= .78)[0][0])\n",
    "else:\n",
    "    print(\"Unsolved Yet!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Problem 3 (SARSA)\n",
    "Implement tabular SARSA (*YOU SHOULD ONLY CHANGE THE CELL BELOW*)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.random.seed(123)\n",
    "\n",
    "# Number of episodes and other hyperparameters\n",
    "num_episodes = 5000\n",
    "episode_max_length = 100\n",
    "gamma = 0   # hyperparameter tuning\n",
    "learn_rate = 0  # hyperparameter tuning\n",
    "eps = 0.9\n",
    "eps_decay = 0.99\n",
    "\n",
    "# Create Q-table for FrozenLake-v1\n",
    "Q = np.zeros([env.observation_space.n, env.action_space.n])\n",
    "\n",
    "# Execute in episodes\n",
    "for i in range(num_episodes):\n",
    "    # Reset the environment at the beginning of an episode\n",
    "    s = env.reset()\n",
    "    d = False  # Not done\n",
    "\n",
    "    for t in range(episode_max_length):\n",
    "        \n",
    "        ########### SELECT ACTION a for state  using current policy ##################\n",
    "        ## Take an action at state s by e-greedy policy\n",
    "        ################# YOUR CODE HERE #####################\n",
    "        \n",
    "        \n",
    "        ######################################################\n",
    "\n",
    "        # Get new state, reward, done\n",
    "        s1, r, d, _ = env.step(a)\n",
    "\n",
    "        ##### Update Q(s,a) ############\n",
    "        ## Take an action at state s1 by e-greedy policy\n",
    "        ################# YOUR CODE HERE #####################\n",
    "        \n",
    "        \n",
    "        ######################################################\n",
    "\n",
    "        ## SARSA Update\n",
    "        ################# YOUR CODE HERE #####################\n",
    "        \n",
    "        \n",
    "        ######################################################\n",
    "\n",
    "        # Break if done, reached a terminal state\n",
    "        if d:\n",
    "            break\n",
    "        \n",
    "        # Update state\n",
    "        ################# YOUR CODE HERE #####################\n",
    "        \n",
    "        \n",
    "        ######################################################\n",
    "\n",
    "    # Epsilon decay\n",
    "    eps = eps * eps_decay\n",
    "\n",
    "    # Policy improvement\n",
    "    pi = np.argmax(Q, axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Solved after 6469 episodes.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmcElEQVR4nO3deZwcdZ3/8dene2Zy35kc5GACCYRwyBERFlAWRDlc8Fp/wdtVUVdWlD2M8tNlcddF/K3rsXig67msLOIqWcOlyCqIAuEmIRdJJAkJuROSITPT09/fH13VU91dPdMzU93V3fV+Ph55UF1dM/2hZvrT3/l8L3POISIijS8VdwAiIhINJXQRkSahhC4i0iSU0EVEmoQSuohIk2iJ64WnTp3qOjo64np5EZGG9Oijj+5yzrWHPRdbQu/o6GDFihVxvbyISEMysz+We04lFxGRJqGELiLSJJTQRUSahBK6iEiTUEIXEWkSSugiIk1CCV1EpEkooUuibNx1iDuf3hZ3GNz+xFZeOtwzqK/56r3ruOuZ+GOX+qWELonylzc/xkdufizWGFZvP8BVtzzBJ3/6VMVf8/SW/Xzpl2v58H/EG7vUNyV0SZRntx0AIM6NXTq7ewF4Yd/hir9m/8uDa81LMimhSyJlso21U9dPH9sSdwjSAJTQJZG6Mtm4QxiU36zdmT/ubbAPI6kdJXRJpG37Xo47hEFpTVv++OWe3hgjkXqmhC6JtG1/5fXrahlMO/t9Z83LH3d2Z6IPRppCRQndzC40szVmtt7MloY8P9fM7jOzx83sKTO7OPpQRYbvyCmjAVj5woFBf+2a7S+x7sWXog6pIgcP9yXx/3zoee58ehtZlV6kyIAJ3czSwI3ARcAi4HIzW1R02f8FbnXOnQIsAb4edaAiUXh+TycAX7hr9aC/9vVf/i0X/Otvhx3DcAfYfPlX6/jIzY9x80Nll8WWhKqkhX46sN45t8E51w3cAlxWdI0DxnvHE4AXogtRJDoxjlbMy3pBdA+iY9as9NwLdVA2kvpSSUKfBWwOPN7inQu6FninmW0B7gD+KpLoRCI2ffyIuEMg05tL6P6Y+Er09JZ+En3jf5+LLCZpDlF1il4OfN85Nxu4GPiRmZV8bzO7wsxWmNmKnTt3lnwTkWpbOGP8wBdVWXDYYaVDEDO9jTXMUuJRSULfCswJPJ7tnQt6P3ArgHPu98BIYGrxN3LO3eScW+ycW9zeHrrHqUgkntqyj46ly+lYupyTrr2bC770GzbtOlQwnvvvb3+m5Osef34vHUuXs/Olrvy5XQe7+ORtlU/TH8h9a3bkjytdz6XcRCjV0SWokoT+CLDAzOaZWRu5Ts9lRdc8D5wPYGbHkUvoaoJLbC79t9/ljw8czrBux0H+4vuPFFzzg9+XJsM3ff1BAD7248fz5264azX/tWJzybVD9e8PbMwfH+qubEx5T2+WSaNbS85f87PSDyVJrgETunMuA1wJ3A08S240y0ozu87MLvUu+2vgg2b2JPBj4L0uzsUyJDGcc/zLPWsqKkls2n2o3+c3eyNgoK8T0jnHrSuqN+2+0lLKhp2H2NvZwwfPmTfwxZJYLZVc5Jy7g1xnZ/DcZwPHq4Czog1NZGBX/vhxlj+1jbtXbueeT7ym32sHKle//wd9LXg/oX/13vWl3yfrSKVChp0MQU+FCf33G3YDsOxJDSCT8jRTVBraQxv2ALD2xYPD/l7B79GTyWX/p7bsK7kuyoW9ujOD+14vHugq+5z+KBYldGlo3ZnqrGsyqi0NhCfvbEji7M06OpYu73co4XGfuYuP3/J4wblKWuiVjITpWLqceZ+6g+VPaQOMJFNCl4Y2cXRb6Pmz55cMshqU8xZOA8LXWwlL8v4koS//am3Z7/lyTy8/f6KwZFJJQq+0LAPw+TuerfhaaT5K6NJQNu/p5Ev3rMmXFw6UGfY3oqWyX+2OpcvJZh27DxaWMjbszJVf0iGl8rAWc082l3S7MtnQjs67ntke+vrX/WJV2dh+9PtNPPb83oLXaw0LKGBrg60iKdFSQpeGcsWPHuWrv17Phl25ESvvPuPI0Ov6q3P/85tPLHj85JZ9XPs/hYk1ncq9NS46cWbJ14cl9F2Bceu/XVc6YvfD//FoweOFM8YB8NSW/WXj/MztK3nz1x8s+H/56Uf+pOz1Ikro0lCKp8uPHhE+UCuTLV+muPz0uVx+et9cOQclmy9v2Zsbwjh+ZOn3L07oOw4cZuOuviGRz+04xI33rc9PGgprsb/1tNn542e27ufXq18sG6//ev9w6fGcNHsim66/pOD5zu4ME0PGqEvyVDRsUaRedfX0JcuuTC8jWnKdmWFrnwQF+zVHtKRKrt9zqBuAsPJ1cUI//fP3Fjz+J6+O/cW717Dp+ktK6uYAYwIfRG/42gMAPLj0PI6YOKrkWv/DKV1mqOTbv/0Q+zr7Sk/OOSxsNS9pemqhS0Py01VXYJRLMElnerOcNX8Kb3/V3NCvD45UMUqTn/986IiWQQ4PXBuyhvpobxRNULmNoP0PkJYyCf2JzfsKHg/0YSbNSy10aWjBsePB5PvY8/sAOGdB+JpB/vPQt0Z60OrtuST8QkgnY3BjiY6lywcVr681XdqW8uPf19nNydf9Mn/eH0FTroVe7OXuXtoq7BSW5qKfujQkP7kFa8dhnZXvPzt8qvz6HX0fBNv3lybtiaNy3zesJT3YiUVhE35mTyotrfiXPV7U4j7UlfsrpCUwwuW2D5/JUe1jQl+vq0pj86X+KaFLwwiOx/aPuwKbRPjrjL8cWPAqrCVcrHiEC/Ql7e6Q8sUHf7gC51xF67A8vHEPG3aWriEzbmRpJ6bfQn9g3a6C83d6Hbb+yBuAxR2T+eSFC0NfM8qZrNJYVHKRhnHbo32LZPlT5rt6+pK3n8i+9+BGyvnEa48B4PR5k3l4456y1/kfGGGTetbvOMgzWw/wq2fLj0zxve1bvw89H1bK8Wv5wdUYAb7269x6MsU19FfNmxz6vTOqoSeWWujSMFYHhiy+3JPhwOEe7lnVl1T90SDb+9ma7UOvOQqAVy/ofybpXm/UyF5vtMurjymsxWedY+fB8uuqDGRka4rf/O25BecOdWc43FO+XFJcQy83SzaTzbLjwGEOdWVCn5fmpRa6NIzg+uV/v2wlz2wtHJPut0x/WLTO+UmzJ+Qn8PglmBNmTRjw9ZY/tY1v/XYDAKteKJwAdKg7w+2PF+/zUrmZE0aV1Oc/94tVdEwJr4sD/Sb7oEzWcfrn72XBtLH88ur+V6CU5qIWujSk4mQOpbXjsd5Y71s/dGb+nN/KPffYaTy49Lx+XyM4iam4dWxYxZtT+EMnzz22r5V/xMRRJS3slS8c4Jf9lHGmjOl/P9Q/9yYr+R9s63YMfwVKaSxK6FLXHtm0h1Ouu4d9nd0F58OG8PUWzQ49YuJIAEa2lo5UyT1fOtIkaPuBvtLNpKLkW+leoJCbCTpxdGto3bxYfx2tLQOs4+JPVrr4q/dXHJs0FyV0qWtf/tVa9nb28NjzewvOfyBkOGLxhJrgmi13ffwcbnz7qf2+1lXnL2DO5L4kb/QNL/ze+17JZ96wKP9c8dICF584gyWvnEOxBdPGMnF0Gz2ZbOia7aOKPmz6+5wIW5jrhreelD+udJy6NC8ldKlrT3gTgIo3gnjmhdJFrYpbzRNG9bWqF84YzyUnlS60FfSBc+Zx/9+dl0/i63YcZPzIVhbNHM/MCaMKxrSvCpRjvv++V/L1d5zG9W85qeR7dnb38tu1O5kyNrxcMlCrOyg4bNH354E1YaaN678kI81PCV3qml+nLp4s87v1u/PHMyfkSiulQwwHN3xvTFuuZNHmdZw+sXkfE0a1sitkNMsNd63JH588Z2L++OoLcsMiJ4/JfZj4y9k+v6eTc0JG1twQ8iFQTtjU/+CaLQumj634e0lzUkKXhhC2porvnd4Suod7ChN6JWXuV3ZMyh/7+4QGW82ZbJb50/oS5d0ff3XB17emraBz82PnL2DT9Zfw2Gcu4MyjphRc+y4vztceNz1/7qITZ5asngi5ktI9nyh8rYFa8+NDJitJsiihS0Po6WfvTb+ResUPV7A/sOpgJQlucUfp5Jyz5/eNRunKZAs2yyheI6W/hbD8jZ2H4vhZ40kVrZjYElJyCdICi6Jx6NIQ+pvO7iful7oy+R2M3nzqLGZ4pZj+/M3rjuU1x7TnyzYAn754Id/93UbedMosVr1woCCJTxkbPpknam88eRabdhcuGlZutUVf+9iB/3+luamFLg3h0z97uuxzbYH1Ws654T4AXl1mlcVi6ZRxxlFTODIwoaclnWLq2BGsffEl1rz4Em0tfSNR/Dp7NXVMGY2ZUZy/Byq5aIVF0W+ANJS3nDq75NzCmeNKzg1m9EiYzu5Mflx38DsNZmjgm0+ZNaTX9l9jsCWXgfYbleanhC4N5U8XFra8500dE9oyHSj5DeT4I8bT2Z1bC+Wk2QMvExDmVUeFL541kOe81RmLa+IDfUi1qoWeePoNkIbyctF0+2suPo7Zk0aXXDfc1mpLKkWntw75iCEmyuJx4/7wxnedGb6xdbHibVEHKve0hSwVXLwHqzQ3dYpKXWtJWUGHaPu4EaHD/E6cNYGnt/ZNNmqpYB30/pjBjpdy489HtIQvHTCQY4rGhU8bPzI09nKKt7obFbLZRlDY2u8HymxrJ81JLXSpa5XWkYtb5DsOlF9CtxIPPrebg97ys60tQ2vtD3ejiTEDJPBifu39rPl94993Hewud7k0ISV0qW8V1pGLW6fb+lkTfbDCptz7HrnmtWWf684MvKNRmIUzcp2808aP5H+uPHvA6x++5vz82uo//+hZfGXJKfnnwrbQk+alkovUteL0XW4sdnHHaPGiV8NRrrX/q6tfTXs/66cMdYijv0E1wIkVdMhOG9c3/jy4DAGAG+TyB9LY1EKXulY8TLBcbby4hX7RiTMii+G3RXt8+gaqqBx/xHgg1+L+6UfO7P/ifnzujSfwnx941aC+5p/edAKg7eiSRi10qWslCb1MC31v0Xrp40dFt65J2OgRGLikkkrZoDpBy/HXgBmMU+bk1qjpbw0caT5qoUtdKym5lKmhP+4ts+sbbsnl9cf3LaD11tPCJwjNGmCDjDj596mf/TKkCSmhS10rHiky0HomAM99/uLQIXyD8bbFfZtVBFdHDJo0Jrp1XaJoyQf5o4OKN+KQ5qaELnXrpcM9dBZNJKpkBmgUO/cE9wsd7pj2OPj3QCWXZGm831RJjLAx1OWS9acvXghEVwbZE7Kphe8nHz6Tryw5OZLXCfPAJ/902N/D/0tGJZdkUaeoxO72J7ayff9hPvSaowvOh7Uuy635feyM3IgSf/u44Rrdz5DDV4asoR6lsKUMBiuVT+jK6EmihC6xu+qWJwBKEroLSejl8pO/3krxvqJDNaK19n+8fvOdp7J5z8uRfK+0qYWeREroUjce2rCbdTsO0pIylpw+l7Dyb7masD+xyJ+uP1xR1OEH68IT+t/EejD8+IvXg5HmVlFCN7MLga8AaeA7zrnrQ655G3AtuZ15n3TOvT3COCUB/s9Nf8gfLzl9bj4ZTRs3Ir9QVrmRJT3emPA/Fu3yM1TFa8g0mnxCVxM9UQZM6GaWBm4ELgC2AI+Y2TLn3KrANQuATwFnOef2mtm0agUsyeHPcvzcG0/gguOmk8m6srvy+AnsuJDNLobCb6CXG7JY7/IlFzXQE6WSQuHpwHrn3AbnXDdwC3BZ0TUfBG50zu0FcM7tiDZMSZqOpcvp8VqXbekUqZT1u8XaSG8i0VHtY8teMxgTR+f+Eoiqk7XW0ml1iiZRJSWXWcDmwOMtQPHCEscAmNnvyJVlrnXO3VX8jczsCuAKgLlz5w4lXkkQf1JRJdvJnTBrAt94x6mce2w0fxyecdQUbnz7qZx/XGP+salO0WSKqiu/BVgAnAtcDnzbzCYWX+Scu8k5t9g5t7i9vbJNfKU5fes3zw24m86+ztzmDJVuJ3fRiTMH3ARiMC45aWa+5d9oNLEomSp5p2wF5gQez/bOBW0BljnnepxzG4G15BK8SAnnHP9852ou+sr9/V7XoqQ0ZH5C12qLyVJJQn8EWGBm88ysDVgCLCu65ufkWueY2VRyJZgN0YUpzaTSseL+deNGanTtYPmduhq2mCwDJnTnXAa4ErgbeBa41Tm30syuM7NLvcvuBnab2SrgPuBvnXO7qxW0NLZKt2bzk1GjDyGMg5mRThnZiCZaSWOoqOnjnLsDuKPo3GcDxw642vsn0q+eCnvq/GQUxySfZpA2G/a+ptJYtDiX1Fywrttf+cVvoSuhD00qpf6HpFFCl5rrCYyN7sr0hl7TMWV0Ptmr5DI0LalUZGvbSGNQQpeaG6iFPn38CMws37pUC31oDNiyN5qlEKQxKKFLzQUTetiwut5sbqcdv9SeVgt9SF7qyjBpdHS7Kkn9U0KXmguWXMI67drSRjbb1yla4bwiKTJj/EjV0BNGbxWpuYOH+5a4Ddvzcvbk0bkWukouw5JOmab+J4wSusQq0+uYPn5Ewbn508bSm+2rr6vkMjQa5ZI8SuhSc8FWeW/WlbQi02b0ZrP5ZJRSC31IcvdRCT1JlNCl5go6RQOJ25crFTi10IcplTJN/U8YJXSpuWBHaMZL3O8588j8ueKErhb60KRNU/+TRgldaq4gofc6sllHKmVcfcExHDdzPC1ey1Lj0IfH/2CU5FBCl5rL9BYOW+x1jrQZHzt/AXdedU6uVBCoravkMjSpwOQsSQYldKm5YAt9b2c3WecKWuEtXkLv6xSteYhNQS305NFbRWou2CnamkqRzeaWe/XlWpZ916mFPjS5TtG4o5BaUkKXmvvOA317n/gTiNKB30R/pyJ/mV3V0IcmbahTNGGU0KXmHn9+X/7YH80SbIWnAgndrLD1LpVTySV5lNAlVn4rPFVUQwfoymRVbhmGlGkcetIooUusujJeWSWQuA9159ZI377/sMagD4O2oEseJXSJVXemtIV+dPuY3HO9aqEPR1ozRRNHCV1i1R3S8dnq9ZB2Z7LqEB2GlGaKJo4SusSqJ6Tk4ifx7kwW5fOhUws9eZTQpeYmjGrldYumA30t9GBlJd8p2qsW+nD8evUOntl6IO4wpIaU0KXmMr1ZRramgb4aejBxB1voSugilVNCl5rryTpGtvbVyYGiqf/+c72k1CkqUjEldKm5TG+WUV4L3R+2mAqroavkIjIoSuhSU9msI+vIl1y6wlro6WCnqBL6UL37zCOZOLo17jCkhlriDkCSZeULuU66+9ftAuD7D24CYPW2vs47f7r6iwe6ahtck0lpC7rEUQtdaur+9TsBWLWtcPTFhl2H8sc92qo+EpopmjxK6FJTmwKJ2+8Yhb6hitA3sUiGR+PQk0fvHKkpv2V+8pyJ+dEsAJ974wn5Y9XNo5GbKRp3FFJLSuhSU51duYW3Zk8aVdAROnvS6PyxBrZEI51CLfSEUUKXmlrcMQmAc4+dVnZIYjC5y9Cl1SmaOEroUlOnzM0l9LPnTy2b0GdMGFnLkJqWv4KlOkaTQwldasofwdKaNna+pGGJ1eQveKayS3IooUtN+VP9W1sq+9VbfOSkaobT1PwWusouyaGELjWzcdch/nH5swC0BYYmtvWT3E+cPaHqcTUrv6SVVQs9MZTQpWbe/d2H8sfBoYkj+0noh3t6qxpTM8uXXNRCTwwldKmZl7v7knOwQ7S/dKMRL0PX1ykacyBSMxUldDO70MzWmNl6M1vaz3VvMTNnZoujC1Gaxa6D3fnj4ACX/lqQmmQ0dN4aZ+oUTZABE7qZpYEbgYuARcDlZrYo5LpxwFXAQ8XPiRSzQKLOhCT0t5w6G8hNjpGhSatTNHEqebucDqx3zm1wznUDtwCXhVz3OeALwOEI45ME8Ee+BHVMyZVaJo5qq3U4TSOlTtHEqSShzwI2Bx5v8c7lmdmpwBzn3PL+vpGZXWFmK8xsxc6dOwcdrDS2GeMLJwwF128p9qHXHM3n3ngCbzltdrXDalrqFE2eYf9Ba2Yp4EvAXw90rXPuJufcYufc4vb29uG+tDSY1pZcgrnkxJkAnDN/atlr21pSvOuMI7Vj0TBoHHryVJLQtwJzAo9ne+d844ATgP81s03AGcAydYxKsTOPmgLAR/90PoCSdZX5LXSVXJKjkoT+CLDAzOaZWRuwBFjmP+mc2++cm+qc63DOdQB/AC51zq2oSsTSsF7Yl+teWXTEeEDrnlebOkWTZ8B3lHMuA1wJ3A08C9zqnFtpZteZ2aXVDlCaQ6Y3ywPrdxWc04jE6lKnaPJUtKeoc+4O4I6ic58tc+25ww9Lms2hrtIZn36eGVHhui4yOH2dojEHIjWjd5LUxP6Xe0rOtXozX46cotmg1eCXXDKaKpoYFbXQRYbrcKa0hT5l7Aj+8Y0ncN7CaTFE1PzSmvqfOEroUhM9Zf7uf+cZR9Y4kuTw+5w19T85VHKRmujpVVKptZQmFiWOErrURLkWulSP1kNPHiV0qYmekPVapLo09T95lNClJrrVQq85bRKdPEroUhM7tCF0zfkll72dpUNGpTkpoUtN+JOH/uZ1x8QcSXJo4lby6CctNeGveX7ZybMGuFKiMrotDWjYYpIooUtN+MMWtSBX7aRVQ08cvbuk6nqzjt0HczV0f7q/VF9+tUW10BNDM0Wl6o7+dN+6bq2q59aMls9NHr27pKbaVHKpGY1DTx69u6SmWrRLUc2ohZ48SuhSUy1qodeMNrhIHr27RJqUNrhIHiV0kSaV0vK5iaOELtKk/Ba6xqEnhxK6SJNq8Zro6hRNDiV0kSaVL7kooSeGErpIk9JM0eRRQhdpUtqCLnmU0EWalBbnSh4ldKmqjAZBxyY/Dl0ll8RQQpeq6tJeorFJpQwztdCTRAldqmrJTX/IH8+bOibGSJIpbUZGCT0xlNClqp7euj9//O13nxZjJMmUSplKLgmihC5VE/xT/7yF05g/bVyM0SRT2kwllwRRQpeqeX5PZ/74b19/bIyRJFc6ZVqcK0GU0KVqgh2ix80cH2MkyZUyLZ+bJEroUjW7vH1EJT65FroSelIooUvVaG+i+KVTKXWKJogSulRNj1qGsUunoLdXP4ekUEKXqvFniV5z8XExR5JcadOwxSRRQpeq6fFahn8yf0rMkSSXmfHH3YfiDkNqRAldqiaTzbXQW7UxdGx2Hexiwqi2uMOQGtE7TarmUFcGgJaUukfjsmD6WJxKLolRUUI3swvNbI2ZrTezpSHPX21mq8zsKTO718yOjD5UaTSf/OnTgFrocdJaLsky4DvNzNLAjcBFwCLgcjNbVHTZ48Bi59xJwG3ADVEHKo2rJa0WelzSKdPEogSppOl0OrDeObfBOdcN3AJcFrzAOXefc86f5/0HYHa0YUojS6vkEpuWVIqMhi0mRiUJfRawOfB4i3eunPcDd4Y9YWZXmNkKM1uxc+fOyqOUhnT2/KkAtI8dEXMkyaWZoskSaXHTzN4JLAa+GPa8c+4m59xi59zi9vb2KF9a6sztT2zlgfW7mDKmDTO10OPSkrb8aCNpfi0VXLMVmBN4PNs7V8DMXgtcA7zGOadFPBLuqlueAGD3oe54A0m4lKmFniSVtNAfARaY2TwzawOWAMuCF5jZKcC3gEudczuiD1NEhqIlpVEuSTJgQnfOZYArgbuBZ4FbnXMrzew6M7vUu+yLwFjgJ2b2hJktK/PtRKSGVENPlkpKLjjn7gDuKDr32cDxayOOS0Qi0JJWQk8SzfiQyPVoi5y6kU6llNATRAldIrevsyfuEMSjGnqyKKFL5NRCrx8a5ZIsSugSuWBCf+cZc2OMRHItdH3AJkVFnaIig+En9K9dfgp/9oojYo4m2dJpozujhJ4UaqFL5LozuT/xtcpi/LozWfaqTyMx9I6TyPkt9LYWTfmPW8qgrUVv86TQT1oit7czN92/JaVfr7hNGqPdipJE7ziJnL/8tpbNjV/ajKxGuSSGErpE7rmdBwGYPl7L5sYtnTJ6tcFFYiihS+Q27MrtMt8+bmTMkUjKDOfQvqIJoYQuVTFhVCsTRrXGHUbi+WUvTS5KBiV0iVxPJsvotnTcYQiBhK4WeiIooUvkenqzGoNeJ1LeblGaLJoMetdJ5Hqyjta0RrjUA/9zVS30ZFBCl8j1ZNRCrxd+C1019GTQu04i9+KBw0rodcKvoWssejJocS6JXK9zvHQ4E3cYQm61RUBroieEmlESuWwW5k0dE3cYAqT8Frpq6ImghC6RW7XtAGNG6I+/epBWDT1RlNAlciNaUvnOOIlXShOLEkUJXSJ14HAPXZksR7er5FIP/Ba6Si7JoIQukVq59QAA40Zq2n890NT/ZFFCl0j5m1ucPGdCzJEIqFM0aZTQJVJ+Qtc49PrQ1ykacyBSE3rXSaSU0OtLfuq/Si6JoHedRKq7VxtE15OUOkUTRe86iVRPxtsgWgm9LqhTNFn0rpNIbd7bCUBri8ah14OU1kNPFCV0iZSRSyATR2m3+XqQH4euFnoiKKFLpP64J7ef6CjtWFQXVHJJFiV0idR/P7Y17hAkQAk9WZTQRZqY9hRNFiV0kSbmD1t8178/zLInX4g5Gqk2JXSJ3IRRWselXvgtdICP/fjxGCORWlBCl8h0dud2KXrTKbNijkR86aJljJ/esj+mSKQWlNAlMn/YsBtQC72ezJw4suDxTfdviCkSqQVtKyOR6OnNsmb7QQAuPfmImKMR39SxI9h0/SUA/NnXHuDJzfvozbqCUow0j4pa6GZ2oZmtMbP1ZrY05PkRZvZf3vMPmVlH5JFKXfuXe9byhbtWYwaTRmtSUT2aMraN5/d08r3fbYw7FKmSARO6maWBG4GLgEXA5Wa2qOiy9wN7nXPzgX8FvhB1oFLftu9/mfZxI7jtw3/C5DFK6PXo+jefBMD2/YdjjkSqpZKSy+nAeufcBgAzuwW4DFgVuOYy4Frv+Dbg38zMnIt+8Outj2zm26oD1p1t+w8zZ/JoTjtyUtyhSBkzJoykfdwIfvLoFn6zdmfc4STax85fwJ+9IvrSZCUJfRawOfB4C/Cqctc45zJmth+YAuwKXmRmVwBXAMydO3dIAU8c3cqC6WOH9LVSPQumj+X8hdPjDkMG8NFzj+bhTXviDiPxqjVwoKados65m4CbABYvXjyk1vvrjp/B646fEWlcIknx3rPm8d6z5sUdhlRJJZ2iW4E5gcezvXOh15hZCzAB2B1FgCIiUplKEvojwAIzm2dmbcASYFnRNcuA93jHbwV+XY36uYiIlDdgycWriV8J3A2kge8651aa2XXACufcMuDfgR+Z2XpgD7mkLyIiNVRRDd05dwdwR9G5zwaODwN/Hm1oIiIyGJr6LyLSJJTQRUSahBK6iEiTUEIXEWkSFtfoQjPbCfxxiF8+laJZqHVIMUZDMUaj3mOs9/igfmI80jnXHvZEbAl9OMxshXNucdxx9EcxRkMxRqPeY6z3+KAxYlTJRUSkSSihi4g0iUZN6DfFHUAFFGM0FGM06j3Geo8PGiDGhqyhi4hIqUZtoYuISBEldBGRJtFwCX2gDaur+LpzzOw+M1tlZivN7Crv/GQz+6WZrfP+O8k7b2b2VS/Op8zs1MD3eo93/Toze0+51xxGrGkze9zMfuE9nudt3r3e28y7zTtfdnNvM/uUd36Nmb0+4vgmmtltZrbazJ41szPr7T6a2Se8n/MzZvZjMxsZ9300s++a2Q4zeyZwLrL7ZmanmdnT3td81cwsohi/6P2snzKzn5nZxMBzofen3Pu83M9guDEGnvtrM3NmNtV7HMt9HDLnXMP8I7d873PAUUAb8CSwqEavPRM41TseB6wlt2n2DcBS7/xS4Ave8cXAnYABZwAPeecnAxu8/07yjidFHOvVwH8Cv/Ae3wos8Y6/CXzEO/5L4Jve8RLgv7zjRd69HQHM8+55OsL4fgB8wDtuAybW030kt6XiRmBU4P69N+77CLwaOBV4JnAusvsGPOxda97XXhRRjK8DWrzjLwRiDL0/9PM+L/czGG6M3vk55JYJ/yMwNc77OOTfkVq9UCTBwpnA3YHHnwI+FVMstwMXAGuAmd65mcAa7/hbwOWB69d4z18OfCtwvuC6COKaDdwLnAf8wvul2hV4Q+XvoffLe6Z33OJdZ8X3NXhdBPFNIJcsreh83dxH+vbInezdl18Ar6+H+wh0UJgsI7lv3nOrA+cLrhtOjEXPvQm42TsOvT+UeZ/397scRYzkNrh/BbCJvoQe230cyr9GK7mEbVg9q9ZBeH9SnwI8BEx3zm3zntoO+Dsll4u12v8PXwb+Dsh6j6cA+5xzmZDXK9jcG/A3965mjPOAncD3LFcW+o6ZjaGO7qNzbivw/4DngW3k7suj1Nd99EV132Z5x9WMFeAvyLVahxJjf7/Lw2JmlwFbnXNPFj1Vr/cxVKMl9NiZ2Vjgp8DHnXMHgs+53EdybONAzewNwA7n3KNxxVCBFnJ/7n7DOXcKcIhcqSCvDu7jJOAych8+RwBjgAvjiqdScd+3gZjZNUAGuDnuWILMbDTwaeCzA11b7xotoVeyYXXVmFkruWR+s3Puv73TL5rZTO/5mcCOAWKt5v/DWcClZrYJuIVc2eUrwETLbd5d/HrlNveuZoxbgC3OuYe8x7eRS/D1dB9fC2x0zu10zvUA/03u3tbTffRFdd+2esdVidXM3gu8AXiH98EzlBh3U/5nMBxHk/vwftJ778wGHjOzGUOIsar3cUC1qu1E8Y9c624DuZvvd5YcX6PXNuCHwJeLzn+Rwk6pG7zjSyjsTHnYOz+ZXA15kvdvIzC5CvGeS1+n6E8o7Ej6S+/4oxR25t3qHR9PYWfVBqLtFL0fONY7vta7h3VzH4FXASuB0d7r/gD4q3q4j5TW0CO7b5R25l0cUYwXAquA9qLrQu8P/bzPy/0Mhhtj0XOb6Kuhx3Yfh/T/VasXiizgXK/zWnK94NfU8HXPJvfn7FPAE96/i8nV9e4F1gG/CvxQDbjRi/NpYHHge/0FsN77974qxXsufQn9KO+XbL33hhjhnR/pPV7vPX9U4Ouv8WJfQ8S99MDJwArvXv7ce0PU1X0E/gFYDTwD/MhLOrHeR+DH5Gr6PeT+0nl/lPcNWOz9/z4H/BtFHdfDiHE9uXqz/7755kD3hzLv83I/g+HGWPT8JvoSeiz3caj/NPVfRKRJNFoNXUREylBCFxFpEkroIiJNQgldRKRJKKGLiDQJJXQRkSahhC4i0iT+PwfGCYTRK72/AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "## DO NOT CHANGE THIS CELL\n",
    "rm=movingaverage(rList, 100)\n",
    "plot(rm)\n",
    "criterion = len(np.where(rm >= .78)[0]) >0\n",
    "if criterion:\n",
    "    print(\"Solved after %d episodes.\" % np.where(rm >= .78)[0][0])\n",
    "else:\n",
    "    print(\"Unsolved Yet!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}