blackjack_dqn.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Copy of blackjack_dqn.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/mia1996/rlcard-tutoirial/blob/master/blackjack_dqn.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "miBl4S8JARzX"
      },
      "source": [
        "\n",
        "\n",
        "# <a href='https://github.com/datamllab/rlcard'> <center> <img src='https://miro.medium.com/max/1000/1*_9abDpNTM9Cbsd2HEXYm9Q.png' width=500 class='center' /></a> \n",
        "\n",
        "## **Deep-Q Learning on Blackjack**\n",
        "This example is to use Deep-Q learning to train an agent on Blackjack. We aim to use this example to show how reinforcement learning algorithms can be developed and applied in our toolkit. To be self-contained, we first install RLCard."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zQ8CiXAJjQGi",
        "outputId": "bda614ff-35d4-4ba3-d171-374c8142af0d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "!pip install rlcard[torch]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting rlcard[torch]\n",
            "  Downloading rlcard-1.0.7.tar.gz (268 kB)\n",
            "\u001b[?25l\r\u001b[K     |█▏                              | 10 kB 16.1 MB/s eta 0:00:01\r\u001b[K     |██▍                             | 20 kB 18.6 MB/s eta 0:00:01\r\u001b[K     |███▋                            | 30 kB 21.5 MB/s eta 0:00:01\r\u001b[K     |████▉                           | 40 kB 20.1 MB/s eta 0:00:01\r\u001b[K     |██████                          | 51 kB 16.5 MB/s eta 0:00:01\r\u001b[K     |███████▎                        | 61 kB 18.6 MB/s eta 0:00:01\r\u001b[K     |████████▌                       | 71 kB 16.1 MB/s eta 0:00:01\r\u001b[K     |█████████▊                      | 81 kB 7.0 MB/s eta 0:00:01\r\u001b[K     |███████████                     | 92 kB 7.6 MB/s eta 0:00:01\r\u001b[K     |████████████▏                   | 102 kB 8.3 MB/s eta 0:00:01\r\u001b[K     |█████████████▍                  | 112 kB 8.3 MB/s eta 0:00:01\r\u001b[K     |██████████████▋                 | 122 kB 8.3 MB/s eta 0:00:01\r\u001b[K     |███████████████▉                | 133 kB 8.3 MB/s eta 0:00:01\r\u001b[K     |█████████████████               | 143 kB 8.3 MB/s eta 0:00:01\r\u001b[K     |██████████████████▎             | 153 kB 8.3 MB/s eta 0:00:01\r\u001b[K     |███████████████████▌            | 163 kB 8.3 MB/s eta 0:00:01\r\u001b[K     |████████████████████▊           | 174 kB 8.3 MB/s eta 0:00:01\r\u001b[K     |██████████████████████          | 184 kB 8.3 MB/s eta 0:00:01\r\u001b[K     |███████████████████████▏        | 194 kB 8.3 MB/s eta 0:00:01\r\u001b[K     |████████████████████████▍       | 204 kB 8.3 MB/s eta 0:00:01\r\u001b[K     |█████████████████████████▋      | 215 kB 8.3 MB/s eta 0:00:01\r\u001b[K     |██████████████████████████▉     | 225 kB 8.3 MB/s eta 0:00:01\r\u001b[K     |████████████████████████████    | 235 kB 8.3 MB/s eta 0:00:01\r\u001b[K     |█████████████████████████████▎  | 245 kB 8.3 MB/s eta 0:00:01\r\u001b[K     |██████████████████████████████▌ | 256 kB 8.3 MB/s eta 0:00:01\r\u001b[K     |███████████████████████████████▊| 266 kB 8.3 MB/s eta 0:00:01\r\u001b[K     |████████████████████████████████| 268 kB 8.3 MB/s \n",
            "\u001b[?25hRequirement already satisfied: numpy>=1.16.3 in /usr/local/lib/python3.7/dist-packages (from rlcard[torch]) (1.21.5)\n",
            "Requirement already satisfied: termcolor in /usr/local/lib/python3.7/dist-packages (from rlcard[torch]) (1.1.0)\n",
            "Requirement already satisfied: torch in /usr/local/lib/python3.7/dist-packages (from rlcard[torch]) (1.10.0+cu111)\n",
            "Collecting GitPython\n",
            "  Downloading GitPython-3.1.27-py3-none-any.whl (181 kB)\n",
            "\u001b[K     |████████████████████████████████| 181 kB 41.9 MB/s \n",
            "\u001b[?25hCollecting gitdb2\n",
            "  Downloading gitdb2-4.0.2-py3-none-any.whl (1.1 kB)\n",
            "Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from rlcard[torch]) (3.2.2)\n",
            "Collecting gitdb>=4.0.1\n",
            "  Downloading gitdb-4.0.9-py3-none-any.whl (63 kB)\n",
            "\u001b[K     |████████████████████████████████| 63 kB 922 kB/s \n",
            "\u001b[?25hCollecting smmap<6,>=3.0.1\n",
            "  Downloading smmap-5.0.0-py3-none-any.whl (24 kB)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.7/dist-packages (from GitPython->rlcard[torch]) (3.10.0.2)\n",
            "Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->rlcard[torch]) (3.0.7)\n",
            "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->rlcard[torch]) (0.11.0)\n",
            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->rlcard[torch]) (1.4.0)\n",
            "Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->rlcard[torch]) (2.8.2)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib->rlcard[torch]) (1.15.0)\n",
            "Building wheels for collected packages: rlcard\n",
            "  Building wheel for rlcard (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for rlcard: filename=rlcard-1.0.7-py3-none-any.whl size=325373 sha256=dd6ec050c7312061d299e960ea393b346a9278484233bb6c92c0f87c7b1d4d90\n",
            "  Stored in directory: /root/.cache/pip/wheels/8a/90/bd/bc402a48ca90970c9a7c2c4387dcb885fdf6073ec231a605ad\n",
            "Successfully built rlcard\n",
            "Installing collected packages: smmap, gitdb, rlcard, GitPython, gitdb2\n",
            "Successfully installed GitPython-3.1.27 gitdb-4.0.9 gitdb2-4.0.2 rlcard-1.0.7 smmap-5.0.0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now we import `rlcard` and `DQNAgent`. The `DQNAgent` will learn how to win the game."
      ],
      "metadata": {
        "id": "2bt-JVoXyTwM"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4_A_Br3Jj0xW"
      },
      "source": [
        "import rlcard\n",
        "from rlcard.agents import DQNAgent"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N2ltkfinYmiU"
      },
      "source": [
        "Let's create the Blackjack environment and take a look at it."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "l_8Kuf47kghG",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "177c5020-2885-465b-9648-40c1ddcd6c06"
      },
      "source": [
        "env = rlcard.make(\"blackjack\")\n",
        "print(\"Number of actions:\", env.num_actions)\n",
        "print(\"Number of players:\", env.num_players)\n",
        "print(\"Shape of state:\", env.state_shape)\n",
        "print(\"Shape of action:\", env.action_shape)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Number of actions: 2\n",
            "Number of players: 1\n",
            "Shape of state: [[2]]\n",
            "Shape of action: [None]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Blackjack is a very simple game with only two possible actions. There is only one player. It's time for us to train our DQN to master this game! We first create a DQNAgent."
      ],
      "metadata": {
        "id": "W2vJYyyqzVj3"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8bqfMncnJTYU"
      },
      "source": [
        "agent = DQNAgent(\n",
        "    num_actions=env.num_actions,\n",
        "    state_shape=env.state_shape[0],\n",
        "    mlp_layers=[64,64],\n",
        ")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Here, we use a 64-64 deep neural network to learn. Then we pass the DQNAgent to the environment."
      ],
      "metadata": {
        "id": "tJVGcqYhzvyG"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "env.set_agents([agent])"
      ],
      "metadata": {
        "id": "5JEwJb_Oztut"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now we are ready to train! We first import some useful classes and functions for training."
      ],
      "metadata": {
        "id": "zy_sHLKf0TVl"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GhgGYyick13x"
      },
      "source": [
        "from rlcard.utils import (\n",
        "    tournament,\n",
        "    reorganize,\n",
        "    Logger,\n",
        "    plot_curve,\n",
        ")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Then start the training and log the performance with our `Logger`. The script below will train DQN for 1000 epochs (i.e., 1000 games). Usually, the agent will become stronger if trained longer."
      ],
      "metadata": {
        "id": "w2bT6ETj04Hq"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rYAj8Q22k5e2",
        "outputId": "e43b29b2-553a-4c17-e3b3-166910af9c42",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "with Logger(\"experiments/leduc_holdem_dqn_result/\") as logger:\n",
        "    for episode in range(1000):\n",
        "\n",
        "        # Generate data from the environment\n",
        "        trajectories, payoffs = env.run(is_training=True)\n",
        "\n",
        "        # Reorganaize the data to be state, action, reward, next_state, done\n",
        "        trajectories = reorganize(trajectories, payoffs)\n",
        "\n",
        "        # Feed transitions into agent memory, and train the agent\n",
        "        for ts in trajectories[0]:\n",
        "            agent.feed(ts)\n",
        "\n",
        "        # Evaluate the performance.\n",
        "        if episode % 50 == 0:\n",
        "            logger.log_performance(\n",
        "                env.timestep,\n",
        "                tournament(\n",
        "                    env,\n",
        "                    10000,\n",
        "                )[0]\n",
        "            )\n",
        "\n",
        "    # Get the paths\n",
        "    csv_path, fig_path = logger.csv_path, logger.fig_path"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "----------------------------------------\n",
            "  timestep     |  552308\n",
            "  reward       |  -1.0\n",
            "----------------------------------------\n",
            "\n",
            "----------------------------------------\n",
            "  timestep     |  571353\n",
            "  reward       |  -1.0\n",
            "----------------------------------------\n",
            "INFO - Step 100, rl-loss: 1.7699662446975708\n",
            "INFO - Copied model parameters to target network.\n",
            "INFO - Step 142, rl-loss: 1.4145885705947876\n",
            "----------------------------------------\n",
            "  timestep     |  590632\n",
            "  reward       |  -0.8554\n",
            "----------------------------------------\n",
            "INFO - Step 210, rl-loss: 1.2130974531173706\n",
            "----------------------------------------\n",
            "  timestep     |  608418\n",
            "  reward       |  -0.7016\n",
            "----------------------------------------\n",
            "INFO - Step 280, rl-loss: 1.1429738998413086\n",
            "----------------------------------------\n",
            "  timestep     |  625811\n",
            "  reward       |  -0.6066\n",
            "----------------------------------------\n",
            "INFO - Step 348, rl-loss: 0.9004282355308533\n",
            "----------------------------------------\n",
            "  timestep     |  642940\n",
            "  reward       |  -0.4844\n",
            "----------------------------------------\n",
            "INFO - Step 413, rl-loss: 0.6521245837211609\n",
            "----------------------------------------\n",
            "  timestep     |  659822\n",
            "  reward       |  -0.365\n",
            "----------------------------------------\n",
            "INFO - Step 480, rl-loss: 0.840827465057373\n",
            "----------------------------------------\n",
            "  timestep     |  676163\n",
            "  reward       |  -0.085\n",
            "----------------------------------------\n",
            "INFO - Step 553, rl-loss: 0.5684747099876404\n",
            "----------------------------------------\n",
            "  timestep     |  691722\n",
            "  reward       |  -0.0584\n",
            "----------------------------------------\n",
            "INFO - Step 624, rl-loss: 0.8044297099113464\n",
            "----------------------------------------\n",
            "  timestep     |  706931\n",
            "  reward       |  -0.0619\n",
            "----------------------------------------\n",
            "INFO - Step 691, rl-loss: 0.5927526354789734\n",
            "----------------------------------------\n",
            "  timestep     |  721525\n",
            "  reward       |  -0.0763\n",
            "----------------------------------------\n",
            "INFO - Step 757, rl-loss: 0.8437865376472473\n",
            "----------------------------------------\n",
            "  timestep     |  736157\n",
            "  reward       |  -0.0601\n",
            "----------------------------------------\n",
            "INFO - Step 828, rl-loss: 0.561453640460968\n",
            "----------------------------------------\n",
            "  timestep     |  750661\n",
            "  reward       |  -0.0781\n",
            "----------------------------------------\n",
            "INFO - Step 899, rl-loss: 0.7222794890403748\n",
            "----------------------------------------\n",
            "  timestep     |  764648\n",
            "  reward       |  -0.0693\n",
            "----------------------------------------\n",
            "INFO - Step 972, rl-loss: 0.4520031809806824\n",
            "----------------------------------------\n",
            "  timestep     |  778987\n",
            "  reward       |  -0.0774\n",
            "----------------------------------------\n",
            "INFO - Step 1040, rl-loss: 0.5117819309234619\n",
            "----------------------------------------\n",
            "  timestep     |  793088\n",
            "  reward       |  -0.0543\n",
            "----------------------------------------\n",
            "INFO - Step 1100, rl-loss: 0.4985390901565552\n",
            "INFO - Copied model parameters to target network.\n",
            "INFO - Step 1116, rl-loss: 0.6727216243743896\n",
            "----------------------------------------\n",
            "  timestep     |  807093\n",
            "  reward       |  -0.0829\n",
            "----------------------------------------\n",
            "INFO - Step 1183, rl-loss: 0.681004524230957\n",
            "----------------------------------------\n",
            "  timestep     |  820958\n",
            "  reward       |  -0.0615\n",
            "----------------------------------------\n",
            "INFO - Step 1256, rl-loss: 0.6855602264404297\n",
            "----------------------------------------\n",
            "  timestep     |  834715\n",
            "  reward       |  -0.0823\n",
            "----------------------------------------\n",
            "INFO - Step 1329, rl-loss: 0.5815291404724121\n",
            "----------------------------------------\n",
            "  timestep     |  848487\n",
            "  reward       |  -0.0985\n",
            "----------------------------------------\n",
            "INFO - Step 1393, rl-loss: 0.6741513013839722\n",
            "Logs saved in experiments/leduc_holdem_dqn_result/\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now we plot the learning curves to monitor how the agent gets improved!"
      ],
      "metadata": {
        "id": "rbIiQS3K2R0u"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "06n2QSTDIqb0",
        "outputId": "4536c8af-e4ca-489e-c7fd-4d32c839f81c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 279
        }
      },
      "source": [
        "plot_curve(csv_path, fig_path, \"DQN\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAEGCAYAAACdJRn3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU9b3/8dcne0JCNkgI+yKrgGAAwbYKdbfWrda23rrUWmqtXbRabW0t3va2aqu9ba+9rbfXSvX+LnVpK2hbVCTVuiGx7BBA1rBkIxACZP/+/pgDN9AAmZDJmTnzfj4e88iZM+fM+Xw4w3zmnO/3fI855xAREQlHgt8BiIhI7FHxEBGRsKl4iIhI2FQ8REQkbCoeIiIStiS/A+huffr0cUOHDg1rnQMHDtCrV6/IBOSDIOWjXKJXkPIJUi7QtXxKS0urnXN9O7t84IrH0KFDWbp0aVjrlJSUMHPmzMgE5IMg5aNcoleQ8glSLtC1fMxsazjL67SViIiETcVDRETCpuIhIiJhC1ybR0eam5spLy+noaGhw9ezs7NZu3ZtD0fV/dLS0hg4cKDfYYhIHIiL4lFeXk5WVhZDhw7FzP7p9f3795OVleVDZN3HOUdNTQ3l5eV+hyIicSAuTls1NDSQn5/fYeEICjMjPz//uEdXIiLdKS6KBxDownFYPOQoItEhLk5biUj8qm9sYdHaCppa2rh0QhG9UvW11x30r9hDEhMTmTBhAs3NzSQlJXHDDTdwxx13kJAQOvj7+9//zp133kldXR3OOb72ta9x2223ATBnzhwefvhhtmzZQkFBAQCZmZnU19f7lo+EOOdYtaOOF1fupHp/E71SE8lISaJXSiIZqUf/XV/TSl75XjJSkshISaRXShIZqYkkJ/pzAqD2QBPrdu9nREEvCrLSfIkhUg4XjJdW7KJkfRVNLW0APLBgDZ84cwCfnT6EkYWx3c7pNxWPHpKens6yZcsAqKys5LrrrqOuro4HHniA3bt3c9111/GnP/2JM888k+rqai666CKKioq46qqrAOjTpw+PPPIIDz30kJ9piGdjZT3zl+9kwfKdbK4+QHKiUZCVxsGmFg40tR75svon7735T7NSEhPISktiZGEmEwZkM35ANhMH5jAkL4OEhO45FdnW5thYVU/p1lpKt9by/rZaNlUdOPL6iL69mDEinxnD+3DW8Dz6ZKZ2y3Z7UkcFo7B3KtdNG8zHJhaRYMbT72zlf5dsZ+7bW5kxPJ/rZwzhgnGF3RbD/oZmSsqqeHlNBa+vr2Jgbjrnjy3kgnGFnN6/d6BOLat4+KCgoIDHH3+cqVOnMmfOHB577DFuuukmzjzzTCBUKB5++GG++93vHikeN998M08++ST33HMPeXl5foYft3bsPcSC5TuZv2wna3bVYQYzhufzxXOGc/H4fuRkpBxZtrm1jYNNraFi0hj6++a7pYwaN54DTa0cbGw56u++Q02s2bWfuW9vPVJ4slKTGD8gmwkDvYIyIJsh+Rmd+gLa39DMsu17eX/rXkq31fKPbbXsb2gBIDcjmeIhuVxTPJCx/XqzvmI/b2+q4Y/v7+Dpd7YBMKowk+nD85kxPJ+zhueT1yvlRJs7Keccew82s732IOW1h0hKMPrnpNM/J53cjOQuf6merGAUD849qgAXD8nlOx8byzNLy3n6na3c9j/vU9g7lRkFbYw9s4HC3uEfgVXtb+TVtRUsXL2btzbW0NTaRn6vFM4bU8C2PQf5+Wsb+NmiDQzISef8sQWcP66Qs4blk5IU203OcVc8HliwmjU7646a19raSmJiYpffc1z/3nzv46eHtc7w4cNpbW2lsrKS1atXc+ONNx71+pQpU1izZs2R55mZmdx888387Gc/44EHHuhyrBKe6vpG/rxyF/OX7WTp1loAJg3K4f7LxnHZxCIKjvNlk5yYQHZ6AtnpyUfm7dmYyMyxJ/6V29zaxvqK/azasY+VO/axsnwfT765haZWr6CkJTG+fzYTvYIyYUA2g/My2LbnYOioYlst72+tpaxiP86BGYwuzOKyif0pHpJL8ZBchh5TgGaNKeCL546gpbWNlTv28c6mPby9qYbnSsv53duh4Y7G9Mti+vB875F3VKE8rL6xhe17DoYetYcorz3I9j2hv+W1h6hvbOkw57TkBPpnp1OUk+b9TWdAThpF2elegUkjIyXpqO2EUzCOlZ+ZypdmjmD2OcMpKavkqXe28kJZFQsefI2LTi/k+ulDmT4874QFbUv1AV5es5uXV1dQuq0W52BQXjo3zBjChaf3o3hILoleDNX1jby2tpKX11Tw+6Who56s1CRmjing/LEFzBxdcNTnJFbEXfGIZV/96leZNGkSd911l9+hBFpdQzMLV+1m/vKdvPVBDa1tjtGFWdx90Wg+PrE/g/MzIrbt5MQETu+fzen9s/nU1NC8ppZjCsqOffy2XUFJTjSaWx0QOlqZNDiHi8f348zBuUwanEPvtM59MSUlJjB5cC6TB+fypZkjaG5tY0X5Pt7ZVMM7m2qY9942nnxrC2Ywtl9vChIbeWZHKdv3HGJ77UH2Hmw+6v16pSQyKC+DgbkZTB+ez6C8DAblpjMwN4OWtjZ27m1g595D7Np3KDS97xCvb6iicn8jzh0dW3Z6Mv1z0slOT+L9bXvDLhgdSUwwzhtbyHljC3nmz6+xkSKeWbqdP6/czciCTK6fMYSrJg8gKy0Z5xyrd9axcHWoYJRV7AdgXFFvvn7eKC48vZAx/bI6LDh9MlO5duogrp06iENNrfx9YzWvrNnNorWVLFi+k6QEY/rwfM4fW8AFp/djQE56WHn4Je6KR0dHCH5cJLhp0yYSExMpKChg3LhxlJaWcsUVVxx5vbS0lClTphy1Tk5ODtdddx2PPfZYj8YaDxqaW3ltXSXzl+3ktbJKmlraGJibzhfPGc7lk/ozpl9v32JLSUpgvNcW8mlv3uGCsnLHPj6orGd430yKh+RyWkHmkV+8pyo5MeHI0cqXZ51GU0sby8v38s4HNby9qYb3trVQ2LifgbkZTByY7RWHDAblhQrEyU5HTTzOYAjNrW1U1DWwc28Du/YdYsfeQ+zyCk11feMpFYzjKchI4NqZY7nzglEsWL6Tp9/Zyv0vrObBv6zj3FF9WVG+jx17D5FgMHVoHvdfNo4LxhUyKC+8HxLpKYlcMC7UBtLa5li2vZaX11TwypoK5ixYw5wFaxhX1JtzRvVl2rBciofkRe1RSdwVj2hQVVXFrbfeyu23346Z8eUvf5mzzjqLq6++mkmTJlFTU8N9993Hgw8++E/r3nnnnUydOpWWlo5PAUjXfOF3S3ljQzV9MkO/Zi+f1J/Jg3KitoGzfUHpyW1OHZrH1KF5fOW8kREbxjw5MYGBuaEjlp6WlpzIJ6cM4pNTBrF8+16eemcrJWVVTBqUw9fPH8l5YwtPuf3nsMQEo3hIHsVD8vjWJWP5oKqeV9dU8OraCn7zxiZ+9TeHGYzp15tpQ3OZOiyPaUPzjnuqtKepePSQQ4cOMWnSpCNdda+//nruvPNOAIqKinj66aeZPXs2+/btY8uWLTz55JOce+65//Q+ffr04aqrruKnP/1pT6cQWNX1jbyxoZpbPjyMb106ttt+uUtsO2NQDmcMyumx7Y3om8mIczP54rkjONTUyj+217Jk8x7e27KHZ5aWM9drfxqanxEq4l4x6Wwniu6m4tFDWltbT/j6Oeecw5IlSwD45S9/yQ9/+EMuvvhicnNzmTNnzlHLPvroozz66KORCjXuvL6+CoArJw9Q4ZCokJ6SyNkj+nD2iD5A6FTe6p11LNlcw5LNtbyytoJnS0Pj2BVkpR4pJGcNz+uxU6wqHlHotttuO3KBoETe4rIq+mSmMq7Iv3YNkRNJTkxg0qAcJg3KYfY5/3fdzrub9/Ced3Ty0opdTBiQzYKvfLhHYlLxkLjW2uZ4fX0V548t7LbGV5FIS0gwRhVmMaowi+unD8E5R3ntIWoPNvVYDHFTPJxzUdv42V3csf0b5aSWba9l36FmZo3p63coIl1mZqHebmH2/joVsX2JYyelpaVRU1MT6C/Xw/fzSEuLjp4YsaKkrIoEg4+cpuIhEo64OPIYOHAg5eXlVFVVdfh6Q0NDIL50D99JcOvWrX6HEjNKyqooHpJLdkZ09qUXiVZxUTySk5MZNmzYcV8vKSlh8uTJPRiRRIPK/Q2s3LGPuy8a7XcoIjEnLk5biXTk9fXVAMwcrVNWIuFS8ZC4tbiskoIsddEV6QoVD4lLLa1tvLG+ipmj+wa+F55IJKh4SFz6x/a91DW0MHN0gd+hiMQkFQ+JSyVllSQmGB86rY/foYjEJBUPiUuL13lddKN0uGuRaKfiIXGnoq6BNbvq1MtK5BSoeEjc+VtZ6GLRWWrvEOkyFQ+JOyXrK+nXO40x/Xr27pEiQaLiIXGlubWNN9ZXq4uuyClS8ZC48v7WWvY3tqi9Q+QUqXhIXFlcVkWSuuiKnDIVD4krJWWVTBmaS1aauuiKnApfioeZ5ZnZK2a2wfub28Eyk8zsbTNbbWYrzOxTfsQqwbFr3yHW7d6vXlYi3cCvI497gUXOuZHAIu/5sQ4CNzjnTgcuBv7dzHJ6MEYJmMNddDUkicip86t4XAHM9abnAlceu4Bzbr1zboM3vROoBNTKKV1WUlZFUXYaowoz/Q5FJOaZH7dmNbO9zrkcb9qA2sPPj7P8NEJF5nTnXFsHr88GZgMUFhYWz5s3L6x46uvrycwMzhdKkPLprlxa2hy3LzrI9KIkbhqf2g2RhS9I+wWClU+QcoGu5TNr1qxS59yUTq/gnIvIA3gVWNXB4wpg7zHL1p7gfYqAMmB6Z7ZbXFzswrV48eKw14lmQcqnu3J5a2O1G3LPi+6vq3Z1y/t1RZD2i3PByidIuTjXtXyApS6M7/iI3YbWOXf+8V4zswozK3LO7TKzIkKnpDparjfwEnCfc+6dCIUqcaCkrJLkRHXRFekufrV5zAdu9KZvBF44dgEzSwH+CPzOOfdcD8YmAVRSVsXUoXlkpkbs95JIXPGreDwIXGBmG4DzveeY2RQz+423zLXAOcBNZrbMe0zyJ1yJZTv3HqKsQl10RbqTLz/DnHM1wHkdzF8K3OJNPw083cOhSQCVHOmiq856It1FV5hL4C0uq2RATjqnFQSnN42I31Q8JNAaW1p5a6NG0RXpbioeEmhLt9RyoKlVV5WLdDMVDwm0krJKUhITOHtEvt+hiASKiocE2uKyKqYNy6OXuuiKdCsVDwms7XsOsrGyXr2sRCJAxUMCq2S9RtEViRQVDwmsv5VVMigvnRF9e/kdikjgqHhIIDW2tPLmxhpmjipQF12RCFDxkEBasnkPh5pbmTVG7R0ikaDiIYFUUlZFSlICM4ZrFF2RSFDxkEBaXFbJWcPySE9J9DsUkUBS8ZDA2VZzkE1VBzSKrkgEqXhI4JSsD91bTNd3iESOiocETklZFUPyMxjWR110RSJFxUMCpaG5lbc+qGbmKI2iKxJJKh4SKO9u3kNDcxszx6i9QySSVDwkUBavqyQ1KYEZwzWKrkgkqXhIoPxtfRUzRuSTlqwuuiKRpOIhgbGl+gCbqw8wc5R6WYlEmoqHBEZJ2eEuumrvEIk0FQ8JjMVlVQzr04uh6qIrEnEqHhIIDc2tvLOphnN1ykqkR6h4SCC8vamGxpY2ZqmLrkiPUPGQQChZV0lacgJnDcvzOxSRuKDiITHPOcfisirOHtFHXXRFeoiKh8S8zdUH2LbnoAZCFOlBKh4S8xaXVQEwc5TaO0R6ioqHxLSa+kb+s+QDJg7MZnB+ht/hiMSNJL8DEOkq5xzf/uNK6g4189Tnp/kdjkhc0ZGHxKzn39/BwtUVfOPCUYwt6u13OCJxRcVDYtL2PQeZM38104blcctHhvsdjkjc8aV4mFmemb1iZhu8v7knWLa3mZWb2X/0ZIwSvVrbHN94ZjkAj3zyDBITdNMnkZ7m15HHvcAi59xIYJH3/Hi+D7zeI1FJTPjNG5tYsmUPcy4/nUF5aiQX8YNfxeMKYK43PRe4sqOFzKwYKARe7qG4JMqt2VnHT14u4+LT+/GJMwf4HY5I3PKreBQ653Z507sJFYijmFkC8AhwV08GJtGrobmVO59ZRnZ6Cj+8eoLuUS7iI3POReaNzV4F+nXw0n3AXOdcTrtla51zR7V7mNntQIZz7mEzuwmY4py7/Tjbmg3MBigsLCyeN29eWLHW19eTmZkZ1jrRLEj5tM9l3rom/rqlmTuKUzmjb+z1Mg/SfoFg5ROkXKBr+cyaNavUOTel0ys453r8AZQBRd50EVDWwTL/A2wDtgDVQB3w4Mneu7i42IVr8eLFYa8TzYKUz+Fc3tpY7Ybe+6L79h9W+BvQKQjSfnEuWPkEKRfnupYPsNSF8T3u18+3+cCNwIPe3xeOXcA59y+Hp9sdeZyoYV0Cqq6hmbueXc6QvAzu+9hYv8MREfxr83gQuMDMNgDne88xsylm9hufYpIoNWf+anbXNfDTT00iIyX2TleJBJEv/xOdczXAeR3MXwrc0sH8J4EnIx6YRJ33drfwh2U7+Op5I5k8+LiXA4lID9MV5hK1KusamLu6kYkDs/nKR0/zOxwRaUfFQ6KSc45vPr+Cplb46acmkZyoj6pINNH/SIlK//PuNkrKqrh2dAoj+ganC6VIUKh4SNTZVFXPv720lo+M7MN5g9VALhKNVDwkqrS0tnHHM8tJSUrgx9ecoavIRaKUiodElccWf8Dy7Xv5t6vG0y87ze9wROQ4VDwkaizfvpefv7aBKyf157KJ/f0OR0RO4IQnlM1sAXDcwa+cc5d3e0QSlw41tXLH75dRkJXKA1eM9zscETmJk7VG/sT7ezWhQQ6f9p5/BqiIVFASf370l7Vsqj7A/7vlLLLTk/0OR0RO4oTFwzn3NwAze8QdPdriAjNbGtHIJG6UlFXyu7e38vkPD+Ps0/r4HY6IdEJn2zx6mdmRG0Wb2TCgV2RCknhSe6CJbz63gpEFmdx90Wi/wxGRTupsJ/qvAyVmtgkwYAje/TNEuso5x3f+tIrag008cdNU0pIT/Q5JRDrppMXDu6NfNjASGOPNXueca4xkYBJ8LyzbyUsrd3H3RaMZPyDb73BEJAwnPW3lnGsDvumca3TOLfceKhxySnbtO8R3X1hF8ZBcbj13hN/hiEiYOtvm8aqZ3WVmg8ws7/AjopFJYLW1Oe5+dgWtbY5Hrz2DxARdRS4Sazrb5vEp7++X281zwPAOlhU5oafe2crfN1bzb1eNZ0i++l2IxKJOFQ/n3LBIByLx4YOqen70l7XMHN2X66YN9jscEemiTg9ZambjgXHAkQGHnHO/i0RQEkwtrW3c+cxy0pITefgTEzXooUgM61TxMLPvATMJFY8/A5cAfwdUPKTTflkSGvTwP66bTEFvDXooEss622B+DaF7ju92zn0OOINQ912RTllZvo+fL9rAFRr0UCQQOls8DnlddlvMrDdQCQyKXFgSJA3NrdzxzDL6ZKbyr5dr0EORIOhsm8dSM8sB/gsoBeqBtyMWlQTKjxeWsbGynqc+P43sDA16KBIEne1tdZs3+Ssz+yvQ2zm3InJhSVC89UE1//33zdwwYwgfGdnX73BEpJt0tsH8KeB14A3n3LrIhiRBUdfQzN3PrmBYn17ce8mYk68gIjGjs20eTwBFwC/MbJOZPW9mX4tgXBIA/7pgDbv2HeLRa88gI6XTvcJFJAZ09rTVYjN7HZgKzAJuBU4HfhbB2CSGLVy9m+dKy/nKR09j8uBcv8MRkW7W2dNWiwjdv+Nt4A1gqnOuMpKBSeyqrm/k239YyfgBvfnKR0f6HY6IREBnT1utAJqA8cBEYLyZpUcsKolZzjnufX4l+xtbePTaSaQkdfYjJiKxpLOnre4AMLMs4Cbgt4TuaZ4ascgkJj1bWs6rayv4zsfGMqowy+9wRCRCOnva6nbgI0AxsIVQA/obkQtLYtH2PQf51wVrOGtYHjd/SGNpigRZZ7vApAGPAqXOuZYIxiMxqq3NcdezywF45NozSNA9OkQCrVMnpJ1zPwGSgesBzKyvmemnpRzxxJubeXfzHr738XEMzM3wOxwRibBOFQ9vVN17gG95s5KBpyMVlMSW9RX7eXhhGReMK+Sa4oF+hyMiPaCzXWGuAi4HDgA453YCXW4N9W5j+4qZbfD+dnghgJkNNrOXzWytma0xs6Fd3aZERlNLG3f8fhlZqUn86OoJukeHSJzobPFocs45QreexcxO9d6h9wKLnHMjgUXe8478Dvixc24sMI3QaL4SRX7x2gZW76zjR1dPoE+mOt+JxIuTFg8L/ZR80cx+DeSY2ReAVwmNsNtVVwBzvem5wJUdbHcckOScewXAOVfvnDt4CtuUbvb+tloeW7yRa4oHcuHp/fwOR0R6kIUOKE6ykNlK4E7gQsCAhYe/1Lu0UbO9zrkcb9qA2sPP2y1zJXALoYsThxEqWPc651o7eL/ZwGyAwsLC4nnz5oUVT319PZmZmV1JJSr1RD6NLY773zpESxv84MPppCdF5nRVkPZNkHKBYOUTpFyga/nMmjWr1Dk3pdMrOOdO+iB0dDC1M8u2W+dVYFUHjyuAvccsW9vB+tcA+4DhhLoUPw98/mTbLS4uduFavHhx2OtEs0jn09ra5r72v++7ofe+6N7aWB3RbQVp3wQpF+eClU+QcnGua/kAS10Y3/Gdvc7jLOBfzGwrXqO5V3gmnqAonX+818yswsyKnHO7zKyIjtsyyoFlzrlN3jp/AqYD/93JmCVCHvrrOv60bCd3XzSaGSPy/Q5HRHzQ2eJxUTdvdz5wI/Cg9/eFDpZ5j1AbS1/nXBXwUWBpN8chYfrNG5v49eubuGHGEG6bOcLvcETEJ50d22prN2/3QeAZM/s8sBW4FsDMpgC3Ouducc61mtldwCKvXaSUU2ukl1P0wrId/OCltVw6oR/f+/jp6pYrEsd8uUOPc64GOK+D+UsJNZIffv4KoVF8xWdvbKjirmeXc9awPB69dhKJGn5EJK5pvGw5qVU79nHrU6WM6JvJ4zdMIS050e+QRMRnKh5yQltrDnDTb5eQk5HC3JunkZ2e7HdIIhIFdGNpOa7q+kZufGIJLW2OeTdPo7B3mt8hiUiU0JGHdOhAYwuf++177K5r4ImbpnJaQXAuoBKRU6cjD/knTS1t3Pp0KWt21fH49cWcObjDcStFJI7pyEOO0tbm+OZzy3ljQzU/unoC540t9DskEYlCKh5ylAfbXT1+7ZRBfocjIlFKxUOO+M0bm3hcV4+LSCeoeAigq8dFJDwqHqKrx0UkbCoecW5lua4eF5HwqXjEsa01B/jck7p6XETCp+IRp6r2N3LDE0tobXPM1dXjIhImXSQYh+obW7j5yfeoqGvg/31huq4eF5GwqXjEmaaWNr7kXT3+Xzfo6nER6Rqdtoozv3htw5Grxz86RlePi0jXqHjEkfLagzz++iYuP6O/rh4XkVOi4hFHHvprGQD3XDLG50hEJNapeMSJ0q17WLB8J7PPGc6AnHS/wxGRGKfiEQfa2hz/+uJaCrJSufVcjVklIqdOxSMOzF++k+Xb9/LNi8fQK1Ud7ETk1Kl4BNyhplYe+us6JgzI5urJA/wOR0QCQsUj4B5/fRO79jXw3cvGkaABD0Wkm6h4BNjufQ386m8fcOmEfkwblud3OCISICoeAfbwwnW0tjm+dclYv0MRkYBR8Qio5dv38of3d3Dzh4cxKC/D73BEJGBUPALIOcf3X1xDn8wUvjxLXXNFpPupeATQe7tbWbq1lm9cOJqsNN2jQ0S6n4pHwDQ0t/L7sibG9MvS+FUiEjG6Yixg/vvvm6lpcPzi4+N0L3IRiRgdeQRI5f4Gfrl4I5MLEjl7RB+/wxGRAFPxCJBHFq6nqbWNT41O8TsUEQk4FY+AWLVjH8+UbufGGUPp10u7VUQiy5dvGTPLM7NXzGyD97fDe6Ga2cNmttrM1prZz81MJ/E7cLhrbk56Ml85b6Tf4YhIHPDrJ+q9wCLn3Ehgkff8KGZ2NvAhYCIwHpgKnNuTQcaKhasreHfzHu68YBTZ6eqaKyKR51fxuAKY603PBa7sYBkHpAEpQCqQDFT0SHQxpLGllR/9ZS0jCzL5zLTBfocjInHCnHM9v1Gzvc65HG/agNrDz49Z7ifALYAB/+Gcu+847zcbmA1QWFhYPG/evLDiqa+vJzMzM7wkosRfNjfz+7ImvlGcyoS+oZ7XsZzPsZRL9ApSPkHKBbqWz6xZs0qdc1M6vYJzLiIP4FVgVQePK4C9xyxb28H6pwEvAZne423gIyfbbnFxsQvX4sWLw14nGlTvb3Dj7/+ru+mJd4+aH6v5dES5RK8g5ROkXJzrWj7AUhfGd3zELhJ0zp1/vNfMrMLMipxzu8ysCKjsYLGrgHecc/XeOn8BZgBvRCTgGPToK+s52NzKfR8b53coIhJn/GrzmA/c6E3fCLzQwTLbgHPNLMnMkgk1lq/tofiiXtnu/fzvkm1cP30IpxUE53BbRGKDX8XjQeACM9sAnO89x8ymmNlvvGWeAz4AVgLLgeXOuQV+BBttnHP84KU1ZKUl8zV1zRURH/gytpVzrgY4r4P5Swk1kOOcawW+2MOhxYTFZZW8saGa+y8bR24vXU0uIj1PlyLHmObWNn7w4lqG9+3F9TOG+B2OiMQpFY8Y89TbW9lUfYD7Lh1LcqJ2n4j4Q98+MaT2QBM/W7SBj4zsw0fHFPgdjojEMRWPGPKzRRvY39DMdz42Dg3zJSJ+UvGIERsr9/PUO1v5zLTBjO6X5Xc4IhLnVDxigHOOBxasISM5kTsvGOV3OCIiKh6x4NnSct7YUM3dF48mPzPV73BERFQ8ol1FXQPff3EN04bl8dmz1DVXRKKDikcUc85x3x9X0dTSxkOfmEhCghrJRSQ6qHhEsQUrdvHq2gruunA0w/r08jscEZEjVDyiVE19I3Pmr+aMQTnc/OFhfocjInIUFY8o9b35q9nf0MyPr5lIok5XiUiUUfGIQgtX7+bFFbv46kdHMqpQ13SISPRR8Ygy+w42850/rYPxGNUAAAtcSURBVGJcUW9unTnC73BERDrky5Dscnzff2kNew408dubpmrgQxGJWvp2iiIlZZU8V1rOl84dwfgB2X6HIyJyXCoeUWJ/QzPf/sNKTivI5CvnneZ3OCIiJ6TTVlHiob+uY1ddA89/6WxSkxL9DkdE5IR05BEF3v6ghqff2cbnPzSMMwfn+h2OiMhJqXj47GBTC/c8v4Ih+Rl848LRfocjItIpOm3ls0deXs+2PQeZN3s66Sk6XSUisUFHHj56f1stT7y5mc9OH8z04fl+hyMi0mkqHj5paG7lm8+toH92OvdeMtbvcEREwqLTVj75xWsb2FhZz9ybp5GZqt0gIrFFRx4+WLVjH7/62yY+WTyQc0f19TscEZGwqXj0sObWNu5+bgX5vVL4zsfG+R2OiEiX6HxJD/tVyQes3VXH49cXk52R7Hc4IiJdoiOPHrS+Yj8/f20DHz+jPxee3s/vcEREukzFo4e0tLZx97PLyUpLZs7HdbpKRGKbTlv1kCfe3Mzy8n384jOTyc9M9TscEZFToiOPHrC5+gCPvLyeC8YVctnEIr/DERE5ZSoeEdbW5rjnuRWkJiXwgyvHY6b7kYtI7FPxiLCn393Kki17+O5l4yjsneZ3OCIi3cKX4mFmnzSz1WbWZmZTTrDcxWZWZmYbzezenoyxO2zfc5AH/7KOc0b15ZrigX6HIyLSbfw68lgFXA28frwFzCwReAy4BBgHfMbMYqabknOOb/1hJQb86OoJOl0lIoHiS28r59xa4GRfqNOAjc65Td6y84ArgDWRiGnvwSY++au3u+39Wtocm6sP8P0rxzMgJ73b3ldEJBqYc86/jZuVAHc555Z28No1wMXOuVu859cDZznnbu9g2dnAbIDCwsLiefPmhRVHfX09Cam9eGJVY/hJnMCAzASuOC2ZhB4+6qivryczM7NHtxkpyiV6BSmfIOUCXctn1qxZpc654zYjHCtiRx5m9irQ0WXU9znnXujObTnnHgceB5gyZYqbOXNmWOuXlJQwc+ZMLr2gO6Pyz+F8gkC5RK8g5ROkXKBn8olY8XDOnX+Kb7EDGNTu+UBvnoiI+Cyau+q+B4w0s2FmlgJ8Gpjvc0wiIoJ/XXWvMrNyYAbwkpkt9Ob3N7M/AzjnWoDbgYXAWuAZ59xqP+IVEZGj+dXb6o/AHzuYvxO4tN3zPwN/7sHQRESkE6L5tJWIiEQpFQ8REQmbioeIiIRNxUNERMLm6xXmkWBmVcDWMFfrA1RHIBy/BCkf5RK9gpRPkHKBruUzxDnXt7MLB654dIWZLQ3nsvxoF6R8lEv0ClI+QcoFeiYfnbYSEZGwqXiIiEjYVDxCHvc7gG4WpHyUS/QKUj5BygV6IB+1eYiISNh05CEiImFT8RARkfA552L6AWwBVgLLgKXevDmE7v2xzHtc2m75bwEbgTLgonbzL/bmbQTubTd/GPCuN//3QIo3P9V7vtF7fWg35ZMDPAesIzSa8AwgD3gF2OD9zfWWNeDnXgwrgDPbvc+N3vIbgBvbzS/2/r02eusePnXZ4TYikEvM7RtgdLt4lwF1wNdjeL8cL5+Y2zfee94BrAZWAf8LpHVl+92VYwRyeRLY3G6/TIqGz9kpf9n5/SBUPPocM28OodvbHrvsOGC59wEaBnwAJHqPD4DhQIq3zDhvnWeAT3vTvwK+5E3fBvzKm/408PtuymcucIs3nULoC/jhwx9a4F7gIW/6UuAv3odoOvBuuw/CJu9vrjd9+Ittibeseete4s3vcBsRyCVm9433fonAbmBIrO6XE+QTc/sGGEDoizW93XZvCnf73ZljBHJ5Erimg+V9/Zx124fQrwfhFY9vAd9q93whoV/DM4CFxy7n/QNXA0ne/CPLHV7Xm07ylrNTzCXb+/DYMfPLgCJvuggo86Z/DXzm2OWAzwC/bjf/1968ImBdu/lHljveNiKQS0zum3bbvxB4M1b3y0nyibl9Q+gLdzuhL8ok4EXgonC33505dnMuF3L84uHr5ywIbR4OeNnMSs1sdrv5t5vZCjN7wsxyvXmHd85h5d68483PB/a60I2p2s8/6r281/d5y5+KYUAV8Fsz+4eZ/cbMegGFzrld3jK7gcIu5jPAmz52PifYRnfnArG5bw77NKHTCRCb++VY7fOBGNs3zrkdwE+AbcAu7/1Ku7D97syx23Jxzr3svfxv3n75qZmlHptLJ2Pu1s9ZEIrHh51zZwKXAF82s3OA/wRGAJMI7YRHfIwvHEnAmcB/OucmAwcIHUIe4UI/DVwkg+imbRwvl1jdN3i3Q74cePbY12JovxzRQT4xt2+8AncFoR8r/YFehNooYk5HuZjZZwkd6YwBphI6KrknknF09nMW88XDq9Y45yoJ3Z1wmnOuwjnX6pxrA/4LmOYtvgMY1G71gd68482vAXLMLOmY+Ue9l/d6trf8qSgHyp1z73rPnyP0BVxhZkXetoqAyi7ms8ObPnY+J9hGt+YSw/sGQj9Q3nfOVXjPY3G/tHdUPjG6b84HNjvnqpxzzcAfgA91YfvdmWN35nK2c26XC2kEfkvX90u3fs5iuniYWS8zyzo8Tej84KrD/wieqwj1XACYD3zazFLNbBgwklAD0nvASDMb5v0a+zQw36vAi4FrvPVvBF5o9143etPXAK95y3eZc243sN3MRnuzzgPWHLOtY2O4wUKmEzrM3UXofO2FZpbr/Zq5kND52F1AnZlNNzMDbjhOPu230a25xOq+8XyGo0/xxNx+OVE+MbpvtgHTzSzD+7c7/H8m3O13Z47dmcvadl/qBlzJ0fvFv8/ZqTTw+P0g1ANiufdYDdznzX+KUHe0Fd4/SlG7de4j1HuiDK+ngTf/UmC999p9x2xjCaGubc8Cqd78NO/5Ru/14d2U0yRgqRf7nwj1lsgHFhHqRvcqkOcta8BjXswrgSnt3udmL7aNwOfazZ/iffg+AP6D/+uq1+E2IpBLTO4bQqdDaoDsdvNicr+cIJ9Y3TcPEOoOvsrLIbUr2++uHCOQy2veflkFPA1kRsPnTMOTiIhI2GL6tJWIiPhDxUNERMKm4iEiImFT8RARkbCpeIiISNhUPEQ6YGY5ZnabN93fzJ6L4LYmmdmlkXp/kUhQ8RDpWA6hEVhxzu10zl1zkuVPxSRC1xKIxAxd5yHSATObR2icoTJCF06Ndc6NN7ObCF3l24vQVcg/ITRU9/VAI6F7YOwxsxGELuDqCxwEvuCcW2dmnwS+B7QSGpTvfEIXcqUTGiriR4RGU/0FMB5IBuY4517wtn0VoSE1BgBPO+ceiPA/hUiHkk6+iEhcuhcY75ybZGZDCX2hHzYemEzoauWNwD3Ouclm9lNCQz78O/A4cKtzboOZnQX8EvgocD+hGw3tMLMc51yTmd1P6Org2wHM7IeEhs242cxygCVm9qq37Wne9g8C75nZS865pZH8hxDpiIqHSPgWO+f2A/vNbB+wwJu/EphoZpnA2cCzoSGEgNAwEwBvAk+a2TOEBr7ryIXA5WZ2l/c8DRjsTb/inKsBMLM/AB8mNASMSI9S8RAJX2O76bZ2z9sI/Z9KIHSvh0nHruicu9U7EvkYUGpmxR28vwGfcM6VHTUztN6x55l13ll8oQZzkY7tB7K6sqJzrg7Y7LVv4I16eoY3PcI5965z7n5CN8sa1MG2FgJf8UY+xcwmt3vtAjPLM7N0Qm0vb3YlRpFTpeIh0gHv1NCbZrYK+HEX3uJfgM+b2eERn6/w5v/YzFZ67/sWoRGhFwPjzGyZmX0K+D6hhvIVZrbae37YEuB5QiPfPq/2DvGLeluJxAivt9WRhnURP+nIQ0REwqYjDxERCZuOPEREJGwqHiIiEjYVDxERCZuKh4iIhE3FQ0REwvb/AbJ7vmKJ8Rs4AAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rtJ7xdSIaPw3"
      },
      "source": [
        "You can find that the agent achieves an increasingly better performance with more training. All the logs and the model weights are saved in `experiments/leduc_holdem_dqn_result/`. Now you have your trained DQN agent on Blackjack!"
      ]
    }
  ]
}