DP/Gamblers Problem Solution.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### This is Example 4.3. Gambler’s Problem from Sutton's book.\n",
    "\n",
    "A gambler has the opportunity to make bets on the outcomes of a sequence of coin flips. \n",
    "If the coin comes up heads, he wins as many dollars as he has staked on that flip; \n",
    "if it is tails, he loses his stake. The game ends when the gambler wins by reaching his goal of $100, \n",
    "or loses by running out of money. \n",
    "\n",
    "On each flip, the gambler must decide what portion of his capital to stake, in integer numbers of dollars. \n",
    "This problem can be formulated as an undiscounted, episodic, finite MDP. \n",
    "\n",
    "The state is the gambler’s capital, s ∈ {1, 2, . . . , 99}.\n",
    "The actions are stakes, a ∈ {0, 1, . . . , min(s, 100 − s)}. \n",
    "The reward is zero on all transitions except those on which the gambler reaches his goal, when it is +1.\n",
    "\n",
    "The state-value function then gives the probability of winning from each state. A policy is a mapping from levels of capital to stakes. The optimal policy maximizes the probability of reaching the goal. Let p_h denote the probability of the coin coming up heads. If p_h is known, then the entire problem is known and it can be solved, for instance, by value iteration.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import sys\n",
    "import matplotlib.pyplot as plt\n",
    "if \"../\" not in sys.path:\n",
    "  sys.path.append(\"../\") "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "\n",
    "### Exercise 4.9 (programming)\n",
    "\n",
    "Implement value iteration for the gambler’s problem and solve it for p_h = 0.25 and p_h = 0.55."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def value_iteration_for_gamblers(p_h, theta=0.0001, discount_factor=1.0):\n",
    "    \"\"\"\n",
    "    Args:\n",
    "        p_h: Probability of the coin coming up heads\n",
    "    \"\"\"\n",
    "    # The reward is zero on all transitions except those on which the gambler reaches his goal,\n",
    "    # when it is +1.\n",
    "    rewards = np.zeros(101)\n",
    "    rewards[100] = 1 \n",
    "    \n",
    "    # We introduce two dummy states corresponding to termination with capital of 0 and 100\n",
    "    V = np.zeros(101)\n",
    "    \n",
    "    def one_step_lookahead(s, V, rewards):\n",
    "        \"\"\"\n",
    "        Helper function to calculate the value for all action in a given state.\n",
    "        \n",
    "        Args:\n",
    "            s: The gambler’s capital. Integer.\n",
    "            V: The vector that contains values at each state. \n",
    "            rewards: The reward vector.\n",
    "                        \n",
    "        Returns:\n",
    "            A vector containing the expected value of each action. \n",
    "            Its length equals to the number of actions.\n",
    "        \"\"\"\n",
    "        A = np.zeros(101)\n",
    "        stakes = range(1, min(s, 100-s)+1) # Your minimum bet is 1, maximum bet is min(s, 100-s).\n",
    "        for a in stakes:\n",
    "            # rewards[s+a], rewards[s-a] are immediate rewards.\n",
    "            # V[s+a], V[s-a] are values of the next states.\n",
    "            # This is the core of the Bellman equation: The expected value of your action is \n",
    "            # the sum of immediate rewards and the value of the next state.\n",
    "            A[a] = p_h * (rewards[s+a] + V[s+a]*discount_factor) + (1-p_h) * (rewards[s-a] + V[s-a]*discount_factor)\n",
    "        return A\n",
    "    \n",
    "    while True:\n",
    "        # Stopping condition\n",
    "        delta = 0\n",
    "        # Update each state...\n",
    "        for s in range(1, 100):\n",
    "            # Do a one-step lookahead to find the best action\n",
    "            A = one_step_lookahead(s, V, rewards)\n",
    "            # print(s,A,V) # if you want to debug.\n",
    "            best_action_value = np.max(A)\n",
    "            # Calculate delta across all states seen so far\n",
    "            delta = max(delta, np.abs(best_action_value - V[s]))\n",
    "            # Update the value function. Ref: Sutton book eq. 4.10. \n",
    "            V[s] = best_action_value        \n",
    "        # Check if we can stop \n",
    "        if delta < theta:\n",
    "            break\n",
    "    \n",
    "    # Create a deterministic policy using the optimal value function\n",
    "    policy = np.zeros(100)\n",
    "    for s in range(1, 100):\n",
    "        # One step lookahead to find the best action for this state\n",
    "        A = one_step_lookahead(s, V, rewards)\n",
    "        best_action = np.argmax(A)\n",
    "        # Always take the best action\n",
    "        policy[s] = best_action\n",
    "    \n",
    "    return policy, V"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimized Policy:\n",
      "[ 0.  1.  2.  3.  4.  5.  6.  7.  8.  9. 10. 11. 12. 12. 11. 15. 16. 17.\n",
      " 18.  6. 20. 21.  3. 23. 24. 25.  1.  2.  3.  4.  5.  6.  7.  8.  9. 10.\n",
      " 11. 12. 38. 11. 10.  9. 42.  7. 44.  5. 46. 47. 48. 49. 50.  1.  2.  3.\n",
      "  4.  5.  6.  7.  8.  9. 10. 11. 12. 13. 11. 10.  9. 17.  7. 19.  5. 21.\n",
      " 22. 23. 24. 25.  1.  2.  3.  4.  5.  6.  7.  8.  9. 10. 11. 12. 12. 11.\n",
      " 10.  9.  8.  7.  6.  5.  4.  3.  2.  1.]\n",
      "\n",
      "Optimized Value Function:\n",
      "[0.00000000e+00 7.24792480e-05 2.89916992e-04 6.95257448e-04\n",
      " 1.16010383e-03 1.76906586e-03 2.78102979e-03 4.03504074e-03\n",
      " 4.66214120e-03 5.59997559e-03 7.08471239e-03 9.03964043e-03\n",
      " 1.11241192e-02 1.56793594e-02 1.61464431e-02 1.69517994e-02\n",
      " 1.86512806e-02 1.98249817e-02 2.24047303e-02 2.73845196e-02\n",
      " 2.83388495e-02 3.04937363e-02 3.61633897e-02 3.84953022e-02\n",
      " 4.44964767e-02 6.25000000e-02 6.27174377e-02 6.33700779e-02\n",
      " 6.45857723e-02 6.59966059e-02 6.78135343e-02 7.08430894e-02\n",
      " 7.46098323e-02 7.64884604e-02 7.93035477e-02 8.37541372e-02\n",
      " 8.96225423e-02 9.58723575e-02 1.09538078e-01 1.10939329e-01\n",
      " 1.13360151e-01 1.18457374e-01 1.21977661e-01 1.29716907e-01\n",
      " 1.44653559e-01 1.47520113e-01 1.53983246e-01 1.70990169e-01\n",
      " 1.77987434e-01 1.95990576e-01 2.50000000e-01 2.50217438e-01\n",
      " 2.50870078e-01 2.52085772e-01 2.53496606e-01 2.55313534e-01\n",
      " 2.58343089e-01 2.62109832e-01 2.63988460e-01 2.66803548e-01\n",
      " 2.71254137e-01 2.77122542e-01 2.83372357e-01 2.97038078e-01\n",
      " 2.98439329e-01 3.00860151e-01 3.05957374e-01 3.09477661e-01\n",
      " 3.17216907e-01 3.32153559e-01 3.35020113e-01 3.41483246e-01\n",
      " 3.58490169e-01 3.65487434e-01 3.83490576e-01 4.37500000e-01\n",
      " 4.38152558e-01 4.40122454e-01 4.43757317e-01 4.47991345e-01\n",
      " 4.53440603e-01 4.62529268e-01 4.73829497e-01 4.79468031e-01\n",
      " 4.87912680e-01 5.01265085e-01 5.18867627e-01 5.37617932e-01\n",
      " 5.78614419e-01 5.82817988e-01 5.90080452e-01 6.05372123e-01\n",
      " 6.15934510e-01 6.39150720e-01 6.83960814e-01 6.92560339e-01\n",
      " 7.11950883e-01 7.62970611e-01 7.83963162e-01 8.37972371e-01\n",
      " 0.00000000e+00]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "policy, v = value_iteration_for_gamblers(0.25)\n",
    "\n",
    "print(\"Optimized Policy:\")\n",
    "print(policy)\n",
    "print(\"\")\n",
    "\n",
    "print(\"Optimized Value Function:\")\n",
    "print(v)\n",
    "print(\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Show your results graphically, as in Figure 4.3.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xd8HNW5//HPI8mqlmRky7jjbmMb\nQhGmJKGH0OEmJIFAQkng5hJCCKSQhCSENNJuknshxaH3UPIjhksghNCbLReMC26Si1xlSbZlyerP\n748ZKWtZZW1rtCrf9+u1L+3Mnp15zs5qnznnTDF3R0REBCAp0QGIiEjPoaQgIiItlBRERKSFkoKI\niLRQUhARkRZKCiIi0kJJIUJmNsbMdplZchcs6z4z+3FXxNVquW5mE8PnfzSz70Wwjv80s99GsNxL\nzewfXb3crrS/283MDjezt6KIqaczszPM7OluWtd3zOyuA3j/GjM7PXx+vZnd3nXRJYaSQhcIvxi7\nwwTQ/Bjh7uvcfaC7N0a8/ivMrDFc704zW2hm5+7rctz9S+7+oy6OLRW4BfjlAS5nbJjAUprnufvD\n7n7Ggca4j3G0/AhEyd0XAdvN7Lwo12NmqWb2azMrCb8/xWb2m5jX96m+XbTz8lOg5cfVAteb2WIz\nqwpjfcLMDjvA9eDuP3X3L4br2es7to9mAZeZ2dADjSuRlBS6znlhAmh+bOzm9b/t7gOBQcDdwONm\nltfNMbTlAuADd9+Q6EB6oYeB/4x4Hd8GCoCZQDZwCrAg4nW2y8yOAXLd/Z2Y2b8DvgpcD+QBk4Gn\ngXO6P8L2uXsN8Hfg84mO5UAoKUSo9Z6Hmb1iZj8yszfNrNLM/mFmQ2LKP2Fmm81sh5m9ZmbT93Wd\n7t4E3ANkAOPD5V5tZqvMrNzMZpvZiHbi3WMvz8wuCFsdO81stZmdaWafMrN5rd53UwfN/bOAV1uV\nb7eeZpYR7rmuDV9/w8wygNfCItvDPdrjwxbSGzHvPcHM5obvm2tmJ8S81uFn3yq+IWb2rJltDz+z\n180sycweBMYAz4QxfLOz+rRabraZvWxm/xPu/aaZ2a/MbJ2ZbbGg+y4j5i2vAKeZWVoby7rYzApb\nzfuamc0On59tZkvDum4ws6+3FRNwDPD/3H2jB9a4+wPhMvapvmZ2DXAp8M2w/DPh/BFm9pSZlVrQ\nErm+nVig1ffFzCYBXwYucfd/uXutu1eHrcTbwzLnmNmC8Hu63sxujXl/8//gNWa20cw2mdlNMa/f\namYPhZNtfccmmNm/zKzMzLaZ2cNmNqiD+F+hhyWrfebuehzgA1gDnN7G/LGAAynh9CvAaoI9nYxw\n+vaY8lcR7K2lAb8FFsa8dh/w43bWfwXwRvg8hWCvqhLIBU4FtgFHhcv9X+C1mPc6MLH1Ogj2HHcA\nHyPYeRgJTA2XUQ4cGrOMBcAn24ltLvCpVvM6qued4ecyEkgGTgjL7fFZtlHvPKAC+Fz4GVwSTg+O\n57NvFd/PgD8CA8LHRwFrb1vHs92AwcCc2G0Ylp0dxp4NPAP8rNWydwKHtxFjZriNJ7X6rC8On28C\nPho+Pwg4qp263gKsA64FDmuuZ0ff7XjqGzOdBMwDvg+kEuyoFAEfbyeeJ4BvxEx/CVjbyf/fyWHs\nScDhwBbgwlb/g48CWWG50uY6AbcCD7X1/xrOm0jwP5AG5BMkjt+29/kQ/J+VJ/L36EAfCQ+gLzzC\nL8YuYHv4eDqcv8eXjOCH6JaY910LPN/OMgeF780Np/f4Z2tV9gqgIVz3NuCdmC/93cAvYsoOBOqB\nseF0e0nhT8Bv2lnfH4CfhM+nE/z4prVTdiVwZgefXUs9w3/q3cCH2ijX1j/sFfw7KXwOmNPqPW8D\nV+zHZ38b8Lfmz6WNbb3XDkAn2+0eYDF7/tgZUAVMiJl3PFDcankbgBPbWddDwPfD55MIkkRmOL2O\noOspp5PvbjLBnvibQC2wEbj8AOsbmxSOBda1es+3gXvbWd6LwJdipr8LvLOP/4+/bf7uxnxvpsa8\n/gvg7vD5rXSQFNpY9oXAgvY+n3A7NO5LvD3toe6jrnOhuw8KHxd2UG5zzPNqgh9pzCzZzG4Pu2l2\nEnzZANrs4mjDO+G6h7j7ce7+z3D+CGBtcyF33wWUEeyJd2Q0wZ51W+4HPmtmRvBj/Li717ZTtoJg\nrxLotJ5DgPQO1tuRPeoZWsue9Wzzs2/DL4FVwD/MrMjMbm5vpXFut3MIWid/jJmXT7C3Py/sptoO\nPB/Oj5VNkOzb8ghBiwjgswQ7I9Xh9CeBs4G1ZvaqmR3f1gLcvdHd73T3DxP8wP8EuMfMDj2A+sY6\nBBjRXMewnt8BDm6n/B7fF4Lv6vB2yjbHdGzYLVdqZjsIWhet41kf83wtwfelU2Y21MweC7vgdhIk\n4o7+J7MJWti9lpJCz/FZgkHZ0wn2mseG8+0Al7uR4B8zWJhZFkFXRmcDv+uBCW294MEgYB1Bt8pn\ngQc7WM4igi6bZh3VcxtQ0856O7uc7x71DI2h83ruvSL3Sne/yd3HA+cBN5rZae3EEc92+zPBD/5z\n4ecPQV13A9NjdiZyPThYIFhAMPaTCixvJ9R/AEPM7AiC5PBITB3muvsFwFCCQdnH46j3bne/k+CH\nedp+1rd1+fUErZ9BMY9sdz+7nTBaf19eAkaZWUEHoT9C0A032t1zCZJv6/+b0THPxxB8X1pr6zv2\ns3D+4e6eA1zWxrJjHQq818HrPZ6SQs+RTdB8LyPYg/xpFy33EeBKMzsiHLD8KfCuu6/p5H13h+87\nLRxkHWlmU2NefwC4A2hw9zfaXgQAzwEnxUy3W0//9yD5f4eDk8nhYF8aQT9wE+HgeTvrmWxmnzWz\nFDP7DMEP27Od1HMvZnaumU0MW0I7gcbwAUF/dWwM8W636wh+3J81s4ywrn8GfmPhIYzhZ/zxmPec\nDPyrvVaYuzcATxK0bPIIul6aDzO91Mxy3b0+pg5t1fUGMzvZggH+FDO7PKxT8xFI+1rf1uXnADvN\n7FvhOpLNbIYFRxm1ZY/vi7uvBH4PPBrGmWpm6RYMtDe34LIJ+vFrzGwmQeJq7XtmlhkOil8J/KWN\nMm19x7IJu4bNbCTwjXbibnYSwRFIvZaSQs/xAEGzdgOwlGBc4IC5+0vA94CnCAYfJwAXx/G+OQT/\nPL8haA6/yp574g8CM+i4lQDB4OlU+/cRT53V8+vA+wSDpuXAz4GksFvkJ8CbYTfEca3iLQPOBW4i\n+MH6JnCuu2/rrK5tmAT8k+DH4G3g9+7+Svjaz4Bbwhi+Hkd9muNz4BqCPee/mVk68C2Cbqp3wq6J\nfwJTYt52KXt2ObXlEYK99ifCJNHsc8CacLlfItjDbctu4NcEXWvbCMYXPunuRftZ37uBaWH5pz04\nR+c84AigOFzHXQStjL24+3xgh5kdGzP7eoIdkDsJutJWA/9B8N2CYHzoNjOrJBjQbqtV9CrBZ/0S\n8Ct33+ukx3a+Yz8kGDzeAfwf8Ne24gYIt+nZBN2rvVbzERUi+8SCQye3EhzVsrKTstcA09z9hm4J\nrg+w4MSsWe7e5lhAX2ZmZwDXdjI2F++yxhIkowGtkmaXM7OvEHRhfTPK9URNSUH2i5ndSLAnfmqi\nYxFpT3cmhb5if0/nln7MzNYQDLYd8J6ciPQsaimIiEgLDTSLiEiLXtd9NGTIEB87dmyiwxAR6VXm\nzZu3zd1bnxy5l16XFMaOHUthYWHnBUVEpIWZtT7jv03qPhIRkRZKCiIi0kJJQUREWigpiIhICyUF\nERFpoaQgIiItlBRERKSFkoKISA/X1OT89LllLCpp7yZ8XUdJQUSkh1uxtZJZrxWxcsuuyNelpCAi\n0sPNKS4HYOa4vMjXpaQgItLDvVtczojcdEYdlBH5upQURER6MHdnTnE5M8flEdw2PFpKCiIiPdja\nsmpKK2s5phu6jkBJQUSkR2seTzi2LyQFMzvTzJab2Sozu7mN18eY2ctmtsDMFpnZ2VHGIyLS27xb\nXE5eVioT8gd2y/oiSwpmlgzcCZwFTAMuMbNprYrdAjzu7kcCFwO/jyoeEZHeaM6aMmaO7Z7xBIi2\npTATWOXuRe5eBzwGXNCqjAM54fNcYGOE8YiI9CqbduxmffnubjkUtVmUSWEksD5muiScF+tW4DIz\nKwGeA77S1oLM7BozKzSzwtLS0ihiFRHpcbrz/IRmUSaFtto63mr6EuA+dx8FnA08aGZ7xeTus9y9\nwN0L8vM7vcWoiEifMKe4nIFpKRw6PKfzwl0kyqRQAoyOmR7F3t1DXwAeB3D3t4F0YEiEMYmI9Bpz\nisspGHsQyUndM54A0SaFucAkMxtnZqkEA8mzW5VZB5wGYGaHEiQF9Q+JSL+3dWcNK7fu6tauI4gw\nKbh7A3Ad8AKwjOAooyVmdpuZnR8Wuwm42szeAx4FrnD31l1MIiL9zs+fX05KkvHx6cO6db0pUS7c\n3Z8jGECOnff9mOdLgQ9HGYOISG/z9uoynppfwrUnT+i28xOa6YxmEZEepLahke8+/T6j8zL4yqmT\nun39kbYURERk3/zp1SKKSqu478pjyEhN7vb1q6UgItJDbNy+mzteXsU5hw3n5ClDExKDkoKISA/x\n0rIt1DU0cdMZkxMWg5KCiEgP8XZRGSNy0xk3JCthMSgpiIj0AE1NzturyzhuwuBuu/hdW5QURER6\ngOVbKqmorueECYm9qIOSgohID/D26jIAjp8wOKFxKCmIiPQAb60u45DBmYwclJHQOJQUREQSrLHJ\nebe4jOPHJ7aVAEoKIiIJt2TjDiprGhLedQRKCiIiCdcynqCWgoiIvLW6jAn5WQzNSU90KEoKIiKJ\nVN/YxNw15Qk/FLWZkoKISALNW1tBdV1jjxhPACUFEZGEqWto4rZnljJkYBofmdQzWgq6dLaISILc\n+fIqlm7ayazPHU1O+oBEhwOopSAikhCLN+zgzpdX8R9HjuSMbr7lZkeUFEREulltQyNff+I98rJS\n+cF50xIdzh7UfSQi0o1Wba3kG08u4oPNldx9eQGDMlMTHdIelBRERLpBQ2MTs14v4rf/XElmajK/\nu/gITjv04ESHtRclBRGRbnDfW2v4xfPLOWvGMG67YAb52WmJDqlNSgoiIt3g+cWbmTEyhz9cdnSi\nQ+mQBppFRCJWUVXH/HUVnDq153UXtaakICISsddWltLkcMqU/ESH0iklBRGRiL38wVYGZ6XyoVGD\nEh1Kp5QUREQi1NjkvLqilJMm55OUZIkOp1NKCiIiEVq4fjsV1fWcMnVookOJi5KCiEiEXlm+leQk\n48RJPX88AZQUREQi9a8PtnL0mIPIzewZF7zrjJKCiEhEtuysYcnGnZw8tXe0EkBJQUQkMq8s3wrA\nqb1kPAGUFEREIlHf2MSs14oYn5/FlIOzEx1O3JQUREQi8NicdawureLmM6di1vMPRW2mpCAi0sV2\n1tTzm3+u5LjxeXxsWs+/tEUsJQURkS5258urqKiu45ZzpvWqVgIoKYiIdKn15dXc+8YaPnHkKGaM\nzE10OPtMSUFEpIts3L6bLz00j6Qk+MbHpyQ6nP0SaVIwszPNbLmZrTKzm9sp82kzW2pmS8zskSjj\nERGJytw15Zx/xxusLavm95cexbDc9ESHtF8iu8mOmSUDdwIfA0qAuWY2292XxpSZBHwb+LC7V5hZ\n7zmYV0Qk9Nf5JXzrqUWMOiiTx645molDe88hqK1Feee1mcAqdy8CMLPHgAuApTFlrgbudPcKAHff\nGmE8IiJd7tUVpXzjyUUcOy6PP1x2NLkZveNyFu2JsvtoJLA+ZroknBdrMjDZzN40s3fM7My2FmRm\n15hZoZkVlpaWRhSuiMi+Wb65kusens+koQOZ9fmCXp8QINqk0NZxWN5qOgWYBJwMXALcZWZ73YXC\n3We5e4G7F+Tn955riIhI31VaWctV980lPTWZe644hoFpfeOW91EmhRJgdMz0KGBjG2X+5u717l4M\nLCdIEiIiPZK78/zizVz0x7coq6rl7ssLGDEoI9FhdZkok8JcYJKZjTOzVOBiYHarMk8DpwCY2RCC\n7qSiCGMSEdkvTU3Ou0VlfGbWO3zpoXkMSE7ivitncngvuMXmvtin9o6ZHQSMdvdFnZV19wYzuw54\nAUgG7nH3JWZ2G1Do7rPD184ws6VAI/ANdy/b51qIiERk6cadPDZ3HS8s2cyWnbUMzkrlxxfO4OJj\nRpOS3PdO9TL31t38rQqYvQKcT5BAFgKlwKvufmPk0bWhoKDACwsLE7FqEelnauobmfmTf1LX2MTJ\nk4dy5oxhnD7t4F45fmBm89y9oLNy8dQs1913mtkXgXvd/Qdm1mlLQUSkt3tr9TZ21jRw75XHcMqU\n/nEaVTxtnxQzGw58Gng24nhERHqMFxZvITsthRMmDE50KN0mnqRwG0Hf/2p3n2tm44GV0YYlIpJY\nDY1NvLhsC6dMHUpaSnKiw+k2nXYfufsTwBMx00XAJ6MMSkQk0QrXVlBeVcfHpw9LdCjdqtOWgplN\nNrOXzGxxOH24md0SfWgiIonz/OLNpKYkcfKU/nXCbDzdR38muGhdPUB4OOrFUQYlIpJI7s4/lmzm\nxElDyOqFRxodiHiSQqa7z2k1ryGKYEREeoL3N+xg444azuhnXUcQX1LYZmYTCK9bZGYXAZsijUpE\nJIFeWLKZ5CTj9EN71/2Vu0I87aIvA7OAqWa2ASgGLo00KhGRBGlqcv6+eDMzx+aRl5Wa6HC6XTxJ\nwd39dDPLApLcvdLMxkUdmIhIItz31hqKSqv4yqkTEx1KQsTTffQUgLtXuXtlOO/J6EISEUmMlVsq\nuf35Dzht6lAuPKL17V/6h3ZbCmY2FZgO5JrZJ2JeygF6581HRUTaUdfQxNceX8jAtBRu/+ThmLV1\nS5i+r6PuoynAucAg4LyY+ZUEt9EUEekz/uellSzesJM/fe5o8rPTEh1OwrSbFNz9b8DfzOx4d3+7\nG2MSEelW7xSV8ftXVnHR0aP63RnMrcUz0LzAzL5M0JXU0m3k7ldFFpWISDcp21XLVx9bwCGDs7j1\n/OmJDifh4hlofhAYBnwceJXgtpqVHb5DRKQXaGpybnriPSqq67njs0f2yvskdLV4ksJEd/8eUOXu\n9wPnAIdFG5aISPTueqOIV5aX8r1zDmX6iNxEh9MjxJMU6sO/281sBpALjI0sIhGRbvCvD7bwi+eX\nc9aMYVx23CGJDqfHiKetNCu8N/P3gNnAQOD7kUYlIhKhV1eU8qUH5zNtRA4/v6j/Hn7alnjup3BX\n+PRVYHy04YiIROutVdu45oFCJg4dyANXzSQnfUCiQ+pROk0KZjYI+DxBl1FLeXe/PrqwRES6VlOT\n8+jcdfz42WWMHZzFQ188lkGZ/e/aRp2Jp/voOeAd4H2gKdpwRES6XvG2Km5+ahHvFpdzwoTB/O7i\nI/vlxe7iEU9SSHf3GyOPRESki63aWsm9b67hyXklpKYk8fNPHsanC0ZrDKED8SSFB83sauBZoLZ5\npruXRxaViMgBWF26i1tnL+H1ldtITUniwiNGcNMZUzg4R5dt60w8SaEO+CXwXcIb7YR/NegsIj3O\nu0VlXPPgPJIMvn7GZC6ZOYbBA/vvtYz2VTxJ4UaCE9i2RR2MiMiB+NvCDXzjiUWMysvgvitmMmZw\nZqJD6nXiSQpLgOqoAxER2V+1DY38+h8rmPVaEceOy+NPnztaRxbtp3iSQiOw0MxeZs8xBR2SKiIJ\n98Hmndzw2EI+2FzJpceO4fvnTSMtJTnRYfVa8SSFp8OHiEiPsWZbFQ++s5YH315LTsYA7rmigFOn\nHpzosHq9eM5ovr87AhER6UxVbQMvL9/KE4UlvLqilJQk47wPjeCWcw7VYHIX6eh2nI+7+6fN7H3+\nfdRRC3c/PNLIRESALTtreHVFKS8u3cJrK0qpbWji4Jw0vnb6ZC6ZOZqhOsy0S3XUUvhq+Pfc7ghE\nRPqnxiansqaeXbUNVNU2snHHbopKqygq3cW8tRV8sDm4fcvw3HQumTmGs2YMo2BsHslJOgEtCh3d\njnNT+PRad/9W7Gtm9nPgW3u/S0Rk31z0x7dYsG77XvNz0lOYMTKXb581lRMn5zN1WLbORO4G8Qw0\nf4y9E8BZbcwTEdknWytrWLBuO+cePpwTJ+WTlZbC0Jw0xg/JIi8rVUkgAToaU/gv4Fpggpktinkp\nG3gz6sBEpO+bW1wBwBc/Op4jRg9KcDQCHbcUHgH+DvwMuDlmfqWueyQiXWFOcRmZqclMH5GT6FAk\n1O7tON19h7uvAW4BNrv7WmAccFl4jwURkQPybnE5Rx9yEAOS47kzsHSHeLbEU0CjmU0E7iZIDI9E\nGpWI9Hk7qutZvqWSY8bmJToUiRFPUmhy9wbgE8Bv3f1rwPB4Fm5mZ5rZcjNbZWY3d1DuIjNzMyuI\nL2wR6e0K15bjDjPHKSn0JPEkhXozu4TglpzPhvM6vampmSUDdxIcqTQNuMTMprVRLhu4Hng33qBF\npPebU1xOanKSBph7mHiSwpXA8cBP3L3YzMYBD8XxvpnAKncvcvc64DHggjbK/Qj4BVATZ8wi0ge8\nW1zOh0bnkj5AF6/rSdpNCmaWA+DuS939end/NJwuJr4xhZHA+pjpknBe7DqOBEa7+7OISL9RVdvA\n4g07NJ7QA3XUUnil+YmZvdTqtXiumtrWWSct11AysyTgN8BNnS7I7BozKzSzwtLS0jhWLSI92YJ1\n22loco0n9EAdJYXYH/XWWy6e0wxLgNEx06OAjTHT2cAM4BUzWwMcB8xua7DZ3We5e4G7F+Tn58ex\nahHpyeYUl5FkcPQhByU6FGmlo6Tg7Txva7otc4FJZjbOzFKBi4HZLQsIzoMY4u5j3X0s8A5wvrsX\nxhe6iPRWc9aUM31ELtnpnR6zIt2sozOah5rZjQStgubnhNOd7q67e4OZXQe8ACQD97j7EjO7DSh0\n99kdL0FE+qLFG3ZQuKaCqz4yLtGhSBs6Sgp/Jujiaf0c4K54Fu7uzwHPtZr3/XbKnhzPMkWk96qq\nbeD6RxcweGAqXzppQqLDkTZ0dOnsH3ZnICLS9/3wmSUUl1Xx8BePJS8rNdHhSBt0wRER6RbPvLeR\nxwtLuPbkCZwwYUiiw5F2xHM/BRGR/bartoG7Xi/iT68WccToQdxw+uREhyQdUFIQkUi4Ow+9u47f\nvriCsqo6zj5sGD84b7quiNrDdZoUzOxg4KfACHc/K7x+0fHufnfk0YlIr/XQu+v43tOLOW58Hnef\ndaiucdRLxJOy7yM4rHREOL0CuCGqgESk93tv/XZ+9MxSTp06lEe+eJwSQi8ST1IY4u6PA00QnH8A\nNEYalYj0WhVVdVz78Hzys9P4709/iKQk3We5N4lnTKHKzAYTnsVsZscBOyKNSkR6pd11jdz4+EJK\nK2t58r+OZ1CmDjvtbeJJCjcSXJ5igpm9SXA280WRRiUivUp1XQMPv7OOP71WxLZdtfz4whkcPkpd\nRr1Rp0nB3eeb2UnAFIJLXCx39/rIIxORHsvdea9kB4VrylmwfjtvrdpGRXU9H5k4hK+efpQuid2L\nxXP00edbzTrKzHD3ByKKSUR6uF//YwV3vLwKgJGDMvjopHwuP+EQjj5EyaC3i6f76JiY5+nAacB8\nQElBpB96dM467nh5FZ86ehTf+PgUhuakJzok6ULxdB99JXbazHKBByOLSER6rJeXb+WWpxdz0uR8\nfvaJw0jRiWh9zv6c0VwNTOrqQESkZ3v5g61c98h8phyczZ2XHqWE0EfFM6bwDP++qU4SMA14PMqg\nRKTnWFdWzW3PLuWfy7YwIT+Le688hoFpukJOXxXPlv1VzPMGYK27l0QUj4j0ABu27+aNlaW8tnIb\nLy7dQkqScfNZU7nqw+NITVELoS+LZ0zh1e4IREQSq7Kmnmfe28Rf5q7jvZLg/NSh2Wl88qiRXH/a\nJIbnZiQ4QukO7SYFM6uk7XsxG+DunhNZVCISGXdne3U9m3fWUFRaxbJNO1m6aSdvry5jd30jUw7O\n5jtnT+XkKUOZNHQgZrpMRX/S0Z3Xstt7TUR6n7qGJm564j3+sWQztQ1NLfOTk4yJ+QO58MiRfLpg\nFEeMHqRE0I/FPVpkZkMJzlMAwN3XRRKRiHS5hsYmvvrYAv6+eDOXzBzDxKEDGZ6bzpi8TCYOHUj6\ngOREhyg9RDxHH50P/Jrg0tlbgUOAZcD0aEMTka7Q1OR888lF/H3xZr537jS+8JFxiQ5JerB4DiP4\nEXAcsMLdxxGc0fxmpFGJSJfYsH03N/xlIX9dsIGbPjZZCUE6FU/3Ub27l5lZkpklufvLZvbzyCMT\nkf1WvK2KP7yyir/O3wDADadP4rpTJyY4KukN4kkK281sIPAa8LCZbSU4X0FEehB3p3BtBX9+rYgX\nl20hNTmJS48dwzUnTWDkIB1OKvGJJylcANQAXwMuBXKB26IMSkTit3lHDf/3/ib+tnADi0p2kJsx\ngGtPnsDlJ4xlaLYuVif7pqPzFO4AHnH3t2Jm3x99SCLSHndn884a3lu/nQXrt1O4poL56ypwh2nD\nc7jtgulcdPQoMlN1GQrZPx19c1YCvzaz4cBfgEfdfWH3hCUiABu37+aV5aW8uXobRaVVrC2rorou\nuEX6gGRj2ohcbjhtMud+aDgT8gcmOFrpCzo6ee13wO/M7BDgYuBeM0sHHgUec/cV3RSjSL/Q3Aoo\nXFNB4Zpy3i0u54PNlQCMyE1n6vAcjh8/mLFDMjlsZC7TRuSQlqLzC6RrmXtbV7Jop7DZkcA9wOHu\nnpBvY0FBgRcWFiZi1SL7zd2pqmuktLKW0spatlbWsHlH8Ni4Yzdry6pZW1bNrtrgGI7M1GSOGnMQ\nJ03O55Sp+UzI1+Um5MCY2Tx3L+isXDwnrw0AziRoLZwGvAr88IAjFOmDZr22mr/O30BDk9PY5NTU\nN1JV20BVXSONTXvvgKUPSGJEbgZjBmdyzNg8xg3J4qgxB3Ho8Gzdr0ASoqOB5o8BlwDnAHOAx4Br\n3L2qm2IT6VXcnbteLyZ9QDIzRuaQkpREakoSA9NSyEpLJid9APnZaS2P4TkZ5GSkqAUgPUpHLYXv\nAI8AX3f38m6KR6TXWltWzdb8d16TAAAQVklEQVTKWn584QwuO+6QRIcjsl86Gmg+pTsDEent5qwJ\n9p2OHZeX4EhE9p86LUW6yJzicg7KHMDEoTo0VHovJQWRLjKnuJxjxuZpjEB6NSUFkS6wacdu1pVX\nM1NdR9LLKSmIdIE5xc3jCYMTHInIgVFSEOkCc9eUk5WazKHDdRdb6d2UFES6wJzico4em6cTzqTX\ni/QbbGZnmtlyM1tlZje38fqNZrbUzBaZ2UvhdZZEepXyqjpWbNmlQ1GlT4gsKZhZMnAncBYwDbjE\nzKa1KrYAKHD3w4EngV9EFY9IVOaG5ydokFn6gihbCjOBVe5e5O51BJfJuCC2gLu/7O7V4eQ7wKgI\n4xGJxNziclJTkjh8VG6iQxE5YFEmhZHA+pjpknBee74A/L2tF8zsGjMrNLPC0tLSLgxR5MDU1Dfy\n4rItHDF6kC5jLX1ClEmhrTN42rxOt5ldBhQAv2zrdXef5e4F7l6Qn5/fhSGKHJif/N8y1pZV8+VT\nJiY6FJEuEeU9+0qA0THTo4CNrQuZ2enAd4GT3L02wnhEutQLSzbz4Dtr+eJHxnHSZO2sSN8QZUth\nLjDJzMaZWSrB/RhmxxYIb9rzJ+B8d98aYSwiXWrTjt1866lFTB+RwzfOnJLocES6TGRJwd0bgOuA\nF4BlwOPuvsTMbjOz88NivwQGAk+Y2UIzm93O4kR6jPdLdnD1A4XUNTTxv5ccqbEE6VOi7D7C3Z8D\nnms17/sxz0+Pcv0iXWltWRW/+scKnnlvIwdlDuC/P30E4/N1RVTpWyJNCiJ9RVHpLs6/400am5yv\nnDqRq08cT076gESHJdLllBREOrG7rpH/emg+qSlJ/O3LH2Z0XmaiQxKJjJKCSAfcne8+/T4rtlby\nwFUzlRCkz9PVu0Q68Je56/nr/A3ccNpkPjpJh51K36eWgkgbVm2t5H//tYpn3tvIiZPz+cqpOjlN\n+gclBRGgsclZsaWS+esqeGPlNp5fspmMAclcfeJ4rjtlIklJusWm9A9KCtLvLVhXwRfuL6S8qg6A\nwVmpfOmkCVz90fHkZaUmODqR7qWkIP3a2rIqvnB/IQPTUvj+udM4cswgxuRlYqaWgfRPSgrSb1VU\n1XHFvXNpcue+K4/RiWgiKClIP7Vjdz1XP1DIhu27eeSLxyohiISUFKRfaWpynppfws+f/4Dyqjr+\n55IjKRirO6aJNFNSkH6hpKKaN1Zu4y+F61mwbjtHjRnEfVfOZMZI3S1NJJaSgvRZ68ureWJeCc++\nt5GibVUAjMhN51ef+hCfOHKkDjMVaYOSgvQZ5VV1LNu0k6Ubd/LaylLeWLUNgA9PGMKlxx3CRycN\nYdLQgTqySKQDSgrSa/39/U08Onc9m3fsZtOOGiprGlpeG3VQBtefOolPHzOakYMyEhilSO+ipCC9\n0rOLNnL9owsYk5fJlGHZHD9+MKMOyuTQ4TkcOjybwQPTEh2iSK+kpCC9zj+XbuGGxxZScEge9181\nk4xU3flMpKsoKUiv4e68sGQz1z+2kGkjcrj7igIlBJEupqQgPV5Tk/Pisi3c+fIqFpXsYOqwbB64\naibZuvOZSJdTUpAeq6a+kacXbOCuN4pZtXUXY/Iy+dknDuMTR40kLUUtBJEoKClIj7NqayWzF27k\nkTnr2LarjmnDc/jdxUdwzmHDSUnWfaFEoqSkIAm3o7qehSXbmb+2gheWbOaDzZWYwUmT87n6o+M5\nYcJgnVsg0k2UFKRbrS+v5vWV21ixpZI1ZVUUb6tibVk1AGZw1JiDuPW8aZx92HCG5qQnOFqR/kdJ\nQSJVtquWwrUVzCku59UVpazauguAzNRkDhmcxfQROXy6YDRHjB7E4aNyNXgskmBKCnJAGpuc8qo6\nNu+oYfPOGjZu383asmrWllWxunQXa8JWQGpKEseOy+OzM8dw8pR8xg3JUpeQSA+kpCDtWlSynScK\nS2hoaqK+0alraKKqtoGqugZ27m6gdFct5VV1NDb5Hu9LH5DE2MFZTB2Ww8Uzx3DM2IOYMTJXRwyJ\n9AJKCtKmrTtruOLeueyuayQ7PYWUJGNAShJZqSkMTEtheG46h4/KJT87jfzsNA7OSWd4bjrDctPJ\nH5imVoBIL6WkIHtpanJufPw9qusaePYrH2Hi0OxEhyQi3UQHfcteZr1exBurtvGD86YrIYj0M0oK\nsofCNeX86oXlnDVjGBcfMzrR4YhIN1P3kQAwb20Ff3p1NS8u28KI3Axu/8ThGhcQ6YeUFPohd6dw\nbQXvFpWxbHMlyzbupGhbFbkZA/jKKRO5/ISx5GbqfAGR/khJoR+pqW9k9nsbuffNNSzbtBOA0XkZ\nTB2Ww+ePP4RPFYwmK01fCZH+TL8AfUxTkzNvXQUlFdVUVNWzvbqO4rJqVm6ppGhbFXUNTUw5OJvb\nP3EY5xw+XGcQi8gelBT6AHdnbVk1f51fwlPzN7Bh++6W18xg5KAMJg0dyImT8zl5cj7H6wJzItIO\nJYVeorHJ2bKzhpKK3ZRW1rJtVy2bd9awdONOFm/YQVlVHWbw0Un53HzWVKaPyGFQZiq5GQNITlIC\nEJH4KCn0ADX1jbyyvJQF6yrYXd/I7rpGdtc3srOmgcqaeiqq6tiwfTf1jXteTiI5yZg0dCCnTB3K\nYSNz+di0gxkxKCNBtRCRvkBJIQGqahtYXbqL1aW7eHt1GX9fvJnKmgZSk5PISksmfUAyGQOSyc4Y\nQE56CiMHZXDmjOGMyctk1EEZ5GenMWRgGnlZqWoFiEiXijQpmNmZwO+AZOAud7+91etpwAPA0UAZ\n8Bl3XxNlTFFzd3bVNrC9up6tlTVs3F7Dph27WVdeTfG2KopKq9i0o6al/MC0FM6YfjAXHDGSD08Y\nrDuLiUhCRZYUzCwZuBP4GFACzDWz2e6+NKbYF4AKd59oZhcDPwc+E1VM8XB3ahuaqKlvpKa+iaq6\nBqprG9lV28CO3XWUV9VTUV1HaWUtWytrKK2sZefu4MqhVbUNVNY00NDqqqEAOekpjM8fyPHjBzM+\nP4uJQwcycehADhmcxQAlAhHpIaJsKcwEVrl7EYCZPQZcAMQmhQuAW8PnTwJ3mJm5+96/qgfo8bnr\nmfV6EU3u4NDkTkOT09DoNDQ1UdsQPOoamuJaXnZaCvk5aQzNTmPskEyyUlPISE0mN2MAgzIHMCgz\nlfyBaQwflM6IQRnk6NBPEekFokwKI4H1MdMlwLHtlXH3BjPbAQwGtsUWMrNrgGsAxowZs1/BDMoc\nwJSDs8EgyQwDUpKNlCQjJTmJtJQk0lKSSUtJIn1AMukDgr+ZqclkpaaQmZbMoIxU8rJSGZQ5gPQB\nujeAiPQ9USaFtkZAW7cA4imDu88CZgEUFBTsVyvijOnDOGP6sP15q4hIvxFlZ3YJEHuZzVHAxvbK\nmFkKkAuURxiTiIh0IMqkMBeYZGbjzCwVuBiY3arMbODy8PlFwL+iGE8QEZH4RNZ9FI4RXAe8QHBI\n6j3uvsTMbgMK3X02cDfwoJmtImghXBxVPCIi0rlIz1Nw9+eA51rN+37M8xrgU1HGICIi8dMB8iIi\n0kJJQUREWigpiIhICyUFERFpYb3tCFAzKwXW7ufbh9DqbOl+oj/Wuz/WGfpnvftjnWHf632Iu+d3\nVqjXJYUDYWaF7l6Q6Di6W3+sd3+sM/TPevfHOkN09Vb3kYiItFBSEBGRFv0tKcxKdAAJ0h/r3R/r\nDP2z3v2xzhBRvfvVmIKIiHSsv7UURESkA0oKIiLSot8kBTM708yWm9kqM7s50fFEwcxGm9nLZrbM\nzJaY2VfD+Xlm9qKZrQz/HpToWLuamSWb2QIzezacHmdm74Z1/kt4+fY+xcwGmdmTZvZBuM2P7yfb\n+mvh93uxmT1qZul9bXub2T1mttXMFsfMa3PbWuB/wt+2RWZ21IGsu18kBTNLBu4EzgKmAZeY2bTE\nRhWJBuAmdz8UOA74cljPm4GX3H0S8FI43dd8FVgWM/1z4DdhnSuALyQkqmj9Dnje3acCHyKof5/e\n1mY2ErgeKHD3GQSX5b+Yvre97wPObDWvvW17FjApfFwD/OFAVtwvkgIwE1jl7kXuXgc8BlyQ4Ji6\nnLtvcvf54fNKgh+JkQR1vT8sdj9wYWIijIaZjQLOAe4Kpw04FXgyLNIX65wDnEhwTxLcvc7dt9PH\nt3UoBcgI79aYCWyij21vd3+Nve9C2d62vQB4wAPvAIPMbPj+rru/JIWRwPqY6ZJwXp9lZmOBI4F3\ngYPdfRMEiQMYmrjIIvFb4JtAUzg9GNju7g3hdF/c3uOBUuDesNvsLjPLoo9va3ffAPwKWEeQDHYA\n8+j72xva37Zd+vvWX5KCtTGvzx6La2YDgaeAG9x9Z6LjiZKZnQtsdfd5sbPbKNrXtncKcBTwB3c/\nEqiij3UVtSXsR78AGAeMALIIuk9a62vbuyNd+n3vL0mhBBgdMz0K2JigWCJlZgMIEsLD7v7XcPaW\n5uZk+HdrouKLwIeB881sDUG34KkELYdBYfcC9M3tXQKUuPu74fSTBEmiL29rgNOBYncvdfd64K/A\nCfT97Q3tb9su/X3rL0lhLjApPEIhlWBganaCY+pyYV/63cAyd//vmJdmA5eHzy8H/tbdsUXF3b/t\n7qPcfSzBdv2Xu18KvAxcFBbrU3UGcPfNwHozmxLOOg1YSh/e1qF1wHFmlhl+35vr3ae3d6i9bTsb\n+Hx4FNJxwI7mbqb90W/OaDazswn2IJOBe9z9JwkOqcuZ2UeA14H3+Xf/+ncIxhUeB8YQ/FN9yt1b\nD2L1emZ2MvB1dz/XzMYTtBzygAXAZe5em8j4upqZHUEwuJ4KFAFXEuzo9eltbWY/BD5DcLTdAuCL\nBH3ofWZ7m9mjwMkEl8feAvwAeJo2tm2YHO8gOFqpGrjS3Qv3e939JSmIiEjn+kv3kYiIxEFJQURE\nWigpiIhICyUFERFpoaQgIiItlBREQmY2zMweM7PVZrbUzJ4zs8n7sZy7mi+4aGbfifM9a8xsyL6u\nS6Sr6ZBUEVpO/HsLuN/d/xjOOwLIdvfXD2C5u9x9YBzl1hBc+XPb/q5LpCuopSASOAWob04IAO6+\nEFhgZi+Z2Xwze9/MLoDggoPhfQzuD69h/6SZZYavvWJmBWZ2O8HVPBea2cPha0+b2bzwfgDXJKCe\nIh1SUhAJzCC42mZrNcB/uPtRBInj12GrAmAKMMvdDwd2AtfGvtHdbwZ2u/sR4aU3AK5y96OBAuB6\nMxscQV1E9puSgkjHDPipmS0C/klwOYWDw9fWu/ub4fOHgI/Esbzrzew94B2Ci5hN6uJ4RQ5ISudF\nRPqFJfz7gmqxLgXygaPdvT7s+08PX2s9INfhAF14babTgePdvdrMXolZlkiPoJaCSOBfQJqZXd08\nw8yOAQ4huF9DvZmdEk43G2Nmx4fPLwHeaGO59eHlzAFygYowIUwluGWqSI+ipCACeHAY3n8AHwsP\nSV0C3Ao8BxSYWSFBq+GDmLctAy4Pu5byaPveuLOAReFA8/NASlj+RwRdSCI9ig5JFdkP4e1Onw1v\nHi/SZ6ilICIiLdRSEBGRFmopiIhICyUFERFpoaQgIiItlBRERKSFkoKIiLT4/4EmbUnRp+/0AAAA\nAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1d9de122198>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plotting Final Policy (action stake) vs State (Capital)\n",
    "\n",
    "# x axis values\n",
    "x = range(100)\n",
    "# corresponding y axis values\n",
    "y = v[:100]\n",
    " \n",
    "# plotting the points \n",
    "plt.plot(x, y)\n",
    " \n",
    "# naming the x axis\n",
    "plt.xlabel('Capital')\n",
    "# naming the y axis\n",
    "plt.ylabel('Value Estimates')\n",
    " \n",
    "# giving a title to the graph\n",
    "plt.title('Final Policy (action stake) vs State (Capital)')\n",
    " \n",
    "# function to show the plot\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAGoxJREFUeJzt3Xu8HGV9x/HP13AXQghJMJDEgA0X\naw2XIwWxlIu0SCmJBSkUMW3B9KJyEbWovFpQq9AqUK9tCmKK3CQg1xZJYyKlhUAihIsgCYgQE5MA\nCQEBTeDXP+Y5sBzO7pk9Z2f27M73/Xrta3dm5/KbmWR/53memedRRGBmZtX1pnYHYGZm7eVEYGZW\ncU4EZmYV50RgZlZxTgRmZhXnRGBmVnFOBDZsSTpB0q1DWH+BpJNbGVMT+x5S7H229bik97ZiWzXb\nPEjS8prpByUd1Mp9WOdwIrAhk/RnkhZJel7SSkn/Jek9Q91uRFwWEX9Qs5+Q9FtD3W6rSJqcYnq+\n5rUE3hh7gTF8R9Jv0r6fkTRX0u7NbicifjsiFhQQonUAJwIbEkkfBy4EvgjsAEwCvglMa2dcJRsV\nEVun19Q27P+fImJrYAKwGvhOG2KwDuZEYIMmaVvgc8BHIuLaiPhVRGyIiBsj4pNpmX0l3SFpXSot\nfF3SZjXbCEmnSHpM0lOS/lnSm9J3fy7p9vT5trTKkvTX759K2k7STZLWSFqbPk/IEfeOkl6UNLpm\n3l5p/5tK+i1JP5L0bJp31SDOzaux1xznX0tammL9hiSl794m6YeSnk77u0zSqGb3GREvAJcD70jb\n3VzShZJWpNeFkjavE++r1U+SRkj6jKRHJT0nabGkiSnmr/RZ70ZJpzUbqw0vTgQ2FPsDWwDfb7DM\ny8DpwJi0/KHA3/ZZ5v1AD7A3WUniL/tuJCIOTB+npr+8ryL793sJ8FayksiLwNcHCjoiVgB3AEfX\nzP4zYE5EbAA+D9wKbEf2V/bXBtpmTkcC7wKmAscCf5jmC/gSsCOwBzAROLvZjUvaGjgBuCfN+iyw\nH7Bn2ue+wFk5NvVx4HjgCGAk2fV4AZgNHF+TqMeQXc8rmo3VhhcnAhuK7YGnImJjvQUiYnFE3BkR\nGyPiceDfgN/vs9h5EfFMRDxBVs10fJ6dR8TTEXFNRLwQEc8B/9jPtuu5vHc/6S/z49I8gA1kyWXH\niHgpIm7vfxOveiqVeNZJ+kSD5c6NiHXpOOeT/UATEcsiYm5E/Doi1gDnN3EcAJ+QtA5YBmwN/Hma\nfwLwuYhYnbZ7DnBiju2dDJwVET+NzJJ0ru8CniX78YfsnC2IiFVNxGrDkBOBDcXTwBhJm9RbQNKu\nqcrml5LWk7UljOmz2JM1n39O9pfxgCRtJenfJP08bfs2YJSkETlWnwPsL2lH4EAggP9J332K7K/0\nu9LdNG8oofQxJiJGpdeXGyz3y5rPL5D9aCNpnKQrJf0iHcd3eeM5auTLad9viYijIuLRNH9HsvPZ\nK++5nQg8Wue72cAH0+cPApc2EacNU04ENhR3AC8B0xss8y3gYWBKRIwEPkP2I1trYs3nScCKnPs/\nA9gN+N207d7qo77bf4OIWEdW/XMsWbXQFZG64o2IX0bEhyNiR+CvgG8WfLfSl8gS0TvTcXyQHMeQ\nwwqykk2vvOf2SeBtdb77LjBN0lSyaqzrhhShDQtOBDZoEfEs8PfANyRNT3+hbyrpfZL+KS22DbAe\neD7d1vg3/Wzqk6nhdyJwKlCvcXYVsEvN9DZk7QLrUsPvPzR5CJcDHyJrK+itFkLSB2oandeS/Ui/\n3OS2m7EN8DzZcewEfLJF270COEvS2FSf//dkP+QDuQj4vKQpyrxT0vYAEbEcuJusJHBNRLzYolit\njZwIbEgi4nyyxsWzgDVkf01+lNf+UvwE2V/czwH/Tv8/8tcDi4F7gZuBi+vs7mxgdqqLP5asPWFL\n4CngTuCWJsO/AZgCrIqIJTXz3wUslPR8WubUiPhZk9tuxjlkDeXPkh3/tS3a7heARcB9wP3Aj9O8\ngZwPfI+sxLSe7HpsWfP9bOB3cLVQ15AHprF2khRk1UbL2h2L5SPpQLKSxeSIeKXd8djQuURgZrlJ\n2pSs+u4iJ4Hu4URgZrlI2gNYB4wnq5azLuGqITOzinOJwMys4uo+CDScjBkzJiZPntzuMMzMOsri\nxYufioixAy3XEYlg8uTJLFq0qN1hmJl1FEk/H3gpVw2ZmVWeE4GZWcU5EZiZVZwTgZlZxTkRmJlV\nnBOBmVnFFXr7qKTHyXqdfBnYGBE9qbvgq4DJwOPAsRGxtsg4zMysvjJKBAdHxJ4R0ZOmzwTmRcQU\nYF6aNjOzNmlH1dA0sv7MSe+NRrcyM7OCFZ0IArhV0mJJM9O8HSJiJUB6H9ffipJmSlokadGaNWsK\nDtNs8C6Y+wgXzH2k3WGYDVrRXUwcEBErJI0D5kp6OO+KETELmAXQ09PjLlLNzApSaIkgIlak99XA\n94F9gVWSxgOk99VFxmBmZo0VlggkvVnSNr2fgT8AHiAbA3ZGWmwG2Xi1ZmbWJkVWDe0AfF9S734u\nj4hbJN0NfE/SScATwAcKjMGs5WrbA04/bNc2RmLWGoUlgoh4DJjaz/yngUOL2q+ZmTXHTxabmVWc\nE4GZWcV1xAhlZu3m5wSsm7lEYGZWcU4EZmYV50RgZlZxbiMwq8PtAlYVLhGYmVWcE4GZWcU5EZiZ\nVZzbCMxquF3AqsglAjOzinMiMDOrOCcCM7OKcyIwM6s4JwIzs4pzIjAzqzgnAjOzinMiMDOrOD9Q\nZpXkAejNXuMSgZlZxTkRmJlVnBOBmVnFORGYmVWcE4GZWcU5EZiZVZwTgZlZxfk5Autqfl7AbGAu\nEZiZVZwTgZlZxTkRmJlVnBOBmVnFFZ4IJI2QdI+km9L0zpIWSloq6SpJmxUdg5mZ1VdGieBU4KGa\n6fOACyJiCrAWOKmEGMzMrI5CE4GkCcAfARelaQGHAHPSIrOB6UXGYGZmjRVdIrgQ+BTwSpreHlgX\nERvT9HJgp/5WlDRT0iJJi9asWVNwmGZm1VVYIpB0JLA6IhbXzu5n0ehv/YiYFRE9EdEzduzYQmI0\nM7Ninyw+ADhK0hHAFsBIshLCKEmbpFLBBGBFgTGYmdkACisRRMSnI2JCREwGjgN+GBEnAPOBY9Ji\nM4Dri4rBzMwG1o7nCP4O+LikZWRtBhe3IQYzM0tK6XQuIhYAC9Lnx4B9y9ivmZkNzE8Wm5lVnBOB\nmVnFORFYR7lg7iOvG2PAzIbOicDMrOKcCMzMKs6JwMys4hrePippC+BI4PeAHYEXgQeAmyPiweLD\nMzOzotVNBJLOBv6Y7P7/hcBqsq4idgXOTUnijIi4r/gwzcysKI1KBHdHxNl1vjtf0jhgUutDMjOz\nMtVNBBFxc+20pDdHxK9qvl9NVkowM7MONmBjsaR3S/oJaZQxSVMlfbPwyMzMrBR57hq6APhD4GmA\niFgCHFhkUGZmVp5ct49GxJN9Zr1cQCxmZtYGeXoffVLSu4GQtBlwCq8fjN7MzDpYnhLBXwMfIRtb\neDmwZ5o2M7MukKdE8EoaWexVknYmtRmYmVlny1MiuFHSyN4JSXsANxYXkpmZlSlPIvgiWTLYWtI+\nwBzgg8WGZWZmZRmwaigibpa0KXArsA0wPSKWFh6ZmZmVolFfQ18DombWSOAx4GOSiIhTig7OzMyK\n16hEsKjP9OIiAzEzs/Zo1NfQ7DIDMTOz9hiwjUDSFOBLwNvJuqEGICJ2KTAuMzMrSZ67hi4BvgVs\nBA4G/gO4tMigzMysPHkSwZYRMQ9QRPw8jVFwSLFhmZlZWfI8WfySpDcBSyV9FPgFMK7YsMzMrCx5\nSgSnAVuRdTa3D9nDZB8qMigzMytPnkQwOSKej4jlEfEXEXE0HqLSzKxr5EkEn845z8zMOlCjJ4vf\nBxwB7CTpqzVfjSS7g8jMzLpAo8biFWRPFx/F658qfg44vcigzMysPI2eLF4CLJF0eURsAJC0HTAx\nItaWFaCZmRUrTxvBXEkjJY0GlgCXSDp/oJUkbSHpLklLJD0o6Zw0f2dJCyUtlXRVGv7SzMzaJE8i\n2DYi1gN/AlwSEfsA782x3q+BQyJiKtnwlodL2g84D7ggIqYAa4GTBhe6mZm1Qp5EsImk8cCxwE15\nNxyZ59PkpukVZE8lz0nzZwPT84drZmatlicRfA74AbAsIu6WtAuQa2AaSSMk3QusBuYCjwLrIqL3\nrqPlwE7Nh21mZq2SZ4Syq4Gra6YfA47Os/GIeBnYU9Io4PvAHv0t1t+6kmYCMwEmTfLza2ZmRalb\nIpB0Vmogrvf9IZKOzLOTiFgHLAD2A0ZJ6k1AE8huU+1vnVkR0RMRPWPHjs2zGzMzG4RGJYL7yQat\nfwn4MbCGbDyCKWSNv/9NNrB9vySNBTZExDpJW5I1MJ8HzAeOAa4EZgDXt+A4zMxskBo9R3A9cH0a\nmOYAYDywHvguMDMiXhxg2+OB2ZJGkJU8vhcRN0n6CXClpC8A9wAXt+A4zMxskPK0ESwlZ+Nwn/Xu\nA/bqZ/5jwL7Nbs/MzIqR564hMzPrYk4EZmYVN2AiaHTnkJmZdb48JYKFkq6WdIQkFR6RmZmVKk8i\n2BWYBZwILJP0RUm7FhuWmZmVZcBEkPoMmhsRxwMnk937f5ekH0nav/AIzcysUAPePippe7IB608E\nVgEfA24ge6jsamDnIgM0M7NiDZgIgDuAS4HpEbG8Zv4iSf9aTFhmZlaWPIlgt4jot2O4iDivxfGY\nmVnJ8jQW35p6DwWy4Sol/aDAmMzMrER5EsHY1HsoAGm84nHFhWRmZmXKkwhelvTqgACS3kqdMQTM\nzKzz5Gkj+Cxwu6QfpekDSQPGmJlZ58vT++gtkvYmG1RGwOkR8VThkZmZWSkajVC2e3rfG5hENpLY\nL4BJaZ6ZmXWBRiWCM4APA1/p57sADikkImurC+Y+8urn0w9zTyLWer3/xvzva/hoNELZh9P7weWF\nY2ZmZaubCCT9SaMVI+La1odjZmZla1Q19McNvgvAicDMrAs0qhr6izIDsfapbRcwK4LbBYa3PCOU\nbSvpfEmL0usrkrYtIzgzMytenieLvw08BxybXuuBS4oMyszMypPnyeK3RcTRNdPnSLq3qIDMzKxc\neUoEL0p6T++EpAOAF4sLyczMypSnRPA3wOzULiDgGbLhKq2DuYHYiuYG4s6Rp6+he4Gpkkam6fWF\nR2VmZqXJc9fQ9pK+CiwA5kv6lzSOsZmZdYE8bQRXAmuAo4Fj0uerigzKzMzKk6eNYHREfL5m+guS\nphcVkJl1LrcLdKY8JYL5ko6T9Kb0Oha4uejAzMysHHkSwV8BlwO/Tq8rgY9Lek6SG47NzDpcnruG\ntikjEDMza488bQTWwTzQjBXN7QKdL0/V0KBImihpvqSHJD0o6dQ0f7SkuZKWpvftiorBzMwGVlgi\nADYCZ0TEHmQD339E0tuBM4F5ETEFmJemzcysTRqNUDa60YoR8cwA368EVqbPz0l6CNgJmAYclBab\nTfag2t/ljtjMzFqqURvBYrKRyNTPdwHskncnkiYDewELgR1SkiAiVkoaV2edmcBMgEmTJuXdleE6\nWzNrTqMRynZuxQ4kbQ1cA5wWEeul/vJKv/ufBcwC6OnpiVbEYmZmb5TrrqHUoDsF2KJ3XkTclmO9\nTcmSwGU1g92vkjQ+lQbGA6ubD9vMzFolT6dzJwO3AT8AzknvZ+dYT8DFwEMRcX7NVzfwWjfWM4Dr\nmwvZzMxaKU+J4FTgXcCdEXGwpN3JEsJADgBOBO6vGdHsM8C5wPcknQQ8AXyg+bDNrEx+HqW75UkE\nL0XES5KQtHlEPCxpt4FWiojb6b+hGeDQpqI0M7PC5EkEyyWNAq4D5kpaC6woNiwzMytLnr6G3p8+\nni1pPrAtcEuhUZmZWWny3jU0AtgB+Fma9Ray+n1rIz8vYGatMGAikPQx4B+AVcAraXYA7ywwLjMz\nK0neu4Z2i4iniw7GzMzKl6fTuSeBZ4sOxMzM2iNPieAxYIGkm8lGKAOgz0NiVhK3C1iR/LxANeVJ\nBE+k12bpZWZmXSTP7aN5niI2M7MO1Wg8ggsj4jRJN5LdJfQ6EXFUoZGZmVkpGpUILk3vXy4jEDMz\na49GiWANQET8qKRYrA43EJtZkRrdPnpd7wdJ15QQi5mZtUGjRFDbc2juYSnNzKyzNEoEUeezmZl1\nkUZtBFMlrScrGWyZPpOmIyJGFh5dhbldwMzK0mjw+hFlBmJmZu2Rp68hMzPrYk4EZmYV50RgZlZx\nTgRmZhXnRGBmVnFOBGZmFZdr8Hoz60weaMbycInAzKzinAjMzCrOicDMrOLcRtAGtf0IuU+hgfkc\nNae2XcAGVu98Venfm0sEZmYV50RgZlZxTgRmZhXnNoKS5Knndl34a3y+mpOnXcDPFLzG5+v1CisR\nSPq2pNWSHqiZN1rSXElL0/t2Re3fzMzyKbJq6DvA4X3mnQnMi4gpwLw0bWZmbVRYIoiI24Bn+sye\nBsxOn2cD04vav5mZ5VN2G8EOEbESICJWShpXb0FJM4GZAJMmTSopvNZyHXZzhnK+qniuW3G+Brt+\nJxrq8xXd/G9s2N41FBGzIqInInrGjh3b7nDMzLpW2YlglaTxAOl9dcn7NzOzPspOBDcAM9LnGcD1\nJe/fzMz6KPL20SuAO4DdJC2XdBJwLnCYpKXAYWnazMzaqLDG4og4vs5Xhxa1z3arYgPcUBXRANfN\nna75fDWnqGPrtobjYdtYbGZm5XAiMDOrOCcCM7OKc6dzQ9TN9atF6bb61aL5fDWn7P+T3XB9XCIw\nM6s4JwIzs4pzIjAzqzi3EQyC2wWaNxzqUTvpuvl8NWe4xDocrttguERgZlZxTgRmZhXnRGBmVnFu\nI8hpuNRB1jMc6yaHY0zDmc9Xc/x/snVcIjAzqzgnAjOzinMiMDOrOLcRNDDc6yCHo06qF+3Vzuvc\niecL2he3z1cxXCIwM6s4JwIzs4pzIjAzqzi3EfThdoHmeJzm5g33+uLhptvO13A8HpcIzMwqzonA\nzKzinAjMzCrObQRUp12gVcdZlfPVSsOxXng4q8r5Gi7H6RKBmVnFORGYmVWcE4GZWcU5EZiZVVxl\nG4vd4Nkcn6/mDZeGwE5R9fPVzuN3icDMrOKcCMzMKs6JwMys4irVRuB67ub4fDWv6vXczfL56l/Z\n56UtJQJJh0v6qaRlks5sRwxmZpYpPRFIGgF8A3gf8HbgeElvLzsOMzPLtKNEsC+wLCIei4jfAFcC\n09oQh5mZAYqIcncoHQMcHhEnp+kTgd+NiI/2WW4mMDNN7gb8dAi7HQM8NYT1O5GPuRp8zNUw2GN+\na0SMHWihdjQWq595b8hGETELmNWSHUqLIqKnFdvqFD7mavAxV0PRx9yOqqHlwMSa6QnAijbEYWZm\ntCcR3A1MkbSzpM2A44Ab2hCHmZnRhqqhiNgo6aPAD4ARwLcj4sGCd9uSKqYO42OuBh9zNRR6zKU3\nFpuZ2fDiLibMzCrOicDMrOK6PhFUoTsLSRMlzZf0kKQHJZ2a5o+WNFfS0vS+XbtjbSVJIyTdI+mm\nNL2zpIXpeK9KNyN0FUmjJM2R9HC63vtX4Dqfnv5dPyDpCklbdNu1lvRtSaslPVAzr9/rqsxX02/a\nfZL2Hur+uzoRVKg7i43AGRGxB7Af8JF0nGcC8yJiCjAvTXeTU4GHaqbPAy5Ix7sWOKktURXrX4Bb\nImJ3YCrZ8XftdZa0E3AK0BMR7yC7weQ4uu9afwc4vM+8etf1fcCU9JoJfGuoO+/qREBFurOIiJUR\n8eP0+TmyH4edyI51dlpsNjC9PRG2nqQJwB8BF6VpAYcAc9IiXXW8AJJGAgcCFwNExG8iYh1dfJ2T\nTYAtJW0CbAWspMuudUTcBjzTZ3a96zoN+I/I3AmMkjR+KPvv9kSwE/BkzfTyNK9rSZoM7AUsBHaI\niJWQJQtgXPsia7kLgU8Br6Tp7YF1EbExTXfjtd4FWANckqrELpL0Zrr4OkfEL4AvA0+QJYBngcV0\n/7WG+te15b9r3Z4IcnVn0S0kbQ1cA5wWEevbHU9RJB0JrI6IxbWz+1m02671JsDewLciYi/gV3RR\nNVB/Ur34NGBnYEfgzWRVI31127VupOX/1rs9EVSmOwtJm5Ilgcsi4to0e1VvkTG9r25XfC12AHCU\npMfJqvsOISshjErVB9Cd13o5sDwiFqbpOWSJoVuvM8B7gZ9FxJqI2ABcC7yb7r/WUP+6tvx3rdsT\nQSW6s0j14xcDD0XE+TVf3QDMSJ9nANeXHVsRIuLTETEhIiaTXdMfRsQJwHzgmLRY1xxvr4j4JfCk\npN3SrEOBn9Cl1zl5AthP0lbp33nvMXf1tU7qXdcbgA+lu4f2A57trUIatIjo6hdwBPAI8Cjw2XbH\nU9AxvoesaHgfcG96HUFWbz4PWJreR7c71gKO/SDgpvR5F+AuYBlwNbB5u+Mr4Hj3BBala30dsF23\nX2fgHOBh4AHgUmDzbrvWwBVkbSAbyP7iP6nedSWrGvpG+k27n+yOqiHt311MmJlVXLdXDZmZ2QCc\nCMzMKs6JwMys4pwIzMwqzonAzKzinAis0iS9RdKVkh6V9BNJ/ylp10Fs56LeDg0lfSbnOo9LGtPs\nvsxazbePWmWlB5T+D5gdEf+a5u0JbBMR/zOE7T4fEVvnWO5xsnvAnxrsvsxawSUCq7KDgQ29SQAg\nIu4F7pE0T9KPJd0vaRpkHfqlcQBmp37g50jaKn23QFKPpHPJesq8V9Jl6bvrJC1OferPbMNxmjXk\nRGBV9g6yniz7egl4f0TsTZYsvpJKDwC7AbMi4p3AeuBva1eMiDOBFyNiz8i6vQD4y4jYB+gBTpG0\nfQHHYjZoTgRmbyTgi5LuA/6brIvfHdJ3T0bE/6bP3yXr3mMgp0haAtxJ1lnYlBbHazYkmwy8iFnX\nepDXOi6rdQIwFtgnIjakuvwt0nd9G9UaNrJJOoisB839I+IFSQtqtmU2LLhEYFX2Q2BzSR/unSHp\nXcBbycY72CDp4DTda5Kk/dPn44Hb+9nuhtQtOMC2wNqUBHYnG0rUbFhxIrDKiuyWufcDh6XbRx8E\nzgb+E+iRtIisdPBwzWoPATNStdFo+h8vdhZwX2osvgXYJC3/ebLqIbNhxbePmuWUhgG9KbJB1M26\nhksEZmYV5xKBmVnFuURgZlZxTgRmZhXnRGBmVnFOBGZmFedEYGZWcf8PGuEWwOrW2QgAAAAASUVO\nRK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1d9e016fe48>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plotting Capital vs Final Policy\n",
    "\n",
    "# x axis values\n",
    "x = range(100)\n",
    "# corresponding y axis values\n",
    "y = policy\n",
    " \n",
    "# plotting the bars\n",
    "plt.bar(x, y, align='center', alpha=0.5)\n",
    " \n",
    "# naming the x axis\n",
    "plt.xlabel('Capital')\n",
    "# naming the y axis\n",
    "plt.ylabel('Final policy (stake)')\n",
    " \n",
    "# giving a title to the graph\n",
    "plt.title('Capital vs Final Policy')\n",
    " \n",
    "# function to show the plot\n",
    "plt.show()\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}