diff --git a/neural_net_optimizers.ipynb b/neural_net_optimizers.ipynb index a2c5300..d89f5be 100644 --- a/neural_net_optimizers.ipynb +++ b/neural_net_optimizers.ipynb @@ -1,5 +1,12 @@ { "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Part 2: Adding optimizers to neural net" + ] + }, { "cell_type": "code", "execution_count": 3, @@ -189,7 +196,7 @@ "cell_type": "code", "execution_count": 19, "metadata": { - "scrolled": true + "scrolled": false }, "outputs": [ { @@ -227,70 +234,11 @@ }, { "cell_type": "code", - "execution_count": 22, + "execution_count": null, "metadata": { - "scrolled": true + "scrolled": false }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Epoch 1. Training loss: 0.7294109890216827, Val loss:0.7571293303654454\n", - "Epoch 2. Training loss: 0.7247515532526804, Val loss:0.7526944168197087\n", - "Epoch 3. Training loss: 0.7201350017092408, Val loss:0.7482783550191122\n", - "Epoch 4. Training loss: 0.7155696414124957, Val loss:0.7438576050205469\n", - "Epoch 5. Training loss: 0.711049988328926, Val loss:0.7394775980364626\n", - "Epoch 6. Training loss: 0.7065726744573672, Val loss:0.7351348327425783\n", - "Epoch 7. Training loss: 0.7021364589743962, Val loss:0.7308175132213711\n", - "Epoch 8. Training loss: 0.6977332146464922, Val loss:0.7265004766416918\n", - "Epoch 9. Training loss: 0.6933646184493438, Val loss:0.7222185128265478\n", - "Epoch 10. Training loss: 0.6890260444381533, Val loss:0.7179350645249138\n", - "Epoch 11. Training loss: 0.6847393914854546, Val loss:0.7136704235845478\n", - "Epoch 12. Training loss: 0.6805139318191527, Val loss:0.7094559866952688\n", - "Epoch 13. Training loss: 0.6763579163374663, Val loss:0.7052955474375573\n", - "Epoch 14. Training loss: 0.6722682699385342, Val loss:0.7011868196998294\n", - "Epoch 15. Training loss: 0.6682293721400434, Val loss:0.6971484073256895\n", - "Epoch 16. Training loss: 0.6642274910930711, Val loss:0.6931804764540437\n", - "Epoch 17. Training loss: 0.6602622929652188, Val loss:0.6892692901051991\n", - "Epoch 18. Training loss: 0.65634224133864, Val loss:0.6854339032397738\n", - "Epoch 19. Training loss: 0.6524612635879831, Val loss:0.6816242638020328\n", - "Epoch 20. Training loss: 0.6486256930113067, Val loss:0.6778815887814797\n", - "Epoch 21. Training loss: 0.6448456739079084, Val loss:0.6742036302083959\n", - "Epoch 22. Training loss: 0.6411319683106386, Val loss:0.6705940886650256\n", - "Epoch 23. Training loss: 0.6375072133972782, Val loss:0.6670354558662032\n", - "Epoch 24. Training loss: 0.6339495923587553, Val loss:0.6635696574891677\n", - "Epoch 25. Training loss: 0.6304528478373312, Val loss:0.6602012622544849\n", - "Epoch 26. Training loss: 0.627030335917739, Val loss:0.6569069986655193\n", - "Epoch 27. Training loss: 0.6236760668369884, Val loss:0.6536681940680158\n", - "Epoch 28. Training loss: 0.620369639093562, Val loss:0.6504880317929598\n", - "Epoch 29. Training loss: 0.6171499591562633, Val loss:0.6473678376101121\n", - "Epoch 30. Training loss: 0.6140064570440803, Val loss:0.6443077860493469\n", - "Epoch 31. Training loss: 0.6109188524329593, Val loss:0.6413186028608349\n", - "Epoch 32. Training loss: 0.6078894544166995, Val loss:0.6383808846278852\n", - "Epoch 33. Training loss: 0.6049231153621669, Val loss:0.635524103288931\n", - "Epoch 34. Training loss: 0.6020271599782544, Val loss:0.6327374810959397\n", - "Epoch 35. Training loss: 0.5992072874731751, Val loss:0.630007390264417\n", - "Epoch 36. Training loss: 0.5964559453696211, Val loss:0.6273438569453254\n", - "Epoch 37. Training loss: 0.5937728085703975, Val loss:0.6247608775982707\n", - "Epoch 38. Training loss: 0.5911450252485898, Val loss:0.6222460034829198\n", - "Epoch 39. Training loss: 0.5885821143077742, Val loss:0.6197946463148126\n", - "Epoch 40. Training loss: 0.5860777305669043, Val loss:0.617412930072183\n" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xl8lNXZ//HPyWSZJDNZJpN9JawJIQkxIgoIonVDrVrc19SWx6VPtVofrY/V1tY+Vv25W8WFaNVK3cWV2kpBqqIBMWGVNRAgZN8XMsn5/XFPhmwkAZLMZLjer1dezHLPzJWF75y57nOfW2mtEUII4V183F2AEEKIoSfhLoQQXkjCXQghvJCEuxBCeCEJdyGE8EIS7kII4YUk3IUQwgtJuAshhBeScBdCCC/k664XttvtOiUlxV0vL4QQo9Lq1asrtNaRA203YLgrpczACiDAuf1bWut7e2wTAPwVOA6oBC7RWu/s73lTUlIoKCgY6OWFEEJ0oZQqHsx2g2nLtAJztdZZQDZwplJqeo9trgOqtdbjgEeBPx9OsUIIIYbWgOGuDQ3Oq37Or56rjf0YeNl5+S3gVKWUGrIqhRBCHJZB7VBVSpmUUmuBMuAzrfWqHpvEA7sBtNYOoBaI6ON5FiilCpRSBeXl5UdXuRBCiEMa1A5VrXU7kK2UCgPeVUplaK3Xddmkr1F6r7WEtdbPAc8B5ObmylrDQoygtrY2SkpKaGlpcXcpYhDMZjMJCQn4+fkd0eMPa7aM1rpGKfVv4Eyga7iXAIlAiVLKFwgFqo6oIiHEsCgpKcFqtZKSkoJ0TT2b1prKykpKSkoYM2bMET3HgG0ZpVSkc8SOUioQOA3Y1GOzJcA1zsvzgc+1nAVECI/S0tJCRESEBPsooJQiIiLiqD5lDWbkHgu8rJQyYbwZvKG1/lApdR9QoLVeArwIvKKU2ooxYr/0iCsSQgwbCfbR42h/VwOGu9a6EJjax+33dLncAlx0VJUM1hdfwG9/Cy+9BHIQlBBC9Gn0LT+waBEsXw4PPeTuSoQQh6GyspLs7Gyys7OJiYkhPj7edf3AgQODeo68vDw2b97c7zZPP/00r7322lCUzMyZM1m7du2QPNdIc9vyA0fs5ptl1C7EKBQREeEKyt/97ndYLBZ+/etfd9tGa43WGh+fvsed+fn5A77OTTfddPTFeoHRN3LPyoKwMNi2DYDtVdv59T9+TUVThZsLE0Icia1bt5KRkcH1119PTk4O+/btY8GCBeTm5jJ58mTuu+8+17adI2mHw0FYWBh33nknWVlZnHjiiZSVlQFw991389hjj7m2v/POO5k2bRoTJ07kyy+/BKCxsZGf/OQnZGVlcdlll5GbmzvgCP3VV19lypQpZGRkcNdddwHgcDi46qqrXLc/8cQTADz66KOkp6eTlZXFlVdeOeQ/s8EYfSN3pYyA//57tNac/urpbKveRnRwNLfPuN3d1QkxOtxyCwym3dDWBqWlEBMDA823zs4GZ6gerg0bNpCfn8+zzz4LwAMPPIDNZsPhcHDKKacwf/580tPTuz2mtraW2bNn88ADD3DrrbeyaNEi7rzzzl7PrbXmm2++YcmSJdx33318+umnPPnkk8TExPD222/z/fffk5OT0299JSUl3H333RQUFBAaGsppp53Ghx9+SGRkJBUVFRQVFQFQU1MDwIMPPkhxcTH+/v6u20ba6Bu5gxHuRUW8s+EttlVvw0f5kDc1z91VCeF9Skth+3bj32E0duxYjj/+eNf1119/nZycHHJycti4cSMbNmzo9ZjAwEDOOussAI477jh27tzZ53NfeOGFvbZZuXIll15qTOrLyspi8uTJ/da3atUq5s6di91ux8/Pj8svv5wVK1Ywbtw4Nm/ezM0338zSpUsJDQ0FYPLkyVx55ZW89tprR3wQ0tEafSN3gKwsmlsbue2TXwHQoTsICQhxc1FCjCKDHWFXVEB+PuTlgd0+bOUEBwe7Lm/ZsoXHH3+cb775hrCwMK688so+53v7+/u7LptMJhwOR5/PHRAQ0Gubwz0M51DbR0REUFhYyCeffMITTzzB22+/zXPPPcfSpUtZvnw577//Pn/84x9Zt24dJpPpsF7zaI3akftDM6C4cQ8XTLoAgPrWejcXJYQXstvh9tuHNdh7qqurw2q1EhISwr59+1i6dOmQv8bMmTN54403ACgqKurzk0FX06dPZ9myZVRWVuJwOFi8eDGzZ8+mvLwcrTUXXXQRv//971mzZg3t7e2UlJQwd+5cHnroIcrLy2lqahry72Ego3LkXpxg4f9mwsUd6Zwx4Rze3fQuDQcaiAjqtVaZEGKUycnJIT09nYyMDFJTU5kxY8aQv8Z///d/c/XVV5OZmUlOTg4ZGRmulkpfEhISuO+++5gzZw5aa84991zmzZvHmjVruO6669Bao5Tiz3/+Mw6Hg8svv5z6+no6Ojq44447sFqtQ/49DES5a5WA3NxcfaQn67j4zYv5sPAtNm2Yy9f3LeCSty5h3Q3rmBzVf99MiGPZxo0bSUtLc3cZHsHhcOBwODCbzWzZsoXTTz+dLVu24OvrWePdvn5nSqnVWuvcgR7rWd/JILy38T3e3PAm/1ObRlLBFtb5WwBoONAwwCOFEMLQ0NDAqaeeisPhQGvNwoULPS7Yj9ao+25W7FoBQGhUEuxairXVuL3+gPTchRCDExYWxurVq91dxrAadeF+16y7iLXEkleVBCzFsnMvICN3IYToatSFuz3IbhystG8fAJYtOwEJdyGE6Gp0ToUE44i5yEgsG4xlCCTchRDioNEb7s5lCKyFxnlDZJ67EEIcNHrDHSAri6DvjYMPZOQuhGebM2dOrwOSHnvsMW688cZ+H2exGDPi9u7dy/z58w/53ANNrX7ssce6HUx09tlnD8m6L7/73e94+OGHj/p5htqoD3ef1gMEmwIl3IXwcJdddhmLFy/udtvixYu57LLLBvX4uLg43nrrrSN+/Z7h/vHHHxMWFnbEz+fpRn24A1jwl3AXwsPNnz+fDz/8kNZWY/7yzp072bt3LzNnznTNO8/JyWHKlCm8//77vR6/c+dOMjIyAGhububSSy8lMzOTSy65hObmZtd2N9xwg2u54HvvvReAJ554gr1793LKKadwyimnAJCSkkJFhbFU+COPPEJGRgYZGRmu5YJ37txJWloaP//5z5k8eTKnn356t9fpy9q1a5k+fTqZmZlccMEFVFdXu14/PT2dzMxM14Jly5cvd52sZOrUqdTXD21redTNlulm0iTw88Pa5iPz3IU4DLd8egtrSwde8retvY3ShlJiLDH4mfpf3TA7JpvHzjz0gmQRERFMmzaNTz/9lB//+McsXryYSy65BKUUZrOZd999l5CQECoqKpg+fTrnnXfeIc8j+swzzxAUFERhYSGFhYXdluy9//77sdlstLe3c+qpp1JYWMgvf/lLHnnkEZYtW4a9xzo5q1evJj8/n1WrVqG15oQTTmD27NmEh4ezZcsWXn/9dZ5//nkuvvhi3n777X7XZ7/66qt58sknmT17Nvfccw+///3veeyxx3jggQfYsWMHAQEBrlbQww8/zNNPP82MGTNoaGjAbDb3+/M9XKN75O7vD2lpWJodMnIXYhiUNpSyvWY7pQ1Ds+Rv19ZM15aM1pq77rqLzMxMTjvtNPbs2cP+/fsP+TwrVqxwhWxmZiaZmZmu+9544w1ycnKYOnUq69evH3BRsJUrV3LBBRcQHByMxWLhwgsv5IsvvgBgzJgxZGdnA/0vKwzG+vI1NTXMnj0bgGuuuYYVK1a4arziiit49dVXXUfCzpgxg1tvvZUnnniCmpqaIT9CdnSP3AEmTsRSvZ6Gxmp3VyLEqNHfCLuriqYK8r/LJ29qHvago18Z8vzzz+fWW29lzZo1NDc3u0bcr732GuXl5axevRo/Pz9SUlL6XOa3q75G9Tt27ODhhx/m22+/JTw8nGuvvXbA5+lvfa3O5YLBWDJ4oLbMoXz00UesWLGCJUuW8Ic//IH169dz5513Mm/ePD7++GOmT5/OP//5TyZNmnREz9+X0T1yBygrw9LcTsO+YndXIoTX6TxocCiCHYyZL3PmzOGnP/1ptx2ptbW1REVF4efnx7Jlyygu7v//88knn+w6Cfa6desoLCwEjOWCg4ODCQ0NZf/+/XzyySeux1it1j772ieffDLvvfceTU1NNDY28u677zJr1qzD/t5CQ0MJDw93jfpfeeUVZs+eTUdHB7t37+aUU07hwQcfpKamhoaGBrZt28aUKVO44447yM3NZdOmTYf9mv0Z/SP3E07AumM5O0MCBt5WCOF2l112GRdeeGG3mTNXXHEF5557Lrm5uWRnZw84gr3hhhvIy8sjMzOT7Oxspk2bBhhnVZo6dSqTJ0/utVzwggULOOuss4iNjWXZsmWu23Nycrj22mtdz/Gzn/2MqVOn9tuCOZSXX36Z66+/nqamJlJTU8nPz6e9vZ0rr7yS2tpatNb86le/IiwsjN/+9rcsW7YMk8lEenq666xSQ2VULvnbzaOP8tPlt/LZrHh231Zy9M8nhJeSJX9Hn6NZ8nf0t2UsFiwH5CAmIYToynvCva3xsM+LKIQQ3mr0h7vVirUVHNpBa3uru6sRwqPJAGj0ONrf1egPd+fIHaQ1I0R/zGYzlZWVEvCjgNaaysrKozqwafTPlrFau4X7UE3ZEsLbJCQkUFJSQnl5ubtLEYNgNptJSEg44seP/nDvMnKXZX+FODQ/Pz/GjBnj7jLECPGKtoxV2jJCCNHN6A/3Hm0ZIYQQ3hDuwcES7kII0cOA4a6USlRKLVNKbVRKrVdK3dzHNnOUUrVKqbXOr3uGp9w+mExYlLH0gCz7K4QQhsHsUHUAt2mt1yilrMBqpdRnWuue62h+obU+Z+hLHJjV3wq0yshdCCGcBhy5a633aa3XOC/XAxuB+OEu7HBYAoxzLEq4CyGE4bB67kqpFGAqsKqPu09USn2vlPpEKTV5CGobtMDAEJSWcBdCiE6DnueulLIAbwO3aK3rety9BkjWWjcopc4G3gPG9/EcC4AFAElJSUdcdE8+FivB7SaZ5y6EEE6DGrkrpfwwgv01rfU7Pe/XWtdprRuclz8G/JRSvQ4V1Vo/p7XO1VrnRkZGHmXpXVgsWB0+MnIXQginwcyWUcCLwEat9SOH2CbGuR1KqWnO560cykL7ZbViaVM0tEm4CyEEDK4tMwO4CihSSnWeLv0uIAlAa/0sMB+4QSnlAJqBS/VIrk5ksWBp1TJyF0IIpwHDXWu9Euh9Jtru2zwFPDVURR02qxVLc4f03IUQwmn0H6EKRs+9uUNG7kII4eQ14W5p1TTIyF0IIQBvCXfn4mES7kIIYfCOcHeu6V4vbRkhhAC8KNytcpJsIYRw8Y5wd7ZlOuigxdHi7mqEEMLtvCPc5STZQgjRjXeEe5ezMcma7kII4S3hbrFgbTUuyshdCCG8KNylLSOEEAd5R7jLSbKFEKIb7wj3wEAsDmP5G1lfRgghvCXclcLqGwzIyF0IIcBbwh2w+Em4CyFEJ+8J9wArIOEuhBDgReEeGBSCj5Z57kIIAV4U7spixdJukpG7EELgReGOxYKlTU6SLYQQ4E3hbrViaZOeuxBCgDeFu/NsTNJzF0IIbwp3qxVri5xHVQghwJvC3WLB0txBQ6uEuxBCeFe4H4CG1jp3VyKEEG7nPeHuXDxM1pYRQghvCvfOkbv03IUQwovC3WrF2goNjiY5SbYQ4pjnPeHuHLlrNM2OZndXI4QQbuV14Q7SdxdCCO8JdzkbkxBCuHhPuFssWCXchRAC8LJwl5G7EEIYvCfcu7RlZH0ZIcSxznvC3d8fS4cvICN3IYQYMNyVUolKqWVKqY1KqfVKqZv72EYppZ5QSm1VShUqpXKGp9z+WeU8qkIIAQxu5O4AbtNapwHTgZuUUuk9tjkLGO/8WgA8M6RVDpLF3wJIuAshxIDhrrXep7Ve47xcD2wE4nts9mPgr9rwNRCmlIod8moH0HmSbJnnLoQ41h1Wz10plQJMBVb1uCse2N3legm93wCGndl5kmwZuQshjnWDDnellAV4G7hFa91zXV3Vx0N6LfCilFqglCpQShWUl5cfXqWDqdFixeqQk2QLIcSgwl0p5YcR7K9prd/pY5MSILHL9QRgb8+NtNbPaa1ztda5kZGRR1Jv/6xWLG1Kwl0IccwbzGwZBbwIbNRaP3KIzZYAVztnzUwHarXW+4awzsGxWDA7oGBvARVNFSP+8kII4SkGM3KfAVwFzFVKrXV+na2Uul4pdb1zm4+B7cBW4HngxuEpdwAWC210UFhWSP53+W4pQQghPIHvQBtorVfSd0+96zYauGmoijpiViuTSjUH4mLIm5rn7mqEEMJtvOcIVQCLhcRajUJhD7K7uxohhHAbrwv3yEYobyrv82xMO6t3cvorp7OvfuR3BwghxEjyrnC3WolqBEeHg5qWml533/353Xy2/TP+sOIPbihOCCFGjneFu8VCZJNxsayxrNfdaZFpAOTEumXpGyGEGDHeFe5WK5GNxsXypt4HSTW2GXc2HmgcyaqEEGLEeVe4WyxEdYZ7Y+9wL20o7favEEJ4K68L9/7aMp2hvq9BdqgKIbybd4X7AG2Z/Y37AQl3IYT3865wt1gIaIcQZe5/5H6IqZAtjhZaHa1H9NKPfPUIF795sSx7IITwCF4X7gBRWHqN3Dt0hyvwDzVyP+dv53DiiyceUUDf/8X9vLnhTVn2QAjhEbwy3CO1udcO1armKhwdDmyBNiqaKmhrb+v18FUlq/iu9LvDDuiGAw1UNVdhD7TLsgdCCI/gXeFuMkFgIJFtAb3aMp0tmakxU4GD/fdOLY4WGtoaMPuaDzug15etB6CDDln2QAjhEbwr3ME4SrXNr1dbZn+DEead4d6z77671jiRVIujhQBTwGG9ZFFZEQDVzdV06I4jKlsIIYaS94V7YCCRJdWUN5Z3C9rOkXt2TDbQu+9eXFvsury7bjeHo2i/Ee4aTW1L7RGVLYQQQ8n7wt1kImr7ftp1e7f1ZXqFe4+Re3FNl3Cv7R3uhfsLGf/EeFcLptt9ZYWuy5XNlUdXvxBCDAHvC/eXXyay1QRAeeUu1837G/cTYApgQsQEFKrXyH1X7cFt+xq5/+mLP7G1eiv3/vvebrdrrSnaX0SMJQYwdtwKIYS7eV+4z5xJ1G33AFB2z23gXPq3tKGUGEsMfiY/7EH2XksQFNcWEx0cjY/y6Rb0nZJCk/p8udKGUiqbK5mTMgeQcBdCeAbvC3cg8tTzACj/+nP4v/8DjJF7tCUagFhrbJ8997G2scRaYvscuXduv2rPqm5rxXfuTJ2TPAeAyiZpywgh3M87wz0oEoDyOcfD//4v/PznlNbscbVOYi2xffbck0OTSQxN7LPnvq1qGwAldSXsrNnpur1zZ+rslNmAjNyFEJ7BO8M92Aj3sp+cCTYbvPACpZVG2wV6j9zbO9opqSshOTSZpNCkPkfu26q3MT1hOgDLi5e7bi8sKyTGEsM42zhAwl0I4Rm8Mtz9Tf6EBoRSfqAG7r6bdgUVNLpG7jHBMZQ2lLqmSpY2lNLW0UZyWDKJIYnsqt3VrfVS31pPWWMZ5044l4jACFYUr3DdV7S/iMzoTHx9fAkNCJXZMkIIj+CV4Q4QFRxlHKV6002Uj4ulA32wLWONxdHhcI2yO+e4J4UmkRiSSIujpVtIb6/eDsA42zhmJc9yjdwdHQ42lG9gStQUACKCImTkLoTwCF4b7pHBkcZRqv7+lN5wFQDRxUZgx1pigYNz3TvnuHe2ZaD7XPdt1Ua/fWz4WGYnz2Z79XZK6krYWrWV1vZWV7jbAm0S7kIIj+C14R4VHOVaPGz/GTMAiHn9A8AYucPBGTCdUx+Tw4wdql1vg4M7U8faxnJy8skArCheQeF+4+ClKdEHw13aMkIIT+C14R4ZFOlaPKz0QDUAMcu+hW++6T1yry3GFmjD4m8hMcQI9647VbdVb8MWaCPMHEZWdBahAaEs37mcov1F+Cgf0iPTAYgIlLaMEMIzeG24RwVHUdFUQYfucK0AGe0XBvff7+q9d47ci2uLXe2YyOBI/E3+vdoyY8PHAmDyMTEzaSYrdq2gqKyICRETMPuaAWnLCCE8h9eGe2RQJO26nermakobSgn2C8Zy7QJYsoTg7OOxdvix75vPYc8edtXuIjk0GQAf5UNiSGK3kfv26u2MtY11XT85+WQ2VWxi5a6Vrn47GOFe3VxNe0f7yH2jQgjRB+8Nd+dc9/KmckobSo2jU0NDjTvb24mtaaf0q8/QqWMortzuCneAxNBEV8+9rb2N4ppi18gdYHayccBSZXNlt3CPCIwwVoZslZUhhRDu5bXhHhUcBUB5Yzn7G/cbrZgFC+DBB+E//yF26iz2TYyjxqeN+vYmkv0jXY/tOnLfVbuLdt3eLdxzYnMI9gsGDu5MBWPkDnIgkxDC/bw23DuXIChrLHMtGobdDrffDna7cZSqPYDiBRcDkPTCm+BwAEa476nbQ3tHu2saZGp4quu5/Ux+5MblApAQkuC6vTPcZX0ZIYS7eW24u0bunW0Z59IDnWItsexrKmPXjVcAkLyiEH7zG8A4mKldt7OvYV+3aZBddb55fL7jc9dtEUERgIzchRDu5+vuAoZL57lM99Ttoaq5yjVDplOMJYbGtkbXyTeSL8yDhx+GkhISbzNWldxdu5tt1dsIMAUQZ43r9vhnznmGafHTup1vVdoyQghP4bXh7mfyI9wczvpyI7x7hnvnXPev93yN2ddM5P97Fj76NyxeTGKCH1iMue7bqreRGp6Kj+r+IcceZOf2Gbd3u83VlpEDmYQQbjZgW0YptUgpVaaUWneI++copWqVUmudX/cMfZlHJjI40rXeeq+2jPMo1VUlq0gKTUL5+8OSJWCxkPTRSsA5cq/a1qslcyjh5nBARu5CCPcbTM/9JeDMAbb5Qmud7fy67+jLGhqRQZGunvmhRu77G/cfnAaZkQEffEDo5mKs7b4U1+405riHDy7cTT4mwsxhEu5CCLcbMNy11iuAUZlWUcFRaIylezvPwtSpc+QOdJvjzpw58Ic/kFjpYPV3H9PY1miEu9ZQVwcHDvT7mrK+jBDCEwzVbJkTlVLfK6U+UUpNHqLnPGqdM1qgd1sm3ByOv8kfMBYM6+bOO0n0t/NNi7HU79g7HoCgIOMgqIAAMJuNaZVpabBnT7eHyvoyQghPMBThvgZI1lpnAU8C7x1qQ6XUAqVUgVKqoLy8fAheun+d0yFDA0IJ9AvsWYurVdPr5Nc+PiSddBYOk3F1bGswHH+8ceWMM+DmmyE5GTZtgv/6r24PlfVlhBCe4Khny2it67pc/lgp9RellF1rXdHHts8BzwHk5ubqnvcPtc4lCHq2ZDrFWmK7rSvTVWLUeNgACkXKe8vAFAD5+ZCXZ4zaKypg1ixYvhzKyyHSeC1boI2tVVuH75sSQohBOOqRu1IqRimlnJenOZ/TI5rOnSP3njtTO3X23Xu1ZcC1rntCSAIB0fHdjm4FjH/feQeam+Hee12Pk7aMEMITDGYq5OvAV8BEpVSJUuo6pdT1SqnrnZvMB9Yppb4HngAu1V1PQOpGnT33Q4V7WEAYCkWAKaDXfZ2tmn6nQaalwY03wsKFsM6YKWoLtFHTUiMrQwoh3Gows2Uu01rHaq39tNYJWusXtdbPaq2fdd7/lNZ6stY6S2s9XWv95fCXPTiutkxw322Z8MBwNJpXC1/tdV/nSTsaDzRS0dSrw3TQvfcaO1pvuw20xhZoQ6Opaak5+m9ACCGOkNeuLQMH2zI/VP7QZ0DfNesuHjztwW5LCHRKDE3EbDLz7d5vyf8u/9AvEhFhBPw//gE/+xkR7cYMHGnNCCHcyavDPTo4mnnj57F029I+A7pzCYHOdWi6MvuaWXv9Wh449YE+w7+bG280dqguWoRtyWeAhLsQwr28OtyVUrx0/kuHHJ0PZKJ9InfMvKPP8O/Gzw/efx+sVmwfGatEyoFMQgh38upwh/5H50PqxBPh22+JcDjbMlu+H97XE0KIfnh9uI+oiROxvfkBAFUP/xH+53+M+fBCCDHCJNyHWNjkXBSKSprgoYeMA5+EEGKEee167u7iWhnS3gZTxxtHtAohxAiTkfswsAXaqLKZjfnv9mHu9QshRB8k3IeBLdBGpdW314qRQggxUiTch0FEUARVZg179xrrwAshxAiTcB8GtkAbVX4OaGw0TvAhhBAjTMJ9GNjMNipVi3FFWjNCCDeQcB8GEUER1HQ00q6QcBdCuIWE+zCwBdoAqDEj4S6EcAsJ92HQGe6VQUi4CyHcQsJ9GEQERgBQFR0i4S6EcAsJ92HQOXKvig+XcBdCuIWE+zBwtWVk5C6EcBMJ92EQEWS0Zd6Kq6GicrebqxFCHIsk3IdBaEAoPsqHJUG7yY8vg7Y2d5ckhDjGSLgPA5OPifG28aSZYsj7DigtdXdJQohjjIT7MJkaO5UmUwf2JqTvLoQYcRLuwyQzKpPiA2XUBiDhLoQYcRLuw2RK9BQA1kUh4S6EGHES7sMkMzoTgKI4k4S7EGLEyWn2hkliSCKhAaEUjumQcBdCjDgZuQ8TpRQZURkURSPhLoQYcRLuwygzOpOikBb0nhJ3lyKEOMZIuA+jKVFTqDW1sbt+j5xuTwgxoiTch1HnTtXCkGY53Z4QYkRJuA+jjKgMAIpkOqQQYoRJuA+jUHMoyeYYCmWnqhBihEm4D7MpEWkyY0YIMeIGDHel1CKlVJlSat0h7ldKqSeUUluVUoVKqZyhL3P0ykzMZZMdWkuK3V2KEOIYMpiR+0vAmf3cfxYw3vm1AHjm6MvyHlPic2j3gU1l691dihDiGDJguGutVwBV/WzyY+Cv2vA1EKaUih2qAkc714yZ+q1urkQIcSwZip57PND1dEMlztsEMN42Hv8ORZFjr7tLEUIcQ4Yi3FUft/V5xI5SaoFSqkApVVBeXj4EL+35/Ex+pDvCKfSvdncpQohjyFCEewmQ2OV6AtDnMFVr/ZzWOldrnRsZGTkELz06TPGNpyjsgJxuTwixgXEKAAAUlElEQVQxYoYi3JcAVztnzUwHarXW+4bgeb1GZuh49oZA5c6N7i5FCHGMGMxUyNeBr4CJSqkSpdR1SqnrlVLXOzf5GNgObAWeB24ctmpHqZwYY3Zo1qJpPPnx72hua3ZzRUIIb6e0mxa0ys3N1QUFBW557ZGmGxqY/8to3kluAiBMBzAlJot7z/gTc1PmolRfuy2EEKI3pdRqrXXugNtJuI+Mit2bWfTkT5m438Fvg78xjloFJvpEcd7kC+mwBHPHjDuIDD529kUIIQ6fhLsHq7jrFp768nFC/C0siW9geTKgwNZh5srkczln1k+ZNWYOZl+zu0sVQngYCXdPVlEB+fmQlwdVVRTdkce9/l/S4A9fJEOLH5g6YLpPMqenzWPO8RdxfMIJBPoFurtyIYSbSbiPJp1hf/75NK3+ml989ivykyqJq4N9VtAKfDogQ0VzXOxxNIYFceMJv+CkxJPwM/m5u3ohxAiScB/FKnZvJv+VW8k75VZ8Nmzk1i/v5eWkKsZWwu5QOOA8rbm/9mGSfxz+wSHMy/gJ05Kmkx6ZTlJoEj5KFvwUwhtJuHsRV9jPu5v2xx/loT1vMl7Z2W6q492xB9gS0X173w7FhOAkJicdR0J4Mnvq9nB11tXkxuUSFRwls3OEGMUk3L1V1369zUbFHb8g/8tnOD84h/3mdh4NLuKdSR1MKgeHD2wLB91lEO/v48c423gm2CcQGxzL3oa9XDz5YrJjskkJSyHIL8h935sQYkAS7seKrmFvt1NRvJH8528kL+UC7Lsq2Pvh6zwesZWp+2BxBryfBmkVChVg5ofgFhw+3X//wX7BTIiYwPiI8UQFRVFSX8JF6ReRFZ1FclgyFn+Lm75RIQRIuItOneE/fz4Vj/yR/DWLyPOfht3HQlnhVzw5pZmTdkF1EPx1qg9LUzsY32oBcwDbdTXtdHR7utCAUMaEjyE6OJr61np+lPoj0qOMPn9iSCIxlhhMPiY3fbNCeD8Jd9Fbj1E+FRXw1FOQkwN791KxeBH5bd+St9WCvayBsiB48gQ4qQRePTGYv6U2cmJbNDZrNAXtu9nf1nulS5MykRCSQKwllpb2FmYlzSLeGs/Gio1clXUVmVGZ2IPs0vcX4ghJuIvD1zX8/fzg66+N6zExVHyxlHzzJvIKfbA3dFARBPnZ8JOdQTSNT+aRzEbyI3Yx13c8cfZU/tO4kR31u/BRPnTo7qN/fx9/LAEWJkZMJCUsBVugjd11uzl/4vmkR6aTEJJAtCUaXx9fN/0ghPBcEu5iaHUG/9VXQ1MT3H8/vPginHQSWK1UFH5N/pha8taCvQkj/I8zcU19KuWpMSwM38ZxqTOoi4vg7abVLC//ljFhY1BKUVxTTLtu7/ZyCkVUcBSxlljaOto4If4ExtnGYfW3UlhWyDVZ15ARlUFIQIh8ChDHFAl3Mbz6avE8/zyccgpUVcHChbBkCaSlQWUllJUdfKhz1J+3JxJ7wgTK48N4Omg9s9PPojE5jkX1K3i35DNy43KpballS9UWzL5mWhwtvcoI8g0iyC+ICfYJpISlEBYQxu663Zw38Twm2ScRZ40j1hIrR/cKryHhLtyra/gDLFoEZ58NtbXw5JPw979Dbi5YLLB2LdTUHHxol/AnLo78+DLyYs4iaGIG6yI6yD/wNdljTqLe5OD9zUtYuXslKWEp+CifPj8FAISZw4gKjqJDd5ATm0NqWCrWACsbKzZyWcZlpNnTiLXGyno+wuNJuAvP1deo/4UX4PTTob4e/vIXeOMNI/xramDrVjCZoL1HaAcHUzEmmvzUOvJCZ2MfO4Vyq4m/lLzD3FN/zl9bVvHCupc5c+yZjLWNZXnxctaVrSPMHEbjgUbaOnqfGSvQN5Ck0CRiLDE0HGhgRuIMxtrGuj4BxFnj5E1AuJWEuxi9eo768/Phmmugo8Po9T/1FJxzDowbB599BuvXG58AGhq6P00Q5Of6kleTgj1uHBVRFvJ9vidvytXY0o/jh9A2niv7lBNST6bR0cTf1/2df2z/B5MjJ1PXWsfuut197hAGY0pogG+Aa6dwaEAou2p3ce7Ec5lkn0SsJZZYa6wcFCaGnIS78E59jfo7r1utUFRk7OjNyjJ6/p98AunpEBgImzf3egMAICgIEhKoiA8n315CXtzZEB1N/r6PueaMO1CT0vjTDy/y2DePuw7o+mjLR3xV8hVJoUkAlNSV9PkmYPG3EB0cjUaTHW0cBRwSEMKmik1cPPliJtknEWOJIcwcJjuGxaBIuAtxqJ2+Z5wBzc1G++dvf4NZsyAmBlatgl27wMfH+JTQ9amCFfkn+JPXOAF77Fgq7EHkt68mL/ta7Om5lNvMPF32IXMnnkVLRysvrnmRNza8wYzEGdS11lFUVoQt0EZzWzPNjt6nWfT18SXWEktCSALh5nCqW6qZnTyb6OBoCssKuSLzCtLsaUQFR8kU0WOchLsQAznUp4BrroHycnjuOTj+eGO0/9Zb8K9/wfjx4O8P27ZBS+/ZOwQGQnw8FXFh5EftIS/qDIiKIr/sH+TNvZWIKSewI6iVZze8woykGTS1NfG3or/x4ZYPmRozFVugjaKyIsoay3o/t1OwXzApYSnGG0FgOPvr93Pa2NMYZxtHjCXG9WX1t8qnAS8k4S7EUOrrjWDhQvjRj4xPAQsXwuuvG58C4uONTwE7doCvLzgcvZ8vIAASEiAxkYrIYPL9N5A3/iLs47OMTwVN/+Hy3J9S5tPMi2sXMS1+Gk1tTby94W3+ueOfpNnTsPhb+KHyB2pba/ss2WwyE+gXSGp4KgkhCYSaQ9lbt5cfjf0RY8PHEmOJIdoSTVRwlLwRjCIS7kKMpEN9Crj2Wqirg2eeMQ74amw03gQ++QQyMyEkBDZsMI4N6IvZbOwTGDcOkpKosJmNJSKyrsE+cSoVNjOLqv7F+TmX00YHj696nOfXPM+88fOob61nxa4VjLONw+xrZkf1DhrbGvt8mUDfQCICI1BKkRaZRmRQJHvr9zJ3zFxSw1OJCo5yfdmD7NIaciMJdyE8VV9vBIsWwfz50NoKjz1mtITmzTPeGL74AlJTjSUhdu40tunJxweio412UEIFedZZYLeT37CSvOOuw552HBVh/rxY9inzMi/CoR089c1TvPjdi5w97mzSI9P5fMfnrCldQ5w1jvrWeuoP1B/yWwg3h+Pr48s42zjiQ+Kx+FkoqSvh1NRTiQiMoGBvAVdMuYIJ9glEBEbIGcOGkIS7EKNVX1NBe7aDzjgDDhyAZ5+FV16BuXNhzBj4z39g0ybjE0FDQ68dw4DxJhEdTUVsqHGAmGUW9oQJxtTR0k/Jm3MLjB3LoorPuGDq5bQrzeOrHufZgmc5f9L5ZEVnsXTrUr7e8zVjwsZg9jVTXFNMk6PpkN+S2ddMnCWOaEs0IQEhVDVXcUL8CSSGJhJgCuC70u+4KP0ixtrGYg+yE24Ol9VFD0HCXYhjQX9TQzs6jBlBp51mvBG88ILREpo9G1JS4Kuv4IcfjDeCpqa+9w34+EBUlPFGkFhJXuCJ2OPHG7OHyj8j76QbsU/KoSLElxf3fsR5GT+hsqmSVwpfYVr8NFrbW3lv03t8tv0zsqKzsAfZWV++ntKGUkzK1OfRxJ3CzGGEBYTRrtuZEDGBGEsMQb5B7Kzdydwxc0kKTcJX+bJy90quyryKcbZxhJpDvf4UkxLuQoju+nsjsNmMI4Gfe87YKfzmm/Daa8YngtRU+PJLY99AeLjRFmo6xCg9KMh4bq2NdYXi46mwmshvXEle1rXYx2VSEeJLfvXnXDvtvwgMjeCHqi28vPZlZiTO4L3N7/H6utc5LfU0JkVMYuWulazdv5Z4azz+Jn/21u+ltb2PtlQXoQGhRARFYPW30tTWRHpkOjGWGAJMAWyt2sqpqaeSFJpEuDmc8MBwwsxhhJvDCQkIGRWfFiTchRBHrr83Arvd6P0vXAhz5hifCv76V2O66KxZRjvou+8gLg6Ugn37+m4PgbHDODzc2G7cOCqireT7rSMv4VzsSZOoMLWS/8PfyTvjTuzpuVQEal7Y9DfOn3g+KNhWtY23N77NcbHHsXTbUj744QNOSjiJlPAUVu9dzebKzcb5A1BUNlf2eaBZVxZ/C7ZAGxZ/C81tzYyPGE90cDQBpgB21Ozg5KSTiQuJw0f58NXur5ifPp/ksGRCAkIIDQglyC9o2GcdSbgLIUZOf/sJysuNfQNnnmmsD7RwIbz0krGQ3OTJsGwZFBRAUpIxm6iysv/X8vU1njcqylh2oq4OcnKMmUT1K8jLuAr7mMlUWHzIr/03edl52OPGUd5ez8LVCzl3wrkopXjm22d4dvWzXJx+Mc2OZj744QPXWkIFewvYUL6BWEss/iZ/9jfsp6W9j+MaevDBB2uAFR/lQ6w1FlugDbOvmfLGcrKis4gMjsSkTNS01HD/qfdjD7If9o9awl0I4Zn6+1QAxuUrrzT6/Vu2GO2h6dONTwjvvmtMI83NNY4n+P5741NEcLBxvMGhPiGAcWxBQADExkJkpNEeCt5CXsSpEBJCfvXn5GVegz0l3dinULPMaCXFpFKhG1m0Np+L0i/C5GNiZ/VOFq9fzMykmQC8teEt3t30LnNT5tLiaOHLki9Jj0wnOjiaHyp/YE/9Hqz+Vtp1O01tRkvrwdMe5PYZtx/2j0/CXQjhffp7Y9Da+IQwb56x7bPPGstNXHih8Wbw6aewYoXxaSEy0phVVFpqHHF84ED/r+vnZ2wXHQ0REca+haoqYw2j6GgqfA+QX/kv8jKvBquV/KK/knf6HcY+Bn8H+TvfJe+4n2EPslPaUEr+d/n8/Lify8hdCCEO20D7Dbpeb2sz3gzOPtvo/z//vDG76MILYdo0443h3/82wjw21ti5vGuXMdPI4Tj0DuauAgIgNNRoJaWnG69tl3AXQoiRczhvDHa7sdN44cKDB5698QbMmGE81zvvwPvvG1NQJ00ylqZYuxYefBBul7aMEEKMTgO9MRwmCXchhPBCgw137z6USwghjlGDCnel1JlKqc1Kqa1KqTv7uP9apVS5Umqt8+tnQ1+qEEKIwRpw3U6llAl4GvgRUAJ8q5RaorXe0GPTv2utfzEMNQohhDhMgxm5TwO2aq23a60PAIuBHw9vWUIIIY7GYMI9Htjd5XqJ87aefqKUKlRKvaWUSuzriZRSC5RSBUqpgvLy8iMoVwghxGAMJtz7WgWn5xSbD4AUrXUm8E/g5b6eSGv9nNY6V2udGxkZeXiVCiGEGLTBhHsJ0HUkngDs7bqB1rpSa925DufzwHFDU54QQogjMZgTIX4LjFdKjQH2AJcCl3fdQCkVq7Xe57x6HrBxoCddvXp1hVKq+DDr7WQHKo7wscNJ6ho8T6wJPLMuT6wJPLMuT6wJhrau5MFsNGC4a60dSqlfAEsBE7BIa71eKXUfUKC1XgL8Uil1HuAAqoBrB/G8R9yXUUoVDGYS/0iTugbPE2sCz6zLE2sCz6zLE2sC99Q1qFOYa60/Bj7ucds9XS7/BvjN0JYmhBDiSMkRqkII4YVGa7g/5+4CDkHqGjxPrAk8sy5PrAk8sy5PrAncUJfbFg4TQggxfEbryF0IIUQ/Rl24D7SI2QjWsUgpVaaUWtflNptS6jOl1Bbnv+EjXFOiUmqZUmqjUmq9UupmD6nLrJT6Rin1vbOu3ztvH6OUWuWs6+9KKf+RrMtZg0kp9Z1S6kMPqmmnUqrIuQhfgfM2d/8Ow5xHn29y/n2d6AE1TeyyWOFapVSdUuoWD6jrV86/83VKqdedf/8j/nc1qsK9yyJmZwHpwGVKqXQ3lfMScGaP2+4E/qW1Hg/8y3l9JDmA27TWacB04Cbnz8fddbUCc7XWWUA2cKZSajrwZ+BRZ13VwHUjXBfAzXQ/LsMTagI4RWud3WX6nLt/h48Dn2qtJwFZGD8zt9aktd7s/BllYxw42QS86866lFLxwC+BXK11Bsb08Utxx9+V1nrUfAEnAku7XP8N8Bs31pMCrOtyfTMQ67wcC2x288/rfYzVPD2mLiAIWAOcgHFQh29fv9sRqiUB4z//XOBDjKU23FqT83V3AvYet7ntdwiEADtw7qPzhJr6qPF04D/urouDa3HZMKaafwic4Y6/q1E1cmfwi5i5S7R2Hqnr/DfKXYUopVKAqcAqT6jL2f5YC5QBnwHbgBqttcO5iTt+l48B/wN0OK9HeEBNYKzd9A+l1Gql1ALnbe78HaYC5UC+s4X1glIq2M019XQp8Lrzstvq0lrvAR4GdgH7gFpgNW74uxpt4T6YRcyOeUopC/A2cIvWus7d9QBordu18fE5AWMZ6bS+NhupepRS5wBlWuvVXW/uY1N3/H3N0FrnYLQfb1JKneyGGrryBXKAZ7TWU4FGRr4tdEjO/vV5wJseUEs4xpLoY4A4IBjj99jTsP9djbZwH3ARMzfbr5SKBWO9HYxR6ohSSvlhBPtrWut3PKWuTlrrGuDfGPsEwpRSnUdJj/TvcgZwnlJqJ8Y5CuZijOTdWRMAWuu9zn/LMHrI03Dv77AEKNFar3Jefwsj7D3l7+osYI3Wer/zujvrOg3YobUu11q3Ae8AJ+GGv6vRFu6uRcyc79aXAkvcXFNXS4BrnJevweh5jxillAJeBDZqrR/xoLoilVJhzsuBGP8BNgLLgPnuqEtr/RutdYLWOgXj7+hzrfUV7qwJQCkVrJSydl7G6CWvw42/Q611KbBbKTXRedOpwAZ31tTDZRxsyYB769oFTFdKBTn/P3b+rEb+78pdO0COYofF2cAPGD3b/3VjHa9j9NTaMEY212H0bP8FbHH+axvhmmZifNwrBNY6v872gLoyge+cda0D7nHengp8A2zF+Egd4Kbf5RzgQ0+oyfn63zu/1nf+jXvA7zAbKHD+Dt8Dwt1dk7OuIKASCO1ym7t/Vr8HNjn/1l8BAtzxdyVHqAohhBcabW0ZIYQQgyDhLoQQXkjCXQghvJCEuxBCeCEJdyGE8EIS7kII4YUk3IUQwgtJuAshhBf6/1e+ni7dMrh3AAAAAElFTkSuQmCC\n", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "nn.fit_epoch(X_train,y_train,X_test,y_test,lr=0.002,epochs=40,bs=512)" ] @@ -412,126 +360,27 @@ }, { "cell_type": "code", - "execution_count": 62, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Epoch 1. Training loss: 1.9782431452162077, Val loss:1.754339275195669\n", - "Epoch 2. Training loss: 1.654076432227679, Val loss:1.5811210360670265\n" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD8CAYAAABw1c+bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xl4VdX18PHvIoR5CJAAMoZ5SAghpICCzEPQOvOqKFoQi1OpVkXRqiCoRVQKCJVSC1ZFqNWiVCVMoigCEpB5TMIUxoQ5zEn2+8e+uT/AkFzITc4d1ud58pjce3LPOoDrnLPP2nuJMQallFLBo4TTASillCpemviVUirIaOJXSqkgo4lfKaWCjCZ+pZQKMpr4lVIqyGjiV0qpIKOJXymlgowmfqWUCjIlnQ4gL+Hh4SYyMtLpMJRSym+sWrUqwxgT4cm2Ppn4IyMjSUpKcjoMpZTyGyKyy9NtdahHKaWCjCZ+pZQKMpr4lVIqyPjkGL9SqnhduHCBtLQ0zp4963QoqgBlypShTp06hIaGXvNnaOJXSpGWlkbFihWJjIxERJwOR12BMYbDhw+TlpZGgwYNrvlzdKhHKcXZs2epVq2aJn0fJyJUq1at0HdmmviVUgCa9P2EN/6eAivxDx8Ojz8OGRlOR6KUUj4rcBL/sWPw3nv2q3NnWLfO6YiUUh44fPgwsbGxxMbGUrNmTWrXru3++fz58x59xqBBg9i6dWu+20yePJkZM2Z4I2Q6derEmjVrvPJZTgich7thYbB+PTz6KCxdCrGxcN99MGoUNGzodHRKqSuoVq2aO4mOHDmSChUq8Oyzz16yjTEGYwwlSuR9rTp9+vQC9/PEE08UPtgAEThX/AD16sE338DOnfD88/Df/0KzZvDEE3DggNPRKaWuQnJyMtHR0Tz66KPExcWxf/9+hgwZQnx8PFFRUYwaNcq9be4VeFZWFmFhYQwfPpzWrVtz/fXXc+jQIQBeeuklxo8f795++PDhtGvXjmbNmvHTTz8BcOrUKe666y5at25N//79iY+PL/DK/uOPP6ZVq1ZER0fz4osvApCVlcUDDzzgfn3ixIkA/PWvf6Vly5a0bt2aAQMGeP3PzFOBc8V/sSpV4C9/gaFDYfRomDoVPvgAnnoKhg2zdwdKqbw99RR4Moxx4YK9oKpZEwqqKY+NBVfSvRqbNm1i+vTpTJkyBYAxY8ZQtWpVsrKy6NatG/369aNly5aX/M7x48fp0qULY8aM4emnn2batGkMHz78V59tjOHnn39mzpw5jBo1isTERN59911q1qzJ559/ztq1a4mLi8s3vrS0NF566SWSkpKoXLkyPXv25KuvviIiIoKMjAzWr18PwLFjxwAYO3Ysu3btolSpUu7XnBBYV/yXq1XLjvlv3gy33QZvvGGHfcaOhdOnnY5OKf924ACkphbp3XSjRo34zW9+4/555syZxMXFERcXx+bNm9m0adOvfqds2bL07dsXgLZt27Jz5848P/vOO+/81TY//vgj9957LwCtW7cmKioq3/hWrFhB9+7dCQ8PJzQ0lPvuu48lS5bQuHFjtm7dypNPPsm8efOoXLkyAFFRUQwYMIAZM2YUagJWYQXmFf/lGjeGTz6B556DP//ZDgNNmAAjRsCgQQVfrSgVTDy9Ms/IgOnT7f9D4eFFEkr58uXd32/fvp0JEybw888/ExYWxoABA/KsZy9VqpT7+5CQELKysvL87NKlS/9qG2PMVcV3pe2rVavGunXrmDt3LhMnTuTzzz9n6tSpzJs3j++//54vv/yS1157jQ0bNhASEnJV+/SGwL7iv1xsLHz9NXz/PURGwiOPQFQU/PvfkJPjdHRK+ZfwcDt0WkRJ/3InTpygYsWKVKpUif379zNv3jyv76NTp058+umnAKxfvz7PO4qLdejQgcWLF3P48GGysrKYNWsWXbp0IT09HWMM/+///T9effVVVq9eTXZ2NmlpaXTv3p233nqL9PR0Tjs08hAcV/yX69wZfvwRvvoKXnwR7r0X3nzTPhfo3Rt0IotSPicuLo6WLVsSHR1Nw4YN6dixo9f3MXToUB588EFiYmKIi4sjOjraPUyTlzp16jBq1Ci6du2KMYZbbrmFm2++mdWrVzN48GCMMYgIb775JllZWdx3332cPHmSnJwcnn/+eSpWrOj1Y/CEXO2tTXGIj483xdaIJTsbZs6El1+21UBdu9oTQIcOxbN/pXzA5s2badGihdNhOC4rK4usrCzKlCnD9u3b6d27N9u3b6dkSd+6Rs7r70tEVhlj4j35/eAa6slLSAgMGABbt8KkSbBpE1x/Pdx+O2zc6HR0SqlilJmZSceOHWndujV33XUXf//7330u6XtD4B3RtSpVytb7/+539sHv2LHQqhU88AC8+qp9JqCUCmhhYWGsWrXK6TCKXIFX/CIyTUQOiciGK7xfRURmi8g6EflZRKIvem+niKwXkTUi4h9NdCtUsJU/qanwzDPw6afQtCk8+SS4JoIopZQ/82So5wMgIZ/3XwTWGGNigAeBCZe9380YE+vp2JPPqFYN3noLtm+HgQNh8mQ7B+CVV+D4caejU0qpa1Zg4jfGLAGO5LNJS2CRa9stQKSI1PBOeD6gTh0783fTJrj5ZjsTuFEjeOcd0G5FSik/5I2Hu2uBOwFEpB1QH6jjes8A80VklYgMye9DRGSIiCSJSFJ6eroXwvKypk1tvX9SErRtC88+C02awD//CVeYIKKUUr7IG4l/DFBFRNYAQ4FfgNxM2NEYEwf0BZ4Qkc5X+hBjzFRjTLwxJj4iIsILYRWRtm1h3jz49luoXRsefhiio+Hzz8EHS2OV8gddu3b91YSs8ePH8/jjj+f7exUqVABg37599OvX74qfXVB5+Pjx4y+ZTHXTTTd5ZS2dkSNH8vbbbxf6c7yt0InfGHPCGDPIGBOLHeOPAHa43tvn+u8hYDbQrrD78xndusGyZTB7ti0J7dcP2rWDhQudjkwpv9O/f39mzZp1yWuzZs2if//+Hv1+rVq1+Oyzz655/5cn/m+++YawAF7MsdCJX0TCRCR3cYyHgSXGmBMiUl5EKrq2KQ/0BvKsDPJbIrbef906u/rnoUPQqxf07AkrVzodnVJ+o1+/fnz11VecO3cOgJ07d7Jv3z46depEZmYmPXr0IC4ujlatWvHll1/+6vd37txJdLQtKDxz5gz33nsvMTEx3HPPPZw5c8a93WOPPeZe1nnEiBEATJw4kX379tGtWze6desGQGRkJBmuTn7jxo0jOjqa6Oho97LOO3fupEWLFvz+978nKiqK3r17X7KfvKxZs4YOHToQExPDHXfcwdGjR937b9myJTExMe4F4r7//nt3M5o2bdpw8uTJa/6zzUuBdfwiMhPoCoSLSBowAggFMMZMAVoAH4pINrAJGOz61RrAbFd/yJLAJ8aYRK9G7ytCQmz9/733wpQp8Prr9ur/zjvhtddAZ0QqP/JU4lOsOVDwsswXsi9wIPMANSvUJDQk/4UOY2vGMj7hyou/VatWjXbt2pGYmMhtt93GrFmzuOeeexARypQpw+zZs6lUqRIZGRl06NCBW2+99Yq9Z9977z3KlSvHunXrWLdu3SVLK7/++utUrVqV7OxsevTowbp16/jjH//IuHHjWLx4MeGXrTu0atUqpk+fzooVKzDG0L59e7p06UKVKlXYvn07M2fO5B//+Ad33303n3/+eb5r7D/44IO8++67dOnShVdeeYVXX32V8ePHM2bMGHbs2EHp0qXdw0tvv/02kydPpmPHjmRmZlKmTJl8/3yvlidVPf2NMdcZY0KNMXWMMf80xkxxJX2MMcuMMU2MMc2NMXcaY466Xk81xrR2fUUZY173auS+qHRpW++fkmInfS1YYMf/H3oIdu92OjqlvOpA5gFSj6VyINM7yzJfPNxz8TCPMYYXX3yRmJgYevbsyd69ezl48OAVP2fJkiXuBBwTE0NMTIz7vU8//ZS4uDjatGnDxo0bC1yE7ccff+SOO+6gfPnyVKhQgTvvvJMffvgBgAYNGhAbGwvkv/wz2B4Bx44do0uXLgD87ne/Y8mSJe4Y77//fj7++GP3LOGOHTvy9NNPM3HiRI4dO+b12cM6c7coVKxo6/0fe8yu+zN5MsyYYWcGv/AC+PLDaxX08rsyv1jG6Qym/zKdQW0GEV6u8Ct03n777Tz99NOsXr2aM2fOuK/UZ8yYQXp6OqtWrSI0NJTIyMg8l2O+WF53Azt27ODtt99m5cqVVKlShYEDBxb4OfmtZZa7rDPYpZ0LGuq5kq+//polS5YwZ84cRo8ezcaNGxk+fDg333wz33zzDR06dGDhwoU0b978mj4/L7pWT1GKiIBx4+wksAED7FIQjRrZuwEvj9kpVdzCy4UzrOMwryR9sBU6Xbt25aGHHrrkoe7x48epXr06oaGhLF68mF27duX7OZ07d3Y3Vd+wYQPr1q0D7LLO5cuXp3Llyhw8eJC5c+e6f6dixYp5jqN37tyZL774gtOnT3Pq1Clmz57NjTfeeNXHVrlyZapUqeK+W/joo4/o0qULOTk57Nmzh27dujF27FiOHTtGZmYmKSkptGrViueff574+Hi2bNly1fvMjyb+4lCvnq3337DBLvs8cqSdBTxhArgeZiml7HDP2rVr3Q85Ae6//36SkpKIj49nxowZBV75PvbYY2RmZhITE8PYsWNp184WE7Zu3Zo2bdoQFRXFQw89dMmyzkOGDKFv377uh7u54uLiGDhwIO3ataN9+/Y8/PDDtGnT5pqO7V//+hfDhg0jJiaGNWvW8Morr5Cdnc2AAQNo1aoVbdq04U9/+hNhYWGMHz+e6OhoWrdufUlHMW/RZZmd8PPPtg/AokX2pPDqq3YxOAc68SgFuiyzv9Flmf1Rbr3/ggVQvbptXRcTA198oZPAlFJFThO/k3r2tFf/n31mG8LccYftBbB4sdORKaUCmCZ+p4nAXXfZ8f9//hP27oXu3aFPHwiCdcGV7/DFYV/1a974e9LE7ytKlrT1/tu325U/V62C+Hi4+27bHUypIlSmTBkOHz6syd/HGWM4fPhwoSd06cNdX3X8uD0BjBtnl39+6CE7N6BOnYJ/V6mrdOHCBdLS0gqsa1fOK1OmDHXq1CE09NLZ0lfzcFcTv687dMguAfHee1CiBAwdCsOH20YxSinlolU9gaR6dVvvv22bXQvonXfsHIDXXoPMTKejU0r5IU38/iIy0q4Aum6dXRL65ZftLOBJk+D8eaejU0r5EU38/iY62tb7L1tmV/0cOhSaNYOPPrIloUopVQBN/P6qQwdb75+YCFWqwIMPQmws/O9/OglMKZUvTfz+TMTW+ycl2X7A587BrbdCp07gWvJVKaUup4k/EJQoYev9N26EqVNh507o0gVuugnWFNxQQykVXApM/CIyTUQOiUiebRNFpIqIzBaRdSLys4hEX/RegohsFZFkERnuzcBVHkJD4fe/h+RkGDsWli+HNm2gf3/7mlJK4dkV/wdAQj7vvwisMcbEYJutTwAQkRBgMtAXaAn0F5GWhYpWeaZsWRg2DFJT7Sqgc+bYB8GPPQb79jkdnVLKYZ60XlwCHMlnk5bAIte2W4BIEakBtAOSXS0YzwOzgNsKH7LyWFiYnfyVkgKPPALvvw+NG9sJYK5Gz0qp4OONMf61wJ0AItIOqA/UAWoDey7aLs31Wp5EZIiIJIlIUnp6uhfCUm41a9p6/61b7YJwY8faSWBjxsDp005Hp5QqZt5I/GOAKiKyBhgK/AJkAb9ueglXrDM0xkw1xsQbY+IjtCdt0WjY0Nb7r1ljK39eeMFOAnvvPbhwwenolFLFpNCJ3xhzwhgzyBgTix3jjwB2YK/w6160aR1AB5h9QUyMrff/8Uc79PP44/YZwCefQE6O09EppYpYoRO/iISJSCnXjw8DS4wxJ4CVQBMRaeB6/15gTmH3p7yoY0db7//111C+PNx/P8TFwTff6CQwpQKYJ+WcM4FlQDMRSRORwSLyqIg86tqkBbBRRLZgK3ieBDDGZAF/AOYBm4FPjTEbi+IgVCGI2Hr/X36xV/wnT8LNN9t5AEuXOh2dUqoI6LLM6lLnz9tOYKNGwYED8Nvf2sqgmBinI1NK5UOXZVbXrlQpW++fnAxvvAE//GDXAHrgATsvQCnl9zTxq7yVL2+rflJT4bnnbEP45s3hD3+wdwJKKb+liV/lr2pVW++fkgKDB8OUKbYE9M9/hmPHnI5OKXUNNPErz9SqZev9N2+2K4C+8YadF/DWW3DmjNPRKaWugiZ+dXWaNIGZM2H1atsT4Lnn7FyAqVN1EphSfkITv7o2bdrYev/vv7dtIR95BKKibF8AnQSmlE/TxK8Kp3NnOwN4zhwoXdo2hI+Ph3nzdBKYUj5KE78qPBG45Ra7BtCHH9qVPxMSoHt32xNAKeVTNPEr7wkJsfX+W7bAu+/Cpk1w/fVw++22O5hSyido4lfeV7q0rfdPSYHRo21T+FatYOBA2xZSKeUoTfyq6FSoAC+9ZCeBPfMMzJoFTZvCk0/CoUNOR6dU0NLEr4petWq23j852V71T55s5wCMGAEnTjgdnVJBRxO/Kj516th6/40b7Yqgo0bZE8C4cXD2rNPRKRU0NPGr4tesGXz6KSQlQdu2dhioSRO7KmhWltPRKRXwNPEr57Rta+v9Fy2yS0I8/DBER8Pnn+scAKWKkCeNWKaJyCER2XCF9yuLyP9EZK2IbBSRQRe9ly0ia1xf2n1L5S233n/2bChRAvr1g3btYOFCpyNTKiB5csX/AZCQz/tPAJuMMa2BrsA7F7ViPGOMiXV93VqoSFVgE7H1/uvXw/TptuqnVy/o2RNWrnQ6OqUCSoGJ3xizBDiS3yZARRERoIJrWx2oVdcmJMRW/mzbBuPHw9q19uq/Xz87MUwpVWjeGOOfhO27uw9YDzxpjMldpauMiCSJyHIRud0L+1LBonRpW++fmgojR9pnAVFRtifA7t1OR6eUX/NG4u8DrAFqAbHAJBGp5HqvnqsH5H3AeBFpdKUPEZEhrpNEUnp6uhfCUgGhYkVb75+aak8EH39sJ4E9/TRkZDgdnVJ+yRuJfxDwX2MlAzuA5gDGmH2u/6YC3wFtrvQhxpipxph4Y0x8RESEF8JSASUiwtb7b98O990HEybYOQCjRsHJk05Hp5Rf8Ubi3w30ABCRGkAzIFVEqohIadfr4UBHYJMX9qeCWb16MG0abNhgH/6OGGFbQU6YAOfOOR2dUn7Bk3LOmcAyoJmIpInIYBF5VEQedW0yGrhBRNYDi4DnjTEZ2HH/JBFZCywGxhhjNPEr72jRwtb7r1hhF4B76ik7BPTBB5Cd7XR0Svk0MT44USY+Pt4kJSU5HYbyJwsXwvDhsGoVtGwJr78Ot91my0SVCgIissr1TLVAOnNXBYbcev///Mcu+3DHHbYXwOLFTkemlM/RxK8Ch4it99+4Ed5/H/butbOC+/SxdwJKKUATvwpEJUvaev9t2+Dtt+1icPHxcM899jWlgpwmfhW4ypa1K3+mpsLLL8PXX9vx/yFDIC3N6eiUcowmfhX4Kle29f4pKfDEE7byp0kTeO45OHzY6eiUKnaa+FXwqFHD1vtv2wZ3322HgRo2tBVAmZlOR6dUsdHEr4JPZCT861+wbh1062b7AjduDJMmwfnzTkenVJHTxK+CV3Q0fPEF/PQTNG8OQ4fa/378sU4CUwFNE79SufX+iYkQFgYPPACxsfC//2knMBWQNPErBXYOQJ8+tvRz1izb/P3WW6FTJ1iyxOnolPIqTfxKXaxECVvvv2kT/P3vsHMndOkCN90Ea9Y4HZ1SXqGJX6m8hIbaev/t2+HNN21P4DZt7JLQyclOR6dUoWjiVyo/5crZev/UVHjxRfjyS7sy6GOPwb59Tken1DXRxK+UJ8LCbL1/crK9E3j/fVsC+sILcPSo09EpdVU08St1Na67DiZPto3f77zTDgM1bAhjxsDp005Hp5RHNPErdS0aNbL1/mvW2MqfF16wdwDvvQcXLjgdnVL58ijxi8g0ETkkIhuu8H5lEfmfiKwVkY0iMuii934nIttdX7/zVuBK+YSYGFvv/8MP9sr/8cftM4CZMyEnx+nolMqTp1f8HwAJ+bz/BLDJGNMa6Aq8IyKlRKQqMAJoD7QDRohIlWsPVykf1amTTf5ffQXly9vqn7g4+OYbnQSmfI5Hid8YswQ4kt8mQEUREaCCa9ssoA+wwBhzxBhzFFhA/icQpfyXCNx8M/zyC8yYASdP2p+7dIGlS52OTik3b43xT8I2V98HrAeeNMbkALWBPRdtl+Z6TanAVaKEveLfvNk+CN6+3d4R3HKLXRhOKYd5K/H3AdYAtYBYYJKIVALy6nSd532viAwRkSQRSUpPT/dSWEo5qFQpO+afnAxvvGGHgmJj7VpAqalOR6eCmLcS/yDgv8ZKBnYAzbFX+HUv2q4O9q7gV4wxU40x8caY+IiICC+FpZQPKF/eVv2kptrJYJ999n+rgR486HR0Kgh5K/HvBnoAiEgNoBmQCswDeotIFddD3d6u15QKPlWr2nr/5GR46CFb+tmwoe0HcPy409GpIOJpOedMYBnQTETSRGSwiDwqIo+6NhkN3CAi64FFwPPGmAxjzBHXeytdX6NcrykVvGrXhilT7DOAW2+1M4IbNoS33oIzZ5yOTgUBMT5YahYfH2+SkpKcDkOp4vHLL3YdoMREe1IYMQIGDYKSJZ2OTPkREVlljIn3ZFuduauU09q0gblz4bvvoF49uxZQy5bw6ac6CUwVCU38SvmK3Hr/L7+0FUH33AO/+Q3Mm6eTwJRXaeJXypeI2HH/tWvhww/hyBFISIDu3W1PAKW8QBO/Ur4oJMTW+2/ZAhMnwsaNtjfw7bfb75UqBE38Svmy0qVtvX9qKowebZvCt2oFAwfatpBKXQNN/Er5gwoVbL1/Sgo8/bRtCN+sGTz1FBw65HR0ys9o4lfKn4SHw9tv2/V/HnwQ3n3X9gYYMQJOnHA6OuUnNPEr5Y/q1oV//AM2bYK+fWHUKDsJbNw4OHvW6eiUj9PEr5Q/a9bM1vuvXGnX/3/mGWjaFKZNg6wsp6NTPkoTv1KBID4e5s+HhQuhZk0YPNg+BP78c50DoH5FE79SgaRHD1ixAv77XzsnoF8/aN8eFi1yOjLlQzTxKxVoROCOO2zTl2nT4MAB6NnTfq1c6XR0ygdo4lcqUJUsaRd727YN/vpXOxu4XTt7F7Bli9PRKQdp4lcq0JUpY+v9U1Jg5Ei79k9UlH0OsHu309EpB2jiVypYVKpk6/1TU+GPf4SPP7YVQM88AxkZTkenipEmfqWCTUSEHfrZts02hR8/3s4BGDUKTp50OjpVDApM/CIyTUQOiciGK7w/TETWuL42iEi2iFR1vbdTRNa73ivyzir7Tu7DFxvLKOWT6te3D3/Xr7cPfkeMsLOAJ06Ec+ecjk4VIU+u+D8AEq70pjHmLWNMrDEmFngB+P6y9ordXO971BnmWmXnZBP1tyiqvFmFB2c/yH83/5cT53QKu1IFatnSln8uXw7R0fDkk3Zi2L/+BdnZTkenikCBid8YswTwtE9uf2BmoSK6Rlk5WfRs0JPj547z6cZPuevTu6g2thpdPujCmB/HsObAGr0bUCo/ufX+8+fbNYEGDoSYGPjiC50EFmA86rkrIpHAV8aY6Hy2KQekAY1zr/hFZAdwFDDA340xUz0J6lp77maczmD6L9MZEDOA5CPJzE2eS2JyIr8c+AWAmhVqktA4gb6N+9KrYS+qlK1y1ftQKigYY2f9/vnP9llA+/YwZgx07ep0ZOoKrqbnrjcT/z3AAGPMLRe9VssYs09EqgMLgKGuO4i8fn8IMASgXr16bXft2uVJ/B7Zf3I/81PmMzd5LvNT5nP07FFKSAk61OlAQqME+jbpS9x1cZQQfdat1CWysuCDD2wZ6N690Ls3vPEGtG3rdGTqMk4l/tnAf4wxn1zh/ZFApjHm7YL2d61X/J7Izsnm570/k5icyNzkuSTtS8JgiCgXQZ/GfUholECfxn0ILxdeJPtXyi+dOQN/+5tN+keOwN1328YwTZs6HZlyKfbELyKVgR1AXWPMKddr5YESxpiTru8XAKOMMYkF7a8oE//l0k+lu+8G5qXMI+N0BoIQXyuevo37ktA4gXa12xFSIqRY4lHKpx0/bvsB/PWvdvnnhx6CV16BOnWcjizoeTXxi8hMoCsQDhwERgChAMaYKa5tBgIJxph7L/q9hsBs148lgU+MMa97ElRxJv6L5ZgcVu1b5b4bWLF3BTkmhyplqtC7UW/6Nu5Ln8Z9qFmhZrHHppRPOXgQXn8dpkyx/YGHDoXnn4dq1ZyOLGh5/Yq/uDmV+C935MwRFqQsIDElkcTkRA5kHgCgTc027ruB6+teT8kSJR2OVCmH7Nhhx/8/+ggqVoTnnrPLQ5Qv73RkQUcTfxHIMTmsO7iOudvnkpiSyNLdS8k22VQuXZmeDXu67wbqVNJbXhWENmywFUBz5kCNGvDyy/D730OpUk5HFjQ08ReD42ePs2jHIuZun8vc5LnsPbkXgFbVW7lLRjvW60ipEP2Hr4LITz/BCy/AkiXQoIFdBqJ/fzscpIqUJv5iZoxhY/pG97OBH3b9wIWcC1QoVYEeDXq4TwT1w+o7HapSRc8YuwLoCy/AmjW2E9gbb8DNN9teAapIaOJ3WOb5TL7d8a37bmDXcTsnoXl4c/ezgc71O1OmZBmHI1WqCOXk2H7AL78Myclwww12EtiNNzodWUDSxO9DjDFsPbzVfTfw/c7vOZd9jrIly9KtQTf3iaBx1cZOh6pU0bhwwS4G9+qrsH8/3HSTrQiKjXU6soCiid+Hnb5wmu92fuc+ESQfSQagcdXG7lnEXSO7Ui60nMORKuVlp0/DpEnwl7/AsWN27H/UKGisFz3eoInfjyQfSSYx2ZaLfrvjW85knaF0SGm6RHZx3w00q9YM0bFRFSiOHoW33rJ9AC5cgIcftsNBtWo5HZlf08Tvp85mneWHXT+4F5fbnLEZgMiwSPfdQPcG3alQqoLDkSrlBfv3w2uvwdSpEBoC1GyHAAAUE0lEQVRql4N+7jmooosnXgtN/AFi57GdzEuex9zkuSzasYjM85mElgjlxvo3uk8EURFRejeg/FtKil324ZNPICwMhg+3M4HL6XDn1dDEH4DOZ59n6e6l7ruB9YfWA1CnUh0SGiWQ0DiBng17UrlMZYcjVeoarV1rJ4F9/TVcd509GQwebO8GVIE08QeBtBNp7ruBBakLOHHuBCESwg11b3A/G4itGat3A8r//PCDnQOwdKltBTl6NNxzD5TQZdPzo4k/yFzIvsDytOXuSqGLG8/0adTHNp5p1IuqZas6HKlSHjIGvvnGngDWr4fWrW01UEKCTgK7Ak38Qe5A5gHmJc8jMSWR+SnzOXLmCCWkBO1rt3fPIm5bq602nlG+LycHZs60VT87dtjJX3/5C3Ts6HRkPkcTv3LLzslm5b6V7sXlVu5dicEQXi7cfTfQu1FvIspHOB2qUld2/jy8/76t+z94EG65xU4Ca9XK6ch8hiZ+dUXpp9JZkLrANp5Jnkf66XR345ncuwFtPKN81qlTMGECjB0LJ07A/ffbGcENGzodmeM08SuP5JgcVu9f7b4bWJ62/JLGMwmNbbWQNp5RPufIEXjzTZg4EbKz4ZFH4KWX7JLQQcrbHbimAb8FDuXVelFEhgH3u34sCbQAIowxR0QkAZgAhADvG2PGeBKUJn5nHDlzhIWpC90lo7mNZ2Jrxv5f45k61xMaouV1ykfs3Wurft5/H0qXhj/9CYYNg8rBV9bs7cTfGcgEPsyv2bpr21uAPxljuotICLAN6AWkASuB/saYTQUFpYnfecYY1h5c664Uym08U6l0JXfjmYTGCdp4RvmG7dvtA+B//xuqVrXVQE88AWXLOh1ZsSn2ZusXbfcJsNgY8w8RuR4YaYzp43rvBQBjzF8K2p8mft+T23gm90SQdiINgOjq0e5ZxB3rdqR0ydIOR6qC2urV8OKLth9A7dowYgQMGgQlA789qiOJX0TKYa/sG7uGefphG7A/7Hr/AaC9MeYPBe1PE79vM8awKX2Te0hoya4lXMi5QPnQ8vRo2MN9IogMi3Q6VBWsvvvOXvUvXw5Nm9o1ge66K6AngV1N4vfmafAWYKkx5khuHHlsc8WzjIgMAYYA1KtXz4thKW8TEaKqRxFVPYpnb3iWzPOZLN6xmLnJtvHMnK1zANt4JvckoI1nVLHq2tW2gZwzxy4DcffdEBdn5wD06hX0k8C8ecU/G/iPMeYT18861BOEjDFsO7zNfTfw3c7vLmk8k3si0MYzqthkZ8OMGXbtn127oFs3ewJo397pyLyq2Id6RKQysAOoa4w55XqtJPbhbg9gL/bh7n3GmI0F7U8Tf+A4feE03+/83n0i2H5kOwCNqjRyPyDu1qCbNp5RRe/cObsE9OjRkJ4Ot99uh4CiopyOzCu8XdUzE+gKhAMHgRFAKIAxZoprm4HY8fx7L/vdm4Dx2HLOacaY1z0JShN/4Eo5kuJ+QHxx45nO9Tu7TwTNw5vr4nKq6Jw8aZvAvPWWnRD2wAN2Elj9+k5HVig6gUv5hdzGM7kngtzGM/Ur13fPIu7eoDsVS1d0OFIVkDIybPP3SZPsonCPPWYrgqpXdzqya6KJX/mlXcd22TaUKYksTF3objzTqV4n94kgunq03g0o79qzx17xT59um788/TQ88wxUquR0ZFdFE7/ye+ezz/PTnp+Yu91WCuU2nqldsbb7JNCjYQ/CyoQ5HKkKGFu22Elgn30G1arZq//HH4cy/lGNpolfBZy9J/a67wYWpCzg+Lnj7sYzuSeC1jVb61LTqvBWrrRJf+FCqFsXRo6EBx/0+UlgmvhVQMvKyWJ52nL34nKr968GoEb5Gu6F5Xo36q2NZ1ThLFpkJ4GtXAnNm9tloO+4w2fnAGjiV0HlQOYB5qfMZ27y3Esaz7Sr3c5dKRRfK17vBtTVMwZmz7aTwLZsgd/8xs4B6NHD6ch+RRO/Clq5jWdyK4UubjzTu1Fvd+OZ6uX9s3JDOSQrCz76yK79s2cP9OwJb7xhTwQ+QhO/Ui4ZpzOYnzLfPh9ITnQ3nmlbq617FnG72u0oWcK3x2+Vjzh7Ft57zyb9jAy7/s9rr9mhIIdp4lcqDzkmh1/2/+JeU+jixjO9GvUioZF9PnBdxeucDlX5uhMnYNw4eOcdOH3argA6YoR9GOwQTfxKeeDomaMsSF3gvhvYn7kfsI1ncu8GtPGMyld6ur36/9vf7EPfJ56wD4TDw4s9FE38Sl0lYwzrDq5zrym0dM9SsnKy3I1ncu8G6lZ27opO+bBdu2zZ54cfQvny8OyzthtYxeKbda6JX6lCOnHuBItSF7mHhXIbz0RFRLkrhTrV66SNZ9SlNm60vX+/+AIiIuz3jzxi20IWMU38SnlRbuOZ3EqhH3b/wPns85QPLU/3Bt3dJ4IGVRo4HaryFcuX2yGf776zi7+9+ioMGAAhIUW2S038ShWh3MYzuSeCHcd2ANCsWjP3LOLO9TtTNjR4+r2qPBgDCxbYE8Dq1Xb559dfh1tvLZJJYJr4lSomxhi2H9nunkX83c7vOJt1lrIly9I1sqv7RNCkWhOnQ1VOycmBzz+3wz7btkGHDnYSWNeuXt2NJn6lHJLbeCb3buDixjO5y0l0i+xG+VLlHY5UFbusLPjgA/sQeO9e6NPHVgTFxXnl4zXxK+UjchvPJKYk8u2Obzl94TSlQkpd0nimRXgLXWo6mJw5A5Mn26v+I0dsP+DRo21T+ELwdgeuacBvgUP5tF7siu20FQpkGGO6uF7fCZwEsoEsT4PSxK8C0dmss/y4+0f3sNCm9E0A1Ktcz30S6NGghzaeCRbHj8Pbb9uJYOfOweDBti9w7drX9HHeTvydgUzgw7wSv4iEAT9hWy/uFpHqxphDrvd2AvHGmIyrOQBN/CoY7Dq2i3kp85ibPNfdeKZkiZJ0qtfJfSJoVb2V3g0EuoMH7bIPU6ZAaKhdDK5evav+mGJtti4ijwO1jDEv5fHeTjTxK1Wg3MYzuc8G1h1cB0CtirXcs4h7NuypjWcC2Qsv2FaQY8fCsGFX/evFnfhzh3iigIrABGPMh673dgBHAQP83Rgz1ZOgNPGrYLfv5D73UhLzU+a7G89cX/d694kgtmasLjUdSDIybPvHQYOuacmH4k78k4B4oAdQFlgG3GyM2SYitYwx+0SkOrAAGGqMWXKFfQwBhgDUq1ev7a5duzyJX6mAl9t4JvdEsGr/KsA2nunTuA8JjWzjmWrlqjkcqXJScSf+4UAZY8xI18//BBKNMf+5bLuRQKYx5u2C9qdX/Epd2cHMg+7GM/NS5nHkzBEE+VXjmZASRTdLVPme4k78LYBJQB+gFPAzcC+wAyhhjDkpIuWxV/yjjDGJBe1PE79SnsnOySZpX5J7cbmf9/6MwVCtbLVLGs/UqFDD6VBVEfN2Vc9MoCsQDhwERmDH9DHGTHFtMwwYBOQA7xtjxotIQ2C262NKAp8YY173JChN/Epdm4zTGSxIWeC+Gzh06hAAba9r674baF+nvTaeCUA6gUsp5W48k1sptCxtGTkmh7AyYfRq2Iu+jfvSp3EfalWs5XSoygs08SulfuXomaMsTF3oPhHkNp5pXaO1e02hG+reoI1n/JQmfqVUvowxrD+03j2L+MfdP5KVk0XFUhVt4xnXiUAbz/gPTfxKqaty4twJvt3xLXO328Yze07sAaBlREv3s4Eb692ojWd8mCZ+pdQ1M8awOWOz+25gya4lnM8+T7nQcpc0nmlYpaHToaqLaOJXSnlN5vlMvtv5nftuILfxTNNqTd2ziLvU76KNZxymiV8pVSRyG8/kPiDObTxTpmQZukZ2dd8NNKnaRBeXK2aa+JVSxeLMhTN8v+v/Gs9sO7wNgIZVGrrvBrTxTPHQxK+UckTq0VT3mkKLdiy6pPFM7olAG88UDU38SinHncs6ZxvPuJaT2Ji+EbCNZxIa2TaUPRr2oFLpSg5HGhg08SulfM7u47vddwMLUxdy8vxJSpYoSce6Hd3PBmJqxOjdwDXSxK+U8mnns8+zbM8y993A2oNrAbiuwnXuyWM9G/akStkqDkfqPzTxK6X8yr6T+5iXbNtQLkhdwLGzxwiREDrU6eA+EbS5ro02nsmHJn6llN/KysliRdoKd6VQbuOZ6uWr06dRH/o27kuvRr0IL3f1XaoCmSZ+pVTAOHTqEPOS55GYksi85HkcPnPY3Xgm925AG89o4ldKBajsnGxW7V/lXk5iRdqKSxrPJDROoE+jPkHZeEYTv1IqKBw+fZj5KfNJTLHVQrmNZ+Kui3NXCnWo0yEoGs94uwPXNOC3wKG8Wi+6tukKjMd25sowxnRxvZ4ATABCsJ25xngSlCZ+pdTVyjE5rDmwxn03sGzPMrJNNpVLV6ZXI1fjmUZ9qF2pttOhFglvJ/7OQCbw4RV67oYBPwEJxpjdIlLdGHNIREKAbUAvIA1YCfQ3xmwqKChN/Eqpwjp29hgLUxe6TwT7Tu4DIKZGjHsW8Q11b6BUSCmHI/WO4m62/jhQyxjz0mWvXw+MNMb0cf38AoAx5i8F7U8Tv1LKm3Ibz+RWCl3ceKZHwx7uYaF6les5Heo1u5rE742Br6ZAqIh8B1QEJhhjPgRqA3su2i4NaO+F/Sml1FUREWJqxBBTI4bnOj7nbjyTeyL4YssXgG08k3s3EMiNZ7yR+EsCbYEeQFlgmYgsB/Kad33F2wsRGQIMAahXz3/Pukop31epdCVub347tze/HWMMWzK2uGcRT1o5iXHLx7kbz+SeCAKp8Yw3En8a9oHuKeCUiCwBWrtev7hhZx1g35U+xBgzFZgKdqjHC3EppVSBRIQWES1oEdGCp69/mlPnT9nGM8m28cxX276CudCkahP3kFDXyK5+3XjGG2P8LYBJQB+gFPAzcC+wBftwtwewF/tw9z5jzMaC9qdj/EopX2CMIflIsvtuYPHOxe7GM13qd3GfCJpWa+r44nLeruqZCXQFwoGDwAhs2SbGmCmubYYBg4AcbNnmeNfrN2HLPEOAacaY1z0JShO/UsoXnblwhiW7lrhPBFsPbwWgQVgD9yzibg26UaFUhWKPTSdwKaVUMdhxdIf7AfG3O77l1IVTlAopxY31bnSfCFpGtCyWuwFN/EopVcxyG8/knghyG8/UrVTXfRIoysYzmviVUsphe47vsY1nUhJZkLLgksYzuScCbzae0cSvlFI+5EL2BZalLWPudlspdHnjmYTGCfRq2KtQjWc08SullA/LbTyTmJLI/JT5HDt7DEGIDIvkp8E/UbNCzav+zKtJ/NrORimlilmtirUY1GYQ/+73b9KHpbP0oaV0b9CdHcd28NHaj4p8/4G/VqlSSvmwkiVKckPdG5jVbxbTf5nOoDaDin6fRb4HpZRSBQovF86wjsOKZV861KOUUkFGE79SSgUZTfxKKRVkNPErpVSQ0cSvlFJBRhO/UkoFGU38SikVZHxyyQYRSQd2XeOvhwMZXgzHH+gxB75gO17QY75a9Y0xEZ5s6JOJvzBEJMnT9SoChR5z4Au24wU95qKkQz1KKRVkNPErpVSQCcTEP9XpABygxxz4gu14QY+5yATcGL9SSqn8BeIVv1JKqXz4beIXkQQR2SoiySIyPI/3S4vIv13vrxCRyOKP0ns8ON6nRWSTiKwTkUUiUt+JOL2poGO+aLt+ImJExO8rQDw5ZhG52/V3vVFEPinuGL3Ng3/b9URksYj84vr3fZMTcXqLiEwTkUMisuEK74uITHT9eawTkTivB2GM8bsvIARIARoCpYC1QMvLtnkcmOL6/l7g307HXcTH2w0o5/r+MX8+Xk+P2bVdRWAJsByIdzruYvh7bgL8AlRx/Vzd6biL4ZinAo+5vm8J7HQ67kIec2cgDthwhfdvAuYCAnQAVng7Bn+94m8HJBtjUo0x54FZwG2XbXMb8C/X958BPcRb7eyLX4HHa4xZbIw57fpxOVCnmGP0Nk/+jgFGA2OBs8UZXBHx5Jh/D0w2xhwFMMYcKuYYvc2TYzZAJdf3lYF9xRif1xljlgBH8tnkNuBDYy0HwkTkOm/G4K+Jvzaw56Kf01yv5bmNMSYLOA5UK5bovM+T473YYOwVgz8r8JhFpA1Q1xjzVXEGVoQ8+XtuCjQVkaUislxEEootuqLhyTGPBAaISBrwDTC0eEJzzNX+/37V/LX1Yl5X7peXJ3myjb/w+FhEZAAQD3Qp0oiKXr7HLCIlgL8CA4sroGLgyd9zSexwT1fsXd0PIhJtjDlWxLEVFU+OuT/wgTHmHRG5HvjIdcw5RR+eI4o8d/nrFX8aUPein+vw69s/9zYiUhJ7i5jf7ZUv8+R4EZGewJ+BW40x54optqJS0DFXBKKB70RkJ3YsdI6fP+D19N/1l8aYC8aYHcBW7InAX3lyzIOBTwGMMcuAMtg1bQKVR/+/F4a/Jv6VQBMRaSAipbAPb+dcts0c4Heu7/sB3xrXkxM/VODxuoY9/o5N+v4+7gsFHLMx5rgxJtwYE2mMicQ+17jVGJPkTLhe4cm/6y+wD/IRkXDs0E9qsUbpXZ4c826gB4CItMAm/vRijbJ4zQEedFX3dACOG2P2e3MHfjnUY4zJEpE/APOwVQHTjDEbRWQUkGSMmQP8E3tLmIy90r/XuYgLx8PjfQuoAPzH9Qx7tzHmVseCLiQPjzmgeHjM84DeIrIJyAaGGWMOOxd14Xh4zM8A/xCRP2GHPAb68UUcIjITO1QX7npuMQIIBTDGTME+x7gJSAZOA4O8HoMf//kppZS6Bv461KOUUuoaaeJXSqkgo4lfKaWCjCZ+pZQKMpr4lVIqyGjiV0qpIKOJXymlgowmfqWUCjL/H2IaxX+8Rmo2AAAAAElFTkSuQmCC\n", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "nn.fit_epoch(X_train,y_train,X_test,y_test,lr=0.01,epochs=2,bs=512,beta1=0.9)" ] }, { "cell_type": "code", - "execution_count": 63, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Epoch 1. Training loss: 1.5014039895713864, Val loss:1.4282706859763334\n", - "Epoch 2. Training loss: 1.356052986532401, Val loss:1.30039896845335\n", - "Epoch 3. Training loss: 1.245414725116916, Val loss:1.2063326238489638\n", - "Epoch 4. Training loss: 1.1591243698569502, Val loss:1.129896227482362\n", - "Epoch 5. Training loss: 1.0839643596784354, Val loss:1.0575557732351633\n", - "Epoch 6. Training loss: 1.0063708394080297, Val loss:0.9781137288597281\n", - "Epoch 7. Training loss: 0.9266249021201841, Val loss:0.9046629082662417\n", - "Epoch 8. Training loss: 0.8594279318762205, Val loss:0.8493280604017351\n", - "Epoch 9. Training loss: 0.8119586484110163, Val loss:0.8094829150730612\n", - "Epoch 10. Training loss: 0.7749317808186631, Val loss:0.7756003862150562\n", - "Epoch 11. Training loss: 0.74166099097438, Val loss:0.7435236769302102\n", - "Epoch 12. Training loss: 0.7093291147775425, Val loss:0.7118740131342062\n", - "Epoch 13. Training loss: 0.6782940451112496, Val loss:0.6823216262085491\n", - "Epoch 14. Training loss: 0.6504175261818713, Val loss:0.6569080249607883\n", - "Epoch 15. Training loss: 0.6270665933881882, Val loss:0.6364130705234229\n", - "Epoch 16. Training loss: 0.6083222807021489, Val loss:0.619957635083624\n", - "Epoch 17. Training loss: 0.5931017588668728, Val loss:0.606578978661161\n", - "Epoch 18. Training loss: 0.5803457991717884, Val loss:0.5952747784471085\n", - "Epoch 19. Training loss: 0.5692909030356361, Val loss:0.5854929484194242\n", - "Epoch 20. Training loss: 0.5594647274891112, Val loss:0.5768556403632602\n" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3XdcleX/x/HXxVCWCgKKiopZuZAlbhJXzlw4yllqkqtl39Kvv4aZlvk1M8scGWTlzJWZIzNzpJm4cGWaEye4twLX74+bEIylHrg5h8/z8TiPM+7r3OfDid7cXvd1X5fSWiOEEMK22JldgBBCCMuTcBdCCBsk4S6EEDZIwl0IIWyQhLsQQtggCXchhLBBEu5CCGGDJNyFEMIGSbgLIYQNcjDrg728vLSfn59ZHy+EEFZp69atCVpr7+zamRbufn5+xMTEmPXxQghhlZRSR3PSTrplhBDCBkm4CyGEDco23JVSZZVSa5RS+5RSe5RSL2fQRimlJiqlDiqlYpVSIblTrhBCiJzISZ97IvCa1nqbUqoIsFUptUprvTdNm5bAYym32sDklHshRD5x584d4uLiuHnzptmliBxwcnLC19cXR0fHB3p/tuGutT4FnEp5fEUptQ8oA6QN93bA19qYHP53pZS7UqpUynuFEPlAXFwcRYoUwc/PD6WU2eWILGitOXfuHHFxcVSoUOGB9nFffe5KKT8gGNh8z6YywPE0z+NSXrv3/ZFKqRilVEx8fPz9VSqEeCg3b97E09NTgt0KKKXw9PR8qH9l5TjclVJuwALgFa315Xs3Z/CWfy3xpLWeprUO1VqHentnO0xTCGFhEuzW42H/W+Uo3JVSjhjBPlNrvTCDJnFA2TTPfYGTD1VZZn77DRo0gBMncmX3QghhC3IyWkYBXwL7tNbjM2m2BOiVMmqmDnAp1/rbo6Jg/Xp4++1c2b0QInecO3eOoKAggoKC8PHxoUyZMqnPb9++naN99O7dm/3792fZZtKkScycOdMSJRMWFsaOHTsssq+8lpPRMvWBnsAupdQ/P+VwoByA1noKsAxoBRwErgO9LV9qipEjYeZMyOEvgxAif/D09EwNyhEjRuDm5sZ//vOfdG201mitsbPL+LgzOjo6288ZNGjQwxdrA7I9ctdab9BaK611gNY6KOW2TGs9JSXY0YZBWuuKWuvqWuvcm1egTBno2BFWrICkpFz7GCFE3jh48CD+/v7079+fkJAQTp06RWRkJKGhoVSrVo2RI0emtv3nSDoxMRF3d3eGDRtGYGAgdevW5ezZswC8+eabTJgwIbX9sGHDqFWrFpUqVWLjxo0AXLt2jY4dOxIYGEjXrl0JDQ3N9gj922+/pXr16vj7+zN8+HAAEhMT6dmzZ+rrEydOBODjjz+matWqBAYG0qNHD4t/Zzlh2twyDyUiAmbNMrpnGjY0uxohrM8rr0BOuhvu3IHTp8HHB7Ibbx0UBCmher/27t1LdHQ0U6ZMAWDMmDEUL16cxMREGjVqRKdOnahatWq691y6dInw8HDGjBnDkCFDiIqKYtiwYf/at9aaP/74gyVLljBy5EhWrFjBp59+io+PDwsWLGDnzp2EhGR93WVcXBxvvvkmMTExFCtWjKZNm7J06VK8vb1JSEhg165dAFy8eBGAsWPHcvToUQoVKpT6Wl6zzukHWrQAJydYmNG5XSGExZw+DYcOGfe5qGLFitSsWTP1+ezZswkJCSEkJIR9+/axd+/ef73H2dmZli1bAlCjRg2OHDmS4b4jIiL+1WbDhg0888wzAAQGBlKtWrUs69u8eTONGzfGy8sLR0dHunXrxrp163j00UfZv38/L7/8MitXrqRYsWIAVKtWjR49ejBz5swHvgjpYVnnkburqxHwCxcaRwqZ9M8JITKR0yPshASIjobevcHLK9fKcXV1TX184MABPvnkE/744w/c3d3p0aNHhuO9CxUqlPrY3t6exMTEDPdduHDhf7UxrrfMuczae3p6Ehsby/Lly5k4cSILFixg2rRprFy5krVr1/L9998zatQodu/ejb29/X195sOy3lSMiDCGQ8q0wULkHi8veP31XA32e12+fJkiRYpQtGhRTp06xcqVKy3+GWFhYcybNw+AXbt2Zfgvg7Tq1KnDmjVrOHfuHImJicyZM4fw8HDi4+PRWtO5c2feffddtm3bRlJSEnFxcTRu3Jj//e9/xMfHc/36dYv/DNmxziN3gKeeAgcH4+i9Vi2zqxFCWEhISAhVq1bF39+fRx55hPr161v8M1588UV69epFQEAAISEh+Pv7p3apZMTX15eRI0fSsGFDtNa0adOG1q1bs23bNvr27YvWGqUUH374IYmJiXTr1o0rV66QnJzM0KFDKVKkiMV/huyo+/3niaWEhobqh16so0UL+Ptv+OsvkCvvhMjSvn37qFKlitll5AuJiYkkJibi5OTEgQMHaNasGQcOHMDBIX8d72b030wptVVrHZrde/PXT3K/IiLghRdg926oXt3saoQQVuLq1as0adKExMREtNZMnTo13wX7w7Lun6ZdO+jf3+iakXAXQuSQu7s7W7duNbuMXGW9J1QBSpaEsDAZEimEEPew7nAHo2smNhYOHjS7EiGEyDesP9w7dDDuFy0ytw4hhMhHrD/cy5eH0FDpmhFCiDSsP9zB6Jr5/XeIizO7EiFEJho2bPivC5ImTJjAwIEDs3yfm5sbACdPnqRTp06Z7ju7odUTJkxIdzFRq1atLDLvy4gRIxg3btxD78fSbCfcARYvNrcOIUSmunbtypw5c9K9NmfOHLp27Zqj95cuXZr58+c/8OffG+7Lli3D3d39gfeX39lGuFeqBFWrSteMEPlYp06dWLp0Kbdu3QLgyJEjnDx5krCwsNRx5yEhIVSvXp3vv//+X+8/cuQI/v7+ANy4cYNnnnmGgIAAnn76aW7cuJHabsCAAanTBb/zzjsATJw4kZMnT9KoUSMaNWoEgJ+fHwkJCQCMHz8ef39//P39U6cLPnLkCFWqVKFfv35Uq1aNZs2apfucjOzYsYM6deoQEBBAhw4duHDhQurnV61alYCAgNQJy9auXZu6WElwcDBXrlx54O82I9Y9zj2tiAh4/31joqM8nAdDCGv0yopX2HE6+yl/7yTd4fTV0/i4+eBon/XshkE+QUxokfmEZJ6entSqVYsVK1bQrl075syZw9NPP41SCicnJxYtWkTRokVJSEigTp06tG3bNtN1RCdPnoyLiwuxsbHExsamm7J39OjRFC9enKSkJJo0aUJsbCwvvfQS48ePZ82aNXjdkw9bt24lOjqazZs3o7Wmdu3ahIeH4+HhwYEDB5g9ezZffPEFXbp0YcGCBVnOz96rVy8+/fRTwsPDefvtt3n33XeZMGECY8aM4fDhwxQuXDi1K2jcuHFMmjSJ+vXrc/XqVZycnLL8fu+XbRy5gxHuycmwZInZlQhhM05fPc2hi4c4fdUyU/6m7ZpJ2yWjtWb48OEEBATQtGlTTpw4wZkzZzLdz7p161JDNiAggICAgNRt8+bNIyQkhODgYPbs2ZPtpGAbNmygQ4cOuLq64ubmRkREBOvXrwegQoUKBAUFAVlPKwzG/PIXL14kPDwcgGeffZZ169al1ti9e3e+/fbb1Cth69evz5AhQ5g4cSIXL160+BWytnPkHhQEfn5G10yfPmZXI0S+ltURdloJ1xOI3h5N7+DeeLk8/L+I27dvz5AhQ9i2bRs3btxIPeKeOXMm8fHxbN26FUdHR/z8/DKc5jetjI7qDx8+zLhx49iyZQseHh4899xz2e4nq/m1/pkuGIwpg7PrlsnMjz/+yLp161iyZAnvvfcee/bsYdiwYbRu3Zply5ZRp04dfv75ZypXrvxA+89IThbIjlJKnVVK7c5kezGl1A9KqZ1KqT1KqdxbPzUrShnL761aBZcvm1KCELbGy8WL1+u/bpFgB2PkS8OGDenTp0+6E6mXLl2iRIkSODo6smbNGo4ePZrlfho0aJC6CPbu3buJjY0FjOmCXV1dKVasGGfOnGH58uWp7ylSpEiG/doNGjRg8eLFXL9+nWvXrrFo0SKeeOKJ+/7ZihUrhoeHR+pR/zfffEN4eDjJyckcP36cRo0aMXbsWC5evMjVq1f5+++/qV69OkOHDiU0NJQ///zzvj8zKznplvkKaJHF9kHAXq11INAQ+EgpVSiL9rknIsJYOPvHH035eCFE9rp27crOnTtTTywCdO/enZiYGEJDQ5k5c2a2R7ADBgzg6tWrBAQEMHbsWGqlTPsdGBhIcHAw1apVo0+fPummC46MjKRly5apJ1T/ERISwnPPPUetWrWoXbs2zz//PMHBwQ/0s82YMYPXX3+dgIAAduzYwdtvv01SUhI9evSgevXqBAcH8+qrr+Lu7s6ECRPw9/cnMDAw3apSlpKjKX+VUn7AUq21fwbb/guUxQh5P2AV8LjWOjmrfVpkyt97JScbC2iHhcF331l230JYOZny1/o8zJS/ljih+hlQBTgJ7AJezi7Yc42dnTEdwbJl8IB9Y0IIYQssEe7NgR1AaSAI+EwpVTSjhkqpSKVUjFIqJj4+3gIfnYGICLh+HX76KXf2L4QQVsAS4d4bWKgNB4HDQIYdZlrraVrrUK11qLe3twU+OgPh4eDhIRc0CZEBs1ZeE/fvYf9bWSLcjwFNAJRSJYFKwCEL7PfBODpC27bGePfbt00rQ4j8xsnJiXPnzknAWwGtNefOnXuoC5uyHeeulJqNMQrGSykVB7wDOKYUMAV4D/hKKbULUMBQrXXCA1dkCR07wowZ8Ouv0KyZqaUIkV/4+voSFxdHrnWJCotycnLC19f3gd+fbbhrrbOc1UdrfRLIXwn65JPg6mp0zUi4CwGAo6MjFSpUMLsMkUesbvqBuMtxPDP/GeKvZXH04eQErVsbs0QmJeVdcUIIkU9YXbgPWTmEuXvmMuSnIVk3jIiAM2dg06a8KUwIIfIRqwv3iS0nUrpIaVYfWs3V21czb9iqFRQqJKNmhBAFktWFu4+bD991/o5TV08xat2ozBsWKWL0ty9cCDI6QAhRwFhduAPUK1uP54KeY/ym8fyZkMVkOx07wtGjsG1b3hUnhBD5gFWGO8CHTT/ExdGFF5e/mPm43TZtwN5eumaEEAWO1YZ7CdcSjG48mp8P/cz8vZmsq+jpCQ0bSrgLIQocqw13gP6h/Qn2CebVla9mfnI1IgL+/BP27cvb4oQQwkRWHe72dvZMajWJE1dOZH5ytX17416O3oUQBYhVhztA3bJ16R3Um482fcS++AyOzkuXhrp1YcGCvC9OCCFMYvXhDjCm6RjcCrllfnI1IgK2b4fDh/O+OCGEMIFNhPs/J1dXH16d8cnViAjjftGivC1MCCFMYhPhDvBCjRcyP7n6yCMQFCT97kKIAsNmwj3tydX31r737wYREbBxI5w6lffFCSFEHrOZcAfj5GqfoD6M/338v0+uRkQY0xAMGgQJ5k43L4QQuc2mwh2yOLlatSqUKGH0u0+dal6BQgiRB2wu3L1dvVNPrn6397u7G5SCyZONx8eOmVOcEELkEZsLd0h/cvXKrSt3N0REwODBMG0arFljXoFCCJHLbDLc7e3s+bz155y8cpL31t1zcvXDD+Gxx6B3b7h82ZwChRAil2Ub7kqpKKXUWaXU7izaNFRK7VBK7VFKrbVsiQ+mjm8d+gb35ePfP05/ctXFBb76Co4fhyHZrOYkhBBWKidH7l8BLTLbqJRyBz4H2mqtqwGdLVPaw/ugyQe4FXJj8PLB6U+u1qsHr78OX34JP/5oXoFCCJFLsg13rfU64HwWTboBC7XWx1Lan7VQbQ/N29Wb9xu/zy+Hf2HennnpN777Lvj7w/PPw7lz5hQohBC5xBJ97o8DHkqpX5VSW5VSvSywT4uJrBFJSKkQhvw0JP3J1cKF4euvjTHvgwebV6AQQuQCS4S7A1ADaA00B95SSj2eUUOlVKRSKkYpFRMfH2+Bj86evZ09n7fK5ORqcDC88w7MmQPz5mW8AyGEsEKWCPc4YIXW+prWOgFYBwRm1FBrPU1rHaq1DvX29rbAR+dMbd/aqSdX98bvTb9x2DCoWRMGDoTTp/OsJiGEyE2WCPfvgSeUUg5KKRegNpDvlj36oMkHuDq60npWa85cPXN3g4MDzJgBV69Cv37GFAVCCGHlcjIUcjawCaiklIpTSvVVSvVXSvUH0FrvA1YAscAfwHStdabDJs3i7epNi0dbcOTiEfou6Zt+Y5Uq8MEHsHSpEfRCCGHlVIaLW+SB0NBQHRMTk6efGX8tnkYzGnHg3AG2999OVe+qdzcmJ0OjRrBjB+zaBeXK5WltQgiRE0qprVrr0Oza2eQVqpnxdvVmda/VFHUqyrOLn+VO0p27G+3sjIubkpKMq1eTk02rUwghHlaBCneAkm4lmdx6MjEnYxizYUz6jRUqwPjx8Msv8Pnn5hQohBAWUODCHaBT1U50q96NketGsu3UtvQb+/WDFi3gjTfgwAFzChRCiIdUIMMd4LOWn1HCtQS9FvXiVuKtuxuUgunTjYucnn3W6KYRQggrU2DD3cPZg+ltprMnfg/v/PpO+o1lysBnn8GmTTBunDkFCiHEQyiw4Q7Q8rGW9Avpx/82/o+Nxzem39itG3TsCG+/bYyeEUIIK1Kgwx3go2YfUa5YOZ5d/CzXbl+7u+GflZvc3aFXL7h927wihRDiPhX4cC9SuAhftfuKg+cPMuznYek3ensb663u2AGjRplToBBCPIACH+4A4X7hvFL7FT7b8hmrD61Ov7F9e+jSxQj3FSvMKVAIIe6ThHuK95u8TyXPSvRZ0odLNy+l3+jvb8w50727LM0nhLAKEu4pnB2dmdF+BnGX4xiy8p7l9wYMMGaNvHABXnzRnAKFEOI+SLinUdu3Nv8N+y9RO6L4Yf8Pdzd4ecGkSfDWW8YCH7Nnm1ekEELkQIGaOCwnbifdpuYXNTlz9Qx7Bu7B08Xz7sbERGjQAPbsgZ07wc/PtDqFEAWTTBz2gArZF+Lr9l9z/sZ5Bi0blH6jgwPMnGn0v/foYYS9EELkQxLuGQj0CWREwxHM3TOXubvnpt9YoYIx/v233+D9980pUAghsiHhnok36r9B7TK1GbhsIKeunEq/sXt34zZyJGzcmPEOhBDCRBLumXCwc2BG+xlcv3OdyKWR/OvcxKRJULasEfKXLmW8EyGEMImEexYqeVViTJMxLP1rKV/t+Cr9xmLFYNYsOH4cBg3K8P1CCGEWCfdsvFj7Rer51mPAjwP448Qf6TfWrWtMLDZzpnETQoh8IicLZEcppc4qpbJc9FopVVMplaSU6mS58sxnp+wI9wvnVtIt2sxqw+Vb91yhOnw41K9vXOh06JA5RQohxD1ycuT+FdAiqwZKKXvgQ2ClBWrKd4bUHUJkSCQJ1xPouagnyTrN+qoODvDtt8YskjI8UgiRT2Qb7lrrdcD5bJq9CCwAzlqiqPzGy8WLqW2m8knLT1iyfwnvrLlncQ8/P2P2yE2b4L33TKlRCCHSeug+d6VUGaADMOXhy8nfBtUcRN/gvoxaP4rv9nyXfuMzzxjzvo8aBRs2mFOgEEKksMQJ1QnAUK11touNKqUilVIxSqmY+Ph4C3x03lJKManVJOqVrcdz3z/HztM70zf47DPjKL5HD7h40ZQahRACLBPuocAcpdQRoBPwuVKqfUYNtdbTtNahWutQb29vC3x03ivsUJgFXRbg4eRBuzntiL+W5o9UkSLG8Mi4OOMEq0nz9gghxEOHu9a6gtbaT2vtB8wHBmqtFz90ZfmYj5sPi59ZzJlrZ+j8XWfuJN25u7F2bXj3XZgzB775xrwihRAFWk6GQs4GNgGVlFJxSqm+Sqn+Sqn+uV9e/hVaOpQv2nzB2qNreXXlq+k3DhtmzB45aBD8/bc5BQohCjSH7BporbvmdGda6+ceqhor0yOgBztP72TcpnEElgykX41+xgZ7e+OoPTDQmJ5g/XpwdDS3WCFEgSJXqD6kMU3H0LxicwYtG8SGY2lGyZQrZwyP3LwZmjaFhATzihRCFDgS7g/J3s6e2R1n4+fuR8d5HTl+6fjdjV26GFMUrFtnDJMUQog8IuFuAR7OHnz/zPfcuHOD9nPbc/3O9bsbFy0yumeWL4cpNn8pgBAin5Bwt5Aq3lWYGTGT7ae20++HfnenCC5ZErZsgaeeMhbZnjXL3EKFEAWChLsFtanUhlGNRzFr1yz+t/F/dzc4OsK8eRAebnTPLFliXpFCiAJBwt3C/hv2X7pU68Kwn4ex7MCyuxucnY1QDwkx+uJXrzavSCGEzZNwtzClFFFtowj0CaTbgm7sT9h/d2ORIkbf+2OPQbt2xkgaIYTIBRLuucC1kCuLn16Mo70j7ea049LNNMvweXrCTz+Bjw+0bAmxseYVKoSwWRLuuaS8e3nmd57PwfMHCZ4azN/n01ypWqoU/PwzuLhAs2Zw4IB5hQohbJKEey4K9wvnmWrPcPjiYcKiwjh55eTdjX5+sGoVJCUZFzkdP57pfoQQ4n5JuOeyCS0n8Hzw81y5fYV6X9ZL3wdfpQqsXGlMD/zkk3DWJtc6EUKYQMI9l3m5ePFF2y9Y+9xart+5Tv2o+ukX2g4JgR9/hGPHoHlzmQdeCGEREu55pEbpGmzsu5FiTsVoNKMRKw6uuLsxLAwWLoQ9e4yLna5dM69QIYRNkHDPQ48Wf5Tf+vzG456P02Z2G77ZmWa+9xYtjKtXN22CiAi4dcu8QoUQVk/CPY/5uPmw9rm1NCjfgF6LezFu47i7Gzt1gunTjaGS3bpBYqJ5hQohrJqEuwmKFi7Ksm7L6FKtC6+vep3//PQfknWysbF3b5gwweimqVMHzpwxt1ghhFWScDdJYYfCzO44m8E1B/PRpo/otagXt5NuGxtfftkYPbN1KzRsCJcvm1qrEML6SLibyE7ZMbHlREY3Hs3MXTNpM7sNV29fNTbOmgUdOsBff0G9enDokLnFCiGsioS7yZRSDH9iOF+2/ZLVh1bTeEZj4q/Fg5eX0TWzahWcPAm1asHatWaXK4SwEjlZIDtKKXVWKbU7k+3dlVKxKbeNSqlAy5dp+/oE92HR04vYdXYX9aPqc/jCYWND48bwxx/g7W1cyfrFF+YWKoSwCjk5cv8KaJHF9sNAuNY6AHgPmGaBugqkNpXasLrXahKuJ1Avqh47T+80Njz6qDFEskkTiIw0+uRlJI0QIgvZhrvWeh1wPovtG7XWF1Ke/g74Wqi2Aqle2Xps6LMBhaL29NpEb482Nri7w9Kl8OqrMHEitG4tV7MKITJl6T73vsByC++zwKnqXZXeQb25lXSLPkv6MGDpAG7cuQEODjB+vDEWfs0aY6ikzCgphMiAg6V2pJRqhBHuYVm0iQQiAcqVK2epj7ZJr9Z9lSKFi3Di8gk+2/IZG45vYG6nuVT1rgp9+xoLfnTsCLVrG0v4NW1qdslCiHzEIkfuSqkAYDrQTmt9LrN2WutpWutQrXWot7e3JT7aZnm5eDEsbBiftvqUFd1XcObqGUKnhRK1PcpYfLtBA+NEa+nSxtQFkyaZXbIQIh956HBXSpUDFgI9tdZ/PXxJ4l7NH23Ozv47qVe2Hn2X9KX7wu5cvnUZKlSAjRuNFZ0GD4aBA+HOHbPLFULkAzkZCjkb2ARUUkrFKaX6KqX6K6X6pzR5G/AEPldK7VBKxeRivQVWqSKlWNljJe83fp95e+YRPDWYmJMxULQoLF4Mb7wBkycbR/HnMz3/LYQoIJTW2pQPDg0N1TEx8nfgQWw8vpGuC7py6sopxjQdwyt1XsFO2cHXX0O/fkZXTceOMGyYcTGUEMJmKKW2aq1Ds2snV6haoXpl67H9he089fhTvPbTa7SZ3ca4qrVXL/j1V4iPh48+gv79waQ/3kIIc0m4W6nizsVZ0GUBk1pNYvWh1QROCWTN4TVQty78/rtx4dOCBdClC1y4kP0OhRA2RcLdiimlGFhzIJuf30zRwkVp8nUT3l7zNolVK8P+/fDhh0Z/fGAgrF9vdrlCiDwk4W4DAn0C2Rq5lWeDnuW9de8RFhXG/615i4TBfYzRNIUKGVMHv/OOTFsgRAEh4W4jXAu5Et0umm87fMv209t5f8P7vLT8JahZE7Zvhx49YORII+SPHjW7XCFELpNwtzHdA7qzofcGyhUrx+zds+m6oCvnHe7AjBkwcybExhrdNPPmmV2qECIXSbjboJplavL3S38zuvFo5u+dT8DkAFb9vcpYl3XHDqhcGZ5+2pjG4No1s8sVQuQCCXcb5WDnwPAnhqeebG32bTNeXv4yN8qWMk6uDh8O0dEQEgLbtpldrhDCwiTcbVxIqRC2Rm7l5dovM/GPiYRMC2FrfCyMHg2rV8PVq8bskuPHQ3Ky2eUKISxEwr0AcHZ0ZkKLCazquYort65Q58s6jF43msTwJ4w++Fat4LXXjPszZ8wuVwhhARLuBUjTR5qya8AuOlftzJtr3qRBdAMOqguwaBF8/rlxdWvFivDdd2aXKoR4SBLuBYyHswezOs5iVsQs9iXsI2hKEF9sm47u3x8GDTJOsHbpAm++KWPihbBiEu4FVNfqXdk1YBd1fOsQuTSStnPacublvjBqlDGqZvRoY3HuuDizSxVCPAAJ9wLMt6gvP/X8iU9afMLPh37Gf244iztWM8bDf/21MYomKAiWy8qJQlgbCfcCzk7Z8VLtl9gauZVSbqXoMLcD/Zb043a3pyEmxpg+uFUrGDpUFgIRwopIuAvAWJS7a/WuAEzfPp0G0Q04VsoFNm+GyEgYO9aYuuDYMXMLFULkiIS7SNUvpB9jm44lqm0U+xL2ETw1mOVxv8LUqTB7tjFsMjgYfvjB7FKFENmQcBepvFy8eL3+6/QO7k1Mvxh8i/rSalYr3vrlLZK6dDb64MuVg7ZtjXHxt2+bXbIQIhMS7iJDj3k+xu99f6dvcF9GrR9Fs2+bcaZUUdi0yRgyOX48PPEEHDlidqlCiAzkZIHsKKXUWaXU7ky2K6XURKXUQaVUrFIqxPJlCjM4Ozozve10ottFs+n4JoKnBrP+zBb47DPjQqc//zTekAOFAAAT7ElEQVS6aRYvNrtUIcQ9cnLk/hXQIovtLYHHUm6RwOSHL0vkJ88FPcfm5zfjVsiNRjMaMfa3seiOHY1umooVoUMHePlluHXL7FKFECmyDXet9TrgfBZN2gFfa8PvgLtSqpSlChT5Q/WS1YmJjCGiSgRDfx5K+7ntuVC6OPz2mxHsEyca67Zu3252qUIILNPnXgY4nuZ5XMprwsYULVyUuZ3m8kmLT1h+YDk1ptVg67ndMGEC9OplXM1avz4sW2Z2qUIUeJYId5XBazrDhkpFKqVilFIx8fHxFvhokdeUUrxU+yXW915PYnIi9aLqMSVmCnrcOHjjDXjkEWjd2pibJinJ7HKFKLAsEe5xQNk0z32Bkxk11FpP01qHaq1Dvb29LfDRwiy1fWuz/YXtNK7QmAE/DqDn+le5+t5bsGWLscLT6NHQrBmcPWt2qUIUSJYI9yVAr5RRM3WAS1rrUxbYr8jnPF08+bHbj4xqNIpZu2ZR4ZMK/HzqN5g+HaKiYONGYzTNhg1mlypEgZOToZCzgU1AJaVUnFKqr1Kqv1Kqf0qTZcAh4CDwBTAw16oV+Y6dsuP/GvwfkTUiSbieQPNvm/P++vdJerYX/P47uLgY0xZ89BHoDHvrhBC5QGmT/ocLDQ3VMTExpny2sLyE6wlM+mMSO07vYPH+xYSVC+ObDt/gpzygTx9YuNAYMhkdDcWKmV2uEFZLKbVVax2aXTu5QlVYhJeLF+80fIeFTy/kmw7fEHsmlsApgXx79Af0d98ZV7T+8APUqAE7d5pdrhA2T8JdWJRSih4BPdjZfycBJQPouagn3RZ150L/54xl/G7cMBbkjo42u1QhbJqEu8gVfu5+/Prsr4xuPJr5e+cTOCWQX8vcMS5yql/f6Krp29cIeyGExUm4i1xjb2fP8CeGs7HPRpwcnGg8ozFDd37E7WU/GOPgo6Kgbl04eNDsUoWwORLuItfVLFOT7S9sN+aL3ziW2tH12PdSN+NK1qNHwd/f6JNPTja7VCFshoS7yBOuhVyZ2mYqi59eTNzlOEKmhTDJ8xB6QH9jwrHXXjMW5D5wwOxShbAJEu4iT7Wr3I5dA3bR0K8hg5cPpvVjWzg95k34+GPYsQMCAowl/RITzS5VCKsm4S7ynI+bD8u6LePTlp/yy4kNVEwcz/s1rnNr13Zo2dJYjLt2bSPshRAPRMJdmEIpxeBagxlcazDXE6/zf7/8H5UXNuabdzqQ9N1cOHECQkNh+HC4edPscoWwOhLuwlTDwobxYdMPmddpHh5OHvRa3Iug+Pf44cfx6F494YMPIDAQ1q83u1QhrIqEuzCVl4sXb9R/g87VOhMTGcPcTnO5mXiTtku780TYATbMH28sxN2gAQwcCJcvm12yEFZBwl3kG3bKji7VurB34F6mtJ7CoQuHeGL3EJ4aWZnY13rA1KlQrRosXWp2qULkexLuIt9xtHfkhdAXOPjSQcY0GcNvJ38nqMhMekx+kkOlnaFNG+jWDWTBFyEyJeEu8i0XRxeGhg3l0EuHGFp/KAvj11H5qSMMHlGLM8u/g0qVjJkmjx0zu1Qh8h0Jd5HveTh78EHTDzj40kH6BvdlitpKxdcceathMhdWLIYqVWDECDmSFyINCXdhNUoXKc3kpyazb9A+nqrcllGBlygxVNGxjxurv36X5HJljZOuMleNEBLuwvo85vkYczrN4dU6r5KoNMt8LtP0WXjkP468u38aR2s+Bp06GStBCVFASbgLqzX8ieGMbTqWAy8eYHbH2Tz+eF3ebZBMhZfhyaLfM7tfXW6E14MlS2RSMlHgyDJ7wqYcvXiUGTtnEL0tiiOXj+J+y45uO5Ppc8GPkN7DUT17gpOT2WUK8cByusxejsJdKdUC+ASwB6Zrrcfcs70cMANwT2kzTGu9LKt9SriL3JSsk/n1yK9EbZvOgj3zuanvEHAa+hwsQveGL+I18D/g4WF2mULcN4uFu1LKHvgLeBKIA7YAXbXWe9O0mQZs11pPVkpVBZZprf2y2q+Eu8grF29eZPauWUSt+4SYq3/hkASVE+DlW0E803Y4bq3ag6Oj2WUKkSOWXCC7FnBQa31Ia30bmAO0u6eNBoqmPC4GnLyfYoXITe5O7gyoOZAtr+1nZ/+d1NGl2V0S+pXbgfeWLnTo68q3Q5pyae1KMKmbUghLy0m4lwGOp3kel/JaWiOAHkqpOGAZ8GJGO1JKRSqlYpRSMfEyJlmYIKBkAIsif2GMQ0sWN4uiX+mn2FLOgZ7FVuO9ugWtI12Jeuspzu3YZHapQjyUnHTLdAaaa62fT3neE6iltX4xTZshKfv6SClVF/gS8NdaZzpEQbplRH6RrJPZ/NcvLFj2EfPP/spRp5vYJ0OjhCJ0KtuM9p3eouSjgWaXKQRg2W6ZOKBsmue+/LvbpS8wD0BrvQlwArxyVqoQ5rJTdtSt1JRxry7n8PvXiYlYwRvOTTnqcpv+txZQ+psgGr7qwacTu3PixD6zyxUiR3Jy5O6AcUK1CXAC44RqN631njRtlgNztdZfKaWqAKuBMjqLncuRu8jvtNbs3rKU+cvGseDS7+xxvw1A2WsO9PJuQrcO71ClYh2UUiZXKgoSSw+FbAVMwBjmGKW1Hq2UGgnEaK2XpIyQ+QJwwzi5+obW+qes9inhLqyK1vy55jteWfQCK70upr786A0X2nnWo22TgdQPaou9nb2JRYqCwKLhnhsk3IU1Sji+n+ivh9Cialt+i1nI9wkb+KXEdW47gOdtB55yCaJdWF+a1e2JayFXs8sVNkjCXYi8oDWXd/7ByiXj+f7YKn70usBFZ3BKUjS1f5x2NbrRJjySkkV8zK5U2AgJdyFMcOfgftYv+Jgl+5fwfdFTHPEApaGG9sG7ZAXebDWWuuXrSz+9eGAS7kKYTJ84wa6Fn/P99jlMdT/EiWLG648mFqNt2Sa0adSfsIqNcLBzMLdQYVUk3IXIRxLefYMJ6/6He7GS/OIWz+ryydx2AI+kQrTyqEXbsL40r96BYk7FzC5V5HMS7kLkJwkJEB0NvXuDqytXfvqBVT9PY0n8BpaWv8U5F3BMVjR0qkybGt1oU6sHfu5+Zlct8iEJdyGsQVISSZt+Y9MPk1lyeAVLSl5kf8rlfwH40LZKO9rW60ONMqHYKVl+QUi4C2F9tIZ9+9i/eDo/xM7nB+fjbCgHyXbglmhHK8+6dGjYn2aVWlHcubjZ1QqTSLgLYe1OnuTc4lkM2jKSuX5XcL4NNwqBnVbUcqpIy8COtAzoSI3SNeSovgCRcBfCRiQc30909Mv0KtOKwzGrWB63huWlrhFTGrQCL1xpXq4RLWs8TfNHW+DlItM62TIJdyFsVXIyxMQQv3QuP22fz/JCx1hZERJcjTH1NYtUokVgR1pWbkPN0jVlSgQbI+EuREERF0fyD0vY+usslp/bzAq/RDaXMfrqnZPsCC9dlycDIwgvH06gT6CMq7dyEu5CFETXr8Pq1Zz7cT4vXZzFrCqJeF6DcynT3BSxcyasXBgNKjYmvHw4NUrXoJB9IXNrFvdFwl2IAi7h2J9Ef96P3s71uLV1M+tPbGRtqTus84O93kYbFzsn6parRwO/cMLLh1PbtzZODk6m1i2yJuEuhEjv5k3YtAlWreLsuuWsv7CDdeVg7SN2xHonoxUUsnOktm8dQkuHcvHmRYbUHYJ/CX+zKxdpSLgLIbJ2/jz88gv8/DMX1q5gQ/JR1vrBukcdifG+g06Z28zHzYcapWoQUiok9d63qK9MfmYSCXchxP05dAh+/hlWreLI+h8YW+MWpa/AX2UKsa1cIfa5XCMZIy+8XbzThX2N0jUoX6y8BH4ekHAXQjy4s2dh7FgoUwZ274aNG7n295/EloStZRTbqnqwrYxij8NFEkkCoLhzcaMLR0PHKh2p5VuLyl6VcXdyN/mHsS0S7kIIyzp/Hn7/3ei337gRNm/m5q1r7CoB2yoVYWu14izzPM8JdSXd23zcfKjsVZkqXlXS3UvXzoORcBdC5K7EROOo/p+w37SJhFN/82UwNDoMZx7x5s/K3uwr7cifrjfYl3iai3cup77d1dHVCHvvKlT2rEyZomXYeXonL9V+iQoeFUz8wfI3Sy+Q3QL4BGOB7Ola6zEZtOkCjMBYIHun1rpbVvuUcBfCBu3dC+PGQenS8PffEBsL+/dDUhIaOFu8MPtqlufPSl7s83HgT9fr7LtziuNXT6TbTXHn4lRwr0AFjwrGfcpjP3c//Nz9CvRwTYuFu1LKHvgLeBKIA7YAXbXWe9O0eQyYBzTWWl9QSpXQWp/Nar8S7kIUEDdvwr59RtDHxsLOncYtISG1ydXypdhcoyRfeZ/gkbKBnPVy5rD9ZQ7fOMWRS0e4nXQ73S5LuZVKDf6SriU5dukYnat2plqJavgW9aVo4aI22+VjyXCvC4zQWjdPef5fAK31B2najAX+0lpPz2mBEu5CFGBaw5kzdwM/NtYYqXPqVPp2zs4kP1qRU1XLcfgRDw6XduZw0WQOF7rO4ZunOHzxMMcvHUeTPsfcCrnhW9SXMkXK4FvUN8Obp7OnVf4ByGm452SSiTLA8TTP44Da97R5POVDf8PouhmhtV6RQVGRQCRAuXLlcvDRQgibpBT4+Bi3Zs2M1xISICoKmjeHc+fgr7/gwAHs/vqLMjsOUGbB34QlJt7dR9Gi8PjjnKpQjYlF91K/cjOu+pYgzjWJOLurnLh+mrjLcaw+vJqTV06SrJPTlVDYvjA+bj4opaheojrli5XHx83nX7cSriVwtHfMwy/HMnJy5N4ZaK61fj7leU+gltb6xTRtlgJ3gC6AL7Ae8NdaX8xsv3LkLoS4L4mJcPRoauhz4IDxeMsWuHAhfVuljH7/8uXBz4/E8mU5U9aDEyWciSsGcU53iLtxhlV/r2LHmR2UcCnB7eTbXLyZcWR5uXhRyq1UauAXLVyUIxeP0PLRllQsXhFvF2+8Xb3xdvHG2dE5V78GSx65xwFl0zz3BU5m0OZ3rfUd4LBSaj/wGEb/vBBCPDwHB6hY0bi1bHn39YQE+OILePJJuHLF+ANw5Mjd+02bcJg3jzKJiZQBav3zvhIlSPArQbRvSXp7PYnXI/7cfKQEZ7ydOV3MgdMuyZy+c4HTV08bt2vG/YFjBzh+6ThJOokfD/z4rzLdCrmlhn0J1xLG4zTPC9kVYueZnbxW77VcnXs/J0fuDhgnVJsAJzACu5vWek+aNi0wTrI+q5TyArYDQVrrc5ntV47chRB5JikJTp78d/CvXWv8C6BwYbh169/vc3cHX1/jYq40t/iSbky5uYHmVduSVKwIZ2+cI/56PPHX4jl77azx+HrK42vG43tPCo9tOpbX679+3z+KxY7ctdaJSqnBwEqM/vQorfUepdRIIEZrvSRlWzOl1F4gCXg9q2AXQog8ZW8PZcsat7Cwu68nJEB0NPTubQT8iRPpb3Fxdx/HxsLp06A13sBbAEwBOzvw9jbOH5QqlXIuoSz41DQeV/ZBlyzJZU9X4u1ucuD8QX49+iu9g3vn6o8sFzEJIUROJSYaAT9mDEyaBO3bQ/XqxmunTxujff55nPbk7z+cnY0/BAEBxh8Vr/vvlrFkn7sQQggw+v19fWHECONkbe/eGQd0crJxkvfewD992piJc+lSI9xfv/9umRyXmmt7FkIIW+XllXUw29mBp6dxq1Yt/ba0XUG5SMJdCCHyUnZ/GCzELtc/QQghRJ6TcBdCCBsk4S6EEDZIwl0IIWyQhLsQQtggCXchhLBBEu5CCGGDTJt+QCkVDxx9wLd7AQnZtirY5DvKmnw/2ZPvKGtmfT/ltdbe2TUyLdwfhlIqJidzKxRk8h1lTb6f7Ml3lLX8/v1It4wQQtggCXchhLBB1hru08wuwArId5Q1+X6yJ99R1vL192OVfe5CCCGyZq1H7kIIIbJgdeGulGqhlNqvlDqolBpmdj35kVLqiFJql1Jqh1KqwC93pZSKUkqdVUrtTvNacaXUKqXUgZR7DzNrNFsm39EIpdSJlN+jHUqpVmbWaCalVFml1Bql1D6l1B6l1Mspr+fb3yOrCnellD0wCWgJVAW6KqWqmltVvtVIax2Un4dq5aGvgBb3vDYMWK21fgxYnfK8IPuKf39HAB+n/B4Faa2X5XFN+Uki8JrWugpQBxiUkj359vfIqsIdqAUc1Fof0lrfBuYA7UyuSeRzWut1wPl7Xm4HzEh5PANon6dF5TOZfEcihdb6lNZ6W8rjK8A+oAz5+PfI2sK9DHA8zfO4lNdEehr4SSm1VSkVaXYx+VRJrfUpMP7HBUqYXE9+NVgpFZvSbZNvuhzMpJTyA4KBzeTj3yNrC3eVwWsy3Off6mutQzC6rwYppRqYXZCwSpOBikAQcAr4yNxyzKeUcgMWAK9orS+bXU9WrC3c44CyaZ77AidNqiXf0lqfTLk/CyzC6M4S6Z1RSpUCSLk/a3I9+Y7W+ozWOklrnQx8QQH/PVJKOWIE+0yt9cKUl/Pt75G1hfsW4DGlVAWlVCHgGWCJyTXlK0opV6VUkX8eA82A3Vm/q0BaAjyb8vhZ4HsTa8mX/gmtFB0owL9HSikFfAns01qPT7Mp3/4eWd1FTCnDsSYA9kCU1nq0ySXlK0qpRzCO1gEcgFkF/TtSSs0GGmLM4ncGeAdYDMwDygHHgM5a6wJ7QjGT76ghRpeMBo4AL/zTv1zQKKXCgPXALiA55eXhGP3u+fL3yOrCXQghRPasrVtGCCFEDki4CyGEDZJwF0IIGyThLoQQNkjCXQghbJCEuxBC2CAJdyGEsEES7kIIYYP+H58jx0XMci2PAAAAAElFTkSuQmCC\n", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "nn.fit_epoch(X_train,y_train,X_test,y_test,lr=0.01,epochs=20,bs=512,beta1=0.9)" ] }, { "cell_type": "code", - "execution_count": 64, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Epoch 1. Training loss: 0.5506057780259906, Val loss:0.5691002192042336\n", - "Epoch 2. Training loss: 0.5425068147147296, Val loss:0.5620410337745381\n", - "Epoch 3. Training loss: 0.5350577253330229, Val loss:0.5555534154743696\n", - "Epoch 4. Training loss: 0.5281538936023978, Val loss:0.5495226085192954\n", - "Epoch 5. Training loss: 0.5217589673422273, Val loss:0.543940726784795\n", - "Epoch 6. Training loss: 0.515789630803349, Val loss:0.5386916415584343\n", - "Epoch 7. Training loss: 0.5101821903238222, Val loss:0.5337678287554231\n", - "Epoch 8. Training loss: 0.50488829727666, Val loss:0.5290617814767321\n", - "Epoch 9. Training loss: 0.4998636617167306, Val loss:0.5246166682357379\n", - "Epoch 10. Training loss: 0.4950760662835879, Val loss:0.5204086443783965\n", - "Epoch 11. Training loss: 0.4905166142281695, Val loss:0.5164064645942483\n", - "Epoch 12. Training loss: 0.4861761918316699, Val loss:0.5126758395938752\n", - "Epoch 13. Training loss: 0.4820594975451327, Val loss:0.5091832719621227\n", - "Epoch 14. Training loss: 0.47814024610846745, Val loss:0.5058423394634673\n", - "Epoch 15. Training loss: 0.474382979546902, Val loss:0.5026866048735866\n", - "Epoch 16. Training loss: 0.4708047604350005, Val loss:0.49968995149563844\n", - "Epoch 17. Training loss: 0.4674129409333824, Val loss:0.49683660335813257\n", - "Epoch 18. Training loss: 0.4641792811470584, Val loss:0.4941960360773917\n", - "Epoch 19. Training loss: 0.46109905036848825, Val loss:0.491662559889695\n", - "Epoch 20. Training loss: 0.458145948846935, Val loss:0.4892692128549704\n" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xd4FWXax/HvnU56SAKJCRCatBBCDEVRqSIgglJUFBFWZXGxwa4ra1kROyIiNmSRInCBCCi8iGJjRVcFQ5UivYWWRgIEAoQ87x8TQoCUA5wwOcn9ua65Ts6cOTN3RvnN5JlnnhFjDEoppSoWN7sLUEop5Xwa7kopVQFpuCulVAWk4a6UUhWQhrtSSlVAGu5KKVUBabgrpVQFpOGulFIVkIa7UkpVQB52bTgsLMzExMTYtXmllHJJK1euTDPGhJe2nG3hHhMTQ1JSkl2bV0oplyQiux1ZTptllFKqAtJwV0qpCqjUcBeRGiKyVEQ2icgGEXmiiGVERMaLyDYRWSciCWVTrlJKKUc40uaeC/zdGLNKRAKAlSLyrTFmY6FlugL186dWwIf5r0qpcuL06dMkJyeTk5NjdynKAT4+PkRHR+Pp6XlZ3y813I0xB4AD+T8fFZFNQBRQONx7Ap8Ya3D430QkWEQi87+rlCoHkpOTCQgIICYmBhGxuxxVAmMM6enpJCcnU7t27ctaxyW1uYtIDNAcWH7BR1HA3kLvk/PnXfj9wSKSJCJJqampl1apUuqK5OTkEBoaqsHuAkSE0NDQK/ory+FwFxF/YB7wpDHmyIUfF/GVix7xZIyZaIxJNMYkhoeX2k1TKeVkGuyu40r/WzkU7iLiiRXsM40x84tYJBmoUeh9NLD/iiorzk8/Qbt2sHNnmaxeKaUqAkd6ywjwMbDJGDO2mMUWAgPye820BrLKrL198mT48UcYM6ZMVq+UKhvp6enEx8cTHx9PREQEUVFRBe9PnTrl0DoGDRrE5s2bS1zm/fffZ+bMmc4omRtvvJE1a9Y4ZV1XmyO9ZdoA9wN/iMjZ3/IZoCaAMWYCsBjoBmwDjgODnF9qvsceg6lToU6dMtuEUsr5QkNDC4Jy5MiR+Pv7849//OO8ZYwxGGNwcyv6vHPKlCmlbmfo0KFXXmwFUOqZuzHmZ2OMGGPijDHx+dNiY8yE/GDHWIYaY+oaY5oaY8puXIHmzSEgQJtllKogtm3bRmxsLEOGDCEhIYEDBw4wePBgEhMTadKkCaNGjSpY9uyZdG5uLsHBwYwYMYJmzZpx/fXXk5KSAsBzzz3HuHHjCpYfMWIELVu2pEGDBvzyyy8AZGdn07t3b5o1a0a/fv1ITEws9Qx9xowZNG3alNjYWJ555hkAcnNzuf/++wvmjx8/HoC3336bxo0b06xZM/r37+/0feYI28aWuWwiEBsLf/xhdyVKua4nnwRHmhtOn4aDByEiAkrrbx0fD/mheqk2btzIlClTmDBhAgCvv/46VatWJTc3l/bt29OnTx8aN2583neysrJo27Ytr7/+OsOHD2fy5MmMGDHionUbY1ixYgULFy5k1KhRfP3117z77rtEREQwb9481q5dS0JCyfddJicn89xzz5GUlERQUBCdOnVi0aJFhIeHk5aWxh/5eZSZmQnA6NGj2b17N15eXgXzrjbXHH6gaVMr3M1FHXKUUs508CDs2GG9lqG6devSokWLgvezZs0iISGBhIQENm3axMaNGy/6TpUqVejatSsA1113Hbt27Spy3b169bpomZ9//pl77rkHgGbNmtGkSZMS61u+fDkdOnQgLCwMT09P7r33XpYtW0a9evXYvHkzTzzxBEuWLCEoKAiAJk2a0L9/f2bOnHnZNyFdKdc7cwcr3CdOhP37Ieqi7vRKqdI4eoadlgZTpsCgQRAWVmbl+Pn5Ffy8detW3nnnHVasWEFwcDD9+/cvsr+3l5dXwc/u7u7k5uYWuW5vb++LljGXeGJY3PKhoaGsW7eOr776ivHjxzNv3jwmTpzIkiVL+PHHH1mwYAEvv/wy69evx93d/ZK2eaVc98wdtGlGqbIWFgZPPVWmwX6hI0eOEBAQQGBgIAcOHGDJkiVO38aNN97InDlzAPjjjz+K/MugsNatW7N06VLS09PJzc1l9uzZtG3bltTUVIwx9O3blxdffJFVq1Zx5swZkpOT6dChA2+++SapqakcP37c6b9DaVz3zB2scO/Sxd5alFJOlZCQQOPGjYmNjaVOnTq0adPG6dt47LHHGDBgAHFxcSQkJBAbG1vQpFKU6OhoRo0aRbt27TDGcPvtt3PbbbexatUqHnzwQYwxiAhvvPEGubm53HvvvRw9epS8vDyefvppAgICnP47lEYu9c8TZ0lMTDRX9LCOqCjo2BE++cR5RSlVgW3atIlGjRrZXUa5kJubS25uLj4+PmzdupXOnTuzdetWPDzK1/luUf/NRGSlMSaxtO+Wr9/kUsTFabOMUuqyHDt2jI4dO5Kbm4sxho8++qjcBfuVct3fpmlTWLoUcnOhgv1HUUqVreDgYFauXGl3GWXKNS+oghXuJ0/C1q12V6KUUuWOa4c7aNOMUkoVwXXDvVEjcHeHdevsrkQppcod1w13b2+49lo9c1dKqSK4brjDuWEIlFLlXrt27S66IWncuHH87W9/K/F7/v7+AOzfv58+ffoUu+7SulaPGzfuvJuJunXr5pRxX0aOHMmYcjgEueuH+86dcPSo3ZUopUrRr18/Zs+efd682bNn069fP4e+f8011zB37tzL3v6F4b548WKCg4Mve33lneuHO8CGDfbWoZQqVZ8+fVi0aBEnT54EYNeuXezfv58bb7yxoN95QkICTZs2ZcGCBRd9f9euXcTGxgJw4sQJ7rnnHuLi4rj77rs5ceJEwXKPPPJIwXDBL7zwAgDjx49n//79tG/fnvbt2wMQExNDWloaAGPHjiU2NpbY2NiC4YJ37dpFo0aNePjhh2nSpAmdO3c+bztFWbNmDa1btyYuLo4777yTw4cPF2y/cePGxMXFFQxY9uOPPxY8rKR58+YcdfJJqmt3EC/cY6Z1a3trUcqFPPn1k6w5WPqQv6fPnObgsYNE+Efg6V7y6IbxEfGM61L8gGShoaG0bNmSr7/+mp49ezJ79mzuvvtuRAQfHx8+//xzAgMDSUtLo3Xr1vTo0aPY54h++OGH+Pr6sm7dOtatW3fekL2vvPIKVatW5cyZM3Ts2JF169bx+OOPM3bsWJYuXUrYBePkrFy5kilTprB8+XKMMbRq1Yq2bdsSEhLC1q1bmTVrFv/5z3+46667mDdvXonjsw8YMIB3332Xtm3b8u9//5sXX3yRcePG8frrr7Nz5068vb0LmoLGjBnD+++/T5s2bTh27Bg+Pj4l7t9L5dpn7jEx4Oen7e5KlZGDxw6yI3MHB485Z8jfwk0zhZtkjDE888wzxMXF0alTJ/bt28ehQ4eKXc+yZcsKQjYuLo64uLiCz+bMmUNCQgLNmzdnw4YNpQ4K9vPPP3PnnXfi5+eHv78/vXr14qeffgKgdu3axMfHAyUPKwzW+PKZmZm0bdsWgAceeIBly5YV1HjfffcxY8aMgjth27Rpw/Dhwxk/fjyZmZlOv0O21LWJyGSgO5BijIkt4vMgYAbWY/c8gDHGmNKfheUMbm764A6lLkNJZ9iFpR1PY8rqKQxqPogw3ysfGfKOO+5g+PDhrFq1ihMnThSccc+cOZPU1FRWrlyJp6cnMTExRQ7zW1hRZ/U7d+5kzJgx/P7774SEhDBw4MBS11PS+FpnhwsGa8jg0pplivPll1+ybNkyFi5cyEsvvcSGDRsYMWIEt912G4sXL6Z169Z89913NGzY8LLWXxRHztynAiUNvTgU2GiMaQa0A94SEa8SlncufXCHUmUmzDeMp9o85ZRgB6vnS7t27fjLX/5y3oXUrKwsqlWrhqenJ0uXLmX37t0lrufmm28ueAj2+vXrWZd/v8uRI0fw8/MjKCiIQ4cO8dVXXxV8JyAgoMh27ZtvvpkvvviC48ePk52dzeeff85NN910yb9bUFAQISEhBWf906dPp23btuTl5bF3717at2/P6NGjyczM5NixY2zfvp2mTZvy9NNPk5iYyJ9//nnJ2yxJqWfuxphlIhJT0iJAgFiHUX8gAyh61Pyy0LQpTJpkPSkmMvKqbVYpdXn69etHr169zus5c99993H77beTmJhIfHx8qWewjzzyCIMGDSIuLo74+HhatmwJWE9Vat68OU2aNLlouODBgwfTtWtXIiMjWbp0acH8hIQEBg4cWLCOhx56iObNm5fYBFOcadOmMWTIEI4fP06dOnWYMmUKZ86coX///mRlZWGMYdiwYQQHB/P888+zdOlS3N3dady4ccFTpZzFoSF/88N9UTHNMgHAQqAhEADcbYz5spj1DAYGA9SsWfO60o7ODvnvf6F9e1iyBDp3vvL1KVVB6ZC/rudKhvx1xgXVW4E1wDVAPPCeiAQWtaAxZqIxJtEYkxgeHu6ETaNjzCilVBGcEe6DgPnGsg3YiXUWf3WEhlrNMRruSilVwBnhvgfoCCAi1YEGwA4nrNdxOgyBUg6x68lr6tJd6X+rUsNdRGYBvwINRCRZRB4UkSEiMiR/kZeAG0TkD+B74GljTNoVVXWpmja17lIt5unnSinw8fEhPT1dA94FGGNIT0+/ohubHOktU+LAD8aY/cBVu5K57tA6nv3+WSb1mER1/+rWzLMP7ti2DZzYT1SpiiQ6Oprk5GRSU1PtLkU5wMfHh+jo6Mv+vssNP/DmL2+yaOsiXvjvC0zoPsGaWfiiqoa7UkXy9PSkdu3adpehrhKXG37glQ6v4OnmyancU+dmNmpk3a2q7e5KKQW4YLjXDKpJz4Y9+Wr7V5zJO2PNrFIF6tfXcFdKqXwuF+4AfRr14eCxg/xv7//OzdQeM0opVcAlw/22a2/Dx8OHuRsLDdzftCns2AHZ2fYVppRS5YRLhru/lz9d63Vl3qZ55Jk8a2bTptbgYfrgDqWUcs1wB+jTuA/7j+7n172/WjN0GAKllCrgsuHe/drueLt7n2uaqVMHfH013JVSChcO90DvQG6tdytzN821mmbc3KBJEw13pZTChcMdrF4zyUeSWbFvhTUjLk7DXSmlcPFw79GgB55unueaZpo2hdRUKOHZi0opVRm4dLgH+QTRuW5n5m6caw2GpBdVlVIKcPFwB+jbuC+7s3aTtD/pXLi/8w6kXd2BKZVSqjxx+XDv0aAHHm4efLbxMwgPh6AgWLQIpkyxuzSllLKNy4d7SJUQOtXpdK5p5t57rZ4zPXrYXZpSStnG5cMdrKaZnZk7WX1wNTzyCOTlwTff2F2WUkrZpkKEe88GPXEXdz7b8JnV7p6QAFOn2l2WUkrZxpHH7E0WkRQRWV/CMu1EZI2IbBCRH51bYulCfUPpULsDczflN80MHAirVmmvGaVUpeXImftUoEtxH4pIMPAB0MMY0wTo65zSLk3fxn3ZlrGNtYfWQr9+4OkJ06bZUYpSStmu1HA3xiwDMkpY5F5gvjFmT/7yKU6q7ZLc0fAO3MTNuqEpLAy6d4cZM+D0aTvKUUopWzmjzf1aIERE/isiK0VkQHELishgEUkSkSRnP6Q33C+cdjHt+GzjZ+eaZg4dgiVLnLodpZRyBc4Idw/gOuA24FbgeRG5tqgFjTETjTGJxpjE8PBwJ2z6fH0b92VL+hbWp6yHrl2tfu/aNKOUqoScEe7JwNfGmGxjTBqwDGjmhPVesjsb3gnAsCXDSDudBf37w8KFkJ5uRzlKKWUbZ4T7AuAmEfEQEV+gFbDJCeu9ZNX9q1MvpB7f7/yeyasnwwMPwKlTMHu2HeUopZRtHOkKOQv4FWggIski8qCIDBGRIQDGmE3A18A6YAUwyRhTbLfJsjb8+uEANAhtAM2aQXy8Ns0opSodR3rL9DPGRBpjPI0x0caYj40xE4wxEwot86YxprExJtYYM65sSy7ZgGYDCPAKYMHmBdaMgQPh99/12apKqUqlQtyhWpiflx99G/dlzoY5HDt1zBprxsNDz96VUpVKhQt3gIHxA8k+nc38TfOtHjO33QbTp0Nurt2lKaXUVVEhw/3GmjdSJ6QO09bmn60PHAgHD8K339pal1JKXS0VMtxFhAeaPcAPO39gd+Zu6NbNumtVBxNTSlUSFTLcwbqwCvDJ2k/Ay8tqe//iCzh82ObKlFKq7FXYcI8JjqFdTDumrZ12bjgC7fOulKokKmy4AwxsNpDth7fzv73/s/q7N24Mr72mz1dVSlV4FTrcezfujZ+nH9PWTAMRqFMH9u61Al4ppSqwCh3u/l7+9Gnch083fMrx08dhzBjr+arZ2XaXppRSZapChztYfd6PnjrK55s+hwYNoE8f+OwzyMmxuzSllCozFT7cb651M7WCap3r8/7ww5CRYfWcUUqpCqrCh7ubuPFAswf4bsd37M3aCx06QO3a8J//2F2aUkqVmQof7mD1eTcYZqybYbW5P/gg/PADbN9ud2lKKVUmKkW4161al5tq3sTUtVOtPu+DBlkh//HHdpemlFJlolKEO1gXVrekb+G35N/gmmuswcSmTNHBxJRSFVKlCfc+jftQxaPK+RdWDx6EL7+0tzCllCoDlSbcA70Due3a25i2Zpo1mFjXrtYZ/KRJdpemlFJO58hj9iaLSIqIlPjoPBFpISJnRKSP88pzrmq+1cg5k8OTXz9pPcBj0CBYvBiSk+0uTSmlnMqRM/epQJeSFhARd+ANYIkTaiozI9uNpEZgDdYeWsuZvDPwl79AXp4OBayUqnAceYbqMiCjlMUeA+YBKc4oqqyE+4UzpvMYdmbuZNGWRdZYM506Wb1m8vLsLk8ppZzmitvcRSQKuBOY4MCyg0UkSUSSUlNTr3TTl6VXo17UDKrJ2N/GWjMeegh27YLvvrOlHqWUKgvOuKA6DnjaGHOmtAWNMRONMYnGmMTw8HAnbPrSebh58ESrJ1i2exlJ+5PgjjsgNFQvrCqlKhRnhHsiMFtEdgF9gA9E5A4nrLfMPJTwEAFeAYz9dSx4e8OAAdZYMzb9NaGUUs52xeFujKltjIkxxsQAc4G/GWPK9ahcgd6BPJzwMHM2zLHGm3n4YTh9Gj75xO7SlFLKKRzpCjkL+BVoICLJIvKgiAwRkSFlX17ZebzV4wC8u+JdaNQI2rSxBhMzxubKlFLqyjnSW6afMSbSGONpjIk2xnxsjJlgjLnoAqoxZqAxZm7ZlOpctYJr0adxHyaunMjRk0etC6ubN8PQofoYPqWUy6s0d6gWZfj1w8k6mcXk1ZOhb1+oUgU+/NAac0YppVxYpQ73llEtaVOjDeOWj+NMFR946inrgyZN7C1MKaWuUKUOd7DO3ndl7uLzPz+Hf/4Tqle3nrWqlFIurNKHe88GPakTUsfqFunnB888A0uXWg/zUEopF1Xpw93dzZ0nWz3Jr8m/8uveX2HwYIiOhuee054zSimXVenDHWBQ80EE+wTz9m9vg48PPP88/PorfPWV3aUppdRl0XAH/L38+et1f2Xuxrn867t/kXb37dagYnr2rpRyURru+R5t+Sgiwuv/e50p62fACy/A6tXw+ed2l6aUUpdMwz1fdGA0j7e07lq9NvRauO8+aNjQaqI5U+qYaEopVa5ouBfySsdXqBVUi2d/eJbT5MGoUbBxI8yebXdpSil1STTcC/H19GV81/FsSN3AO8vfgd69oVkzGDnSGlhMKaVchIb7BXo06MHt197OyP+OJPnYfnjpJdi2TUeMVEq5FA33IrzT5R3OmDMMWzIMuneHli2tJpqTJ+0uTSmlHKLhXoTaIbV59qZnmbtxLt/s+BZefhn27NGnNSmlXIaGezGeuuEp6letz9DFQ8lpdyNcfz08/bQ1LLBSSpVzGu7F8Pbw5v1u77MtYxtv/jIGWreG7Gy45x67S1NKqVJpuJfglrq3cFeTu3j151fZ+eh9cMstsGYNLFhgd2lKKVUiRx6zN1lEUkRkfTGf3yci6/KnX0SkmfPLtM/YzmPxcPPgsd9ewCxaZHWNHDIEMjLsLk0ppYrlyJn7VKBLCZ/vBNoaY+KAl4CJTqir3IgKjGJk25F8ufVLFu74CqZOtR7D9+STdpemlFLFcuQZqsuAYk9TjTG/GGMO57/9DYh2Um3lxuOtHqdhWEMe+OIBtkT7wr/+BdOnw5df2l2aUkoVydlt7g8CxY6TKyKDRSRJRJJSU1OdvOmy4+nuSec6nck6mcUdn96BefZZiI21xn7PzLS7PKWUuojTwl1E2mOF+9PFLWOMmWiMSTTGJIaHhztr01fF822fp2eDnmxK28TYle9ZD9E+dAiGD7e7NKWUuohTwl1E4oBJQE9jTLoz1lnehPmG8fndn9OrUS9GfD+C3yJyrWeuTpkCX39td3lKKXWeKw53EakJzAfuN8ZsufKSyi8R4eMeH1MjsAb3zL2HjKcehUaN4OGHISvL7vKUUqqAI10hZwG/Ag1EJFlEHhSRISIyJH+RfwOhwAciskZEksqwXtsF+wTzaZ9P2X90P4OWPIKZPBn274ennrK7NKWUKuBR2gLGmH6lfP4Q8JDTKnIBLaJaMPqW0QxbMox3Ytrz5PDhMGaM9VCPN96AsDC7S1RKVXJ6h+pleqLVE/Rs0JN/fvtPfn+kB0REwOTJMHas3aUppZSG++USESb3nExkQCR3/d8AMj+bDt7e8M03cOqU3eUppSo5DfcrULVKVT7t8ynJR5J5cP+HmE8+gZUrYcQIu0tTSlVyGu5XqHV0a17v+DrzN82nR+500h5/CN5+G+bPt7s0pVQlpuHuBMOvH06T8CYs2rqI5zuK9eSmQYNg+3a7S1NKVVIa7k4gIvxfv/8j3DecuZs/J3nKO+DuDn37Qk6O3eUppSohDXcnqR1Smx8H/khObg59fnmSk1M/htWrdfRIpZQtNNydqFF4I6bdMY3l+5bzhNsS67F8H30EM2faXZpSqpLRcHeyXo16MaLNCD5a+REf96kDN90Ef/0rbNpkd2lKqUpEw70MvNzhZTrV6cTQrx/n93dHQJUq0L497Nljd2lKqUpCw70MuLu5M6v3LCL8I+j9wxBSe91qDQ981112l6aUqiQ03MtImG8Y8++eT+rxVO5psYfc9m1h+XKYM8fu0pRSlYCGexlKiExgwm0T+GHfT3S635B2c6I1PPCOHXaXppSq4DTcy9gD8Q/QKqoVP+5Zxr/+Whfc3KBfPx1/RilVpjTcr4I5fedQza8aC/cvJXXCW7BiBTz3nN1lKaUqMA33q6BmUE2+vf9bsnKy+Eve55hHhsCbb8JXxT5LXCmlroiG+1USVz2O0beMZtGWRXzQvwHExcGAAdZTnJRSyskceczeZBFJEZH1xXwuIjJeRLaJyDoRSXB+mRXDYy0fo2u9rvz9hxGs/+glOH4c+ve3nuCklFJO5MiZ+1SgSwmfdwXq50+DgQ+vvKyKSUSYesdUgn2C6bf6WU6MHwtLl8Jrr9ldmlKqgik13I0xy4CMEhbpCXxiLL8BwSIS6awCK5pqftWYesdU1qes55+Rf0Dv3vDvf8OXX9pdmlKqAnFGm3sUsLfQ++T8eRcRkcEikiQiSampqU7YtGvqUq8Lw1oP473f32dRnA8YYzXPZGXZXZpSqoJwRrhLEfNMUQsaYyYaYxKNMYnh4eFO2LTreq3jazSr3oxB3l9z4LFBcOQIPPKIFfRKKXWFnBHuyUCNQu+jAe0CUgpvD29m9Z7FsdPZtK3/Mykv/ANmzYIZM+wuTSlVATgj3BcCA/J7zbQGsowxB5yw3gqvUXgjul3bja0ZW7m/9mpreOChQ3V4AqXUFXOkK+Qs4FeggYgki8iDIjJERIbkL7IY2AFsA/4D/K3Mqq2AJtw2geYRzflmx7cseeNha3iC++6D06ftLk0p5cLE2NTGm5iYaJKSkmzZdnmTfSqb1h+3Zv/R/ayKGkWt/o/C88/DqFF2l6aUKmdEZKUxJrG05fQO1XLAz8uP+XfNJzcvlz7Hp5IzqD+88gr89JPdpSmlXJSGezlRP7Q+0+6YRtL+JJ7o7gm1a1vdIzMz7S5NKeWCNNzLkTsa3sHTbZ5m4h9TmDr6XmvcmSFDtHukUuqSabiXMy93eJn2Me15ZNObrBk5BD791Ho8X1qa3aUppVyIhns54+Hmwew+swmtEkov/y853KQOzJ1rXWBVSikHabiXQ9X8qvFZ389IPpLM/U/WIi88DGbPhq1b7S5NKeUiNNzLqetrXM/YW8fy5b6ldHixDql+QPfukFHSGG5KKWXRcC/HhrYYSuuo1vyYsoLeT9XC7NppjSKpz19VSpVCw70cExEW9ltImxpt+ClzLY++0Za8H/+rA4wppUql4V7OhfuF89Ogn3jqhqf4IOs7/vrv5pyZMhlGj7a7NKVUOeZhdwGqdCLCG53ewMfDh5eWvcTJJ2OY/MwIPOrXh1697C5PKVUOabi7CBFhVPtReLt789zS58gZHMrMAffhWfMnSCx1mAmlVCWjzTIu5tmbn2XMLWP4LCKdvncJJzu2ha+/trsspVQ5o+Hugv5+w995r+t7LKh1gqYDjvPnA91g5ky7y1JKlSMa7i5qaMuh9K3bg61h0GKwsPjF/vDMM5CXZ3dpSqlyQMPdhX3Q62OGtR5GdPX63HYf/G3da2T37QnHjtldmlLKZhruLizMN4yxt45l9ZA1DG89jA9bQMI1i1jRvTns3m13eUopGzkU7iLSRUQ2i8g2ERlRxOc1RWSpiKwWkXUi0s35pari+Hj48NatY/l+wPecuCacG9puY9TfmpD78zK7S1NK2cSRZ6i6A+8DXYHGQD8RaXzBYs8Bc4wxzYF7gA+cXagqXYfaHVg3bAv31L6dF1pm02pGW/7xSF3Sdm20uzSl1FXmyJl7S2CbMWaHMeYUMBvoecEyBgjM/zkI2O+8EtWlCPYJZsbAhcy+9T9sChfeithB79cTOLJmud2lKaWuIkfCPQrYW+h9cv68wkYC/UUkGVgMPFbUikRksIgkiUhSamrqZZSrHHV364dYdf/PtDxRlWURJ2kwszXTX+yNOXnS7tKUUleBI+HzpZpXAAATm0lEQVQuRcy7cNSqfsBUY0w00A2YLiIXrdsYM9EYk2iMSQwPD7/0atUlaXjtDSx/PZ0VfZZQ0z2UAcznpn9UZc332ideqYrOkXBPBmoUeh/Nxc0uDwJzAIwxvwI+QJgzClRXrkVsZ359LYWPaz3OZr8crlvWn6HPxpORnmx3aUqpMuJIuP8O1BeR2iLihXXBdOEFy+wBOgKISCOscNd2l3LETdz4y8B32DJsB0NPxPKhx1qi367Bq+N6czpXx4dXqqIpNdyNMbnAo8ASYBNWr5gNIjJKRHrkL/Z34GERWQvMAgYaowOOl0ch1WsxfvQfPHkijhOe8GzWfGKfDWHhN++i/8mUqjjErn/QiYmJJikpyZZtK0jbu5nJ04YRLUG8lDKHP6vm0f50NG/dP4PmDdraXZ5SqhgistIYU+pQsBruitMpB5n4xl284PETGVVgoO8NvDx4NtcE1yj9y0qpq8rRcNfhBxSe1SIY+tYytvVZxt/3RjPz6C/Ue6smt/wrml07VtldnlLqMmi4qwLBLW7izUl72NRgPPUOu/Gdzz7iJrfg469f40zeGbvLU0pdAg13dT4R6vR/jB+eWMXQAzVomGp4aPkzxI+szpK18+yuTinlIA13VaSw+s14b8Ielr+QzNzUDpzISqfLF33o/Goj1u5baXd5SqlSaLirEsk119D7ve/ZOGA547bWY2XWn8T/J5Hrngkjaf23dpenlCqGhrtyiNd1LXli+ha2tZjOzfs8WOWdTqvPOvPIlN7sO7LP7vKUUhfQcFeOEyGkT3/m/Ws1zx2ozwObvPh453zqvlWT4Z/+hZTsFLsrVErl03BXlyysTiwvTdjC5EmpbOEx7l0vvLNxCnXejObZ/3uSwycO212iUpWehru6fIGBxIwaz+T39rIx9S5u35DLq6veIfK1MIbOGsCerD12V6hUpaXhrq5cZCQNPviUWS9tZNiWqpx0z+PDP6dTZ1wM/Wb1ZsW+FXZXqFSlo+GunKdhQ555/RdGH2/Dqj9uYNj/DIv/mE+rSa24cWIr5m+arzdDKXWV6NgyquysW8fRV19g8u4vGNda2BVsqOkXRZPIOF7t+CrxEfF2V6iUy9GxZZT94uIImP05T3y8gW2p9zB3juCevI+vtn1F84+a025qOyatmkRmTqbdlSpV4eiZu7p6tm4lbdDdjPFZjQDzm/uwxT8HL3cvul/bnf5N+9Otfje8PbztrlSpckuH/FXlU1oavP8+eHpipn/CyiObmdHCm1nN3ElxO463uze31LmFOxvdSZd6Xbgm4Bq7K1aqXNFwV+WfMfDjjzBhArmfz+OvXXKZnACBeHOEkwDER8TTrV43utXvRqvoVni4edhctFL2cmq4i0gX4B3AHZhkjHm9iGXuAkYCBlhrjLm3pHVquKvzHDpE2kP3MiXjBwaugQMBsLhzDIsbuPPLmV2cMWcI8g4iKjCKvo36cmu9W7numuvwcveyu3KlriqnhbuIuANbgFuAZKwHZvczxmwstEx9YA7QwRhzWESqGWNKvBddw11dJC0NpkyB66+H77+HuXNh/XoyfeDbrtfyZrNj/M7+gsV9PHxoGdWSm2rexI01b6RVVCtCqoTY+AsoVfacGe7XAyONMbfmv/8XgDHmtULLjAa2GGMmOVqghrtyyJ9/wrx5MHcuaVvWMCUebjvgx59dW/JzfW9+9jrIqrQ/OGOs/vO1gmrRMqolCZEJXBd5HQmRCYT6htr8SyjlPM4M9z5AF2PMQ/nv7wdaGWMeLbTMF1hn922wmm5GGmO+LmJdg4HBADVr1rxu9+7djv9GSq1cCa++Crm58PPPkJEBQHazxgzvbJjot4mmQdeS7ZbLjsM7Cr4WHRBNcJVgutbrSsuolsRWi6Ve1Xrafq9ckqPh7sj/3VLEvAuPCB5AfaAdEA38JCKxxpjzOjAbYyYCE8E6c3dg20qdc9111lk8QF4erFkD336L33ff8cqHP1AvDgat2ULYNfU43OYuVjWPZFWUMDP1B9amrGNDygZM/v+6Xu5eNAxrSGy1WGoH1Wbv0b38NeGvJEYlaju+qhCc1SwzAfjNGDM1//33wAhjzO/FrVebZZRT7d0Lr70G4eGwbh38+iscOgRAWpgvU6734Z5a3Um7riHrI9xZ75bG+rQNrE9Zf94AZ27iRu3g2jQIa0CD0AZEBUSxJX0Lj7R4hKbVmuLu5m7Xb6gU4NxmGQ+sJpeOwD6sC6r3GmM2FFqmC9ZF1gdEJAxYDcQbY9KLW6+GuypTxsDOnVbIv/++9eruDmfyx7bx94dmzSAhgZ21Q3g7ZQGN2vdlv0cOWzK2sDltM1vSt3Ai90TBKr3cvYgJjqFuSF1rqlqXcN9wVh5YyeMtHycmJMae31VVKs7uCtkNGIfVnj7ZGPOKiIwCkowxC0VEgLeALsAZ4BVjzOyS1qnhrq6as71w7rsPUlNh1SpYvfrclJ1tLScC9epBbCzExpLXpDF/1PDi/dTFNKzWmJTsFLYf3s72jO1sP7ydIyePnLeZqlWqUiuoFjHBMdQKqkWYbxhb0rcwoNkAmlZvSrhvONY/FaUun97EpJQj8vJgxQr44AOIiIAdO2D9eti61foMwM0N6ta1Qr9BA2jYENOgARm1qrHy+DY+WfsJdUPqknY8jV1Zu9iVaU3HTx8/b1Pe7t5EB0ZTI6gGNQJrEFollN1Zu+nVqBeNwhoRFRhFNb9quIkO+aSKp+Gu1JXIybG6Yb7xBsyeDU2aWE0627ZZvXXOqlYN/PygdWto2tQ6869XD1O3LptP7ufDpA9JiEwg62QWe7P2svdI/pS1l+QjyQUXeM/ycPMg0j+San7VOHnmJC2vaUmdkDpEBkQS6R9Z8BruF64HgUpKw10pZzjbpDNoEISFwenTVlv+n39a02efQVISBATA0aPnfzc8HHx9oUULaNQI6tSB2rWtKSqKQyfSeG/Fe9xc62ayT2ez78g+9h21pl/2/MK2w9vw8fAhJzfnorIEoZpfNSIDIonwjyDCP4LqftWJ8I+gikcVVh1YRf+4/jQMa0jVKlX1QnAFouGu1NVQOPx9fGD7duvsfts2mD/favIJDoYjR8418wB4ekJ0tHWRt2VLq7mnVq2CKa2qD1PWz2BQ80H4efpx8NhBDhw7wIGjB5i+bjoLNi+gxTUtqOZXjUPZhzh47CCHjh3idN7pi0p0EzfCfMOo5leNYJ9gjp48SmJkIjWDaxLuG041v2qE+4UT7htOuF84IT4hejAoxzTclbJb4eAPCoI9e6w2/Z07rdevvrK6bRZ11i9iza9f3zrTr1HDmqKjSasewJTsnxnU5lHCAiMKvmKM4XDOYf5M/ZPp66aTeE0iObk5HMo+REp2CinZKaw6sIrdWbuL/YvgrBCfEKr5VSPMN6xg8vX0ZXvGdjrX7UxMcAyhvqGEVgmlapWqVK1SFU93z7Lak6oQDXelyrvC4e/vb/XV373bmmbNssbXqVfPuqC7dy+cOHH+90Wsi8BRUedPgYGwdi3cf791raBqVWtZIO14GlNWT2FQ80EEeQeRdjyNlOwUUo+nkpqdyuwNs1m4eSGto1tTI7AG6SfSSTueRtrxNA4eO0ieySviF7H4e/nj6eZJzaCaVPOrVhD6VatUxcvdi/Up6+l+bXdqBdUi2CeYkCohhPiE4O/lr72ILoGGu1Ku7MK2fmPg8GEr5N95x/qsY0eoWRP274d9+6zp8OGL1+XlZR0EIiOtoD98GG6+2boGEBFxbqpenbS8YwXhH+Ybdt5qUrNT+WjlR3Sr3w1jDOkn0kk/nk7GiQzST6TzzfZv+N/e/9EwrCHBPsFknMgomEo6KLjhho+Hj3VA8K1KsE8wwT7BBHkHEewTjKebJ3+m/UmnOp2ICowiyDuIIJ+g814r0wNeNNyVqqguDP7Cjh+3unJOnQoJCVYf/gMHzk3r1hXcuVukgADw9ra6fkZHQ/XqVo+g6tWhShXrGsL990PDhlZTU6Ez7sJ/FRQ+MOSZPHYe3smkVZO4pe4tABw+cZjMnEwO5xzmq61f8cOuH4iPiCcqIIrMnEwyczLJOplFZk4mx04dK3WXeLl5EeQTRKB34HmTt4c3yUeSSYxMJMI/ggDvAAK9Awnwyn/1DiDAK4AA7wD8vfwJ8Aoo981LGu5KqYudPTD0729d4D148Pxp8WL45RfrrN7b2zoQ5A/QdhFPT6tH0NkpMBBSUuDGG60Lw+Hh1sHn7GtoqHUB+cKSijkonHXw2EE+SvqI7td2x93NnaycLLJOZpGVk8W8TfNYsHkB7Wq1o1F4I7JOZnHk5BGOnDxCVk4We7L2cDjnMO7iXjByaGm83b0J8A7A18OXU2dOER1oDTzn7+VvTZ7+uIkbWzK20Ca6DdX9q+Pn5Yefpx/+Xv4FPxd+9fX0ddqYRRruSqlLV9RfBadPW3f2bt4MM2daA7jl5FhBnppqvaakWDd+FXcgOCso6PywDw217hPYuRM6dbKamUJDreajs6++vuf9hXBeuaUcGAp/HuAVwNFTRzl68ihHTx3lo6SP+CDpA/o37U/72u0L5h89eZRjp47xW/JvrDq4ivpV6xPuF86xU8cKpowTGeTm5RZRUfE83Dzw8/TDx8OHhMgEPrnzkyJrLo2Gu1Lq6jp7YLj3XiuM09Ks8E9Lsy4QL1gAN9xgndWnpUF6uvV64IB1ACmOl5f1V0RkpHVgqFoVQkKsVy8v66Bz663WgSEk5PzJu/i2+Es5MFz4edrxNCavnsxdTe7Cx8OH7FPZZJ/O5tgp65rFpNWTuKfJPbSv3Z7sU9kcP32c7NPZZJ/K5rfk31ixfwWjO43mqTZPXfJu1nBXSpUfJV0nSEuDiROhRw/rwnFGhjWlp1uvixdbz9qNjbXCPSPDuiickQHHSmmP9/Gxmo8iIqzvBgefm7y8YMsW68J0VJT1V8WFk59fsX81FPurXsFBwxEa7kqpiqGkA8P+/fDRR9C1q/X+8GHIzLReDx+Gb76BZcusA0P16tZnZ6eMDOtgUhIR61pCcLAV9oGB1qu3t7XtxERrvYGB508BAedeAwKsrq5FXG+4HBruSilV0oEhNdU6MPTsaQVvVpYV+llZ1rRwofVXww03WPcbZGVZdxpnZcGuXdbBwcPj/LGGSuLnZwW9ry80bmzVdWFNDtBwV0qpK1FaU9LZzwIDrTuMjx61wn/CBPjwQ2uI6bZtz3129vMVK6ybzEaPhqe0zV0ppVxDSQcFRz4vhYa7UkpVQI6Guw4IrZRSFZBD4S4iXURks4hsE5ERJSzXR0SMiJR6VFFKKVV2Sg13EXEH3ge6Ao2BfiLSuIjlAoDHgeXOLlIppdSlceTMvSWwzRizwxhzCpgN9CxiuZeA0UDxg0QrpZS6KhwJ9yhgb6H3yfnzCohIc6CGMWZRSSsSkcEikiQiSampqZdcrFJKKcc4Eu5F3Xtb0MVGRNyAt4G/l7YiY8xEY0yiMSYxPDzc8SqVUkpdEkfCPRmoUeh9NLC/0PsAIBb4r4jsAloDC/WiqlJK2afUfu4i4gFsAToC+4DfgXuNMRuKWf6/wD+MMSV2YheRVGD3ZdQMEAakXeZ3KxvdV47R/eQY3U+OKcv9VMsYU2rTh0dpCxhjckXkUWAJ4A5MNsZsEJFRQJIxZuHlVOdIccURkSRHOvEr3VeO0v3kGN1PjikP+6nUcAcwxiwGFl8w79/FLNvuystSSil1JfQOVaWUqoBcNdwn2l2AC9F95RjdT47R/eQY2/eTbQOHKaWUKjuueuaulFKqBC4X7o4OYlbZiMhkEUkRkfWF5lUVkW9FZGv+a4idNZYHIlJDRJaKyCYR2SAiT+TP1311ARHxEZEVIrI2f1+9mD+/togsz99Xn4qIl921lgci4i4iq0VkUf57W/eTS4W7o4OYVVJTgS4XzBsBfG+MqQ98n/++sssF/m6MaYR1w93Q/P+HdF9d7CTQwRjTDIgHuohIa+AN4O38fXUYeNDGGsuTJ4BNhd7bup9cKtxxfBCzSscYswzIuGB2T2Ba/s/TgDuualHlkDHmgDFmVf7PR7H+MUah++oixnIs/61n/mSADsDc/Pm6rwARiQZuAyblvxds3k+uFu6lDmKmzlPdGHMArFADqtlcT7kiIjFAc6xhqnVfFSG/qWENkAJ8C2wHMo0xZ58Krf8GLeOAfwJ5+e9DsXk/uVq4lziImVKOEhF/YB7wpDHmiN31lFfGmDPGmHisMaVaAo2KWuzqVlW+iEh3IMUYs7Lw7CIWvar7yaE7VMuR0gYxU+c7JCKRxpgDIhKJdfZV6YmIJ1awzzTGzM+frfuqBMaYzPxxo1oDwSLikX9Wqv8GoQ3QQ0S6AT5AINaZvK37ydXO3H8H6udfhfYC7gEua2ybSmIh8ED+zw8AC2yspVzIbwv9GNhkjBlb6CPdVxcQkXARCc7/uQrQCesaxVKgT/5ilX5fGWP+ZYyJNsbEYGXSD8aY+7B5P7ncTUz5R8dxnBvE7BWbSyoXRGQW0A5rNLpDwAvAF8AcoCawB+hrjLnwomulIiI3Aj8Bf3CuffQZrHZ33VeFiEgc1oVAd6wTwTnGmFEiUgerM0NVYDXQ3xhz0r5Kyw8RaYc1Km53u/eTy4W7Ukqp0rlas4xSSikHaLgrpVQFpOGulFIVkIa7UkpVQBruSilVAWm4K6VUBaThrpRSFZCGu1JKVUD/DyY6FD8TbOmbAAAAAElFTkSuQmCC\n", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "nn.fit_epoch(X_train,y_train,X_test,y_test,lr=0.01,epochs=20,bs=512,beta1=0.9)" ] @@ -1032,43 +881,9 @@ }, { "cell_type": "code", - "execution_count": 127, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Epoch 1. Training loss: 0.29180977605399294, Val loss:0.3333131512457867\n", - "Epoch 2. Training loss: 0.28590664025453655, Val loss:0.33280775472681706\n", - "Epoch 3. Training loss: 0.284354744492831, Val loss:0.3319412315286279\n", - "Epoch 4. Training loss: 0.2836359602338351, Val loss:0.3318411543204399\n", - "Epoch 5. Training loss: 0.2840274762633642, Val loss:0.3315891587513862\n", - "Epoch 6. Training loss: 0.28405648106200443, Val loss:0.3314421561281658\n", - "Epoch 7. Training loss: 0.2834176966591488, Val loss:0.3310195950602498\n", - "Epoch 8. Training loss: 0.2834370919823108, Val loss:0.33119587451175175\n", - "Epoch 9. Training loss: 0.2817027694235941, Val loss:0.33058750121545133\n", - "Epoch 10. Training loss: 0.2812265904175054, Val loss:0.33029430245223507\n", - "Epoch 11. Training loss: 0.28164920994081216, Val loss:0.33000137582860756\n", - "Epoch 12. Training loss: 0.2799430241697258, Val loss:0.3300854585871695\n", - "Epoch 13. Training loss: 0.2801152434978174, Val loss:0.3296515518446343\n", - "Epoch 14. Training loss: 0.2799086282887769, Val loss:0.3293390927042878\n", - "Epoch 15. Training loss: 0.279855535753892, Val loss:0.32994998145139315\n" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xl8VNX9//HXJzuEAAkJiyyCS9UQAsSI8gNlVXEB1GIFpRZbi7Vqbf3Wr9Rqq7Q8Sv36VUpLK9hK3WpKtSoqSGtLXfq1CChEVkEEiSgkyJqwZJLz++NOhsk+JBOGmbyfj8d9zJ07J3fOnYH3nDn33DPmnENERGJLXKQrICIi4adwFxGJQQp3EZEYpHAXEYlBCncRkRikcBcRiUEKdxGRGKRwFxGJQQp3EZEYlBCpJ87MzHS9e/eO1NOLiESllStXljjnshorF7Fw7927NytWrIjU04uIRCUz2xZKOXXLiIjEIIW7iEgMUriLiMSgiPW5i8iJVV5eTlFREYcPH450VSQEKSkp9OjRg8TExCb9vcJdpJUoKioiLS2N3r17Y2aRro40wDnH7t27KSoqok+fPk3ah7plRFqJw4cP06lTJwV7FDAzOnXq1KxvWQp3kVZEwR49mvteRV+4v/oqjBwJu3ZFuiYiIiet6Av3uXNh6VJ47LFI10REjsPu3bsZMGAAAwYMoGvXrnTv3j1w/+jRoyHt46abbmLjxo0NlpkzZw7PPvtsOKrM0KFDWbVqVVj2daJF3wnVSy7xWu9XXBHpmojIcejUqVMgKB944AHatWvHD3/4w2plnHM454iLq7vdOX/+/Eaf57bbbmt+ZWNA9LXce/WKdA1EJIw2b95MTk4O3/nOd8jLy+Pzzz9n6tSp5Ofn07dvX6ZPnx4oW9WS9vl8dOzYkWnTptG/f38GDx7MLn9X7X333cesWbMC5adNm8agQYM466yz+L//+z8ASktL+epXv0r//v2ZNGkS+fn5jbbQn3nmGfr160dOTg733nsvAD6fj69//euB7bNnzwbg0UcfJTs7m/79+zN58uSwv2ahiL6We3q6d7tnT2TrIRLNvv99CKW7obwcvvgCunaFxsZbDxgA/lA9XuvWrWP+/Pk85u9unTlzJhkZGfh8PkaMGMGECRPIzs6u9jf79u1j2LBhzJw5k7vuuosnnniCadOm1dq3c4733nuPhQsXMn36dF5//XV+/etf07VrV1544QVWr15NXl5eg/UrKirivvvuY8WKFXTo0IHRo0fz6quvkpWVRUlJCR9++CEAe/fuBeChhx5i27ZtJCUlBbadaNHXcs/I8G4V7iIt74svYMsW77YFnX766Zx33nmB+8899xx5eXnk5eWxfv161q1bV+tv2rRpw2WXXQbAueeey9atW+vc9zXXXFOrzDvvvMPEiRMB6N+/P3379m2wfsuWLWPkyJFkZmaSmJjI9ddfz1tvvcUZZ5zBxo0bufPOO1myZAkdOnQAoG/fvkyePJlnn322yRchNVejLXczewK4EtjlnMup43EDfgVcDpQBU5xz74e7ogFquYs0X6gt7JISmD8fbroJMjNbrDqpqamB9U2bNvGrX/2K9957j44dOzJ58uQ6x3snJSUF1uPj4/H5fHXuOzk5uVYZ59xx1a++8p06daKwsJDFixcze/ZsXnjhBebNm8eSJUt48803efnll/n5z3/OmjVriI+PP67nbK5QWu5/BMY08PhlwJn+ZSrwu+ZXqwFV4f7lly36NCKCF+h3392iwV7T/v37SUtLo3379nz++ecsWbIk7M8xdOhQFixYAMCHH35Y5zeDYBdccAFLly5l9+7d+Hw+CgoKGDZsGMXFxTjnuPbaa3nwwQd5//33qaiooKioiJEjR/I///M/FBcXU1ZWFvZjaEyjLXfn3Ftm1ruBIuOBp5z30fYfM+toZt2cc5+HqY7VtWkDyclquYvEqLy8PLKzs8nJyeG0005jyJAhYX+OO+64gxtvvJHc3Fzy8vLIyckJdKnUpUePHkyfPp3hw4fjnGPs2LFcccUVvP/++3zrW9/COYeZ8ctf/hKfz8f111/PgQMHqKys5J577iEtLS3sx9AYC+XriT/cX62nW+ZVYKZz7h3//X8A9zjnGvwljvz8fNfkH+vo1g3GjoV585r29yKt0Pr16znnnHMiXY2Tgs/nw+fzkZKSwqZNm7jkkkvYtGkTCQkn1xiTut4zM1vpnMtv7G/DcSR1XSNb5yeGmU3F67qhV3OGNKanq1tGRJrs4MGDjBo1Cp/Ph3OOuXPnnnTB3lzhOJoioGfQ/R7AjroKOufmAfPAa7k3+RnT09UtIyJN1rFjR1auXBnparSocAyFXAjcaJ4LgH0t1t9eJSND4S4i0oBQhkI+BwwHMs2sCPgpkAjgnHsMWIQ3DHIz3lDIm1qqsgHp6bBmTYs/jYhItApltMykRh53wImdzEF97iIiDYq+K1TBC/f9+6GiItI1ERE5KUVnuFdNQRChORtE5PgNHz681gVJs2bN4rvf/W6Df9euXTsAduzYwYQJE+rdd2NDq2fNmlXtYqLLL788LPO+PPDAAzz88MPN3k+4RWe4awoCkagzadIkCgoKqm0rKChg0qQGe34DTjnlFJ5//vkmP3/NcF+0aBEdO3Zs8v5OdtEd7up3F4kaEyZM4NVXX+XIkSMAbN26lR07djB06NDAuPO8vDz69evHyy+/XOvvt27dSk6Odx3loUOHmDhxIrm5uVx33XUcOnQoUO7WW28NTBf805/+FIDZs2ezY8cORowYwYgRIwDo3bs3JSUlADzyyCPk5OSQk5MTmC5469atnHPOOXz729+mb9++XHLJJdWepy6rVq3iggsuIDc3l6uvvpo9/gbo7Nmzyc7OJjc3NzBh2Ztvvhn4sZKBAwdy4MCBJr+2dYnOUfuaGVKkWb7/+vdZ9UXjU/6WV5TzxcEv6NquK4nxDc9uOKDrAGaNqX9Csk6dOjFo0CBef/11xo8fT0FBAddddx1mRkpKCi+++CLt27enpKSECy64gHHjxtX7O6K/+93vaNu2LYWFhRQWFlabsnfGjBlkZGRQUVHBqFGjKCws5Hvf+x6PPPIIS5cuJbPGPDkrV65k/vz5LFu2DOcc559/PsOGDSM9PZ1Nmzbx3HPP8fjjj/O1r32NF154ocH52W+88UZ+/etfM2zYMH7yk5/w4IMPMmvWLGbOnMknn3xCcnJyoCvo4YcfZs6cOQwZMoSDBw+SkpLS4Ot7vKK75a5wF2lRXxz8gi17t/DFwfBM+RvcNRPcJeOc49577yU3N5fRo0fz2WefsXPnznr389ZbbwVCNjc3l9zc3MBjCxYsIC8vj4EDB7J27dpGJwV75513uPrqq0lNTaVdu3Zcc801vP322wD06dOHAQMGAA1PKwze/PJ79+5l2LBhAHzjG9/grbfeCtTxhhtu4JlnnglcCTtkyBDuuusuZs+ezd69e8N+hWx0ttzVLSPSLA21sIOVlJUw/4P53DTwJjLbNn9myKuuuoq77rqL999/n0OHDgVa3M8++yzFxcWsXLmSxMREevfuXec0v8HqatV/8sknPPzwwyxfvpz09HSmTJnS6H4aml+rarpg8KYMbqxbpj6vvfYab731FgsXLuRnP/sZa9euZdq0aVxxxRUsWrSICy64gDfeeIOzzz67Sfuvi1ruIlKvzLaZ3D3k7rAEO3gjX4YPH843v/nNaidS9+3bR+fOnUlMTGTp0qVs27atwf1cdNFFgR/BXrNmDYWFhYA3XXBqaiodOnRg586dLF68OPA3aWlpdfZrX3TRRbz00kuUlZVRWlrKiy++yIUXXnjcx9ahQwfS09MDrf6nn36aYcOGUVlZyfbt2xkxYgQPPfQQe/fu5eDBg3z88cf069ePe+65h/z8fDZs2HDcz9mQ6Gy5JydD27YKd5EoNGnSJK655ppqI2duuOEGxo4dS35+PgMGDGi0BXvrrbdy0003kZuby4ABAxg0aBDg/arSwIED6du3b63pgqdOncpll11Gt27dWLp0aWB7Xl4eU6ZMCezj5ptvZuDAgQ12wdTnySef5Dvf+Q5lZWWcdtppzJ8/n4qKCiZPnsy+fftwzvGDH/yAjh07cv/997N06VLi4+PJzs4O/KpUuIQ05W9LaNaUvwA9esCll8If/hC+SonEME35G32aM+VvdHbLgKYgEBFpQPSGu2aGFBGpV/SGu+Z0FzlukeqGlePX3PcqusNd3TIiIUtJSWH37t0K+CjgnGP37t3NurApOkfLgFruIsepR48eFBUVUVxcHOmqSAhSUlLo0aNHk/8+esM9IwNKS6G8HBIbvixaRCAxMZE+ffpEuhpygkR3twyo9S4iUoeQwt3MxpjZRjPbbGbT6nj8VDP7h5kVmtm/zKzp3yVCpSkIRETq1Wi4m1k8MAe4DMgGJplZdo1iDwNPOedygenAL8Jd0VrUchcRqVcoLfdBwGbn3Bbn3FGgABhfo0w28A//+tI6Hg8/TfsrIlKvUMK9O7A96H6Rf1uw1cBX/etXA2lm1qn51WuAWu4iIvUKJdzrmi2/5kDZHwLDzOwDYBjwGeCrtSOzqWa2wsxWNHs4lvrcRUTqFUq4FwE9g+73AHYEF3DO7XDOXeOcGwj82L9tX80dOefmOefynXP5WVlZzag2armLiDQglHBfDpxpZn3MLAmYCCwMLmBmmWZWta8fAU+Et5p1SEiAtDSFu4hIHRoNd+ecD7gdWAKsBxY459aa2XQzG+cvNhzYaGYfAV2AGS1U3+o0BYGISJ1CukLVObcIWFRj20+C1p8Hng9v1UKgKQhEROoUvVeogqb9FRGpR3SHu1ruIiJ1iv5wV5+7iEgt0R3u6pYREalTdId7ejocPuwtIiISEP3hDmq9i4jUEBvhrn53EZFqojvcNTOkiEidojvc1S0jIlKn2Ah3dcuIiFQTG+GulruISDXRHe4dOoCZwl1EpIboDvf4eC/gFe4iItVEd7iDpiAQEalD9Ie7piAQEakl+sNdM0OKiNSicBcRiUGxEe7qcxcRqSakcDezMWa20cw2m9m0Oh7vZWZLzewDMys0s8vDX9V6VPW5O3fCnlJE5GTXaLibWTwwB7gMyAYmmVl2jWL34f1w9kBgIvDbcFe0XunpUF4OZWUn7ClFRE52obTcBwGbnXNbnHNHgQJgfI0yDmjvX+8A7AhfFRuhKQhERGoJJdy7A9uD7hf5twV7AJhsZkXAIuCOunZkZlPNbIWZrSguLm5CdeugmSFFRGoJJdytjm01O7gnAX90zvUALgeeNrNa+3bOzXPO5Tvn8rOyso6/tnXR/DIiIrWEEu5FQM+g+z2o3e3yLWABgHPuXSAFyAxHBRulcBcRqSWUcF8OnGlmfcwsCe+E6cIaZT4FRgGY2Tl44R6mfpdGqM9dRKSWRsPdOecDbgeWAOvxRsWsNbPpZjbOX+y/gG+b2WrgOWCKcydobKL63EVEakkIpZBzbhHeidLgbT8JWl8HDAlv1UKUlubNDqlwFxEJiP4rVM2gY0eFu4hIkOgPd9AUBCIiNcRGuGvaXxGRamIj3DUzpIhINbET7uqWEREJiI1wV7eMiEg1sRHu6emwd6+m/RUR8YudcK+ogAMHIl0TEZGTQuyEO6jfXUTELzbCXVMQiIhUExvhrpkhRUSqia1wV7eMiAgQK+GubhkRkWpiI9zVLSMiUk1shHvbtpCYqHAXEfGLjXA30xQEIiJBYiPcQVMQiIgECSnczWyMmW00s81mNq2Oxx81s1X+5SMz2xv+qjZCM0OKiAQ0+jN7ZhYPzAEuBoqA5Wa20P/TegA4534QVP4OYGAL1LVh6emwc+cJf1oRkZNRKC33QcBm59wW59xRoAAY30D5SXg/kn1iqc9dRCQglHDvDmwPul/k31aLmZ0K9AH+Wc/jU81shZmtKC4uPt66Nkx97iIiAaGEu9Wxrb65dScCzzvnKup60Dk3zzmX75zLz8rKCrWOoUlPh337vNkhRURauVDCvQjoGXS/B7CjnrITiUSXDHjh7pwX8CIirVwo4b4cONPM+phZEl6AL6xZyMzOAtKBd8NbxRBpCgIRkYBGw9055wNuB5YA64EFzrm1ZjbdzMYFFZ0EFDgXoZ9D0hQEIiIBjQ6FBHDOLQIW1dj2kxr3HwhftZpA4S4iEhA7V6hq2l8RkYDYCXf1uYuIBMROuKtbRkQkIHbCPSXFWxTuIiIxFO6gKQhERPxiK9w1BYGICBBr4a5pf0VEgFgMd3XLiIjEWLirW0ZEBIi1cFe3jIgIEIvhfuAAlJdHuiYiIhEVe+EOsPfE/4SriMjJJLbCXVMQiIgAsRbumoJARARQuIuIxKTYCveqbhmNdReRVi62wl0tdxERIMRwN7MxZrbRzDab2bR6ynzNzNaZ2Voz+1N4qxkihbuICBDCz+yZWTwwB7gYKAKWm9lC59y6oDJnAj8Chjjn9phZ55aqcIMSEyE1Vd0yItLqhdJyHwRsds5tcc4dBQqA8TXKfBuY45zbA+Cc2xXeah4HTUEgIhJSuHcHtgfdL/JvC/YV4Ctm9m8z+4+ZjQlXBY+bpiAQEWm8WwawOra5OvZzJjAc6AG8bWY5zrlql4qa2VRgKkCvXr2Ou7IhUbiLiITUci8Cegbd7wHsqKPMy865cufcJ8BGvLCvxjk3zzmX75zLz8rKamqdG6Zpf0VEQgr35cCZZtbHzJKAicDCGmVeAkYAmFkmXjfNlnBWNGTqcxcRaTzcnXM+4HZgCbAeWOCcW2tm081snL/YEmC3ma0DlgJ3O+d2t1SlG6RuGRGRkPrccc4tAhbV2PaToHUH3OVfIis9HcrK4MgRSE6OdG1ERCIitq5QBc0MKSJCLIa7rlIVEVG4i4jEotgNdw2HFJFWLPbCXX3uIiIxGO7qlhERicFw79jRu1W4i0grFnvhHh8PHTqoz11EWrXYC3fQVaoi0uop3EVEYlDshru6ZUSkFYvNcNfMkCLSysVmuKtbRkRaudgOd1fzB6NERFqH2A33I0fg0KFI10REJCJiM9w1BYGItHKxGe6agkBEWrmQwt3MxpjZRjPbbGbT6nh8ipkVm9kq/3Jz+Kt6HBTuItLKNfoze2YWD8wBLgaKgOVmttA5t65G0T87525vgToev6puGY11F5FWKpSW+yBgs3Nui3PuKFAAjG/ZajWTWu4i0sqFEu7dge1B94v822r6qpkVmtnzZtYzLLVrKoW7iLRyoYS71bGt5gDyV4Dezrlc4A3gyTp3ZDbVzFaY2Yri4uLjq+nxaN8ezNQtIyKtVijhXgQEt8R7ADuCCzjndjvnjvjvPg6cW9eOnHPznHP5zrn8rKysptQ3NHFxukpVRFq1UMJ9OXCmmfUxsyRgIrAwuICZdQu6Ow5YH74qNpHCXURasUZHyzjnfGZ2O7AEiAeecM6tNbPpwArn3ELge2Y2DvABXwJTWrDOoVG4i0grFtI4d+fcIufcV5xzpzvnZvi3/cQf7DjnfuSc6+uc6++cG+Gc29BSFf7XJ//iovkXUVzaSJ99Rob63EWk1Yq6K1RnLZvF25++zR2L72i4oFruItKKRV24z71yLr069GLJx0vYVbqr/oIKdxFpxaIu3Lu068LiGxZTerSUHyz5Qf0FNe2viLRiURfuANlZ2fz4wh/zpw//xGsfvVZ3oYwM8Png4METWzkRkZNAVIY7wLSh08jOyubW127lwJEDtQvoKlURacWiNtyTE5L5/djfU7S/iB//88e1C6SkeLeLF5/YiomInASiNtwBBvcczO2Dbuc37/2Gd7e/W/3BTz/1bn/wA9i+vfYfi4jEsKgOd4AZI2fQo30Pbn7lZo74jhx74Nvfhv/+b2+Oma9/HSoqIldJEZETLOrDPS05jceufIx1xeuY+c7MYw9kZsIvfwm//S28+SbMnFn/TkREYkzUhzvA5WdezvX9rmfG2zNYV1zjN0RuvBEmTYKf/hTefbfuHYiIxJiYCHeAWZfOon1ye25eeDOVrvLYA2bwu99Bz55w/fWwb1/kKikicoLETLhnpWbx6KWP8m7Ru/x2+W+rP9ihAzz3nHdi9dZbdWGTiMS8mAl3gMm5k7n09Ev50T9+xKf7Pq3+4AUXwIMPeiH/1FORqaCIyAkSU+FuZjx25WNUukpufe1WXM0W+rRpMGwY3HYbbN4cmUqKiJwAMRXuAL079mbGyBks2rSIgjUF1R+Mj4enn4akJO8k69GjkamkiEgLi7lwB7hj0B0M7DqQm1+5mVc2vlL9wZ494Q9/gBUr4P77I1NBEZEWFpPhHh8Xz8g+IykrL2NcwTiu+fM1bNq96ViBq6+GW26Bhx6CN96IXEVFRFqI1eqXPkHy8/PdihUrWmz/JWUlzF05l4NHDvLr937N0Yqj3Hbebdw/7H4y2mRAWRmcd573a02FhdCSP9gtIhImZrbSOZffWLmQWu5mNsbMNprZZjOb1kC5CWbmzKzRJ25pmW0z+fGFP+YXo3/Bpjs28Y3+32D2e7M5Y/YZPPruoxxNTvBGzuzZAzfdpOGRIhJTGg13M4sH5gCXAdnAJDPLrqNcGvA9YFm4K9lc3dK68fi4x1l1yyrO634ed/3tLrLnZPNCwibcQw/Ba6/BuefC1q2RrqqISFiE0nIfBGx2zm1xzh0FCoDxdZT7GfAQcDiM9Qurfl36sWTyEhbfsJiUhBQm/GUCF7X7C/+5+jzKV38AgwbBmjWRrqaISLMlhFCmOxA8Z24RcH5wATMbCPR0zr1qZj+sb0dmNhWYCtCrV6/jr22YjDljDKNPG80THzzB/UvvZ3D/XdAf4iqLafunfrRpk0abtHTaJLQhJSGFNoltaJPQhsT4RHp36M2MUTPIbJsZsfqLiDQmlJa71bEt0EFtZnHAo8B/NbYj59w851y+cy4/K8InMBPiEph67lQ237GZq8++GoDhPYbw7eKefHXZAUYUJTIgM4dTO55KWlIa5ZXl/KfoP8x7fx4/euNHEa27iEhjQmm5FwE9g+73AHYE3U8DcoB/mRlAV2ChmY1zzrXccJgwSUtOY97YeQzuMZibBt5E5jfTYcYMeOABODsJFiyAnBwAPir5iOFPDufpwqe5tu+1XHL6JZGtvIhIPRodCmlmCcBHwCjgM2A5cL1zbm095f8F/LCxYG/poZDN9s9/erNI7t8Pc+Z4I2qA3WW7GfXUKDaUbGDhpIUKeBE5ocI2FNI55wNuB5YA64EFzrm1ZjbdzMY1v6onqZEjYdUqb8Kxb34TpkyB0lI6te3EP278B2dnns2458bxt4//FumaiojUErMXMYVNRQX87GcwfTqceSZcdRXcfTe72xqjnx7N+uL1vDzxZS4949JI11REWoGwXsTUqsXHe/3vf/87FBV5UxbMmUOntp144+tvcE7WOYwvGM+SzUsiXVMRkQCFe6hGjYI//xni4uBf/wKfr1bAv7759UjXUkQEULgfnyuvhLlzvXC/5x6AQMBnZ2VzVcFVCngROSmEMhRSgt18szfR2COPQL9+MGWKF/A3vsHop0ZzVcFVvDTxJcacMabJT+Gr9FFcWsyu0l3sLN3J5t2b+duWv5F/Sj4pCSn4Kn3VlorKCnyVPg4cOcCa4jXkds6lfUp7EuMSSYxPJCk+KbCeGJfIYd9h3vvsPa455xrOzjybbmndyGiTQZzps14kVuiEalOUl8OYMfDOO14rfvBgAL489CWjnxrN2l1rOfeUc8ntnEvbpLYYhpkRZ3GBdcM47DvMih0ryErNYu/hvews3cmu0l3sLtuNI7T3Jc7iSIhLICEugYrKCo5UHCExLhEzo7yiPOT9JMYl0qVdF7q160a3tG50a9eNru26kpaUxgeff8CVX7mSnh160j65PWnJad5tUhqJ8YlNfRVFpAlCPaGqcG+q3bu9uWjKymD5cujRA/AC/rzHz2PLni0kxyeTGJ+Icw6Ho9JVBtadc1S4CipdJZ3adOLszLPpnNqZLqld6NKuS7X1hLgEFm9azI39bwzcj7d44uPiq7W2S8pKmP/BfO9irLaZgecoryinvLI8cLvz4E6e/fBZLux1IYd8h/ji4Bd8fuBzPj/oLVX3i8uKG30Z2iS0oX1ye9omtuVQ+SG6tutKu+R2JMcnk5yQTHJ8MikJKYH1yspK1pWs44yMM4izOI5WHK1zKT1ayoGjB7j4tIsZ3HMwuV1yOTvzbJLik1rsLRWJBgr3E2HtWm8c/FlnwdtvQ5s2QO2QrU+o5SKlvKKcDSUbeHL1k4w+bTRxFseBIwfYf2R/rWXZZ8tYvXM1Z3U6i1PSTuFIxRGO+I5wpOIIh32HA+v7D+/ncMVhUhNTSW+TTlJ8Uq0lOT6Zov1FbPpyE/EWT4WrALxvF+dknUP/Lv3J7ZIbuO3SrkuEXymRE0fhfqIsXOiNfZ80CZ55BqyuqXhiX7g/0KrKTc6dzJeHvqRwZyGrd64O3O44cGwGjMQ477yCf/oLLGg6pKptla6S8opyEuISAtuCv0VVdV9VVlYSZ3Fcfc7V3Hn+nQzqPihQXuRkoHA/kWbMgPvug5kzA6NopGWVlJVQuLOQWf+ZxSsfvcKFvS7kvFPOq3aOIfjf9oodK3hn+zsM7TmU87qfF/gAqDr/UXW77LNlvLntTRLjEimvLKd/l/7ccu4t3JB7A+2T25/w4xSpSeF+IjnntdwXLIBXXoErroh0jVqNlvrGcG32tSz5eAlzV87lgy8+IDUxlUk5k7gl/xbyT4n4D41JK6ZwP9HKymDoUNi8GZYtg3POiXSNJAyccyzfsZy5K+ZSsLaAsvIy8rrlccu5tzApZxJpyWmRrqK0Mgr3SPj0U+9Ht9u3h3//Gzp3Ds9+jxyBXbvg8OGGl+Ji78Tu//t/Xh3i4upeDh706nfttZCdDd26Qdu24alrDNt3eB/PFD7D3JVz+XDXh6TEp5CSmEL7pPYkJSQFRjElxCUQHxcfWK90lZSUldCrQy/aJbXzrjvwX3+QFJcUuO+r9LFu1zr6delHalJqra6jqvVD5YdYtXMVeV3zaJfUjjiL84bZ+ofbVg25LSsvY/mO5Zzf/XzSktOqdT8F77O0vJR3i95laM+hdEzpGKilab92AAAL3UlEQVR78G2cxVF6tJR/fvJPRp02ivbJ7XHOPwIs6LxF1bb9R/bz5rY3ueT0S8hokxEYrlvXEm/xgXo3tAQfX52PY4FycKxbrqqrruZ9oNrxnajrPKpeq6Y+n8I9Uv79bxg+HHw+yMz0Jhs7/XRvOeOMY+tZWcdOvjrnDa3csgU+/rj2bVFRy9e7Qwc45RQv6E855dh6aiqsWAETJ3r1z8jwPgha8UlG5xz/KfoPd//9bv69/d8M7DqQszLPoqKyggpXEbiwrGr9kz2f8PGej+nZvieZbTMDwz3LK8urDf8sKy/DV+mrFjTBJ3uD16Vl1PxQMwxfpY82iW1IiEto8MPFOceew3tok9AGh6t1sWHVUukquebsa5g7dm6TRskp3CPpjjvgN7/xWvGpqccCOvi1btcOTj0V9uzx5ow/eLD6Prp1g9NO8z4IPv/cm7hs4kSYMAFSUupeysrg+efhuusgPR0qK2svFRXeB8lf/uK18EtLYccO7zmCb3fsgKNH6z6+5GQv5Dt1Onbbti3s3Anjx0NuLvTp431AxMe33OscYS3V3x9KuSc+eIIpA6aQ0SYj0FquakVXrZeUlvB04dNc3+96r1yNFjZ4rdiSshIK1hRwbfa1dEjpEPhgqrqtdJVUVFaw+9BuXt7wMmPPGkt6SnqglRz8TaBq255De3hxw4tc8ZUrSEtKqzfonl/3PE8XPs0NOTdwxVeuCNS95rJ482Je3PAiV511FaNOG1VvuTe3vsnrH7/OmNPHcOGpF1b7xgPHRlK9te0tFm1exJjTxzCk15Bax1x1u/yz5byz/R0G9xhM/y79qz8f1Z977a61rN65mnO7ncvArgPr/aay7LNl/H3L33lo9EPcPeTu4/53p3CPpJISmD/f+4GPTP9/0sOHYetWL+irlr/9DTZs8EJ2woRjrfo+fap3k9S1v5bmnPfBs3Yt/OlP3nj+8nLvg+HLL73b4PVt22p/QCUmQq9e3vFULZ06efPk33ijd16ifftW/S2gtYvkB2S4r0U5Ude3KNyjQSRCu6WUlMDjj8OIEd43kU8+qb5s3eqdE6gpOdk7N9Gly7HbLl28bzxr1sDYsdCzp9dt1LGjd1t1PkGkFVK4y8nn4EH44AN46inIy/O6kXbu9E4W79x5bH3XLu9bQkPS0rywT031yg4dCgMHelcLn3WW940hhruEpPUKa7ib2RjgV0A88Hvn3Mwaj38HuA2oAA4CU51z6xrap8Jd6uWcN6T097+Hiy/2um327oV9+7wleH35cq/rKCXF6/qqkpwMX/nKsbA/66xjJ4SrvgWkpETuGEWaKGzhbmbxeD+QfTFQhPcD2ZOCw9vM2jvn9vvXxwHfdc41OOetwl3Coqpra8oU70Nh40Zv2bDh2PqWLd6J5JqSk48FfdWSkuJ1H915J3z1q5CgWbHl5BJquIfyL3cQsNk5t8W/4wJgPBAI96pg90sFjdeSEyQzE+4OGnHQuTNceGH1MkePwr33wv/+r3cl8YUXeq3/qqXq28DevV63UXExvPuut6+JE+GGG7yRTzrxK1EklHDvDmwPul8EnF+zkJndBtwFJAEjw1I7kXBISoJp07wTtY2dvK46Mdy9uzcp3Ny5MHu2d73CDTd4yxlnnLi6izRRKN0y1wKXOudu9t//OjDIOXdHPeWv95f/Rh2PTQWmAvTq1evcbdu2NbP6Ii1s71544QV49lnvh1mcg/PP90L+uuvCdxWySIjC2ec+GHjAOXep//6PAJxzv6infBywxznXoaH9qs9dok5RETz3nBf0q1d73TQZGXDuud5PLgafvO3cWd040iLCGe4JeCdURwGf4Z1Qvd45tzaozJnOuU3+9bHATxt7coW7RLU1a+Cuu7wrh7t29Vr4waN1OnTwQv7ss71f6dqwAUaP9sq2bestqam115OTNYZfGhS2E6rOOZ+Z3Q4swRsK+YRzbq2ZTQdWOOcWAreb2WigHNgD1OqSEYkpOTnelbtVF6FlZHgTx9UcrfOPf8Bnn3l/89e/hrZvM+/q3uAlIeHYOsCBA95YfzNvJFDV1BLBt+Xl3gdOUlL1D4yaDbqqsm3aeGWrnq9qqbrvnHdFcufOx+pRn/Jy7wrn00/3rkpOTfWm3EhNrb5eUeH9FvHFF3uvYXx87SUh4dg1CxUVx5aqYw1e9u6Ff/4Txo3zPlRrPl9S6/mZRl3EJNLStm3zTtKOHesNtSwt9S7gKiurvr54sbeMGgX5+V5Alpd7k9BVrZeXe98aCguhf39vVs+qEIyLq367apU36mfIEK/rKFhwl9HKlV7ADh7sdS/5fMeWquf3+eCjj7wPrDPPhN69Gz7mTz7xrlU45RTvG0lpqXcRW2mpF8qRkpDgBX1KijeKquoD0rnqS2Wld1tRAYcOecdQ9dqaeUvN9cpK7+rsDh28slWvcfBrbea9loMGwaxZTboyPZxDIUWkOU49FX7+88bLTZoU2nQUoU5bEalyDZV1zpvCuirot2/3TlhfeaUXujVb4lXLggXwxz/CN78JX/ta9ZZ91YdZfLz3jeall2DkSC/Iq54n+MOltNT70Fu+3HtvcnKOhXTN0C4shPfe8+ZB6tevevDXXF+71rumont3rzuu6niDjx28D8hnn/U+nO8+/onDQqWWu4ic/MI9D9OJ/EBrzj7roLllRERiUKjhrtPyIiIxSOEuIhKDFO4iIjFI4S4iEoMU7iIiMUjhLiISgxTuIiIxKGLj3M2sGGjqnL+ZQEkYqxNJOpaTT6wcB+hYTlbNOZZTnXNZjRWKWLg3h5mtCGUQfzTQsZx8YuU4QMdysjoRx6JuGRGRGKRwFxGJQdEa7vMiXYEw0rGcfGLlOEDHcrJq8WOJyj53ERFpWLS23EVEpAFRF+5mNsbMNprZZjObFun6NIeZbTWzD81slZlF1fzHZvaEme0yszVB2zLM7O9mtsl/mx7JOoainuN4wMw+878vq8zs8kjWMVRm1tPMlprZejNba2Z3+rdH1fvSwHFE3ftiZilm9p6ZrfYfy4P+7X3MbJn/PfmzmYX99/+iqlvGzOLxfqz7YqAI78e6Jznn1kW0Yk1kZluBfOdc1I3dNbOLgIPAU865HP+2h4AvnXMz/R+86c65eyJZz8bUcxwPAAedcw9Hsm7Hy8y6Ad2cc++bWRqwErgKmEIUvS8NHMfXiLL3xcwMSHXOHTSzROAd4E7gLuCvzrkCM3sMWO2c+104nzvaWu6DgM3OuS3OuaNAATA+wnVqlZxzbwFf1tg8HnjSv/4k3n/Ik1o9xxGVnHOfO+fe968fANYD3Ymy96WB44g6znPQfzfRvzhgJPC8f3uLvCfRFu7dge1B94uI0jfdzwF/M7OVZjY10pUJgy7Ouc/B+w8KdI5wfZrjdjMr9HfbnNTdGHUxs97AQGAZUfy+1DgOiML3xczizWwVsAv4O/AxsNc55/MXaZEci7Zwtzq2RU+/Um1DnHN5wGXAbf4uAom83wGnAwOAz4H/jWx1jo+ZtQNeAL7vnNsf6fo0VR3HEZXvi3Ouwjk3AOiB1/twTl3Fwv280RbuRUDPoPs9gB0RqkuzOed2+G93AS/ivfHRbKe/v7Sq33RXhOvTJM65nf7/kJXA40TR++Lv130BeNY591f/5qh7X+o6jmh+XwCcc3uBfwEXAB3NLMH/UIvkWLSF+3LgTP+Z5iRgIrAwwnVqEjNL9Z8swsxSgUuANQ3/1UlvIfAN//o3gJcjWJcmqwpCv6uJkvfFf/LuD8B659wjQQ9F1ftS33FE4/tiZllm1tG/3gYYjXcOYSkwwV+sRd6TqBotA+Af/jQLiAeecM7NiHCVmsTMTsNrrQMkAH+KpmMxs+eA4Xiz2+0Efgq8BCwAegGfAtc6507qk5X1HMdwvK/+DtgK3FLVZ30yM7OhwNvAh0Clf/O9eP3VUfO+NHAck4iy98XMcvFOmMbjNaYXOOem+///FwAZwAfAZOfckbA+d7SFu4iINC7aumVERCQECncRkRikcBcRiUEKdxGRGKRwFxGJQQp3EZEYpHAXEYlBCncRkRj0/wFdOvEF5WOZWgAAAABJRU5ErkJggg==\n", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "nn.fit_epoch(X_train,y_train,X_test,y_test,lr=0.0001,epochs=15,bs=512,l2=1)" ] @@ -1162,7 +977,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.0" + "version": "3.6.6" } }, "nbformat": 4, diff --git a/scratch_neural_net.ipynb b/scratch_neural_net.ipynb index 90f9faf..4a01ed2 100644 --- a/scratch_neural_net.ipynb +++ b/scratch_neural_net.ipynb @@ -1,5 +1,12 @@ { "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Part 1: Building a customized neural net from scratch with weight decay, non-linear activations and dropouts" + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -9,7 +16,7 @@ }, { "cell_type": "code", - "execution_count": 14, + "execution_count": 1, "metadata": {}, "outputs": [], "source": [ @@ -141,7 +148,7 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 6, "metadata": {}, "outputs": [ { @@ -150,7 +157,7 @@ "(72.94035223214286, 90.02118235130519)" ] }, - "execution_count": 7, + "execution_count": 6, "metadata": {}, "output_type": "execute_result" } @@ -163,7 +170,7 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 7, "metadata": {}, "outputs": [], "source": [ @@ -173,7 +180,7 @@ }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 8, "metadata": {}, "outputs": [ { @@ -182,7 +189,7 @@ "(60000, 784)" ] }, - "execution_count": 9, + "execution_count": 8, "metadata": {}, "output_type": "execute_result" } @@ -193,7 +200,7 @@ }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 9, "metadata": {}, "outputs": [ { @@ -202,7 +209,7 @@ "-1.74808013869143e-17" ] }, - "execution_count": 10, + "execution_count": 9, "metadata": {}, "output_type": "execute_result" } @@ -224,7 +231,7 @@ }, { "cell_type": "code", - "execution_count": 15, + "execution_count": 10, "metadata": {}, "outputs": [], "source": [ @@ -303,92 +310,18 @@ }, { "cell_type": "code", - "execution_count": 197, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Epoch 1. Training loss: 0.7880122308672165, Val loss:0.8010807662240778\n", - "Epoch 2. Training loss: 0.7798141842254848, Val loss:0.7965325472238913\n", - "Epoch 3. Training loss: 0.776284811586093, Val loss:0.7922179527083798\n", - "Epoch 4. Training loss: 0.7729580080155023, Val loss:0.7881014526576602\n", - "Epoch 5. Training loss: 0.7697823764153524, Val loss:0.7841551603336737\n", - "Epoch 6. Training loss: 0.7667299825805938, Val loss:0.7803635988999236\n", - "Epoch 7. Training loss: 0.7637853705691704, Val loss:0.7767307703624498\n", - "Epoch 8. Training loss: 0.7609325763091156, Val loss:0.7732467182668896\n", - "Epoch 9. Training loss: 0.7581627220311515, Val loss:0.7699111202394039\n", - "Epoch 10. Training loss: 0.755468867019022, Val loss:0.766712567968624\n", - "Epoch 11. Training loss: 0.752842235300128, Val loss:0.763628956743271\n", - "Epoch 12. Training loss: 0.7502778439909424, Val loss:0.7606525061851516\n", - "Epoch 13. Training loss: 0.7477681225849878, Val loss:0.7577799056290795\n", - "Epoch 14. Training loss: 0.7453111821348567, Val loss:0.7550101451280937\n", - "Epoch 15. Training loss: 0.7428985610123593, Val loss:0.7523220825172293\n", - "Epoch 16. Training loss: 0.7405277397305936, Val loss:0.7497162650199105\n", - "Epoch 17. Training loss: 0.7381991356449792, Val loss:0.747188668358643\n", - "Epoch 18. Training loss: 0.7359096907002791, Val loss:0.7447291206710276\n", - "Epoch 19. Training loss: 0.7336504385309983, Val loss:0.7423369465461572\n", - "Epoch 20. Training loss: 0.7314198935746877, Val loss:0.7400043543850221\n" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd8VFX6+PHPSTIpM6lk0iCB0AOENCOCIKCgq65tXVZBsWTxy1rWLsrqrnV3VfSnFLEvsaCiYkdQV6WIq3QSCCGCtBQISSAJKZBMcn5/3EkMyaRAAsMMz/v1mldm7px755kwPHPy3HPPUVprhBBCuBcPZwcghBCi60lyF0IINyTJXQgh3JAkdyGEcEOS3IUQwg1JchdCCDckyV0IIdyQJHchhHBDktyFEMINeTnrha1Wq46NjXXWywshhEtat25dsdY6rL12TkvusbGxrF271lkvL4QQLkkptbsj7aQsI4QQbkiSuxBCuCFJ7kII4YacVnMXQpxctbW15OXlcfjwYWeHIjrA19eX6OhoTCbTce0vyV2I00ReXh4BAQHExsailHJ2OKINWmtKSkrIy8ujd+/ex3UMKcsIcZo4fPgwoaGhkthdgFKK0NDQTv2VJcldiNOIJHbX0dl/q9Mqub+y9hUeXfYoxVXFzg5FCCFOqNMmuecU53Dzlzfz2PLHSN+Q7uxwhDjtlJSUkJSURFJSEpGRkfTo0aPxcU1NTYeOkZaWRk5OTptt5s6dyzvvvNMVITNq1Cg2btzYJcc62U6bE6rLdi0DYET0CNKS05wbjBCnodDQ0MZE+eijj+Lv78999913VButNVprPDwc9zvT09vvmN12222dD9YNnDY992W7lwFgNVuxmq3ODUYI0Wj79u3Ex8dz8803k5KSwt69e5k6dSqpqakMGTKExx9/vLFtQ0/aZrMRHBzM9OnTSUxMZMSIEezfvx+Av//978ycObOx/fTp0xk2bBgDBw7kf//7HwCVlZX88Y9/JDExkUmTJpGamtpuD33+/PkMHTqU+Ph4HnzwQQBsNhvXXXdd4/bZs2cD8PzzzzN48GASExOZPHlyl//OOuK06LlrrVm+azkAueW5To5GiFPAXXdBR8oNtbWwbx9ERkJ7462TksCeVI/Vli1bSE9P5+WXXwbgqaeeolu3bthsNs4991wmTJjA4MGDj9qnrKyMMWPG8NRTT3HPPfcwb948pk+f3uLYWmtWr17N559/zuOPP85XX33FnDlziIyM5KOPPiIjI4OUlJQ248vLy+Pvf/87a9euJSgoiPHjx7No0SLCwsIoLi5m06ZNAJSWlgIwY8YMdu/ejbe3d+O2k+206LlvO7CNvRV78ff2J7dMkrsQHbZvH+zYYfw8gfr27cuZZ57Z+Pi9994jJSWFlJQUsrOz2bJlS4t9/Pz8uOiiiwA444wz2LVrl8NjX3nllS3arFy5kokTJwKQmJjIkCFD2oxv1apVnHfeeVitVkwmE9dccw0rVqygX79+5OTkcOedd/L1118TFBQEwJAhQ5g8eTLvvPPOcV+E1FmnRc+9odf+x0F/5M2MN6mqrcJsMjs5KiGcqKM97OJiSE+HtDSwnrhypsViaby/bds2Zs2axerVqwkODmby5MkOx3t7e3s33vf09MRmszk8to+PT4s2Wutjiq+19qGhoWRmZrJkyRJmz57NRx99xKuvvsrXX3/N8uXL+eyzz/jnP//J5s2b8fT0PKbX7KzToue+bPcyIv0jGdd7HAB55XlOjkgIF2G1wrRpJzSxN1deXk5AQACBgYHs3buXr7/+ustfY9SoUXzwwQcAbNq0yeFfBk0NHz6cpUuXUlJSgs1mY8GCBYwZM4aioiK01vzpT3/iscceY/369dTV1ZGXl8d5553HM888Q1FREVVVVV3+Htrj9j33hnr7mF5jiAmKAWBP2R4GhA5wcmRCCEdSUlIYPHgw8fHx9OnTh5EjR3b5a9x+++1cf/31JCQkkJKSQnx8fGNJxZHo6Ggef/xxxo4di9aaSy+9lN///vesX7+eKVOmoLVGKcXTTz+NzWbjmmuu4dChQ9TX1/PAAw8QEBDQ5e+hPepY/zzpKqmpqfpkLNax/cB2+s/pz0u/f4nz+5xPvzn9mHfZPBkOKU472dnZDBo0yNlhnBJsNhs2mw1fX1+2bdvGBRdcwLZt2/DyOrX6u47+zZRS67TWqe3te2q9kxOgYXz7mF5jiA6MBmTEjBCnu4qKCsaNG4fNZkNrzSuvvHLKJfbOavfdKKV8gRWAj739Qq31I83a3Ag8A+TbN72gtX69a0M9Pst3LyfcEk6cNQ6lFBGWCBkxI8RpLjg4mHXr1jk7jBOqI19VR4DztNYVSikTsFIptURr/XOzdu9rrf/a9SEeP601y3YtY0yvMY2T8MQExUjPXQjh9todLaMNFfaHJvvNOYX6Y7SzdCd55XmMjR3buC0mUJK7EML9dWgopFLKUym1EdgP/FdrvcpBsz8qpTKVUguVUjFdGuVxalpvbxATGCNlGSGE2+tQctda12mtk4BoYJhSKr5Zky+AWK11AvAt8Kaj4yilpiql1iql1hYVFR1XwB9v+ZjYmbFk7c9qt+3y3cuxmq0MDvvtsuWYoBgO1Ryi7HDZcb2+EEK4gmO6iElrXQosAy5str1Ea33E/vA14IxW9n9Va52qtU4NCws7jnDhq1+/YnfZbuasntNu2+b1djB67iAjZoQ42caOHdvigqSZM2dy6623trmfv78/AAUFBUyYMKHVY7c3tHrmzJlHXUx08cUXd8m8L48++ijPPvtsp4/T1dpN7kqpMKVUsP2+HzAe2NqsTVSTh5cB2V0ZZFO3nWlM59mvW7822+0q3cWesj1H1duBxguZpDQjxMk1adIkFixYcNS2BQsWMGnSpA7t3717dxYuXHjcr988uS9evJjg4ODjPt6priM99yhgqVIqE1iDUXNfpJR6XCl1mb3NHUqpLKVUBnAHcOOJCReGRgzFYrKQX57fZjtH9Xb4ree+p2zPCYlPCOHYhAkTWLRoEUeOGH/k79q1i4KCAkaNGtU47jwlJYWhQ4fy2Weftdh/165dxMcbFeHq6momTpxIQkICV199NdXV1Y3tbrnllsbpgh95xBi1PXv2bAoKCjj33HM599xzAYiNjaW42FiV7bnnniM+Pp74+PjG6YJ37drFoEGD+L//+z+GDBnCBRdccNTrOLJx40aGDx9OQkICf/jDHzh48GDj6w8ePJiEhITGCcuWL1/euFhJcnIyhw4dOu7frSPtDoXUWmcCyQ62P9zk/t+Av3VpZK3wUB4MtA5ka8nWNtt9tf0rzF5mIvwjjtoeFRCFh/KQsow4rd311V1s3Nf+lL+1dbXsq9hHpH8kJs+2ZzdMikxi5oWtT0gWGhrKsGHD+Oqrr7j88stZsGABV199NUopfH19+eSTTwgMDKS4uJjhw4dz2WWXtbqO6EsvvYTZbCYzM5PMzMyjpuz917/+Rbdu3airq2PcuHFkZmZyxx138Nxzz7F06VKszebJWbduHenp6axatQqtNWeddRZjxowhJCSEbdu28d577/Haa69x1VVX8dFHH7U5P/v111/PnDlzGDNmDA8//DCPPfYYM2fO5KmnnmLnzp34+Pg0loKeffZZ5s6dy8iRI6moqMDX17fN3++xcsmJwwaGDmRrcdvJ/bsd31Flq+LNjUef2/Xy8KJHQA9J7kJ0wL6Kfewo3cG+iq6Z8rdpaaZpSUZrzYMPPkhCQgLjx48nPz+fwsLCVo+zYsWKxiSbkJBAQkJC43MffPABKSkpJCcnk5WV1e6kYCtXruQPf/gDFosFf39/rrzySn744QcAevfuTVJSEtD2tMJgzC9fWlrKmDFGteCGG25gxYoVjTFee+21zJ8/v/FK2JEjR3LPPfcwe/ZsSktLu/wKWZe83jbOGseCzQuorq3Gz+TX4vmKmgqKq4s5v8/5DueQiQmS4ZDi9NZWD7up4qpi0jekk5ac1iUrmF1xxRXcc889rF+/nurq6sYe9zvvvENRURHr1q3DZDIRGxvrcJrfphz16nfu3Mmzzz7LmjVrCAkJ4cYbb2z3OG3Nr9UwXTAYUwa3V5ZpzZdffsmKFSv4/PPPeeKJJ8jKymL69On8/ve/Z/HixQwfPpxvv/2WuLi44zq+Iy7bc9doth3Y5vD5hmGSt555q8MPpFzIJETHWM1Wpo2c1mVLU/r7+zN27Fj+/Oc/H3UitaysjPDwcEwmE0uXLmX37t1tHmf06NGNi2Bv3ryZzMxMwJgu2GKxEBQURGFhIUuWLGncJyAgwGFde/To0Xz66adUVVVRWVnJJ598wjnnnHPM7y0oKIiQkJDGXv/bb7/NmDFjqK+vJzc3l3PPPZcZM2ZQWlpKRUUFv/76K0OHDuWBBx4gNTWVrVvbrkYcK5ftuQPkFOeQEJHQ4vnMQuMf2tFzYCT3z3I+a5ymUwhx8kyaNIkrr7zyqJEz1157LZdeeimpqakkJSW124O95ZZbSEtLIyEhgaSkJIYNGwYYqyolJyczZMiQFtMFT506lYsuuoioqCiWLl3auD0lJYUbb7yx8Rg33XQTycnJbZZgWvPmm29y8803U1VVRZ8+fUhPT6euro7JkydTVlaG1pq7776b4OBg/vGPf7B06VI8PT0ZPHhw46pSXcUlp/ytqq3C/9/+PDb2Mf4x5h8tnr998e28mfEmpdNL8VAt/ziZvWo2d351J/vv20+Y5fjG2wvhamTKX9fTmSl/XbIsYzaZ6RnUs9URMxmFGQyNGOowsYNcyCSEcH8umdzBKM3kFOe02K61JrMwk4RwxyUZkAuZhBDuz2WT+8DQgeSU5LQ4051bnkvZkbJW6+0gPXdx+nJWGVYcu87+W7lsco+zxlFRU0HBoYKjtjecTE2MTGx13zBLGN6e3nKVqjit+Pr6UlJSIgneBWitKSkp6dSFTS45WgZ+GzGztXgrPQJ7NG5vSO7x4c0nrvyNh/IgOjBaeu7itBIdHU1eXh7HOyOrOLl8fX2Jjo4+7v1dNrkPtA4EjOQ+rs+4xu0ZhRn0Du5NoE9gm/vLvO7idGMymejdu7ezwxAnicuWZaL8owjwDiCn5OiTqpmFmW3W2xv0DOopPXchhNty2eSulDImEGsyx0x1bTW/lPxCYkTr9fYGMYEx5JfnU1dfdyLDFEIIp3DZ5A5G3b1pct9StIV6Xd+hnntMUAx1uq7LJkQSQohTiWsn99A4cstzqaypBNqfdqApGQ4phHBnLp3cG06q/lLyC2CcTDWbzPQJ6dPuvnIhkxDCnbl0cm+cQMx+UjWzMJP48Hg8PTzb3Vd67kIId+bSyb1ft34oFFuLtzZOO9CRk6kAwb7BWEwW6bkLIdySSyd3Xy9feof0ZmvxVvZW7KWkuqRD9XYwRtvEBMWwp1yuUhVCuB+XTu5gn0CsJOeYTqY2kAuZhBDuyuWT+8DQgeQU57Bh7wYAhoYP7fC+siKTEMJduXxyj7PGUW2r5sttX9IzqCchfiEd3rdnUE8KKwqpqas5gREKIcTJ5/LJfWCoMRzyx9wfj6kkA8ZJVY1mU+GmExGaEEI4jcsn94bhkECbC3Q4klGYAcCbGW92aUxCCOFs7SZ3pZSvUmq1UipDKZWllHrMQRsfpdT7SqntSqlVSqnYExGsI+GWcIJ9g4FjO5kKcNmAywAY13tcOy2FEMK1dKTnfgQ4T2udCCQBFyqlhjdrMwU4qLXuBzwPPN21YbZOKdVYmmlrgQ5HogKiADB5mro8LiGEcKZ2k7s2VNgfmuy35ku5XA401DYWAuOUUqrLomxH75DeeHl4NfbgO8ribQFonJtGCCHcRYdq7kopT6XURmA/8F+t9apmTXoAuQBaaxtQBoR2ZaBt6RXUC1u9jbcz3j6m/SwmI7lX1VadiLCEEMJpOrQSk9a6DkhSSgUDnyil4rXWm5s0cdRLb7FQo1JqKjAVoGfPnscRrmP3nX0foX6hpCWnHdN+ZpMZgMpa6bkLIdzLMY2W0VqXAsuAC5s9lQfEACilvIAg4ICD/V/VWqdqrVPDwsKOK2BHrGYr00ZOw2q2HtN+UpYRQrirjoyWCbP32FFK+QHjga3Nmn0O3GC/PwH4Xp+oJdZ37oSrr4YuWOTXz8sPkLKMEML9dKQsEwW8qZTyxPgy+EBrvUgp9TiwVmv9OfAf4G2l1HaMHvvEExbx3/4GH3wAQUHw6qudOpSnhye+Xr5SlhFCuJ12k7vWOhNIdrD94Sb3DwN/6trQWvHss/DFF5Cf3yWHs5gsUpYRQrgd17tCNToabr8dvvoK9nR+ul6zyUyVTcoyQgj34nrJHeDmm42fr7zS6UNZvKXnLoRwP66Z3GNj4ZJL4LXX4MiRTh3KYrLICVUhhNtxzeQOcNttxoiZhQs7dRizySwnVIUQbsd1k/v48dC/P7zwQqcOI2UZIYQ7ct3k7uFh9N5//hnWrz/uw5hNZinLCCHcjusmd4AbbgCzGebOPe5DWEwWKcsIIdyOayf34GCYPBnefRdKSo7rEDLOXQjhjlw7uYNRmjl8GNLTj33fH3/E/OXXVElyF0K4GddP7gkJMHw4PPoovPUWdGRKm0OH4L77YPRoLDk7qaqt4kRNhSOEEM7g+skdYMQIqKw0avAXXwy//OK4ndZGCWfgQHjuORg+HEsNaAWHbYdPbsxCCHECdWg+91Pegw9CRATU1cHTT0N8PNx7L1x6KezeDbt2GbdVqyAjAxIT4dNPwccH89QkwJjT3c/k59S3IYQQXcU9krvVCg88YNyfMgWmT4ennjJuDcLCwMfHuD9pEgwbBjt3Yqk1NlXWVB7zfPBCCHGqco+yTFMREcbJ1bvuMh7fey9UVMD+/bBhA8yYYXwBAAQGYrYndxnrLoRwJ+7Rc3fkoYege3dISwOLseISVitMm/Zbm8BALDXGXRnrLoRwJ+6b3JsnckdMJizKG6iRse5CCLfifmWZY2T2MXr1UpYRQriT0z65W3wCACnLCCHciyR330AAKcsIIdzKaZ/czX5GcpeyjBDCnZz2yd1iCQakLCOEcC+nfXI3+4cA0nMXQriX0z65ewYG42OTmrsQwr2c9sm94UKmytoKZ0cihBBdRpJ7UBDmWqg6fMjZkQghRJdpN7krpWKUUkuVUtlKqSyl1J0O2oxVSpUppTbabw+fmHBPgMBALLVQWVXm7EiEEKLLdGT6ARtwr9Z6vVIqAFinlPqv1npLs3Y/aK0v6foQT7CGskx1ubMjEUKILtNuz11rvVdrvd5+/xCQDfQ40YGdNPaZIauOSM1dCOE+jqnmrpSKBZKBVQ6eHqGUylBKLVFKDWll/6lKqbVKqbVFRUXHHOwJ0VCWqZHkLoRwHx1O7kopf+Aj4C6tdfMaxnqgl9Y6EZgDfOroGFrrV7XWqVrr1LCwsOONuWs19NxlnLsQwo10KLkrpUwYif0drfXHzZ/XWpdrrSvs9xcDJqWUayxr1FBzt1U7OxIhhOgyHRkto4D/ANla6+daaRNpb4dSapj9uCVdGegJ01CWqZMFsoUQ7qMjo2VGAtcBm5RSG+3bHgR6AmitXwYmALcopWxANTBRa61PQLxdr6Eso484OxIhhOgy7SZ3rfVKQLXT5gXgha4K6qTy8cFS70mlrkFrjf0PECGEcGlyhSpg8fBFKzhsk9KMEMI9SHIHzF6+gIyYEUK4D0nugMXLWEdV5nQXQrgLSe6A2VsWyRZCuBdJ7oDFxx+QOd2FEO5DkjtNFsmWsowQwk1IcgfMZlkkWwjhXiS5AxazfZFsKcsIIdyEJHfAYl8ku7Kq1MmRCCFE15DkDpgDQwGoOnTAyZEIIUTXkOQOWAKNCSwrKyS5CyHcgyR3wBxkJPeqSllHVQjhHiS5A15BIXjbpOYuhHAfktzhtzndZZFsIYSbkOQOTRbJPuTsSIQQoktIcoffltqTce5CCDchyR1+K8vI9ANCCDchyR3A1xezDapkkWwhhJuQ5A6gFJZ6L1kkWwjhNiS525mVN1X1ski2EMI9SHK3s+BNJTXODkMIIbqEJHc7i6cvlcrm7DCEEKJLSHK3M3v6USXJXQjhJiS521lMZio969FaOzsUIYTotHaTu1IqRim1VCmVrZTKUkrd6aCNUkrNVkptV0plKqVSTky4J47FZKHeA47UyUlVIYTr60jP3Qbcq7UeBAwHblNKDW7W5iKgv/02FXipS6M8Ccz2RbJlqT0hhDtoN7lrrfdqrdfb7x8CsoEezZpdDrylDT8DwUqpqC6P9gSy+AYAUFl50MmRCCFE5x1TzV0pFQskA6uaPdUDyG3yOI+WXwCntMZFskuLnByJEEJ0XoeTu1LKH/gIuEtr3XxuXOVglxZnJpVSU5VSa5VSa4uKTq0k2rhItiR3IYQb6FByV0qZMBL7O1rrjx00yQNimjyOBgqaN9Jav6q1TtVap4aFhR1PvCeMxb8bAJVlxU6ORAghOq8jo2UU8B8gW2v9XCvNPgeut4+aGQ6Uaa33dmGcJ5w5wEjuski2EMIdeHWgzUjgOmCTUmqjfduDQE8ArfXLwGLgYmA7UAWkdX2oJ5YlMBSAykMlTo5ECCE6r93krrVeieOaetM2Gritq4JyBkuQUSaS0TJCCHcgV6jamYON5F5VJeuoCiFcnyR3O0u3CAAqq8ucHIkQQnSeJHc7c6AVgKojFU6ORAghOk+Su53JyxtTHVTWSHIXQrg+Se5NWGyKyhpZJFsI4fokuTdhrveURbKFEG5BknsTski2EMJdSHJvwoKJSi3zuQshXJ8k9ybMypsqLYtkCyFcnyT3JiwevlRS6+wwhBCi0yS5N2H29KXSQxbJFkK4PknuTVhMZqq8NNgkwQshXJsk9yYsJguVJuDQIWeHIoQQnSLJvQmzjz9VJqBcJg8TQrg2Se5NWHwDqPQGXSaThwkhXJsk9yYs5mDqPKCmTBbsEEK4NknuTZjNQQBUySLZQggXJ8m9CYslBJBFsoUQrk+SexPmAHtyL5fkLoRwbZLcm7A0LNhxSNZRFUK4NknuTVgCugGySLYQwvVJcm/C7O0PQFWVDIUUQrg2Se5NWLwtAFRWy0VMQgjXJsm9CYvJntyPyPQDQgjX1m5yV0rNU0rtV0ptbuX5sUqpMqXURvvt4a4P8+Qwm8wAVB2RRbKFEK7NqwNt3gBeAN5qo80PWutLuiQiJ2osy9TKItlCCNfWbs9da70COHASYnG6hp57pSySLYRwcV1Vcx+hlMpQSi1RSg3pomOedN6e3nhpRZUkdyGEi+uK5L4e6KW1TgTmAJ+21lApNVUptVYptbao6NScv8WiTVTWVMKePc4ORQghjlunk7vWulxrXWG/vxgwKaWsrbR9VWudqrVODQsL6+xLnxBm7WWsxvTQQ84ORQghjlunk7tSKlIppez3h9mP6bJz5lpCIqg0e8FBuUpVCOG62h0to5R6DxgLWJVSecAjgAlAa/0yMAG4RSllA6qBiVprfcIiPsEsfoFU9o2B//etsSJTYKCzQxJCiGPWbnLXWk9q5/kXMIZKugWzyUxVpDcc2QmffQbXXefskIQQ4pjJFarNWLwtRlmmVy947z1nhyM64GD1QYa/PpzElxN58ocn2bB3A3X1dc4OSwin6shFTKcVs8nM/sr9cPXV8NxzUFwMVofnh8Up4vud37MqfxUAmYWZPPj9g1hMFvqE9OGLSV/QK7iXkyMU4uSTnnszFpOFqtoqmDQJbDb4+GNnhyTasSp/FSYPE/8+799s+MsG3rnyHeLD49m0fxMTF050dnhCOIUk92YsJosxzj0xEQYOhAULnB2SaMfq/NUkRyXzt3P+RlJkEtcMvYZF1yxidM/R/Jz/M1/+8qWzQxTipJPk3oyPlw+FlYX8Z8M8mDgRli2DvXudHZZoRV19HWsL1nJWj7OO2m41W/nmum8YGj6UKZ9Poajy1LxorjN+zv2ZwXMHs6t0l7NDEacgSe7NhPiGUK/ruemLm7i5bzbVnho+/NDZYYlWZBVlUVlb2SK5g/FF/c6V73Dw8EGmLpqKC4/QdeifP/yT7OJsnvzhSWeHIk5BktybuXP4nTw57kluH3Y7r+z4gDPv8CXri3nODku0YlWecSL1rOiWyR1gaMRQ/n3ev/l066ekb0w/maF12JxVcxjy4hAyCzOPab8QX2NBdzlhLByR5N6M1Wxl+qjpzL5oNl9P/pqiQC9Sz8rginnns69in7PDc2vvbnqX0BmhzFk1p8O97FX5qwj1C6VvSN9W29w94m7OjT2XO7+6kx0Hd3RVuB2SX57f7nuZtWoWW4q2cPZ/zmbpzqUdPvbust0AUpYRDklyb8MFfS8g40/f0bMMPsv9lpSXkvhh9w/ODsttzfx5JgeqD3DHV3cw7q1x/FLyS7v7rMpfxbAew7DPgOGQh/LgjSveQKEYOW8kG/Zu6MqwW5Vblkuvmb24/pPrKa4qbrVd+ZFy+ob0JSogivFvj+eZH5/p0Jfb1uKtAGze73AdHXGak+TejsjBw1i5NJbJGaCKSxj9xmiu+ega8svznR2a26moqaB/t/48e/6zrN+7noSXEnhi+RPU1NU4bH/oyCGy9mcxrMewdo/dM6gnVw25in0V+xg5bySLflnU1eG3sKZgDXW6jvmb5pO+wXFJqLiqmKKqIm5JvYX1U9dz5aAruf/b+5nw4QTKj7S+lu+B6gMUVRVh8jCRVZTlducTROdJcu+AsC+X8XbZefzyAvz9J28+3vwhA+YMYPxb49lWss3Z4bmFyppKckpymBg/kXvPvpfs27K5PO5yHl72MAkvJTjs+a7buw6Ndngy1ZGnxj/FAyMfoF+3flz63qVM/3Y6tnpbV7+VRhn7MgDoG9KXtOQ0h22y9mcBEB8eT4BPAB9M+ID/d8H/49PsT7ng7Qta7fHnFOcAcH7f8yk/Uk5eeV6rcSzftZwrFlzBHUvu4PpPrmfcW+MY+tJQ/rr4r23+RSFcmyT3jujVC777DktGNk94jmfLLBt9imx8t/M7hr4Yz0PfPST/STpp/d711Ov6xl54VEAU7094n/P7nE9OSQ5PrXyqxT4NJ1M70nMH43zKU+OfYvX/rWZqylSe/vEXAFgVAAAcwUlEQVRpRqeP5uGlD5+Qf7+MQiO5lx4uJdQv1GGbhpLKkHBjjRulFPeMuIeB1oGsyl/Vao+/oSQzYdCEo47jyBXvX8FnOZ8xd81clu9eTnVtNVn7s5i7Zm6rxxeuT5L7sejXDxYtos+bn7P0QzN3/w8u/FXx5Mon6fV8L8a+MfaYRzwIw5qCNQCc2f3Mo7bPunAWHsqD6tqWq2Otyl9Fv279CDU7Tpyt8fXy5ZVLX+HtP7zNmoI1PLHiiROS5DIKM/BQHpRUl7R6Mj6rKIsgnyB6BPQ4avtF/S5CoZgY7/gK25ySHEweJi4ZYCxd3FpyzyvPo/RwKRf3u5i99+5l9127+d+U/9G3W1/iw+Nb/YtCuD5J7sdKKbj0Uqxrt/Bc8FV8uqI7m+dqBhTVsXz3cpJeTuJPH/6JFbtXSB30GKwpWENMYAwR/hFHbR8UNojLB17OwuyF1NbVHvXcqvxVHS7JODI5YTIX9LmAbn7dujzJlR0uY1fpLs7vcz5Aq1/6m/dvZkj4kBYnhEf2HIlGU1hZ6HC/rcVb6R/anzBLGN0DurO5yHFy/yn3JwAeGfsI4Zbwxu3x4fHU63qsZpk3yV1Jcj9eUVHw/vvwyy8Mfu5t/vuxP9N/gFs3mvhuy5eMeWMMQ14cwpXvX8mvB351drSnvNX5q1str/w5+c/sr9x/1EnQvPI8Cg4VdCq5A5zX+zwOVB9o8cXRWQ3JfHLCZOC3Ek1TWmuyirKID4tv8VxSZBJAqyN7ckpyiLPGATAkbEirPfef8n7C18u38XgN4kLj2Fay7YSecxDOJcm9s7y8YPJkrGuyeLL7dbywP5W8f1Xz+mIvSvft4pOtnzDwhYFMXDiRJduWyH8mB0qqSthxcEeLkkyDC/tdSJR/FPM2/nYxWXsXL3XUqJ6jAPgx98dOHae5hmQ+NnYsMYExDnvuhZWFHKg+0Fhvb6p3cG8CfQLZuG9ji+dq62rZfmA7A0MHAkYvPLso2+E0xz/l/cQZUWfg7el91PY4axy19bXsPLjzuN6fOPVJcu8qERHw1lvw44+Y129iSvIUMl6s546f4cbt/nyb9QUXv3sxPZ7rwTnzzmHxtsXU63pnR31KWFuwFoAzezhO7l4eXtyYdCOLty2m4FABYPT0vT29SYxI7NRrJ0cl4+flx497uja5ZxZm0s2vGz0CepAQkeAwuTf0tuPDW/bclVIkRSaxYV/LnvuOgzuw1dsae+7x4fFU26rZWXp0oj5iO8L6vesZET2ixTEa9m04MSvcjyT3EyE+Hl5+mbA7H2TWV/D6skAKnqji4w88sO6vZGXuSn7/7u/p+XxP7lxyJz/s/uG0Xlxidf5qAM6IOqPVNmlJadTret7c+CZg1NuTIpPw8fLp1Gt7e3ozrMcwVuau7NRxmssozCAxIhGlFAkRCWQXZ7cYr98wDHJIWMueO0ByZDKZhZktPhs5JcYwyKY9d2h5UnX93vXU1NUwIqZlch9oNfaV5O6+JLmfSLfeCjNmwPr1eG/O5g+XP8Dytzx4/Ht4abHizLx6XlnzEqPfGE3gU4FcvfBqFm5Z2ObFK+5oTcEa4qxxBPkGtdqmf2h/RvcazbyN87DV2xzOBHm8RvUcxYa9G4ypnrtAXX0dmwo3Nf5VkRCRgK3eRnZR9lHtNu/fjNVsPepEZ1NJkUlU1lay/cD2o7Y3JOSGBD04bHDj8Zr6Kc84meqo5x7sG0ykf6Qkdzcmyf1Eslph2jTjZ1wc/PvfWDO3848ht3BzwhQ+WQBF/6rlms0eVNVW8UXWx/zpwz8R+nQo/Wf357Flj5FZmOnWo2601qwpWNNqvb2pKclT2H5gOy+vfbnVmSCPx6ieo6jTdY2rOXXW9gPbqbZVkxhpJPeGJN+8NJNVlMWQsJYjZRokRyYDtKi75xTnEGGJINg3GAB/b396B/cmqyjrqHY/5f1Er6BeRAVEOTx+nDWO7OJsh88J1yfJ/WQLD4cXX4TXXoO8PAK++4FZxanM+AZ2PGNjeTqM3O/D9oPbeXT5oyS+nEj357pz1YdXMXHhRLcbR59/KJ99Ffs6lNwnDJ5AgHcAf//+70DnT6Y2GBE9AoVi5Z6uKc00nExtSOr9Q/vj4+lz1L9d40gZB/X2BoPCBmHyMLWou28t2dpYM28wJLzliJmfcn9yWJJpEBcax9birW7deTidSXJ3Jg8PGDUK64dfMu3yp4lcvJzRU//Fwq+DmPENZLwI8761cG6+N19mf8b7We+T+HIi/Wb346bPb2J+5nxyy3Kd/S46paHePqzHMFi4ECIjYehQOPtsOO88GD8eRo6EL77A7OnLpPhJlB0pa3cmyGMR5BvE0IihxzRi5pZFt9B7Vm+Hi4Bk7MvAy8OrsVzi5eHFkPAhZO7/LbnnledRfqS81Xo7GOcD4sPjHfbcmyf3+LB4thZvbazr55blkn8o32FJpkGcNY6Dhw9SVOV+C5kIWSD71GC1wv33G/dHj8Y6dSrT5syB67uT8NNPpC34hv1lNTx5DoT6BLN2kI2Pyt7jPxv+A0B3/+6Mjh3NyJiRjIwZydCIoXh5uMY/7Zr8NXh5eJEYkQB3/AEKC6FbN/D3h+pq2LkT8vPhssugRw+mTBrNq/7GVAIl1SXtX4RTVwelpXDgAOzYAd9//1uprIlRMaN4K/MtbPW2dn93B6sP8vqG17HV2/jnin8y66JZRz2fUZhBnDXuqJO9CREJLNm2pPFxWyNlmkqKTOLLbV+itUYpRXFVMSXVJY0nUxvEh8djq7exrWQbQ8KHtFlvb9B0xExrdX/hulwjA5xurFZ47DHj/l/+AloTfu+9PP/88zAoCpYXUFdexb0XwKwRYC2u5Ieqr1iw2Vjv1eRhIrV7KqN6juKsHmcxrMcwogOj25wW11nWFKwhISIB36++NZYzvOoqmDv3t+RbXAwvvwxhYbBkCWfOXsiZ18MackifNIhp9cOhb1/jC2HFCqPsVVwMu3ZBbi4cPtzyRd97z/gynTix8XVG9hzJi2tfZFPhJpKjktuM+c2MN7HV2/BQHg5PwmYUZjCm15ijtiVGJPLGxjcorCgkwj+isT7uaIx7U8mRyaRvTGdfxT6iAqIaT4C26Lk3GTEzJHwIP+UaFy811P0daZrcR/ca3WYcwvW0m9yVUvOAS4D9WusW3QxlZIxZwMVAFXCj1np9Vwd6WlMKHnzQuCo2LQ1CQvDcuJG/P/E3enzzX9JyaggtqWZPEEy/yMSCuFry9mxmVv4aarRx0ZTFZOGs6LMYET2ClKgUzog6g55BPZ2a8Ot1PWsK1nBN/CR4/Ano3RvmzweT6bdGViv83aix85e/oEpLWXxbGunffEpadSjU7YGlS6HSnmQDA42T197eRmIfPx4uvdRI/h4eRmLfuRNuvx3uvhvGjQObjVFn9AMz/Pj6wyTrM6G2Fu68s0UPX2vNS2tfYnj0cGKDY/kk5xPm2uY29tIPVB8grzyvxfj7hIgEADbt30SEfwSb928myj+Kbn7d2vwdNV6pum8DUQFRjbNBNoyUaTDQOhBP5dn4pfFT3k+kdk9tcfFSUzFBMZhNZhkx46Y60nN/A3gBeKuV5y8C+ttvZwEv2X+KrtQw8qbBGWdgff1dpqWnw403woED9PrxR+a8PpuUPRmkbTxEwBHIiIR//87MZz0r2bJjNct3LqMO4+KpAO8AkiKTSIxIpE9IH3aU7uDeEfcSGxx7Ut7StpJtlB8p58yDZlizBl599ejE7khwMNZZrzEt/Wzji85qBa1h+3b4+GOYMsXYVlwM6em/tWlwzTXGz8xMePtt4zXLy+n53XfE3A0rcxfx14X2aQ6ysozzAB6/nZr6fuf3/FLyC28FpRH++WoWpBxg0a3j+WNhNygtJSP4IKTQosc8NHyo8bJ7NzDeNNAYKdNOrx1+O87GfRu5uP/FbC3eio+nD72Cjl5az9fLl37d+rF5/2YO2w6zfu967hp+V5vH9lAeDAwdKMndTbWb3LXWK5RSsW00uRx4Sxun3H9WSgUrpaK01nu7KEbRmqYJPywMBg7EetllRsKfPwF27WLYqlW8/sEbjNyaQ9rGCiw1cP/58MJZMKTYA9uRnaTnrqZSHwHghdUv0Du4N0PCh9AnuA+FlYVMSZ7C8OjhBPgEdGn4jTNBvrMUoqPh+uuP/X2D8ZdN//7wwAOtt2kuIQGeecbYx/4FOXLZ7fwQvRJ9z4eo666DTz4xTua++qpxkreoiJfm30noYQ/+9M90THXQvb/izfr1/HFXPzhwgIxoY171xFsfhymlcMUVUFRE2NdfE1XrS+ash6hfeD9bHoSpB/vC/v8Ho0ZBSorDL7ZAn0D6hvRtHDGTU5JD/9D+eHp4tmgbHx5PZmEm6/eup7a+ts16e4M4axw/5/3cbjvherqi5t4DaDpkI8++TZK7MzRNar17w7nnYr3pJiPhp18Oe/bwyJyn6fnNt6TlmbDm76W+vo61UTD3LEV371B2RteTVbaGxfpL6tG8n/U+ANGB0fQJ6UNtXS2XDLiElKgUBoQOoFdQL4fJpj1r8tdg8fBl8DcbYfYc8Onc1abHpcnva1Svc1iw5X32DOpOr23bjBLRPfdAcjL070/B/l/59K+13F3SC9+3noTcXK4blMezG16kcOY3RBz2JGPOeCL0diJy8uDqq8FiaSwZJUzxISPaxK7gWqpMMGRHObx4nxGHxQLTp8N994Gv71EhJkclszF/HbzwAlv3/0DigHMcvpX48Hg+zv6Y73d+D9DmMMgGcdY4FmxeQHVtNX4mv+P9LYpTUFckd0dFW4cDZ5VSU4GpAD179uyClxYd0jThDxiANSnJSPZpaeDvj0d2NsOeeophH3wAA0Phf1WQW8heCzx3NsQXK/J7h7I1GpYeXEeeZ2XjaAww/rzvHdybAaED6NetHxGWCH49+CtTkqeQHJWM2WR2GNbqgtWkHPTBMyLYKKc4WcMkYiv3rKRXwrVw3XVw0UXGbe1aXrs2mjqPPG5+6jvoZgzDvKEom6fXz+GdTe9wz4h7yOjuQaJlJPy6GL75Bh55xCg53X03iRd5MfPnWWycdh3UvE38a5+DV6zxbzN/PvzjH/DKK8bPtDTw9ITvviNp2VYWhu+k5Onb2TENrn73G1h3N/zxj8aQ0aIiyM4mfkMeGk36oifo7RdE5KJlxhoE/ftDkOOrf+OscWg02w5sazwvINxDVyT3PCCmyeNooMBRQ631q8CrAKmpqXLlhLM0L1kkJxsjVFJTf6tRl5cT9dBDPPPCC3DuWNDB8G0OxbvzmZcIl/wCJWaYfbYHC+Pq8csrZG9xOT94fk8FRoknfaOxAEakfyTRAdHY6m2M7jWaQWGD6BXUi40F67k1qwbu+wf4Ob/XGB8eT6BPICv3rOTahGuNjVYrLFlC7bzXedVjFr+L/B19u/02vn5Q2CCG9RjGGxvf4PZht5NVlMWdfe40EvNFF8GZZzbW/hMKvqamvoaPB9bDJvu0AT6B8PzzRpmoXz+jVPSXv8C//w1lZVBaSnJKAFwGH796F3W/zmRgSH946SWYOdN4nTpj7pl4K/BX2GGp4ZrMGnh80m9vzmSCkBAIDjZOOlsskJxM3F+uAIwRM5Lc3UtXJPfPgb8qpRZgnEgtk3q7C2qe8AMDjV5nz55HnZS07tvH/bNmwV0pUFTEoIVvMWzPKtLy/bDur0BXHyHbCi+eCUP3Q3GYmZ3dbSwL38KvvlVkFmZQ3+QPu/3dfCi+fgKnwpIRnh6ejIge0fJiJquVLy4dQMEH+3gx9eUW+92QeAO3Lb6N97Pep6au5uiRMk1+rwl1RvL8dOun9AzqSaBPYIs2XHEFLFkCt91mjM+/9lqSZj4Bc/uwoN64CCrumXR4aQDcfLMx+ueyy+Cvf6XfwP54vzmQmvoaRvzfY/DYlbBtG7z+OixeDLGxRqmuvBy+/RaWL6d/VBgKJSdV3ZHWus0b8B5G/bwWo5c+BbgZuNn+vALmAr8Cm4DU9o6pteaMM87Qwg0UFWk9Y4bxs75e6717tb71Vq1B6wsv1HrqVK0vuEAXxYTqGWej95nReQHolTHoGy9H8yh6xtePOPtdNHp82eNaPar0weqDR20f/9Z4HfNcjK6tq22xT0lVifZ+wlvHPBejeRS9qXCTw2MfsR3RpsdNmkfRF82/qO1Amvxe6+vrdfgz4Vo9qjSPossOl7Vo0yDhpQTNo+i1+WsdHqvR1Klae3hovXWr7j2zt560cFLb8YhTBrBWdyDHdmS0zKR2ntfAbcf/9SJcWvMef2SkcQFWbOzRPf7iYqbNmwfpV0BlJT1ycxm4YB6Dv/mMND/gAqdE38KonqPQaPrP7k9cWBwxgTH4efnx7Y5vmT5yusOrV7v5deOygZexcMtCvD29W1w92sDb05tBYYPILMxs98rUpr9XhTHe/Ztfv6F7QHfHPX67/t36k12UTfeA7g6P1ej2241RQIsXExceJz13NyRzy4iu13Q2zKbb7r8fBgwwavyXXYZ19utMu2IG1j//1XmxNjOq5yjG9BpDcXUxe8r2sDp/NfM3zQfA5Nn6GPwbE28EjGkRyo6UtdquoWTT1pwyjjTMENnaF0eDft36UVtfy/zM+W0fMD4ehg+H114jLnQgOSU5sniMm5HpB4TztDcW3QlMniYWXrWQ9A3ppCWnYTVbKa4qZt6Gefw5+c+t7ve7fr8j2DeYgkMFpG9IZ9pIx++rYbKzHoE9jimuhitVm0870Nx9Z99HqF9oxxb8vukmuOkm4sovpaq2irzyPHoGySg2dyE9dyGasZqtTBs5rXFSMqvZyv0j729zkjIvDy9W37Saf533rzYTa502RrasK1h3TDE1rFJVcKiA4qriDsfepquvBn9/4r43TtRKaca9SHIXoov0D+3Pg+c82GZiveOsO5gxfgZTUo5tXH//0P6kJaXxWc5npG9I72yoBn9/mDSJuI+WA5Lc3Y2UZYQ4iRp61sdjxvkzGGQd1LGSS0fddBNhr71GiJIJxNyNJHchXERnvhhadeaZqIQE4kp+leTuZqQsI8TpTCnjpOruSrL3bnJ2NKILSc9diNPd5MnEfHYX+44U8/TKp4kNjiXEL4Rg32CCfYMJ8gki2Df4qJWlxKlPkrsQp7uQEIoHxwI7mP7d9Fab+Xj6EOgTiEYTHRBNqDmUIN8gfDx9yC3L5azos4j0j8RDebCuYB2XDLiEmKAYAn0CCfAOIMAngADvAPmSOEkkuQsheMxjHD2+3cFlv4DSUOoL/0mG9BS4IhvOLPGhzN/E8h7lrAo7QmhRBdXaj32mevK8qijzrGXlnpVHzRH77uZ3Hb6WycNEgLc/ZpOF2vpaogOj6ebXDX9vf0yeJnLLcjkj6ozGdV03Fm7kgj4XEOkfib+3P/7e/li8LVhMlsaffiY/PJRUmZtSxuwBJ19qaqpeu3atU15bCNFMw8pVEycayxPm5lL8+mzSd39GWsA5WONSoKKC4oyfSPfeQtqBnsaQz4oKiksLSO9XwQ0bwc8Gu4LhzSQ4fzt4AuU+8P4Q+CAeLtwGSfvgkA/8FA3ru0P/gx6E1nhR4QN5Zhul3vV41StsHseWm7y0B4FeFgJ8ArCYzJi8fDhUW0HvkD6EmEPx8PBgT+kekiOTCTWHYjaZqdf1rN+3nvG9xxPhH4HZZMbPyw8/k99R9/28jMfent5OX4tYKbVOa53abjtJ7kIIhxwtVdjWthtvNGYT3bPHeHzJJcZatpWVFM9/lfQt75LW63Ks5/zO2Pb9ItLLlpNmOhNr36FQVUVx9jrSLdtIK+pBsL+VPaW7mde7lEt2eeNdBxXU8GYizEuBqzbD6N1Q6Q3/7QPf9oXhuTCgxNi2OQxywiC6DAKPQEEAlPqBqQ5sHqCPM0eblQ9+Ht5440ltzWHCLGEE+Abh5elFSW05fYN6E+QXjJ+PP9pD8evBHZzR/QxC/ULx9fKlTteRW5bLI2Mf6djFZs1IchdCnDqO9YuiYVvzx/X1FG/PJP3daaRd/BDWoCiorqZ4dzbpy2eSlpSG1S/U2LboA9LzvySt23lYk0dS/L9vSa/+iTSSCO0Vx5HDleTt2sQbIbu49GA4vv5BVNdWM697Ia8l1DJ5k+L8X6HaU7OoPyyKg3G/QlIhVHvB6h6wtgcM3g/R5bCtG+zsBhGHIKAGqk1wwBeqvTGWL2r2ZTJjxCNMu+DRY/5VSnIXQpze2vuicNQGKM7NIf3te0i77jmsMQPBZjO+UN57gLSLH8Qa0gMOH6Z4z1bSlz5P2hlTsFrCKP7obdJ//Yi07hdjHXm+0Wb5EtLLV3CjzzCCBiRw+Egl+b+sY6HXL9zy+0ew3v/oMb8tSe5CCHEyHe9fJ8dIkrsQQrihjiZ3GTskhBBuSJK7EEK4IUnuQgjhhiS5CyGEG5LkLoQQbkiSuxBCuCFJ7kII4YacNs5dKVUE7D7O3a1A66sEn/okfudx5djBteN35djh1Im/l9Y6rL1GTkvunaGUWtuRQfynKonfeVw5dnDt+F05dnC9+KUsI4QQbkiSuxBCuCFXTe6vOjuATpL4nceVYwfXjt+VYwcXi98la+5CCCHa5qo9dyGEEG1wueSulLpQKZWjlNqulGp9qfZThFJqnlJqv1Jqc5Nt3ZRS/1VKbbP/DHFmjK1RSsUopZYqpbKVUllKqTvt210lfl+l1GqlVIY9/sfs23srpVbZ439fKeXt7Fhbo5TyVEptUEotsj92pdh3KaU2KaU2KqXW2re5ymcnWCm1UCm11f75H+EqsTdwqeSulPIE5gIXAYOBSUqpwc6Nql1vABc22zYd+E5r3R/4zv74VGQD7tVaDwKGA7fZf9+uEv8R4DytdSKQBFyolBoOPA08b4//IDDFiTG2504gu8ljV4od4FytdVKTIYSu8tmZBXyltY4DEjH+DVwldoPW2mVuwAjg6yaP/wb8zdlxdSDuWGBzk8c5QJT9fhSQ4+wYO/g+PgPOd8X4ATOwHjgL40IUL0efqVPpBkRjJJHzgEUYq3C6ROz2+HYB1mbbTvnPDhAI7MR+TtKVYm96c6meO9ADyG3yOM++zdVEaK33Ath/hjs5nnYppWKBZGAVLhS/vayxEdgP/Bf4FSjVWtvsTU7lz9BM4H6g3v44FNeJHYxlob9RSq1TSk21b3OFz04foAhIt5fEXldKWXCN2Bu5WnJXDrbJcJ8TTCnlD3wE3KW1Lnd2PMdCa12ntU7C6AUPAwY5anZyo2qfUuoSYL/Wel3TzQ6annKxNzFSa52CUUa9TSk12tkBdZAXkAK8pLVOBio51UswDrhacs8DYpo8jgYKnBRLZxQqpaIA7D/3OzmeVimlTBiJ/R2t9cf2zS4TfwOtdSmwDOPcQbBSysv+1Kn6GRoJXKaU2gUswCjNzMQ1YgdAa11g/7kf+ATjy9UVPjt5QJ7WepX98UKMZO8KsTdyteS+BuhvHzHgDUwEPndyTMfjc+AG+/0bMGrZpxyllAL+A2RrrZ9r8pSrxB+mlAq23/cDxmOcGFsKTLA3OyXj11r/TWsdrbWOxficf6+1vhYXiB1AKWVRSgU03AcuADbjAp8drfU+IFcpNdC+aRywBReI/SjOLvofx8mOi4FfMGqnDzk7ng7E+x6wF6jF6BFMwaidfgdss//s5uw4W4l9FMaf/ZnARvvtYheKPwHYYI9/M/CwfXsfYDWwHfgQ8HF2rO28j7HAIleK3R5nhv2W1fB/1YU+O0nAWvtn51MgxFVib7jJFapCCOGGXK0sI4QQogMkuQshhBuS5C6EEG5IkrsQQrghSe5CCOGGJLkLIYQbkuQuhBBuSJK7EEK4of8PMJokt/R7ebsAAAAASUVORK5CYII=\n", - "text/plain": [ - "
" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "nn.fit_epoch(X_train,y_train,X_test,y_test,lr=0.001,epochs=20,bs=512)" ] }, { "cell_type": "code", - "execution_count": 200, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Epoch 1. Training loss: 0.671948775732667, Val loss:0.6821920287204709\n", - "Epoch 2. Training loss: 0.670222472202753, Val loss:0.680555022827165\n", - "Epoch 3. Training loss: 0.6685046216756554, Val loss:0.6789231193281474\n", - "Epoch 4. Training loss: 0.6667965940664408, Val loss:0.6773057499397158\n", - "Epoch 5. Training loss: 0.6650977879905501, Val loss:0.6756997275584913\n", - "Epoch 6. Training loss: 0.6634090545244806, Val loss:0.6741022647227787\n", - "Epoch 7. Training loss: 0.6617319267915919, Val loss:0.672508543400481\n", - "Epoch 8. Training loss: 0.6600597993170945, Val loss:0.6709184436565648\n", - "Epoch 9. Training loss: 0.6583971985551863, Val loss:0.6693364404986417\n", - "Epoch 10. Training loss: 0.6567442757826135, Val loss:0.66776031876229\n", - "Epoch 11. Training loss: 0.6550973927654985, Val loss:0.6661960653047748\n", - "Epoch 12. Training loss: 0.6534659717820085, Val loss:0.6646451835606706\n", - "Epoch 13. Training loss: 0.6518448486575243, Val loss:0.6630994595183812\n", - "Epoch 14. Training loss: 0.650226757733936, Val loss:0.6615597805032299\n", - "Epoch 15. Training loss: 0.6486133316286764, Val loss:0.6600349845961935\n", - "Epoch 16. Training loss: 0.6470136235238605, Val loss:0.6585176499743152\n", - "Epoch 17. Training loss: 0.6454236139155769, Val loss:0.6570074794926403\n", - "Epoch 18. Training loss: 0.6438407802164837, Val loss:0.6555035995885308\n", - "Epoch 19. Training loss: 0.6422584003092452, Val loss:0.6539940005867619\n", - "Epoch 20. Training loss: 0.6406781871002952, Val loss:0.6524968697284922\n" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VNXdx/HPyWSW7AmZkIQESEAUAcMWEARlrXUFt7pUXFKrrdpHrT4odatbW+GhSFGr4hKxKrjggrigtlTEBSWIYdWAEAkJgQTInkwmOc8fd2ZIQkIWhgwz83u/Xnllljtzf8Pod05+c+65SmuNEEKIwBLi6wKEEEJ4n4S7EEIEIAl3IYQIQBLuQggRgCTchRAiAEm4CyFEAJJwF0KIACThLoQQAUjCXQghAlCor3Zst9t1Wlqar3YvhBB+KScnp0RrndDedj4L97S0NNauXeur3QshhF9SSuV3ZDtpywghRACScBdCiAAk4S6EEAHIZz13IUT3qq+vp6CggNraWl+XIjrAZrORmpqK2Wzu0uMl3IUIEgUFBURFRZGWloZSytfliCPQWlNaWkpBQQHp6eldeg5pywgRJGpra4mPj5dg9wNKKeLj44/qrywJdyGCiAS7/zja9yrgwn1X2S4ufeNS9lXt83UpQgjhMwEX7jM/mckbm9/gr5//1delCCGaKC0tZdiwYQwbNoykpCRSUlI81x0OR4eeIysrix9++OGI2zz55JO88sor3iiZ8ePHs379eq88V3cLuC9Ue0X1AuD0Pqf7uBIhRFPx8fGeoHzggQeIjIzkf//3f5tto7VGa01ISOvjzuzs7Hb3c/PNNx99sQEg4Ebuuyt2A9BIo48rEUJ0xLZt2xgyZAi///3vGTFiBEVFRdxwww1kZmYyePBgHnroIc+27pG00+kkNjaWWbNmMXToUMaOHcvevXsBuPfee5k/f75n+1mzZjF69GhOOukkvvzySwCqqqq4+OKLGTp0KFdccQWZmZntjtBffvllTjnlFIYMGcLdd98NgNPp5KqrrvLcvmDBAgAee+wxBg0axNChQ5kxY4bX/806IuBG7nmleQAcqDng40qEOI7ddht0pN1QXw979kBSErQ333rYMHCFamdt3ryZ7Oxsnn76aQAeffRRevTogdPpZNKkSVxyySUMGjSo2WPKysqYMGECjz76KLfffjsvvPACs2bNOuy5tdZ88803LFu2jIceeoiPPvqIxx9/nKSkJJYuXcr333/PiBEjjlhfQUEB9957L2vXriUmJoapU6eyfPlyEhISKCkpYcOGDQAcPHgQgDlz5pCfn4/FYvHc1t0CauSutWbb/m0AHKz1zT+oEAFlzx746Sfj9zHUv39/Ro0a5bm+ePFiRowYwYgRI9iyZQubN28+7DFhYWGcffbZAIwcOZKdO3e2+twXXXTRYdusXr2ayy+/HIChQ4cyePDgI9a3Zs0aJk+ejN1ux2w28+tf/5pVq1Zxwgkn8MMPP3DrrbeyYsUKYmJiABg8eDAzZszglVde6fJBSEcroEbue6v2UuGoAOBArYzchWhTR0fYJSWQnQ1ZWWC3H7NyIiIiPJfz8vL4xz/+wTfffENsbCwzZsxodb63xWLxXDaZTDidzlaf22q1HraN1rpT9bW1fXx8PLm5uXz44YcsWLCApUuXsnDhQlasWMFnn33Gu+++yyOPPMLGjRsxmUyd2ufRCqiRe97+PM9lacsI4QV2O8yceUyDvaXy8nKioqKIjo6mqKiIFStWeH0f48eP5/XXXwdgw4YNrf5l0NSYMWNYuXIlpaWlOJ1OlixZwoQJE9i3bx9aa371q1/x4IMPsm7dOhoaGigoKGDy5Mn83//9H/v27aO6utrrr6E9ATVyd/fbrSarjNyF8FMjRoxg0KBBDBkyhH79+jFu3Div7+N//ud/uPrqq8nIyGDEiBEMGTLE01JpTWpqKg899BATJ05Ea83555/Pueeey7p167juuuvQWqOUYvbs2TidTn79619TUVFBY2Mjd911F1FRUV5/De1Rnf3zxFsyMzO1t0/Wcc+/72HOl3PISMzAHm5nxQzvf+IL4a+2bNnCySef7OsyjgtOpxOn04nNZiMvL48zzzyTvLw8QkOPr/Fua++ZUipHa53Z3mOPr1dylPL255EWm0ZCeAL7a/b7uhwhxHGqsrKSKVOm4HQ60VrzzDPPHHfBfrTafTVKKRuwCrC6tn9Ta/3nFttcC/wfsNt10xNa6+e8W2r78vbnMaDHAKKt0Ww/sL27dy+E8BOxsbHk5OT4uoxjqiMfVXXAZK11pVLKDKxWSn2otf66xXavaa3/4P0SO0ZrTV5pHmf0OQNHg0OmQgohglq7s2W0odJ11ez68U2j/giKq4qpqq/ihB4nEBcWx4GaA52e7iSEEIGiQ1MhlVImpdR6YC/widZ6TSubXayUylVKvamU6t3G89yglFqrlFq7b1/XVm2sqKtgXdE66px1zW53z5QZED+AOFscDbqBSkdla08hhBABr0PhrrVu0FoPA1KB0UqpIS02eQ9I01pnAJ8Ci9p4noVa60ytdWZCQkKXCl68cTEjF45kbWHzmTbuOe4DegwgLiwOkAOZhBDBq1MHMWmtDwL/Bc5qcXup1to9lH4WGOmV6lqRU2h8CfLS9y81uz2vNI/QkFD6xvYlzuYKdzmQSYjjxsSJEw87IGn+/PncdNNNR3xcZGQkAIWFhVxyySVtPnd7U6vnz5/f7GCic845xyvrvjzwwAPMnTv3qJ/H29oNd6VUglIq1nU5DJgKbG2xTXKTq9OALd4ssqmrh14NwMhezT8/th3YRnpsOqEhocTaYgEZuQtxPLniiitYsmRJs9uWLFnCFVdc0aHH9+rVizfffLPL+28Z7h988AGxsbFdfr7jXUdG7snASqVULvAtRs99uVLqIaXUNNc2tyilNimlvgduAa49NuXCQPtAAGrqa5rdnleax4D4AQCetozMmBHi+HHJJZewfPly6uqMP/J37txJYWEh48eP98w7HzFiBKeccgrvvvvuYY/fuXMnQ4YYHeGamhouv/xyMjIyuOyyy6ipOZQHN954o2e54D//2Zi1vWDBAgoLC5k0aRKTJk0CIC0tjZKSEgDmzZvHkCFDGDJkiGe54J07d3LyySdz/fXXM3jwYM4888xm+2nN+vXrGTNmDBkZGVx44YUcOHDAs/9BgwaRkZHhWbDss88+85ysZPjw4VRUVHT537Y17U6F1FrnAsNbuf3+Jpf/BPzJq5W1IS4sjtCQUIqripvWQl5pHpGWSEqqS6QtI0Q7bvvoNtbvaX/J3/qGevZU7iEpMgmz6cirGw5LGsb8s9pekCw+Pp7Ro0fz0UcfMX36dJYsWcJll12GUgqbzcbbb79NdHQ0JSUljBkzhmnTprV5HtGnnnqK8PBwcnNzyc3NbbZk71/+8hd69OhBQ0MDU6ZMITc3l1tuuYV58+axcuVK7C3WycnJySE7O5s1a9agtebUU09lwoQJxMXFkZeXx+LFi3n22We59NJLWbp06RHXZ7/66qt5/PHHmTBhAvfffz8PPvgg8+fP59FHH2XHjh1YrVZPK2ju3Lk8+eSTjBs3jsrKSmw22xH/fTvL7xYOC1EhJIQnUFx5KNwP1B6g2lnNF7u+IPu7bPlCVQgv2VO5h58O/sSeSu8s+du0NdO0JaO15u677yYjI4OpU6eye/duiouL23yeVatWeUI2IyODjIwMz32vv/46I0aMYPjw4WzatKndRcFWr17NhRdeSEREBJGRkVx00UV8/vnnAKSnpzNs2DDgyMsKg7G+/MGDB5kwYQIA11xzDatWrfLUeOWVV/Lyyy97joQdN24ct99+OwsWLODgwYNeP0LWL4+3TYxMZG/1Xs/13eXGgbFXDrmSrOFZRFujUSgZuQvRhiONsJsqqS4h+7tssoZnYQ8/+pUhL7jgAm6//XbWrVtHTU2NZ8T9yiuvsG/fPnJycjCbzaSlpbW6zG9TrY3qd+zYwdy5c/n222+Ji4vj2muvbfd5jnQ8jHu5YDCWDG6vLdOW999/n1WrVrFs2TIefvhhNm3axKxZszj33HP54IMPGDNmDJ9++ikDBw7s0vO3xu9G7gCJEYnNRu7uU+v9ftTvsYfbCVEhxNpiZeQuxFGyh9uZOW6mV4IdjJkvEydO5De/+U2zL1LLysro2bMnZrOZlStXkp+ff8TnOeOMMzwnwd64cSO5ubmAsVxwREQEMTExFBcX8+GHH3oeExUV1Wpf+4wzzuCdd96hurqaqqoq3n77bU4/vfPnYI6JiSEuLs4z6v/Xv/7FhAkTaGxsZNeuXUyaNIk5c+Zw8OBBKisr2b59O6eccgp33XUXmZmZbN26tZ09dI7fjty3lByakFNYUQhASlSK57a4sDgJdyGOQ1dccQUXXXRRs5kzV155Jeeffz6ZmZkMGzas3RHsjTfeSFZWFhkZGQwbNozRo0cDxlmVhg8fzuDBgw9bLviGG27g7LPPJjk5mZUrV3puHzFiBNdee63nOX77298yfPjwI7Zg2rJo0SJ+//vfU11dTb9+/cjOzqahoYEZM2ZQVlaG1po//vGPxMbGct9997Fy5UpMJhODBg3ynFXKW/xyyd87P7mTBWsWUHNPDUopHv7sYe7/7/3U3lOLNdT4M2rkwpEkRSbx/q/f92bZQvgtWfLX/xzNkr9+25apa6ijvK4cMNoy8WHxnmAHiLPFSc9dCBG0/DPcIxMBPNMhCysKSYlOabaNtGWEEMHMP8M9whXuri9Vd1fsbtZvBxm5C9EaWSnVfxzte+Wf4d5i5L67vI1wr5Vlf4Vws9lslJaWyv8TfkBrTWlp6VEd2OSfs2WajNzrG+rZW7WXXlG9mm0TFxaHo8FBjbOGcHO4L8oU4riSmppKQUEBXV1uW3Qvm81Gampqlx/vl+EeHx6PQlFcVcyeyj1o9OE99yZLEEi4CwFms5n09HRflyG6iV+2ZUJDQrGH2ymuLPYcwNSyLeNeGVIWDxNCBCO/DHc4tASBe+mB1mbLgKwvI4QITv4b7q4lCNwj98N67rIypBAiiPlvuEcmUlxVTGFFIeYQ82FrX8jIXQgRzPw33JuM3HtF9SJENX8pMnIXQgQzvw73qvoqfiz98bB+O0CMLQaQkbsQIjj5b7i7DmTKLc49bKYMGDNqoixRMltGCBGU/DfcXQcy1TprD/sy1U3WlxFCBCu/DfeeET09l1sbuYOsLyOECF5+G+7utgwcPsfdTUbuQohg5bfh3pGRe3hoOHmleZRUl3RXWUIIcVzw23C3mCye6Y5t9dyLKosoriom+7vs7ixNCCF8rt1wV0rZlFLfKKW+V0ptUko92Mo2VqXUa0qpbUqpNUqptGNRbEvu1kxbbZmB9oHEWGPIGp7VHeUIIcRxoyMj9zpgstZ6KDAMOEspNabFNtcBB7TWJwCPAbO9W2breth6YAu1UV1f3er9MdYYrKFWr525XQgh/EW74a4Nla6rZtdPy9X+pwOLXJffBKYopZTXqmyrNjS1zto22y4WkwVHg+NYlyGEEMedDvXclVImpdR6YC/widZ6TYtNUoBdAFprJ1AGxHuz0NYsvXQpf5n8lzbbLhLuQohg1aFw11o3aK2HAanAaKXUkBabtDZKP+xcXkqpG5RSa5VSa71xNpjkqGTuPv3uNtsuFpOFOmfdUe9HCCH8Tadmy2itDwL/Bc5qcVcB0BtAKRUKxAD7W3n8Qq11ptY6MyEhoUsFd4bFZKFBN9DQ2HDM9yWEEMeTjsyWSVBKxbouhwFTga0tNlsGXOO6fAnwH32szsK7ZAkkJMCWLe1uajFZAKhvrD8mpQghxPGqIyP3ZGClUioX+Baj575cKfWQUmqaa5vngXil1DbgdmDWsSkXWL0aSkrgb39rd1NrqBVA+u5CiKDT7gmytda5wPBWbr+/yeVa4FfeLa0Nd98NzzwDNlu7m7pH7hLuQohg439HqPbqBRMmwLfftruphLsQIlj5X7gDnHEGfP89HDzyWu0S7kKIYOW/4a41fPHFETdzh7tMhxRCBBv/DPdTTwWzGVatOuJmMnIXQgQr/wz3sDAYPVrCXQgh2uCf4Q5Ga2btWqiqanMTq0mmQgohgpN/h7vTCV991eYmMnIXQgQr/w33006DkJAjtmYs23cA4Dhw9OvYCCGEP/HfcI+OhlNOgRdegKKiw+/fvBnL724CoG7F+91cnBBC+Jb/hjvAiSfC7t1w1llGi8btxx9hyhQsVcYUSMekCT4qUAghfMO/w/2f/4Rzz4XcXMjKMtadeeghGDUK6uuxjDJOGOWIaH+pAiGECCTtri1zXLPbYflyeOQRuO8+ePnlQ/f98Y9Y1UHga/lCVQgRdPx75O52773wm98Yl++4A+bMgbvvxmKLAGS2jBAi+Pj3yL2p2bNh4ECjPWM3zsxksYaDBoejxsfFCSFE9wqccLfbYebMZjdZbBFQI+EuhAg+gdGWaYO7LVNX2/ZRrEIIEYiCItwdjmofVyKEEN0roMPdZAtHaXDUSVtGCBFcAjrcVVgYVqf03IUQwSegw52wMCwN4KiXcBdCBJfADnebzQh3R62vKxFCiG4VHOHulHAXQgSXoAj3unoJdyFEcGk33JVSvZVSK5VSW5RSm5RSt7ayzUSlVJlSar3r5/5jU24neUbucoJsIURw6cgRqk7gDq31OqVUFJCjlPpEa725xXafa63P836JR8Fmw9oga8sIIYJPuyN3rXWR1nqd63IFsAVIOdaFeYV7toyM3IUQQaZTPXelVBowHFjTyt1jlVLfK6U+VEoN9kJtR8/dlpGRuxAiyHR44TClVCSwFLhNa13e4u51QF+tdaVS6hzgHWBAK89xA3ADQJ8+fbpcdIe5w72x/tjvSwghjiMdGrkrpcwYwf6K1vqtlvdrrcu11pWuyx8AZqWUvZXtFmqtM7XWmQkJCUdZegdYrRLuQoig1JHZMgp4HtiitZ7XxjZJru1QSo12PW+pNwvtktBQLI2KOgl3IUSQ6UhbZhxwFbBBKbXeddvdQB8ArfXTwCXAjUopJ1ADXK611seg3k6z6BAcWsJdCBFc2g13rfVqQLWzzRPAE94qypushOLQTl+XIYQQ3Sqwj1AFLMqEgwZflyGEEN0qCMI9FIeWcBdCBJfgCHcl4S6ECC5BEO5m6iTchRBBJvDDPcSMQx0XE3eEEKLbBH64m8w4VCPHycxMIYToFgEf7laTFa2gQb5UFUIEkYAPd4vJAsjiYUKI4CLhLoQQASjwwz3UCki4CyGCS+CHu9kI9zo5YYcQIogEfriH2gAZuQshgkvAh7vVEgbIqfaEEMEl4MPdYnaFe22ljysRQojuE/jhbnG1Zaol3IUQwSMIwj0ckJG7ECK4BH64W11tmZoqH1cihBDdJwjC3Ri518nIXQgRRAI/3G0RADhqZeQuhAgeAR/uVlskIOEuhAguAR/unpG7o8bHlQghRPcJnnCvq/ZxJUII0X0CP9zDXG0ZGbkLIYJI4Id7eBQg4S6ECC7thrtSqrdSaqVSaotSapNS6tZWtlFKqQVKqW1KqVyl1IhjU27nuUfudRLuQoggEtqBbZzAHVrrdUqpKCBHKfWJ1npzk23OBga4fk4FnnL99jlrZAwADmetjysRQoju0+7IXWtdpLVe57pcAWwBUlpsNh14SRu+BmKVUsler7YLzO62TL2EuxAieHSq566USgOGA2ta3JUC7GpyvYDDPwBQSt2glFqrlFq7b9++zlXaRSFh4YQ2yJK/Qojg0uFwV0pFAkuB27TW5S3vbuUh+rAbtF6otc7UWmcmJCR0rtKuMpuxNIDDKSfrEEIEjw6Fu1LKjBHsr2it32plkwKgd5PrqUDh0ZfnBUphaQRHg4zchRDBoyOzZRTwPLBFaz2vjc2WAVe7Zs2MAcq01kVerPOoWBqVnGZPCBFUOjJbZhxwFbBBKbXeddvdQB8ArfXTwAfAOcA2oBrI8n6pXWdpVNRpCXchRPBoN9y11qtpvafedBsN3OytorzN2hiCQ0m4CyGCR0dG7n7PokNwaKevyxBCiG4T8MsPAFgw4Wis93UZQgjRbYIn3GXkLoQIIhLuQggRgIIj3FUodTT4ugwhhOg2QRHuVhWKQ8JdCBFEgiLcLcqMQ0m4CyGCR3CEe0goDtXo6zKEEKLbBEm4myXchRBBJTjC3WTBoQ5bpFIIIQJWcIR7iAVHiAYtAS+ECA7BEe6hVupCAafMdRdCBIegCHdrqBWHCaiVU+0JIYJDUIS7RcJdCBFkgibc602ga2p8XYoQQnSL4Ah3sw2A+uoKH1cihBDdIzjC3WKEu6Om0seVCCFE9wiScA8HwFFV7uNKhBCiewRFuFstYQDUychdCBEkgiLcPSP3Wgl3IURwCI5wtxojd0dNlY8rEUKI7hEk4R4BgKNOwl0IERyCI9xt7rZMtY8rEUKI7tFuuCulXlBK7VVKbWzj/olKqTKl1HrXz/3eL/PoWGyRADhqZeQuhAgOoR3Y5kXgCeClI2zzudb6PK9UdAxYwlxtGYeM3IUQwaHdkbvWehWwvxtqOWasYVEA1NVJuAshgoO3eu5jlVLfK6U+VEoNbmsjpdQNSqm1Sqm1+/bt89Ku22exuUbu3+dASUm37VcIIXzFG+G+DuirtR4KPA6809aGWuuFWutMrXVmQkKCF3bdMe61ZRy56yE7u9v2K4QQvnLU4a61LtdaV7oufwCYlVL2o67MiywmCwAOexxkZfm4GiGEOPaOOtyVUklKKeW6PNr1nKVH+7ze5An3qnKIivJxNUIIcey1O1tGKbUYmAjYlVIFwJ8BM4DW+mngEuBGpZQTqAEu1/r4OlmpJ9xpgA0bIDPTxxUJIcSx1W64a62vaOf+JzCmSh63rCYrgHE2ppycoAn3Lfu2MP/r+Tw46UGSIpN8XY4Qoht1ZJ6733OP3OuiwoxwDxLXLbuOrwq+4rVNr/G7kb8jwhLBTaNuwh5+XH0lIoQ4BoIq3B29U4Iq3K2hVmJtsQxPGs6cL+cAEBYaxsxxM31cmRDiWAuOtWXc4Z6abPTcHQ4fV9Q9CisK+UW/X/Cfa/7DTZk3EaJCOGfAOb4uSwjRDYIi3ENDjD9QPo0tpcRcDxtbXSYnoDTqRvIP5pMWmwbAbWNuo1E38u4P7/qknvd/fJ+7Pr2Lkmo5iEyI7hAU4a6UIj02nf9Wb+bRcQRFa6a4spi6hjpPuA+IH8CU9CkszFlIQ2NDt9cz4+0ZzPliDtnfyUFkQnSHoAh3gHcuf4cIcwTvDFKUr/vK1+Ucc/ll+QD0jenrue13I39Hflk+H2//2Ov7e2btM9z8/s2tjsy11lTWVZIem07WcDmITIjuEDThnpGYwfu/fp8dsZrTwhezt2qvr0s6am9tfovJL06muLL4sPt2HtwJ4Bm5A0wfOJ3EiESeznna67Xc8fEd/HPtP5n31bzD7jtQewCndmIKMclMHSG6SdCEO8CEtAmc09CfTdG1TH91GsfZsVad9vCqh1mZv5Jpiw9/Le5w7xt7aORuMVn4zfDf8N4P7/GnT//Upf53YUUhJyw4gbs+OdQ/dzQ4qKo31sr/ePvH1DfUN3uM+8OnqKKo0/sTQnRNUIU7wAt9b2HCDvi6cA23fnSrXwe82WTGpEx8U/gNd//77mb37Ty4k/iweCItkc1uvzHzRiwmC49+8SjPrXuu0/vMKcxh+4HtzPnyUP/cHd6/7P9LcopyuG/lfc0eU1xl3F9VX0VFXUWn9ymE6LygmOfeVEJpDSsXwR2/hMd4nO+KvmPpZUvpGdHT16V1WmFFITMyZmALtfHoF4+y/cB2/nnuP7GH28kvy2/WknHrHdObJ855guvfu56fD/7c6X3urtgNwNknnO3pnxdWFAJw86ibSYtNY/YXs6l11nLvGfdiD7c3axsVVhRykvUkAOob6ln982pyi3PJKcrBarLyt6l/k9aNEF4QdCN3rrsO9be/8XfOZHw+rN61mikvTKS63r9O5FHfUE9hRSF9Y/ry5DlP0jO8J29sfsMzmt55cGer4Q7w2xG/5Q+j/sBTOU/x9pa3O7Xf3eVGuJ/Q4wRPCBdVGu2W5KhkZp5mHCD1jzX/ODSyrzoU7u5tAe79z71Mfmkyt624jX/l/ovnvntOZtMI4SXBF+52O8yahXp3GW/l9Oe8rbCxdAvjnh/Hff+5z2/mYReUF6DR9I3tiynExNkDzsZqsnL10KvRWh8x3AHmnjmXoYlDueadazr1mt0jd/doHQ710ntF9SIlOgUwWjTukX3TkXvTvvtH2z8C4L4z7uP0PqfTJ6aPzKYRwkuCL9zdrFYSPvqc97aP4t0lsKnoex75/BFmfTrL15V1SMupjpPTJ1PXUMfeqr3srdpLrbO22TTIlqyhVlKjU6lwVPDkN092eL/ucHf/BmM0rlD0jOiJLdRGQngCabFpnpF9cVUxEWbjbFhNPxRCCGFAjwHccuot9I7pjUnJbBohvCV4wx0gORnWrGHab2azMluTWB3C8989z4TsCZ72w/Eq/6Ar3F2zYcamjgXgq4KvPMF/pJE7wKWDLwVgfJ/xHd5vQXkBQLN/n6KKInpG9PQcCZwanerZDoxwP6HHCYSFhjVryxRUFDApbRL2cDuJEYnN2jdCiKMT3OEOoBTceSfjpmaxY14jYwoUq35exfBnhrP659W+rq5N7gDvHd0bONQD/6rgq1bnuLdmWNIwAPbXdPz85+5QL6osolE3ei4nRyV7tjks3CuLSYxMJDkq2RPulY5KSqpLSI9LByAxIpHq+moqHZUdrkUI0TYJd7c5cwi76x7e2zyU63LAur+c07NPZ8xzY9i6b6uvqztM/sF8kiKTsIYaa9UrpRiTOoavdn3V6hz31qTHGsG6/cD2Du2zylFFWV0ZvaN742x0sq/KOMl5UWURyZFHCPeqYhIjEukV1cvTltlxYEezGhIjE41tWzkgSwjReRLubnY7PPII9mmX89x7sOUfTk7fFcKa3WsY/sww5n01D0fD8bOaZH5Z/mE99dNST+OH0h9YV7SOOFsc0dboIz5HlDWKnhE9+enATx3ap7vPPiplVLPrRRWHh3tpTSlJj04cAAAXc0lEQVQ19TVorY2Re0QiyZHJni9Udxx0hbtr5O4+mYi0ZoTwDgn3lq67DubMIXLV17z1dV/u+BLG7Wzkjo/vIGluEo+veRxno9PXVRrh3mJkPra30Xd/78f32m3JuPWL69fhkbu7JTO612jP9YbGBoqriukV1cuzXWp0KmD05yscFdQ11BltmchDbZnDRu4RMnIXwpsk3Fuy22HmTMjMxL5yDXMTr+LTj5P4bY6xRsotH91C8t+TWfD1Ap+N5Bt1I7vKdh02ch/VaxQmZaK6vrrD4d4/rn+HR+7uVsupqacCxsh9b9VeGnVjs567+3uAgvICT1i72zLldeVUOarYcXAHEeYIz+wYd1tmT+WeDtUihDgyCfcjSUiAl16CH3/kb0xh9sdw2QYoqS7h1hW3kj4/nWmLp3lGod1lb9Ve6hrqDgv3CEsEGYkZAEecBtlUv7h+/Fz2c6sfVPUN9ZTVlnmuu9swI5JHEKJCKKwoPHQAU4u2DLjC3dVm6RnR0/MBUFRZxI6DO0iPS0cpBUBCeAIgbRkhvEXCvSNsNuwvLOHOXz7IEwfGMPtjWPIGWPcd4L0f32PgEydx/bLrmfnxzG45CMozDfLDL8BsNv7aSE+H00/ntOjBQPszZdz6x/X3nNijpbs+vYs+8/t4VtDcXb6bGGsM0dZokiKT2F2+29NDbzpydx/I1Gzk7mrLgNGj33Fgh6clA8Y6OfFh8dKWEcJLgm5tmS6z2+H++7GX3MSdTz4J1dVMeXYBD4+GooRQXmh8nkY0b2x6g9P6nMbsqbPpHdP7mJTiOYDptRXgdMKAAVBeDqtXM7b2W548D9ateJGSlLOw9zbWcWH/fti+HfLzYdcuqKiAm26iX1w/AH468BMD4gc028/rm16nvK6cOV/MYe6Zc9ldsdsT3L2ierG7YnerI/dwczg9wnpQUF5AjC0GMNoy7nnwhRWF7Di4g0lpk5rtLykySUbuQniJhHtn2e3w5z8bF2+8kX/88Y/wRS6bP/yJ/z3HxFdpRSwuX8xbW97iV4N+RZQ1igcmPuDVhcnco+w+P+2Hc8+FF1807njkEaav+ZwJO9bxUvp6hlyfycz8VCP4CwsPf6K8PPo/+Shw+HTI8rpyz7RFd4tnd8VuUqKMcE+JSmH7ge2ekbt7totbanQqBRUF2MPtKBQJEQmeaZsb926k0lHpmSnjlhgpBzIJ4S3ttmWUUi8opfYqpVo98agyLFBKbVNK5SqlRni/zONUWhq8/Tbk5TFo2m/44KUGfpjr4Ia1MH13JK9tWMxTa59iwOMDuHH5jfzhgz945SQh+WX5xKgwYuqARx81PnDsdpg/n8gv1/LmeYuYs+MEsnbGwdatkJgI57hOjH3bbbB8OfTqBS+/TNLJo7A1hPDTM4/CVVdBidFW+mrXV2iM5ZA379sMWlNQXuDpp6dEpXh67vFh8Z7gdnPPdS+uKiY+PJ7QkFDibHFYTVa+LPgSoFlbBozRvbRlhPCOjozcXwSeAF5q4/6zgQGun1OBp1y/g0dICMyeDQMH0rOoiGceewzsip31Ddz9C9ifamFh3TM0onlx/Yuc0vMUbjn1Fi4YeAFh5rBO7y6/LJ++1RZIjYfBg5vfqRT2S65m5iVXG0GdnQ1ZrsW43JftdqM9c+GFhCxfTr9yM9sde+Dll43XsmgRn+/8DJNWjCo28fVHz+K84Bn23KNJqTEDRl99f81+fvrsHZJTDl8PJjUqlbWFa+kT08czzVEpRVJkEmsK1gAcPnKPSJTZMkJ4SbvhrrVepZRKO8Im04GXtHHWi6+VUrFKqWStdXCddsc9hbKkxFiz5tprSXvwQV598kmIqmN7qOa+KbA+rZGv67/m67e+xmaykRaXxlUZV3HxyRdzYvyJntkjR5J/YCdphVVw1q+M5RPaq8mt6eXQUCPss7Pp1/NTfir7GT6rhiVL4Lzz+PyrpxhRqZmy3cmj42HH5JE0huSQsiAb9g6mV85r0A9yKGL4d3uM4wN69oQ77gC7ndToVPZW7WVX2S7PNEcwevXu7wwOG7lHJlJVX0WVo4oIS0TH/t2FEK3yxmyZFGBXk+sFrtuCkztQExLggQdgzhyjt33Z73n1TVj1guavn8Lrr8PIPYqtJVu55z/3MPDJgcTNjmP64ulc+salfLv72zbPEvXzgZ30LXHCWWd5pdb+iSezvXIXet06GDCAuisuZU3EQcanjOXUqdfQEALv3TkdgJTY3nDrraR8+wMAJRGQHNsbFi0yWkQPPghAaqRxUNOGgnUkVh36AHLPqokPiyfKGtWsHDlKVQjv8cYXqq0NHVtNJaXUDcANAH369PHCro9zTUfODz8M/fphP/98/rRwIURGMumNRWT3+JnJP8FfJ4bw1sAyPvxhOfWqkTc2v0FiRCKJkYlcNvgyftHvFwxNGkpNfQ1lzkr6liuYMsUrZfaL60dVfRX7bA30XLGCtb/9JXWhmzjdMoBTb5kDcxex9Gdj7fWUZxbDX7NJWXroJNvJ510Bl19g9PWffx6GDiX13ccgExwmTeKyf8MXM6CkhOTk7yAN0vPLjB6/0wmXXw4DB5IY5loiuLLYM4tHCNE13gj3AqDpnL9UoJWpGaC1XggsBMjMzPTfk5d2RdOgnzfPuOmWW5j59NOQns4zzz/OmJ/XcN4PjTw3EpKqFK+OKmd9bDG5xbnc8597MCmTJ/TievWH2FivlOZ+zu37t9Oz91g+v+1C+HoT42f8iYSInqTHpvPlLuNL0JS4vvDww6SkJ0ONMWsoOTIZxoyBzZuNgL/+elJTLJBpPH/i4FPhscXQ2EjyedFGuO9zwtJXobHRaAUBiWeeAqfJyF0Ib/BGW2YZcLVr1swYoCzo+u1dZbfDvffClVdif305My+Yw8nX3cnfP4aZlol88lECcz6G7/8JV34PDbqBkj3GUgHbTvLeSS36x/UH8CxD8HnpOgbaB5LQeyBwaLkBc4iZhIgEsNuJnnkf4eZwgEPryiQlwWefwXnnkbrv0BGviVf+DjZsgPvvp9edDwOQPnQifP+90aNfuhTS0kj8agMg68sI4Q3tjtyVUouBiYBdKVUA/BkwA2itnwY+AM4BtgHVgJwnrSuafiFrt0NWFnZg5nPPwejRzH/q7wz9+AOmFdhYllpF1riRXtu1+2jWBd8s4CT7SXzx8xeeE3kAjEkZw5KNS+gV1YsQZYwHlFKkRKWQtz+v2dGpREVBdjZR2dlEOx+m3FFhzPE/cRA8+CDJ24z2Tt4JcZT0S8I+d67xuNxcej5s9OtlxowQR68js2WuaOd+DdzstYqCXcsZLrOM0/7ZMzKYmZ0N06Yxc9myQ9MbvSDMHEbP8J58s/sbRj1rLOdbXldOSXUJ9nC7Z+TuPjrVLSXaFe5Njk5t+hpS//kim/dtbjZbJiMxg2hrNG9vfZuxqWOZOc71Wn/xC8wPPkgPU5S0ZYTwAjlC1V80Df2m4e8lX//2a+Z8OYcBPQbw8faPeW3Ta4xMHsnMcTMZljSMUBVKZV2lJ/ABeoT1AIx1YVqTGp1qhHtE86mQ22/ZTvZ32c1Phj1yJJjNJNVbJNyF8AJZOEwAxgFFT537FLePvZ2XL3qZOVPneMLXFmrjnBPPIXdvLtnfZXse43AaffXXNr7W6nO6V3p0t3Lc7OF2Zo6b2fxk2DYbjBxJ4kGn9NyF8AIJd3GY1sL3+WnPNwt8gBemv8DsqbObj8CbcC8U9uqGVzu243HjSNxTQbH03IU4atKWER3iDvymEiISuHPcnW0+Zu6ZcxmcMLjN8D/MaaeR+OzfKS6XyVZCHC0ZuYtjptX2y5GMHUtiJVQ0VFNdX31sixMiwEm4i+NHcjJJTY5SFUJ0nYS7OK4kpg0BIK/0Rxp1o4+rEcJ/Sc9dHFf6DBkHB/7LL185i1BCibJFMaDHAJIik9hfs5/J6ZNJjEhkffF6rs64mpMTTiYuLO6wGTlCBDvV1sqDx1pmZqZeu3atT/Ytjl9682ZumDWY50ZC//2wvQf0rwmjzNxISWhdq48xKRP2cDumEBMD7QOxh9vZU7GHU1NOJb8sn+kDp9M/rj8JEQnYw+1EWaI6tLSyEMcjpVSO1jqz3e0k3MXxpmT1x2QvvpNpW2FZ7fdk7UmCqiqy+1fwq02wJxIWDYOScHhzMEz6CSqssDYFUqtMVJg1ZZa2WzqhmLCZrCTaEmg0aQYmDCLOdc7X0/ucTmxYLOv3rOeCky4gLS6NOFscsbZYYmwxnumdQviKhLvwf62dSerSS6G0FBYtouTAbrK3LyUr6Syoria7fBVZziFQU8MLUduZWmjl9f51TNoJrw2B7OFw7g9QbYaV/SCxAoqjIKnCuK3c1n5JEViJMYXT6KilV3QqNaGaQfZBxETGs7Msn3G9xxFti2Zd0TrOO/E8+sT0IdoaTbQ1mihLFNHW6DaP6BWiIyTcReBr6zSC7svTpsGyZXDNNZSU/Ez2q3eSdcYt8PbbZOe+xDR9IsvUj2RFnwG1tWQ7vuHyuhMoa6whO2E3ByJCyB7ayKUboSYU3hsI4/ONyzkp0KMa9ocbv2tCocbSsbKtmAhtgB6mKBqcDvrE9CEyIo59zjIyep5CZHgseQe3MzxpONv2b+PMfmcSaY3k8/zPuWTQJaRGpxJpifT8yIdFcJFwF6It7g8Fd/i38eFQ8svTyV7+MFkz/g41NWS/MpOsybdDRQXZ/5nHtH1xLNuxgqyeZ0JtLS+Ur+Ky0KFUNFSzKDyPg9Fmnjulnqs2m6lVDbxxciO/3Aa1ofBZGvQqh8Jo6Lcf6kywOwaiao376zvR/TETQgRWaHDSM8xOhCWKcl1D78gUDjZUckrPUwi3RbHtwHbGpo4lwhLBuqJ1jE0dy8a9G5l20jRSolOIMEcQaYkkwhJBhDkCi8ki300chyTchTjW2vnLwfPhcJVxcpbsl24n67K/QWUl2W/ew7TBF7Ns7StkDbsWPv6Y7J/eMj4o6up4ofwzplb25O0eezm3ri9Vuo43e+zh9P2RvN+nlldPdnLyXtjSEybuMD4cvuoDQ4rBYYIf7RBda7SaomqhLhQcnfy6QAGWxhBiQyMJN4dTi5PU8CTCLRGUOA6SFpfOvrr9DE8cTpg5jK0lWxnXZxzh5nC+3f0t4/uMZ/2e9Vx48oX0iupFhDmCcHM4ERbjt8XUwT91RDMS7kL4k9Y+KNr4y6KkppTsf93OtKl/YNmKf5B13n3GF87LHyZrzI3wwQdkb3rlUNspforxgVG5mitChlLVUMOi8B8Ztz+CFUlVTC6N4t0+NSwa4uSSzeAIgWUDoX8pbI+H0QXGB8b6ZDih1Lj8cyyEOYxWlNVpPEZ3cjZqCGBuDCHWHEWYJZxaXU9KeDJhlnBK3R8eNSVk9Mwg3BzOD6U/cFrqaYSZw8gpymFM6hg2FG/g/BPPJzkqmXBzeLOfMHMYVpM14P76kHAXIlh1sO3U9H73B0bWVfOgvt748Bh1FctWP0fWGbfCu++SnfsSWf0vNu7/eRnT6tJYZt1Jlm0Muq6W59R6LnH0p7qxjldiCxhbYuOTXrVMKrKyPL2elzIauXiT8eHwXosPj3oTfJcMA0qM+/PjIMIBVRYIq4f6EHCauvBvoSE8xEpYqI0GNAm2HoSbwylzVtIrOpWyujJOsp9EWGgYOw/uZETyCMJCw9i4dyMjk0eytXQrU/tNJSw0jFX5q7hg4AWeD5Kw0DDPh0hYaBimkK4U2HkS7kII7+nEXxat3V9SU3qoLVVTTfbrdzNt2GUsW/Mvsk79HXz0EdmbXyWr73RwOMgu+pBpNX1YFvYzWaGZUF/P8yHfc3l1f6p1HS/FFzC+2MqK1Dqm7rawrF89zw/XXL7B+HB4axCctA9+SIDTdxofHl/3hlP2GJe3JkBMDZSFQXw11JsU5VaNqREaung8nIkQwk1hhIVacdJInDWOOu2gd3QfbGYbe6v2clL8SdhCbZhCTMz75byOr7vUhIS7EMJ/eOvD41d/Mb78fvMepg29jGXftPjwSLvA+PAo/ODQXx6W0cZfI/o7smpOosHp4NnYHUwpDue91GrO3x1BtXKytF8dk3fA8hNh0XDI2AO5SXDOj8ZfFp+cAOPyjQ+Pb1IPTbXtt9+4bVcMxNco6k1QbtHMGftnZp75QKf/qSTchRDC7Sg/PDy3XXONcXzF4juNttUXzxttq/feI3v9i2SdeCk4nWT/9BbTnP1ZFrrdmGrrdJJd/SVZeqhx2byJrOl/xn7nA51+KRLuQgjRXbry4ZGVZZw+s5Mk3IUQIgB1NNxlKT0hhAhAEu5CCBGAOhTuSqmzlFI/KKW2KaVmtXL/tUqpfUqp9a6f33q/VCGEEB3V7gHJSikT8CTwC6AA+FYptUxrvbnFpq9prf9wDGoUQgjRSR0ZuY8Gtmmtf9JaO4AlwPRjW5YQQoij0ZFwTwF2Nble4LqtpYuVUrlKqTeVUr29Up0QQogu6Ui4t7bqTsv5k+8BaVrrDOBTYFGrT6TUDUqptUqptfv27etcpUIIITqsI4uAFgBNR+KpQGHTDbTWpU2uPgvMbu2JtNYLgYUAri9g8ztV7SF2oKSLjz2eBeLrCsTXBIH5uuQ1+Ye+HdmoI+H+LTBAKZUO7AYuB37ddAOlVLLWush1dRqwpb0n1VondKTA1iil1nZkEr+/CcTXFYivCQLzdclrCizthrvW2qmU+gOwAjABL2itNymlHgLWaq2XAbcopaYBTmA/cO0xrFkIIUQ7OnRuFq31B8AHLW67v8nlPwF/8m5pQgghuspfj1Bd6OsCjpFAfF2B+JogMF+XvKYA4rOFw4QQQhw7/jpyF0IIcQR+F+7trXPjD5RSvZVSK5VSW5RSm5RSt7pu76GU+kQplef6HefrWjtLKWVSSn2nlFruup6ulFrjek2vKaX87pT3SqlY18F5W13v2Vh/f6+UUn90/be3USm1WCll88f3Sin1glJqr1JqY5PbWn1vlGGBKztylVIjfFf5sedX4d5knZuzgUHAFUqpQb6tqkucwB1a65OBMcDNrtcxC/i31noA8G/XdX9zK82nws4GHnO9pgPAdT6p6uj8A/hIaz0QGIrx+vz2vVJKpQC3AJla6yEYs+Auxz/fqxeBs1rc1tZ7czYwwPVzA/BUN9XoE34V7gTIOjda6yKt9TrX5QqMsEjBeC3uo3sXARf4psKuUUqlAucCz7muK2Ay8KZrE398TdHAGcDzAFprh9b6IH7+XmHMlAtTSoUC4UARfvheaa1XYUy/bqqt92Y68JI2fA3EKqWSu6fS7udv4d7RdW78hlIqDRgOrAES3QeDuX739F1lXTIfuBNodF2PBw5qrZ2u6/74fvUD9gHZrnbTc0qpCPz4vdJa7wbmAj9jhHoZkIP/v1dubb03AZcfR+Jv4d6RdW78hlIqElgK3Ka1Lvd1PUdDKXUesFdrndP05lY29bf3KxQYATyltR4OVOFHLZjWuHrQ04F0oBcQgdGyaMnf3qv2BMJ/jx3mb+He7jo3/kIpZcYI9le01m+5bi52/5no+r3XV/V1wThgmlJqJ0a7bDLGSD7W9ac/+Of7VQAUaK3XuK6/iRH2/vxeTQV2aK33aa3rgbeA0/D/98qtrfcmYPKjI/wt3D3r3Li+yb8cWObjmjrN1Yt+HtiitZ7X5K5lwDWuy9cA73Z3bV2ltf6T1jpVa52G8b78R2t9JbASuMS1mV+9JgCt9R5gl1LqJNdNU4DN+PF7hdGOGaOUCnf9t+h+TX79XjXR1nuzDLjaNWtmDFDWZE2swKO19qsf4BzgR2A7cI+v6+niaxiP8edgLrDe9XMORo/630Ce63cPX9faxdc3EVjuutwP+AbYBrwBWH1dXxdezzBgrev9egeI8/f3CngQ2ApsBP4FWP3xvQIWY3xvUI8xMr+urfcGoy3zpCs7NmDMFvL5azhWP3KEqhBCBCB/a8sIIYToAAl3IYQIQBLuQggRgCTchRAiAEm4CyFEAJJwF0KIACThLoQQAUjCXQghAtD/AzhAY9wWHuZwAAAAAElFTkSuQmCC\n", - "text/plain": [ - "
" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "nn.fit_epoch(X_train,y_train,X_test,y_test,lr=0.001,epochs=20,bs=512)" ] @@ -591,7 +524,7 @@ }, { "cell_type": "code", - "execution_count": 232, + "execution_count": 11, "metadata": {}, "outputs": [], "source": [ @@ -684,7 +617,7 @@ }, { "cell_type": "code", - "execution_count": 40, + "execution_count": 12, "metadata": {}, "outputs": [ { @@ -693,7 +626,7 @@ "[784, 100, 10]" ] }, - "execution_count": 40, + "execution_count": 12, "metadata": {}, "output_type": "execute_result" } @@ -804,7 +737,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "With sigmoid, model is slower to learn. This can be the result of vanishing gradient problem as sigmoid's gradient is easier to approach 0 when inputs are at two extreme ends." + "With sigmoid, model is slower to learn. This can be the result of **vanishing gradient problem** as sigmoid's gradient is easier to approach 0 when inputs are at two extreme ends." ] }, { @@ -826,31 +759,9 @@ }, { "cell_type": "code", - "execution_count": 49, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Epoch 1. Training loss: 2.9122617252387832, Val loss:2.831944970324196\n", - "Epoch 2. Training loss: 2.39801878740374, Val loss:2.261103771704497\n", - "Epoch 3. Training loss: 2.427667933233322, Val loss:3.199589164086086\n", - "Epoch 4. Training loss: 2.418664644373875, Val loss:3.1486781831217554\n", - "Epoch 5. Training loss: 2.566634661636582, Val loss:3.2339314052081995\n" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XlcVNX7wPHPERFEEBBwBQMtcxeRzMDS1MqltEx/Zmqm5paVtn21fS/bzVZBoUVNyxbNUlu0rMwFUHHfN8INF5RVlvP74wIpggw4w50ZnvfrxcuBOffehyvzzJmzKq01QgghnEs1swMQQghhfZLchRDCCUlyF0IIJyTJXQghnJAkdyGEcEKS3IUQwglJchdCCCckyV0IIZyQJHchhHBC1c26sL+/vw4ODjbr8kII4ZDi4+NTtNYBZZUzLbkHBwcTFxdn1uWFEMIhKaUOWFJOmmWEEMIJSXIXQggnJMldCCGckGlt7iXJyckhKSmJrKwss0MRFnB3dycwMBBXV1ezQxFCFGNXyT0pKQkvLy+Cg4NRSpkdjrgErTUnTpwgKSmJkJAQs8MRQhRjV80yWVlZ+Pn5SWJ3AEop/Pz85FOWEHbKrpI7IIndgcj/lRD2y66aZYQQwlll5mQSfzieX/b8wuG0w7za/VX8Pfxtdj1J7uc5ceIE3bt3B+DIkSO4uLgQEGBMBFu7di01atQo8xwjRoxgypQpXH311aWW+fDDD/Hx8WHIkCGXHXPnzp354IMPCA0NvexzCSGs5/DZw6w6tIpVh1bx96G/STicQE5+TtHzV9W5iscjH7fZ9SW5n8fPz48NGzYA8Pzzz+Pp6cljjz12QRmtNVprqlUruUUrNja2zOtMmDDh8oMVQtiNvPw8Nh3bVJTMVx1axb7T+wBwr+7ONQ2v4ZHrHiEiKIJmdZrxw84fGNF+hE1jkuRugd27d3P77bfTuXNn1qxZw+LFi3nhhRdISEggMzOTQYMG8eyzzwL/1aRbt26Nv78/48aNY8mSJXh4eLBw4ULq1q3L008/jb+/P5MmTaJz58507tyZ5cuXk5qaSmxsLBEREaSnp3PPPfewe/duWrZsya5du5g5c+Yla+izZ8/m9ddfR2tN3759efXVV8nNzWXEiBFs2LABrTVjxozhoYce4t133yU6OhpXV1fatGnD7NmzK+t2CuHwUrNSWfPvGv4++DerklaxOmk1aefSAGjg2YDIxpE82PFBIoIiaN+gPTVcLvzU3zyguc1jLDO5K6XcgZWAW0H5BVrr54qVeQS4D8gFjgMjtdYWrX9QqkmToKAWfUk5OXDkCNSvD2WNtw4NhWnTKhTO1q1biY2N5ZNPPgFg6tSp1KlTh9zcXG688UYGDBhAy5YtLzgmNTWVLl26MHXqVB555BFiYmKYMmXKRefWWrN27VoWLVrEiy++yNKlS3n//fepX78+33zzDRs3biQsLOyS8SUlJfH0008TFxeHt7c3PXr0YPHixQQEBJCSksKmTZsAOH36NABvvPEGBw4coEaNGkU/E0JcTGvN3lN7i5pXVh1axeZjm9FoqqlqtK3XluHthhMRFEFEUARXeF9hF4MNLKm5ZwPdtNZpSilX4C+l1BKt9erzyqwHwrXWGUqp8cAbwCAbxHuxI0dg717jcVCQzS7TtGlTrrnmmqLvv/zyS2bNmkVubi7Jycls3br1ouRes2ZNevXqBUCHDh34888/Szx3//79i8rs378fgL/++ovJkycD0K5dO1q1anXJ+NasWUO3bt3w9zc6aO6++25WrlzJ5MmT2bFjBxMnTqR3797cfPPNALRq1YqhQ4fSr18/br/99nLeDSGcV1ZuFgmHEy5I5sfSjwFQ26021wVex8CWA4kIiqBjo454uXmZHHHJykzuWmsNpBV861rwpYuVWXHet6uBoZcdmaU17JQUiI2FESPA33Y9z7Vq1Sp6vGvXLt577z3Wrl2Lj48PQ4cOLXG89/kdsC4uLuTm5pZ4bjc3t4vKGLfdcqWV9/PzIzExkSVLljB9+nS++eYboqKiWLZsGX/88QcLFy7k5ZdfZvPmzbi4uJTrmkI4g6NpR/9rK09aRVxyHOfyzgFwZZ0r6XllTyKDIokIiqBlQEuqKbsbQV4ii9rclVIuQDxwJfCh1nrNJYqPApZYITbL+PvD47brcS7JmTNn8PLyonbt2hw+fJhly5bRs2dPq16jc+fOfPXVV1x//fVs2rSJrVu3XrJ8p06dePzxxzlx4gTe3t7MmzePxx57jOPHj+Pu7s7AgQMJCQlh3Lhx5OXlkZSURLdu3ejcuTNz5swhIyMDLy/7rIFUprz8PF758xXWH1nP4FaDCWsYxhXeV+DqIkssOIO8/Dy2HN9yQcfnnlN7AHBzcSO8YTgTr51Y1MRSt1ZdkyOuOIuSu9Y6DwhVSvkA3ymlWmutNxcvp5QaCoQDXUo6j1JqDDAGoHHjxhUO2mxhYWG0bNmS1q1b06RJEyIjI61+jQcffJB77rmHtm3bEhYWRuvWrfH29i61fGBgIC+++CJdu3ZFa81tt91Gnz59SEhIYNSoUWitUUrx+uuvk5uby913383Zs2fJz89n8uTJktgLLNyxkOd+N7qUvt/+PQDVVDUaezemqW9T46tOU5r4Nil6XNuttpkhi0s4m32W1Umri2rlq5NWcyb7DAD1atUjsnEk48PHExEUQViDMNyqu5kcsfWo8n78V0o9B6Rrrd8q9vMewPtAF631sbLOEx4erotv1rFt2zZatGhRrnicVW5uLrm5ubi7u7Nr1y5uvvlmdu3aRfXq9jXAydn+z3rO7kni0URGth9Jp8BOnMg4wZ5Te4yvk3vYe2ovxzOOX3CMv4c/TX0vTPiF/9b3rO8wH+Mdndaa/af3X9BWvunYJvJ1PgpFm3ptippXIoIiCPEJsYuOz/JSSsVrrcPLKmfJaJkAIEdrfVopVRPoAbxerEx7YAbQ05LELsqWlpZG9+7dyc3NRWvNjBkz7C6xO5t9p/bx856fea7LczzX9blSy53JPsPeU3vZc9JI+ntP7WXPqT2sTlrNV1u+Ik/nFZV1r+7+X9IvVusP9gl2qppiZcvOzWb9kfUXJPMjaUcA8KrhRafATjxzwzNEBkVybeC1Ve4TliXZogHwWUG7ezXgK631YqXUi0Cc1noR8CbgCXxd8E54UGvd11ZBVwU+Pj7Ex8ebHUaVMjNhJkopRoWNumS52m61Ca0fSmj9i+cc5OTlcCD1wAXJv7DWv3zfctJz0ovKKhRB3kEl1vqb+DbBt6av1X9HR3Ys/Rj/HPqnKJnHJceRnZcNQBPfJvRo0oOIwAgiG0fSKqAVLtWq9gABS0bLJALtS/j5s+c97mHluISoVDl5OcRsiKHPVX0IrB1Y4fO4urhyZZ0rubLOldD0wue01hxLP1aU7AsT/95Te1m8czFH049eUN7X3fe/Jp7CN4CC7xvVbuTUzT35Op+tx7de0PG56+QuAGq41KBDgw480PGBoiaW+p71TY7Y/sjnfCGAxTsXcyTtCGM6jLHZNZRS1POsRz3PekQERVz0fNq5tKIaf2FTz55Te4hLjuObbd+Qm//fUFo3FzdCfEMuavJp6tuUEN8Q3Ku72+z3sIW0c2msSVpT1PH5z6F/SM1OBSDAI4DIxpGMDhtNRFAEHRp2cLjfzwyS3IUAohKiCKwdSM8rrTuktTw8a3jStl5b2tZre9Fzufm5HEo9VGKt/88Df3L23NkLyjfyalRqrb9OzTqmdiRqrTmYevCCtvKNRzcWdXy2qtuKu1rfVVQrb+rb1CE7Ps0myV1UeftP72fZ7mU82+VZqlezz5dE9WrVCfENIcQ3hB5NLmwF1VqTkpFywYiewuS/dPdSDqcdvqC8t5v3hcM5z6v1B9YOtHpb9bm8c2w4suGCZJ58NhmAWq616BTYiaeuf6qo49PH3ceq16+q7PMv2SRdu3bliSee4JZbbin62bRp09i5cycfffRRqcd5enqSlpZGcnIyDz30EAsWLCjx3G+99Rbh4aWPYJo2bRpjxozBw8MDgN69ezN37lx8fC7vj720FS6FYVbCLJRSjGw/0uxQKkQpRUCtAAJqBdApsNNFz2fkZLDv1L6Lav2JRxNZuH3hBcvQulZzJdgnuMRafxPfJni4epQZT0pGygUdn+uS15GVa8zgDvYJpmtw16KOz9Z1W9vtG6qjk7t6nsGDBzNv3rwLkvu8efN48803LTq+YcOGJSZ2S02bNo2hQ4cWJfeffvqpwucSlsnJy2HW+ln0urIXjb0dd2LdpXi4etCqbita1b14faK8/DySziSVWOs/v927UAPPBhfV+oO8g9h5YmdRMt95YidgvFGENQgrmiQUERRBQ6+GlfI7C0nuFxgwYABPP/002dnZuLm5sX//fpKTk+ncuTNpaWn069ePU6dOkZOTw8svv0y/fv0uOH7//v3ceuutbN68mczMTEaMGMHWrVtp0aIFmZmZReXGjx/PunXryMzMZMCAAbzwwgtMnz6d5ORkbrzxRvz9/VmxYgXBwcHExcXh7+/PO++8Q0xMDAD33XcfkyZNYv/+/fTq1YvOnTuzatUqGjVqxMKFC6lZs2apv+OGDRsYN24cGRkZNG3alJiYGHx9fZk+fTqffPIJ1atXp2XLlsybN48//viDiRMnAkbtcOXKlU43k/XHXT9yOO2wTTtS7ZlLNReu8LmCK3yuoFtItwue01pzMvPkfwn/vFr/8n3L+Xzj5xeUr1OzDp0bd2Zk6EgigiIIbxhOTdfS/xaFbdltcp+0dBIbjpS95G9OXg5H0o5Q37N+met/hNYPZVrP0hck8/Pzo2PHjixdupR+/foxb948Bg0ahFIKd3d3vvvuO2rXrk1KSgqdOnWib9++pXb0fPzxx3h4eJCYmEhiYuIFS/a+8sor1KlTh7y8PLp3705iYiIPPfQQ77zzDitWrCha2bFQfHw8sbGxrFmzBq011157LV26dMHX15ddu3bx5ZdfEh0dzf/93//xzTffMHRo6eu23XPPPbz//vt06dKFZ599lhdeeIFp06YxdepU9u3bh5ubW9ESwG+99RYffvghkZGRpKWl4e7ufCMUouKjaOTViN5X9TY7FLujlMLPww8/Dz+uaXTNRc9n5Wax79Q+3v7nbWatn8XkyMn8L/J/JkQqSuLwA2WPpB1h7+m9RTPTLldh0wwYTTKDBw8GjFrMk08+Sdu2benRowf//vsvR48eLfU8K1euLEqybdu2pW3b/0ZAfPXVV4SFhdG+fXu2bNlS5qJgf/31F3fccQe1atXC09OT/v37Fy0fHBISUrSBx/lLBpckNTWV06dP06WLsfTP8OHDWblyZVGMQ4YMYfbs2UUzYSMjI3nkkUeYPn06p0+fdroZsgdOH2Dp7qWMaj9K2n0rwL26Oy0CWjC1x1Te6PGGw/ZZOCu7/Yu+VA37fCkZKcSuj2VE+xFW2Wz29ttv55FHHinaZamwxj1nzhyOHz9OfHw8rq6uBAcHl7jM7/lKqtXv27ePt956i3Xr1uHr68u9995b5nkutf5P4XLBYCwZfH7zT3n8+OOPrFy5kkWLFvHSSy+xZcsWpkyZQp8+ffjpp5/o1KkTv/76K82b234Hmcoya/0sgDJnpIpL8/fwt+leoKJiHL7mXviHZa1dxD09PenatSsjR44sqrWDUeutW7curq6urFixggMHLr3R1A033MCcOXMA2Lx5M4mJiYCxXHCtWrXw9vbm6NGjLFny3+rIXl5enD17tsRzff/992RkZJCens53333H9ddfX+7fzdvbG19f36Ja/xdffEGXLl3Iz8/n0KFD3HjjjbzxxhucPn2atLQ09uzZQ5s2bZg8eTLh4eFs37693Ne0V7n5uUZH6lXO25Eqqja7rbmbafDgwfTv37+oeQZgyJAh3HbbbYSHhxMaGlpmDXb8+PGMGDGCtm3bEhoaSseOHQFjV6X27dvTqlWri5YLHjNmDL169aJBgwasWPHf/idhYWHce++9Ree47777aN++/SWbYErz2WefFXWoNmnShNjYWPLy8hg6dCipqalorXn44Yfx8fHhmWeeYcWKFbi4uNCyZcuiXaWcwU+7fiL5bDIf9S59iKsQjqzcS/5aiyz56xwc9f+sz9w+rD+8noMPH5T2duFQLF3y1+GbZYQor4OpB1mya4l0pAqnJsldVDmzEoyO1PvC7jM5EiFsx+6Su1nNRKL8HPH/qrAjteeVPbnC5wqzwxHCZuwqubu7u3PixAmHTBpVjdaaEydOONzEpiW7lvDv2X+r7IxUUXXYVYNjYGAgSUlJHD9+vOzCwnTu7u4EBlZ8YwszRCVE0cCzAX2u6mN2KELYlF0ld1dXV0JCQswOQzipQ6mH+GnXTzzR+Ykyl6oQwtHZVbOMELY0a/0stNaMai8zUoXzk+QuqoTCjtSbm95MiK98OhTOT5K7qBKW7l5K0pkkxnYYa3YoQlQKSe6iSoiKj6K+Z31ubXar2aEIUSkkuQunl3QmiR93/cjI0JHSkSqqDEnuwunFrI8hX+fLjFRRpUhyF04tLz+PmQkzpSNVVDmS3IVTW7p7KYfOHGJMmMxIFVWLJHfh1KISoqhXqx59r+5rdihCVCpJ7sJp/XvmXxbvXMzI9tKRKqoeSe7CaUlHqqjKJLkLp5SXn8fM9TO5qclNNPFtYnY4QlQ6Se7CKf2852cOph6UpX1FlSXJXTilGfEzqFurrnSkiiqrzOSulHJXSq1VSm1USm1RSr1QQhk3pdR8pdRupdQapVSwLYIVwhKFHakjQkdQw6WG2eEIYQpLau7ZQDetdTsgFOiplOpUrMwo4JTW+krgXeB164YphOViN8SSp/OkI1VUaWUmd21IK/jWteCr+D54/YDPCh4vALorpZTVohTCQoUzUns06cGVda40OxwhTGNRm7tSykUptQE4BvyitV5TrEgj4BCA1joXSAX8rBmoEJb4Ze8vHEg9IDNSRZVnUXLXWudprUOBQKCjUqp1sSIl1dIv2uVaKTVGKRWnlIqTfVKFLUTFRxHgEUC/5v3MDkUIU5VrtIzW+jTwO9Cz2FNJQBCAUqo64A2cLOH4KK11uNY6PCAgoEIBC1Gaw2cPs2jHIulIFQLLRssEKKV8Ch7XBHoA24sVWwQML3g8AFiutb6o5i6ELcWsj5GOVCEKVLegTAPgM6WUC8abwVda68VKqReBOK31ImAW8IVSajdGjf0um0UsRAnydT7RCdF0C+nGVX5XmR2OEKYrM7lrrROB9iX8/NnzHmcBA60bmhCW+2WP0ZH6eg8ZhSsEyAxV4SSiEoyO1Dta3GF2KELYBUnuwuEVdqTeG3qvdKQKUUCSu3B4n274lNz8XOlIFeI8ktyFQyvsSL0x+Eaa+TUzOxwh7IYkd+HQft37K/tO75OlfYUoRpK7cGhR8VH41fTjjubSkSrE+SS5C4d1JO0IC3cs5N7Qe3Gr7mZ2OELYFUnuwmEVdqRKk4wQF5PkLhxSYUdq1+Cu0pEqRAkkuQuHtHzfcvae2itL+wpRCknuwiEVdaTKjFQhSuR4yV1rWLfO7CiEiY6mHeW77d8xvN1w3Ku7mx2OEHbJ8ZL79OnQsSN8/bXZkQiTFHakju4w2uxQhLBbjpfc09ONf0eOhKNHzY1FVLrCjtQbrriB5v7NzQ5HCLvleMl9zBh45BHIyYHhwyE/3+yIRCVasW8Fe07tYWyHsWaHIoRdc7zk7u8Pb78N06bBsmXw7rtmRyQqUVRCFHVq1qF/i/5mhyKEXXO85F5o7Fjo3x+eeALi4syORlSCY+nH+G6bdKQKYQnHTe5KQXQ01K8Pd90FZ8+aHZGwsc82fEZOfg6jw6QjVYiyOG5yB6hTB+bOhX374P77zY5G2FC+zicqIYrrG19Pi4AWZocjhN1z7OQO0LkzPPcczJ4NX3xhdjTCRn7f/zu7T+6WdWSEsJDjJ3eAp56CG26A8eNh1y6zoxE2EBUfha+7L3e2uNPsUIRwCM6R3F1cYM4ccHMz2t/PnTM7ImFFx9OP8+22bxnebjg1XWuaHY4QDsE5kjtAYCDExEBCAjz5pNnRCCv6bGNBR6rMSBXCYs6T3AH69YMJE4xx8EuWmB2NsAKtNVHxUXRu3JmWAS3NDkcIh+FcyR3gzTehTRtj9uqRI2ZHIy7THwf+YNfJXbK0rxDl5HzJvWZNmDcP0tJg2DBZnsDBzYifgY+7DwNaDjA7FCEcivMld4CWLeG99+DXX+Gtt8yORlRQYUfqPW3vkY5UIcrJIZN7bn5u2YXuuw8GDjSGSa5ZY/ughNV9vvFzzuWdk7HtQlSAwyX3+ZvnU/fNumw+tvnSBZWCqCho1AgGD4bU1MoJUFiF1pqohCgigyJpVbeV2eEI4XAcLrknHE7gVNYpBi0YhNb60oV9fIzlCQ4eNCY4lVVe2I2VB1ay88ROqbULUUEOl9wfj3ycPlf1YevxrczfMr/sAyIi4IUX4Msv4bPPbB+gsIqohCh83H0Y2HKg2aEI4ZAcLrn7e/iz8K6FdGzUkQeXPMjx9ONlHzRlCnTtCg88ADt22DxGcXlSMlJYsHUBw9oOk45UISqozOSulApSSq1QSm1TSm1RSk0soYy3UuoHpdTGgjIjbBOuwaWaCzF9YziTfYYHlzxowQEuxsJi7u5G+3t2ti3DE5epsCNVlvYVouIsqbnnAo9qrVsAnYAJSqniUwUnAFu11u2ArsDbSqkaVo20mFZ1W/HsDc8yf8t8vtv2XdkHNGoEsbGwfr1Rkxd2qXBG6nWB19GmXhuzwxHCYZWZ3LXWh7XWCQWPzwLbgEbFiwFeSikFeAInMd4UbOp/kf8jtH4o9/90PyczT5Z9wG23wYMPGlv0/fijrcMTFfDnwT/ZcWKH7JEqxGUqV5u7UioYaA8UHzj+AdACSAY2ARO11hdNDVVKjVFKxSml4o4ft6CtvAyuLq7E9oslJSOFR5Y9YtlBb7wB7drBvfdCcvJlxyCsKyo+Cm83bwa2ko5UIS6HxcldKeUJfANM0lqfKfb0LcAGoCEQCnyglKpd/Bxa6yitdbjWOjwgIOAywv5PaP1QpkRO4bONn7FklwWLhbm7G8sTZGQYyxPk5VklDnH5TmScKOpI9XD1MDscIRyaRcldKeWKkdjnaK2/LaHICOBbbdgN7AOaWy/MS3v6hqdpGdCSMYvHcCa7+PtOCZo3h/ffh+XLjZq8sAtfJH5Bdl62jG0XwgosGS2jgFnANq31O6UUOwh0LyhfD7ga2GutIMviVt2NmL4xJJ9N5n+//M+yg0aMgEGD4JlnYPVq2wYoyqS1Zkb8DDoFdpKOVCGswJKaeyQwDOimlNpQ8NVbKTVOKTWuoMxLQIRSahPwGzBZa51io5hLdG3gtTzc6WFmxM9g+b7lZR+gFMyYAUFBxvDI06dtH6Qo1V8H/2J7ynZZ2lcIK1FlTuG3kfDwcB0XF2fVc2bkZNDuk3bk5eexafwmatWoVfZBq1cbm2wPGGDMYlXKqjEJywz7bhiLdiwi+ZFky/7fhKiilFLxWuvwsso53AzVS/Fw9WBW31nsO72Pp5Y/ZdlBnTrBSy/B/PnGOHhR6U5mnuTrLV8zrO0wSexCWIlTJXeAG664gQnXTGD6mun8ffBvyw6aPBm6dzfGwG/bZtsAxUW+2Gh0pMqMVCGsx+mSO8Br3V+jsXdjRi4aSWZOZtkHVKsGn38OHh5G+3tWlu2DFMB/S/te2+ha2tVvZ3Y4QjgNp0zuXm5eRN8Wzc4TO3nhjxcsO6hhQ2PVyI0b4X8WjrgRl23VoVVsPb5Vhj8KYWVOmdwBbmp6E6Paj+LNVW+y7t91lh3UuzdMmmSMgf/hB9sGKABjj1SvGl4MajXI7FCEcCpOm9wB3rr5Lep71mfUolGcyztn2UFTp0L79sY4+H//tW2AVdzJzJN8teUrhrYdKh2pQliZUyd3H3cfZtw6g03HNvHan69ZdpCbm7E8QVYWDB0qyxPY0OzE2TIjVQgbcerkDnBrs1sZ0mYIL//5MolHEy07qFkz+OAD+P13oyYvrK5wad+OjToSWj/U7HCEcDpOn9wB3uv5HnVq1mHkwpHk5lu4EvHw4XD33fDcc7BqlW0DrIL+SfqHLce3yIxUIWykSiR3Pw8/Puz9IfGH43l71duWHaQUfPwxNG4syxPYQFR8lNGR2lo6UoWwhSqR3AEGtBzAnS3u5Lnfn2N7ynbLDqpd22h/T06G0aPBpKUanM2pzFPM3zKfIW2G4FnD0+xwhHBKVSa5A3zQ+wNq1ajFyIUjycu3sKO0Y0d45RVYsABmzrRtgFXE7MTZZOVmSUeqEDZUpZJ7fc/6vNfzPf5J+ocP1n5g+YGPPQY33QQTJ8LWrbYLsAoonJEa3jCc9g3amx2OEE6rSiV3gCFthtDnqj488dsT7Dm5x7KDCpcn8PSEu+6CTAuWNBAlWp20ms3HNktHqhA2VuWSu1KKT279BFcXV0b/MJr8i7d6LVn9+kaC37QJHn/ctkE6saiEKDxreDK4zWCzQxHCqVW55A4QWDuQt256ixX7VxAdH235gT17wqOPwocfwvff2y5AJ3U66zTzN0tHqhCVoUomd4D7wu6je0h3Hv/lcQ6mHrT8wFdfhQ4dYORIOHTIdgE6oTmJc8jMzZSOVCEqQZVN7kopom+LJk/nMXbxWCzekapGDWPHppwcWZ6gHAr3SO3QoANhDcLMDkcIp1dlkztAiG8IU7tPZenupXy+8XPLD7zqKvjoI1i50hgmKcq05t81bDq2SWrtQlSSKp3cASZ0nEBkUCSTlk3i8NnDlh84bJhRc3/hBfjrL9sF6CSi4qOo5VqLwa2lI1WIylDlk3s1VY2YfjFk5WYx/sfxljfPgFF7b9LEWIPm5EnbBengUrNSmbd5Hne3uRsvNy+zwxGiSqjyyR2gmV8zXuz6Igt3LOSrLV9ZfqCXl9H+fviwLE9wCXM2GR2pYzuMNTsUIaoMSe4FHr7uYa5peA0PLHmA4+nHLT8wPBxeew2+/RZmzLBdgA6qsCM1rEEYHRp2MDscIaoMSe4FqlerTky/GFKzUpm4dGL5Dn7kEbjlFnj4Ydi82TYBOqh1yetYpdKbAAAZgklEQVRIPJooM1KFqGSS3M/Tum5rnrnhGb7c/CULty+0/MBq1YzNtb29ZXmCYoo6UmVGqhCVSpJ7MVM6T6FdvXaM/3E8pzJPWX5gvXrG8gRbthg1eUFqVipfbv6Swa0HU9utttnhCFGlSHIvxtXFlZh+MRxLP8ajPz9avoNvvtlYd+aTT4w2+Cpu7qa5ZORkyNh2IUwgyb0EYQ3CmBw5mdgNsSzbvax8B7/8MlxzDYwaBQfLsayBkynsSA2tH0p4w3CzwxGiypHkXopnujxDC/8WjP5hNGeyz1h+YOHyBHl5MGQI5Fq4Z6uTiUuOY+PRjYztMBallNnhCFHlSHIvhXt1d2L6xZB0Jokpv04p38FNmxr7r/71F7z0km0CtHNR8VF4uHpwd5u7zQ5FiCpJkvsldArsxMOdHubjuI/5ff/v5Tt4yBAYPtxopvnjD5vEZ6/OZJ+RjlQhTFZmcldKBSmlViiltimltiilShwErpTqqpTaUFDGabLZS91eoqlvU0YtGkX6ufTyHfzBB0YtfuhQOHHCNgHaobmb5pKeky4dqUKYyJKaey7wqNa6BdAJmKCUanl+AaWUD/AR0Fdr3QoYaPVITeLh6sGsvrPYe2ovz6x4pnwHe3rCvHlw9KjRwVoFlico7EhtV68d1zS8xuxwhKiyykzuWuvDWuuEgsdngW1Ao2LF7ga+1VofLCh3zNqBmqlLcBfGh49n2upp/HPon/IdHBYGr78OCxca7fBOLv5wPBuObGBMhzHSkSqEicrV5q6UCgbaA2uKPdUM8FVK/a6UildK3WOd8OzH6z1eJ8g7iJGLRpKVm1W+gydOhF69jMlNiYm2CdBOFHakDmkzxOxQhKjSLE7uSilP4Btgkta6+NjA6kAHoA9wC/CMUqpZCecYo5SKU0rFHT9ejsW57ICXmxdRt0axPWU7L/7xYvkOrlYNPv0UfH2N5QkyMmwSo9nOZp9l7qa53NXqLrzdvc0OR4gqzaLkrpRyxUjsc7TWJU29TAKWaq3TtdYpwEqgXfFCWusorXW41jo8ICDgcuI2xS1X3sKI0BG88fcbxCfHl+/gunXhiy9g+3aYNMk2AZrsy81fSkeqEHbCktEyCpgFbNNav1NKsYXA9Uqp6kopD+BajLZ5p/P2zW9Tt1ZdRi4aybm8c+U7uEcPmDwZoqPh669tE6CJouKjaFuvLR0bdTQ7FCGqPEtq7pHAMKBbwVDHDUqp3kqpcUqpcQBa623AUiARWAvM1Fo75dq3vjV9+eTWT0g8msjUv6aW/wQvvgjXXmts7nHggPUDNEl8cjzxh+MZEyYdqULYA1WubeWsKDw8XMfFxZlybWu4+5u7WbB1AfFj4mlTr035Dt63D0JDoXVrY4JT9eq2CbISjf1hLF8kfkHyo8n4uPuYHY4QTkspFa+1LnPBJpmhWkHTe03Hx92HkYtGkptfzvVjQkKMlSNXrTI22HZwZ7PPMnfzXAa1HiSJXYiy5OdDdrbNLyPJvYL8Pfz5oPcHxCXH8e4/75b/BIMHw4gR8MorsGKF9QOsRPM2zyPtXJrstiTEpRw5YmzJ2aQJ9O8PKSk2vZwk98swsOVA7mh+B8+seIYdKTvKf4L334dmzYzlCWz8H21LUQlRtKnbhk6BncwORQj7kpcHS5YYyTwoCJ580hga/dNPEBtr00tLcr8MSik+7P0hHq4ejFo0inydX74T1KplLE+QkgIjRzrk8gQJhxOIS46TGalCnO/QIaPJtUkT6N0b/vzTGAK9fTusXQtvvGF8crchSe6XqYFXA6b1nMbfh/7mw7Uflv8EoaHw5pvwww/GQmMOJjo+Gvfq7gxtO9TsUIQwV04OfP899OkDwcHw/PNw9dXw1Vfw77/G6/zqq8Hf39ixzd/fpuFIcreCYW2H0evKXkz5bQr7Tu0r/wkefND4g3jsMdi40foB2kjauTTmbJrDoFbSkSqqsL174amn4Ior4I47YP16mDIF9uyBn3+GgQONTXwqmSR3K1BKMePWGbgoF+774T7KPbxUKaP9zc8PBg2C9HIuLWySeZvncfbcWZmRKqqec+eMGvlNNxnLek+daiwS+P33xvaar7xiNMmYSJK7lQR5B/HWzW+xfN9yZibMLP8JAgJg9mzYudNYaMwBRMVH0SqgFdcFXmd2KEJUjh07jCaVwECjIrZzp9G2vn8/LF4M/frZzbwVSe5WNDpsNN1CuvHoz49yKPVQ+U/QrRs88QTMmgXz51s/QCtaf3g965LXSUeqcH6ZmUbFq0sXaN4cpk2Dzp2NES9798KzzxojYeyMJHcrUkoRfVs0eTqPcT+OK3/zDBidMNddB2PGGDNZ7VR0gtGROqztMLNDEcI2Nm2Chx6CRo1g2DCjU/S114yRMN9+ayzj7eJidpSlkuRuZU18m/Bqt1f5addPzE6cXf4TuLrC3LlGO/zddxs98HYm/Vw6sxNn83+t/g/fmr5mhyOE9aSnQ0yMUcFq2xZmzIBbboHffjOaYKZMgfr1zY7SIpLcbeCBjg8QERTBxKUTOZJ2pPwnCA42Vo5cvRqee87q8V2u+VvmGx2pMiNVOIuEBBg3Dho0MLbEPH0a3n7bqK1/+aXRZFrNsdKlY0XrIFyquTCr7ywycjK4/8f7K9Y8M3Ag3Hef0Qv/22/WD/IyzIifQcuAlkQERZgdihAVd+aMscZThw7G12efGUMZ//wTtm41dk6z8Vh0W5LkbiPN/ZvzQtcX+G77dyzYuqBiJ5k2zZj0MGwY2MnOVRuObGDtv2tlaV/hmLSGf/4xZoQ3aADjx0NurjGB8PBhI8F37mw0izo4Se429GjEo3Ro0IEJP00gJaMCa8cULk9w8iTce69dLE8QHR+Nm4sbw9pJR6pwICdPwvTpRjt6RIQxRv3uu2HNGtiwASZMAB/nmognyd2GqlerTmy/WE5nnWbi0gqOXW/XDt56yxh29d571g2wnNLPpTN702wGthpInZp1TI1FiDJpbeyXMHQoNGxozB+pWROiooxaenQ0dOzoFLX0kkhyt7E29drw1PVPMXfTXBbtWFSxk0yYAH37wv/+Z3T8mOSrLV9xJvsMYzuMNS0GIcp0/LhRIWrRArp2NdZtGjXKWBZg7VpjFzQvL7OjtDnZiakSnMs7R3hUOCkZKWydsLVi67CcOGHU4mvVgvh48PS0fqBluG7WdaRmpbLl/i3S3i7sS36+MfAgOtpYAiAnByIjjUQ+cCB4eJgdodXITkx2pIZLDWL7xXIs/RiP/fxYxU7i5wdz5sCuXcZCY5Us8Wgiq5NWy4xUYV+Sk411XK68Em6+2UjwEybAli3w118wfLhTJfbykOReSTo07MDjEY8za/0sftnzS8VO0qULPP00fPqpMdGpEkXFRxkdqTIjVZgtLw9+/BFuvx0aNzZeE8HBxmvi33/h3XehZUuzozSdNMtUoqzcLEI/CSUrN4tN4zfh5VaBdr/cXKMdMTHRaENs2tTqcRaXkZNBg7cbcFuz25jdvwKzboWwhoMHjXWXYmIgKQnq1jU2vBg1Cq66yuzoKo00y9gh9+ruxPSL4WDqQZ747YmKnaR6daN5xsXF2If13DnrBlmCwo5UWdpXVLqcHPjuO2M3o+BgeOklaNUKFiww1niZOrVKJfbykOReySKCIph47UQ+XPchKw+srNhJrrgCZs6EdeuMFelsLCo+iub+zbm+8fU2v5YQgLHRxRNPGM0u/fsbn1SfftpYhXHpUrjzTlM2wHAkktxN8HK3l2ni24RRi0aRkZNRsZPceSeMHQuvvw6/VLAN3wKbjm7in6R/ZEaqsL3sbGOp6+7djQ7SN980xqH/8IOxXvqLLxq1d2ERSe4mqFWjFjNvm8nuk7t5dsVl1LzfecfoOBo2DI4ds16A54lOiKaGSw3uaXePTc4vBNu3w6OPGkvr3nWXUTt/6SU4cAAWLoRbb7WbDTAciSR3k9wYciPjOozj3dXvsjppdcVO4uFhLE+QmmoM+crPt2qMGTkZfL7xcwa0HICfh59Vzy2quMxM+OILuP56Y7LR9Olw442wbJnRJPP000ayFxUmyd1Er9/0Oo28GjFy4Uiyc7MrdpI2bYwa/NKlxkJjVvT1lq9JzU6VpX2F9SQmGvM0GjaEe+6Bo0fhjTeMIYxff22MVXewpXXtldxFE9V2q03UbVFsS9nGSytfqviJxo0zliqdMsWYvWolUQlRNPNrxg1X3GC1c4oqKC3NGABw7bXGLOvoaGP0y4oV/+1JWreu2VE6HUnuJut5ZU+GtxvO1L+msv7w+oqdRCnjxVOvntFmefbsZce1+dhmVh1aJR2pomK0hrg4o9O/QQNjGYD0dOPTZXKyMZy3a1enXbTLHkgvhR1455Z3WLZnGSMWjmDd6HW4uriW/yR16hgz9Lp2hQceMNalvgzR8UZH6vDQ4Zd1HruUng4pKf99HT9+4ePDh43OvGbNoHZtY8idm5vxb0lfpT1Xnp/b8V6c5ZKaaiTu6GhjKd2aNWHQICO5X3edJPNKJMndDtSpWYeP+3zMHfPv4PW/X+fpG56u2Imuv94Y9/7883DTTcZSpxWQmZPJ54mfc2eLO/H3sPOdaHJyjLW6iyfo0pJ3SorRmVcSFxdj5538fOOYpCSj0zo725gsVviVl2f936NaNeu8SdjyjcjVteTkXLgBRlSUsU56Zia0bw8ffWSsme7tbf37Jcokyd1O3N78dga1GsSLf7zIHc3voFXdVhU70VNPGYsnjR8PnToZ44XLacHWBZzOOl35M1K1Nmp+pdWoS0rYp0+Xfj5vbyNZBwQYIy/atfvve3//ix97extJNiUFYmONqe0lbbOWl2e8qRRP+oVftv55aqpl5W3B1fXipJ+dbXSM1qpldJKOHm1sWydMVebaMkqpIOBzoD6QD0RprUvcNUIpdQ2wGhiktb7k3nJVcW2ZshxPP07Lj1oS4hPCqlGrqF6tgu+9hw4ZiaxpU/j773LP5Osc05lj6cfY8cCOy2tvz8q6dC26+HMpKcbaOSVxcys9KZeUsOvUqdozGLU27mVlvNls3Gisc/TSS8YQRmFTlq4tY0n2yAUe1VonKKW8gHil1C9a663FLugCvA4sq1DEgoBaAbzf630GfzOYaaun8VhEBZcHDgoyFljq39+oyb/5psWHbjm2hb8P/c2bN715YWLPyzOaP8pq9jj/cXp6yRdRyljCuDARX3WV0R57qYRdq5a015aHUkYt29XVuHe2dP4nHWE3yr0qpFJqIfCB1vqXYj+fBOQA1wCLpeZeMVpr7ph/B8v2LGPjuI0082tW8ZPdfz98/DEsWQI9e55/EWNETQkJetLpeXzkksC/m28m4MjZ/547dar0PVy9vC5diy7+nI+P83QgClHJLK25lyu5K6WCgZVAa631mfN+3giYC3QDZiHJ/bIkn02m1UetaF23NX/c+wfVVAVHrGZmGmtzJCUZibVGDaONOiWlxDbZzOrQ6FG4+V935iU2syxh+/mBu/tl/sZCCEtZs1mm8ISewDfApPMTe4FpwGStdd6l2miVUmOAMQCNGze29NJVTkOvhrx7y7uMWDiCj9d9zISOEyp2opo1jYWYeveG3buNdWh69So1YX9z/DdOLR3D2Bd/gpAbrftLCSEqlUU1d6WUK7AYWKa1fqeE5/cBhVndH8gAxmitvy/tnFJzvzStNb3m9OKvg3+x+f7NBPsEV/xkZY3+KHBD7A0cTjvMzgd2ysQlIeyU1TbrUMarfBawraTEDqC1DtFaB2utg4EFwP2XSuyibEopom6LQinF6B9Gc1k7Zvn7G1O8L5HYtx3fxp8H/5QZqUI4CUsacyOBYUA3pdSGgq/eSqlxSqlxNo6vSmvs3Zg3b3qTX/f+Ssz6GJteKyo+Ctdqrs45I1WIKkj2ULVz+Tqf7p93J+FwAlvv30qj2tZfBjUrN4uGbzfkpqY3MX/AfKufXwhhPbKHqpOopqox87aZ5OTlMO7HcZfXPFOKb7Z+w6msU7K0rxBORJK7A2hapymvdn+VxTsXM3fTXKufPyohiqa+TblRRsgI4TQkuTuIBzs+yHWB1/HQ0oc4mnbUaufdnrKdlQdWMqbDmIqPpxdC2B15NTsIl2ouzOo7i7RzaTyw5AGrnTc6PhrXaq7cG3qv1c4phDCfJHcH0iKgBc93eZ4FWxewYOslJwBbJCs3i882fsbtzW+nbi3ZCUcIZyLJ3cE8FvEYYQ3CmPDTBE5knLisc3277VtOZJ6o/KV9hRA2J8ndwbi6uBLTN4aTmSeZtGzSZZ0rKj6KJr5N6BbSzUrRCSHshSR3B9Sufjue7PwksxNns3jn4gqdY0fKDv448Aejw0ZLR6oQTkhe1Q7qqRueonXd1oxdPJbTWZfYjagU0QnRVK9WXTpShXBSktwdVA2XGsT0jeFI2hEe//nxch2bnZvNpxs+5fbmt1Pfs76NIhRCmEmSuwO7ptE1PHbdY8xcP5Nf9vxS9gEFvtv+ndGRKjNShXBaktwd3PNdn6eZXzNG/zCatHNpFh0TFR9FiE8I3Zt0t3F0QgizSHJ3cDVdaxLTN4aDqQd58rcnyyy/88ROVuxfIR2pQjg5eXU7gcjGkTzY8UHeX/s+fx7485Jlo+ONjtQR7WUzYyGcmSR3J/Fq91cJ8Qlh1KJRZOZkllgmOzebTzd+St+r+0pHqhBOTpK7k6hVoxbRt0Wz6+Qunvv9uRLLfL/9e1IyUqQjVYgqQJK7E+nepDtjwsbw9j9vs/bftRc9H5UQRbBPMDc1vcmE6IQQlUmSu5N546Y3aOjVkJELR5Kdm130810ndrF833LpSBWiipBXuZPxdvdmxq0z2HJ8C6/8+UrRz2cmzMRFuTAiVDpShagKJLk7od5X9eaedvfw2l+vseHIBs7lnSN2Qyx9r+5LA68GZocnhKgEktyd1Lu3vItfTT9GLhzJ11u+5njGcVnaV4gqRJK7k6pTsw4f9fmI9UfWM3bxWK7wvoKbmkhHqhBVhSR3J9a/RX8GthxIek46zf2bcyrrlNkhCSEqiSR3J/dh7w+JDIpk2Z5lxK6PNTscIUQlqW52AMK2AmoF8P1d3xO7PlaWHBCiCpHkXgX4e/jzeGT51nwXQjg2aZYRQggnJMldCCGckCR3IYRwQpLchRDCCUlyF0IIJyTJXQghnJAkdyGEcEJKa23OhZU6Dhyo4OH+QIoVw7EWe40L7Dc2iat8JK7ycca4rtBaB5RVyLTkfjmUUnFa63Cz4yjOXuMC+41N4iofiat8qnJc0iwjhBBOSJK7EEI4IUdN7lFmB1AKe40L7Dc2iat8JK7yqbJxOWSbuxBCiEtz1Jq7EEKIS7Dr5K6U6qmU2qGU2q2UmlLC825KqfkFz69RSgXbSVz3KqWOK6U2FHzdV0lxxSiljimlNpfyvFJKTS+IO1EpFWYncXVVSqWed7+erYSYgpRSK5RS25RSW5RSE0soU+n3y8K4Kv1+FVzXXSm1Vim1sSC2F0ooU+mvSQvjMus16aKUWq+UWlzCc7a9V1pru/wCXIA9QBOgBrARaFmszP3AJwWP7wLm20lc9wIfmHDPbgDCgM2lPN8bWAIooBOwxk7i6gosruR71QAIK3jsBews4f+x0u+XhXFV+v0quK4CPAseuwJrgE7FypjxmrQkLrNek48Ac0v6/7L1vbLnmntHYLfWeq/W+hwwD+hXrEw/4LOCxwuA7kopZQdxmUJrvRI4eYki/YDPtWE14KOUamAHcVU6rfVhrXVCweOzwDagUbFilX6/LIzLFAX3Ia3gW9eCr+KddpX+mrQwrkqnlAoE+gAzSyli03tlz8m9EXDovO+TuPiPvKiM1joXSAX87CAugDsLPsovUEoF2TgmS1kauxmuK/hYvUQp1aoyL1zwcbg9Ro3vfKber0vEBSbdr4Jmhg3AMeAXrXWp96wSX5OWxAWV/5qcBvwPyC/leZveK3tO7iW9gxV/N7akjLVZcs0fgGCtdVvgV/57dzabGffLEgkYU6rbAe8D31fWhZVSnsA3wCSt9ZniT5dwSKXcrzLiMu1+aa3ztNahQCDQUSnVulgRU+6ZBXFV6mtSKXUrcExrHX+pYiX8zGr3yp6TexJw/rtrIJBcWhmlVHXAG9t//C8zLq31Ca11dsG30UAHG8dkKUvuaaXTWp8p/Fittf4JcFVK+dv6ukopV4wEOkdr/W0JRUy5X2XFZdb9KhbDaeB3oGexp8x4TZYZlwmvyUigr1JqP0bTbTel1OxiZWx6r+w5ua8DrlJKhSilamB0OCwqVmYRMLzg8QBguS7onTAzrmLtsn0x2k3twSLgnoJRIJ2AVK31YbODUkrVL2xrVEp1xPi7PGHjaypgFrBNa/1OKcUq/X5ZEpcZ96vgWgFKKZ+CxzWBHsD2YsUq/TVpSVyV/ZrUWj+htQ7UWgdj5IjlWuuhxYrZ9F5Vt9aJrE1rnauUegBYhjFCJUZrvUUp9SIQp7VehPEi+EIptRvjHe8uO4nrIaVUXyC3IK57bR0XgFLqS4yRFP5KqSTgOYzOJbTWnwA/YYwA2Q1kACPsJK4BwHilVC6QCdxVCW/SkcAwYFNBWy3Ak0Dj8+Iy435ZEpcZ9wuMkTyfKaVcMN5QvtJaLzb7NWlhXKa8JourzHslM1SFEMIJ2XOzjBBCiAqS5C6EEE5IkrsQQjghSe5CCOGEJLkLIYQTkuQuhBBOSJK7EEI4IUnuQgjhhP4fm8SShtk/clYAAAAASUVORK5CYII=\n", - "text/plain": [ - "
" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "nn.fit_epoch(X_train,y_train,X_test,y_test,lr=0.01,epochs=5,bs=512,l2=2)" ] @@ -920,31 +831,9 @@ }, { "cell_type": "code", - "execution_count": 52, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Epoch 1. Training loss: 2.0999233784694957, Val loss:2.0358802555227933\n", - "Epoch 2. Training loss: 2.0020841962999687, Val loss:1.969175427216433\n", - "Epoch 3. Training loss: 1.9316250087626763, Val loss:1.898417676366854\n", - "Epoch 4. Training loss: 1.857334229971547, Val loss:1.8245502295970901\n", - "Epoch 5. Training loss: 1.7818421924295407, Val loss:1.7505339650271199\n" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD8CAYAAABw1c+bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XdcVfUbwPHPo+DeollaOXAhIiI5osSVWu5Rau4cOcHMymxqWWplipgrt6aZ5sgcmTMrB5qKAwXBitRcudIs9Pv741z9EaFcFM5lPO/Xi1dwz7n3PJzkued+z/N9vmKMQSmlVOaRxdUBKKWUspcmfqWUymQ08SulVCajiV8ppTIZTfxKKZXJaOJXSqlMRhO/UkplMpr4lVIqk9HEr5RSmYybqwNIjIeHhylZsqSrw1BKqXRj165dZ4wxRZzZN00m/pIlSxIWFubqMJRSKt0QkZ+d3VeHepRSKpPRxK+UUpmMJn6llMpk0uQYv1LKXv/88w+xsbH89ddfrg5FJSFHjhyUKFECd3f3u34NTfxKKWJjY8mbNy8lS5ZERFwdjroNYwxnz54lNjaWUqVK3fXrJDnUIyIPishGETkkIgdEJDiRfSqIyI8ick1EhiTY1lhEDotIlIgMvetIlVKp5q+//qJw4cKa9NM4EaFw4cL3/MnMmSv+OOBFY8xuEckL7BKRdcaYg/H2OQcEAS0TBJkVmAg8AcQCO0VkRYLnKqXSAE366UNK/H9K8orfGHPCGLPb8f0l4BBQPME+p4wxO4F/Ejy9OhBljIk2xvwNLARa3HPUt/PqqxAcDGfOpNohlFIqvUtWVY+IlASqAtudfEpx4Nd4P8eS4E0j3mv3FpEwEQk7ffp0csKynDsHkyZBSAh07w43biT/NZRStjt79iy+vr74+vpSrFgxihcvfuvnv//+26nX6N69O4cPH77jPhMnTmT+/PkpETKPPfYYe/bsSZHXcgWnb+6KSB5gCTDIGHPR2acl8liiq7sbY6YCUwH8/f2TvwJ8oUKwYwe0agUrV0KTJjB7NhQtmuyXUkrZp3DhwreS6Ntvv02ePHkYMuRftwoxxmCMIUuWxK9VZ86cmeRx+vfvf+/BZhBOXfGLiDtW0p9vjPkyGa8fCzwY7+cSwPFkPD95ypWD/fth4kTYuBF8fGDdulQ7nFIq9URFReHt7U2fPn3w8/PjxIkT9O7dG39/fypVqsSIESNu7XvzCjwuLo4CBQowdOhQqlSpQq1atTh16hQAr7/+OuPGjbu1/9ChQ6levTrly5fnhx9+AODPP/+kTZs2VKlShQ4dOuDv75/klf28efOoXLky3t7eDBs2DIC4uDg6d+586/GQkBAAPv74Y7y8vKhSpQqdOnVK8XPmrCSv+MW6kzAdOGSMGZvM198JlBWRUsBvQHvg2WRHmRwi0K8fPP44tG8PDRvCyy/Du+/CPdS9KpVpDBoEzgxj/PMPnDwJxYol/bfl6wuOpJscBw8eZObMmUyePBmAUaNGUahQIeLi4qhbty5t27bFy8vrX8+5cOECgYGBjBo1isGDBzNjxgyGDv1vQaExhh07drBixQpGjBjBmjVrmDBhAsWKFWPJkiXs3bsXPz+/O8YXGxvL66+/TlhYGPnz56dBgwasXLmSIkWKcObMGcLDwwE4f/48AGPGjOHnn38mW7Zstx5zBWeu+AOAzkA9Ednj+HpKRPqISB8AESkmIrHAYOB1EYkVkXzGmDhgALAW66bwImPMgVT6Xf6tcmXYuROefx7GjIHHHoOjR205tFKZwsmTEB1t/TeVlClThkceeeTWzwsWLMDPzw8/Pz8OHTrEwYP/LRDMmTMnTz75JADVqlXj2LFjib5269at/7PP1q1bad++PQBVqlShUqVKd4xv+/bt1KtXDw8PD9zd3Xn22WfZsmULnp6eHD58mODgYNauXUv+/PkBqFSpEp06dWL+/Pn3NAHrXiV5xW+M2UriY/Xx9zmJNYyT2LZVwKq7iu5e5coFkydDgwbQqxdUrWr9/GzqfuhQKl1z9sr8zBmYOdMqpvDwSJVQcufOfev7yMhIxo8fz44dOyhQoACdOnVKtJ49W7Zst77PmjUrcXFxib529uzZ/7OPMcm7vXi7/QsXLsy+fftYvXo1ISEhLFmyhKlTp7J27Vo2b97M8uXLeffdd9m/fz9Zs2ZN1jFTQubo1dO2rfXR1ccHOnaEbt3g8mVXR6VU+ubhAS+9lGpJP6GLFy+SN29e8uXLx4kTJ1i7dm2KH+Oxxx5j0aJFAISHhyf6iSK+mjVrsnHjRs6ePUtcXBwLFy4kMDCQ06dPY4zh6aefZvjw4ezevZvr168TGxtLvXr1+OCDDzh9+jRXrlxJ8d/BGZmnZcPDD8OmTTBihDXe/8MPsHAhJDGGp5RKG/z8/PDy8sLb25vSpUsTEBCQ4scYOHAgXbp0wcfHBz8/P7y9vW8N0ySmRIkSjBgxgjp16mCMoVmzZjRp0oTdu3fTo0cPjDGICKNHjyYuLo5nn32WS5cucePGDV555RXy5s2b4r+DMyS5H23s4O/vb1J1IZZNm6BTJzh1CkaPtiZ93aZMTKnM4NChQ1SsWNHVYbhcXFwccXFx5MiRg8jISBo2bEhkZCRubmnrGjmx/18isssY4+/M89PWb2OXOnVg717o0QMGD7ZKPmfN0pp/pTK5y5cvU79+feLi4jDGMGXKlDSX9FNCxvuNnFW4MCxdCp98Ai++aI3/z50LTzzh6siUUi5SoEABdu3a5eowUl3mHt8Qgf79rRm/hQpZNf+vvGLVJyulVAaVuRP/TT4+EBYGvXv/v+Y/OtrVUSmlVKrQxH9TrlwwZQp88QUcOWLNNFywwNVRKaVUitPEn1D8mv9nn7Ump2jNv1IqA9HEn5ibNf9vvGF1+PTzg927XR2VUhlWnTp1/jMha9y4cfTr1++Oz8uTJw8Ax48fp23btrd97aTKw8eNG/evyVRPPfVUivTSefvtt/nwww/v+XVSmib+23FzsyZ7bdgAV65AzZrw8ceQBuc9KJXedejQgYULF/7rsYULF9KhQwennv/AAw+wePHiuz5+wsS/atUqChQocNevl9Zp4k/KzZr/p56yav6bNrUmfimlUkzbtm1ZuXIl165dA+DYsWMcP36cxx577FZtvZ+fH5UrV2b58uX/ef6xY8fw9vYG4OrVq7Rv3x4fHx/atWvH1atXb+3Xt2/fW22d33rrLQBCQkI4fvw4devWpW7dugCULFmSM46V/MaOHYu3tzfe3t632jofO3aMihUr0qtXLypVqkTDhg3/dZzE7Nmzh5o1a+Lj40OrVq34448/bh3fy8sLHx+fWw3iNm/efGsxmqpVq3Lp0qW7PreJybx1/MmRsOa/ShWr5r9BA1dHplSKG7RmEHtOJt2W+Z/r/3Dy8kmK5SmGe9Y7d5r0LebLuMa3b/5WuHBhqlevzpo1a2jRogULFy6kXbt2iAg5cuRg6dKl5MuXjzNnzlCzZk2aN29+27VnJ02aRK5cudi3bx/79u37V2vlkSNHUqhQIa5fv079+vXZt28fQUFBjB07lo0bN+KRoO/Qrl27mDlzJtu3b8cYQ40aNQgMDKRgwYJERkayYMECpk2bxjPPPMOSJUvu2GO/S5cuTJgwgcDAQN58802GDx/OuHHjGDVqFDExMWTPnv3W8NKHH37IxIkTCQgI4PLly+TIkeOO5ze59IrfWfFr/gsWtGr+hw7Vmn+VaZ28fJLo89GcvJwybZnjD/fEH+YxxjBs2DB8fHxo0KABv/32G7///vttX2fLli23ErCPjw8+Pj63ti1atAg/Pz+qVq3KgQMHkmzCtnXrVlq1akXu3LnJkycPrVu35rvvvgOgVKlS+Pr6Andu/wzWGgHnz58nMDAQgK5du7Jly5ZbMXbs2JF58+bdmiUcEBDA4MGDCQkJ4fz58yk+e1iv+JPLx8fq8//CC1afn40brbLP0qVdHZlSKeJOV+bxnblyhpk/zaR71e545Lr3Dp0tW7Zk8ODB7N69m6tXr966Up8/fz6nT59m165duLu7U7JkyUTbMceX2KeBmJgYPvzwQ3bu3EnBggXp1q1bkq9zp15mN9s6g9XaOamhntv5+uuv2bJlCytWrOCdd97hwIEDDB06lCZNmrBq1Spq1qzJt99+S4UKFe7q9ROjV/x3I3dumDoVFi2Cw4e15l9lSh65PHgp4KUUSfpgVejUqVOH55577l83dS9cuEDRokVxd3dn48aN/Pzzz3d8ndq1a99aVH3//v3s27cPsNo6586dm/z58/P777+zevXqW8/JmzdvouPotWvXZtmyZVy5coU///yTpUuX8vjjjyf7d8ufPz8FCxa89Wlh7ty5BAYGcuPGDX799Vfq1q3LmDFjOH/+PJcvX+bo0aNUrlyZV155BX9/fyIiIpJ9zDvRK/578fTTUL26Ve//7LPwzTcwYQI4SsyUUsnToUMHWrdu/a8Kn44dO9KsWTP8/f3x9fVN8sq3b9++dO/eHR8fH3x9falevTpgrahVtWpVKlWq9J+2zr179+bJJ5/k/vvvZ+PGjbce9/Pzo1u3brdeo2fPnlStWvWOwzq3M3v2bPr06cOVK1coXbo0M2fO5Pr163Tq1IkLFy5gjOGFF16gQIECvPHGG2zcuJGsWbPi5eV1a0WxlJI52zKntLg4GD4cRo6EsmWtPv9Vq7o6KqWcpm2Z05d7bcusQz0pwc0N3nnHqvm/fNmq+R83Tmv+lVJpUpKJX0QeFJGNInJIRA6ISHAi+4iIhIhIlIjsExG/eNuux1ukfUVK/wJpys2a/8aNrZu/TZvC6dOujkoppf7FmSv+OOBFY0xFoCbQX0S8EuzzJFDW8dUbmBRv21VjjK/jq3lKBJ2meXjAsmUQGgrr11tVQOvXuzoqpZKUFod91X+lxP+nJBO/MeaEMWa34/tLwCGgeILdWgBzjGUbUEBE7r/n6NKrhDX/TzwBr76qNf8qzcqRIwdnz57V5J/GGWM4e/bsPU/oSlZVj4iUBKoC2xNsKg78Gu/nWMdjJ4AcIhKG9clhlDFm2W1euzfWpwUeeuih5ISVdsWv+R81yroHoDX/Kg0qUaIEsbGxnNahyTQvR44clChR4p5ew+nELyJ5gCXAIGPMxYSbE3nKzUuHh4wxx0WkNLBBRMKNMUf/s7MxU4GpYFX1OBtXmnez5r9BA2uhF19fq++/k82nlLKDu7s7pUqVcnUYyiZOVfWIiDtW0p9vjPkykV1igQfj/VwCOA5gjLn532hgE9YnhsznmWesPv/e3lbN/3PPaZ9/pZRLOFPVI8B04JAxZuxtdlsBdHFU99QELhhjTohIQRHJ7ngdDyAAuHNzjIysZEnYsgVefx1mzYJq1eCnn1wdlVIqk3Hmij8A6AzUi1eW+ZSI9BGRPo59VgHRQBQwDbi5ekJFIExE9gIbscb4M2/ih//X/K9frzX/SimX0Jm7rnTmjDXk89VXVr//WbOgSBFXR6WUSod05m564eEBy5db/X205l8pZZMMlfgXhi9k+KbhnLlyxtWhOE8EBgyA7duhQAGr5n/YMK35V0qlmgyT+C/8dYEuy7rw9ua3abe4HWevnHV1SMlTpQqEhUGPHvD++/D44xAT4+qolFIZUIZJ/Plz5Gdd53VULlqZDTEbKDm+JMPWD0tfV/+5c8O0afD55xARYdX8J1iAWiml7lWGSfwAgSUD2dd3H+F9w2lStgmjto6i5LiSDP12KKf/TEczEm/W/FeqZE30eu45+PNPV0ellMogMlTiv8m7qDcL2y5kf7/9NC/fnDHfj6HU+FK8vO5lTv15ytXhOedmzf9rr2nNv1IqRWXIxH+TVxEvPmvzGQf7H6RlhZZ89ONHlBpfiiHfDOH3y7dfrDnNcHODd9+1Kn0uXbJq/seP15p/pdQ9ydCJ/6YKHhWY13oeB/sdpE3FNny87WNKjS/F4LWDOXn5pKvDS1rdulaf/0aNYNAgaNZM+/wrpe5apkj8N5X3KM+cVnOI6B/BM5WeIWR7CKXGl2LQmkEcv3Tc1eHd2c2a/5AQWLfOqgLSmn+l1F3IVIn/prKFyzKr5SwiBkTQwbsDoTtCKT2+NEGrg/jt4m+uDu/2RGDgQKvPf/78WvOvlLormTLx3+RZyJMZLWZwZOAROvl0YlLYJEqHlGbAqgH8euHXpF/AVbTmXyl1DzJ14r+pdMHSfNr8UyIHRtK1Slem7JqC5wRP+q7syy8XfnF1eInTmn+l1F3SxB9PyQIlmdpsKlEDo+ju253pP03HM8ST5796np/P/+zq8BJ3s+bfy8uq+e/RQ2v+lVJ3pIk/EQ8XeJjJTScTFRRFT7+ezNo7C88JnvRa0YuYP9LgkEr8mv+ZM62a/z17XB2VUiqN0sR/Bw/lf4hPmnzC0aCjPF/teebsm0O50HL0WN6D6D+iXR3ev7m7/7vmv0YNqwJIa/6VUglo4ndCiXwlCH0qlOigaPr592N++HzKTShH9+XdiToX5erw/i1+zX9wsNb8K6X+QxN/MhTPV5zxT44nJjiGgdUHsnD/QiqEVqDrsq5Eno10dXj/l1jN/4YNro5KKZVGaOK/C/fnvZ+PG39MTHAMwTWC+eLAF1SYWIHOSztz+MxhV4dnuVnzv3075MsHDRpozb9SCtDEf0+K5SnGR40+IiY4hsE1B/PloS/x+sSLjl925NDpQ64Oz+LrC7t2WR0+338fatfWmn+lMrkkE7+IPCgiG0XkkIgcEJHgRPYREQkRkSgR2ScifvG2dRWRSMdX15T+BdKC+/LcxwcNPyAmOIYhtYawPGI5lT6pRIclHTh4Og2sLZ87N3z6qVXnf/Cg9Wbw+eeujkop5SLOXPHHAS8aYyoCNYH+IuKVYJ8ngbKOr97AJAARKQS8BdQAqgNviUjBFIo9zSmauyijnxhNTHAMrwS8wsojK/H+xJt2i9ux/9R+V4cH7dr9v+a/fXut+Vcqk0oy8RtjThhjdju+vwQcAoon2K0FMMdYtgEFROR+oBGwzhhzzhjzB7AOaJyiv0EaVCR3Ed5v8D4xwTG8+tirrI5cTeVJlWm7qC37ft/n2uBKlbJq/ocNgxkz4IEH4MMPdexfqUwkWWP8IlISqApsT7CpOBC/uU2s47HbPZ4peOTyYGT9kRwbdIzXH3+dddHrqDK5Cq0/b82eky6cYOXuDiNHQr9+cPEivPQSVKgA8+bB9euui0spZQunE7+I5AGWAIOMMRcTbk7kKeYOjyf2+r1FJExEwk5nsLrzQjkL8U69dzgWfIw3a7/JhpgNVJ1SlZYLW/LTCReuqjV8OIweDZ99BnnzQufO1vj/8uU68UupDMypxC8i7lhJf74x5stEdokFHoz3cwng+B0e/w9jzFRjjL8xxr9IkSLOhJXuFMxZkOF1h3Ns0DHeDnybzT9vxm+qH80XNGfX8V32B+ThAS+/bPX42b3buuH799/QsqW12pf2+1cqQ3KmqkeA6cAhY8zY2+y2AujiqO6pCVwwxpwA1gINRaSg46ZuQ8djmVqBHAV4q85bHAs+xog6I9j6y1b8p/nT9LOm7Pxtp2uCypLFavh24IBVAXTihFX7X78+bNvmmpiUUqnCmSv+AKAzUE9E9ji+nhKRPiLSx7HPKiAaiAKmAf0AjDHngHeAnY6vEY7HFJA/R37eCHyDY4OO8W7dd/kx9keqf1qdp+Y/xfbYhLdRbOLmZlX7HDkC48ZBeDjUqgUtWljfK6XSPTFpcCzX39/fhIWFuToM2126donQHaF89ONHnL16lkZlGvFW4FvUerCW64K6fNla4P2DD6wbwc8+C2+/DZ6erotJKfUfIrLLGOPvzL46czcNyZs9L68+/irHBh1jVP1R7Dqxi0dnPErDuQ35/pfvXRNUnjxWu+foaOt+wJdfQsWK0KcP/JaGl6lUSt2WJv40KE+2PLzy2CvEBMcwpsEY9pzcw2MzH6PBnAZ89/N3rgmqUCEYNQqOHoXnn7fmAJQpA0OGwJkzrolJKXVXNPGnYXmy5eGlgJeICY7hwyc+JPxUOLVn1abe7HpsPrbZNUHdfz+EhsLhw9bs348/htKlreGfiwmrfJVSaZEm/nQgd7bcvPjoi8QExzC24VgOnTlEndl1qDOrDhtjNuKS+zSlSsGsWdYN34YNrTkBpUtbs4CvXrU/HqWU0zTxpyO53HPxQq0XiA6KZnzj8Rw5e4R6c+oROCuQ9dHrXfMG4OUFixfDzp3g72/NAvb0hClTtA2EUmmUJv50KKd7ToJqBBEdHM2EJycQ/Uc0DeY24PGZj7Pu6DrXvAH4+8OaNbBpk7UGcJ8+VhuI+fO1DYRSaYwm/nQsh1sOBlQfQFRQFBOfmsjPF36m4byGBMwIYG3UWte8AQQGwtatsHKl1QaiUydtA6FUGqOJPwPI4ZaDfo/0I2pgFJOaTCL2YiyN5zem1vRarIpcZf8bgAg0aWK1gVi4EK5ds9pA1KqlS0AqlQZo4s9Asrtlp49/H6KCopjSdAonL5+kyWdNqPFpDVYeWWn/G0CWLNYaAAcPWm0gjh+3WkA0aGAtCamUcglN/BlQtqzZ6F2tN0cGHmFas2mcvnKaZgua8ci0R1hxeIX9bwAJ20Ds22c1gWvZUttAKOUCmvgzsGxZs9HTrydHBhxhevPp/PHXH7RY2IJqU6uxLGKZ/W8AOXJAcLA1Ceydd2DjRqhSxboPcPSovbEolYlp4s8E3LO681zV54joH8HMFjO5eO0irT5vReVJlem6rCun/7R5/YO8eeH1161F32+2gahQQdtAKGUTTfyZiHtWd7r5diNiQASzW87m98u/M2fvHKpNrcaGGBfcdE2sDYSnp7aBUCqVaeLPhNyyuNGlShfC+4XTtmJb4m7EUX9OfRrMacCO33bYH1D8NhDt2v2/DcTw4doGQqlUoIk/EyuWpxhfPPMF0cHRfNzoY/b+vpcan9ag9eetOXDqgP0BJWwD8fbb1hvARx9pGwilUpAmfkUOtxwMqjmI6KBohtcZzrfR394a/z92/pj9AcVvA1GtmjX0o20glEoxmvjVLXmz5+XNwDeJDo7mxVovsujAIspNKMfAVQM5efmk/QH5+8PatVb1z802EBUrahsIpe6RJn71Hx65PPig4QdEDYyiu293JoVNokxIGYatH8YfV/+wP6A6df7fBiJPnv+3gVixQttAKHUXNPGr2yqerzhTmk0hYkAELcq34P2t71M6pDTvf/c+f/79p73BJNYGokULbQOh1F1IMvGLyAwROSUi+2+zvaCILBWRfSKyQ0S84207JiLhjgXaM98iuhmEZyFPPmvzGXue30PAgwEM2zCMMiFlmLhjIn9f/9veYOK3gZg2zar71zYQSiWLM1f8s4DGd9g+DNhjjPEBugDjE2yva4zxdXYRYJV2VSlWhZXPrmRr962U9yjPgNUDKB9anjl753D9hs1j7m5u0LMnREZa5Z/x20DsT/QaRSnlkGTiN8ZsAc7dYRcvYL1j3wigpIjclzLhqbQo4KEANnXdxJqOayiUsxBdl3WlyuQqrmsDMWjQv9tA+PhA587aBkKp20iJMf69QGsAEakOPAyUcGwzwDcisktEeqfAsVQaISI08mzEzl47WdR2EXE34mj1eStqTq/J+uj19geUsA3EkiVWG4i+fbUNhFIJpETiHwUUFJE9wEDgJyDOsS3AGOMHPAn0F5Hat3sREektImEiEnb6tM29Y9RdyyJZeLrS0+zvt5/pzadz4tIJGsxtQIM5Ddge64Ix94RtIKZPt+YAvPSStoFQykGc+WguIiWBlcYY7yT2EyAG8DHGXEyw7W3gsjHmw6SO5+/vb8LC9F5wevRX3F9MDpvMyO9GcubKGVpWaMm7dd+lUtFKrgkoJsaaATx3rlUK+uKL8MILkC+fa+JRKpWIyC5n76Xe8xW/iBQQkWyOH3sCW4wxF0Ukt4jkdeyTG2gI6F23DC7+LOARdUawIWYDlSdVpsvSLsT8EWN/QKVKwezZVhuIJ574fxuIsWO1DYTKtJwp51wA/AiUF5FYEekhIn1EpI9jl4rAARGJwBrSCXY8fh+wVUT2AjuAr40xa1L+V1BpUd7seXkj8A2ig6IZ8ugQvjj4BeVDyzNg1QBOXDphf0CVKlnj/jt2WG0gXnwRypaFqVO1DYTKdJwa6rGbDvVkPL9d/I13trzD9J+m457FneAawbwc8DIFcxZ0TUCbNsGwYfDjj1CmDIwYAe3bW/MElEqHbB3qUcoZxfMVZ3LTyRzqf4hWFVsx+vvRlBpfive+e8/+WcBgtYH4/vv/t4Ho2FHbQKhMQxO/spVnIU/mt57Pnj57qP1wbV7b8BplQsoQuiPU/lnACdtA/PWX1Qbi0Uet+QBKZVCa+JVL+Nznw4oOK/j+ue+p4FGBgasHUj60PLP3zLZ/FvDNNhAHDlhtIGJjoV4962bwDhcsTKNUKtPEr1zq0QcfZWPXjbdmAXdb3g2fyT4sPbTU/lnA7u7/bgOxZw/UqAGtWmkbCJWhaOJXLndzFnBYrzC+ePoLrt+4TutFranxaQ2+jf7W/oButoGIjrbaQGzYYLWBePppGDpUJ4KpdE+relSaE3cjjrl75/L25rf55cIv1CtVj/fqvUeNEjVcE9C5czB6tFX7HxcHjz8Oy5ZZs4SVSiO0qkela25Z3OhetTtHBhxhXKNxhP8eTs3pNWm5sCX7T7lgyKVQISvx//QTVK9uLQpTpgx8+KF1Q1ipdEYTv0qzsrtlJ7hmMNHB0bxT9x02HtuIzyQfOi/tTPQf0fYH5O1t9fzft89aAOall6ylIBcsgBs37I9HqbukiV+leXmy5eH12q8THRTNS4++xOKDi6kQWoH+X/d3zSxgb29YtQrWrYMCBeDZZ621ALZssT8Wpe6CJn6VbhTOVZjRT4zmaNBRelTtwdTdUykTUoah3w51zVrADRrArl1WL6ATJyAw0JoHEBFhfyxKJYMmfpXuPJD3ASY1nURE/whaV2zNmO/H3JoFfPnvy/YGkyULdOkCR47Ae+9ZE7+8vaF/fzh1yt5YlHKSJn6VbpUpVIZ5reexp88eAksG3poFPGH7BK7FXbM3mJw54dVXISo26/v6AAAdI0lEQVQK+vSBKVOsdQBGjoQrV+yNRakkaOJX6Z7PfT4sb7+cH577Aa8iXgStCXLdLOCiRSE01JoFXL++tSpYuXIwaxZctzkWpW5DE7/KMGo9WIsNXTawttNaPHJ50G15NypPqsyXh760fxZw+fKwdKl1w7d4ceje3WoHvW6dvXEolQhN/CpDEREalmnIzl47Wfz0YgyGNovaUP3T6qw7us7+N4DHH4dt26wmcBcuQMOG0LixtTCMUi6iiV9lSCJCG682hPcNZ2aLmZz68xQN5zWk/pz6bIvdZncwVhO4iAj46COr8ZuvL/TooQvBK5fQxK8yNLcsbnTz7caRAUcY33g8B04foNb0WrRY2ML+WcDZs8PgwdYN4EGDYN48axWwN9+ES5fsjUVlapr4VaaQ3S07QTWCOBp0lHfrvsumY5tcNwu4UCHryv/QIWje3GoEV7asVQkUF2dvLCpT0sSvMpU82fLwWu3XiAmO4eWAl1lycAnlQ8vT7+t+9s8CLl3aGvvfts1K/H36WF1AV67UVcBUqtLErzKlQjkLMarBKKKCoujl14tpu6fdmgV87uo5e4OpUcOq/lm61Cr5bNbMWghm1y5741CZRpKJX0RmiMgpEUl0QFRECorIUhHZJyI7RMQ73rbGInJYRKJEZGhKBq5USngg7wN80uQTIvpH0MarDWO+H0Pp8aUZuWWkvbOARaBlS2vBl9BQ67/+/tCpE/z8s31xqEzBmSv+WUDjO2wfBuwxxvgAXYDxACKSFZgIPAl4AR1ExOueolUqlZQpVIa5reayt89e6pSsw+sbX6dMSBlCtofYOwvY3d1q9xAVZc0EXrLEmhPwyitw/rx9cagMLcnEb4zZAtzps68XsN6xbwRQUkTuA6oDUcaYaGPM38BCoMW9h6xU6ql8X2WWtV/Gjz1+pFKRSgSvCaZ8aHlm7ZlF3A0bb7zmz2/1/jlyxCoF/eADqwXE+PHwt82L0qsMJyXG+PcCrQFEpDrwMFACKA78Gm+/WMdjiRKR3iISJiJhp0+fToGwlLp7NUvUZH2X9XzT6RuK5C5C9+XdqTixIl2WduH0nzb++3zwQav7565dVu3/oEHg5QWLF+sNYHXXUiLxjwIKisgeYCDwExAHSCL73vZfqjFmqjHG3xjjX6RIkRQIS6l7IyI8UeYJdvTcwZJnlnDxr4vM3TeXR6Y9wg+//mBvMFWrWu0eVq2y1gR++mkICIAfbI5DZQj3nPiNMReNMd2NMb5YY/xFgBisK/wH4+1aAjh+r8dTym4iQuuKrdnbdy9tKrbhatxVAmYE0HZRWyLPRtoZCDz5JOzZA9OmwbFjVvJv29a6J6CUk+458YtIARHJ5vixJ7DFGHMR2AmUFZFSju3tgRX3ejylXKVYnmIsfmYx0UHRDK8znDVRa/D6xIuBqwbaO/zj5gY9e0JkJAwfDmvWWMM/wcFw5ox9cah0y5lyzgXAj0B5EYkVkR4i0kdE+jh2qQgcEJEIrAqeYABjTBwwAFgLHAIWGWMOpMYvoZSdcmfLzZuBb3I06Cg9q/ZkUtgkyoSU4b3v3uPKPzb23s+d22r3EBlpdf8MDbVuAI8Zo4vAqzsS27sVOsHf39+EhYW5OgylnBJxJoKh3w5l+eHlFM9bnHfrvUtnn85kzZLV3kAOHLDKPr/+Gh56yKoK6tDBWiVMZXgisssY4+/MvvovQql7VMGjAsvaL2Nzt80Uz1ec7su74zfVj7VRa+0NpFIlq93D+vVQuLA1+euRR6zlIJWKRxO/Uimk9sO12dZjG5+3/ZzLf1+m8fzGNJzbkD0n99gbSL16EBYGc+bA6dPWz82aWU3hlEITv1IpSkR4ptIzHOx3kHGNxrHrxC78pvjRdVlXfrnwi32BZMkCnTvD4cMwapTVC6hyZasR3MmT9sWh0iRN/Eqlguxu2QmuGczRoKO89OhLfL7/c8pNKMcr617h/F82tl7ImdMa94+Kgn79YPp06wbwO+/An3/aF4dKUzTxK5WKCuQowOgnRnNk4BGeqfQMH/zwAZ4hnozfNp6/r9vYeqFIEQgJgYMHoVEjqxqoXDnrjUAXgc90NPErZYOH8j/EnFZz2NV7F77FfBm0dhAVJ1Zk0YFF9q4DXLas1fht61arHUTPntas4LU234hWLqWJXykbVb2/Kus6r2N1x9Xkds9Nu8XtqDm9Jt/9/J29gQQEwI8/wqJF1pBP48bWQvB799obh3IJTfxK2UxEaOzZmJ+e/4kZzWcQezGW2rNq03JhSyLORNgZiNXz5+BB+PhjqxKoalVrMlhsrH1xKNtp4lfKRbJmyUr3qt2JHBjJyHoj2RCzAe9PvOm7si+/X/7dvkCyZ7e6fh49Ci++CJ99Zo3/v/YaXLxoXxzKNpr4lXKxXO65GPb4MKKCoujj34dPf/oUzwmejNg8gj//trHypmBBq+//4cPWamDvvWdVAH3yCfzzj31xqFSniV+pNKJo7qKEPhXKgX4HaFSmEW9teouyE8ry6e5P7V0EpmRJ66p/xw6oWNFaEaxyZVi+XNcAyCA08SuVxpQrXI7Fzyzm++e+p2SBkvT6qhe+k335+sjX9lYAPfIIbNpkJXywPgXUqQM7d9oXg0oVmviVSqMeffBRvn/ue5Y8s4S/r/9N0wVNqT+nPmHHbWxgKALNm0N4uDXkc+gQVK9uNX+LibEvDpWiNPErlYbdXATmQL8DhD4ZSvipcB6Z9gjPLnmWY+eP2ReIuzv07WvNAH7tNetTQIUKMGQI/PGHfXGoFKGJX6l0wD2rO/2r9+do0FGGPTaMpRFLKR9aniHfDOHc1XP2BZIvH7z7rrUIfMeOMHYslCljlYNeu2ZfHOqeaOJXKh3Jlz0fI+uPJHJgJB0rd2Tsj2PxDPHkox8+4lqcjYm3RAmYMQN++gn8/WHwYOtG8KJFegM4HdDEr1Q6VCJfCWa0mMGePnuoUaIGQ9YNocLECnwW/hk3zA37AqlSBb75xlr+MU8eaNcOatWyWkKoNEsTv1LpmM99PqzuuJp1nddRIEcBOn7ZkerTqrMxxubFVxo1sq7+p0+HX3+Fxx+HJk3g5Zd1HeA0SBO/UhlAg9IN2NV7F3NazuHUn6eoN6ceTT9ryoFTNi5znTUrPPecNf4/YgR8+601IaxVKzhn430IlSRnFlufISKnRGT/bbbnF5GvRGSviBwQke7xtl0XkT2OrxUpGbhS6t+ySBY6V+nM4QGHGd1gNFt/2YrPZB96rejFiUsn7Askd2544w3YvRtq1IDvv///DeC/bWxFrW7LmSv+WUDjO2zvDxw0xlQB6gAfiUg2x7arxhhfx1fze4pUKeWUnO45eTngZaKCohhYfSCz987Gc4Inb218i0vXLtkXSKVKsG0b7Ntn1f4PHmw99uWXegPYxZJM/MaYLcCdPqcZIK+ICJDHsa+N88uVUonxyOXBuMbjONT/EE3LNWXElhGUnVCWyWGT+ee6jb13vL2tfv+rV0O2bNCmDQQGWt1AlUukxBh/KFAROA6EA8HG3CoryCEiYSKyTURapsCxlFLJVKZQGT5v+znbemyjXOFy9P26L5UnVWZ5xHJ7W0A0bmz1+580CSIirJYQnTtbN4OVrVIi8TcC9gAPAL5AqIjkc2x7yBjjDzwLjBORMrd7ERHp7XiTCDt9+nQKhKWUiq9GiRps7raZ5e2XIyK0/LwlgbMC2R673b4g3NysBd+jomDoUPjiC6sF9BtvwCUbh6EyuZRI/N2BL40lCogBKgAYY447/hsNbAKq3u5FjDFTjTH+xhj/IkWKpEBYSqmERITm5ZsT3jecSU0mcfjsYWpOr0m7xe04eu6ofYHkywfvv29d+bdqZc0GLlsWPv1U1wC2QUok/l+A+gAich9QHogWkYIikt3xuAcQABxMgeMppe6RWxY3+vj3IWpgFG/WfpOVR1ZScWJFBq0ZxJkrNtbd32wBvW2bVfnTq5e1Cti6dfbFkAk5U865APgRKC8isSLSQ0T6iEgfxy7vAI+KSDiwHnjFGHMGa9w/TET2AhuBUcYYTfxKpSF5s+dleN3hRA6MpJtvNybsmIBniCejt47m6j9X7QukRg1rtu+iRXD5srX+b5Mm1rKQKsWJrTd3nOTv72/C9I6/UrY7cOoAQ9cPZeWRlTyY70HerfcunXw6kUVsnOt57RpMmADvvGMtBP/88/D226BDwHckIrsc91STpDN3lVK3VCpaia86fMWGLhsomrsoXZd1pdrUanwb/a19QWTPbrV7joqybgRPmWItATlmDPz1l31xZGCa+JVS/1G3VF129NrBZ60/4/xf53li7hM0nteYfb/vsy+IIkUgNNRaBKZ2bXjlFasD6Oef6wSwe6SJXymVqCyShQ6VOxDRP4IPn/iQ7b9tx3eyL92Xdyf2Yqx9gVSsCF99Zd3wzZcP2reHgADrhrC6K5r4lVJ3lN0tOy8++iJHg44yuNZgPgv/jLITyjJs/TAu/HXBvkAaNLD6/0yfbi37WKuWtQTksWP2xZBBaOJXSjmlUM5CfNjwQyL6R9C6Ymve3/o+nhM8Cd0Ryt/XbWq+drMDaGSkNenr5hKQQ4fCBRvfhNI5TfxKqWQpVbAU81vPZ2evnXgX9Wbg6oFU+qQSSw4usa8FRJ48VuvnI0esxV9Gj7YmgE2eDHHaKiwpmviVUnfF/wF/NnTZwMoOK8mWNRttv2hLwIwAfvj1B/uCKFECZs+2Gr55eVkLwlepYjWE0xvAt6WJXyl110SEJuWasLfPXqY1m8ax88cImBFAm0VtOHL2iH2BVKsGGzfC0qVWz/+nnrKawoWH2xdDOqKJXyl1z9yyuNHTryeRAyMZUWcE3xz9Bq+JXjw6/VEOnT5kTxAi0LIlHDgA48bBzp3g6wu9e8PJk/bEkE5o4ldKpZjc2XLzRuAbRA2M4pEHHuHH2B+pOqUqI7eM5Mo/V+wJIls2CA62JoAFBcHMmdb4/8iRcNXGNhRpmCZ+pVSKuy/PfXz17FcMqTWEuiXr8vrG1yk3oRwzf5rJ9Rs2dd8sVMha7vHgQXjiCXj9dShfHubNgxs3kn5+BqaJXymVKjxyefBBww9Y3Wk1m7ttpni+4jy34jn8pvqxNmqtfYGULWst97hpkzUbuHNnqyncd9/ZF0Mao4lfKZXqaj9cm209trGwzUIuXbtE4/mNaTi3IXtP7rUviMBAa9x/zhw4ccJqA9G2LRy1cR2CNEITv1LKFiJCO+92HOp/iLENxxJ2PIyqU6rSbVk3+1pAZMliXfEfOWLNA1izxmoJ8eKL8Mcf9sSQBmjiV0rZKrtbdl6o9QJHg47yYq0XWbB/gf0tIHLlsmb+RkZCly7WvQBPTwgJgX9sXIjeRTTxK6VcomDOgnzQ8AMODzhMm4ptbrWAmLB9gn0tIO6/31ru8aefrJW/goPB2xtWrMjQE8A08SulXKpkgZLMaz2PsF5hVC5amaA1Qfa3gKhSxer+uXKlNRzUogXUr2+9IWRAmviVUmlCtQeqsb7L+n+1gHhs5mP2tYAQsZZ73LcPJk60Zv1Wqwbdu8Nvv9kTg0008Sul0oz4LSCmNp1K9B/RBMwIoO2itkSejbQnCHd36NfPGv8fMsRaDL5cOWv5xz//tCeGVOZU4heRGSJySkT232Z7fhH5SkT2isgBEekeb1tXEYl0fHVNqcCVUhmXWxY3elXrReTASIbXGc6aqDV4feJF0Oogzlw5Y08QBQpYyz1GREDTpjB8uPUGMGtWup8A5uwV/yyg8R229wcOGmOqAHWAj0Qkm4gUAt4CagDVgbdEpODdh6uUykzyZMvDm4FvEhUURY+qPfhk5yeUCSnD+9+9z9V/bGq/UKqUtdzj99/Dgw9aQz/+/lZTuHTKqcRvjNkCnLvTLkBeEREgj2PfOKARsM4Yc84Y8wewjju/gSil1H8Uy1OMyU0nE943nMCHAxm2YRjlQssxe89s+1pAPPoo/PgjLFgA585BvXrWTeDDh+05fgpKqTH+UKAicBwIB4KNMTeA4sCv8faLdTymlFLJVrFIRVZ0WMGmrpsolqcY3ZZ3o9rUaqw7us6eAESsNX8jImDUKOuq39vbKgM9e9aeGFJASiX+RsAe4AHAFwgVkXyAJLJvovVZItJbRMJEJOz06dMpFJZSKiMKLBnI9p7b+az1Z1y4doGG8xrSeF5j9v2+z54AcuSAV16xOoD27AmhodYEsLFj4do1e2K4BymV+LsDXxpLFBADVMC6wn8w3n4lsD4V/IcxZqoxxt8Y41+kSJEUCksplVFlkSx0qNyBiP4RfNTwI3b8tgPfyb48t/w5frtoU/ll0aIwaZJVAlqrltX6oVIlWLIkTU8AS6nE/wtQH0BE7gPKA9HAWqChiBR03NRt6HhMKaVSRHa37AyuNZijQUcZXGsw88PnU3ZCWV5b/xoXr120J4hKlWDVKqv3T86cVvO32rWtpnBpkLPlnAuAH4HyIhIrIj1EpI+I9HHs8g7wqIiEA+uBV4wxZ4wx5xzbdjq+RjgeU0qpFFUwZ0E+bPghEf0jaFmhJe9tfQ/PEE8m7pjIP9dt6r/TqJE123fqVKsRXPXq0KkT/PKLPcd3ktg2JToZ/P39TVhYmKvDUEqlY2HHwxjyzRA2/7yZcoXLMar+KFpWaIlVfGiDS5esG8Bjx1o/Dx4MQ4dC3rypcjgR2WWM8XdmX525q5TKkPwf8Gdj14181eErskpWWi9qzeMzH2db7DZ7Asib11ru8fBhaNMG3nvPWhRm2jS4blMJ6m1o4ldKZVgiQtNyTdnXdx9Tmk4h6lwUtabX4ukvnibqXJQ9QTz0kLXc4/btVuVP795WJ9BvvrHn+InQxK+UyvDcsrjRu1pvooKieCvwLVZFrsJrohfBq4PtawFRvbq13OPixVbPn0aN4KmnrDWBbaaJXymVaeTJloe367xN1MAouvl2I3RnKJ4hnozeOtqeFhAi1rDPwYPw0Ufwww/g4wN9+8KpU6l/fAdN/EqpTOf+vPcztdlU9vXZx+MPP87Q9UMpH1qeuXvncsPY0IAte3brZm9UlNUJ9NNPoXRp6xPA8USnOqUoTfxKqUyrUtFKfNXhKzZ02UDR3EXpsqwL1aZW49vob+0JwMPDWu5x/354+GFYvRrmzk31w2riV0plenVL1WVHrx3Mbz2fP67+wRNzn+DJ+U8S/nu4PQGULw+bN1sLwPfokeqH0zp+pZSK56+4vwjdEcrI70Zy8dpFulXpxoi6IyieL233l9Q6fqWUuks53HIw5NEhHA06yqAag5gXPo+yE8ryxoY3uHTtkqvDSxGa+JVSKhGFchbio0YfEdE/ghYVWvDud+/iOcGTSTsn2dcCIpVo4ldKqTsoVbAUC9osYHvP7VTwqEC/Vf2oPKkyyyKWkRaHyp2hiV8ppZxQvXh1NnXdxPL2yxERWn3eisBZgWyP3e7q0JJNE79SSjlJRGhevjnhfcOZ3GQyR84eoeb0mrRb3I7oP6JdHZ7TNPErpVQyuWVx43n/54kcGMmbtd9k5ZGVVAitwAtrXuDslbS/BKMmfqWUukt5s+dleN3hRA6MpGuVroTsCKFMSBk++P4D/or7y9Xh3ZYmfqWUukcP5H2Aac2nsbfPXgIeCuDlb1+mQmgF5u+bb08LiGTSxK+UUinEu6g3Xz/7Neu7rKdwrsJ0WtqJR6Y9woaYDa4O7V808SulVAqrV6oeO3vtZF6reZy5cob6c+rT5LMmHDh1wNWhAZr4lVIqVWSRLHT06cjhAYcZ02AM3//yPT6Tfei1ohcnLp1wbWwuPbpSSmVwOdxy8FLASxwNOkpQ9SBm752N5wRP3tr4Fpf/vuySmJJM/CIyQ0ROicj+22x/SUT2OL72i8h1ESnk2HZMRMId27TrmlIq0yqcqzAfN/6YiAERNCvXjBFbRuAZ4smUsCnE3YizNRZnrvhnAY1vt9EY84ExxtcY4wu8Cmw2xpyLt0tdx3anusYppVRGVrpgaRa2Xci2HtsoV7gcfb7uQ+VJlVlxeIVtLSCSTPzGmC3AuaT2c+gALLiniJRSKhOoUaIGm7ttZlk7q+dPi4Ut8Azx5JcLv6T6sVNsjF9EcmF9MlgS72EDfCMiu0SkdxLP7y0iYSISdvr06ZQKSyml0iwRoUWFFoT3DadVhVZEn4/m8/2fp/px3VLwtZoB3ycY5gkwxhwXkaLAOhGJcHyC+A9jzFRgKlgLsaRgXEoplaa5Z3VnarOp1CpRi+5Vu6f68VKyqqc9CYZ5jDHHHf89BSwFqqfg8ZRSKsPwyOXBSwEv4ZHLI9WPlSKJX0TyA4HA8niP5RaRvDe/BxoCiVYGKaWUsk+SQz0isgCoA3iISCzwFuAOYIyZ7NitFfCNMebPeE+9D1gqIjeP85kxZk3Kha6UUupuJJn4jTEdnNhnFlbZZ/zHooEqdxuYUkqp1KEzd5VSKpPRxK+UUpmMJn6llMpkNPErpVQmI3b1hkgOETkN/HyXT/cAzqRgOClF40oejSt5NK7kyYhxPWyMKeLMjmky8d8LEQlLiw3hNK7k0biSR+NKnswelw71KKVUJqOJXymlMpmMmPinujqA29C4kkfjSh6NK3kydVwZboxfKaXUnWXEK36llFJ3kG4Tv4g0FpHDIhIlIkMT2Z5dRD53bN8uIiXTSFzdROR0vHWKe9oQU1LrJouIhDhi3icifqkdk5Nx1RGRC/HO1Zs2xfWgiGwUkUMickBEghPZx/Zz5mRctp8zEckhIjtEZK8jruGJ7GP736OTcdn+9xjv2FlF5CcRWZnIttQ9X8aYdPcFZAWOAqWBbMBewCvBPv2AyY7v2wOfp5G4ugGhNp+v2oAfsP82258CVgMC1AS2p5G46gArXfDv637Az/F9XuBIIv8fbT9nTsZl+zlznIM8ju/dge1AzQT7uOLv0Zm4bP97jHfswcBnif3/Su3zlV6v+KsDUcaYaGPM38BCoEWCfVoAsx3fLwbqi6NHtIvjsp1Jet3kFsAcY9kGFBCR+9NAXC5hjDlhjNnt+P4ScAgonmA328+Zk3HZznEOLjt+dHd8Jbx5aPvfo5NxuYSIlACaAJ/eZpdUPV/pNfEXB36N93Ms//0DuLWPMSYOuAAUTgNxAbRxDA8sFpEHUzkmZzgbtyvUcnxUXy0ilew+uOMjdlWsq8X4XHrO7hAXuOCcOYYt9gCngHXGmNueLxv/Hp2JC1zz9zgOeBm4cZvtqXq+0mviT+ydL+E7uTP7pDRnjvkVUNIY4wN8y//f1V3JFefKGbuxpqFXASYAy+w8uIjkAZYAg4wxFxNuTuQptpyzJOJyyTkzxlw3xvgCJYDqIuKdYBeXnC8n4rL971FEmgKnjDG77rRbIo+l2PlKr4k/Foj/zlwCOH67fUTEDchP6g8rJBmXMeasMeaa48dpQLVUjskZzpxP2xljLt78qG6MWQW4i0jqL0gKiIg7VnKdb4z5MpFdXHLOkorLlefMcczzwCagcYJNrvh7TDIuF/09BgDNReQY1nBwPRGZl2CfVD1f6TXx7wTKikgpEcmGdfNjRYJ9VgBdHd+3BTYYx50SV8aVYBy4OdY4rautALo4KlVqAheMMSdcHZSIFLs5riki1bH+vZ614bgCTAcOGWPG3mY328+ZM3G54pyJSBERKeD4PifQAIhIsJvtf4/OxOWKv0djzKvGmBLGmJJYOWKDMaZTgt1S9XwlufRiWmSMiRORAcBarEqaGcaYAyIyAggzxqzA+gOZKyJRWO+U7dNIXEEi0hyIc8TVLbXjkqTXTV6FVaUSBVwBuqd2TE7G1RboKyJxwFWgvQ1v3mBdkXUGwh3jwwDDgIfixeaKc+ZMXK44Z/cDs0UkK9YbzSJjzEpX/z06GZftf4+3Y+f50pm7SimVyaTXoR6llFJ3SRO/UkplMpr4lVIqk9HEr5RSmYwmfqWUymQ08SulVCajiV8ppTIZTfxKKZXJ/A8YEwsPcnXavwAAAABJRU5ErkJggg==\n", - "text/plain": [ - "
" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "nn.fit_epoch(X_train,y_train,X_test,y_test,lr=0.01,epochs=5,bs=512,l2=0)" ] @@ -1017,31 +906,9 @@ }, { "cell_type": "code", - "execution_count": 66, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Epoch 1. Training loss: 2.713198038507928, Val loss:2.742190507090039\n", - "Epoch 2. Training loss: 2.5720206370237206, Val loss:2.992309190585378\n", - "Epoch 3. Training loss: 2.499240076003374, Val loss:3.2752241617101348\n", - "Epoch 4. Training loss: 2.5238658626905726, Val loss:3.1381103575641176\n", - "Epoch 5. Training loss: 2.5950786943316455, Val loss:2.6091254751795403\n" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd0VEUbx/HvEAKh1yBIbyIQQoghBAgJTaqi0psUQURR6UVEqoUaEOG1gCBKV0EBpbeAQCD03nuRUAKEmjLvHzeBEFM2sMnd3Tyfc/a4yU72/tywT2bnzp1RWmuEEEI4lnRmBxBCCGF9UtyFEMIBSXEXQggHJMVdCCEckBR3IYRwQFLchRDCAUlxF0IIByTFXQghHJAUdyGEcEDpk2qglHIBAoGM0e1/01oPi9OmO9ADiATCgG5a60OJPW/evHl1sWLFnjG2EEKkTTt37rymtXZNqp1KavkBpZQCsmitw5RSzsBmoKfWelusNtm11rej7zcBPtBaN0jseb28vHRwcLAF/ytCCCFiKKV2aq29kmqXZM9dG9U/LPpL5+ibjtPmdqwvs8R9XAghROpKsrgDKKWcgJ1AKWCq1joonjY9gD5ABqC2NUMKIYRIHotOqGqtI7XWHkAhwFsp5RZPm6la65LAQGBIfM+jlOqmlApWSgWHhIQ8T24hhBCJSHLM/T8/oNQw4K7WenwCj6cDbmqtcyT2PPGNuYeHh3PhwgUePHiQrEzCHC4uLhQqVAhnZ2ezowiRZlhtzF0p5QqEa61DlVKZgLrAmDhtSmutj0d/2Rg4zjO4cOEC2bJlo1ixYhjncYWt0lpz/fp1Lly4QPHixc2OI4SIw5Ix9wLArOhx93TAQq31MqXUSCBYa70E+FApVRcIB24CHZ8lzIMHD6Sw2wmlFHny5EGG14SwTZbMltkHVIrn+0Nj3e9prUBS2O2H/K6EsF1yhaoQsTyMeMjozaPps7IP1+5dMzuOEM9Minss169fx8PDAw8PD/Lnz0/BggUff/3o0SOLnqNz584cPXo00TZTp05lzpw51oiMr68ve/bsscpzpXV3H93l9Xmv88naT5i4bSK+M3xZf3o9ss+wsEcWzXNPK/LkyfO4UA4fPpysWbPSr1+/p9pordFaky5d/H8XZ86cmeRxevTo8fxhhVXdenCL1+a9xpbzWxhTdwzrTq9j5+Wd1P65Nl4vetG/Wn+alm1K+nTylhH2QXruFjhx4gRubm50794dT09PLl++TLdu3fDy8qJ8+fKMHDnycduYnnRERAQ5c+Zk0KBBVKxYkapVq3L16lUAhgwZwqRJkx63HzRoEN7e3pQpU4YtW7YAcPfuXZo1a0bFihVp06YNXl5eSfbQZ8+eTYUKFXBzc2Pw4MEARERE8Pbbbz/+/uTJkwGYOHEi5cqVo2LFirRv397qr5k9uXbvGrV/rk3QhSAWNF/AgOoDWNF+Bed6neO7xt9x68EtWv3WijJTyjB1+1Tuhd8zO7IQSbLdbkivXmDJcEN4OFy5AvnzQ1LzrT08ILqoJtehQ4eYOXMm3333HQCjR48md+7cREREUKtWLZo3b065cuWe+plbt27h7+/P6NGj6dOnDzNmzGDQoEH/eW6tNdu3b2fJkiWMHDmSFStW8M0335A/f35+//139u7di6enZ6L5Lly4wJAhQwgODiZHjhzUrVuXZcuW4erqyrVr19i/fz8AoaGhAIwdO5azZ8+SIUOGx99Liy7ducSrv7zKqZun+KP1HzQq3ejxY5mcM/Ge13t09ezKkqNLGPPPGD5c/iHDNgzjQ+8P+dD7Q/JmzmtieiESZv899ytX4NQp478pqGTJklSuXPnx1/PmzcPT0xNPT08OHz7MoUP/XQQzU6ZMNGzYEIBXXnmFM2fOxPvcTZs2/U+bzZs307p1awAqVqxI+fLlE80XFBRE7dq1yZs3L87OzrRt25bAwEBKlSrF0aNH6dmzJytXriRHDuPasvLly9O+fXvmzJmTZi9COhN6hhoza3Du1jlWtFvxVGGPzSmdE2+VfYutXbYS2CmQaoWrMWLjCIpMLEKPv3pw8sbJVE4uRNJst+duaQ/72jWYORM6d4a8KdeLypIly+P7x48f5+uvv2b79u3kzJmT9u3bx3tVbYYMGR7fd3JyIiIiIt7nzpgx43/aJPckXkLt8+TJw759+1i+fDmTJ0/m999/54cffmDlypVs3LiRP//8k88//5wDBw7g5OSUrGPasyPXjlD357rcC7/H2g5r8S7oneTPKKWoUbQGNYrW4FDIISZsmcC0XdP4bud3NCvbjP7V+lO5YOUkn0eI1GD/Pfe8eaF//xQt7HHdvn2bbNmykT17di5fvszKlSutfgxfX18WLlwIwP79++P9ZBCbj48P69ev5/r160RERDB//nz8/f0JCQlBa02LFi0YMWIEu3btIjIykgsXLlC7dm3GjRtHSEgI9+6lnXHkPVf24DfTj4ioCDZ22mhRYY+rnGs5fnzjR870OkP/av1ZeXIl3tO9qTWrFsuPL5cZNsJ0tttzt2Genp6UK1cONzc3SpQoQfXq1a1+jI8++ogOHTrg7u6Op6cnbm5uj4dU4lOoUCFGjhxJzZo10Vrz+uuv07hxY3bt2kWXLl3QWqOUYsyYMURERNC2bVvu3LlDVFQUAwcOJFu2bFb/f7BFW89vpdHcRmTLkI01HdbwUp6Xnuv5Xsz2IqPrjmZwjcFM2zmNidsm0mhuI9zyudG/Wn9au7Umg1OGpJ9ICCtL9sJh1hLfwmGHDx+mbNmypuSxNREREURERODi4sLx48epV68ex48fJ3162/p7bE+/s3Wn19FkXhMKZCvA2g5rKZKjiNWP8SjyEfMPzGfclnEcuHqAQtkL0atKL9595V2yZ8xu9eOJtMfShcPsf1jGQYWFhVG9enUqVqxIs2bN+P77722usNuTpUeX0mhOI4rnKs6mzptSpLADZHDKQIeKHdjXfR9/t/2bUrlL0W91P4pMLMKgNYO4dOdSihxXiLik5y6eiz38zuYfmM/bi9+mUv5KrGi/gtyZcqfq8Xdc3MG4LeP4/fDvOCkn3nZ/m37V+lHW1bZfN2GbpOcuBDB913Ta/t6WaoWrsabDmlQv7ACVC1ZmYYuFHPvwGO96vsu8A/Mo979yNJnXhM3nNsvJV5EipLgLhzVp2yTeXfouDUo1YHm75aaPeZfMXZKpjadyttdZhvkPY8v5LdSYWYNqM6qx6PAiIqMiTc0nHIsUd+FwtNaM2jiK3it706xsM/5o/QeZnTObHesx1yyuDK85nHO9zzGl4RT+DfuXZgubUXZqWb4P/p774ffNjigcgBR34VC01gxYPYChG4bSsWJH5jefb7NTETM7Z6aHdw+OfXSMBc0XkD1jdrr/1Z1iXxfji8AvuHH/htkRhR2T4h5LzZo1/3NB0qRJk/jggw8S/bmsWbMCcOnSJZo3b57gc8c9gRzXpEmTnrqYqFGjRlZZ92X48OGMHx/vlrcOJUpH8f5f7zN+63g+rPwhM96YYRerOKZPl56W5Vuy490drOuwDs8CngxZP4QiE4vQa0UvzoaeNTuisENJFnellItSartSaq9S6qBSakQ8bfoopQ4ppfYppdYqpYqmTNyU1aZNG+bPn//U9+bPn0+bNm0s+vkXX3yR33777ZmPH7e4//333+TMmfOZny8tCY8Mp8PiDny/83s+8f2EyQ0nk07ZV99FKUWt4rVY3m45e7vvpWnZpkzdMZWSk0vSblE79lyRdfuF5Sz51/8QqK21rgh4AA2UUj5x2uwGvLTW7sBvwFjrxkwdzZs3Z9myZTx8+BCAM2fOcOnSJXx9fQkLC6NOnTp4enpSoUIF/vzzz//8/JkzZ3BzcwPg/v37tG7dGnd3d1q1asX9+0/GUd9///3HywUPGzYMgMmTJ3Pp0iVq1apFrVq1AChWrBjXrhm7AQUEBODm5oabm9vj5YLPnDlD2bJleffddylfvjz16tV76jjx2bNnDz4+Pri7u/PWW29x8+bNx8cvV64c7u7ujxcs27hx4+PNSipVqsSdO3ee+bVNSQ8jHtLi1xbM2T+HL2t/yZd1vrT7LQDdX3Dn57d+5tTHp+hZpSdLji6h0veVePWXV1l9crXMsBFJsmQPVQ2ERX/pHH3Tcdqsj/XlNuC5FwjvtaKXRT2V8MhwroRdIX/W/Dg7Jb66oUd+DyY1SHhBsjx58uDt7c2KFSt44403mD9/Pq1atUIphYuLC4sXLyZ79uxcu3YNHx8fmjRpkmAR+fbbb8mcOTP79u1j3759Ty3Z+8UXX5A7d24iIyOpU6cO+/bt4+OPPyYgIID169eTN846OTt37mTmzJkEBQWhtaZKlSr4+/uTK1cujh8/zrx585g2bRotW7bk999/T3R99g4dOvDNN9/g7+/P0KFDGTFiBJMmTWL06NGcPn2ajBkzPh4KGj9+PFOnTqV69eqEhYXh4uKS6OtrhruP7vLWgrdYfWo13zT8hg+9PzQ7klUVzlGYCfUn8Jn/Z3wX/B1fB31Nvdn18MjvQf9q/WlZvqVdDD2J1GfR51allJNSag9wFVittQ5KpHkXYHkCz9NNKRWslAoOCQlJftp4XAm7wqnQU1wJs86Sv7GHZmIPyWitGTx4MO7u7tStW5eLFy/y77//Jvg8gYGBj4usu7s77u7ujx9buHAhnp6eVKpUiYMHDya5KNjmzZt56623yJIlC1mzZqVp06Zs2rQJgOLFi+Ph4QEkvqwwGOvLh4aG4u/vD0DHjh0JDAx8nLFdu3bMnj378ZWw1atXp0+fPkyePJnQ0FCbu0L21oNb1J9dn7Wn1/LTGz85XGGPLadLTgb5DuJMzzNMf306DyIe0G5RO0pNLsXX274m7FFY0k8i0hSL3q1a60jAQymVE1islHLTWh+I204p1R7wAvwTeJ4fgB/AuEI1sWMm1sOO7dq9a8zcPZPOlTpbZeOEN998kz59+rBr1y7u37//uMc9Z84cQkJC2LlzJ87OzhQrVizeZX5ji69Xf/r0acaPH8+OHTvIlSsXnTp1SvJ5EvsIHrNcMBhLBic1LJOQv/76i8DAQJYsWcKoUaM4ePAggwYNonHjxvz999/4+PiwZs0aXn755Wd6fmsLuRtCgzkN2P/vfhY0X0DzcvGfyHY0GdNnpItnFzpX6syyY8sYt2UcvVb2YsTGEXxQ+QM+8v6IF7K+YHZMYQOSdcZJax0KbAAaxH1MKVUX+BRoorV+aJV0FsibOS/9q/e32o44WbNmpWbNmrzzzjtPnUi9desW+fLlw9nZmfXr13P2bOIzGPz8/B5vgn3gwAH27dsHGMsFZ8mShRw5cvDvv/+yfPmTDznZsmWLd1zbz8+PP/74g3v37nH37l0WL15MjRo1kv3/liNHDnLlyvW41//LL7/g7+9PVFQU58+fp1atWowdO5bQ0FDCwsI4efIkFSpUYODAgXh5eXHkyJFkHzMlXLpzCf+f/DkUcog/W/+ZZgp7bOlUOpqUacKmzpvY8s4WaharyZebvqTopKK8t/Q9jl0/ZnZEYbIke+5KKVcgXGsdqpTKBNQFxsRpUwn4Hmigtb6aIklTUZs2bWjatOlTM2fatWvH66+/jpeXFx4eHkn2YN9//306d+6Mu7s7Hh4eeHsba4ZXrFiRSpUqUb58+f8sF9ytWzcaNmxIgQIFWL/+yWkMT09POnXq9Pg5unbtSqVKlRIdgknIrFmz6N69O/fu3aNEiRLMnDmTyMhI2rdvz61bt9Ba07t3b3LmzMlnn33G+vXrcXJyoly5co93lTLT6ZunqftLXULuhrCi3Qr8i8X7ITFNqVq4KotaLeLotaMEbA1g1t5ZTNs1jTdffpMB1QfgUyju/AeRFiS5cJhSyh2YBThh9PQXaq1HKqVGAsFa6yVKqTVABeBy9I+d01o3Sex5ZeEwx5Cav7PYuyetbL9Sdj1KwL9h//LN9m+YumMqoQ9C8S3iy4BqA2j8UmO7mx4q/svShcNkVUjxXFLrd7b78m7qza6Hk3Ji9durqfBChRQ/pr0LexTGj7t+JGBbAOdunePlvC/Tv1p/2lVoR8b0GZN+AmGTZFVI4TC2nt9KrVm1yJQ+E5s6b5LCbqGsGbLS06cnJz46wey3ZpPRKSNdlnSh+NfFGbN5DKEPnv/qZ2G7bK64y8UZ9iM1fldrT63l1V9eJV+WfGx+ZzOl85RO8WM6GmcnZ9q5t2P3e7tZ2X4l5fOVZ9DaQRSZWIR+q/px4fYFsyOKFGBTxd3FxYXr169LgbcDWmuuX7+eohc2LT26lMZzG1MiVwkCOwem2O5JaYVSinol67H67dXs6raL1156jUnbJlH86+J0/KMj+//db3ZEYUU2NeYeHh7OhQsXkpz3LWyDi4sLhQoVwtk58SuDn8W8/fN4e/HbvPLiKyxvt9yUTTbSgjOhZ5i4dSLTd0/nXvg9GpZqSP9q/alZrKbdL+HgqOzyhKoQYOye1G1pN/yK+rG0zVKyZcxmdiSHd/3edb4N/pbJQZMJuReC14te9K/Wn6Zlm8ryBjZGTqgKuzRx68THuyf93e5vKeypJE/mPAzxG8LZXmf5rvF3hD4IpdVvrSgzpQxTt0/lXvi9pJ9E2BQp7sImaK0ZsWEEfVb1oXm55ja3e1Jakck5E+95vceRHkf4veXvuGZ25cPlH1JkYhGGbxhOyF3rrAklUp4Ud2E6rTX9V/dn+MbhdPLoxLxm82x296S0wimdE03LNmVrl60EdgqkauGqjNg4gqKTitLjrx6cvHHS7IgiCVLchakioyLpvqw7E7ZO4CPvj/ixyY8yxmtDlFLUKFqDpW2WcvCDg7R2a820XdN4acpLtPy1JTsu7jA7okiAnFAVpgmPDKfTn52Yu38ug30H83ntz2WGhh24ePsik4Mm893O77j98DbVClejRpEa9KvWz2oL+ImEyQlVYdMeRDygxa8tmLt/Ll/V+Yov6nwhhd1OFMxekDGvjuF87/OMe3UcOy/tZMw/Y5ixe4bZ0UQs8vlXpLq7j+7y5oI3WXNqDVMaTqGHdw+zI4lnkD1jdvpV60d4ZDiD1w2mYLaCZkcSsUjPXaSq0Aeh1Jtdj3Wn1/HTGz9JYXcA/ar1o0yeMowMHEl4ZLjZcUQ0Ke4i1YTcDaH2rNrsuLiDhc0X0tGjo9mRhBU4Ozkzvt54jl0/xrfB35odR0ST4i5SxcXbF/H/yZ/D1w6zpM0SmpVrZnYkYUWNSzembom6DN8wnBv3b5gdRyDFXaSC0zdPU2NmDS7cvsDK9itpUOo/uzQKO6eUYkK9Cdx6eItRG0eZHUdgQXFXSrkopbYrpfYqpQ4qpUbE08ZPKbVLKRWhlEp7G1qKBB0OOYzvTF9uPbzF2g5r8SvqZ3YkkULcX3CnS6UuTNkxRfZwtQGW9NwfArW11hUBD6CBUirupozngE7AXOvGE/Zs9+Xd+P3kR2RUJBs7bZRt8dKAkbVG4pLehQGrB5gdJc1LsrhrQ1j0l87RNx2nzRmt9T4gyvoRhT3acn4LtWbVIrNzZjZ13oRbPjezI4lUkD9rfgb7DubPo3+y/vT6pH9ApBiLxtyVUk5KqT3AVWC11jooZWMJe7bm1JrHuydt6rxJdk9KY3pX7U3RHEXps6oPkVGRZsdJsywq7lrrSK21B1AI8FZKPVM3TCnVTSkVrJQKDgmR1eUc0ZKjS2g8tzElc5VkU+dNsntSGuSS3oXRdUez58oeZu2dZXacNCtZs2W01qHABuCZpjtorX/QWntprb1cXV2f5SmEDZu3fx5NFzTFI78HGzpt4IWsL5gdSZikVflW+BTy4dN1nxL2KCzpHxBWZ8lsGVelVM7o+5mAusCRlA4m7Mu0ndNot6gdNYrWYM3ba2RbvDROKcXE+hO5EnaFMZvHmB0nTbKk514AWK+U2gfswBhzX6aUGqmUagKglKqslLoAtAC+V0odTLnIwtYEbA2g27JuNCzdkL/byu5JwuBTyIc2bm0Yv3U8526dMztOmiNL/opnprVmxMYRjNg4ghblWjC76WzZZEM85WzoWV6e+jLNyjZjdtPZZsdxCLLkr0hRWmv6rerHiI0j6OzRWXZPEvEqmrMofXz6MGf/HLZf3G52nDRFirtItsioSN5b9h4B2wL42PtjpjeZjlM6J7NjCRs1yHcQL2R5gd4re2PWSEFaJMVdJEt4ZDhvL36babum8WmNT5nUYBLplPwzEgnLljEbn9f+nC3nt/DroV/NjpNmyLtSWOxBxAOa/9qceQfmMbrOaNkWT1iss0dn3F9wZ+CagTyIeGB2nDRBiruwSNijMF6b+xpLji5haqOpDPQdaHYkYUec0jkRUC+AM6Fn+Hrb12bHSROkuIskhT4Ipf7s+qw/s55Zb87ig8ofmB1J2KE6Jerw+kuv88WmL/g37F+z4zg8Ke4iUSF3Q6g1qxY7Lu7g1xa/0qFiB7MjCTs27tVx3I+4z7ANw8yO4vCkuIsEXbx9Eb+f/Dh67ShL2yyladmmZkcSdq5M3jJ84PUB03ZNY/+/+82O49CkuIt4nbp5ihoza3Dx9kVWtF9B/VL1zY4kHMRQ/6HkyJiDvqv6ytTIFCTFXfzHoZBD1JhZg1sPb7Gu4zrZPUlYVZ7MeRjqP5TVp1az/MRys+M4LCnu4im7Lu/C/yd/onQUGzttxOvFJK9yFiLZPqj8AaVzl6bvqr6ER4abHcchSXEXj/1z7h/ZPUmkigxOGRj36jiOXDvCDzt/MDuOQ5LiLgBj96R6s+uRP2t+NnfeTKncpcyOJBxckzJNqFWsFsM2DCP0QajZcRyOFHfBn0f+pPHcxpTKXYrAToEUzlHY7EgiDVBKEVA/gBv3b/B54Odmx3E4UtzTuLn759JsYTMq5a/E+o7rZfckkao88nvQ2aMzk4Mmc+LGCbPjOBQp7mnYDzt/oP2i9tQoWoPVb6+W3ZOEKT6v/TkZnDIwcI0saWFNUtzTqAlbJvDesvdoVLqR7J4kTFUgWwEG+Q5i0eFFbDyz0ew4DsOSPVRdlFLblVJ7lVIHlVIj4mmTUSm1QCl1QikVpJQqlhJhxfPTWjNs/TD6re5Hy/ItWdRqEZmcM5kdS6Rxfar2oVD2QvRZ1YcoHWV2HIdgSc/9IVBba10R8AAaKKV84rTpAtzUWpcCJgKyI64N0lrTd1VfRgaO5B2Pd5jbdK7sniRsQmbnzIyuM5pdl3fxy95fzI7jEJIs7toQFv2lc/Qt7jXDbwCzou//BtRRstC3TYmMiqTb0m5M3DaRj70/ZlqTabJ7krApbSq0ofKLlRm8bjB3H901O47ds2jMXSnlpJTaA1wFVmutg+I0KQicB9BaRwC3gDzxPE83pVSwUio4JCTk+ZILi4VHhtN+cXum757OkBpDZPckYZPSqXRMrD+RS3cuMW7LOLPj2D2L3uFa60ittQdQCPBWSsW9dDG+Xvp/VgTSWv+gtfbSWnu5uromP61Itvvh93lt3mvMPzCfoX5DGVV7lOyeJGxW9SLVaVm+JWP/GcuF2xfMjmPXktV901qHAhuABnEeugAUBlBKpQdyADeskE88h8t3LuP/kz+rTq4CIGuGrCYnEiJpo+uMJlJH8um6T82OYtcsmS3jqpTKGX0/E1AXOBKn2RKgY/T95sA6LWt5mmr35d14T/fmUMghfn7zZ8bWHUvnSp3NjiVEkornKk5vn978vPdngi8Fmx3HbqmkarBSyh3jZKkTxh+DhVrrkUqpkUCw1nqJUsoF+AWohNFjb621PpXY83p5eengYPnFpYTFhxfTfnF78mTKw5I2S/DI72F2JCGS5fbD25SaXIqX877Mxk4bZSgxFqXUTq11ksu1pk+qgdZ6H0bRjvv9obHuPwBaJDeksC6tNaM3j2bwusFUKViFP1r/Qf6s+c2OJUSyZc+YnVG1RtH9r+4sOryIZuWamR3J7siUCQfxMOIhHf/oyOB1g2lboS0bOm2Qwi7sWhfPLpR3Lc+ANQN4GPHQ7Dh2R4q7A7h69yq1f67NL/t+YVStUcx+azYu6V3MjiXEc0mfLj0B9QM4dfMU32z/xuw4dkeKu53b/+9+vKd5s/vybn5t8StD/IbI+KRwGPVK1qNR6UaMChxFyF25NiY5pLjbsWXHllFtRjXCo8IJ7BxI83LNzY4khNWNf3U8dx/dZfiG4WZHsStS3O2Q1poJWybQZF4TyuQpw/au22WvU+GwyrqWpbtXd77f+T2HQg6ZHcduSHG3M48iH/Hu0nfpt7ofzco1I7BzIAWzFzQ7lhApanjN4WTNkJV+q/qZHcVuSHG3I9fuXePVX17lx90/8pnfZyxovoDMzpnNjiVEisubOS+f+X3G8hPLWXlipdlx7IIUdztxKOQQVaZXIehCEHOazmFkrZGy+JdIUz70/pCSuUrSd1VfIqIizI5j86Q62IGVJ1ZS9ceq3H10lw2dNtC2QluzIwmR6jKmz8jYV8dyMOQg03dNNzuOzZPibsO01nwT9A2N5jaieM7ibH93Oz6F4u6TIkTa8dbLb+FX1I+h64dy68Ets+PYNCnuNio8Mpwef/fg4xUf89pLr7H5nc0UyVHE7FhCmEopRUC9AK7du8aXm740O45Nk+Jug27ev0nDOQ35NvhbBlYfyOJWi2W5XiGivfLiK3So2IFJQZM4dTPR9QnTNCnuNub49eP4/OhD4NlAZr4xk9F1R8uJUyHi+KL2F6RPl55BawaZHcVmSdWwIetOr6PK9CrcuH+DdR3X0cmjk9mRhLBJBbMXZEC1Afx66Fc2n9tsdhybJMXdRnwf/D31Z9fnxWwvsr3rdnyL+JodSQib1q9aP17M9iJ9VvYhSkeZHcfmSHE3WURUBL1W9KL7X92pV7IeW7psoXiu4mbHEsLmZcmQha/qfMWOSzuYu3+u2XFsjiXb7BVWSq1XSh1WSh1USvWMp00updRipdQ+pdT2eDbQFvG49eAWTeY14eugr+lVpRdLWi8he8bsZscSwm60d2/PKwVe4ZO1n3Av/J7ZcWyKJT33CKCv1ros4AP0UEqVi9NmMLBHa+0OdAC+tm5Mx3Pq5imqzajG6lOr+f6175nYYCJO6ZzMjiWEXUmn0hECj6AbAAAdbElEQVRQP4ALty8wYcsEs+PYlCSLu9b6stZ6V/T9O8BhIO5KVeWAtdFtjgDFlFIvWDmrwwg8G4j3NG8u37nMqvar6PZKN7MjCWG3/Ir60bRsU0b/M5pLdy6ZHcdmJGvMXSlVDGM/1aA4D+0Fmka38QaKAoWeP57jmbl7JnV/rkuezHkI6hpEreK1zI4khN0bW3csEVERDFk3xOwoNsPi4q6Uygr8DvTSWt+O8/BoIJdSag/wEbAbYzgn7nN0U0oFK6WCQ0LS1q4qkVGRDFg9gHeWvIN/MX+2ddlG6TylzY4lhEMombskH3t/zE97fmLX5V1mx7EJSmuddCOlnIFlwEqtdUASbRVwGnCP54/AY15eXjo4ODiZce3TnYd3aL+4PUuOLuEDrw+Y1GASzk7OZscSwqGEPgil9DelccvnxroO6xx2u0ml1E6tdZK781gyW0YBPwKHEyrsSqmcSqkM0V92BQITK+xpydnQs/jO9OWvY38xpeEUpjaeKoVdiBSQ0yUnI2qOYMOZDfx59E+z45guyZ67UsoX2ATsB2KuFBgMFAHQWn+nlKoK/AxEAoeALlrrm4k9b1rouW89v5U3F7zJw4iHLGyxkHol65kdSQiHFhEVgfu37oRHhXPwg4NkcMqQ9A/ZGUt77umTaqC13gwk+vlGa70VkAHkWObsm0OXJV0olL0QGzpuoKxrWbMjCeHw0qdLz4R6E2g0txFTt0+ld9XeZkcyjVyhamVROooh64bQfnF7fAr5ENQ1SAq7EKmoQakG1CtZj5GBI7l+77rZcUwjxd2K7j66S4tfW/DFpi/oWqkrq95eRZ7MecyOJUSaopRiQr0J3H54mxEbR5gdxzRS3K3k4u2L+P3kx+LDiwmoF8APr//gkON9QtgDt3xuvOv5Lv/b8T+OXDtidhxTSHG3guBLwVSeVplj14+xtM1Selft7bDTsISwFyNrjSRLhiz0X93f7CimkOL+nBYeXEiNmTXImD4jW7tspfFLjc2OJIQA8mXJx6c1PmXZsWWsObXG7DipTor7M9JaM3LjSFr91opXCrxCUNcg3PLJYphC2JKPq3xMsZzF6LOyD5FRkWbHSVVS3J/B/fD7tF3UlmEbhtGhYgfWdlhLviz5zI4lhIjDJb0LY+uOZf/V/czYPcPsOKlKinsyXb5zmZqzarLgwAJG1xnNT2/8RMb0Gc2OJYRIQPNyzaleuDpD1g/hzsM7ZsdJNVLck2H35d14T/fmwNUDLGq1iIG+A+XEqRA2TilFQP0Art69ylebvzI7TqqR4m6hP478ge9MY1/Tf975hzdfftPkREIIS3kX9KZdhXYEbA3gbOhZs+OkCinuSdBaM2bzGJouaIpbPje2d92OR34Ps2MJIZLpqzpfoZRi0NpBZkdJFVLcE/Ew4iGd/uzEoLWDaOXWig0dN1AgWwGzYwkhnkHhHIXpV7Uf8w/MZ+v5rWbHSXFS3BNw9e5V6vxch5/3/szImiOZ23QumZwzmR1LCPEcBvoOpEDWAvRe2RtL9rKwZ1Lc43Hg6gGqTK/Czss7WdB8AZ/5fyYnToVwAFkzZOWL2l8QdDGI+Qfmmx0nRUlxj+OvY39R9ceqPIx4SGCnQFqWb2l2JCGEFXWo2AGP/B4MWjuI++H3zY6TYqS4R9NaE7A1gNfnvc5LeV5ix7s7qFywstmxhBBW5pTOiYB6AZy7dY6J2yaaHSfFSHEHHkU+otvSbvRd1Ze3yr5FYKdACmYvaHYsIUQKqVW8Fm+UeYOvNn/FlbArZsdJEZbsoVpYKbVeKXVYKXVQKdUznjY5lFJLlVJ7o9t0Tpm41nf93nXq/VKP6bun82mNT/m1xa9kyZDF7FhCiBQ27tVxPIh4wGfrPjM7SoqwpOceAfTVWpcFfIAeSqlycdr0AA5prSsCNYEJsTbMtllHrh2hyvQqbLuwjdlvzebz2p+TTsmHGSHSgtJ5SvNh5Q/5cfeP7L2y1+w4VpdkJdNaX9Za74q+fwc4DMQds9BANmVMKckK3MD4o2CzVp1chc90H+48usP6jutp597O7EhCiFQ21H8ouTLlou+qvg43NTJZ3VSlVDGgEhAU56EpQFngErAf6Km1jorn57sppYKVUsEhISHPFNgapmyfQqM5jSiSowjbu26nauGqpmURQpgnV6ZcDPcfztrTa1l2bJnZcazK4uKulMoK/A700lrfjvNwfWAP8CLgAUxRSmWP+xxa6x+01l5aay9XV9fniP1swiPD6fFXDz5a/hGNSjfin3f+oWjOoqmeQwhhO7p7dadMnjL0W92P8Mhws+NYjUXFXSnljFHY52itF8XTpDOwSBtOAKeBl60X8/ndvH+TRnMb8b/g/9G/Wn8Wt1pMtozZzI4lhDCZs5Mz4+uN59j1Y3wb/K3ZcazGktkyCvgROKy1Dkig2TmgTnT7F4AywClrhXxex68fp+qPVdl4ZiMzmsxg7KtjcUrnZHYsIYSNaFy6MXWK12H4huHcuH/D7DhWYUnPvTrwNlBbKbUn+tZIKdVdKdU9us0ooJpSaj+wFhiotb6WQpmTZf3p9VSZXoVr966xtsNaOleym1maQohUErPme+iDUEZtHGV2HKtIn1QDrfVmINGFVbTWl4B61gplLdN2TuODvz/gpTwvsbTNUkrkKmF2JCGEjXJ/wZ0ulbowZccU3q/8Pi/lecnsSM/FISd1R0ZF0ntFb7ot60bdEnXZ8s4WKexCiCSNqj0Kl/QuDFg9wOwoz83hivvth7d5fd7rTAqaRM8qPVnaZik5XHKYHUsIYQfyZ83PJ76f8OfRP1l/er3ZcZ6LQxX30zdPU/XHqqw+tZrvGn/HpAaTSJ8uyZEnIYR4rLdPb4rkKEKfVX2IjIo0O84zc5jivvncZryne3P5zmVWtl/Je17vmR1JCGGHMjlnYkzdMey5sodZe2eZHeeZOURx/2nPT9SeVZvcmXKzres2ahevbXYkIYQda1W+FT6FfPh03aeEPQozO84zseviHhkVycDVA+n8Z2f8ivqxrcs2uz/DLYQwn1KKifUnciXsCmM2jzE7zjOx2+Ie9iiMpgubMnbLWLq/0p3l7ZaTK1Mus2MJIRyETyEfWru1ZvzW8Zy/dd7sOMlml8X93K1z+M7wZdmxZXzT8Bv+1/h/ODs5mx1LCOFgRtcZjdaaT9Z+YnaUZLO74r7q5CrKTy3PyZsn+bvt33zo/aFsXi2ESBFFcxalT9U+zNk/h+0Xt5sdJ1nsrrgvO7aMsPAw3nvlPeqXqm92HCGEg/vE9xPyZclHn5V97GrNd7sr7kP9hzKm7hgG+Q4yO4oQIg3IljEbn9f6nH/O/8Ovh341O47FlFl/iby8vHRwcLApxxZCiOSIjIrE8wdPbj+8zeEeh3FJ72JaFqXUTq21V1Lt7K7nLoQQqc0pnRMT6k3gTOgZvt72tdlxLCLFXQghLFC3RF1ee+k1vtj0BVfvXjU7TpKkuAshhIXGvzqe+xH3Gbp+qNlRkmR/xf3MGWjVCi5dMjuJECKNKZO3DO97vc+0XdM4cPWA2XESZck2e4WVUuuVUoeVUgeVUj3jadM/1i5NB5RSkUqp3CmSuH9/WLgQypWDSZPg7t0UOYwQQsRnmP8wsmfMTt9VfW16aqQlPfcIoK/WuizgA/RQSpWL3UBrPU5r7aG19gA+ATZqrVNmI8L//Q+6dDGKe+/eULQojBwJNxxj30MhhG3LkzkPQ/2GsurkKpafWG52nAQlWdy11pe11rui798BDgMFE/mRNsA868SLh6srTJ8OW7bAP/9A1aowbJhR5Pv1k+EaIUSK6+Hdg1K5S9F3VV/CI8PNjhOvZI25K6WKAZWAoAQezww0AH5/3mAWqVYNli6FffugSROYOBGKF4du3eDEiVSJIIRIezI4ZWDcq+M4cu0IP+z8wew48bK4uCulsmIU7V5a69sJNHsd+CehIRmlVDelVLBSKjgkJCT5aRNSoQLMmQPHj8M778DPP0OZMtC6NezZY73jCCFEtDfKvEHNYjUZtmEYoQ9CzY7zHxYVd6WUM0Zhn6O1XpRI09YkMiSjtf5Ba+2ltfZydXVNXlJLlCgB334Lp08bQzR//w2VKkGjRrBpk/WPJ4RIs5RSBNQL4Mb9G3we+LnZcf7DktkyCvgROKy1DkikXQ7AH/jTevGeUYECMGYMnD0Ln38OO3aAnx/4+sJff4ENn+EWQtiPSgUq0cmjE5ODJnPyxkmz4zzFkp57deBtoHas6Y6NlFLdlVLdY7V7C1iltbaduYm5csGnnxpFfvJkOH8eXnsNPDxg3jyIiDA7oRDCzn1e+3MyOGVgwJoBZkd5iiWzZTZrrZXW2j1muqPW+m+t9Xda6+9itftJa906ZeM+o8yZ4aOPjJOsP/0Ejx5B27bGuPz338ODB2YnFELYqRezvcjA6gNZdHgRgWcDzY7zmP1dofo8nJ2hY0c4eBAWLYI8eaB7d2OGzbhxcOeO2QmFEHaob7W+FMpeiD4r+xClo8yOA6S14h4jXTp46y0ICoI1a6B8eRgwAIoUgc8+A2vO5BFCOLzMzpn5qs5X7Ly8k1/2/mJ2HCCtFvcYSkGdOkaB374datUyTsAWLQo9e8K5c2YnFELYibYV2lL5xcoMXjeYu4/MP/WYtot7bJUrG0M1hw5By5bGMgclS0LnznDkiNnphBA2Lp1KR0D9AC7ducS4LePMjiPF/T/KljVOup44Ae+/DwsWGOvYNGsGsnOUECIRvkV8aVGuBWP/GcvF2xdNzSLFPSFFixrTJ8+cgcGDYe1ao3f/6quwbp3MlRdCxGtM3TFE6kgGrxtsag4p7knJl88Yhz93zrgwav9+Y5zexwf++AOibOPMuBDCNhTPVZxeVXrx896fCb5k3qd9Ke6Wyp7dmFFz5oyxxEFIiDHjpkIFYy2bcNtcGU4IkfoG1xiMa2ZX+qzsY9qa71Lck8vFxZgbf+yYsViZk5Mxd750aZgyBe7dMzuhEMJkOVxyMLLWSDad28Siw4ktx5VypLg/q/Tpjatc9+41lh0uWNC4CrZYMfjySwi1vVXihBCpp6tnV8q7lmfAmgE8jHiY6seX4v68lDLWq9m8GTZuhFdeMdazKVoUBg2CK1fMTiiEMEH6dOmZUG8Cp26e4pvt36T68aW4W4tSxsqTy5fDrl3QoAGMHWv05D/4wFiGWAiRptQvVZ+GpRoyKnAUIXdT98p3Ke4poVIlY3780aPw9tvGtoClS0P79nDAtndMF0JY1/h647n76C7DNwxP1eNKcU9JpUvDtGlGr71nT2PqZIUKxpaAW7eanU4IkQrKuZbjvVfe4/ud33Mo5FCqHVeKe2ooWBAmTDDWlR8+3NjYu1o1qFkTVq6UC6KEcHDDaw4na4as9FvVL9WOKcU9NeXJA8OGGUU+IMBY4qBBA+Mk7K+/QmSk2QmFECnANYsrQ/yGsPzEclaeWJkqx5TiboasWaF3bzh50hiPDwszFisrVw5+/NHYTEQI4VA+8v6IErlK0HdVXyKiUn4XOEv2UC2slFqvlDqslDqolOqZQLua0VvwHVRKbbR+VAeUMSN06QKHD8PChZAlC3Ttamz0PXGiUfSFEA4hY/qMjK07loMhB2n5a0uu3buWosezpOceAfTVWpcFfIAeSqlysRsopXIC/wOaaK3LAy2sntSROTlBixawcyesWAGlSkGfPsZc+REj4MYNsxMKIaygadmmVHyhIouPLGbm7pkpeixL9lC9rLXeFX3/DnAYKBinWVtgkdb6XHS7q9YOmiYoBfXrw4YNsGULVK9unIAtUgT69oWL5i4hKoR4Pkop1nRYw9i6Y+lcqXOKHitZY+5KqWJAJSAozkMvAbmUUhuUUjuVUh0S+PluSqlgpVRwiGxll7iqVWHJEti3D958E77+2tjr9d134fhxs9MJIZ5FRAR5D56mf743yZs5b4oeyuLirpTKCvwO9NJa347zcHrgFaAxUB/4TCn1Utzn0Fr/oLX20lp7ubq6PkfsNKRCBZg921iorGtX+OUXKFMGWrWC3bvNTieESMy9e7B+PYwcaewFkTMneHsb59qumT/mjlLKGaOwz9Fax7fE2QVghdb6rtb6GhAIVLReTEGJEsbWf2fOGEsPL18Onp7QsCEEBspceSFswY0bxkKCAwYYn75z5oTatY3h1atXoVMnaNcONm2CmSk75q6SWmtYKaWAWcANrXWvBNqUBaZg9NozANuB1lrrBK+19/Ly0sGybd2zCw01iv2kScba8tWqwSefQOPGxti9ECLlXbhgFOqYW8zyIs7Oxs5tNWoYt2rVIFcu47Fr14zC3rkz5E3+0IxSaqfW2ivJdhYUd19gE7AfiNl2aDBQBEBr/V10u/5A5+g207XWkxJ7XinuVnLvHsyYAePGGbtFVahgrEbZsqWxLLEQwjq0NoZHYxfzmAUBs2Y1CnhMMff2hkyZUiSG1Yp7SpHibmXh4TBvHowebcybL1EC+vc3Pga6uJidTgj7ExFh7NcQU8g3bzaGVgBcXcHX11gJtkYNqFgx1TpTUtzTqqgoY5bNV1/B9u3wwgvGDJv792HgQOMfpRDiv+7fN94zMcV8y5YnFxIWK/akV16jhjGpwaThTynuaZ3Wxln6r76CNWuM72XJYqxl4+9v9DgqVIB0sgKFSKNCQ41F/GKKeXDwk6U/3NyeLuaFCpmbNRYp7uKJ7duNMXmlICjIGJsH40x+jRpGoff3N9ahl3F64aguXXp6vHz/fqMTlD49eHk9KeTVq0Pu3GanTZAUd5Gws2eN6ZMbNxr/jbkoKuakUEzPvnJlY/0bIeyN1saqq7GL+cmTxmNZshjTFGOKeZUqkDmzuXmTQYq7sNzly0aRj7nFTOdycQEfnyc9ex8fu3oTiDQkMtK4mjt2Mf/3X+OxvHmNk58xxdzDw5iqaKekuItnd+3akw2/AwNhzx7jRG3M3F0/P+NWvTpkz252WpEWPXgAO3Y8ffLzdvSF80WLPj1e/vLLDnXthxR3YT23bhknnmJ69jt2GNPE0qUzxuljeva+vsaGJEJY261bRgGPKeY7dsDDh8Zj5cs/XcwLFzY3awqT4i5Szt27sG3bk579tm1P3mgVKjzp2fv5Qf785mYV9unKlaeHWPbtMz49pk9v7FwW++RnGutQSHEXqefhQ2NGTkzP/p9/jD8AAC+99KRn7+dnLF8sRGxaw6lTTwp5YKBxMhSMczxxT35myWJuXpNJcRfmCQ83VqyMmZGzaZPxsRqMi0FievX+/lCypEONhwoLREYaJ+1j98wvXzYey5376ZOfnp52ffIzJUhxF7YjMtKYUxx7Rk7Mev4FCjzp1fv5GfvISrF3LA8fGhcIxRTyf/558se+cOGnx8vLlpUL65IgxV3YLq3hyJEnPfuNG40LTMCYthb7wip3d2MbQmE/7tx5+uTn9u3G7BYwinfsYl60qLlZ7ZAUd2E/YsZcY3r1Gzc+WW0vR44nCzT5+Rkn0+Rjuu2IiDBOfgYFPSnmMVNnnZyMYZWYQu7r+0xL3IqnSXEX9u38+aeHcY4cMb6fOfPTV9F6e8uql9b06JExZBb7dvXqf78Xc7t588nPurg8ffLTx8e46llYlRR34Vj+/ffJTIqNG5+sC5IxozGDImYYp2rVND+b4in37ydcmOMr2rfj7qAZzcnJ6HW7uj59y5fP6KkvXgxffmlsGCNSlBR34dhu3DBOzMXMtd+1yzhxGzMPOqZnX726sUCaI9DamGKaULGOr2jHTEmNy9n5v4U6btGO/XXOnAmf6HzOnYVE8lhzJ6bCwM9Afoxdln7QWn8dp01N4E8geqCURVrrkYk9rxR3YVUxJ/FievbbtxtTMpUy1hKJ6dnXqGE7BUhrI3dSQx+xb/fvx/9cGTP+tyAnVrSzZ5dZSXbKmsW9AFBAa71LKZUN2Am8qbU+FKtNTaCf1vo1SwNKcRcp6v594yRfTM9+69YnhbFcuaenX774onWOqbWxRril49UhIU/WD48rc+aEe9Hx3bJmlWKdRlha3JNcvFtrfRm4HH3/jlLqMFAQOJToDwphpkyZoGZN4wZGEQ0OftKznz0bvv3WeKxUqaevoi1WzPh+VJQx/GPpePW1a8bskfhky/akEBcqZKzJk1jhltU3xXNK1pi7UqoYEAi4aa1vx/p+TeB34AJwCaMXfzCx55KeuzBVzP6YMT37wMAnMz9eeMHYXu3ePaM3Hp+cOS0fAsmbV2b0CKux+glVpVRWYCPwhdZ6UZzHsgNRWuswpVQj4Gutdel4nqMb0A2gSJEir5w9e9aiYwuR4qKi4OBBo8jPmGGcoK1TB5o0+W/RzpsXMmQwO7FIo6xa3JVSzsAyYKXWOsCC9mcAL631tYTaSM9d2CyZ/SFsmNXG3JVSCvgROJxQYVdK5Qf+1VprpZQ3kA64nszMQtiGvHmhf3+zUwjxXCzZDbk68DawXym1J/p7g4EiAFrr74DmwPtKqQjgPtBamzWBXgghhEWzZTYDic6x0lpPAaZYK5QQQojnI2trCiGEA5LiLoQQDkiKuxBCOCAp7kII4YCkuAshhAMybclfpVQI8KyXqOYFErxAykS2mgtsN5vkSh7JlTyOmKuo1to1qUamFffnoZQKtuQKrdRmq7nAdrNJruSRXMmTlnPJsIwQQjggKe5CCOGA7LW4/2B2gATYai6w3WySK3kkV/Kk2Vx2OeYuhBAicfbacxdCCJEImy7uSqkGSqmjSqkTSqlB8TyeUSm1IPrxoOidomwhVyelVIhSak/0rWsq5ZqhlLqqlDqQwONKKTU5Ovc+pZSnjeSqqZS6Fev1GpoKmQorpdYrpQ4rpQ4qpXrG0ybVXy8Lc6X66xV9XBel1Hal1N7obCPiaZPq70kLc5n1nnRSSu1WSi2L57GUfa201jZ5A5yAk0AJIAOwFygXp80HwHfR91sDC2wkVydgigmvmR/gCRxI4PFGwHKMVT59gCAbyVUTWJbKr1UBwDP6fjbgWDy/x1R/vSzMleqvV/RxFZA1+r4zEAT4xGljxnvSklxmvSf7AHPj+32l9Gtlyz13b+CE1vqU1voRMB94I06bN4BZ0fd/A+pEby5idi5TaK0DgRuJNHkD+FkbtgE5lVIFbCBXqtNaX9Za74q+fweI2fg9tlR/vSzMZYro1yEs+kvn6Fvck3ap/p60MFeqU0oVAhoD0xNokqKvlS0X94LA+VhfX+C//8gft9FaRwC3gDw2kAugWfRH+d+UUoVTOJOlLM1uhqrRH6uXK6XKp+aBoz8OV8Lo8cVm6uuVSC4w6fWKHmbYA1wFVmutE3zNUvE9aUkuSP335CRgABCVwOMp+lrZcnGP7y9Y3L/GlrSxNkuOuRQoprV2B9bw5K+z2cx4vSyxC+OS6orAN8AfqXVgZWz8/jvQS2t9O+7D8fxIqrxeSeQy7fXSWkdqrT2AQoC3UsotThNTXjMLcqXqe1Ip9RpwVWu9M7Fm8XzPaq+VLRf3C0Dsv66FgEsJtVFKpQdykPIf/5PMpbW+rrV+GP3lNOCVFM5kKUte01Sntb4d87Faa/034KyUSvGdqZWx8fvvwByt9aJ4mpjyeiWVy6zXK06GUGAD0CDOQ2a8J5PMZcJ7sjrQRCl1BmPotrZSanacNin6Wtlycd8BlFZKFVdKZcA44bAkTpslQMfo+82BdTr67ISZueKMyzbBGDe1BUuADtGzQHyAW1rry2aHUkrljxlrVKm0wXr08RLd+B0TXi9LcpnxekUfy1UplTP6fiagLnAkTrNUf09akiu135Na60+01oW01sUwasQ6rXX7OM1S9LWyZINsU2itI5RSHwIrMWaozNBaH1RKjQSCtdZLMN4EvyilTmD8xWttI7k+Vko1ASKic3VK6VwASql5GDMp8iqlLgDDME4uoY2NzP/GmAFyArgHdLaRXGZssG7Jxu9mvF62vCF9AWCWUsoJ4w/KQq31MrPfkxbmMuU9GVdqvlZyhaoQQjggWx6WEUII8YykuAshhAOS4i6EEA5IirsQQjggKe5CCOGApLgLIYQDkuIuhBAOSIq7EEI4oP8Dd2KXhO1Tu4AAAAAASUVORK5CYII=\n", - "text/plain": [ - "
" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "nn.fit_epoch(X_train,y_train,X_test,y_test,lr=0.01,epochs=5,bs=512,l2=0)" ] @@ -1103,7 +970,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Deeper Neural Net" + "# Deeper Neural Net (2 hidden layers)" ] }, { @@ -1274,46 +1141,9 @@ }, { "cell_type": "code", - "execution_count": 60, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Epoch 1. Training loss: 1.3097806084877954, Val loss:1.3291423040506751\n", - "Epoch 2. Training loss: 1.3023353101308597, Val loss:1.388434451500173\n", - "Epoch 3. Training loss: 1.2952853606324042, Val loss:1.3787619869211034\n", - "Epoch 4. Training loss: 1.284319419502578, Val loss:1.3119930882260993\n", - "Epoch 5. Training loss: 1.2760227007072154, Val loss:1.308110890061248\n", - "Epoch 6. Training loss: 1.2713430875181182, Val loss:1.2972796858589568\n", - "Epoch 7. Training loss: 1.258513988011788, Val loss:1.286891446536391\n", - "Epoch 8. Training loss: 1.261942550865047, Val loss:1.3149182906880768\n", - "Epoch 9. Training loss: 1.2528386308585762, Val loss:1.2792056243931689\n", - "Epoch 10. Training loss: 1.2550406141664268, Val loss:1.3111686767129291\n", - "Epoch 11. Training loss: 1.2491439091543683, Val loss:1.280495698054411\n", - "Epoch 12. Training loss: 1.235385449546147, Val loss:1.2622582621445406\n", - "Epoch 13. Training loss: 1.2417737529380717, Val loss:1.2679078438689884\n", - "Epoch 14. Training loss: 1.2355673889524585, Val loss:1.2499621798661336\n", - "Epoch 15. Training loss: 1.2284222621214052, Val loss:1.2480887023022964\n", - "Epoch 16. Training loss: 1.220532216583229, Val loss:1.1952922376018427\n", - "Epoch 17. Training loss: 1.2321207479437544, Val loss:1.2174530693827375\n", - "Epoch 18. Training loss: 1.2265321446461552, Val loss:1.1963013709143089\n", - "Epoch 19. Training loss: 1.213665369649875, Val loss:1.212512521275759\n", - "Epoch 20. Training loss: 1.193507316304565, Val loss:1.167832833844976\n" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XlYlNUXwPHvFVEEFNRByRWXLBVBCNGE3HPJtVyQNHdcs8Uts3LPn5qVqeUSiqaG5pZoLqmh5L4iamqWK+6445IC9/fHi6SJgArODJzP88wDM3Pnfc/gcLzc995zldYaIYQQmUs2cwcghBAi/UlyF0KITEiSuxBCZEKS3IUQIhOS5C6EEJmQJHchhMiEJLkLIUQmJMldCCEyIUnuQgiRCWU314lNJpN2c3Mz1+mFEMIq7dq1K0Zr7ZJaO7Mldzc3N3bu3Gmu0wshhFVSSp1ISzsZlhFCiExIkrsQQmRCktyFECITMtuYuxDi+bp37x7R0dHcuXPH3KGINLCzs6NIkSLY2to+1esluQuRRURHR5M7d27c3NxQSpk7HJECrTWXLl0iOjqaEiVKPNUxZFhGiCzizp075M+fXxK7FVBKkT9//mf6K0uSuxBZiCR26/Gs/1bWl9x//x2qV4eTJ80diRBCWCzrS+4hIRARAaNGmTsSIcQTuHTpEhUrVqRixYq4urpSuHDhpPt3795N0zE6duzI4cOHU2zz7bffMnfu3PQIGX9/fyIjI9PlWM+b9V1Q/fhjI8Hny2fuSIQQTyB//vxJiXLo0KE4OjrSr1+/h9pordFaky1b8v3OkJCQVM/Tq1evZw82E7C6nvupAna06pyHC/u2mjsUIUQ6+Ouvv3B3d6d79+54e3tz9uxZunbtio+PD+XLl2f48OFJbe/3pOPi4nB2dmbgwIF4enry6quvcuHCBQA+/fRTxo8fn9R+4MCB+Pr68tJLL7F582YAbt68SfPmzfH09CQwMBAfH59Ue+hz5syhQoUKuLu7M2jQIADi4uJ45513kh6fMGECAF9//TXlypXD09OTtm3bpvvPLC2srufeZ3UfFha9zq7c61kb8xclTKXNHZIQ1ueDDyAtww337sG5c+DqCqnNt65YERKT6pP6448/CAkJYcqUKQCMHj2afPnyERcXR82aNWnRogXlypV76DXXrl2jevXqjB49mj59+jBjxgwGDhz4yLG11mzfvp2wsDCGDx/OqlWrmDhxIq6urixatIi9e/fi7e2dYnzR0dF8+umn7Ny5EycnJ+rUqcPy5ctxcXEhJiaGffv2AXD16lUAxo4dy4kTJ8iRI0fSY8+b1fXcv2v4HS0dfDmfS1NhigfTdk1Da23usITInM6dg6NHja8ZqFSpUlSqVCnpfmhoKN7e3nh7e3Pw4EH++OOPR16TK1cuGjRoAMArr7zC8ePHkz32W2+99UibjRs30rp1awA8PT0pX758ivFt27aNWrVqYTKZsLW15e233yYiIoLSpUtz+PBh3n//fVavXo2TkxMA5cuXp23btsydO/epFyE9K6vrubs4uPBT83mc8CpJ5wHF6La8G/P2z6Nq0ap8UOUDTPYmc4cohOVLaw87Jsa4xtWxI5gy7nfLwcEh6fsjR47wzTffsH37dpydnWnbtm2y871z5MiR9L2NjQ1xcXHJHjtnzpyPtHnSDuHj2ufPn5+oqChWrlzJhAkTWLRoEdOmTWP16tVs2LCBpUuXMnLkSPbv34+Njc0TnfNZpdpzV0oVVUqFK6UOKqUOKKXeT6ZNG6VUVOJts1LKM2PCTeTmRnH7F/j1gDffvvEtG09u5PPfP6fb8m4k6IQMPbUQWYrJBP37Z2hi/6/r16+TO3du8uTJw9mzZ1m9enW6n8Pf35+ffvoJgH379iX7l8GDqlSpQnh4OJcuXSIuLo558+ZRvXp1Ll68iNaali1bMmzYMHbv3k18fDzR0dHUqlWLL774gosXL3Lr1q10fw+pSUvPPQ7oq7XerZTKDexSSq3RWj/40zgGVNdaX1FKNQCmAZUzIF6DUuDvT7aNm+g590d8C/vSemFrFh9cTO0fahPcOJhS+Upl2OmFEBnH29ubcuXK4e7uTsmSJfHz80v3c/Tu3Zt27drh4eGBt7c37u7uSUMqySlSpAjDhw+nRo0aaK1p3LgxDRs2ZPfu3XTu3BmtNUopxowZQ1xcHG+//TY3btwgISGBjz76iNy5c6f7e0iNetI/T5RSS4FJWus1j3k+L7Bfa104peP4+PjoZ9qs45tvjItCJ09C0aJorQneHUy/Nf24G3eXOiXrENwkmIKOBZ/+HEJkIgcPHqRs2bLmDsMixMXFERcXh52dHUeOHKFu3bocOXKE7Nkta6Q6uX8zpdQurbVPaq99oguqSik3wAvYlkKzzsDKJznuU/H3N75u2gQYS3WDXgniQM8DlMhbguVHluMb7MvhmJQXPAghsp7Y2Fj8/Pzw9PSkefPmTJ061eIS+7NKc3JXSjkCi4APtNbXH9OmJkZy/+gxz3dVSu1USu28ePHi08T7L09PcHBISu73FclThA0dNtC6fGuu3bmG5xRPvtj0BfEJ8c92PiFEpuHs7MyuXbvYu3cvUVFR1K1b19whpbs0JXellC1GYp+rtV78mDYeQDDQVGt9Kbk2WutpWmsfrbWPi0uq+7umLHt2qFIFNm585CkXBxdCW4Ry6N1DNHixAQPWDsD3e1/6/9qfmFsxz3ZeIYSwAmmZLaOA6cBBrfVXj2lTDFgMvKO1/jN9Q0yBvz9ERcH1ZP+QwNXRlcWtFhPaPJRDMYcYt2UcncM6y4waIUSml5aeux/wDlBLKRWZeHtDKdVdKdU9sc1gID/wXeLzz3Cl9An4+UFCAmx9fCkCpRSt3VuzLWgbL5teJuxwGPXm1CP6evRzCVEIIcwh1SsIWuuNQIqFhbXWXYAu6RVUmlWpAtmyGUMzqYyZuRdw54+ef/D97u/5cPWHlPu2HI3KNGJCgwmy8EkIkelYXfmBh+TObVxY/c9F1cdRStH1la7s7b6XvLnyEro/lAZzG3Dz7s0MDlQIUaNGjUcWJI0fP56ePXum+DpHR0cAzpw5Q4sWLR577NSmVo8fP/6hxURvvPFGutR9GTp0KOPGjXvm46Q3607uYIy7b91qFDhKo9L5SrOtyzZqlajFzjM78fneh6jzURkYpBAiMDCQefPmPfTYvHnzCAwMTNPrCxUqxMKFC5/6/P9N7itWrMDZ2fmpj2fpMkdyv3UrbRXuHuDq6Mq6dutY124dV+9cxfd7XybvmCxFyITIIC1atGD58uX8888/ABw/fpwzZ87g7+9PbGwstWvXxtvbmwoVKrB06dJHXn/8+HHc3d0BuH37Nq1bt8bDw4OAgABu376d1K5Hjx5J5YKHDBkCwIQJEzhz5gw1a9akZs2aALi5uRETY8ye++qrr3B3d8fd3T2pXPDx48cpW7YsQUFBlC9fnrp16z50nuRERkZSpUoVPDw8ePPNN7ly5UrS+cuVK4eHh0dSwbINGzYkbVbi5eXFjRs3nvpnmxzrn7V/f2nyoEEQGvrENTBqlajF3u57CVwUSM8VPVl7bC2z35yNva19BgQrhGX4YNUHRJ5LvUN0L/4e52LP4eroiq1NytUNK7pWZHz9xxcky58/P76+vqxatYqmTZsyb948AgICUEphZ2fHkiVLyJMnDzExMVSpUoUmTZo8dh/RyZMnY29vT1RUFFFRUQ+V7P3888/Jly8f8fHx1K5dm6ioKN577z2++uorwsPDMf0nR+zatYuQkBC2bduG1prKlStTvXp18ubNy5EjRwgNDeX777+nVatWLFq0KMX67O3atWPixIlUr16dwYMHM2zYMMaPH8/o0aM5duwYOXPmTBoKGjduHN9++y1+fn7ExsZiZ2eX4s/3SVl/z71wYShRAtauhZo14dSpJz5EAYcC1CtVD4DFBxdTdXpVjl45mt6RCmF1zsWe4+jVo5yLTZ+Svw8OzTw4JKO1ZtCgQXh4eFCnTh1Onz7N+fPnH3uciIiIpCTr4eGBh4dH0nM//fQT3t7eeHl5ceDAgVSLgm3cuJE333wTBwcHHB0deeutt/j9998BKFGiBBUrVgRSLisMRn35q1evUr16dQDat29PREREUoxt2rRhzpw5SSth/fz86NOnDxMmTODq1avpvkLW+nvuYGya3aULrF8PZcvC0KHw/vupby7wgE5enVAoiuYpSo8VPfCZ5kNo81Dqla6XYWELYS4p9bAfFHMrhpA9IXT06pgus8qaNWtGnz592L17N7dv307qcc+dO5eLFy+ya9cubG1tcXNzS7bM74OS69UfO3aMcePGsWPHDvLmzUuHDh1SPU5KQ7H3ywWDUTI4tWGZx/nll1+IiIggLCyMESNGcODAAQYOHEjDhg1ZsWIFVapUYe3atbz88stPdfzkWH/PHYze+8qVcPAg1K5tlCgtWhSSGbd7HJO9if5+/WldoTU7g3ZSJE8R6s+tT8O5Dbl48xlLJQhhpe7/XqTXdGFHR0dq1KhBp06dHrqQeu3aNQoUKICtrS3h4eGcOHEixeNUq1YtaRPs/fv3ExVlTIi4fv06Dg4OODk5cf78eVau/LfMVe7cuZMd165WrRo///wzt27d4ubNmyxZsoTXXnvtid+bk5MTefPmTer1z549m+rVq5OQkMCpU6eoWbMmY8eO5erVq8TGxvL3339ToUIFPvroI3x8fDh06NATnzMlmSO53+fmZiT09u3h/Hlo1gzatIEzZ57oMKXylWJL5y14FPRgxV8raPRjI+7Fp302jhDi8QIDA9m7d2/ShUWANm3asHPnTnx8fJg7d26qPdgePXoQGxuLh4cHY8eOxdfXFzB2VfLy8qJ8+fJ06tTpoXLBXbt2pUGDBkkXVO/z9vamQ4cO+Pr6UrlyZbp06YKXl9dTvbdZs2bRv39/PDw8iIyMZPDgwcTHx9O2bVsqVKiAl5cXH374Ic7OzowfPx53d3c8PT0f2lUqvTxxyd/08swlf1MSEwPTpsGVKzBxojE8M2wY9O79REM1F25eIHBRIL8d+43XS77OgpYLcLJ7fM1nISyZlPy1Ps+t5K/VMJmM2TNffAEHDkC1atC3rzFU88svaT5MAYcCrGu3jhlNZhB+PBy/GX6cuJryn4tCCGEJMmdyf1CpUrB8+b9DNY0aQVAQXEq2cGWyOnp1ZFWbVZy8dhL379zZcHxDBgYshBDPLvMndzC25Rs3DkaMgF69jA1/y5SBgABIY1352iVr0/WVrsTei6X+3PpsjX58sTIhLJUs0rMez/pvlTWSOxhDNZ9+CpMmwZ49Rl2an34yFkGlMHf1QQP9BzLQfyCuDq7U+aEOa4+uzdiYhUhHdnZ2XLp0SRK8FdBac+nSpWda2JQ5L6imxYUL0K0brFkDWsOoUfDuu2Bjk+pLz944S905dfnz0p/MbzGfZi83ew4BC/Fs7t27R3R0dKrzvoVlsLOzo0iRItj+ZxJIWi+oZt3kft/Jk9C9uzFPvlgxmD/fKCWcisu3L/PG3DfYcXoHAe4BUjpYCPFcZO3ZMk+iWDFjBk3r1kaif+01+PZbozefgny58rG23VrcnN0I3R/Kh6s+fE4BCyFE6iS5g3HBdeJEY0y+WjVjeKZ+fTh9OsWXOeZwZF37dRR3Kk7o/lCWHkr7ilghhMhIktzvM5mM2TRr18LkycbuTuXKwTvvGIuiHsPN2Y2oHlH4FPKh5YKWrDiy4jkGLYQQyZPk/l9KGWPwkZHg5ARz5kDjxpDCRag8OfOwqu0qKhSswJvz3iQoLIiYW4//D0EIITKaJPfHefFF2LbNGKbZuhVefRX++uuxzZ3tnFnzzhry2+cneE8wQ9cPfX6xCiHEf0hyT8kLL8CGDRAWBidOQMWK0LbtY4dp8uXKx7p268ifKz9zouZw4MKB5xywEEIYJLmnRePGxsKn/Plh7lxo0QLi45NtWtalLDuCdmBva0/dOXWlFo0QwiwkuadV8eKwZYsxB37DBiPhP2bn9BJ5S7C67Wpu3btF3Tl1pR68EOK5k+T+JAoVMhL85MnGytYqVeDw4WSbVihYgeWByzl57SSvz36dkREj5SKrEOK5keT+NLp3h3XrjMqSlSoZVSaTGYf3K+bHgpYLiDofxWfhnzF993QzBCuEyIokuT+tatVg505wcIDgYGPhUzIalWnEmDpjADh25djzjFAIkYVJcn8WxYvDpk1Gzfj582HMmGTLFvT3688HlT9g6u6pTN4x2QyBCiGymlSTu1KqqFIqXCl1UCl1QCn1fjJtlFJqglLqL6VUlFLKO2PCtUAlSxq7PQUGwsCB8N57yc6kGVd3HI3KNKL3yt6s/mu1GQIVQmQlaem5xwF9tdZlgSpAL6VUuf+0aQC8mHjrCmSt7mnOnMZK1r59jXrxXl6PbMptk82G0OahuBdwp8WCFvRd3VcusAohMkyqyV1rfVZrvTvx+xvAQaDwf5o1BX7Qhq2As1LqhXSP1pJly2bs9tSoEezbB3XqPFKywDGHI8sCl6FQfLX1KyZtn2SmYIUQmd0TjbkrpdwAL2Dbf54qDJx64H40j/4HkDWEhMCbb8LBg9C0Kdy+/dDTRZ2KsrDVQmyUDWuPruVe/D0zBSqEyMzSnNyVUo7AIuADrfX1/z6dzEseubKolOqqlNqplNp5MY17l1odkwkWL4bp04258A0bws2bDzWpW6ouM5rOYNOpTfT9ta+ZAhVCZGZpSu5KKVuMxD5Xa704mSbRQNEH7hcBzvy3kdZ6mtbaR2vt4+Li8jTxWo9OneCHH4zVrLVrw8iRD82Fb+fZjj5V+jBx+0SZ/y6ESHfZU2uglFLAdOCg1vqrxzQLA95VSs0DKgPXtNZn0y9MK9W2LeTIYezytG2bMS4/aFDS02NeH8P+i/vp8UsPyrqUpWrRqmYMVgiRmaSl5+4HvAPUUkpFJt7eUEp1V0p1T2yzAjgK/AV8D/TMmHCtUKtWRrkCgPBwuHs36ans2bIzr/k8iuQpQr3Z9dh7bq+ZghRCZDap9ty11htJfkz9wTYa6JVeQWU63boZm4B06wZt2kBoKGQ3fvR5c+XlrbJv8eWWL2n4Y0OO9D5CLttcZg5YCGHtZIXq89K1K3z5JSxcCF26QEJC0lMD/QfSwbMDp2+cJmhZEDqVzbmFECI1qfbcRTrq0wdu3IChQ41dnZYsARcXTPYmQpqFUDpfaT4N/5SKrhXpV7WfuaMVQlgx6bk/b4MHw2uvGTVp2rd/6KlBrw2iZbmWfLT2I1b9tcpMAQohMgNJ7s+bUsbQTMWKsHIlzJr1wFOKkKYhuBdwp9WCVgxYM0BKFAghnookd3MoUMCYGlm7NnTuDKv+7aU75HBgaeulxOt4vtj8BZN3Zq0yPUKI9CHJ3Vxy5DBWslaoYOzJumNH0lNuzm7MfXMuCsXmk5tJ0AkpHEgIIR4lyd2c8uQxhmby5YOaNR9K8M3KNmN8/fGs+nsVIzaMMGOQQghrJMnd3FxdjRWsN28adWguX056qrdvb9p7tmfohqEsPbTUjEEKIayNJHdLMGAA9OgBV68aFSX/+QcwLrBOaTSFSoUq0XZJW/64+IeZAxVCWAtJ7pbAZILvvoOZMyEiwljklLiQyS67HYsDFmOX3Y7qIdX5+/Lf5o1VCGEVJLlbkrffNqpHzpljLHRKVCRPEVqWa0nM7RgahTaSC6xCiFRJcrc0gwYZ5YKHD4eAgKQywcNrDqfZy804FHOIIeFDzBykEMLSSXK3NErBlClQqhT89BN89hkAJnsTi1stplPFToz8fSSLDyZXVl8IIQyS3C2Rra2xsMnFBebNM+rQYFxg/bbht/gW9qX9z+3lAqsQ4rEkuVuq0qVh82Zjg4+GDeHKFSDxAmurxTjYOtDox0YM3zBcShQIIR4hyd2SlS5tVI48dsxYxXrP2Ey7cJ7CLGy1kBPXTjBk/RDZpk8I8QhJ7pauWjUIDobffgN/f0jcWNy/mD9j64wF4NT1U+aMUAhhgSS5W4N27aBWLdi+3ZhJk6hv1b708OnBtzu+ZU7UHDMGKISwNJLcrUVoqFFkbPlyWLYs6eFv6n9D9eLV6RLWhR2nd6RwACFEViLJ3VoUKABbt8Irr0BgIOw1NtO2tbFlQcsFuDq60mx+M87eOGvmQIUQlkCSuzWxt4ewMHB2hsaN4dw5AFwcXFjaeilXbl+hcnBlTl2TMXghsjpJ7tamUCEjwV+6ZEyRHDUKYmLwdPWkZbmWnLp+iiahTWSTbSGyOEnu1sjbG+bOhd274ZNPYMYMAL6s9yX1S9Un8nwkIyKkBrwQWZkkd2vVrJmx2TbA+fOAUaJgRZsVtPNsx5D1Q5i/f74ZAxRCmFN2cwcgnsHQoXD6NHz1FXh6Qrt2KKWY1mgaf1/+mw5LO1Aibwl8C/uaO1IhxHMmPXdrppRRB75WLaMGfEQEADmz52RJwBIK2Beg9qza7Dm7x8yBCiGeN0nu1i5HDli4EEqWNHZxSiwy5uLgQsvyLYm9F0v9ufW5eueqmQMVQjxPktwzg7x5jcVNWsOrr8LhwwAM9B9IV++uXL51mbfmv8Xd+LtmDlQI8bykmtyVUjOUUheUUvsf87yTUmqZUmqvUuqAUqpj+ocpUlW6tLG4KSYG6tWD27cx2ZuY2ngqM5rOIPx4OJ3DOssUSSGyiLT03GcC9VN4vhfwh9baE6gBfKmUyvHsoYknNmwYtG0LJ04Y9WgSjO343vF8hxE1RzAnag6DwwebOUghxPOQanLXWkcAl1NqAuRWSinAMbFtXPqEJ56IyQSzZ8O4ccY4/IABSU998tondPbqzMjfR9Lyp5ZSA16ITC49pkJOAsKAM0BuIEBr2cHZrPr0gePH4csvwc0N3n0XpRSTG07m9xO/s/DgQhxyODCz2UwzByqEyCjpcUG1HhAJFAIqApOUUnmSa6iU6qqU2qmU2nkxsS65yABKwfjxxth7794QEgIYRcZWv7OaonmKMm//PH4/8buZAxVCZJT0SO4dgcXa8BdwDHg5uYZa62laax+ttY+Li0s6nFo8lo0NvPaa8X1QEKxfD4Cbsxu7u+3GzdmNJvOasO/8PvPFKITIMOmR3E8CtQGUUgWBl4Cj6XBc8ay6dTNWsZYsCU2bQmQkYJQpWN12Nfa29tSfW58TV0+YN04hRLpTqU2NU0qFYsyCMQHngSGALYDWeopSqhDGjJoXAAWM1lqnui2Qj4+P3rlz57PELtLq1CmoWhXi4mDTJiPZA/vO78Nvhh922e3Y0GEDZV3KmjlQIURqlFK7tNY+qbYz17xnSe7P2cGDxh6sTk7GdMn33gOTiZ6/9GTyzskUyl2IAz0P4GznbO5IhRApSGtylxWqWUXZsvDLLxAdDSNGGDVpgOE1h9PZqzMXYi/Q8MeG3Lx708yBCiHSgyT3rKRKFZgzx7jYGhYGsbGY7E0ENwkmtEUoW6O38ub8N/kn7h9zRyqEeEaS3LOaVq1gwQLj4mqTJnD7NgAtyrUguHEwa46uIXBRIHEJsg5NCGsmyT0revNNmDXLmB7ZsiXcNQqKdfTqyPh641lyaAm+3/tyPva8eeMUQjw1Se5ZVZs2MGWKMQ5fqVLSZtvvV3mf+qXqs+fcHhqHNiZBFhsLYZUkuWdlXbtCo0YQFQX16xtTJYHZb82mdona7Dizg94rekslSSGskCT3rC4kBN54A/buhfbtIS4Ok72JNe+soX/V/ny38zv6/tpXErwQVkb2UM3qTCZjaGb0aPj4Y6MuzaxZKBsbxtQZw524O3y99Wsiz0Yyv+V8XBykbIQQ1kB67sIwcCCMGgVz50KHDhAfj1KKb+p/Q+XClQk/EU7rRa2lBy+ElZDkLv718ccwcqQxF97HB86eRSlFWGAYvoV8+e3YbwxdP9TcUQoh0kCSu3jYJ59AgwbGPPjXX4e7dyngUIAtXbbQqWInhkcMZ9j6YeaOUgiRCknu4lE//GAscDpwwJgTf/s22VQ2vm/yPR0qdmDohqEM3zDc3FEKIVIgyV08ymSCpUth6lRYudKYLhkbSzaVjeDGwQSUD2DI+iEMWDMg9WMJIcxCkrt4vK5djZWs4eFQvjz89Rc22WzwcvUC4IvNXzA4fLBcZBXCAslUSJGyd96BX381LrK+9hrs3k1n784opdh3fh8jIkYQlxDH57U+x9gjXQhhCSS5i9R9/TXY2cGPP4K/P6Y1axjgN4AEnYBDDgf+t/F/xCXEMabOGEnwQlgIGZYRqTOZ4Pvv4bff4OpV8PODffvIprIxueFkelXqxRebv6BaSDUu3pSNz4WwBJLcRdpVrgwREaA1+PrC8uUopZjYYCL+Rf3ZeGqjFBsTwkJIchdPpnx56NgR7twxpkkuWYJSisUBi6lZvCbbTm+jc1hn4hPizR2pEFmaJHfx5Pr2hSFDwNMTmjeH777DxcGFde3XMazGMGZGzqTtkrbci79n7kiFyLIkuYsnZzLB0KHGEE3jxtCrFwwahAIGVx/M2Dpjmbd/Hl5TvTh9/bS5oxUiS5LkLp6evT0sWmTMh//f/4x6NGfO0N+vP01fasqBiweoNasWt+7dMnekQmQ5ktzFs8me3djRqV492L3bmAt/5QrBTYJpWa4lRy4fof6c+lz/57q5IxUiS5HkLp6dUsYip9at4dQpqFoV07nr/NTyJ+a1mMeW6C3U/qE2l25dMnekQmQZktxF+jCZIDQU1qyB8+ehShXYto1W5Vvxc8DP7Du/D78Zfnz222fE3Ioxd7RCZHqS3EX6ql4dtmwxxuP9/SE4mIZlGrKyzUqOXjnKyN9HMm7zOHNHKUSmJ8ldpL+XXkraj5WgIBgxgppuNVgWuIxc2XMxM3Im+y/sN3eUQmRqqSZ3pdQMpdQFpdRjfxuVUjWUUpFKqQNKqQ3pG6KwSr17G9v2tWwJgwdD27bUK1KdHUE7sMlmQ7WQamyN3mruKIXItNLSc58J1H/ck0opZ+A7oInWujzQMn1CE1bNZDK27Zs/30jyiUXHyk+az8ZmYeTLlY9as2oRFBYkY/BCZIBUq0JqrSOUUm4pNHkbWKy1PpnY/kL6hCYyBaWMJP/SSxAQALt2UeL6dTaO3Ij02d/XAAAawUlEQVTXVC+C9wRz+95t5jSfY+5IhchU0mPMvQyQVym1Xim1SynVLh2OKTKbt96C1avByQmmTsV1RQQbO26khHMJ5u6fy8RtE80doRCZSnok9+zAK0BDoB7wmVKqTHINlVJdlVI7lVI7L16U0rBZTq1acPgweHtDQAClvp7Jge77aPZyM95b9R6frPtEdnUSIp2kx2Yd0UCM1vomcFMpFQF4An/+t6HWehowDcDHx0d+i7OiggWNuvA9e8LIkeRavJgFYT/Ty74AozaO4vjV43gU9KCzd2dM9iZzRyuE1UqP5L4UmKSUyg7kACoDX6fDcUVmlTMnBAdDTAyEhZG9qj9TIiJwdXRleMRwftz/I/cS7vFptU/NHakQVistUyFDgS3AS0qpaKVUZ6VUd6VUdwCt9UFgFRAFbAeCtdYyiVmkTCmYPt0oOnbvHqpyZYbd8mXc6+NQKBb8sYCzN86aO0ohrJYy1xinj4+P3rlzp1nOLSzM8ePGxh9798LAgazIfY5WCfPJ72BixdsrKF+gvLkjFMJiKKV2aa19UmsnK1SF+bm5waZNEBgI//sfbwwKISK+HXfj7/Lq9FfptqybzIUX4glJcheWwd7eqCw5YgQohfcPa9j62g/kss3FtN3T6La8m7kjFMKqSHIXlkMp+PRTWL8eYmMpXrMZWwp+wkv5X2LxwcW8v/J94hLizB2lEFZBkruwPNWqGRt/eHlRst37HJhv4kP3ICZsn0CjHxtx9c5Vc0cohMWT5C4sU6FCxnx4Pz9sNm7iqw9XEVx1DOuOrqPMxDJsObXF3BEKYdEkuQvLlSMH/PwztG0Lly/TOWA0XfK/zsVbF6k5qyaLDy42d4RCWCxJ7sKymUwwezZERkLx4owYsJJPjhejnHMZmv/UnI/Xfkx8Qry5oxTC4khyF9ahdGnYsgVThcqMnHmSLeOu0O3FQEZvGk2tH2oxJHyITJcU4gGS3IX1sLOD5cuhbVtyXr7GlO7LmVEgiE0nNzE8Yjif/faZuSMUwmJIchfW5f4wzb594OFBx57fs+aoPybsmbprKkPXD5XpkkIgyV1Yq+LFjfnwQ4ZQ84cIjo26xTu3X2TYhmHU/qE20dejzR2hEGYlyV1Yr+zZYehQWLYMR8d8zBrzJ7Nu12fn6Z2UmViGabummTtCIcxGkruwfg0bwokT0KMH7casotfeHNyOu0235d0IWBggF1pFliTJXWQOjo7w3XewejUDNsQxeg18cqwYSw4uwf07d5YdXmbuCIV4riS5i8ylbl1MmyP56G4lRs46yY4wVwriQJN5TfCa4sUfF/4wd4RCPBeS3EXmU6oUbN8O69bhedmWHQOPUueWK5HnI/GZ9grBu4NJ0AnmjlKIDCXJXWRetWrBvn3k+LAfoZPO0W8TVLzuQNCyIPym+9Hv134yHi8yLdmJSWQNv/0GHTqQEH2KkIAy9C53gtsJ//BasdcICwzD2c7Z3BEKkSayE5MQD6pVC44fJ9vUaXRefZFdk+7iey03v5/8nTITyxC8O1hq1IhMRZK7yDqyZYOgIDh8mLLFfdj29Q12zbanTLwzQcuCKDa+GIv/kEqTInOQ5C6yHhcXWLEC3n0X7zwv8fvHR3j7dH7O3DhD8wXNqT+nPpHnIs0dpRDPRMbcRdaWkAA//EDM4H5MK36JOI/yjC96miv/XKV52eaUzleaflX7YbI3mTtSIYC0j7lLchcCjI25Bw8GGxuu5snBmPdfYVy2rcQlxFGpUCXmt5hPibwlzB2lEHJBVYgn0qMHjB0LW7fiXLsh/xu6kT3znPG7ZWLvub2UmVSGoLAgjl05Zu5IhUgT6bkLkZyICHj7bTh9mtMlTYzpU5mpl9dwL+EeLcq1YHD1wbgXcDd3lCILkp67EM+iWjXYswfatqWwcmLCu7/Q/1BeNJqfD/1MhckVaBLahM2nNps7UiGSJcldiMdxcTE2Bjl0CKZP54N1txj7K+wPzcvQ/M3ZdGoTfjP8cBvvxszImTJPXlgUGZYRIq2io+HDD40e/d9/E1uqKG3ecSCMQwCUcC7B+5Xfp5NXJ3LnzG3mYEVmlW7DMkqpGUqpC0qp/am0q6SUildKtXiSQIWwGkWKwIIF8OefEBaGY+ESTB97iNFrYEZUSV6wceKD1R/g8oULXcO68uelP80dscjCUu25K6WqAbHAD1rrZK8gKaVsgDXAHWCG1nphaieWnrvIFNauhY8/hqgouHuX3h0LMqn4ebKpbCToBOqVqse7vu/SoHQDbLLZmDtakQmkW89dax0BXE6lWW9gEXAhbeEJkUnUqQM7dsCZM/DFFwxZ/Q9jf4Wo4BwMu+tH5OldNA5tjNt4Nz6P+JyzN86aO2KRRTzzBVWlVGHgTWDKs4cjhJXKnx/69cO05zD9y3amfIVaDB6zlQ+WGSWF7W7f5dPwTyk2vhiNf2xMUFgQF25KX0hknOzpcIzxwEda63ilVIoNlVJdga4AxYoVS4dTC2FhChSA4GDj+zNn6NK1LTa/htMx8gJXSrgyLaA03x1bx6242yw8uJAPq3xIx4odKepU1LxxP6G4hDiyZ0uP9CEySppmyyil3IDlyY25K6WOAfezugm4BXTVWv+c0jFlzF1kCTExMH06vPCCMa1y7VpO57XhoxZORFdwY8Pl3QCUyVeGPq/2oZ1nO3LZ5jJz0CmbGTmTTks7Ua9UPXpX7k29UvXkesJzlK61ZVJK7v9pNzOxnVxQFSI5Bw9Cp06wdStky8axoJYEeUWz7twmAPLkzEOrcq1oX7E9fkX9SO2vYXNwG+/GiWsncLR1JPZeLMWcihFYPhCHnA708OkhRdYyWFqTe6p/VymlQoEagEkpFQ0MAWwBtNYyzi7EkyhbFpYtg/Hj4dw5SoTMZt7cOGa8UZgXG7Th5zxnCN0fSvCeYPLa5aWdZzu6eHexmFIHUeejOHHtBI1ebMTUxlPZcmoLU3dNZczmMQDY2djR36+/maMUIIuYhDCv6Gho1Qq2bDHuu7gQ26IJXcr/zfyY9SgUGs3Lppcp7lScIdWH8GrRV80Wbq9fejF9z3RO9zlNfvv8SY8HLgpkycElHOl9xOquH1gbqS0jhDUoUgTCwmDkSJgxA2rWxHFWKJMGrGfselsORDdl0ovvc/vebVb/vZqqM6pS/rvyDA4fTOS5SJ5n5yz2biyzo2YT4B7wUGIH6OLVhX/i/5FaOxZEeu5CWJpbt6BbN5gzB2xt4d49YoqZGF89J47VarM6+0kiTkaQoBNwyulE05ea8lbZt6hdsjaOORwzLKxpu6bRbXk3Nnfa/MhfD/EJ8RQfXxyvF7xYFrgsw2IQslmHENYtJgZCQowhm+3bYfhw2J9YAaRYMS4ENKJX6T9ZeHYtOW1y8k/8P9hms6VIniI0LtOY10u9zqtFXn2kh/20tNZ4T/MmQScQ2S0y2Qu9H635iK+2fsWZPmdwcXBJl/OKR0lyFyIziYmBKVOMxVLLl8OvvxKTI46Q1xxpUzmIw42qMHT/t0ScjEgqfQBgsjdRqVAlPAt68rLpZV42vUxZl7LkyZnniU6/LXobVaZXYXLDyXT36Z5sm/0X9lNhcgUm1J9A78q9n/kti+RJchciM7t0CYKCYMkS475SxNSqQkiJawTU/ZDj5V7gf3smservVRSwL8CVO1e4l3Av6eVFchfBw9WDks4luXT7Eh/7f0yFghUee7qOSzuy8I+FnOlzJsWKlxWnVCRn9pxs67It3d6qeJgkdyEyu/tDN9Wrwy+/wMSJcOXKv0+XeoEQj3g6lgnAyb8Ox140MergNGbtnUVF14ok6AQOXDhAvDbq0JfKW4rqxavj5erF6djT9KnSBxcHFy7fvkzhrwrTwbMDkxtNTjGkLzd/Sb81/TjU6xAvmV7K0LefVUlyFyKriYmBCROgfHk4dgzmzYO9ex9u4laAEN8cdGz8KaZWHTn7zyXGbBpD/lz52XV2FxEnIrhyx/gPwimnE43KNOJ23G0WH1xMeLtwapSokWIIZ2+cpcjXRRjkP4gRtUZk1DvN0iS5C5HV3e/Zt2wJJ0/Crl0wa9a/Cd/JCerWhVy5YNgwcHMjQSfw+4nfGb91PBrN1uitnL95HoCxdcamaYFSvTn1+PPSnxx976hFrrC1dpLchRCPiomB778HNzf49Vejd3/nDtjYQK1a8MYb0LAhvPgiYMyS2XFmByuPrKSXb680lRaYvXc27X5ux+8df8e/mH8Gv6GsR5K7ECJ1Z84YvXYbGwgPN/aLBWP/2NatITAQKleGbGlf7xh7N5aC4wrytvvbTGs87al771prhoQP4dzNc4yqPUpq1iSS5C6EeHJHj0KfPrB0qZHQExKMRF+iBLz3Hrz1ljGMk4qAhQH8dOAncufITfkC5SlrKoubkxt34u/Q59U+aUrUQ8KHMDxiOACfvPYJI2uNfDTcK0epO7suXq5eTG40OUv8ByDlB4QQT65kSaMe/dixxl6xc+ca5Yq3b4e2bcFkMoZtAgKMCpePUaGAMa2yfIHy2GW3Y8WRFQzZMIT/bfwfdX6ow47TO1IMY1bkLIZHDKdB6QZkz5adrdFbHym1cCfuDi0XtOTvK3+z8OBChqwf8uzvPzPRWpvl9sorr2ghhBW4eFHrUaO0nj9f6x49tM6TR2swbpUqaT10qNY7dmidkPDvS25e1GM3jtUXb15Memz3md262oxq2nGUo2YouvL3lXW7xe30uRvnHjrdb0d/07bDbXXtWbX13bi7esLWCZqh6OBdwQ+167m8p2YoetK2STrfmHy64BcF9dkbZzP2Z2EBgJ06DTlWkrsQ4slcuKD1++9rPXCg1lWqaK2UkUoKFNB6/Hit79xJ8eXX7lzTX2/5WjuPdtYMRTuPdtZfbPpCX751Wf9x4Q/t9D8nXe7bcvrK7Staa63jE+J1zZk1de5RufXxK8e11lrP2zdPMxTdb3U/rbXWe8/t1faf22v/Gf76btzdjH3/ZpbW5C5j7kKIZ3PxIvTqBQsWGPddXaF3b+jeHfLle+zLzsWeY8CaAfx9+W82R2/GzsYOm2w22NvasyNoB8Wdiye1PX71OBUmV8C3sC/fvfEdPt/74FHQg/Xt12NrYwtA6L5Q3l78Nu/5vsc3Db7J0LdsTnJBVQjx/MTEGCWLS5c2plquWmXMwKlbF9q3h0aNwMHhsS/fe24vQcuC2HFmB719ezOhwYRH2gTvDiZoWRDOds7YKBv2dNvzSO34D1d9yPht4wkoH8CkNyZlygusktyFEObTpw98/TXkzg03boC9vZHoAbp2hWrVHkn2MbdiCNkTQkevjskmZa01r89+nXXH1jGv+TwC3AMeaXMv/h4vT3qZo1ePUrtEbVa3XZ3p9neV2TJCCPMZNMiYcXPkCKxfD+3awZo18PPPxkKpPHmgTBkjySfOujHZm+jv1/+xvW2lFDXdagJw8trJZNvY2tgS3iEcn0I+rDu2jtdnv8652HOPtLsbf/e5bnRiDtJzF0I8HxcvGnvHvvwy/P03zJ9vLJqytYWPP4a+fY2kn4LUevcPmhk5k56/9CRPzjxMazyNuIQ4Np3cxMZTG9l1Zheujq4sarWIykUqp+e7zHAyLCOEsGwxMUbv/vBhY6vBvHnhlVeM5H/6NPz1l3HLmxd69jSGc1yebBOQ/Rf28+b8N/nr8l8A5LTJiW9hX7TWbDy1kRw2OZjZdCaBFQIz4h1mCEnuQgjrsWuXUergyBHjQuyLLxq3y5dh0yajTfbs8PrrxgpZd3eIjYXz540SCnfvQpcuxgKr//wHMDJiJJ+Ff0avSr34su6X5Myek5hbMXy95WvWHl3L9jPb6eLVhW8afIO9rb0Z3vyTkeQuhLAuMTEwaRL06AEFC/77WEiIUd8mLMzYjermTeM5e3uj3d27Rk//vjJloHBhY9ZOqVIpDuXEJcQxJHwIozaOwimnE56unhTKXQhTLhOOORzJmT0n7/q+a1GzbiS5CyEyn7NnjZr13boZlS3h30qXFSsa5YxDQozSCfb28OWXRo8+e/Z/j3H3rlERc+1aY6y/YEGCwoII3hNMMadi5LQxevb369qntdTx8yLJXQiRNcXEwMiRRj2cLVuMzUsGDzZq2q9bBxERcOuW0dbODurXJ6aGLyFFY+hYbyCmxM29e/3Si8k7J7MjaAevFHrFjG/oYZLchRBZm9aweLEx5/5k4tTJsmWhdm2oVAlWrjTG9zdvNnauAihSBJo2hUaNiPYuTcmp5eju0z3ZRVUcOmQs3Dp5Evr1My4GP4fNSdKa3KW2jBAic/v8c6P2zSefPL7NwIFGm3LltLa3N77PlUu37+Cs7Yfl0DHnjv3bdvdurVu0+Lemzv2bq6vWjRppHRio9fnzGfZ2SGNtmeypZn8hhLBmXbsac+k7dnx8m759jTo4HTuCo6Ox8Gr4cPqt2MKsnvBdYGk+s61t9NIPHTLm4w8cCG3awPTpRlnkvXth+XK4ds3Y+GT6dGjQ4Ln05pOT6rCMUmoG0Ai4oLV2T+b5NsBHiXdjgR5a673/bfdfMiwjhLBoifVyGuZYwI4rBzgxyZZcl69DvXowbx4br+/n3RXv4unqyZd1vzRm1Fy4AB98YAz1nDgBr70Go0dD1arpFla6jbkrpaphJO0fHpPcqwIHtdZXlFINgKFa61SXfElyF0JYg/XH11NzVk2mVP+CbtsToFMnll/eSssFLfkn7h80mqYvNeXn1j//+6K7d7kVPJm+vw3kp1J3+M2mE56fB6dLLz7dastorSOAyyk8v1lrfSXx7lagSJqjFEIIC1e9eHV8Cvnw5b5pxPfryw+nV9BsXjPcC7izq+suyrmUY+nhpfRd3ZcEnQDA5vM7qRj/LVMq3OGyPXSJmWHsZHX79nOLO70Lh3UGVqbzMYUQwmyUUgyoOoAjl4/QamEr2v/cnhpuNfit3W94veBFVPco3q30Ll9t/YqAhQH0/7U//jP8uRt/l8X1Z1ItWwl2FoJ9a3+EmjXh3KOFzDIk7tSGZQCUUm7A8uSGZR5oUxP4DvDXWl96TJuuQFeAYsWKvXLixImnCFkIIZ6v+IR4Sk0oxYlrJ2hcpjELWi4gZ/acSc9rrfl669f0/bUvAO082zGpwSRy58zN5duXKTWhFH62pVj+2UFwcoJWreDTT409aZ/Qcy35q5TyAIKBpo9L7ABa62laax+ttY/LExYAEkIIc7HJZkP90vUB8Cvq91BiB6N33+fVPgR5BwHg7uJO7py5AciXKx8f+3/MLzd2sWHpN8YCqm++MVbSZqBnngqplCoGLAbe0Vr/+ewhCSGE5RlZaySl8paio9fjp1SOqj2KF/O9+Eib3r69mbh9Ih8dD2bL1q2opUtTnpqZDtIyWyYUqAGYgPPAEMAWQGs9RSkVDDQH7o+xxKXlTwaZLSOEyEpC9oTQKawTC1supHm55k99HCk/IIQQFiQ+IR7PKZ7cjb/LgZ4Hkjb2flKyzZ4QQlgQm2w2jK4zmiOXjxCwMICYWzEZej5J7kII8Zw0fLEhFQtWZMmhJYTssfALqkIIIdJGKcWadmuSNg/JSJLchRDiOTLZm57L5h8yLCOEEJmQJHchhMiEJLkLIUQmJMldCCEyIUnuQgiRCUlyF0KITEiSuxBCZEJmqy2jlLrIv8XGnpQJyNi1u+nP2mKWeDOWxJuxMnO8xbXWqdZMN1tyfxZKqZ1pKZxjSawtZok3Y0m8GUvilWEZIYTIlCS5CyFEJmStyX2auQN4CtYWs8SbsSTejJXl47XKMXchhBAps9aeuxBCiBRYXXJXStVXSh1WSv2llBpo7nj+Syk1Qyl1QSm1/4HH8iml1iiljiR+zWvOGB+klCqqlApXSh1USh1QSr2f+LhFxqyUslNKbVdK7U2Md1ji4yWUUtsS452vlMph7lgfpJSyUUrtUUotT7xvsfEqpY4rpfYppSKVUjsTH7PIz8N9SilnpdRCpdShxM/yq5Yas1LqpcSf7f3bdaXUB+kdr1Uld6WUDfAt0AAoBwQqpcqZN6pHzATq/+exgcA6rfWLwLrE+5YiDuirtS4LVAF6Jf5MLTXmf4BaWmtPoCJQXylVBRgDfJ0Y7xWgsxljTM77wMEH7lt6vDW11hUfmJ5nqZ+H+74BVmmtXwY8MX7WFhmz1vpw4s+2IvAKcAtYQnrHq7W2mhvwKrD6gfsfAx+bO65k4nQD9j9w/zDwQuL3LwCHzR1jCrEvBV63hpgBe2A3UBljAUj25D4n5r4BRRJ/WWsBywFl4fEeB0z/ecxiPw9AHuAYidcQrSHmB2KsC2zKiHitqucOFAZOPXA/OvExS1dQa30WIPFrATPHkyyllBvgBWzDgmNOHOKIBC4Aa4C/gata67jEJpb2uRgPDAASEu/nx7Lj1cCvSqldSqmuiY9Z7OcBKAlcBEISh76ClVIOWHbM97UGQhO/T9d4rS25q2Qek+k+6UAp5QgsAj7QWl83dzwp0VrHa+NP2iKAL1A2uWbPN6rkKaUaARe01rsefDiZphYRbyI/rbU3xvBnL6VUNXMHlIrsgDcwWWvtBdzEQoZgUpJ4naUJsCAjjm9tyT0aKPrA/SLAGTPF8iTOK6VeAEj8esHM8TxEKWWLkdjnaq0XJz5s0TEDaK2vAusxrhU4K6Xu7wlsSZ8LP6CJUuo4MA9jaGY8lhsvWusziV8vYIwF+2LZn4doIFprvS3x/kKMZG/JMYPxn+durfX5xPvpGq+1JfcdwIuJMw1yYPxJE2bmmNIiDGif+H17jHFti6CUUsB04KDW+qsHnrLImJVSLkop58TvcwF1MC6ehQMtEptZTLxa64+11kW01m4Yn9fftNZtsNB4lVIOSqnc97/HGBPej4V+HgC01ueAU0qplxIfqg38gQXHnCiQf4dkIL3jNfcFhae4APEG8CfGOOsn5o4nmfhCgbPAPYweRWeMMdZ1wJHEr/nMHecD8fpjDAlEAZGJtzcsNWbAA9iTGO9+YHDi4yWB7cBfGH/m5jR3rMnEXgNYbsnxJsa1N/F24P7vmKV+Hh6IuyKwM/Fz8TOQ15JjxpgMcAlweuCxdI1XVqgKIUQmZG3DMkIIIdJAkrsQQmRCktyFECITkuQuhBCZkCR3IYTIhCS5CyFEJiTJXQghMiFJ7kIIkQn9Hy5a3J3/iaRmAAAAAElFTkSuQmCC\n", - "text/plain": [ - "
" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "nn.fit_epoch(X_train,y_train,X_test,y_test,lr=0.001,epochs=20,bs=512,l2=0)" ] @@ -1368,25 +1198,25 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Deeper and bigger NN -> Error" + "# Deeper and bigger NN -> Exploding Gradient" ] }, { "cell_type": "code", - "execution_count": 81, + "execution_count": 39, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "[784, 100, 100, 10]\n" + "[784, 30, 20, 10, 10]\n" ] } ], "source": [ "act_obj = ReLU()\n", - "layers = [X_train.shape[1],100,100,10]\n", + "layers = [X_train.shape[1],30,20,10,10]\n", "print(layers)\n", "\n", "nn = CustomNeuralNetwork(layers,act_obj)" @@ -1394,7 +1224,7 @@ }, { "cell_type": "code", - "execution_count": 82, + "execution_count": 40, "metadata": {}, "outputs": [ { @@ -1405,9 +1235,7 @@ " return np.where(x >= 0, x, 0)\n", "C:\\Users\\qtran\\basic_model_scratch\\model\\activation_classes.py:30: RuntimeWarning: invalid value encountered in greater_equal\n", " return np.where(x >= 0, 1, 0)\n", - "C:\\Users\\qtran\\basic_model_scratch\\model\\neural_network.py:87: RuntimeWarning: invalid value encountered in multiply\n", - " grad_wrt_input = grad_wrt_input * self.act_objs[i].grad(self.X_inputs[i+1]) # actual grad_wrt_input\n", - "C:\\Users\\qtran\\basic_model_scratch\\model\\neural_network.py:92: RuntimeWarning: invalid value encountered in multiply\n", + "C:\\Users\\qtran\\basic_model_scratch\\model\\neural_network.py:97: RuntimeWarning: invalid value encountered in multiply\n", " grad_w+= (l2/bs) * self.weights[i][0] # l2 reg\n" ] }, @@ -1415,14 +1243,14 @@ "name": "stdout", "output_type": "stream", "text": [ - "Epoch 1. Training loss: 3.1526923071050046, Val loss:2.3027694560379373\n", - "Epoch 2. Training loss: 2.302759264788691, Val loss:2.3027303694485006\n", - "Epoch 3. Training loss: 2.302724240305615, Val loss:2.302699413238316\n" + "Epoch 1. Training loss: 2.9692335484626002, Val loss:2.3036101377610736\n", + "Epoch 2. Training loss: 2.303244003517184, Val loss:2.3028717283500204\n", + "Epoch 3. Training loss: 2.3027978699055693, Val loss:2.30266012746749\n" ] }, { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl4VOXZx/HvTVgishMsCmpAq7IYIKYUi8oir1V83QBlERdEKUotiFoookWwFlARoSjSKpcKghTcXqrYWlFcwYDIIqUgi1JAAsomuASe949nSGPMMiEzc2b5fa4rl5OZ58z8cnK8OXnOOfcx5xwiIpJcKgUdQEREIk/FXUQkCam4i4gkIRV3EZEkpOIuIpKEVNxFRJKQiruISBJScRcRSUIq7iIiSahyUB+ckZHhMjMzg/p4EZGEtHTp0p3OuQZljQusuGdmZpKbmxvUx4uIJCQz2xzOOE3LiIgkIRV3EZEkpOIuIpKEAptzF5HY+v7779myZQvffPNN0FEkDOnp6TRu3JgqVaoc1fIq7iIpYsuWLdSsWZPMzEzMLOg4UgrnHLt27WLLli00adLkqN5D0zIiKeKbb76hfv36KuwJwMyoX79+hf7KUnEXSSEq7Imjor+rxCvuq1bBuefC1q1BJxERiVuJV9zHj4d33oGePYNOIiLlsGvXLlq3bk3r1q1p2LAhjRo1Kvj+u+++C+s9+vXrx9q1a0sdM2XKFGbOnBmJyJxzzjksX748Iu8Va4l3QHXCBNi40Rf4OXPgqquCTiQiYahfv35BoRw1ahQ1atTgjjvu+MEY5xzOOSpVKn6/c/r06WV+zqBBgyoeNgkk3p57Rgb8859w9tnQvz/8619BJxKRCli/fj0tW7Zk4MCBZGdns23bNgYMGEBOTg4tWrRg9OjRBWOP7Enn5+dTp04dhg8fTqtWrTj77LPZsWMHACNHjmTixIkF44cPH07btm05/fTTee+99wD4+uuv6d69O61ataJ3797k5OSUuYc+Y8YMzjzzTFq2bMmIESMAyM/P55prril4ftKkSQA8/PDDNG/enFatWtG3b9+Ir7NwJN6eO0DVqn6vvU0b6NEDFi+GY48NOpVI4hgyBMKZbvj+e9i+HRo2hLLOt27dGkJFtbw++eQTpk+fztSpUwEYO3Ys9erVIz8/n06dOtGjRw+aN2/+g2X27NlDhw4dGDt2LEOHDuXJJ59k+PDhP3pv5xxLlizh5ZdfZvTo0SxYsIDJkyfTsGFD5s2bx8cff0x2dnap+bZs2cLIkSPJzc2ldu3adOnShfnz59OgQQN27tzJypUrAdi9ezcA48ePZ/PmzVStWrXguVhLvD33Ixo3hlmz4JNP4Fe/AueCTiSSfLZvhw0b/H+j6JRTTuFnP/tZwfezZs0iOzub7Oxs1qxZwyeffPKjZY455hguuugiAM466yw2bdpU7Ht369btR2PeeecdevXqBUCrVq1o0aJFqfkWL15M586dycjIoEqVKvTp04dFixZx6qmnsnbtWgYPHsxrr71G7dq1AWjRogV9+/Zl5syZR30RUkUl5p77EV26wOjRcPfd0L493Hxz0IlEEkO4e9g7d8L06dCvn58SjZJjC/3lvW7dOh555BGWLFlCnTp16Nu3b7Hne1etWrXgcVpaGvn5+cW+d7Vq1X40xpVzZ7Ck8fXr12fFihW8+uqrTJo0iXnz5jFt2jRee+013nrrLV566SXuu+8+Vq1aRVpaWrk+s6ISd8/9iBEjoGtX/2fmkiVBpxFJLhkZcOedUS3sRe3du5eaNWtSq1Yttm3bxmuvvRbxzzjnnHOYM2cOACtXriz2L4PC2rVrx8KFC9m1axf5+fnMnj2bDh06kJeXh3OOK6+8knvvvZdly5Zx6NAhtmzZQufOnXnggQfIy8vjwIEDEf8ZypLYe+4AlSrBM89AdjZceSUsWwb16wedSkSOUnZ2Ns2bN6dly5Y0bdqU9u3bR/wzbr31Vq699lqysrLIzs6mZcuWBVMqxWncuDGjR4+mY8eOOOe45JJLuPjii1m2bBn9+/fHOYeZMW7cOPLz8+nTpw/79u3j8OHDDBs2jJo1a0b8ZyiLlffPk0jJyclxEb1ZR26un5rp3Bn+9jdf9EWkwJo1a2jWrFnQMeJCfn4++fn5pKens27dOi644ALWrVtH5crxtb9b3O/MzJY653LKWja+fpKKyMmBSZNg4EC47z64556gE4lInNq/fz/nn38++fn5OOd4/PHH466wV1Ry/TQDBsC778KoUdCuHVxwQdCJRCQO1alTh6VLlwYdI6qSa+7CDKZOhRYtoE8f+PzzoBOJiAQiuYo7QPXqMG8efPedP8AaZs8KEZFkknzFHeC00+DJJ/2Vq0V6V4iIpILkLO7g2xLcdhtMngyzZwedRkQkppK3uAOMG+dPj7zxRlizJug0IimtY8eOP7ogaeLEidxyyy2lLlejRg0Atm7dSo8ePUp877JOrZ44ceIPLibq2rVrRPq+jBo1igcffLDC7xNpyV3cq1SB557zTcW6d4f9+4NOJJKyevfuzewif0XPnj2b3r17h7X8CSecwNy5c4/684sW91deeYU6deoc9fvFu+Qu7gCNGvkGY2vXwk03qcGYSEB69OjB/Pnz+fbbbwHYtGkTW7du5Zxzzik47zw7O5szzzyTl1566UfLb9q0iZYtWwJw8OBBevXqRVZWFj179uTgwYMF426++eaCdsG///3vAZg0aRJbt26lU6dOdOrUCYDMzEx27twJwIQJE2jZsiUtW7YsaBe8adMmmjVrxk033USLFi244IILfvA5xVm+fDnt2rUjKyuLK664gq+++qrg85s3b05WVlZBw7K33nqr4GYlbdq0Yd++fUe9bouTXOe5l6RzZ39h04gRfprm178OOpFIoIYsGMLy7WW3/P3+0Pds37+dhjUaUiWt9O6GrRu2ZuKFJTckq1+/Pm3btmXBggVcdtllzJ49m549e2JmpKen88ILL1CrVi127txJu3btuPTSS0u8j+hjjz1G9erVWbFiBStWrPhBy94//OEP1KtXj0OHDnH++eezYsUKfvOb3zBhwgQWLlxIRpE+OUuXLmX69OksXrwY5xw///nP6dChA3Xr1mXdunXMmjWLP//5z1x11VXMmzev1P7s1157LZMnT6ZDhw7cc8893HvvvUycOJGxY8eyceNGqlWrVjAV9OCDDzJlyhTat2/P/v37SU9PL3X9llfy77kfMWwY/O//wtCh8MEHQacRSQjb929nw+4NbN8fmZa/hadmCk/JOOcYMWIEWVlZdOnShf/85z988cUXJb7PokWLCopsVlYWWVlZBa/NmTOH7Oxs2rRpw+rVq8tsCvbOO+9wxRVXcOyxx1KjRg26devG22+/DUCTJk1o3bo1UHpbYfD95Xfv3k2HDh0AuO6661i0aFFBxquvvpoZM2YUXAnbvn17hg4dyqRJk9i9e3fEr5BNjT138L1mnn4azjrL35pv2bKYdroTiSel7WEXtvPATqZ/NJ1+bfqRUb3i/79cfvnlDB06lGXLlnHw4MGCPe6ZM2eSl5fH0qVLqVKlCpmZmcW2+S2suL36jRs38uCDD/Lhhx9St25drr/++jLfp7T+WkfaBYNvGVzWtExJ/va3v7Fo0SJefvllxowZw+rVqxk+fDgXX3wxr7zyCu3ateP111/njDPOOKr3L07q7LkD1K0Lc+fCjh1w9dVw6FDQiUTiWkb1DO5sf2dECjv4M186duzIDTfc8IMDqXv27OG4446jSpUqLFy4kM2bN5f6Puedd17BTbBXrVrFihUrAN8u+Nhjj6V27dp88cUXvPrqqwXL1KxZs9h57fPOO48XX3yRAwcO8PXXX/PCCy9w7rnnlvtnq127NnXr1i3Y63/mmWfo0KEDhw8f5vPPP6dTp06MHz+e3bt3s3//fj799FPOPPNMhg0bRk5ODv+K8C1DU2fP/YjsbH/u+4ABMGaM70MjIjHTu3dvunXr9oMzZ66++mouueQScnJyaN26dZl7sDfffDP9+vUjKyuL1q1b07ZtW8DfValNmza0aNHiR+2CBwwYwEUXXcTxxx/PwoULC57Pzs7m+uuvL3iPG2+8kTZt2pQ6BVOSp556ioEDB3LgwAGaNm3K9OnTOXToEH379mXPnj0457jtttuoU6cOd999NwsXLiQtLY3mzZsX3FUqUpKn5W95OOfvLPP00/DKK3DhhcHkEIkhtfxNPBVp+Zta0zJHmMGjj8KZZ/rpmc8+CzqRiEhEpWZxB99gbO5cyM/3DcZC596KiCSD1C3uAD/9qb/575IlcPvtQacRibqgpmGl/Cr6u0rt4g7QrZsv7FOmwLPPBp1GJGrS09PZtWuXCnwCcM6xa9euCl3YlHpnyxTnj3/07YFvuglatfI3+xBJMo0bN2bLli3k5eUFHUXCkJ6eTuPGjY96eRV3+G+Dsexs32Dsww8hgLuVi0RTlSpVaNKkSdAxJEY0LXPECSf4vu/r1vkWwfrTVUQSmIp7YR07wv33w5w5/kInEZEEVWZxN7N0M1tiZh+b2Wozu7eYMdXM7DkzW29mi80sMxphY+K3v4VLL/UHWd9/P+g0IiJHJZw992+Bzs65VkBr4EIza1dkTH/gK+fcqcDDwLjIxowhM3jqKTjpJN9gTAefRCQBlVncnXfkFkZVQl9FJ6QvA54KPZ4LnG8lNWJOBHXq+Auc8vKgTx81GBORhBPWnLuZpZnZcmAH8A/n3OIiQxoBnwM45/KBPUD9SAaNuTZt/Lnvr78O9/5oJkpEJK6FVdydc4ecc62BxkBbM2tZZEhxe+k/Ot3EzAaYWa6Z5SbEubb9+/sGY2PG+AZjIiIJolxnyzjndgNvAkXbKG4BTgQws8pAbeDLYpaf5pzLcc7lNGjQ4KgCx9yUKf7Cpr594ShagIqIBCGcs2UamFmd0ONjgC5A0a7yLwPXhR73AN5wyXKN8zHH+Pn3Q4fUYExEEkY4e+7HAwvNbAXwIX7Ofb6ZjTazS0NjngDqm9l6YCgwPDpxA3Lqqb73e24uDBkSdBoRkTKV2X7AObcCaFPM8/cUevwNcGVko8WZyy7z58CPHw/t2/tpGhGROKUrVMvjD3+ADh38LfpWrQo6jYhIiVTcy6NyZd9/pnZt32Bs796gE4mIFEvFvbwaNvQdJD/91J8qmSTHjUUkuai4H43zzvM94OfOhUceCTqNiMiPqLgfrTvugMsvhzvvhHffDTqNiMgPqLgfLTN//9WTT/YNxnbsCDqRiEgBFfeKqFMH5s2DL7+E3r3VYExE4oaKe0W1agWPPgpvvAH33FP2eBGRGFBxj4R+/fyZM/ffD/PnB51GRETFPWImT/Ztgq+5BjZuDDqNiKQ4FfdIOdJgDKBHD/jmm2DziEhKU3GPpKZNfYOxZctg8OCg04hIClNxj7RLLoHhw2HaNF/oRUQCoOIeDWPGQKdOMHAgrFwZdBoRSUEq7tFQuTLMmuXPg+/eHfbsCTqRiKQYFfdo+clPfIOxDRvghhvUYExEYkrFPZrOPRfGjYPnn4cJE4JOIyIpRMU92oYOhW7dYNgwePvtoNOISIpQcY82M3jySWjSBHr2hO3bg04kIilAxT0Watf2DcZ27/YNxvLzg04kIklOxT1WsrJg6lR48024++6g04hIklNxj6Vrr/U31x47Fl5+Oeg0IpLEVNxj7ZFHIDvbF/oNG4JOIyJJSsU91tLTfYOxSpXUYExEokbFPQhNmsAzz8BHH8GttwadRkSSkIp7UC6+GEaMgL/8xd+LVUQkglTcgzR6NHTuDLfcAsuXB51GRJKIinuQ0tJ8g7F69fz8++7dQScSkSSh4h60446DOXNg82Z/L1Y1GBORCFBxjwft28MDD8CLL8KDDwadRkSSgIp7vBg8GK68En73O1i0KOg0IpLgVNzjhZk/c+aUU3yDsW3bgk4kIglMxT2e1KrlG4zt3Qu9eqnBmIgcNRX3eNOyJTz+uJ+aueuuoNOISIJScY9Hffv6m2uPHw8vvRR0GhFJQCru8WriRMjJgeuug/Xrg04jIglGxT1eVasGf/3rfxuMHTwYdCIRSSAq7vEsMxNmzICPP4ZBg4JOIyIJpMzibmYnmtlCM1tjZqvNbHAxY2qb2f+Z2cehMf2iEzcFde0KI0f65mJPPBF0GhFJEOHsuecDtzvnmgHtgEFm1rzImEHAJ865VkBH4CEzqxrRpKls1Cjo0sXvvX/0UdBpRCQBlFncnXPbnHPLQo/3AWuARkWHATXNzIAawJf4fxQkEtLS4NlnoUEDNRgTkbCUa87dzDKBNsDiIi/9CWgGbAVWAoOdc4eLWX6AmeWaWW5eXt5RBU5ZDRr4BmOffebPoDn8o9UrIlIg7OJuZjWAecAQ59zeIi//ElgOnAC0Bv5kZrWKvodzbppzLsc5l9OgQYMKxE5RZ58NDz3kb679wANBpxGROBZWcTezKvjCPtM593wxQ/oBzztvPbAROCNyMaXArbfCVVf5uzi9+WbQaUQkToVztowBTwBrnHMTShj2GXB+aPxPgNOBDZEKKYUcaTB22mm+/4wajIlIMcLZc28PXAN0NrPloa+uZjbQzAaGxowBfmFmK4F/AsOcczujlFlq1oS5c2HfPr8X//33QScSkThTuawBzrl3ACtjzFbggkiFkjC0aAF//jNcfbXvAa+bfIhIIbpCNZH16eNvrv3QQ/B8cYdCRCRVqbgnugkToG1bf//VdeuCTiMicULFPdFVq+bPf69cGbp3hwMHgk4kInFAxT0ZnHwyzJwJq1b5aRrngk4kIgFTcU8WF14I99wDTz3lT5UUkZSm4p5M7r4bLrjAX+i0bFnQaUQkQCruySQtzU/PHHecn3//6qugE4lIQFTck01Ghr+D03/+A9deqwZjIilKxT0Z/fzn/hTJ+fNh3Lig04hIAFTck9WgQb73zMiR8M9/Bp1GRGJMxT1Zmfn2BKefDr17+2kaEUkZKu7JrEYNmDfPX9jUs6cajImkEBX3ZNesmT/v/d13YdiwoNOISIyouKeCXr3g17+Ghx/2rYJFJOmpuKeKhx7yZ9HccAP8+99BpxGRKFNxTxVVq/rz36tV8xc4ff110IlEJIpU3FPJiSfCs8/C6tVw881qMCaSxFTcU83//A+MGgXPPAPTpgWdRkSiRMU9FY0c6btI/uY3kJsbdBoRiQIV91RUqRLMmAENG0KPHrBrV9CJRCTCVNxTVf36/gDr1q1wzTVqMCaSZFTcU1nbtjBxIrz6Ktx/f9BpRCSCVNxT3c03Q58+/i5Or78edBoRiRAV91Rn5s+aadbMNxjbsiXoRCISASruAsce6xuMffMNXHUVfPdd0IlEpIJU3MU74wx48kl4/3347W+DTiMiFaTiLv915ZUweDA88gjMmRN0GhGpABV3+aHx4+Hss6F/f/jXv4JOIyJHScVdfqhqVb/Xnp7uG4zt3x90IhE5Ciru8mONG8OsWbBmDfzqV2owJpKAVNyleF26wOjRvovkY48FnUZEyknFXUo2YgR07QpDhsCSJUGnEZFyUHGXklWq5FsDn3CCP5NGDcZEEoaKu5SuXj1/39Xt26FvXzUYE0kQKu5Stpwcf+77ggVw331BpxGRMKi4S3h+9SvfGnjUKPj734NOIyJlUHGX8JjB1KnQooXvIvn550EnEpFSlFnczexEM1toZmvMbLWZDS5hXEczWx4a81bko0rgqlf3Dca++84fYFWDMZG4Fc6eez5wu3OuGdAOGGRmzQsPMLM6wKPApc65FsCVEU8q8eG003yDscWL4Y47gk4jIiUos7g757Y555aFHu8D1gCNigzrAzzvnPssNG5HpINKHOnRA267DSZPhtmzg04jIsUo15y7mWUCbYDFRV46DahrZm+a2VIzuzYy8SRujRsH7dvDjTfCJ58EnUZEigi7uJtZDWAeMMQ5t7fIy5WBs4CLgV8Cd5vZacW8xwAzyzWz3Ly8vArElsBVqQLPPedv9NGjhxqMicSZsIq7mVXBF/aZzrnnixmyBVjgnPvaObcTWAS0KjrIOTfNOZfjnMtp0KBBRXJLPGjUyDcYW7sWbrpJDcZE4kg4Z8sY8ASwxjk3oYRhLwHnmlllM6sO/Bw/Ny/JrnNnGDPGz71PmRJ0GhEJqRzGmPbANcBKM1seem4EcBKAc26qc26NmS0AVgCHgb8451ZFI7DEoeHD/e35hg71V7O2axd0IpGUZy6gP6VzcnJcbm5uIJ8tUfDVV5CdDYcOwbJlkJERdCKRpGRmS51zOWWN0xWqEhl16/oLnHbsgKuv9kVeRAKj4i6Rk53tz33/+9/9PLyIBEbFXSLrxhvhuuv8XZwWLAg6jUjKUnGXyDKDRx+FM8/00zObNwedSCQlqbhL5FWv7m/wkZ/vG4x9+23QiURSjoq7RMdPfwrTp8OHH/pTJEUkplTcJXq6dYPbb/fTNM8+G3QakZSi4i7R9cc/wjnn+PYEq1cHnUYkZai4S3QdaTBWsyZ07w779gWdSCQlqLhL9J1wgu89s26dP1VSDcZEok7FXWKjY0e4/36YM8df6CQiUaXiLrHz29/CpZf6g6zvvx90GpGkpuIusWMGTz0FJ50EV10FumGLSNSouEts1anjL3DKy4M+fdRgTCRKVNwl9tq08Tf2eP11GDUq6DQiSUnFXYLRvz/06wf33QevvBJ0GpGko+IuwZkyBVq1gr59YdOmoNOIJBUVdwnOMcf4+fdDh9RgTCTCVNwlWKee6s+gyc2FIUOCTiOSNFTcJXiXXw533glTp8KMGUGnEUkKKu4SH+6/H847DwYMgFWrgk4jkvBU3CU+VK7sG4zVru0bjO3dG3QikYSm4i7xo2FDX+A//dSfKqkGYyJHTcVd4st55/ke8HPnwiOPBJ1GJGGpuEv8ueOO/x5kfffdoNOIJCQVd4k/Zv7+qyef7BuM7dgRdCKRhKPiLvGpTh2YNw++/BJ691aDMZFyUnGX+NWqlb+59htvwD33BJ1GJKGouEt869fPnzlz//0wf37QaUQShoq7xL/Jk6F1a7jmGti4Meg0IglBxV3i3zHH+Pl3gB494Jtvgs0jkgBU3CUxNG0KTz8Ny5bB4MFBpxGJeyrukjguuQSGD4dp03yhF5ESqbhLYhkzBjp1goEDYcWKoNOIxC0Vd0kslSvDrFn+PPju3WHPnqATicQlFXdJPD/5iW8wtnEj3HCDGoyJFEPFXRLTuefCuHHw/PMwYULQaUTijoq7JK6hQ6FbNxg2DN5+O+g0InGlzOJuZiea2UIzW2Nmq82sxPPQzOxnZnbIzHpENqZIMczgySehSRPo2RO2bw86kUjcCGfPPR+43TnXDGgHDDKz5kUHmVkaMA54LbIRRUpRu7a/wGn3bt9gLD8/6EQicaHM4u6c2+acWxZ6vA9YAzQqZuitwDxA/VkltrKy4LHH4M034e67g04jEhfKNeduZplAG2BxkecbAVcAU8tYfoCZ5ZpZbl5eXvmSipTmuuv8zbXHjoWXXw46jUjgwi7uZlYDv2c+xDlX9O7FE4FhzrlSm24756Y553KcczkNGjQof1qR0jzyCGRnw7XXwoYNQacRCVRYxd3MquAL+0zn3PPFDMkBZpvZJqAH8KiZXR6xlCLhSE/3916tVMk3GDt4MOhEIoEJ52wZA54A1jjnij2h2DnXxDmX6ZzLBOYCtzjnXoxoUpFwNGkCzzwDH30Et94adBqRwISz594euAbobGbLQ19dzWygmQ2Mcj6R8rv4YhgxAp54wt+LVSQFmQvo0u2cnByXm5sbyGdLCjh0CC64AN57D95/39/sQyQJmNlS51xOWeN0haokp7Q032CsXj0//757d9CJRGJKxV2S13HHwZw5sHmzvxerGoxJClFxl+TWvj2MHw8vvggPPhh0GpGYUXGX5DdkiJ+a+d3vYNGioNOIxISKuyQ/M3/mzCmn+AZj27YFnUgk6lTcJTXUquUbjO3dC716qcGYJD0Vd0kdLVvC44/7qZm77go6jUhUqbhLaunb199c+8hBVpEkpeIuqWfiRMjJ8Z0k168POo1IVKi4S+qpVg3++ld/oZMajEmSUnGX1JSZCTNmwMcfw6BBQacRiTgVd0ldXbvCyJG+udgTTwSdRiSiVNwltY0aBV26+L33jz4KOo1IxKi4S2pLS4Nnn4WMDDUYk6Si4i7SoIE/wPrZZ/4MmsOHg04kUmEq7iIAZ58NDz3kb679wANBpxGpsIS7WcfslbO59dVbqVWtFmmV0qKQLLb8XQyTg5EEP8vWreTv38feY4za+WlUdtr/CVcS/PZjIp/DnH64HtNvX0TGiaeXe/lwb9ZR+ajSBeidz99h58GdNK7VmGYNmgUdp0IcydNfPKidhIjLaMGaDxfwafUDHL+7MmdUahB0ooSQJL/9mFh7OI/5dXcw/Zmh3Dnib1H7nIQr7qM6juLk2ifTr00/MqpnBB1HktDO9muZ/sxQ+g2dcFR7ViKl2fl5aPu6ZkJUPyfhpmVERFKZ7qEqIpLCVNxFRJKQiruISBJScRcRSUIq7iIiSUjFXUQkCam4i4gkocDOczezPGDzUS6eAeyMYJxIiddcEL/ZlKt8lKt8kjHXyc65Mi+dDqy4V4SZ5YZzEn+sxWsuiN9sylU+ylU+qZxL0zIiIklIxV1EJAklanGfFnSAEsRrLojfbMpVPspVPimbKyHn3EVEpHSJuucuIiKliLvibmYXmtlaM1tvZsOLeb2amT0Xen2xmWUWeu13oefXmtkvY5xrqJl9YmYrzOyfZnZyodcOmdny0NfLMc51vZnlFfr8Gwu9dp2ZrQt9XRfjXA8XyvRvM9td6LVorq8nzWyHma0q4XUzs0mh3CvMLLvQa9FcX2XlujqUZ4WZvWdmrQq9tsnMVobWV0T7aIeRq6OZ7Sn0+7qn0GulbgNRznVnoUyrQttUvdBrUVlfZnaimS00szVmttrMBhczJnbbl3Mubr6ANOBToClQFfgYaF5kzC3A1NDjXsBzocfNQ+OrAU1C75MWw1ydgOqhxzcfyRX6fn+A6+t64E/FLFsP2BD6b93Q47qxylVk/K3Ak9FeX6H3Pg/IBlZKCjOTAAAEHElEQVSV8HpX4FX8XePaAYujvb7CzPWLI58HXHQkV+j7TUBGQOurIzC/ottApHMVGXsJ8Ea01xdwPJAdelwT+Hcx/z/GbPuKtz33tsB659wG59x3wGzgsiJjLgOeCj2eC5xvZhZ6frZz7lvn3EZgfej9YpLLObfQOXcg9O0HQOMIfXaFcpXil8A/nHNfOue+Av4BXBhQrt7ArAh9dqmcc4uAL0sZchnwtPM+AOqY2fFEd32Vmcs5917ocyF221c466skFdk2I50rJtuXc26bc25Z6PE+YA3QqMiwmG1f8VbcGwGfF/p+Cz9eOQVjnHP5wB6gfpjLRjNXYf3x/zofkW5muWb2gZldHqFM5cnVPfQn4FwzO7Gcy0YzF6HpqybAG4Wejtb6CkdJ2aO5vsqr6PblgL+b2VIzGxBAnrPN7GMze9XMWoSei4v1ZWbV8UVyXqGno76+zE8XtwEWF3kpZttXvN1DtbgbqBc9naekMeEse7TCfm8z6wvkAB0KPX2Sc26rmTUF3jCzlc65T2OU6/+AWc65b81sIP6vns5hLhvNXEf0AuY65w4Vei5a6yscQWxfYTOzTvjifk6hp9uH1tdxwD/M7F+hPdtYWIa/HH6/mXUFXgR+SpysL/yUzLvOucJ7+VFdX2ZWA/+PyRDn3N6iLxezSFS2r3jbc98CnFjo+8bA1pLGmFlloDb+z7Nwlo1mLsysC3AXcKlz7tsjzzvntob+uwF4E/8vekxyOed2FcryZ+CscJeNZq5CelHkT+Yorq9wlJQ9musrLGaWBfwFuMw5t+vI84XW1w7gBSI3HVkm59xe59z+0ONXgCpmlkEcrK+Q0raviK8vM6uCL+wznXPPFzMkdttXpA8qVPCARGX8gYQm/PcgTIsiYwbxwwOqc0KPW/DDA6obiNwB1XBytcEfQPppkefrAtVCjzOAdUTowFKYuY4v9PgK4AP33wM4G0P56oYe14tVrtC40/EHtywW66vQZ2RS8gHCi/nhAa8l0V5fYeY6CX8c6RdFnj8WqFno8XvAhTHM1fDI7w9fJD8LrbuwtoFo5Qq9fmTH79hYrK/Qz/00MLGUMTHbviK2oiP4C+uKP8r8KXBX6LnR+L1hgHTgr6ENfQnQtNCyd4WWWwtcFONcrwNfAMtDXy+Hnv8FsDK0ca8E+sc41x+B1aHPXwicUWjZG0LrcT3QL5a5Qt+PAsYWWS7a62sWsA34Hr+31B8YCAwMvW7AlFDulUBOjNZXWbn+AnxVaPvKDT3fNLSuPg79nu+Kca5fF9q+PqDQPz7FbQOxyhUacz3+JIvCy0VtfeGnyhywotDvqWtQ25euUBURSULxNucuIiIRoOIuIpKEVNxFRJKQiruISBJScRcRSUIq7iIiSUjFXUQkCam4i4gkof8HYE+G0UEg45IAAAAASUVORK5CYII=\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8FdX9//HXBwgi+xaLBRHUVtlJjIpiZdFal1oXQEBAQZBiqYKoPxApKu6oSEFaxQU3BCm41brUVr5StGJDyiqloGCloAIKgrgQ/Pz+mEsaQpJ7k9zcuffm/Xw8eHAzc2bum8nlk8nMmXPM3RERkfRSLewAIiISfyruIiJpSMVdRCQNqbiLiKQhFXcRkTSk4i4ikoaiFnczq2Vm75nZcjNbbWa3FNPmEDN71szWm9kSM2tVGWFFRCQ2sZy5fwv0dPdOQGfgLDPrUqTNUOALdz8GuB+4O74xRUSkLKIWdw/sjnyZEflT9Mmn84EnIq/nA6ebmcUtpYiIlEmNWBqZWXVgKXAMMMPdlxRp0hz4GMDd881sJ9AE2FbSPps2beqtWrUqT2YRkSpr6dKl29w9M1q7mIq7u+8DOptZQ+B5M2vv7qsKNSnuLP2gcQ3MbDgwHKBly5bk5ubG8vYiIhJhZh/F0q5MvWXcfQfwf8BZRVZtAo6IvHENoAHweTHbz3T3HHfPycyM+oNHRETKKZbeMpmRM3bM7FDgDOBfRZq9BFwWed0beNM1IpmISGhiuSxzOPBE5Lp7NWCeu79sZpOAXHd/CXgUeMrM1hOcsfertMQiIhJV1OLu7iuArGKWTyz0+hugT3yjiUg87d27l02bNvHNN9+EHUViUKtWLVq0aEFGRka5to/phqqIpL5NmzZRr149WrVqhXoqJzd3Z/v27WzatInWrVuXax8afkCkivjmm29o0qSJCnsKMDOaNGlSod+yVNxFqhAV9tRR0e9V6hX3VavgJz+BzZvDTiIikrRSr7hPngyLF0PfvmEnEZEy2L59O507d6Zz5840a9aM5s2bF3z93XffxbSPIUOGsHbt2lLbzJgxg9mzZ8cjMqeeeirLli2Ly74SLfVuqE6ZAhs2BAV+3jy4+OKwE4lIDJo0aVJQKG+++Wbq1q3Lddddd0Abd8fdqVat+PPOWbNmRX2fkSNHVjxsGki9M/emTeGvf4WTT4ahQ+FfRZ+nEpFUsn79etq3b8+IESPIzs5my5YtDB8+nJycHNq1a8ekSZMK2u4/k87Pz6dhw4aMGzeOTp06cfLJJ/PZZ58BMGHCBKZOnVrQfty4cZx44okce+yxvPPOOwB89dVX9OrVi06dOtG/f39ycnKinqE//fTTdOjQgfbt2zN+/HgA8vPzGTRoUMHyadOmAXD//ffTtm1bOnXqxMCBA+N+zGKRemfuADVrBmft2dnQqxcsWQJ164adSiR1jB4NsVxu2LsXPvkEmjWDaP2tO3eGSFEtq/fff59Zs2bx4IMPAnDXXXfRuHFj8vPz6dGjB71796Zt27YHbLNz5066devGXXfdxZgxY3jssccYN27cQft2d9577z1eeuklJk2axGuvvcb06dNp1qwZCxYsYPny5WRnZ5eab9OmTUyYMIHc3FwaNGjAGWecwcsvv0xmZibbtm1j5cqVAOzYsQOAyZMn89FHH1GzZs2CZYmWemfu+7VoAXPmBGfuv/wlaLQDkfj75BP48MPg70p09NFHc8IJJxR8PWfOHLKzs8nOzmbNmjW8//77B21z6KGHcvbZZwNw/PHHs3HjxmL3fdFFFx3UZvHixfTrFzxI36lTJ9q1a1dqviVLltCzZ0+aNm1KRkYGl1xyCYsWLeKYY45h7dq1jBo1itdff50GDRoA0K5dOwYOHMjs2bPL/RBSRaXmmft+p58OkybBhAnQtSv86ldhJxJJDbGeYW/bBrNmwZAhwSXRSlKnTp2C1+vWreO3v/0t7733Hg0bNmTgwIHF9veuWbNmwevq1auTn59f7L4POeSQg9qUdeirkto3adKEFStW8OqrrzJt2jQWLFjAzJkzef3113nrrbd48cUXue2221i1ahXVq1cv03tWVOqeue93ww1w7rnBr5lLig4zLyIV0rQpXH99pRb2or788kvq1atH/fr12bJlC6+//nrc3+PUU09l3rx5AKxcubLY3wwK69KlCwsXLmT79u3k5+czd+5cunXrxtatW3F3+vTpwy233EJeXh779u1j06ZN9OzZk3vuuYetW7eyZ8+euP8bokntM3eAatXgqaeC6+99+kBeXkI/iCISX9nZ2bRt25b27dtz1FFH0bVr17i/x1VXXcWll15Kx44dyc7Opn379gWXVIrTokULJk2aRPfu3XF3zjvvPM4991zy8vIYOnQo7o6Zcffdd5Ofn88ll1zCrl27+P777xk7diz16tWL+78hGgtrZN6cnByP62QdS5fCKadAjx7wpz9Bgn8FEkl2a9asoU2bNmHHSAr5+fnk5+dTq1Yt1q1bx5lnnsm6deuoUSO5zneL+56Z2VJ3z4m2bXL9Syri+ONh+vTg5uptt8FNN4WdSESS1O7duzn99NPJz8/H3XnooYeSrrBXVHr9a664At55B265BU46Cc4qOmGUiAg0bNiQpUuXhh2jUqX+DdXCzOB3v4MOHWDAAPjPf8JOJCISivQq7gC1a8P8+ZCfH9xg/fbbsBOJiCRc+hV3gB/9KOib+957cO21YacREUm49CzuABddBNddBzNmwDPPhJ1GRCSh0re4A9x5ZzD2+xVXwOrVYacRqdK6d+9+0ANJU6dO5VdRniyvGxk3avPmzfTu3bvEfUfrWj116tQDHiY655xz4jLuy80338y9995b4f3EW3oX9xo14NlnoV69YICxXbvCTiRSZfXv35+5c+cesGzu3Ln0798/pu1/+MMfMn/+/HK/f9Hi/sorr9CwYcNy7y/ZRS3uZnaEmS00szVmttrMRhXTppGZPW9mK8zsPTNrXzlxy+Hww2HuXFi3DoYN0wBjIiHp3bs3L7/8Mt9GOjls3LiRzZs3c+qppxb0O8/OzqZDhw68+OKLB22/ceNG2rcPSsvXX39Nv3796NixI3379uXrr78uaHfllVcWDBd8U+R5l2nTprF582Z69OhBjx49AGjVqhXbtm0DYMqUKbRv35727dsXDBe8ceNG2rRpwxVXXEG7du0488wzD3if4ixbtowuXbrQsWNHLrzwQr744ouC92/bti0dO3YsGLDsrbfeKpisJCsri11xPvmMpZ97PnCtu+eZWT1gqZm94e6FB2MYDyxz9wvN7DhgBnB6XJNWRPfuwSWasWODp1hHHfTzSaRKGf3aaJZ9En3I37379vLJ7k9oVrcZGdVLH92wc7POTD2r5AHJmjRpwoknnshrr73G+eefz9y5c+nbty9mRq1atXj++eepX78+27Zto0uXLvziF78ocR7R3//+99SuXZsVK1awYsWKA4bsvf3222ncuDH79u3j9NNPZ8WKFVx99dVMmTKFhQsX0rTI8CRLly5l1qxZLFmyBHfnpJNOolu3bjRq1Ih169YxZ84cHn74YS6++GIWLFhQ6vjsl156KdOnT6dbt25MnDiRW265halTp3LXXXexYcMGDjnkkIJLQffeey8zZsyga9eu7N69m1q1apV6fMsq6pm7u29x97zI613AGqB5kWZtgb9G2vwLaGVmP4hr0oq6/no4//zgJmtkwH4RKd0nuz/hwx0f8snu+Az5W/jSTOFLMu7O+PHj6dixI2eccQb//e9/+fTTT0vcz6JFiwqKbMeOHenYsWPBunnz5pGdnU1WVharV6+OOijY4sWLufDCC6lTpw5169bloosu4m9/+xsArVu3pnPnzkDpwwpDML78jh076NatGwCXXXYZixYtKsg4YMAAnn766YInYbt27cqYMWOYNm0aO3bsiPsTsmXam5m1ArKAosMvLgcuAhab2YnAkUALoOTvTqKZweOPQ05OMDVfXh4cdljYqURCUdoZdmHb9mxj1j9nMSRrCE1rV3xAvgsuuIAxY8aQl5fH119/XXDGPXv2bLZu3crSpUvJyMigVatWxQ7zW1hxZ/UbNmzg3nvv5R//+AeNGjVi8ODBUfdT2vha+4cLhmDI4GiXZUrypz/9iUWLFvHSSy9x6623snr1asaNG8e5557LK6+8QpcuXfjLX/7CcccdV679FyfmG6pmVhdYAIx29y+LrL4LaGRmy4CrgH8SXM4puo/hZpZrZrlbt26tQOxyatgweMBp+3a45BLYty/xGURSSNPaTbm+6/VxKewQ9Hzp3r07l19++QE3Unfu3Mlhhx1GRkYGCxcu5KOPPip1P6eddlrBJNirVq1ixYoVQDBccJ06dWjQoAGffvopr776asE29erVK/a69mmnncYLL7zAnj17+Oqrr3j++ef5yU9+UuZ/W4MGDWjUqFHBWf9TTz1Ft27d+P777/n444/p0aMHkydPZseOHezevZsPPviADh06MHbsWHJycvhXnKcMjenM3cwyCAr7bHd/ruj6SLEfEmlrwIbIn6LtZgIzIRgVsvyxK6Bz52CIgssvDwYXu+22UGKIVFX9+/fnoosuOqDnzIABAzjvvPPIycmhc+fOUc9gr7zySoYMGULHjh3p3LkzJ554IhDMqpSVlUW7du0OGi54+PDhnH322Rx++OEsXLiwYHl2djaDBw8u2MewYcPIysoq9RJMSZ544glGjBjBnj17OOqoo5g1axb79u1j4MCB7Ny5E3fnmmuuoWHDhvzmN79h4cKFVK9enbZt2xbMKhUvUYf8jRTrJ4DP3X10CW0aAnvc/TszuwL4ibtfWtp+4z7kb1kNGwaPPgovvxxM9iGS5jTkb+qp7CF/uwKDgJWRyy4Q9I5pCeDuDwJtgCfNbB/wPjA09vghmT49GAN+0KDg+nurVmEnEhGJm6jF3d0XA8X3R/pfm78DP4pXqIQ49FBYsCCYwal3b1i8GOLcFUlEJCzp/YRqNEcdBU8+GZzBq++7VAFhzbwmZVfR71XVLu4Av/gFjBsHM2cGhV4kTdWqVYvt27erwKcAd2f79u0VerApvWZiKq9bb4UlS2DEiKA3TaEHIkTSRYsWLdi0aROhdEOWMqtVqxYtWrQo9/Yq7hAMMDZnDmRlBQOM5eZCKTOhi6SijIwMWrduHXYMSRBdltnvBz+AefNgw4agD7x+dRWRFKbiXtipp8LkyfDcczBlSthpRETKTcW9qGuuCS7NjB0LkceIRURSjYp7UWbw2GNBN8mLL4ZP4jManohIIqm4F6d+/eABp507oV8/yD9oDDQRkaSm4l6SDh3goYfgrbdgwoSw04iIlImKe2kGDYJf/hLuvhuKmfZLRCRZqbhHM3VqMMHHZZfBBx+EnUZEJCYq7tHUqgV/+ANUqxb0oinnTCwiIomk4h6LVq3g6adh+XL49a/DTiMiEpWKe6zOOSe4sfrYY8EkHyIiSUzFvSxuvhnOOANGjoR//jPsNCIiJVJxL4vq1eGZZyAzM5jgY8eOsBOJiBRLxb2sMjODAcb+85+gB83334edSETkICru5XHyyXDfffDSS3DPPWGnERE5iIp7eV11FfTtC+PHw8KFYacRETmAint5mcEjj8CPfxyMP7N5c9iJREQKRC3uZnaEmS00szVmttrMDppJ2swamNkfzWx5pM2QyombZOrWDQYY++qr4Cx+796wE4mIALGduecD17p7G6ALMNLM2hZpMxJ43907Ad2B+8ysZlyTJqu2beHhh2HxYrjhhrDTiIgAMRR3d9/i7nmR17uANUDzos2AemZmQF3gc4IfClVD//7Bk6v33RecyYuIhKxM19zNrBWQBSwpsuoBoA2wGVgJjHL3g/oImtlwM8s1s9y0m4H9vvvgpJNgyBD497/DTiMiVVzMxd3M6gILgNHu/mWR1T8DlgE/BDoDD5hZ/aL7cPeZ7p7j7jmZmZkViJ2EatYM+r/XrBk84LRnT9iJRKQKi6m4m1kGQWGf7e7PFdNkCPCcB9YDG4Dj4hczRbRsGTzBumoVjBgB7mEnEpEqKpbeMgY8Cqxx9yklNPsPcHqk/Q+AY4EP4xUypZx5ZjAGzVNPwcyZYacRkSqqRgxtugKDgJVmtiyybDzQEsDdHwRuBR43s5WAAWPdfVsl5E0NEybA3/8OV18Nxx8fTPYhIpJA5iFdOsjJyfHc3NxQ3jshtm+H7OzgYae8PGjcOOxEIpIGzGypu0c9Y9QTqpWlSROYPx+2bIGBAzXAmIgklIp7ZTrhhGAO1ldfhTvuCDuNiFQhKu6VbcQIGDAAJk6Ev/wl7DQiUkWouFc2M3jooWCYgv79YdOmsBOJSBWg4p4IdeoEwxJ88w306QPffRd2IhFJcyruiXLssTBrFrz7Llx/fdhpRCTNqbgnUu/eMHo0TJsGzz4bdhoRSWMq7ok2eTKccgoMGwZr1oSdRkTSlIp7omVkBAOMHXoo9OoFu3eHnUhE0pCKexiaN4e5c2HtWhg+XAOMiUjcqbiHpWdPuPVWmDMHfve7sNOISJpRcQ/TuHHw85/DNdfAkqLzn4iIlJ+Ke5iqVYMnnwwu0/TpA9uq7kCaIhJfKu5ha9QoeMDps8+CYQr27Qs7kYikARX3ZJCdDdOnw5//HFyHFxGpIBX3ZDFsGFx2GUyaBK+9FnYaEUlxKu7JwizoNdOhQ3B55qOPwk4kIilMxT2Z1K4dXH/Pzw9usH77bdiJRCRFqbgnm2OOgccfh3/8A8aMCTuNiKQoFfdkdOGFcN11wWWa2bPDTiMiKShqcTezI8xsoZmtMbPVZjaqmDbXm9myyJ9VZrbPzDQjdEXceSecdlowPMHq1WGnEZEUE8uZez5wrbu3AboAI82sbeEG7n6Pu3d2987ADcBb7v55/ONWITVqBOPP1KsXDDC2a1fYiUQkhUQt7u6+xd3zIq93AWuA5qVs0h+YE594Vdzhhwfjvq9fD0OHaoAxEYlZma65m1krIAsodiAUM6sNnAUsqGgwiejWLbhE84c/BJN8iIjEIObibmZ1CYr2aHf/soRm5wFvl3RJxsyGm1mumeVu3bq17GmrquuugwsuCP5+++2w04hICoipuJtZBkFhn+3uz5XStB+lXJJx95nunuPuOZmZmWVLWpWZBfOvHnkkXHxxMA6NiEgpYuktY8CjwBp3n1JKuwZAN+DF+MWTAg0bBg84ff459O+vAcZEpFSxnLl3BQYBPQt1dzzHzEaY2YhC7S4E/uzuX1VKUoFOneD3v4c334SJE8NOIyJJrEa0Bu6+GLAY2j0OPF7xSFKqwYOD6+533AEnnxxM9iEiUoSeUE1F06dDVhYMGgQbNoSdRkSSkIp7KqpVC+bPD1737g3ffBNuHhFJOiruqeqoo4Ip+vLy4Oqrw04jIklGxT2VnXce3HADPPwwPPFE2GlEJImouKe6SZOgRw8YMQJWrAg7jYgkCRX3VFejBsyZE0y03asX7NwZdiIRSQIq7ungBz8Ixp7ZuDHoKqkBxkSqPBX3dNG1K0yeDC+8APfdF3YaEQmZins6GT066Bo5bhwsWhR2GhEJkYp7OjGDRx+Fo4+Gvn3hk0/CTiQiIVFxTzf16wcDjH35ZVDg8/PDTiQiIVBxT0ft28NDDwWXZm68Mew0IhICFfd0NXBg0Pd98mR4UaMwi1Q1Ku7pbOpUyMmByy4L5mEVkSpDxT2dHXJI0P+9evWgF83XX4edSEQSRMU93bVqBU8/HQxNMHJk2GlEJEFU3KuCs8+GCROCeVgffTTsNCKSACruVcVNN8FPfxqcvf/zn2GnEZFKpuJeVVSvDrNnQ2ZmMMDYF1+EnUhEKpGKe1WSmRncYN20KehB8/33YScSkUqi4l7VdOkSDCz2xz8GfeBFJC1FLe5mdoSZLTSzNWa22sxGldCuu5kti7R5K/5RJW5+/Wvo1y94enXhwrDTiEgliOXMPR+41t3bAF2AkWbWtnADM2sI/A74hbu3A/rEPanEj1kwNd+xxwZF/r//DTuRiMRZ1OLu7lvcPS/yehewBmhepNklwHPu/p9Iu8/iHVTirG7dYICxr74KBhjbuzfsRCISR2W65m5mrYAsYEmRVT8GGpnZ/5nZUjO7ND7xpFK1aQOPPAJvvx2MAS8iaaNGrA3NrC6wABjt7l8Ws5/jgdOBQ4G/m9m77v7vIvsYDgwHaNmyZUVyS7z06wfvvANTpsDJJwfDFIhIyovpzN3MMggK+2x3f66YJpuA19z9K3ffBiwCOhVt5O4z3T3H3XMyMzMrklvi6d57g140l18O//539PYikvRi6S1jwKPAGnefUkKzF4GfmFkNM6sNnERwbV5SQc2aMG9eMNBYr17BdXgRSWmxnLl3BQYBPSNdHZeZ2TlmNsLMRgC4+xrgNWAF8B7wiLuvqrTUEn9HHAHPPAOrVwfjwLuHnUhEKiDqNXd3XwxYDO3uAe6JRygJyU9/CrfcAhMnQteuQZEXkZSkJ1TlQDfeGIwiOWoU5OaGnUZEyknFXQ5UrRo89RQ0axb0nNm+PexEIlIOKu5ysCZNYP582LIFBg3SAGMiKUjFXYp3wgnw29/Cq6/C7beHnUZEykjFXUr2y1/CwIHBRB9vvBF2GhEpAxV3KZkZPPggtG0Ll1wCH38cdiIRiZGKu5SuTp1ggLFvv4WLL4bvvgs7kYjEQMVdojv2WHjsMXj3XbjuurDTiEgMVNwlNr17wzXXwPTpMHdu2GlEJAoVd4nd3XcHT64OGwZrNHSQSDJTcZfYZWTAs88G1+F79YLdu8NOJCIlUHGXsmneHObMgbVr4YorNMCYSJJScZey69kTbrstuPY+Y0bYaUSkGCruUj5jx8LPfw5jxgS9aEQkqai4S/lUqwZPPgktWkCfPrB1a9iJRKQQFXcpv0aNggectm6FAQNg376wE4lIhIq7VExWFjzwQDD2zKRJYacRkQgVd6m4oUNh8GC49VZ47bWw04gIKu4SD2ZBr5mOHYPLMx99FHYikSpPxV3io3btYIKP/PxgqIJvvw07kUiVpuIu8XPMMfDEE8Hcq9dcE3YakSotanE3syPMbKGZrTGz1WY2qpg23c1sp5kti/yZWDlxJeldcAFcfz38/vcwe3bYaUSqrBoxtMkHrnX3PDOrByw1szfc/f0i7f7m7j+Pf0RJOXfcAUuWwPDh0KkTtG8fdiKRKifqmbu7b3H3vMjrXcAaoHllB5MUVqNGMMBY/frBAGNffhl2IpEqp0zX3M2sFZAFLClm9clmttzMXjWzdiVsP9zMcs0sd6ueaExvzZoFBf6DD4KukhpgTCShYi7uZlYXWACMdveip2J5wJHu3gmYDrxQ3D7cfaa757h7TmZmZnkzS6o47TS4886gF81vfxt2GpEqJabibmYZBIV9trs/V3S9u3/p7rsjr18BMsysaVyTSmq67rr/3WR9++2w04hUGbH0ljHgUWCNu08poU2zSDvM7MTIfrfHM6ikKDN4/HFo1SqYYPuzz8JOJFIlxHLm3hUYBPQs1NXxHDMbYWYjIm16A6vMbDkwDejnrousEtGgQXBp5vPPoX9/DTAmkgBRu0K6+2LAorR5AHggXqEkDXXqFPR9HzIEJk6E228PO5FIWtMTqpI4gwcHU/PdcQf88Y9hpxFJayrukljTpkF2Nlx6KXz4YdhpRNKWirskVq1awfV3CGZw+uabcPOIpCkVd0m81q3hqacgLw+uvjrsNCJpScVdwvHzn8P48fDww0FXSRGJKxV3Cc+kSdCzJ1x5JSxfHnYakbSi4i7hqV4d5syBxo2DCT527gw7kUjaUHGXcB12GMybBxs3Bl0l9eybSFyouEv4unaFe+6BF16Ae+8NO41IWlBxl+QwalTQNfKGG2DRorDTiKQ8FXdJDmbwyCNw9NHQty9s2RJ2IpGUpuIuyaN+fViwIJi5qV8/yM8PO5FIylJxl+TSvj3MnBlcmhk/Puw0IilLxV2Sz4ABQd/3/TdZRaTMVNwlOd1/P5xwAlx2GaxfH3YakZSj4i7J6ZBD4A9/gBo1oFcv2LMn7EQiKUXFXZLXkUfC7NmwciWMHKkHnETKQMVdkttZZ8FvfhMMLvboo2GnEUkZKu6S/CZOhDPPhF//OhgmWESiUnGX5Fe9enB55rDDggHGvvgi7EQiSS9qcTezI8xsoZmtMbPVZjaqlLYnmNk+M+sd35hS5TVtGtxg3bQpmKLv++/DTiSS1GI5c88HrnX3NkAXYKSZtS3ayMyqA3cDr8c3okjESSfBlCnw8stw991hpxFJalGLu7tvcfe8yOtdwBqgeTFNrwIWAJ/FNaFIYSNHBkMTTJgAb74ZdhqRpFWma+5m1grIApYUWd4cuBB4MF7BRIplFkzNd+yxQZH/73/DTiSSlGIu7mZWl+DMfLS7f1lk9VRgrLvvi7KP4WaWa2a5W7duLXtaEYC6dYMBxvbsCUaQ3Ls37EQiSSem4m5mGQSFfba7P1dMkxxgrpltBHoDvzOzC4o2cveZ7p7j7jmZmZkViC1VXps2Qb/3t9+GsWPDTiOSdGpEa2BmBjwKrHH3KcW1cffWhdo/Drzs7hrxSSpX375Bcb//fjjllKCbpIgAMRR3oCswCFhpZssiy8YDLQHcXdfZJTz33gv/+AcMGQIdOgTX4kUkenF398WAxbpDdx9ckUAiZVKzZtD/PSsrGGBsyRKoUyfsVCKh0xOqkvpatIBnnoH334cRIzTAmAgq7pIufvpTuOUWePppeOihsNOIhE7FXdLHjTfC2WfDqFHBdXiRKkzFXdJHtWrBmfvhhwc9Z7ZvDzuRSGhU3CW9NG4M8+fDJ5/AoEEaYEyqLBV3ST85OTBtGrz6Ktx+e9hpREKh4i7pafjw4Mz9ppvgz38OO41Iwqm4S3oygwcfhHbt4JJL4OOPw04kklAq7pK+atcOBhj77jvo0yf4W6SKUHGX9PbjH8OsWcGTq9ddF3YakYRRcZf016sXjBkD06fD3LlhpxFJCBV3qRruugtOPRWGDQuGKRBJcyruUjVkZMCzzwaDivXuDbt3h51IpFKpuEvV8cMfBpdl1q6FK67QAGOS1lTcpWrp0SN4sGnuXHjggbDTiFSS33ImAAAIqElEQVQaFXepev7f/4PzzoNrr4V33w07jUilUHGXqqdaNXjiiWAc+D59QJO1SxpScZeqqVGj4AGnrVthwADYty/sRCJxpeIuVVdWFsyYAW+8EUz0IZJGVNylahs6NJhc+9Zbg1EkRdKEirvIjBnQqRMMHAgbN4adRiQuohZ3MzvCzBaa2RozW21mo4ppc76ZrTCzZWaWa2anVk5ckUpw6KHBBB/79gU3WL/9NuxEIhVWI4Y2+cC17p5nZvWApWb2hrsXfob7r8BL7u5m1hGYBxxXCXnZs3cP2/ZsK/jasP+9NitxWeHlxS1Lpu0T/V4CHHNM0IPmggtg9OjgbL6oosdMx1CSWNTi7u5bgC2R17vMbA3QHHi/UJvCz3LXASrt0b85K+cw7I/DKmv3VVoYP+gS+V4xbT+xDt/veZCvb3yQQ/dC9Sif5FjKu8X4vyGmfcW2q/+950Eb2MFt4vWe5VpS/ve0Ur46YE2Jx6KYNsWujOGYFdp3tOOab86x3phZ1/6NpkccW3rjCojlzL2AmbUCsoAlxay7ELgTOAw4Nw7ZirX+8/UA9Gnbh58d/bOC5R75eeKFHin3Qj9j9i8vblkybZ9KWdPy3/r99yx/ewHv1drBcd/Vp2P9YyiWF95LCYptU/xWB7QsYcelt/GSVxXKU4bW/1sbwzANB/1Li9kkhiNWzDEr6XgduE3JuUoNFPy1vxpHiVdifo+hTSFr8z/l5YZbmfXUGK4f/6eo7csr5uJuZnWBBcBod/+y6Hp3fx543sxOA24FzihmH8OB4QAtW7YsV+BrT7mWxoc2ZkjWEJrWblqufYiUZlv29cx6agxDRk+p1DMrqZq2fbw2+HwNmlKp72Mew09lM8sAXgZed/eoicxsA3CCu28rqU1OTo7n5uaWJauISJVnZkvdPSdau1h6yxjwKLCmpMJuZsdE2mFm2UBNYHvZIouISLzEclmmKzAIWGlmyyLLxgMtAdz9QaAXcKmZ7QW+Bvp6LL8SiIhIpYilt8xioty8dve7gbvjFUpERCpGT6iKiKQhFXcRkTSk4i4ikoZU3EVE0lBM/dwr5Y3NtgIflXPzpkCJfehDlKy5IHmzKVfZKFfZpGOuI909M1qj0Ip7RZhZbiyd+BMtWXNB8mZTrrJRrrKpyrl0WUZEJA2puIuIpKFULe4zww5QgmTNBcmbTbnKRrnKpsrmSslr7iIiUrpUPXMXEZFSJF1xN7OzzGytma03s3HFrD/EzJ6NrF8SmUBk/7obIsvXmtnPim5bybnGmNn7kblk/2pmRxZaty8yv+wyM3spwbkGm9nWQu8/rNC6y8xsXeTPZQnOdX+hTP82sx2F1lXm8XrMzD4zs1UlrDczmxbJvSIyyun+dZV5vKLlGhDJs8LM3jGzToXWbTSzlZHjFddxtGPI1d3Mdhb6fk0stK7Uz0Al57q+UKZVkc9U48i6SjleFtt804n7fLl70vwBqgMfAEcRDBu8HGhbpM2vgAcjr/sBz0Zet420PwRoHdlP9QTm6gHUjry+cn+uyNe7Qzxeg4EHitm2MfBh5O9GkdeNEpWrSPurgMcq+3hF9n0akA2sKmH9OcCrBIPldQGWVPbxijHXKfvfDzh7f67I1xuBpiEdr+7AyxX9DMQ7V5G25wFvVvbxAg4HsiOv6wH/Lub/Y8I+X8l25n4isN7dP3T374C5wPlF2pwPPBF5PR843cwssnyuu3/r7huA9ZH9JSSXuy909z2RL98FWsTpvSuUqxQ/A95w98/d/QvgDeCskHL1B+bE6b1L5e6LgM9LaXI+8KQH3gUamtnhVO7xiprL3d+JvC8k7vMVy/EqSUU+m/HOlZDPl7tvcfe8yOtdwP75pgtL2Ocr2Yp7c+DjQl9v4uCDU9DG3fOBnUCTGLetzFyFDSX46bxfLTPLNbN3zeyCOGUqS65ekV8B55vZEWXctjJzEbl81Rp4s9DiyjpesSgpe2Uer7Iq+vly4M9mttSCqSwT7WQzW25mr5pZu8iypDheZlaboEguKLS40o+XlTzfdMI+X2WaIDsBihs3vmh3npLaxLJtecW8bzMbCOQA3Qotbunum83sKOBNM1vp7h8kKNcfgTnu/q2ZjSD4radnjNtWZq79+gHz3X1foWWVdbxiEcbnK2Zm1oOguJ9aaHHXyPE6DHjDzP4VObNNhDyCx+F3m9k5wAvAj0iS40VwSeZtdy98ll+px8tKn286YZ+vZDtz3wQcUejrFsDmktqYWQ2gAcGvZ7FsW5m5MLMzgBuBX7j7t/uXu/vmyN8fAv9H8BM9IbncfXuhLA8Dx8e6bWXmKqQfRX5lrsTjFYuSslfm8YqJmXUEHgHOd/eCaSwLHa/PgOeJ3+XIqNz9S3ffHXn9CpBhZk1JguMVUdrnK+7Hy4L5phcAs939uWKaJO7zFe+bChW8IVGD4EZCa/53E6ZdkTYjOfCG6rzI63YceEP1Q+J3QzWWXFkEN5B+VGR5I+CQyOumwDridGMpxlyHF3p9IfCu/+8GzoZIvkaR140TlSvS7liCm1uWiONV6D1aUfINwnM58IbXe5V9vGLM1ZLgPtIpRZbXAeoVev0OcFYCczXb//0jKJL/iRy7mD4DlZUrsn7/iV+dRByvyL/7SWBqKW0S9vmK24GO4zfsHIK7zB8AN0aWTSI4GwaoBfwh8kF/Dziq0LY3RrZbC5yd4Fx/AT4FlkX+vBRZfgqwMvLhXgkMTXCuO4HVkfdfCBxXaNvLI8dxPTAkkbkiX98M3FVku8o+XnOALcBegrOlocAIYERkvQEzIrlXAjkJOl7Rcj0CfFHo85UbWX5U5Fgtj3yfb0xwrl8X+ny9S6EfPsV9BhKVK9JmMEEni8LbVdrxIrhU5sCKQt+nc8L6fOkJVRGRNJRs19xFRCQOVNxFRNKQiruISBpScRcRSUMq7iIiaUjFXUQkDam4i4ikIRV3EZE09P8Bd+e8LCgAEk8AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] @@ -1432,16 +1260,47 @@ } ], "source": [ - "nn.fit_epoch(X_train,y_train,X_test,y_test,lr=0.01,epochs=3,bs=512,l2=0)" + "nn.fit_epoch(X_train,y_train,X_test,y_test,lr=0.2,epochs=3,bs=2048,l2=0)" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array(0)" + ] + }, + "execution_count": 43, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "np.where(np.NaN >= 0, 1, 0)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "There is a problem with the code: X_input will grow really fast (exponentially?) even though weights are still between 0 and 1 (probably due to large hidden layers, i.e >1000 or **multiple hidden layers**, i.e 2 hidden layers 50->20 and matrix multiplication is essentially adding up as many numbers as the weights' shape) which makes grad_wrt_input grows fast (grad_wrt_input needs grad_relu(X_input) which will return X_input itself if it's >=0) -> grad_w grow fast -> weight update fast -> X_input grow fast ...\n", + "A deep 'plain' neural net is harder to train due to vanishing gradients/exploding gradients. In this case we have **exploding gradient problem**. Why? As the depth of the neural net model grows, there will be more layer to backpropagate. When calculating loss gradient w.r.t weight, at one point we will do matrix multiplication between 'loss grad w.r.t some input Xi' and 'input Xi activation' to calculate loss grad w.r.t weight. E.g We are using ReLU so the code will look like this:\n", + "```\n", + "grad_wrt_weight = relu(Xi).T @ grad_wrt_input\n", + "```\n", + "relu(Xi) will return itself if it's positive, thus grad_wrt_weight will get bigger extremly fast as we add more hidden layer, resulting in weights getting NaN when it's updated and we end up with the error above." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To solve this, we can increase dropout probability to discard more node in hidden layers + try another activation function other than ReLU\n", "\n", - "## Solution: dropout + pick another activation function" + "## Solution: dropout + pick another activation function (Tanh)" ] }, { @@ -1643,7 +1502,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Evaluate neural net" + "Model is trained without any error. Let's evaluate it!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Evaluate deep neural net" ] }, { @@ -1716,6 +1582,15 @@ "plt.ylabel('True label')\n", "plt.xlabel('Predicted label')" ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "84% is good, but it still doesn't achieve the 87% accuracy from Pytorch version. Let's add some optimizers!\n", + "\n", + "## Part 2 is [here](https://github.com/anhquan0412/basic_model_scratch/blob/master/neural_net_optimizers.ipynb) where I add few optimizers such as momentum, RMSProp and Adam" + ] } ], "metadata": { @@ -1734,7 +1609,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.0" + "version": "3.6.6" }, "varInspector": { "cols": {