Madelon-Example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Binary Classification of Madelon"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Download datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x27784d16e10>"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch\n",
    "torch.manual_seed(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Madelon dataset X: (2600, 500), y: (2600,)\n"
     ]
    }
   ],
   "source": [
    "import openml as oml\n",
    "import pandas as pd\n",
    "madelon_dataset = oml.datasets.get_dataset(1485)\n",
    "X, y, categorical_indicator, attribute_names = madelon_dataset.get_data(\n",
    "target=madelon_dataset.default_target_attribute, dataset_format='dataframe')\n",
    "print(f'Madelon dataset X: {X.shape}, y: {y.shape}')\n",
    "X_np = X.to_numpy()\n",
    "X_np = X_np/999\n",
    "y_np = y.factorize()[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Start with 1 server and 1 clients."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [],
   "source": [
    "client_id_list = [0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The __Courier__ class is in charge of transmiting message between clients and server. Since we'are doing local simulation, a synchronized local courier is enough"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [],
   "source": [
    "from Courier import SyncLocalCourier\n",
    "courier = SyncLocalCourier(client_id_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also need a dataloader. The __DataLoader__ class distributes the data to each clients based on thier id.\n",
    "Since the data source is numpy arrays, we use __SimpleNumpyDataLoader__ here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [],
   "source": [
    "from VFLDataUtils import SimpleNumpyDataLoader\n",
    "loader = SimpleNumpyDataLoader(\n",
    "    clients_id_list=client_id_list,\n",
    "    data_source=(X_np, y_np), dim_split=[250]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__dim_split__ here is a list contains the split points.\n",
    "The length of __dim_split__ should be the same as the number of clients in clients list. Features remained is used as server's feature"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Client 0: Feature Index 0-250\n",
      "Server : Feature Index 250-500\n"
     ]
    }
   ],
   "source": [
    "loader_dict = loader.distribute()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "loader.distribute() will return a dictionary whose keys are clients'id, and valuese are the corresponding train, test loaders."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{0: {'train_loader': <torch.utils.data.dataloader.DataLoader object at 0x000002778158CE50>, 'test_loader': <torch.utils.data.dataloader.DataLoader object at 0x00000277815AA7F0>}, 'server': {'train_loader': <torch.utils.data.dataloader.DataLoader object at 0x00000277815AA4C0>, 'test_loader': <torch.utils.data.dataloader.DataLoader object at 0x00000277815AA850>}}\n"
     ]
    }
   ],
   "source": [
    "print(loader_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we create client's neural models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch import nn\n",
    "client_model = nn.Sequential(\n",
    "    nn.Linear(250, 64),\n",
    "    nn.ReLU(),\n",
    "    nn.Linear(64, 32),\n",
    "    nn.ReLU()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Declare a Synchronized local clients. simple_config.ini records hyper parameters "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [],
   "source": [
    "from Client import SyncFNNClient\n",
    "client = SyncFNNClient(id=0, \n",
    "    model=client_model,\n",
    "    courier=courier,\n",
    "    train_loader=loader_dict[0]['train_loader'],\n",
    "    test_loader=loader_dict[0]['test_loader'],\n",
    "    config_dir='simple_config.ini'\n",
    ")\n",
    "clients = [client]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Strategy is used for server to control clients' behaviour"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [],
   "source": [
    "from Strategy import SyncConcatStrategy\n",
    "strategy = SyncConcatStrategy(\n",
    "    courier=courier,\n",
    "    clients=clients\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we create models of server. Server should have two models, one is to generate embedding from a partition of the features and the other one is to combine all embedding from clients "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [],
   "source": [
    "server_emb_model = nn.Sequential(\n",
    "    nn.Linear(250, 64),\n",
    "    nn.ReLU(),\n",
    "    nn.Linear(64, 32),\n",
    "    nn.ReLU()\n",
    ")\n",
    "server_top_model = nn.Sequential(\n",
    "    nn.Linear(2*32, 32),\n",
    "    nn.ReLU(True),\n",
    "    nn.Linear(32, 1),\n",
    "    nn.Sigmoid()\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [],
   "source": [
    "from Server import SyncFNNServer\n",
    "server = SyncFNNServer(\n",
    "    strategy=strategy,\n",
    "    courier=courier,\n",
    "    top_model=server_top_model,\n",
    "    emb_model=server_emb_model,\n",
    "    train_loader=loader_dict['server']['train_loader'],\n",
    "    test_loader=loader_dict['server']['test_loader'],\n",
    "    config_dir='simple_config.ini'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Starts\n",
      "-----------------------------------------------------------------------------------------\n",
      "Epoch 000: | Loss: 0.69329 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 001: | Loss: 0.69317 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 002: | Loss: 0.69315 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 003: | Loss: 0.69315 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 004: | Loss: 0.69314 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 005: | Loss: 0.69314 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 006: | Loss: 0.69313 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 007: | Loss: 0.69313 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 008: | Loss: 0.69312 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 009: | Loss: 0.69312 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 010: | Loss: 0.69311 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 011: | Loss: 0.69310 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 012: | Loss: 0.69310 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 013: | Loss: 0.69309 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 014: | Loss: 0.69308 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 015: | Loss: 0.69307 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 016: | Loss: 0.69307 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 017: | Loss: 0.69304 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 018: | Loss: 0.69301 | Acc: 0.500 | Val ACC: 0.504\n",
      "Epoch 019: | Loss: 0.69297 | Acc: 0.500 | Val ACC: 0.504\n",
      "Epoch 020: | Loss: 0.69291 | Acc: 0.512 | Val ACC: 0.521\n",
      "Epoch 021: | Loss: 0.69285 | Acc: 0.515 | Val ACC: 0.600\n",
      "Epoch 022: | Loss: 0.69276 | Acc: 0.530 | Val ACC: 0.606\n",
      "Epoch 023: | Loss: 0.69265 | Acc: 0.522 | Val ACC: 0.613\n",
      "Epoch 024: | Loss: 0.69256 | Acc: 0.522 | Val ACC: 0.615\n",
      "Epoch 025: | Loss: 0.69235 | Acc: 0.534 | Val ACC: 0.610\n",
      "Epoch 026: | Loss: 0.69207 | Acc: 0.537 | Val ACC: 0.612\n",
      "Epoch 027: | Loss: 0.69173 | Acc: 0.544 | Val ACC: 0.602\n",
      "Epoch 028: | Loss: 0.69130 | Acc: 0.541 | Val ACC: 0.617\n",
      "Epoch 029: | Loss: 0.69056 | Acc: 0.541 | Val ACC: 0.560\n",
      "Epoch 030: | Loss: 0.68949 | Acc: 0.561 | Val ACC: 0.508\n",
      "Epoch 031: | Loss: 0.68878 | Acc: 0.541 | Val ACC: 0.500\n",
      "Epoch 032: | Loss: 0.68864 | Acc: 0.543 | Val ACC: 0.571\n",
      "Epoch 033: | Loss: 0.68639 | Acc: 0.565 | Val ACC: 0.506\n",
      "Epoch 034: | Loss: 0.68731 | Acc: 0.561 | Val ACC: 0.579\n",
      "Epoch 035: | Loss: 0.68430 | Acc: 0.565 | Val ACC: 0.504\n",
      "Epoch 036: | Loss: 0.68674 | Acc: 0.563 | Val ACC: 0.573\n",
      "Epoch 037: | Loss: 0.68374 | Acc: 0.564 | Val ACC: 0.521\n",
      "Epoch 038: | Loss: 0.68115 | Acc: 0.586 | Val ACC: 0.515\n",
      "Epoch 039: | Loss: 0.68098 | Acc: 0.577 | Val ACC: 0.494\n",
      "Epoch 040: | Loss: 0.68202 | Acc: 0.566 | Val ACC: 0.563\n",
      "Epoch 041: | Loss: 0.67743 | Acc: 0.591 | Val ACC: 0.519\n",
      "Epoch 042: | Loss: 0.67871 | Acc: 0.582 | Val ACC: 0.510\n",
      "Epoch 043: | Loss: 0.67925 | Acc: 0.578 | Val ACC: 0.537\n",
      "Epoch 044: | Loss: 0.67587 | Acc: 0.590 | Val ACC: 0.533\n",
      "Epoch 045: | Loss: 0.67692 | Acc: 0.586 | Val ACC: 0.535\n",
      "Epoch 046: | Loss: 0.67576 | Acc: 0.590 | Val ACC: 0.552\n",
      "Epoch 047: | Loss: 0.67494 | Acc: 0.589 | Val ACC: 0.533\n",
      "Epoch 048: | Loss: 0.67527 | Acc: 0.589 | Val ACC: 0.544\n",
      "Epoch 049: | Loss: 0.67399 | Acc: 0.592 | Val ACC: 0.550\n",
      "Epoch 050: | Loss: 0.67359 | Acc: 0.594 | Val ACC: 0.529\n",
      "Epoch 051: | Loss: 0.67409 | Acc: 0.595 | Val ACC: 0.575\n",
      "Epoch 052: | Loss: 0.67157 | Acc: 0.602 | Val ACC: 0.558\n",
      "Epoch 053: | Loss: 0.67254 | Acc: 0.597 | Val ACC: 0.538\n",
      "Epoch 054: | Loss: 0.67278 | Acc: 0.598 | Val ACC: 0.583\n",
      "Epoch 055: | Loss: 0.67041 | Acc: 0.605 | Val ACC: 0.544\n",
      "Epoch 056: | Loss: 0.67184 | Acc: 0.600 | Val ACC: 0.581\n",
      "Epoch 057: | Loss: 0.67138 | Acc: 0.601 | Val ACC: 0.571\n",
      "Epoch 058: | Loss: 0.67058 | Acc: 0.602 | Val ACC: 0.554\n",
      "Epoch 059: | Loss: 0.67062 | Acc: 0.604 | Val ACC: 0.583\n",
      "Epoch 060: | Loss: 0.66951 | Acc: 0.604 | Val ACC: 0.556\n",
      "Epoch 061: | Loss: 0.66978 | Acc: 0.607 | Val ACC: 0.581\n",
      "Epoch 062: | Loss: 0.66872 | Acc: 0.611 | Val ACC: 0.577\n",
      "Epoch 063: | Loss: 0.66932 | Acc: 0.611 | Val ACC: 0.583\n",
      "Epoch 064: | Loss: 0.66797 | Acc: 0.613 | Val ACC: 0.581\n",
      "Epoch 065: | Loss: 0.66906 | Acc: 0.608 | Val ACC: 0.583\n",
      "Epoch 066: | Loss: 0.66747 | Acc: 0.613 | Val ACC: 0.587\n",
      "Epoch 067: | Loss: 0.66807 | Acc: 0.612 | Val ACC: 0.585\n",
      "Epoch 068: | Loss: 0.66767 | Acc: 0.614 | Val ACC: 0.587\n",
      "Epoch 069: | Loss: 0.66714 | Acc: 0.615 | Val ACC: 0.577\n",
      "Epoch 070: | Loss: 0.66674 | Acc: 0.616 | Val ACC: 0.581\n",
      "Epoch 071: | Loss: 0.66633 | Acc: 0.618 | Val ACC: 0.588\n",
      "Epoch 072: | Loss: 0.66602 | Acc: 0.615 | Val ACC: 0.588\n",
      "Epoch 073: | Loss: 0.66691 | Acc: 0.610 | Val ACC: 0.571\n",
      "Epoch 074: | Loss: 0.66579 | Acc: 0.615 | Val ACC: 0.617\n",
      "Epoch 075: | Loss: 0.66656 | Acc: 0.603 | Val ACC: 0.565\n",
      "Epoch 076: | Loss: 0.66558 | Acc: 0.611 | Val ACC: 0.623\n",
      "Epoch 077: | Loss: 0.66813 | Acc: 0.603 | Val ACC: 0.550\n",
      "Epoch 078: | Loss: 0.66783 | Acc: 0.606 | Val ACC: 0.627\n",
      "Epoch 079: | Loss: 0.66998 | Acc: 0.600 | Val ACC: 0.540\n",
      "Epoch 080: | Loss: 0.66822 | Acc: 0.602 | Val ACC: 0.627\n",
      "Epoch 081: | Loss: 0.67262 | Acc: 0.593 | Val ACC: 0.556\n",
      "Epoch 082: | Loss: 0.66434 | Acc: 0.608 | Val ACC: 0.629\n",
      "Epoch 083: | Loss: 0.66851 | Acc: 0.602 | Val ACC: 0.554\n",
      "Epoch 084: | Loss: 0.66478 | Acc: 0.611 | Val ACC: 0.629\n",
      "Epoch 085: | Loss: 0.66887 | Acc: 0.600 | Val ACC: 0.554\n",
      "Epoch 086: | Loss: 0.66374 | Acc: 0.612 | Val ACC: 0.627\n",
      "Epoch 087: | Loss: 0.67063 | Acc: 0.597 | Val ACC: 0.558\n",
      "Epoch 088: | Loss: 0.66207 | Acc: 0.610 | Val ACC: 0.631\n",
      "Epoch 089: | Loss: 0.66735 | Acc: 0.604 | Val ACC: 0.571\n",
      "Epoch 090: | Loss: 0.66068 | Acc: 0.614 | Val ACC: 0.638\n",
      "Epoch 091: | Loss: 0.66654 | Acc: 0.604 | Val ACC: 0.579\n",
      "Epoch 092: | Loss: 0.65944 | Acc: 0.617 | Val ACC: 0.640\n",
      "Epoch 093: | Loss: 0.66478 | Acc: 0.602 | Val ACC: 0.581\n",
      "Epoch 094: | Loss: 0.65842 | Acc: 0.619 | Val ACC: 0.638\n",
      "Epoch 095: | Loss: 0.66251 | Acc: 0.602 | Val ACC: 0.588\n",
      "Epoch 096: | Loss: 0.65778 | Acc: 0.614 | Val ACC: 0.640\n",
      "Epoch 097: | Loss: 0.66204 | Acc: 0.600 | Val ACC: 0.585\n",
      "Epoch 098: | Loss: 0.65718 | Acc: 0.615 | Val ACC: 0.646\n",
      "Epoch 099: | Loss: 0.66021 | Acc: 0.601 | Val ACC: 0.598\n",
      "Epoch 100: | Loss: 0.65605 | Acc: 0.609 | Val ACC: 0.638\n",
      "Epoch 101: | Loss: 0.65878 | Acc: 0.602 | Val ACC: 0.612\n",
      "Epoch 102: | Loss: 0.65566 | Acc: 0.608 | Val ACC: 0.629\n",
      "Epoch 103: | Loss: 0.65636 | Acc: 0.607 | Val ACC: 0.625\n",
      "Epoch 104: | Loss: 0.65665 | Acc: 0.604 | Val ACC: 0.623\n",
      "Epoch 105: | Loss: 0.65604 | Acc: 0.608 | Val ACC: 0.627\n",
      "Epoch 106: | Loss: 0.65475 | Acc: 0.609 | Val ACC: 0.638\n",
      "Epoch 107: | Loss: 0.65467 | Acc: 0.606 | Val ACC: 0.633\n",
      "Epoch 108: | Loss: 0.65392 | Acc: 0.610 | Val ACC: 0.640\n",
      "Epoch 109: | Loss: 0.65364 | Acc: 0.608 | Val ACC: 0.640\n",
      "Epoch 110: | Loss: 0.65367 | Acc: 0.606 | Val ACC: 0.642\n",
      "Epoch 111: | Loss: 0.65159 | Acc: 0.609 | Val ACC: 0.646\n",
      "Epoch 112: | Loss: 0.65133 | Acc: 0.610 | Val ACC: 0.646\n",
      "Epoch 113: | Loss: 0.65139 | Acc: 0.610 | Val ACC: 0.644\n",
      "Epoch 114: | Loss: 0.65057 | Acc: 0.609 | Val ACC: 0.640\n",
      "Epoch 115: | Loss: 0.64891 | Acc: 0.611 | Val ACC: 0.642\n",
      "Epoch 116: | Loss: 0.64739 | Acc: 0.615 | Val ACC: 0.650\n",
      "Epoch 117: | Loss: 0.64894 | Acc: 0.614 | Val ACC: 0.642\n",
      "Epoch 118: | Loss: 0.64670 | Acc: 0.615 | Val ACC: 0.646\n",
      "Epoch 119: | Loss: 0.64452 | Acc: 0.615 | Val ACC: 0.654\n",
      "Epoch 120: | Loss: 0.64623 | Acc: 0.617 | Val ACC: 0.648\n",
      "Epoch 121: | Loss: 0.64954 | Acc: 0.602 | Val ACC: 0.640\n",
      "Epoch 122: | Loss: 0.64828 | Acc: 0.612 | Val ACC: 0.648\n",
      "Epoch 123: | Loss: 0.64304 | Acc: 0.624 | Val ACC: 0.644\n",
      "Epoch 124: | Loss: 0.64301 | Acc: 0.626 | Val ACC: 0.642\n",
      "Epoch 125: | Loss: 0.64169 | Acc: 0.625 | Val ACC: 0.635\n",
      "Epoch 126: | Loss: 0.64163 | Acc: 0.624 | Val ACC: 0.637\n",
      "Epoch 127: | Loss: 0.64366 | Acc: 0.620 | Val ACC: 0.621\n",
      "Epoch 128: | Loss: 0.64256 | Acc: 0.623 | Val ACC: 0.642\n",
      "Epoch 129: | Loss: 0.64561 | Acc: 0.625 | Val ACC: 0.629\n",
      "Epoch 130: | Loss: 0.64189 | Acc: 0.636 | Val ACC: 0.648\n",
      "Epoch 131: | Loss: 0.63677 | Acc: 0.642 | Val ACC: 0.646\n",
      "Epoch 132: | Loss: 0.63688 | Acc: 0.640 | Val ACC: 0.658\n",
      "Epoch 133: | Loss: 0.63541 | Acc: 0.637 | Val ACC: 0.644\n",
      "Epoch 134: | Loss: 0.63824 | Acc: 0.634 | Val ACC: 0.654\n",
      "Epoch 135: | Loss: 0.64539 | Acc: 0.630 | Val ACC: 0.621\n",
      "Epoch 136: | Loss: 0.63827 | Acc: 0.641 | Val ACC: 0.633\n",
      "Epoch 137: | Loss: 0.63420 | Acc: 0.642 | Val ACC: 0.654\n",
      "Epoch 138: | Loss: 0.63749 | Acc: 0.642 | Val ACC: 0.644\n",
      "Epoch 139: | Loss: 0.63848 | Acc: 0.641 | Val ACC: 0.644\n",
      "Epoch 140: | Loss: 0.63450 | Acc: 0.643 | Val ACC: 0.646\n",
      "Epoch 141: | Loss: 0.63114 | Acc: 0.644 | Val ACC: 0.646\n",
      "Epoch 142: | Loss: 0.62981 | Acc: 0.648 | Val ACC: 0.650\n",
      "Epoch 143: | Loss: 0.63120 | Acc: 0.636 | Val ACC: 0.640\n",
      "Epoch 144: | Loss: 0.63140 | Acc: 0.643 | Val ACC: 0.623\n",
      "Epoch 145: | Loss: 0.63120 | Acc: 0.638 | Val ACC: 0.629\n",
      "Epoch 146: | Loss: 0.62945 | Acc: 0.643 | Val ACC: 0.621\n",
      "Epoch 147: | Loss: 0.62908 | Acc: 0.642 | Val ACC: 0.621\n",
      "Epoch 148: | Loss: 0.62826 | Acc: 0.647 | Val ACC: 0.638\n",
      "Epoch 149: | Loss: 0.62818 | Acc: 0.640 | Val ACC: 0.627\n",
      "Epoch 150: | Loss: 0.62991 | Acc: 0.644 | Val ACC: 0.619\n",
      "Epoch 151: | Loss: 0.62885 | Acc: 0.639 | Val ACC: 0.619\n",
      "Epoch 152: | Loss: 0.62735 | Acc: 0.648 | Val ACC: 0.625\n",
      "Epoch 153: | Loss: 0.62693 | Acc: 0.642 | Val ACC: 0.623\n",
      "Epoch 154: | Loss: 0.62610 | Acc: 0.647 | Val ACC: 0.627\n",
      "Epoch 155: | Loss: 0.62747 | Acc: 0.641 | Val ACC: 0.613\n",
      "Epoch 156: | Loss: 0.62704 | Acc: 0.648 | Val ACC: 0.617\n",
      "Epoch 157: | Loss: 0.62744 | Acc: 0.642 | Val ACC: 0.610\n",
      "Epoch 158: | Loss: 0.62612 | Acc: 0.647 | Val ACC: 0.612\n",
      "Epoch 159: | Loss: 0.62632 | Acc: 0.644 | Val ACC: 0.602\n",
      "Epoch 160: | Loss: 0.62667 | Acc: 0.644 | Val ACC: 0.606\n",
      "Epoch 161: | Loss: 0.62619 | Acc: 0.642 | Val ACC: 0.621\n",
      "Epoch 162: | Loss: 0.62490 | Acc: 0.652 | Val ACC: 0.615\n",
      "Epoch 163: | Loss: 0.62737 | Acc: 0.644 | Val ACC: 0.608\n",
      "Epoch 164: | Loss: 0.62774 | Acc: 0.648 | Val ACC: 0.613\n",
      "Epoch 165: | Loss: 0.62747 | Acc: 0.645 | Val ACC: 0.623\n",
      "Epoch 166: | Loss: 0.62691 | Acc: 0.653 | Val ACC: 0.617\n",
      "Epoch 167: | Loss: 0.62714 | Acc: 0.646 | Val ACC: 0.629\n",
      "Epoch 168: | Loss: 0.62665 | Acc: 0.651 | Val ACC: 0.637\n",
      "Epoch 169: | Loss: 0.62672 | Acc: 0.649 | Val ACC: 0.635\n",
      "Epoch 170: | Loss: 0.62446 | Acc: 0.653 | Val ACC: 0.652\n",
      "Epoch 171: | Loss: 0.62147 | Acc: 0.658 | Val ACC: 0.667\n",
      "Epoch 172: | Loss: 0.62073 | Acc: 0.658 | Val ACC: 0.658\n",
      "Epoch 173: | Loss: 0.62113 | Acc: 0.656 | Val ACC: 0.660\n",
      "Epoch 174: | Loss: 0.62111 | Acc: 0.654 | Val ACC: 0.638\n",
      "Epoch 175: | Loss: 0.62037 | Acc: 0.657 | Val ACC: 0.648\n",
      "Epoch 176: | Loss: 0.62048 | Acc: 0.654 | Val ACC: 0.644\n",
      "Epoch 177: | Loss: 0.61978 | Acc: 0.658 | Val ACC: 0.648\n",
      "Epoch 178: | Loss: 0.62113 | Acc: 0.654 | Val ACC: 0.640\n",
      "Epoch 179: | Loss: 0.62120 | Acc: 0.655 | Val ACC: 0.638\n",
      "Epoch 180: | Loss: 0.61981 | Acc: 0.662 | Val ACC: 0.658\n",
      "Epoch 181: | Loss: 0.61875 | Acc: 0.659 | Val ACC: 0.650\n",
      "Epoch 182: | Loss: 0.61902 | Acc: 0.658 | Val ACC: 0.646\n",
      "Epoch 183: | Loss: 0.61834 | Acc: 0.660 | Val ACC: 0.652\n",
      "Epoch 184: | Loss: 0.61829 | Acc: 0.661 | Val ACC: 0.656\n",
      "Epoch 185: | Loss: 0.61789 | Acc: 0.663 | Val ACC: 0.658\n",
      "Epoch 186: | Loss: 0.61818 | Acc: 0.666 | Val ACC: 0.656\n",
      "Epoch 187: | Loss: 0.61788 | Acc: 0.665 | Val ACC: 0.658\n",
      "Epoch 188: | Loss: 0.61798 | Acc: 0.666 | Val ACC: 0.656\n",
      "Epoch 189: | Loss: 0.61757 | Acc: 0.665 | Val ACC: 0.658\n",
      "Epoch 190: | Loss: 0.61767 | Acc: 0.664 | Val ACC: 0.658\n",
      "Epoch 191: | Loss: 0.61745 | Acc: 0.665 | Val ACC: 0.658\n",
      "Epoch 192: | Loss: 0.61777 | Acc: 0.664 | Val ACC: 0.654\n",
      "Epoch 193: | Loss: 0.61755 | Acc: 0.666 | Val ACC: 0.660\n",
      "Epoch 194: | Loss: 0.61784 | Acc: 0.664 | Val ACC: 0.667\n",
      "Epoch 195: | Loss: 0.61721 | Acc: 0.667 | Val ACC: 0.667\n",
      "Epoch 196: | Loss: 0.61720 | Acc: 0.670 | Val ACC: 0.669\n",
      "Epoch 197: | Loss: 0.61656 | Acc: 0.670 | Val ACC: 0.663\n",
      "Epoch 198: | Loss: 0.61573 | Acc: 0.670 | Val ACC: 0.663\n",
      "Epoch 199: | Loss: 0.61515 | Acc: 0.672 | Val ACC: 0.667\n",
      "Epoch 200: | Loss: 0.61509 | Acc: 0.671 | Val ACC: 0.667\n",
      "Epoch 201: | Loss: 0.61449 | Acc: 0.674 | Val ACC: 0.662\n",
      "Epoch 202: | Loss: 0.61366 | Acc: 0.668 | Val ACC: 0.665\n",
      "Epoch 203: | Loss: 0.61336 | Acc: 0.670 | Val ACC: 0.669\n",
      "Epoch 204: | Loss: 0.61295 | Acc: 0.670 | Val ACC: 0.667\n",
      "Epoch 205: | Loss: 0.61116 | Acc: 0.672 | Val ACC: 0.663\n",
      "Epoch 206: | Loss: 0.61197 | Acc: 0.669 | Val ACC: 0.669\n",
      "Epoch 207: | Loss: 0.61285 | Acc: 0.667 | Val ACC: 0.665\n",
      "Epoch 208: | Loss: 0.61321 | Acc: 0.664 | Val ACC: 0.663\n",
      "Epoch 209: | Loss: 0.61348 | Acc: 0.665 | Val ACC: 0.652\n",
      "Epoch 210: | Loss: 0.61213 | Acc: 0.666 | Val ACC: 0.654\n",
      "Epoch 211: | Loss: 0.61179 | Acc: 0.666 | Val ACC: 0.658\n",
      "Epoch 212: | Loss: 0.61135 | Acc: 0.663 | Val ACC: 0.646\n",
      "Epoch 213: | Loss: 0.61054 | Acc: 0.669 | Val ACC: 0.658\n",
      "Epoch 214: | Loss: 0.61099 | Acc: 0.664 | Val ACC: 0.640\n",
      "Epoch 215: | Loss: 0.61025 | Acc: 0.666 | Val ACC: 0.654\n",
      "Epoch 216: | Loss: 0.61049 | Acc: 0.665 | Val ACC: 0.637\n",
      "Epoch 217: | Loss: 0.60899 | Acc: 0.670 | Val ACC: 0.646\n",
      "Epoch 218: | Loss: 0.61013 | Acc: 0.666 | Val ACC: 0.627\n",
      "Epoch 219: | Loss: 0.60957 | Acc: 0.672 | Val ACC: 0.637\n",
      "Epoch 220: | Loss: 0.60963 | Acc: 0.667 | Val ACC: 0.623\n",
      "Epoch 221: | Loss: 0.60902 | Acc: 0.672 | Val ACC: 0.625\n",
      "Epoch 222: | Loss: 0.60877 | Acc: 0.668 | Val ACC: 0.633\n",
      "Epoch 223: | Loss: 0.60756 | Acc: 0.672 | Val ACC: 0.638\n",
      "Epoch 224: | Loss: 0.60794 | Acc: 0.670 | Val ACC: 0.633\n",
      "Epoch 225: | Loss: 0.60808 | Acc: 0.671 | Val ACC: 0.623\n",
      "Epoch 226: | Loss: 0.60821 | Acc: 0.671 | Val ACC: 0.613\n",
      "Epoch 227: | Loss: 0.60845 | Acc: 0.671 | Val ACC: 0.617\n",
      "Epoch 228: | Loss: 0.60818 | Acc: 0.672 | Val ACC: 0.619\n",
      "Epoch 229: | Loss: 0.60965 | Acc: 0.668 | Val ACC: 0.610\n",
      "Epoch 230: | Loss: 0.60921 | Acc: 0.670 | Val ACC: 0.604\n",
      "Epoch 231: | Loss: 0.60945 | Acc: 0.669 | Val ACC: 0.608\n",
      "Epoch 232: | Loss: 0.61065 | Acc: 0.670 | Val ACC: 0.604\n",
      "Epoch 233: | Loss: 0.60699 | Acc: 0.669 | Val ACC: 0.615\n",
      "Epoch 234: | Loss: 0.60721 | Acc: 0.674 | Val ACC: 0.613\n",
      "Epoch 235: | Loss: 0.60562 | Acc: 0.670 | Val ACC: 0.615\n",
      "Epoch 236: | Loss: 0.60579 | Acc: 0.673 | Val ACC: 0.615\n",
      "Epoch 237: | Loss: 0.60485 | Acc: 0.673 | Val ACC: 0.613\n",
      "Epoch 238: | Loss: 0.60604 | Acc: 0.674 | Val ACC: 0.608\n",
      "Epoch 239: | Loss: 0.60528 | Acc: 0.672 | Val ACC: 0.617\n",
      "Epoch 240: | Loss: 0.60569 | Acc: 0.676 | Val ACC: 0.613\n",
      "Epoch 241: | Loss: 0.60493 | Acc: 0.675 | Val ACC: 0.608\n",
      "Epoch 242: | Loss: 0.60498 | Acc: 0.677 | Val ACC: 0.612\n",
      "Epoch 243: | Loss: 0.60538 | Acc: 0.677 | Val ACC: 0.623\n",
      "Epoch 244: | Loss: 0.60269 | Acc: 0.681 | Val ACC: 0.625\n",
      "Epoch 245: | Loss: 0.60308 | Acc: 0.682 | Val ACC: 0.619\n",
      "Epoch 246: | Loss: 0.60291 | Acc: 0.681 | Val ACC: 0.625\n",
      "Epoch 247: | Loss: 0.60165 | Acc: 0.683 | Val ACC: 0.631\n",
      "Epoch 248: | Loss: 0.60113 | Acc: 0.681 | Val ACC: 0.615\n",
      "Epoch 249: | Loss: 0.60301 | Acc: 0.680 | Val ACC: 0.619\n",
      "Epoch 250: | Loss: 0.60138 | Acc: 0.682 | Val ACC: 0.625\n",
      "Epoch 251: | Loss: 0.60081 | Acc: 0.682 | Val ACC: 0.627\n",
      "Epoch 252: | Loss: 0.60113 | Acc: 0.684 | Val ACC: 0.619\n",
      "Epoch 253: | Loss: 0.60092 | Acc: 0.681 | Val ACC: 0.623\n",
      "Epoch 254: | Loss: 0.60012 | Acc: 0.683 | Val ACC: 0.619\n",
      "Epoch 255: | Loss: 0.60041 | Acc: 0.682 | Val ACC: 0.617\n",
      "Epoch 256: | Loss: 0.60232 | Acc: 0.679 | Val ACC: 0.615\n",
      "Epoch 257: | Loss: 0.60171 | Acc: 0.684 | Val ACC: 0.633\n",
      "Epoch 258: | Loss: 0.59878 | Acc: 0.686 | Val ACC: 0.627\n",
      "Epoch 259: | Loss: 0.59898 | Acc: 0.684 | Val ACC: 0.629\n",
      "Epoch 260: | Loss: 0.59880 | Acc: 0.685 | Val ACC: 0.625\n",
      "Epoch 261: | Loss: 0.59570 | Acc: 0.685 | Val ACC: 0.638\n",
      "Epoch 262: | Loss: 0.59624 | Acc: 0.684 | Val ACC: 0.625\n",
      "Epoch 263: | Loss: 0.59733 | Acc: 0.686 | Val ACC: 0.631\n",
      "Epoch 264: | Loss: 0.59787 | Acc: 0.683 | Val ACC: 0.625\n",
      "Epoch 265: | Loss: 0.60084 | Acc: 0.681 | Val ACC: 0.627\n",
      "Epoch 266: | Loss: 0.59735 | Acc: 0.686 | Val ACC: 0.635\n",
      "Epoch 267: | Loss: 0.59650 | Acc: 0.689 | Val ACC: 0.619\n",
      "Epoch 268: | Loss: 0.59593 | Acc: 0.687 | Val ACC: 0.623\n",
      "Epoch 269: | Loss: 0.59647 | Acc: 0.690 | Val ACC: 0.631\n",
      "Epoch 270: | Loss: 0.59545 | Acc: 0.690 | Val ACC: 0.635\n",
      "Epoch 271: | Loss: 0.59414 | Acc: 0.692 | Val ACC: 0.629\n",
      "Epoch 272: | Loss: 0.59357 | Acc: 0.689 | Val ACC: 0.640\n",
      "Epoch 273: | Loss: 0.59236 | Acc: 0.690 | Val ACC: 0.633\n",
      "Epoch 274: | Loss: 0.59140 | Acc: 0.693 | Val ACC: 0.629\n",
      "Epoch 275: | Loss: 0.59204 | Acc: 0.689 | Val ACC: 0.625\n",
      "Epoch 276: | Loss: 0.59554 | Acc: 0.687 | Val ACC: 0.615\n",
      "Epoch 277: | Loss: 0.59443 | Acc: 0.692 | Val ACC: 0.625\n",
      "Epoch 278: | Loss: 0.59418 | Acc: 0.686 | Val ACC: 0.623\n",
      "Epoch 279: | Loss: 0.59530 | Acc: 0.686 | Val ACC: 0.625\n",
      "Epoch 280: | Loss: 0.59413 | Acc: 0.689 | Val ACC: 0.629\n",
      "Epoch 281: | Loss: 0.59450 | Acc: 0.688 | Val ACC: 0.629\n",
      "Epoch 282: | Loss: 0.59378 | Acc: 0.690 | Val ACC: 0.638\n",
      "Epoch 283: | Loss: 0.59184 | Acc: 0.694 | Val ACC: 0.633\n",
      "Epoch 284: | Loss: 0.59194 | Acc: 0.693 | Val ACC: 0.637\n",
      "Epoch 285: | Loss: 0.59009 | Acc: 0.694 | Val ACC: 0.644\n",
      "Epoch 286: | Loss: 0.58877 | Acc: 0.699 | Val ACC: 0.640\n",
      "Epoch 287: | Loss: 0.58923 | Acc: 0.698 | Val ACC: 0.638\n",
      "Epoch 288: | Loss: 0.58927 | Acc: 0.694 | Val ACC: 0.637\n",
      "Epoch 289: | Loss: 0.58873 | Acc: 0.698 | Val ACC: 0.635\n",
      "Epoch 290: | Loss: 0.58914 | Acc: 0.698 | Val ACC: 0.637\n",
      "Epoch 291: | Loss: 0.58748 | Acc: 0.699 | Val ACC: 0.631\n",
      "Epoch 292: | Loss: 0.58737 | Acc: 0.697 | Val ACC: 0.642\n",
      "Epoch 293: | Loss: 0.58718 | Acc: 0.702 | Val ACC: 0.638\n",
      "Epoch 294: | Loss: 0.58909 | Acc: 0.696 | Val ACC: 0.638\n",
      "Epoch 295: | Loss: 0.58933 | Acc: 0.694 | Val ACC: 0.631\n",
      "Epoch 296: | Loss: 0.58727 | Acc: 0.696 | Val ACC: 0.629\n",
      "Epoch 297: | Loss: 0.58630 | Acc: 0.701 | Val ACC: 0.635\n",
      "Epoch 298: | Loss: 0.58796 | Acc: 0.699 | Val ACC: 0.635\n",
      "Epoch 299: | Loss: 0.58758 | Acc: 0.698 | Val ACC: 0.631\n",
      "Epoch 300: | Loss: 0.58803 | Acc: 0.699 | Val ACC: 0.635\n",
      "Epoch 301: | Loss: 0.58788 | Acc: 0.699 | Val ACC: 0.635\n",
      "Epoch 302: | Loss: 0.58750 | Acc: 0.699 | Val ACC: 0.629\n",
      "Epoch 303: | Loss: 0.58770 | Acc: 0.696 | Val ACC: 0.635\n",
      "Epoch 304: | Loss: 0.58676 | Acc: 0.697 | Val ACC: 0.644\n",
      "Epoch 305: | Loss: 0.58129 | Acc: 0.705 | Val ACC: 0.638\n",
      "Epoch 306: | Loss: 0.58264 | Acc: 0.707 | Val ACC: 0.637\n",
      "Epoch 307: | Loss: 0.58204 | Acc: 0.702 | Val ACC: 0.642\n",
      "Epoch 308: | Loss: 0.58460 | Acc: 0.701 | Val ACC: 0.637\n",
      "Epoch 309: | Loss: 0.58419 | Acc: 0.700 | Val ACC: 0.640\n",
      "Epoch 310: | Loss: 0.58508 | Acc: 0.700 | Val ACC: 0.644\n",
      "Epoch 311: | Loss: 0.58363 | Acc: 0.698 | Val ACC: 0.644\n",
      "Epoch 312: | Loss: 0.58453 | Acc: 0.702 | Val ACC: 0.642\n",
      "Epoch 313: | Loss: 0.58521 | Acc: 0.701 | Val ACC: 0.637\n",
      "Epoch 314: | Loss: 0.58307 | Acc: 0.702 | Val ACC: 0.646\n",
      "Epoch 315: | Loss: 0.58455 | Acc: 0.707 | Val ACC: 0.638\n",
      "Epoch 316: | Loss: 0.57928 | Acc: 0.705 | Val ACC: 0.650\n",
      "Epoch 317: | Loss: 0.58187 | Acc: 0.707 | Val ACC: 0.644\n",
      "Epoch 318: | Loss: 0.58050 | Acc: 0.703 | Val ACC: 0.646\n",
      "Epoch 319: | Loss: 0.58124 | Acc: 0.704 | Val ACC: 0.644\n",
      "Epoch 320: | Loss: 0.58125 | Acc: 0.699 | Val ACC: 0.650\n",
      "Epoch 321: | Loss: 0.58288 | Acc: 0.705 | Val ACC: 0.642\n",
      "Epoch 322: | Loss: 0.58085 | Acc: 0.701 | Val ACC: 0.642\n",
      "Epoch 323: | Loss: 0.58291 | Acc: 0.703 | Val ACC: 0.644\n",
      "Epoch 324: | Loss: 0.58035 | Acc: 0.702 | Val ACC: 0.644\n",
      "Epoch 325: | Loss: 0.58115 | Acc: 0.708 | Val ACC: 0.642\n",
      "Epoch 326: | Loss: 0.57951 | Acc: 0.701 | Val ACC: 0.642\n",
      "Epoch 327: | Loss: 0.58202 | Acc: 0.707 | Val ACC: 0.646\n",
      "Epoch 328: | Loss: 0.58092 | Acc: 0.702 | Val ACC: 0.638\n",
      "Epoch 329: | Loss: 0.58266 | Acc: 0.704 | Val ACC: 0.640\n",
      "Epoch 330: | Loss: 0.57973 | Acc: 0.705 | Val ACC: 0.648\n",
      "Epoch 331: | Loss: 0.57954 | Acc: 0.705 | Val ACC: 0.652\n",
      "Epoch 332: | Loss: 0.57764 | Acc: 0.706 | Val ACC: 0.642\n",
      "Epoch 333: | Loss: 0.57862 | Acc: 0.704 | Val ACC: 0.638\n",
      "Epoch 334: | Loss: 0.57914 | Acc: 0.708 | Val ACC: 0.644\n",
      "Epoch 335: | Loss: 0.57902 | Acc: 0.705 | Val ACC: 0.638\n",
      "Epoch 336: | Loss: 0.57638 | Acc: 0.710 | Val ACC: 0.646\n",
      "Epoch 337: | Loss: 0.57718 | Acc: 0.710 | Val ACC: 0.644\n",
      "Epoch 338: | Loss: 0.57598 | Acc: 0.708 | Val ACC: 0.646\n",
      "Epoch 339: | Loss: 0.57494 | Acc: 0.710 | Val ACC: 0.644\n",
      "Epoch 340: | Loss: 0.57656 | Acc: 0.714 | Val ACC: 0.648\n",
      "Epoch 341: | Loss: 0.57407 | Acc: 0.709 | Val ACC: 0.648\n",
      "Epoch 342: | Loss: 0.57483 | Acc: 0.708 | Val ACC: 0.638\n",
      "Epoch 343: | Loss: 0.57647 | Acc: 0.710 | Val ACC: 0.644\n",
      "Epoch 344: | Loss: 0.57351 | Acc: 0.711 | Val ACC: 0.644\n",
      "Epoch 345: | Loss: 0.57549 | Acc: 0.708 | Val ACC: 0.654\n",
      "Epoch 346: | Loss: 0.57498 | Acc: 0.708 | Val ACC: 0.648\n",
      "Epoch 347: | Loss: 0.57439 | Acc: 0.710 | Val ACC: 0.644\n",
      "Epoch 348: | Loss: 0.57320 | Acc: 0.715 | Val ACC: 0.648\n",
      "Epoch 349: | Loss: 0.57332 | Acc: 0.710 | Val ACC: 0.652\n",
      "Epoch 350: | Loss: 0.57120 | Acc: 0.716 | Val ACC: 0.652\n",
      "Epoch 351: | Loss: 0.57331 | Acc: 0.712 | Val ACC: 0.652\n",
      "Epoch 352: | Loss: 0.57147 | Acc: 0.711 | Val ACC: 0.638\n",
      "Epoch 353: | Loss: 0.57451 | Acc: 0.713 | Val ACC: 0.633\n",
      "Epoch 354: | Loss: 0.57375 | Acc: 0.710 | Val ACC: 0.648\n",
      "Epoch 355: | Loss: 0.57173 | Acc: 0.716 | Val ACC: 0.635\n",
      "Epoch 356: | Loss: 0.57286 | Acc: 0.711 | Val ACC: 0.635\n",
      "Epoch 357: | Loss: 0.57376 | Acc: 0.711 | Val ACC: 0.648\n",
      "Epoch 358: | Loss: 0.57178 | Acc: 0.709 | Val ACC: 0.627\n",
      "Epoch 359: | Loss: 0.57124 | Acc: 0.709 | Val ACC: 0.648\n",
      "Epoch 360: | Loss: 0.57123 | Acc: 0.711 | Val ACC: 0.648\n",
      "Epoch 361: | Loss: 0.57524 | Acc: 0.706 | Val ACC: 0.637\n",
      "Epoch 362: | Loss: 0.57085 | Acc: 0.713 | Val ACC: 0.637\n",
      "Epoch 363: | Loss: 0.57086 | Acc: 0.709 | Val ACC: 0.656\n",
      "Epoch 364: | Loss: 0.56957 | Acc: 0.711 | Val ACC: 0.640\n",
      "Epoch 365: | Loss: 0.56709 | Acc: 0.710 | Val ACC: 0.642\n",
      "Epoch 366: | Loss: 0.56933 | Acc: 0.716 | Val ACC: 0.662\n",
      "Epoch 367: | Loss: 0.56632 | Acc: 0.718 | Val ACC: 0.642\n",
      "Epoch 368: | Loss: 0.56635 | Acc: 0.712 | Val ACC: 0.656\n",
      "Epoch 369: | Loss: 0.56657 | Acc: 0.716 | Val ACC: 0.669\n",
      "Epoch 370: | Loss: 0.56092 | Acc: 0.719 | Val ACC: 0.656\n",
      "Epoch 371: | Loss: 0.56503 | Acc: 0.723 | Val ACC: 0.648\n",
      "Epoch 372: | Loss: 0.56454 | Acc: 0.722 | Val ACC: 0.658\n",
      "Epoch 373: | Loss: 0.56554 | Acc: 0.716 | Val ACC: 0.650\n",
      "Epoch 374: | Loss: 0.56434 | Acc: 0.722 | Val ACC: 0.656\n",
      "Epoch 375: | Loss: 0.56320 | Acc: 0.722 | Val ACC: 0.656\n",
      "Epoch 376: | Loss: 0.56421 | Acc: 0.722 | Val ACC: 0.648\n",
      "Epoch 377: | Loss: 0.56453 | Acc: 0.720 | Val ACC: 0.658\n",
      "Epoch 378: | Loss: 0.56478 | Acc: 0.720 | Val ACC: 0.648\n",
      "Epoch 379: | Loss: 0.56368 | Acc: 0.719 | Val ACC: 0.660\n",
      "Epoch 380: | Loss: 0.56213 | Acc: 0.724 | Val ACC: 0.658\n",
      "Epoch 381: | Loss: 0.56252 | Acc: 0.720 | Val ACC: 0.654\n",
      "Epoch 382: | Loss: 0.56249 | Acc: 0.719 | Val ACC: 0.656\n",
      "Epoch 383: | Loss: 0.55975 | Acc: 0.723 | Val ACC: 0.648\n",
      "Epoch 384: | Loss: 0.55878 | Acc: 0.723 | Val ACC: 0.660\n",
      "Epoch 385: | Loss: 0.55870 | Acc: 0.724 | Val ACC: 0.654\n",
      "Epoch 386: | Loss: 0.55994 | Acc: 0.720 | Val ACC: 0.652\n",
      "Epoch 387: | Loss: 0.55939 | Acc: 0.723 | Val ACC: 0.656\n",
      "Epoch 388: | Loss: 0.55807 | Acc: 0.722 | Val ACC: 0.648\n",
      "Epoch 389: | Loss: 0.55862 | Acc: 0.724 | Val ACC: 0.646\n",
      "Epoch 390: | Loss: 0.55952 | Acc: 0.720 | Val ACC: 0.663\n",
      "Epoch 391: | Loss: 0.55512 | Acc: 0.722 | Val ACC: 0.660\n",
      "Epoch 392: | Loss: 0.55387 | Acc: 0.726 | Val ACC: 0.648\n",
      "Epoch 393: | Loss: 0.55393 | Acc: 0.723 | Val ACC: 0.650\n",
      "Epoch 394: | Loss: 0.55343 | Acc: 0.727 | Val ACC: 0.648\n",
      "Epoch 395: | Loss: 0.55498 | Acc: 0.726 | Val ACC: 0.644\n",
      "Epoch 396: | Loss: 0.56037 | Acc: 0.721 | Val ACC: 0.660\n",
      "Epoch 397: | Loss: 0.55713 | Acc: 0.721 | Val ACC: 0.656\n",
      "Epoch 398: | Loss: 0.55519 | Acc: 0.723 | Val ACC: 0.658\n",
      "Epoch 399: | Loss: 0.55678 | Acc: 0.722 | Val ACC: 0.648\n",
      "Epoch 400: | Loss: 0.55489 | Acc: 0.722 | Val ACC: 0.652\n",
      "Epoch 401: | Loss: 0.55611 | Acc: 0.722 | Val ACC: 0.648\n",
      "Epoch 402: | Loss: 0.55805 | Acc: 0.723 | Val ACC: 0.652\n",
      "Epoch 403: | Loss: 0.55610 | Acc: 0.724 | Val ACC: 0.660\n",
      "Epoch 404: | Loss: 0.55672 | Acc: 0.721 | Val ACC: 0.648\n",
      "Epoch 405: | Loss: 0.55930 | Acc: 0.722 | Val ACC: 0.650\n",
      "Epoch 406: | Loss: 0.55407 | Acc: 0.724 | Val ACC: 0.660\n",
      "Epoch 407: | Loss: 0.55497 | Acc: 0.723 | Val ACC: 0.658\n",
      "Epoch 408: | Loss: 0.55854 | Acc: 0.721 | Val ACC: 0.650\n",
      "Epoch 409: | Loss: 0.55734 | Acc: 0.726 | Val ACC: 0.654\n",
      "Epoch 410: | Loss: 0.55367 | Acc: 0.726 | Val ACC: 0.646\n",
      "Epoch 411: | Loss: 0.55581 | Acc: 0.721 | Val ACC: 0.654\n",
      "Epoch 412: | Loss: 0.55625 | Acc: 0.723 | Val ACC: 0.652\n",
      "Epoch 413: | Loss: 0.55322 | Acc: 0.725 | Val ACC: 0.648\n",
      "Epoch 414: | Loss: 0.55818 | Acc: 0.712 | Val ACC: 0.627\n",
      "Epoch 415: | Loss: 0.56193 | Acc: 0.712 | Val ACC: 0.633\n",
      "Epoch 416: | Loss: 0.55977 | Acc: 0.717 | Val ACC: 0.642\n",
      "Epoch 417: | Loss: 0.55734 | Acc: 0.720 | Val ACC: 0.627\n",
      "Epoch 418: | Loss: 0.56487 | Acc: 0.710 | Val ACC: 0.608\n",
      "Epoch 419: | Loss: 0.56602 | Acc: 0.710 | Val ACC: 0.612\n",
      "Epoch 420: | Loss: 0.56601 | Acc: 0.710 | Val ACC: 0.606\n",
      "Epoch 421: | Loss: 0.56426 | Acc: 0.714 | Val ACC: 0.627\n",
      "Epoch 422: | Loss: 0.56084 | Acc: 0.720 | Val ACC: 0.660\n",
      "Epoch 423: | Loss: 0.55742 | Acc: 0.723 | Val ACC: 0.660\n",
      "Epoch 424: | Loss: 0.55504 | Acc: 0.725 | Val ACC: 0.648\n",
      "Epoch 425: | Loss: 0.55489 | Acc: 0.727 | Val ACC: 0.658\n",
      "Epoch 426: | Loss: 0.55128 | Acc: 0.729 | Val ACC: 0.660\n",
      "Epoch 427: | Loss: 0.54736 | Acc: 0.737 | Val ACC: 0.662\n",
      "Epoch 428: | Loss: 0.55000 | Acc: 0.727 | Val ACC: 0.662\n",
      "Epoch 429: | Loss: 0.55086 | Acc: 0.732 | Val ACC: 0.669\n",
      "Epoch 430: | Loss: 0.54847 | Acc: 0.732 | Val ACC: 0.660\n",
      "Epoch 431: | Loss: 0.54805 | Acc: 0.739 | Val ACC: 0.656\n",
      "Epoch 432: | Loss: 0.54458 | Acc: 0.739 | Val ACC: 0.654\n",
      "Epoch 433: | Loss: 0.54244 | Acc: 0.738 | Val ACC: 0.658\n",
      "Epoch 434: | Loss: 0.54290 | Acc: 0.737 | Val ACC: 0.660\n",
      "Epoch 435: | Loss: 0.54389 | Acc: 0.739 | Val ACC: 0.654\n",
      "Epoch 436: | Loss: 0.54459 | Acc: 0.739 | Val ACC: 0.652\n",
      "Epoch 437: | Loss: 0.54445 | Acc: 0.737 | Val ACC: 0.663\n",
      "Epoch 438: | Loss: 0.54210 | Acc: 0.742 | Val ACC: 0.658\n",
      "Epoch 439: | Loss: 0.54027 | Acc: 0.744 | Val ACC: 0.662\n",
      "Epoch 440: | Loss: 0.54139 | Acc: 0.736 | Val ACC: 0.658\n",
      "Epoch 441: | Loss: 0.54078 | Acc: 0.739 | Val ACC: 0.656\n",
      "Epoch 442: | Loss: 0.54216 | Acc: 0.737 | Val ACC: 0.660\n",
      "Epoch 443: | Loss: 0.54868 | Acc: 0.733 | Val ACC: 0.663\n",
      "Epoch 444: | Loss: 0.55373 | Acc: 0.725 | Val ACC: 0.644\n",
      "Epoch 445: | Loss: 0.56268 | Acc: 0.711 | Val ACC: 0.627\n",
      "Epoch 446: | Loss: 0.55239 | Acc: 0.723 | Val ACC: 0.638\n",
      "Epoch 447: | Loss: 0.55345 | Acc: 0.725 | Val ACC: 0.646\n",
      "Epoch 448: | Loss: 0.54628 | Acc: 0.731 | Val ACC: 0.656\n",
      "Epoch 449: | Loss: 0.54642 | Acc: 0.730 | Val ACC: 0.675\n",
      "Epoch 450: | Loss: 0.53413 | Acc: 0.742 | Val ACC: 0.671\n",
      "Epoch 451: | Loss: 0.53832 | Acc: 0.738 | Val ACC: 0.667\n",
      "Epoch 452: | Loss: 0.54303 | Acc: 0.730 | Val ACC: 0.669\n",
      "Epoch 453: | Loss: 0.54649 | Acc: 0.730 | Val ACC: 0.660\n",
      "Epoch 454: | Loss: 0.53917 | Acc: 0.736 | Val ACC: 0.675\n",
      "Epoch 455: | Loss: 0.53815 | Acc: 0.738 | Val ACC: 0.669\n",
      "Epoch 456: | Loss: 0.53970 | Acc: 0.736 | Val ACC: 0.673\n",
      "Epoch 457: | Loss: 0.53483 | Acc: 0.740 | Val ACC: 0.667\n",
      "Epoch 458: | Loss: 0.53918 | Acc: 0.734 | Val ACC: 0.669\n",
      "Epoch 459: | Loss: 0.53923 | Acc: 0.735 | Val ACC: 0.665\n",
      "Epoch 460: | Loss: 0.53781 | Acc: 0.736 | Val ACC: 0.671\n",
      "Epoch 461: | Loss: 0.53031 | Acc: 0.742 | Val ACC: 0.662\n",
      "Epoch 462: | Loss: 0.53773 | Acc: 0.738 | Val ACC: 0.669\n",
      "Epoch 463: | Loss: 0.53648 | Acc: 0.744 | Val ACC: 0.669\n",
      "Epoch 464: | Loss: 0.53446 | Acc: 0.740 | Val ACC: 0.665\n",
      "Epoch 465: | Loss: 0.53701 | Acc: 0.738 | Val ACC: 0.673\n",
      "Epoch 466: | Loss: 0.54053 | Acc: 0.734 | Val ACC: 0.681\n",
      "Epoch 467: | Loss: 0.54364 | Acc: 0.725 | Val ACC: 0.665\n",
      "Epoch 468: | Loss: 0.54291 | Acc: 0.731 | Val ACC: 0.652\n",
      "Epoch 469: | Loss: 0.54927 | Acc: 0.727 | Val ACC: 0.671\n",
      "Epoch 470: | Loss: 0.54330 | Acc: 0.730 | Val ACC: 0.656\n",
      "Epoch 471: | Loss: 0.54930 | Acc: 0.725 | Val ACC: 0.658\n",
      "Epoch 472: | Loss: 0.54549 | Acc: 0.734 | Val ACC: 0.638\n",
      "Epoch 473: | Loss: 0.54587 | Acc: 0.730 | Val ACC: 0.662\n",
      "Epoch 474: | Loss: 0.53751 | Acc: 0.738 | Val ACC: 0.654\n",
      "Epoch 475: | Loss: 0.54141 | Acc: 0.727 | Val ACC: 0.671\n",
      "Epoch 476: | Loss: 0.53179 | Acc: 0.738 | Val ACC: 0.671\n",
      "Epoch 477: | Loss: 0.53396 | Acc: 0.741 | Val ACC: 0.663\n",
      "Epoch 478: | Loss: 0.53894 | Acc: 0.729 | Val ACC: 0.671\n",
      "Epoch 479: | Loss: 0.53280 | Acc: 0.738 | Val ACC: 0.673\n",
      "Epoch 480: | Loss: 0.53036 | Acc: 0.743 | Val ACC: 0.663\n",
      "Epoch 481: | Loss: 0.53242 | Acc: 0.743 | Val ACC: 0.660\n",
      "Epoch 482: | Loss: 0.53493 | Acc: 0.738 | Val ACC: 0.660\n",
      "Epoch 483: | Loss: 0.53690 | Acc: 0.736 | Val ACC: 0.663\n",
      "Epoch 484: | Loss: 0.53083 | Acc: 0.740 | Val ACC: 0.677\n",
      "Epoch 485: | Loss: 0.52918 | Acc: 0.743 | Val ACC: 0.671\n",
      "Epoch 486: | Loss: 0.52811 | Acc: 0.740 | Val ACC: 0.671\n",
      "Epoch 487: | Loss: 0.53043 | Acc: 0.742 | Val ACC: 0.669\n",
      "Epoch 488: | Loss: 0.53282 | Acc: 0.733 | Val ACC: 0.667\n",
      "Epoch 489: | Loss: 0.53195 | Acc: 0.740 | Val ACC: 0.662\n",
      "Epoch 490: | Loss: 0.53314 | Acc: 0.734 | Val ACC: 0.656\n",
      "Epoch 491: | Loss: 0.53144 | Acc: 0.734 | Val ACC: 0.667\n",
      "Epoch 492: | Loss: 0.53158 | Acc: 0.738 | Val ACC: 0.673\n",
      "Epoch 493: | Loss: 0.53113 | Acc: 0.739 | Val ACC: 0.673\n",
      "Epoch 494: | Loss: 0.52878 | Acc: 0.736 | Val ACC: 0.673\n",
      "Epoch 495: | Loss: 0.53007 | Acc: 0.741 | Val ACC: 0.662\n",
      "Epoch 496: | Loss: 0.53390 | Acc: 0.733 | Val ACC: 0.669\n",
      "Epoch 497: | Loss: 0.53687 | Acc: 0.732 | Val ACC: 0.669\n",
      "Epoch 498: | Loss: 0.53392 | Acc: 0.735 | Val ACC: 0.662\n",
      "Epoch 499: | Loss: 0.53236 | Acc: 0.740 | Val ACC: 0.669\n",
      "Epoch 500: | Loss: 0.53277 | Acc: 0.736 | Val ACC: 0.671\n",
      "Epoch 501: | Loss: 0.53202 | Acc: 0.742 | Val ACC: 0.663\n",
      "Epoch 502: | Loss: 0.52990 | Acc: 0.744 | Val ACC: 0.660\n",
      "Epoch 503: | Loss: 0.53101 | Acc: 0.743 | Val ACC: 0.673\n",
      "Epoch 504: | Loss: 0.53131 | Acc: 0.742 | Val ACC: 0.675\n",
      "Epoch 505: | Loss: 0.52679 | Acc: 0.748 | Val ACC: 0.679\n",
      "Epoch 506: | Loss: 0.52897 | Acc: 0.744 | Val ACC: 0.663\n",
      "Epoch 507: | Loss: 0.52940 | Acc: 0.747 | Val ACC: 0.669\n",
      "Epoch 508: | Loss: 0.53021 | Acc: 0.748 | Val ACC: 0.673\n",
      "Epoch 509: | Loss: 0.52624 | Acc: 0.750 | Val ACC: 0.679\n",
      "Epoch 510: | Loss: 0.52421 | Acc: 0.749 | Val ACC: 0.679\n",
      "Epoch 511: | Loss: 0.52213 | Acc: 0.751 | Val ACC: 0.669\n",
      "Epoch 512: | Loss: 0.52327 | Acc: 0.750 | Val ACC: 0.675\n",
      "Epoch 513: | Loss: 0.52317 | Acc: 0.749 | Val ACC: 0.677\n",
      "Epoch 514: | Loss: 0.52293 | Acc: 0.748 | Val ACC: 0.683\n",
      "Epoch 515: | Loss: 0.52166 | Acc: 0.748 | Val ACC: 0.677\n",
      "Epoch 516: | Loss: 0.52069 | Acc: 0.753 | Val ACC: 0.669\n",
      "Epoch 517: | Loss: 0.52149 | Acc: 0.754 | Val ACC: 0.671\n",
      "Epoch 518: | Loss: 0.51886 | Acc: 0.752 | Val ACC: 0.662\n",
      "Epoch 519: | Loss: 0.52009 | Acc: 0.747 | Val ACC: 0.665\n",
      "Epoch 520: | Loss: 0.51601 | Acc: 0.750 | Val ACC: 0.669\n",
      "Epoch 521: | Loss: 0.51385 | Acc: 0.751 | Val ACC: 0.677\n",
      "Epoch 522: | Loss: 0.51498 | Acc: 0.748 | Val ACC: 0.675\n",
      "Epoch 523: | Loss: 0.51349 | Acc: 0.749 | Val ACC: 0.677\n",
      "Epoch 524: | Loss: 0.51310 | Acc: 0.750 | Val ACC: 0.673\n",
      "Epoch 525: | Loss: 0.51317 | Acc: 0.751 | Val ACC: 0.663\n",
      "Epoch 526: | Loss: 0.51361 | Acc: 0.756 | Val ACC: 0.667\n",
      "Epoch 527: | Loss: 0.51273 | Acc: 0.751 | Val ACC: 0.663\n",
      "Epoch 528: | Loss: 0.51200 | Acc: 0.750 | Val ACC: 0.665\n",
      "Epoch 529: | Loss: 0.51102 | Acc: 0.751 | Val ACC: 0.669\n",
      "Epoch 530: | Loss: 0.50931 | Acc: 0.753 | Val ACC: 0.669\n",
      "Epoch 531: | Loss: 0.51313 | Acc: 0.754 | Val ACC: 0.660\n",
      "Epoch 532: | Loss: 0.51120 | Acc: 0.751 | Val ACC: 0.662\n",
      "Epoch 533: | Loss: 0.51455 | Acc: 0.747 | Val ACC: 0.656\n",
      "Epoch 534: | Loss: 0.51626 | Acc: 0.746 | Val ACC: 0.654\n",
      "Epoch 535: | Loss: 0.51792 | Acc: 0.744 | Val ACC: 0.654\n",
      "Epoch 536: | Loss: 0.51568 | Acc: 0.748 | Val ACC: 0.654\n",
      "Epoch 537: | Loss: 0.51656 | Acc: 0.749 | Val ACC: 0.658\n",
      "Epoch 538: | Loss: 0.50898 | Acc: 0.751 | Val ACC: 0.662\n",
      "Epoch 539: | Loss: 0.51099 | Acc: 0.751 | Val ACC: 0.662\n",
      "Epoch 540: | Loss: 0.51401 | Acc: 0.749 | Val ACC: 0.658\n",
      "Epoch 541: | Loss: 0.51215 | Acc: 0.750 | Val ACC: 0.648\n",
      "Epoch 542: | Loss: 0.51742 | Acc: 0.745 | Val ACC: 0.656\n",
      "Epoch 543: | Loss: 0.51503 | Acc: 0.748 | Val ACC: 0.648\n",
      "Epoch 544: | Loss: 0.51298 | Acc: 0.745 | Val ACC: 0.652\n",
      "Epoch 545: | Loss: 0.51456 | Acc: 0.745 | Val ACC: 0.656\n",
      "Epoch 546: | Loss: 0.51668 | Acc: 0.748 | Val ACC: 0.660\n",
      "Epoch 547: | Loss: 0.51675 | Acc: 0.747 | Val ACC: 0.660\n",
      "Epoch 548: | Loss: 0.51486 | Acc: 0.745 | Val ACC: 0.660\n",
      "Epoch 549: | Loss: 0.51366 | Acc: 0.746 | Val ACC: 0.660\n",
      "Epoch 550: | Loss: 0.51147 | Acc: 0.752 | Val ACC: 0.663\n",
      "Epoch 551: | Loss: 0.50393 | Acc: 0.755 | Val ACC: 0.662\n",
      "Epoch 552: | Loss: 0.50408 | Acc: 0.754 | Val ACC: 0.665\n",
      "Epoch 553: | Loss: 0.50382 | Acc: 0.748 | Val ACC: 0.662\n",
      "Epoch 554: | Loss: 0.50859 | Acc: 0.750 | Val ACC: 0.665\n",
      "Epoch 555: | Loss: 0.50726 | Acc: 0.750 | Val ACC: 0.663\n",
      "Epoch 556: | Loss: 0.50337 | Acc: 0.753 | Val ACC: 0.663\n",
      "Epoch 557: | Loss: 0.50129 | Acc: 0.755 | Val ACC: 0.665\n",
      "Epoch 558: | Loss: 0.49994 | Acc: 0.755 | Val ACC: 0.663\n",
      "Epoch 559: | Loss: 0.50421 | Acc: 0.753 | Val ACC: 0.660\n",
      "Epoch 560: | Loss: 0.50711 | Acc: 0.752 | Val ACC: 0.660\n",
      "Epoch 561: | Loss: 0.50567 | Acc: 0.752 | Val ACC: 0.660\n",
      "Epoch 562: | Loss: 0.50406 | Acc: 0.753 | Val ACC: 0.660\n",
      "Epoch 563: | Loss: 0.50250 | Acc: 0.756 | Val ACC: 0.658\n",
      "Epoch 564: | Loss: 0.50144 | Acc: 0.754 | Val ACC: 0.658\n",
      "Epoch 565: | Loss: 0.49792 | Acc: 0.759 | Val ACC: 0.660\n",
      "Epoch 566: | Loss: 0.49620 | Acc: 0.762 | Val ACC: 0.654\n",
      "Epoch 567: | Loss: 0.49583 | Acc: 0.760 | Val ACC: 0.662\n",
      "Epoch 568: | Loss: 0.49840 | Acc: 0.757 | Val ACC: 0.662\n",
      "Epoch 569: | Loss: 0.50121 | Acc: 0.750 | Val ACC: 0.656\n",
      "Epoch 570: | Loss: 0.49897 | Acc: 0.753 | Val ACC: 0.662\n",
      "Epoch 571: | Loss: 0.49603 | Acc: 0.758 | Val ACC: 0.662\n",
      "Epoch 572: | Loss: 0.49380 | Acc: 0.756 | Val ACC: 0.665\n",
      "Epoch 573: | Loss: 0.49252 | Acc: 0.760 | Val ACC: 0.656\n",
      "Epoch 574: | Loss: 0.49477 | Acc: 0.760 | Val ACC: 0.658\n",
      "Epoch 575: | Loss: 0.50536 | Acc: 0.751 | Val ACC: 0.648\n",
      "Epoch 576: | Loss: 0.50032 | Acc: 0.752 | Val ACC: 0.662\n",
      "Epoch 577: | Loss: 0.49574 | Acc: 0.756 | Val ACC: 0.656\n",
      "Epoch 578: | Loss: 0.49370 | Acc: 0.756 | Val ACC: 0.656\n",
      "Epoch 579: | Loss: 0.49500 | Acc: 0.756 | Val ACC: 0.656\n",
      "Epoch 580: | Loss: 0.49944 | Acc: 0.753 | Val ACC: 0.662\n",
      "Epoch 581: | Loss: 0.49700 | Acc: 0.755 | Val ACC: 0.650\n",
      "Epoch 582: | Loss: 0.49518 | Acc: 0.756 | Val ACC: 0.662\n",
      "Epoch 583: | Loss: 0.49498 | Acc: 0.756 | Val ACC: 0.662\n",
      "Epoch 584: | Loss: 0.49527 | Acc: 0.754 | Val ACC: 0.656\n",
      "Epoch 585: | Loss: 0.49397 | Acc: 0.756 | Val ACC: 0.660\n",
      "Epoch 586: | Loss: 0.49958 | Acc: 0.759 | Val ACC: 0.650\n",
      "Epoch 587: | Loss: 0.49944 | Acc: 0.758 | Val ACC: 0.663\n",
      "Epoch 588: | Loss: 0.49896 | Acc: 0.755 | Val ACC: 0.665\n",
      "Epoch 589: | Loss: 0.49854 | Acc: 0.754 | Val ACC: 0.660\n",
      "Epoch 590: | Loss: 0.49656 | Acc: 0.756 | Val ACC: 0.665\n",
      "Epoch 591: | Loss: 0.49740 | Acc: 0.758 | Val ACC: 0.662\n",
      "Epoch 592: | Loss: 0.49548 | Acc: 0.756 | Val ACC: 0.665\n",
      "Epoch 593: | Loss: 0.49614 | Acc: 0.756 | Val ACC: 0.663\n",
      "Epoch 594: | Loss: 0.49595 | Acc: 0.760 | Val ACC: 0.677\n",
      "Epoch 595: | Loss: 0.49440 | Acc: 0.758 | Val ACC: 0.696\n",
      "Epoch 596: | Loss: 0.49039 | Acc: 0.767 | Val ACC: 0.698\n",
      "Epoch 597: | Loss: 0.48632 | Acc: 0.769 | Val ACC: 0.690\n",
      "Epoch 598: | Loss: 0.48771 | Acc: 0.765 | Val ACC: 0.671\n",
      "Epoch 599: | Loss: 0.49285 | Acc: 0.766 | Val ACC: 0.679\n"
     ]
    }
   ],
   "source": [
    "print('Training Starts')\n",
    "print('-'*89)\n",
    "for epoch in range(600):\n",
    "    server.fit(epoch)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>train-acc</th>\n",
       "      <th>test-acc</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>595</th>\n",
       "      <td>0.758272</td>\n",
       "      <td>0.696154</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>596</th>\n",
       "      <td>0.767463</td>\n",
       "      <td>0.698077</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>597</th>\n",
       "      <td>0.768842</td>\n",
       "      <td>0.690385</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>598</th>\n",
       "      <td>0.765165</td>\n",
       "      <td>0.671154</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>599</th>\n",
       "      <td>0.765625</td>\n",
       "      <td>0.678846</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     train-acc  test-acc\n",
       "595   0.758272  0.696154\n",
       "596   0.767463  0.698077\n",
       "597   0.768842  0.690385\n",
       "598   0.765165  0.671154\n",
       "599   0.765625  0.678846"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "history1 = server.get_history()\n",
    "history1.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How's the performance when we have more clients? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Client 0: Feature Index 0-166\n",
      "Client 1: Feature Index 167-333\n",
      "Server : Feature Index 334-499\n"
     ]
    }
   ],
   "source": [
    "client_id_list = list(range(2))\n",
    "courier = SyncLocalCourier(client_id_list)\n",
    "loader = SimpleNumpyDataLoader(\n",
    "    clients_id_list=client_id_list,\n",
    "    data_source=(X_np, y_np)\n",
    ")\n",
    "loader_dict = loader.distribute()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [],
   "source": [
    "clients = [\n",
    "    SyncFNNClient(id=id, \n",
    "    model=nn.Sequential(\n",
    "        nn.Linear(167, 64),\n",
    "        nn.ReLU(),\n",
    "        nn.Linear(64, 32),\n",
    "        nn.ReLU()),\n",
    "    courier=courier,\n",
    "    train_loader=loader_dict[id]['train_loader'],\n",
    "    test_loader=loader_dict[id]['test_loader'],\n",
    "    config_dir='simple_config.ini')\n",
    "for id in client_id_list]\n",
    "strategy = SyncConcatStrategy(courier=courier, clients=clients)\n",
    "server =  SyncFNNServer(\n",
    "    strategy=strategy,\n",
    "    courier=courier,\n",
    "    top_model=nn.Sequential(\n",
    "        nn.Linear(3 * 32, 32),\n",
    "        nn.ReLU(True),\n",
    "        nn.Linear(32, 1),\n",
    "        nn.Sigmoid()),\n",
    "    emb_model=nn.Sequential( \n",
    "        nn.Linear(166, 64),\n",
    "        nn.ReLU(),\n",
    "        nn.Linear(64, 32),\n",
    "        nn.ReLU()),\n",
    "    train_loader=loader_dict['server']['train_loader'],\n",
    "    test_loader=loader_dict['server']['test_loader'],\n",
    "    config_dir='simple_config.ini')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Starts\n",
      "-----------------------------------------------------------------------------------------\n",
      "Epoch 000: | Loss: 0.69339 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 001: | Loss: 0.69319 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 002: | Loss: 0.69318 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 003: | Loss: 0.69315 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 004: | Loss: 0.69312 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 005: | Loss: 0.69309 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 006: | Loss: 0.69306 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 007: | Loss: 0.69302 | Acc: 0.502 | Val ACC: 0.506\n",
      "Epoch 008: | Loss: 0.69298 | Acc: 0.505 | Val ACC: 0.504\n",
      "Epoch 009: | Loss: 0.69292 | Acc: 0.504 | Val ACC: 0.521\n",
      "Epoch 010: | Loss: 0.69284 | Acc: 0.516 | Val ACC: 0.556\n",
      "Epoch 011: | Loss: 0.69273 | Acc: 0.523 | Val ACC: 0.575\n",
      "Epoch 012: | Loss: 0.69259 | Acc: 0.538 | Val ACC: 0.602\n",
      "Epoch 013: | Loss: 0.69242 | Acc: 0.541 | Val ACC: 0.615\n",
      "Epoch 014: | Loss: 0.69216 | Acc: 0.531 | Val ACC: 0.615\n",
      "Epoch 015: | Loss: 0.69193 | Acc: 0.529 | Val ACC: 0.546\n",
      "Epoch 016: | Loss: 0.69147 | Acc: 0.532 | Val ACC: 0.540\n",
      "Epoch 017: | Loss: 0.69101 | Acc: 0.537 | Val ACC: 0.537\n",
      "Epoch 018: | Loss: 0.69029 | Acc: 0.538 | Val ACC: 0.560\n",
      "Epoch 019: | Loss: 0.68938 | Acc: 0.542 | Val ACC: 0.615\n",
      "Epoch 020: | Loss: 0.68788 | Acc: 0.549 | Val ACC: 0.569\n",
      "Epoch 021: | Loss: 0.68656 | Acc: 0.566 | Val ACC: 0.502\n",
      "Epoch 022: | Loss: 0.68639 | Acc: 0.558 | Val ACC: 0.531\n",
      "Epoch 023: | Loss: 0.68397 | Acc: 0.573 | Val ACC: 0.508\n",
      "Epoch 024: | Loss: 0.68366 | Acc: 0.562 | Val ACC: 0.498\n",
      "Epoch 025: | Loss: 0.68357 | Acc: 0.570 | Val ACC: 0.531\n",
      "Epoch 026: | Loss: 0.67993 | Acc: 0.590 | Val ACC: 0.537\n",
      "Epoch 027: | Loss: 0.68081 | Acc: 0.574 | Val ACC: 0.506\n",
      "Epoch 028: | Loss: 0.68007 | Acc: 0.580 | Val ACC: 0.538\n",
      "Epoch 029: | Loss: 0.67783 | Acc: 0.589 | Val ACC: 0.538\n",
      "Epoch 030: | Loss: 0.67830 | Acc: 0.586 | Val ACC: 0.537\n",
      "Epoch 031: | Loss: 0.67706 | Acc: 0.589 | Val ACC: 0.542\n",
      "Epoch 032: | Loss: 0.67652 | Acc: 0.591 | Val ACC: 0.544\n",
      "Epoch 033: | Loss: 0.67651 | Acc: 0.591 | Val ACC: 0.535\n",
      "Epoch 034: | Loss: 0.67554 | Acc: 0.595 | Val ACC: 0.573\n",
      "Epoch 035: | Loss: 0.67423 | Acc: 0.592 | Val ACC: 0.537\n",
      "Epoch 036: | Loss: 0.67580 | Acc: 0.591 | Val ACC: 0.575\n",
      "Epoch 037: | Loss: 0.67250 | Acc: 0.597 | Val ACC: 0.569\n",
      "Epoch 038: | Loss: 0.67297 | Acc: 0.595 | Val ACC: 0.556\n",
      "Epoch 039: | Loss: 0.67325 | Acc: 0.595 | Val ACC: 0.569\n",
      "Epoch 040: | Loss: 0.67162 | Acc: 0.598 | Val ACC: 0.573\n",
      "Epoch 041: | Loss: 0.67117 | Acc: 0.600 | Val ACC: 0.575\n",
      "Epoch 042: | Loss: 0.67161 | Acc: 0.599 | Val ACC: 0.567\n",
      "Epoch 043: | Loss: 0.67087 | Acc: 0.602 | Val ACC: 0.594\n",
      "Epoch 044: | Loss: 0.66982 | Acc: 0.603 | Val ACC: 0.571\n",
      "Epoch 045: | Loss: 0.67108 | Acc: 0.598 | Val ACC: 0.569\n",
      "Epoch 046: | Loss: 0.66996 | Acc: 0.602 | Val ACC: 0.596\n",
      "Epoch 047: | Loss: 0.66889 | Acc: 0.603 | Val ACC: 0.590\n",
      "Epoch 048: | Loss: 0.66849 | Acc: 0.604 | Val ACC: 0.594\n",
      "Epoch 049: | Loss: 0.66824 | Acc: 0.607 | Val ACC: 0.602\n",
      "Epoch 050: | Loss: 0.66792 | Acc: 0.602 | Val ACC: 0.583\n",
      "Epoch 051: | Loss: 0.66833 | Acc: 0.605 | Val ACC: 0.602\n",
      "Epoch 052: | Loss: 0.66762 | Acc: 0.605 | Val ACC: 0.579\n",
      "Epoch 053: | Loss: 0.66756 | Acc: 0.607 | Val ACC: 0.604\n",
      "Epoch 054: | Loss: 0.66677 | Acc: 0.602 | Val ACC: 0.594\n",
      "Epoch 055: | Loss: 0.66778 | Acc: 0.607 | Val ACC: 0.600\n",
      "Epoch 056: | Loss: 0.66671 | Acc: 0.605 | Val ACC: 0.596\n",
      "Epoch 057: | Loss: 0.66627 | Acc: 0.608 | Val ACC: 0.604\n",
      "Epoch 058: | Loss: 0.66471 | Acc: 0.608 | Val ACC: 0.606\n",
      "Epoch 059: | Loss: 0.66513 | Acc: 0.602 | Val ACC: 0.596\n",
      "Epoch 060: | Loss: 0.66800 | Acc: 0.595 | Val ACC: 0.587\n",
      "Epoch 061: | Loss: 0.66649 | Acc: 0.597 | Val ACC: 0.604\n",
      "Epoch 062: | Loss: 0.66572 | Acc: 0.597 | Val ACC: 0.588\n",
      "Epoch 063: | Loss: 0.66544 | Acc: 0.597 | Val ACC: 0.598\n",
      "Epoch 064: | Loss: 0.66436 | Acc: 0.597 | Val ACC: 0.596\n",
      "Epoch 065: | Loss: 0.66462 | Acc: 0.598 | Val ACC: 0.606\n",
      "Epoch 066: | Loss: 0.66293 | Acc: 0.597 | Val ACC: 0.604\n",
      "Epoch 067: | Loss: 0.66325 | Acc: 0.597 | Val ACC: 0.602\n",
      "Epoch 068: | Loss: 0.66321 | Acc: 0.599 | Val ACC: 0.608\n",
      "Epoch 069: | Loss: 0.66151 | Acc: 0.601 | Val ACC: 0.606\n",
      "Epoch 070: | Loss: 0.66248 | Acc: 0.600 | Val ACC: 0.621\n",
      "Epoch 071: | Loss: 0.65969 | Acc: 0.602 | Val ACC: 0.600\n",
      "Epoch 072: | Loss: 0.66259 | Acc: 0.600 | Val ACC: 0.606\n",
      "Epoch 073: | Loss: 0.65803 | Acc: 0.610 | Val ACC: 0.623\n",
      "Epoch 074: | Loss: 0.65942 | Acc: 0.602 | Val ACC: 0.619\n",
      "Epoch 075: | Loss: 0.65829 | Acc: 0.598 | Val ACC: 0.613\n",
      "Epoch 076: | Loss: 0.65906 | Acc: 0.598 | Val ACC: 0.602\n",
      "Epoch 077: | Loss: 0.66005 | Acc: 0.610 | Val ACC: 0.613\n",
      "Epoch 078: | Loss: 0.65902 | Acc: 0.608 | Val ACC: 0.642\n",
      "Epoch 079: | Loss: 0.65154 | Acc: 0.609 | Val ACC: 0.617\n",
      "Epoch 080: | Loss: 0.65691 | Acc: 0.607 | Val ACC: 0.621\n",
      "Epoch 081: | Loss: 0.65332 | Acc: 0.606 | Val ACC: 0.625\n",
      "Epoch 082: | Loss: 0.65290 | Acc: 0.602 | Val ACC: 0.623\n",
      "Epoch 083: | Loss: 0.65304 | Acc: 0.608 | Val ACC: 0.631\n",
      "Epoch 084: | Loss: 0.65094 | Acc: 0.609 | Val ACC: 0.633\n",
      "Epoch 085: | Loss: 0.65096 | Acc: 0.605 | Val ACC: 0.625\n",
      "Epoch 086: | Loss: 0.65299 | Acc: 0.608 | Val ACC: 0.613\n",
      "Epoch 087: | Loss: 0.64768 | Acc: 0.607 | Val ACC: 0.625\n",
      "Epoch 088: | Loss: 0.64878 | Acc: 0.611 | Val ACC: 0.619\n",
      "Epoch 089: | Loss: 0.64793 | Acc: 0.612 | Val ACC: 0.627\n",
      "Epoch 090: | Loss: 0.64850 | Acc: 0.608 | Val ACC: 0.631\n",
      "Epoch 091: | Loss: 0.64894 | Acc: 0.618 | Val ACC: 0.637\n",
      "Epoch 092: | Loss: 0.64455 | Acc: 0.623 | Val ACC: 0.633\n",
      "Epoch 093: | Loss: 0.64136 | Acc: 0.627 | Val ACC: 0.646\n",
      "Epoch 094: | Loss: 0.64284 | Acc: 0.628 | Val ACC: 0.637\n",
      "Epoch 095: | Loss: 0.64222 | Acc: 0.625 | Val ACC: 0.631\n",
      "Epoch 096: | Loss: 0.64268 | Acc: 0.628 | Val ACC: 0.629\n",
      "Epoch 097: | Loss: 0.63775 | Acc: 0.632 | Val ACC: 0.637\n",
      "Epoch 098: | Loss: 0.63801 | Acc: 0.635 | Val ACC: 0.635\n",
      "Epoch 099: | Loss: 0.63735 | Acc: 0.632 | Val ACC: 0.648\n",
      "Epoch 100: | Loss: 0.63872 | Acc: 0.633 | Val ACC: 0.625\n",
      "Epoch 101: | Loss: 0.63799 | Acc: 0.636 | Val ACC: 0.629\n",
      "Epoch 102: | Loss: 0.63586 | Acc: 0.639 | Val ACC: 0.640\n",
      "Epoch 103: | Loss: 0.63801 | Acc: 0.635 | Val ACC: 0.637\n",
      "Epoch 104: | Loss: 0.63639 | Acc: 0.633 | Val ACC: 0.644\n",
      "Epoch 105: | Loss: 0.63736 | Acc: 0.637 | Val ACC: 0.644\n",
      "Epoch 106: | Loss: 0.63766 | Acc: 0.639 | Val ACC: 0.627\n",
      "Epoch 107: | Loss: 0.63275 | Acc: 0.646 | Val ACC: 0.650\n",
      "Epoch 108: | Loss: 0.63009 | Acc: 0.647 | Val ACC: 0.650\n",
      "Epoch 109: | Loss: 0.63422 | Acc: 0.636 | Val ACC: 0.654\n",
      "Epoch 110: | Loss: 0.63484 | Acc: 0.638 | Val ACC: 0.644\n",
      "Epoch 111: | Loss: 0.62917 | Acc: 0.649 | Val ACC: 0.646\n",
      "Epoch 112: | Loss: 0.62843 | Acc: 0.652 | Val ACC: 0.650\n",
      "Epoch 113: | Loss: 0.63068 | Acc: 0.646 | Val ACC: 0.646\n",
      "Epoch 114: | Loss: 0.63183 | Acc: 0.642 | Val ACC: 0.642\n",
      "Epoch 115: | Loss: 0.62852 | Acc: 0.649 | Val ACC: 0.648\n",
      "Epoch 116: | Loss: 0.62930 | Acc: 0.647 | Val ACC: 0.648\n",
      "Epoch 117: | Loss: 0.62924 | Acc: 0.644 | Val ACC: 0.656\n",
      "Epoch 118: | Loss: 0.62781 | Acc: 0.650 | Val ACC: 0.663\n",
      "Epoch 119: | Loss: 0.62772 | Acc: 0.658 | Val ACC: 0.656\n",
      "Epoch 120: | Loss: 0.62869 | Acc: 0.655 | Val ACC: 0.656\n",
      "Epoch 121: | Loss: 0.62769 | Acc: 0.654 | Val ACC: 0.656\n",
      "Epoch 122: | Loss: 0.62735 | Acc: 0.656 | Val ACC: 0.652\n",
      "Epoch 123: | Loss: 0.62632 | Acc: 0.658 | Val ACC: 0.662\n",
      "Epoch 124: | Loss: 0.62572 | Acc: 0.661 | Val ACC: 0.677\n",
      "Epoch 125: | Loss: 0.62422 | Acc: 0.663 | Val ACC: 0.683\n",
      "Epoch 126: | Loss: 0.62524 | Acc: 0.659 | Val ACC: 0.679\n",
      "Epoch 127: | Loss: 0.62367 | Acc: 0.658 | Val ACC: 0.683\n",
      "Epoch 128: | Loss: 0.62140 | Acc: 0.659 | Val ACC: 0.679\n",
      "Epoch 129: | Loss: 0.62037 | Acc: 0.661 | Val ACC: 0.679\n",
      "Epoch 130: | Loss: 0.62014 | Acc: 0.664 | Val ACC: 0.683\n",
      "Epoch 131: | Loss: 0.62128 | Acc: 0.659 | Val ACC: 0.681\n",
      "Epoch 132: | Loss: 0.62146 | Acc: 0.653 | Val ACC: 0.675\n",
      "Epoch 133: | Loss: 0.62137 | Acc: 0.658 | Val ACC: 0.679\n",
      "Epoch 134: | Loss: 0.62027 | Acc: 0.663 | Val ACC: 0.681\n",
      "Epoch 135: | Loss: 0.61872 | Acc: 0.657 | Val ACC: 0.679\n",
      "Epoch 136: | Loss: 0.62027 | Acc: 0.656 | Val ACC: 0.681\n",
      "Epoch 137: | Loss: 0.62134 | Acc: 0.658 | Val ACC: 0.679\n",
      "Epoch 138: | Loss: 0.61958 | Acc: 0.659 | Val ACC: 0.681\n",
      "Epoch 139: | Loss: 0.61865 | Acc: 0.659 | Val ACC: 0.681\n",
      "Epoch 140: | Loss: 0.61878 | Acc: 0.661 | Val ACC: 0.683\n",
      "Epoch 141: | Loss: 0.61850 | Acc: 0.658 | Val ACC: 0.677\n",
      "Epoch 142: | Loss: 0.61910 | Acc: 0.656 | Val ACC: 0.679\n",
      "Epoch 143: | Loss: 0.61991 | Acc: 0.658 | Val ACC: 0.681\n",
      "Epoch 144: | Loss: 0.61764 | Acc: 0.660 | Val ACC: 0.683\n",
      "Epoch 145: | Loss: 0.61574 | Acc: 0.659 | Val ACC: 0.681\n",
      "Epoch 146: | Loss: 0.61559 | Acc: 0.659 | Val ACC: 0.685\n",
      "Epoch 147: | Loss: 0.61604 | Acc: 0.653 | Val ACC: 0.679\n",
      "Epoch 148: | Loss: 0.61716 | Acc: 0.658 | Val ACC: 0.683\n",
      "Epoch 149: | Loss: 0.61571 | Acc: 0.658 | Val ACC: 0.683\n",
      "Epoch 150: | Loss: 0.61519 | Acc: 0.659 | Val ACC: 0.687\n",
      "Epoch 151: | Loss: 0.61430 | Acc: 0.660 | Val ACC: 0.685\n",
      "Epoch 152: | Loss: 0.61445 | Acc: 0.659 | Val ACC: 0.687\n",
      "Epoch 153: | Loss: 0.61464 | Acc: 0.658 | Val ACC: 0.677\n",
      "Epoch 154: | Loss: 0.61522 | Acc: 0.658 | Val ACC: 0.677\n",
      "Epoch 155: | Loss: 0.61663 | Acc: 0.657 | Val ACC: 0.662\n",
      "Epoch 156: | Loss: 0.61525 | Acc: 0.660 | Val ACC: 0.667\n",
      "Epoch 157: | Loss: 0.61355 | Acc: 0.662 | Val ACC: 0.683\n",
      "Epoch 158: | Loss: 0.61307 | Acc: 0.658 | Val ACC: 0.675\n",
      "Epoch 159: | Loss: 0.61498 | Acc: 0.659 | Val ACC: 0.658\n",
      "Epoch 160: | Loss: 0.61464 | Acc: 0.661 | Val ACC: 0.677\n",
      "Epoch 161: | Loss: 0.61259 | Acc: 0.663 | Val ACC: 0.683\n",
      "Epoch 162: | Loss: 0.61216 | Acc: 0.662 | Val ACC: 0.685\n",
      "Epoch 163: | Loss: 0.61195 | Acc: 0.658 | Val ACC: 0.677\n",
      "Epoch 164: | Loss: 0.61410 | Acc: 0.657 | Val ACC: 0.663\n",
      "Epoch 165: | Loss: 0.61376 | Acc: 0.663 | Val ACC: 0.662\n",
      "Epoch 166: | Loss: 0.61191 | Acc: 0.665 | Val ACC: 0.679\n",
      "Epoch 167: | Loss: 0.61025 | Acc: 0.663 | Val ACC: 0.683\n",
      "Epoch 168: | Loss: 0.61087 | Acc: 0.664 | Val ACC: 0.675\n",
      "Epoch 169: | Loss: 0.61128 | Acc: 0.660 | Val ACC: 0.673\n",
      "Epoch 170: | Loss: 0.61183 | Acc: 0.658 | Val ACC: 0.671\n",
      "Epoch 171: | Loss: 0.60961 | Acc: 0.666 | Val ACC: 0.677\n",
      "Epoch 172: | Loss: 0.60812 | Acc: 0.661 | Val ACC: 0.681\n",
      "Epoch 173: | Loss: 0.60824 | Acc: 0.665 | Val ACC: 0.685\n",
      "Epoch 174: | Loss: 0.60879 | Acc: 0.664 | Val ACC: 0.677\n",
      "Epoch 175: | Loss: 0.61023 | Acc: 0.666 | Val ACC: 0.671\n",
      "Epoch 176: | Loss: 0.61061 | Acc: 0.660 | Val ACC: 0.663\n",
      "Epoch 177: | Loss: 0.60919 | Acc: 0.664 | Val ACC: 0.667\n",
      "Epoch 178: | Loss: 0.60883 | Acc: 0.660 | Val ACC: 0.667\n",
      "Epoch 179: | Loss: 0.60773 | Acc: 0.665 | Val ACC: 0.673\n",
      "Epoch 180: | Loss: 0.60647 | Acc: 0.668 | Val ACC: 0.675\n",
      "Epoch 181: | Loss: 0.60599 | Acc: 0.669 | Val ACC: 0.677\n",
      "Epoch 182: | Loss: 0.60561 | Acc: 0.663 | Val ACC: 0.675\n",
      "Epoch 183: | Loss: 0.60564 | Acc: 0.668 | Val ACC: 0.663\n",
      "Epoch 184: | Loss: 0.60441 | Acc: 0.670 | Val ACC: 0.675\n",
      "Epoch 185: | Loss: 0.60379 | Acc: 0.668 | Val ACC: 0.677\n",
      "Epoch 186: | Loss: 0.60460 | Acc: 0.664 | Val ACC: 0.675\n",
      "Epoch 187: | Loss: 0.60615 | Acc: 0.670 | Val ACC: 0.669\n",
      "Epoch 188: | Loss: 0.60509 | Acc: 0.666 | Val ACC: 0.677\n",
      "Epoch 189: | Loss: 0.60481 | Acc: 0.673 | Val ACC: 0.663\n",
      "Epoch 190: | Loss: 0.60388 | Acc: 0.666 | Val ACC: 0.679\n",
      "Epoch 191: | Loss: 0.60424 | Acc: 0.672 | Val ACC: 0.667\n",
      "Epoch 192: | Loss: 0.60388 | Acc: 0.676 | Val ACC: 0.671\n",
      "Epoch 193: | Loss: 0.60390 | Acc: 0.674 | Val ACC: 0.673\n",
      "Epoch 194: | Loss: 0.60273 | Acc: 0.677 | Val ACC: 0.667\n",
      "Epoch 195: | Loss: 0.60093 | Acc: 0.679 | Val ACC: 0.665\n",
      "Epoch 196: | Loss: 0.60214 | Acc: 0.678 | Val ACC: 0.667\n",
      "Epoch 197: | Loss: 0.60243 | Acc: 0.675 | Val ACC: 0.671\n",
      "Epoch 198: | Loss: 0.60318 | Acc: 0.676 | Val ACC: 0.669\n",
      "Epoch 199: | Loss: 0.60184 | Acc: 0.678 | Val ACC: 0.669\n",
      "Epoch 200: | Loss: 0.60235 | Acc: 0.680 | Val ACC: 0.665\n",
      "Epoch 201: | Loss: 0.60307 | Acc: 0.677 | Val ACC: 0.663\n",
      "Epoch 202: | Loss: 0.60017 | Acc: 0.681 | Val ACC: 0.671\n",
      "Epoch 203: | Loss: 0.59924 | Acc: 0.682 | Val ACC: 0.671\n",
      "Epoch 204: | Loss: 0.60103 | Acc: 0.683 | Val ACC: 0.673\n",
      "Epoch 205: | Loss: 0.59948 | Acc: 0.681 | Val ACC: 0.673\n",
      "Epoch 206: | Loss: 0.59809 | Acc: 0.688 | Val ACC: 0.669\n",
      "Epoch 207: | Loss: 0.59547 | Acc: 0.693 | Val ACC: 0.673\n",
      "Epoch 208: | Loss: 0.59541 | Acc: 0.691 | Val ACC: 0.660\n",
      "Epoch 209: | Loss: 0.59499 | Acc: 0.690 | Val ACC: 0.681\n",
      "Epoch 210: | Loss: 0.59477 | Acc: 0.693 | Val ACC: 0.675\n",
      "Epoch 211: | Loss: 0.59448 | Acc: 0.692 | Val ACC: 0.667\n",
      "Epoch 212: | Loss: 0.59406 | Acc: 0.697 | Val ACC: 0.667\n",
      "Epoch 213: | Loss: 0.59402 | Acc: 0.693 | Val ACC: 0.665\n",
      "Epoch 214: | Loss: 0.59275 | Acc: 0.698 | Val ACC: 0.667\n",
      "Epoch 215: | Loss: 0.59312 | Acc: 0.698 | Val ACC: 0.669\n",
      "Epoch 216: | Loss: 0.59124 | Acc: 0.699 | Val ACC: 0.662\n",
      "Epoch 217: | Loss: 0.59195 | Acc: 0.699 | Val ACC: 0.660\n",
      "Epoch 218: | Loss: 0.59159 | Acc: 0.697 | Val ACC: 0.658\n",
      "Epoch 219: | Loss: 0.58952 | Acc: 0.700 | Val ACC: 0.660\n",
      "Epoch 220: | Loss: 0.58822 | Acc: 0.702 | Val ACC: 0.654\n",
      "Epoch 221: | Loss: 0.58734 | Acc: 0.703 | Val ACC: 0.656\n",
      "Epoch 222: | Loss: 0.58744 | Acc: 0.703 | Val ACC: 0.669\n",
      "Epoch 223: | Loss: 0.58784 | Acc: 0.699 | Val ACC: 0.673\n",
      "Epoch 224: | Loss: 0.58869 | Acc: 0.704 | Val ACC: 0.669\n",
      "Epoch 225: | Loss: 0.58702 | Acc: 0.703 | Val ACC: 0.673\n",
      "Epoch 226: | Loss: 0.58655 | Acc: 0.705 | Val ACC: 0.663\n",
      "Epoch 227: | Loss: 0.58629 | Acc: 0.705 | Val ACC: 0.669\n",
      "Epoch 228: | Loss: 0.58596 | Acc: 0.703 | Val ACC: 0.665\n",
      "Epoch 229: | Loss: 0.58597 | Acc: 0.704 | Val ACC: 0.654\n",
      "Epoch 230: | Loss: 0.58588 | Acc: 0.703 | Val ACC: 0.658\n",
      "Epoch 231: | Loss: 0.58593 | Acc: 0.702 | Val ACC: 0.658\n",
      "Epoch 232: | Loss: 0.58522 | Acc: 0.705 | Val ACC: 0.656\n",
      "Epoch 233: | Loss: 0.58450 | Acc: 0.703 | Val ACC: 0.663\n",
      "Epoch 234: | Loss: 0.58473 | Acc: 0.705 | Val ACC: 0.662\n",
      "Epoch 235: | Loss: 0.58477 | Acc: 0.704 | Val ACC: 0.654\n",
      "Epoch 236: | Loss: 0.58346 | Acc: 0.708 | Val ACC: 0.658\n",
      "Epoch 237: | Loss: 0.58293 | Acc: 0.706 | Val ACC: 0.660\n",
      "Epoch 238: | Loss: 0.58270 | Acc: 0.705 | Val ACC: 0.658\n",
      "Epoch 239: | Loss: 0.58264 | Acc: 0.709 | Val ACC: 0.654\n",
      "Epoch 240: | Loss: 0.58210 | Acc: 0.707 | Val ACC: 0.658\n",
      "Epoch 241: | Loss: 0.58002 | Acc: 0.710 | Val ACC: 0.667\n",
      "Epoch 242: | Loss: 0.58218 | Acc: 0.709 | Val ACC: 0.673\n",
      "Epoch 243: | Loss: 0.58475 | Acc: 0.703 | Val ACC: 0.667\n",
      "Epoch 244: | Loss: 0.58513 | Acc: 0.699 | Val ACC: 0.662\n",
      "Epoch 245: | Loss: 0.58405 | Acc: 0.705 | Val ACC: 0.654\n",
      "Epoch 246: | Loss: 0.58357 | Acc: 0.700 | Val ACC: 0.662\n",
      "Epoch 247: | Loss: 0.58507 | Acc: 0.705 | Val ACC: 0.669\n",
      "Epoch 248: | Loss: 0.58737 | Acc: 0.701 | Val ACC: 0.671\n",
      "Epoch 249: | Loss: 0.58996 | Acc: 0.699 | Val ACC: 0.663\n",
      "Epoch 250: | Loss: 0.58912 | Acc: 0.701 | Val ACC: 0.660\n",
      "Epoch 251: | Loss: 0.58729 | Acc: 0.703 | Val ACC: 0.662\n",
      "Epoch 252: | Loss: 0.58598 | Acc: 0.704 | Val ACC: 0.663\n",
      "Epoch 253: | Loss: 0.58517 | Acc: 0.707 | Val ACC: 0.652\n",
      "Epoch 254: | Loss: 0.58330 | Acc: 0.704 | Val ACC: 0.660\n",
      "Epoch 255: | Loss: 0.58295 | Acc: 0.704 | Val ACC: 0.640\n",
      "Epoch 256: | Loss: 0.57940 | Acc: 0.707 | Val ACC: 0.656\n",
      "Epoch 257: | Loss: 0.57933 | Acc: 0.708 | Val ACC: 0.675\n",
      "Epoch 258: | Loss: 0.58140 | Acc: 0.706 | Val ACC: 0.681\n",
      "Epoch 259: | Loss: 0.58542 | Acc: 0.697 | Val ACC: 0.673\n",
      "Epoch 260: | Loss: 0.58517 | Acc: 0.699 | Val ACC: 0.673\n",
      "Epoch 261: | Loss: 0.58780 | Acc: 0.698 | Val ACC: 0.662\n",
      "Epoch 262: | Loss: 0.58447 | Acc: 0.705 | Val ACC: 0.660\n",
      "Epoch 263: | Loss: 0.58083 | Acc: 0.707 | Val ACC: 0.658\n",
      "Epoch 264: | Loss: 0.58287 | Acc: 0.705 | Val ACC: 0.663\n",
      "Epoch 265: | Loss: 0.58213 | Acc: 0.703 | Val ACC: 0.656\n",
      "Epoch 266: | Loss: 0.58294 | Acc: 0.699 | Val ACC: 0.650\n",
      "Epoch 267: | Loss: 0.58020 | Acc: 0.705 | Val ACC: 0.656\n",
      "Epoch 268: | Loss: 0.57998 | Acc: 0.704 | Val ACC: 0.652\n",
      "Epoch 269: | Loss: 0.57911 | Acc: 0.710 | Val ACC: 0.669\n",
      "Epoch 270: | Loss: 0.57734 | Acc: 0.706 | Val ACC: 0.662\n",
      "Epoch 271: | Loss: 0.57571 | Acc: 0.713 | Val ACC: 0.671\n",
      "Epoch 272: | Loss: 0.57783 | Acc: 0.711 | Val ACC: 0.663\n",
      "Epoch 273: | Loss: 0.57806 | Acc: 0.710 | Val ACC: 0.669\n",
      "Epoch 274: | Loss: 0.57787 | Acc: 0.710 | Val ACC: 0.667\n",
      "Epoch 275: | Loss: 0.57923 | Acc: 0.704 | Val ACC: 0.665\n",
      "Epoch 276: | Loss: 0.57785 | Acc: 0.707 | Val ACC: 0.662\n",
      "Epoch 277: | Loss: 0.57406 | Acc: 0.711 | Val ACC: 0.671\n",
      "Epoch 278: | Loss: 0.57695 | Acc: 0.711 | Val ACC: 0.665\n",
      "Epoch 279: | Loss: 0.57557 | Acc: 0.709 | Val ACC: 0.671\n",
      "Epoch 280: | Loss: 0.57443 | Acc: 0.713 | Val ACC: 0.667\n",
      "Epoch 281: | Loss: 0.57527 | Acc: 0.710 | Val ACC: 0.669\n",
      "Epoch 282: | Loss: 0.57738 | Acc: 0.710 | Val ACC: 0.671\n",
      "Epoch 283: | Loss: 0.57485 | Acc: 0.709 | Val ACC: 0.665\n",
      "Epoch 284: | Loss: 0.57428 | Acc: 0.712 | Val ACC: 0.667\n",
      "Epoch 285: | Loss: 0.57404 | Acc: 0.712 | Val ACC: 0.663\n",
      "Epoch 286: | Loss: 0.57321 | Acc: 0.712 | Val ACC: 0.671\n",
      "Epoch 287: | Loss: 0.57241 | Acc: 0.712 | Val ACC: 0.665\n",
      "Epoch 288: | Loss: 0.57469 | Acc: 0.712 | Val ACC: 0.665\n",
      "Epoch 289: | Loss: 0.57389 | Acc: 0.713 | Val ACC: 0.665\n",
      "Epoch 290: | Loss: 0.57179 | Acc: 0.714 | Val ACC: 0.671\n",
      "Epoch 291: | Loss: 0.57246 | Acc: 0.711 | Val ACC: 0.669\n",
      "Epoch 292: | Loss: 0.57371 | Acc: 0.710 | Val ACC: 0.665\n",
      "Epoch 293: | Loss: 0.57311 | Acc: 0.713 | Val ACC: 0.673\n",
      "Epoch 294: | Loss: 0.57265 | Acc: 0.712 | Val ACC: 0.667\n",
      "Epoch 295: | Loss: 0.57358 | Acc: 0.710 | Val ACC: 0.665\n",
      "Epoch 296: | Loss: 0.57380 | Acc: 0.711 | Val ACC: 0.665\n",
      "Epoch 297: | Loss: 0.57311 | Acc: 0.711 | Val ACC: 0.669\n",
      "Epoch 298: | Loss: 0.57313 | Acc: 0.712 | Val ACC: 0.665\n",
      "Epoch 299: | Loss: 0.57151 | Acc: 0.713 | Val ACC: 0.669\n",
      "Epoch 300: | Loss: 0.57067 | Acc: 0.713 | Val ACC: 0.662\n",
      "Epoch 301: | Loss: 0.57250 | Acc: 0.712 | Val ACC: 0.669\n",
      "Epoch 302: | Loss: 0.57050 | Acc: 0.714 | Val ACC: 0.658\n",
      "Epoch 303: | Loss: 0.56889 | Acc: 0.714 | Val ACC: 0.663\n",
      "Epoch 304: | Loss: 0.56927 | Acc: 0.710 | Val ACC: 0.654\n",
      "Epoch 305: | Loss: 0.56856 | Acc: 0.712 | Val ACC: 0.669\n",
      "Epoch 306: | Loss: 0.56944 | Acc: 0.713 | Val ACC: 0.656\n",
      "Epoch 307: | Loss: 0.56787 | Acc: 0.713 | Val ACC: 0.662\n",
      "Epoch 308: | Loss: 0.56967 | Acc: 0.711 | Val ACC: 0.658\n",
      "Epoch 309: | Loss: 0.56970 | Acc: 0.714 | Val ACC: 0.660\n",
      "Epoch 310: | Loss: 0.56924 | Acc: 0.714 | Val ACC: 0.673\n",
      "Epoch 311: | Loss: 0.56931 | Acc: 0.714 | Val ACC: 0.656\n",
      "Epoch 312: | Loss: 0.56947 | Acc: 0.714 | Val ACC: 0.667\n",
      "Epoch 313: | Loss: 0.57134 | Acc: 0.708 | Val ACC: 0.665\n",
      "Epoch 314: | Loss: 0.56942 | Acc: 0.707 | Val ACC: 0.654\n",
      "Epoch 315: | Loss: 0.57019 | Acc: 0.708 | Val ACC: 0.660\n",
      "Epoch 316: | Loss: 0.57017 | Acc: 0.710 | Val ACC: 0.660\n",
      "Epoch 317: | Loss: 0.57041 | Acc: 0.706 | Val ACC: 0.662\n",
      "Epoch 318: | Loss: 0.57100 | Acc: 0.708 | Val ACC: 0.656\n",
      "Epoch 319: | Loss: 0.57016 | Acc: 0.710 | Val ACC: 0.663\n",
      "Epoch 320: | Loss: 0.57108 | Acc: 0.705 | Val ACC: 0.656\n",
      "Epoch 321: | Loss: 0.57233 | Acc: 0.705 | Val ACC: 0.660\n",
      "Epoch 322: | Loss: 0.57255 | Acc: 0.707 | Val ACC: 0.665\n",
      "Epoch 323: | Loss: 0.57298 | Acc: 0.705 | Val ACC: 0.660\n",
      "Epoch 324: | Loss: 0.57314 | Acc: 0.704 | Val ACC: 0.663\n",
      "Epoch 325: | Loss: 0.56871 | Acc: 0.711 | Val ACC: 0.663\n",
      "Epoch 326: | Loss: 0.56912 | Acc: 0.706 | Val ACC: 0.663\n",
      "Epoch 327: | Loss: 0.57055 | Acc: 0.710 | Val ACC: 0.662\n",
      "Epoch 328: | Loss: 0.56979 | Acc: 0.707 | Val ACC: 0.662\n",
      "Epoch 329: | Loss: 0.57038 | Acc: 0.712 | Val ACC: 0.663\n",
      "Epoch 330: | Loss: 0.57020 | Acc: 0.707 | Val ACC: 0.665\n",
      "Epoch 331: | Loss: 0.57132 | Acc: 0.710 | Val ACC: 0.663\n",
      "Epoch 332: | Loss: 0.56931 | Acc: 0.710 | Val ACC: 0.663\n",
      "Epoch 333: | Loss: 0.57124 | Acc: 0.709 | Val ACC: 0.665\n",
      "Epoch 334: | Loss: 0.56759 | Acc: 0.708 | Val ACC: 0.660\n",
      "Epoch 335: | Loss: 0.56896 | Acc: 0.712 | Val ACC: 0.663\n",
      "Epoch 336: | Loss: 0.56978 | Acc: 0.707 | Val ACC: 0.665\n",
      "Epoch 337: | Loss: 0.56851 | Acc: 0.709 | Val ACC: 0.667\n",
      "Epoch 338: | Loss: 0.56836 | Acc: 0.709 | Val ACC: 0.663\n",
      "Epoch 339: | Loss: 0.56956 | Acc: 0.708 | Val ACC: 0.669\n",
      "Epoch 340: | Loss: 0.56905 | Acc: 0.707 | Val ACC: 0.656\n",
      "Epoch 341: | Loss: 0.57163 | Acc: 0.704 | Val ACC: 0.663\n",
      "Epoch 342: | Loss: 0.56975 | Acc: 0.705 | Val ACC: 0.665\n",
      "Epoch 343: | Loss: 0.57015 | Acc: 0.705 | Val ACC: 0.662\n",
      "Epoch 344: | Loss: 0.57334 | Acc: 0.702 | Val ACC: 0.656\n",
      "Epoch 345: | Loss: 0.57820 | Acc: 0.700 | Val ACC: 0.665\n",
      "Epoch 346: | Loss: 0.58153 | Acc: 0.697 | Val ACC: 0.667\n",
      "Epoch 347: | Loss: 0.57846 | Acc: 0.699 | Val ACC: 0.662\n",
      "Epoch 348: | Loss: 0.57783 | Acc: 0.699 | Val ACC: 0.662\n",
      "Epoch 349: | Loss: 0.58766 | Acc: 0.690 | Val ACC: 0.667\n",
      "Epoch 350: | Loss: 0.58417 | Acc: 0.695 | Val ACC: 0.687\n",
      "Epoch 351: | Loss: 0.57765 | Acc: 0.700 | Val ACC: 0.704\n",
      "Epoch 352: | Loss: 0.58277 | Acc: 0.682 | Val ACC: 0.662\n",
      "Epoch 353: | Loss: 0.57835 | Acc: 0.708 | Val ACC: 0.700\n",
      "Epoch 354: | Loss: 0.57299 | Acc: 0.705 | Val ACC: 0.700\n",
      "Epoch 355: | Loss: 0.57035 | Acc: 0.711 | Val ACC: 0.700\n",
      "Epoch 356: | Loss: 0.57931 | Acc: 0.695 | Val ACC: 0.667\n",
      "Epoch 357: | Loss: 0.58485 | Acc: 0.693 | Val ACC: 0.677\n",
      "Epoch 358: | Loss: 0.57882 | Acc: 0.695 | Val ACC: 0.692\n",
      "Epoch 359: | Loss: 0.56666 | Acc: 0.710 | Val ACC: 0.698\n",
      "Epoch 360: | Loss: 0.56308 | Acc: 0.718 | Val ACC: 0.700\n",
      "Epoch 361: | Loss: 0.56359 | Acc: 0.710 | Val ACC: 0.706\n",
      "Epoch 362: | Loss: 0.56544 | Acc: 0.712 | Val ACC: 0.688\n",
      "Epoch 363: | Loss: 0.56101 | Acc: 0.718 | Val ACC: 0.692\n",
      "Epoch 364: | Loss: 0.56172 | Acc: 0.715 | Val ACC: 0.700\n",
      "Epoch 365: | Loss: 0.57139 | Acc: 0.700 | Val ACC: 0.679\n",
      "Epoch 366: | Loss: 0.56957 | Acc: 0.706 | Val ACC: 0.694\n",
      "Epoch 367: | Loss: 0.55760 | Acc: 0.721 | Val ACC: 0.700\n",
      "Epoch 368: | Loss: 0.56005 | Acc: 0.719 | Val ACC: 0.696\n",
      "Epoch 369: | Loss: 0.55855 | Acc: 0.718 | Val ACC: 0.694\n",
      "Epoch 370: | Loss: 0.56280 | Acc: 0.714 | Val ACC: 0.696\n",
      "Epoch 371: | Loss: 0.56310 | Acc: 0.717 | Val ACC: 0.687\n",
      "Epoch 372: | Loss: 0.56948 | Acc: 0.704 | Val ACC: 0.671\n",
      "Epoch 373: | Loss: 0.56931 | Acc: 0.713 | Val ACC: 0.688\n",
      "Epoch 374: | Loss: 0.56449 | Acc: 0.704 | Val ACC: 0.677\n",
      "Epoch 375: | Loss: 0.56051 | Acc: 0.719 | Val ACC: 0.687\n",
      "Epoch 376: | Loss: 0.55618 | Acc: 0.720 | Val ACC: 0.688\n",
      "Epoch 377: | Loss: 0.55652 | Acc: 0.718 | Val ACC: 0.687\n",
      "Epoch 378: | Loss: 0.55873 | Acc: 0.716 | Val ACC: 0.685\n",
      "Epoch 379: | Loss: 0.57198 | Acc: 0.700 | Val ACC: 0.669\n",
      "Epoch 380: | Loss: 0.57572 | Acc: 0.702 | Val ACC: 0.681\n",
      "Epoch 381: | Loss: 0.55849 | Acc: 0.718 | Val ACC: 0.683\n",
      "Epoch 382: | Loss: 0.55650 | Acc: 0.722 | Val ACC: 0.692\n",
      "Epoch 383: | Loss: 0.55594 | Acc: 0.723 | Val ACC: 0.694\n",
      "Epoch 384: | Loss: 0.56371 | Acc: 0.707 | Val ACC: 0.681\n",
      "Epoch 385: | Loss: 0.56334 | Acc: 0.711 | Val ACC: 0.692\n",
      "Epoch 386: | Loss: 0.56888 | Acc: 0.705 | Val ACC: 0.667\n",
      "Epoch 387: | Loss: 0.56914 | Acc: 0.711 | Val ACC: 0.685\n",
      "Epoch 388: | Loss: 0.55706 | Acc: 0.715 | Val ACC: 0.681\n",
      "Epoch 389: | Loss: 0.55982 | Acc: 0.720 | Val ACC: 0.690\n",
      "Epoch 390: | Loss: 0.56506 | Acc: 0.711 | Val ACC: 0.702\n",
      "Epoch 391: | Loss: 0.58957 | Acc: 0.692 | Val ACC: 0.637\n",
      "Epoch 392: | Loss: 0.59258 | Acc: 0.687 | Val ACC: 0.692\n",
      "Epoch 393: | Loss: 0.56205 | Acc: 0.719 | Val ACC: 0.696\n",
      "Epoch 394: | Loss: 0.55701 | Acc: 0.710 | Val ACC: 0.688\n",
      "Epoch 395: | Loss: 0.55672 | Acc: 0.712 | Val ACC: 0.683\n",
      "Epoch 396: | Loss: 0.55891 | Acc: 0.710 | Val ACC: 0.681\n",
      "Epoch 397: | Loss: 0.57153 | Acc: 0.705 | Val ACC: 0.667\n",
      "Epoch 398: | Loss: 0.57001 | Acc: 0.709 | Val ACC: 0.677\n",
      "Epoch 399: | Loss: 0.55291 | Acc: 0.716 | Val ACC: 0.692\n",
      "Epoch 400: | Loss: 0.55174 | Acc: 0.713 | Val ACC: 0.692\n",
      "Epoch 401: | Loss: 0.55324 | Acc: 0.719 | Val ACC: 0.690\n",
      "Epoch 402: | Loss: 0.56342 | Acc: 0.706 | Val ACC: 0.673\n",
      "Epoch 403: | Loss: 0.56523 | Acc: 0.707 | Val ACC: 0.688\n",
      "Epoch 404: | Loss: 0.56167 | Acc: 0.710 | Val ACC: 0.673\n",
      "Epoch 405: | Loss: 0.55746 | Acc: 0.714 | Val ACC: 0.698\n",
      "Epoch 406: | Loss: 0.55211 | Acc: 0.717 | Val ACC: 0.690\n",
      "Epoch 407: | Loss: 0.55167 | Acc: 0.720 | Val ACC: 0.692\n",
      "Epoch 408: | Loss: 0.55754 | Acc: 0.713 | Val ACC: 0.671\n",
      "Epoch 409: | Loss: 0.57255 | Acc: 0.698 | Val ACC: 0.662\n",
      "Epoch 410: | Loss: 0.57264 | Acc: 0.705 | Val ACC: 0.665\n",
      "Epoch 411: | Loss: 0.56015 | Acc: 0.716 | Val ACC: 0.702\n",
      "Epoch 412: | Loss: 0.55907 | Acc: 0.714 | Val ACC: 0.700\n",
      "Epoch 413: | Loss: 0.55932 | Acc: 0.715 | Val ACC: 0.704\n",
      "Epoch 414: | Loss: 0.54918 | Acc: 0.719 | Val ACC: 0.692\n",
      "Epoch 415: | Loss: 0.54716 | Acc: 0.723 | Val ACC: 0.692\n",
      "Epoch 416: | Loss: 0.55263 | Acc: 0.717 | Val ACC: 0.683\n",
      "Epoch 417: | Loss: 0.56484 | Acc: 0.699 | Val ACC: 0.673\n",
      "Epoch 418: | Loss: 0.56451 | Acc: 0.705 | Val ACC: 0.685\n",
      "Epoch 419: | Loss: 0.56436 | Acc: 0.706 | Val ACC: 0.665\n",
      "Epoch 420: | Loss: 0.56299 | Acc: 0.716 | Val ACC: 0.667\n",
      "Epoch 421: | Loss: 0.55072 | Acc: 0.722 | Val ACC: 0.692\n",
      "Epoch 422: | Loss: 0.55134 | Acc: 0.716 | Val ACC: 0.692\n",
      "Epoch 423: | Loss: 0.55228 | Acc: 0.716 | Val ACC: 0.690\n",
      "Epoch 424: | Loss: 0.55190 | Acc: 0.718 | Val ACC: 0.687\n",
      "Epoch 425: | Loss: 0.55195 | Acc: 0.718 | Val ACC: 0.690\n",
      "Epoch 426: | Loss: 0.54922 | Acc: 0.719 | Val ACC: 0.685\n",
      "Epoch 427: | Loss: 0.54470 | Acc: 0.725 | Val ACC: 0.690\n",
      "Epoch 428: | Loss: 0.54360 | Acc: 0.724 | Val ACC: 0.692\n",
      "Epoch 429: | Loss: 0.54479 | Acc: 0.722 | Val ACC: 0.690\n",
      "Epoch 430: | Loss: 0.56223 | Acc: 0.708 | Val ACC: 0.673\n",
      "Epoch 431: | Loss: 0.56195 | Acc: 0.708 | Val ACC: 0.683\n",
      "Epoch 432: | Loss: 0.56176 | Acc: 0.709 | Val ACC: 0.681\n",
      "Epoch 433: | Loss: 0.55466 | Acc: 0.716 | Val ACC: 0.688\n",
      "Epoch 434: | Loss: 0.54919 | Acc: 0.719 | Val ACC: 0.692\n",
      "Epoch 435: | Loss: 0.54668 | Acc: 0.722 | Val ACC: 0.685\n",
      "Epoch 436: | Loss: 0.54401 | Acc: 0.724 | Val ACC: 0.688\n",
      "Epoch 437: | Loss: 0.54517 | Acc: 0.723 | Val ACC: 0.687\n",
      "Epoch 438: | Loss: 0.54543 | Acc: 0.722 | Val ACC: 0.681\n",
      "Epoch 439: | Loss: 0.54081 | Acc: 0.727 | Val ACC: 0.688\n",
      "Epoch 440: | Loss: 0.53956 | Acc: 0.728 | Val ACC: 0.694\n",
      "Epoch 441: | Loss: 0.54181 | Acc: 0.722 | Val ACC: 0.692\n",
      "Epoch 442: | Loss: 0.56188 | Acc: 0.705 | Val ACC: 0.677\n",
      "Epoch 443: | Loss: 0.56253 | Acc: 0.704 | Val ACC: 0.685\n",
      "Epoch 444: | Loss: 0.55628 | Acc: 0.717 | Val ACC: 0.683\n",
      "Epoch 445: | Loss: 0.55059 | Acc: 0.718 | Val ACC: 0.681\n",
      "Epoch 446: | Loss: 0.55068 | Acc: 0.718 | Val ACC: 0.681\n",
      "Epoch 447: | Loss: 0.55745 | Acc: 0.713 | Val ACC: 0.688\n",
      "Epoch 448: | Loss: 0.56259 | Acc: 0.709 | Val ACC: 0.665\n",
      "Epoch 449: | Loss: 0.55042 | Acc: 0.721 | Val ACC: 0.685\n",
      "Epoch 450: | Loss: 0.54285 | Acc: 0.725 | Val ACC: 0.687\n",
      "Epoch 451: | Loss: 0.54289 | Acc: 0.721 | Val ACC: 0.690\n",
      "Epoch 452: | Loss: 0.54271 | Acc: 0.726 | Val ACC: 0.685\n",
      "Epoch 453: | Loss: 0.54275 | Acc: 0.722 | Val ACC: 0.688\n",
      "Epoch 454: | Loss: 0.54460 | Acc: 0.726 | Val ACC: 0.685\n",
      "Epoch 455: | Loss: 0.53969 | Acc: 0.726 | Val ACC: 0.685\n",
      "Epoch 456: | Loss: 0.53974 | Acc: 0.723 | Val ACC: 0.694\n",
      "Epoch 457: | Loss: 0.54321 | Acc: 0.722 | Val ACC: 0.685\n",
      "Epoch 458: | Loss: 0.54492 | Acc: 0.722 | Val ACC: 0.688\n",
      "Epoch 459: | Loss: 0.54989 | Acc: 0.720 | Val ACC: 0.683\n",
      "Epoch 460: | Loss: 0.54878 | Acc: 0.716 | Val ACC: 0.687\n",
      "Epoch 461: | Loss: 0.54600 | Acc: 0.725 | Val ACC: 0.692\n",
      "Epoch 462: | Loss: 0.54607 | Acc: 0.720 | Val ACC: 0.681\n",
      "Epoch 463: | Loss: 0.54433 | Acc: 0.720 | Val ACC: 0.698\n",
      "Epoch 464: | Loss: 0.54566 | Acc: 0.725 | Val ACC: 0.673\n",
      "Epoch 465: | Loss: 0.55212 | Acc: 0.710 | Val ACC: 0.681\n",
      "Epoch 466: | Loss: 0.55463 | Acc: 0.705 | Val ACC: 0.673\n",
      "Epoch 467: | Loss: 0.54555 | Acc: 0.721 | Val ACC: 0.681\n",
      "Epoch 468: | Loss: 0.54580 | Acc: 0.719 | Val ACC: 0.704\n",
      "Epoch 469: | Loss: 0.53967 | Acc: 0.727 | Val ACC: 0.694\n",
      "Epoch 470: | Loss: 0.54688 | Acc: 0.720 | Val ACC: 0.683\n",
      "Epoch 471: | Loss: 0.55421 | Acc: 0.709 | Val ACC: 0.685\n",
      "Epoch 472: | Loss: 0.55328 | Acc: 0.710 | Val ACC: 0.685\n",
      "Epoch 473: | Loss: 0.54137 | Acc: 0.720 | Val ACC: 0.685\n",
      "Epoch 474: | Loss: 0.54067 | Acc: 0.717 | Val ACC: 0.692\n",
      "Epoch 475: | Loss: 0.54795 | Acc: 0.715 | Val ACC: 0.683\n",
      "Epoch 476: | Loss: 0.55174 | Acc: 0.715 | Val ACC: 0.679\n",
      "Epoch 477: | Loss: 0.54310 | Acc: 0.724 | Val ACC: 0.688\n",
      "Epoch 478: | Loss: 0.53883 | Acc: 0.721 | Val ACC: 0.687\n",
      "Epoch 479: | Loss: 0.53878 | Acc: 0.718 | Val ACC: 0.675\n",
      "Epoch 480: | Loss: 0.53963 | Acc: 0.720 | Val ACC: 0.679\n",
      "Epoch 481: | Loss: 0.54011 | Acc: 0.723 | Val ACC: 0.665\n",
      "Epoch 482: | Loss: 0.53853 | Acc: 0.724 | Val ACC: 0.679\n",
      "Epoch 483: | Loss: 0.54061 | Acc: 0.722 | Val ACC: 0.665\n",
      "Epoch 484: | Loss: 0.53892 | Acc: 0.724 | Val ACC: 0.677\n",
      "Epoch 485: | Loss: 0.54044 | Acc: 0.724 | Val ACC: 0.681\n",
      "Epoch 486: | Loss: 0.53865 | Acc: 0.727 | Val ACC: 0.673\n",
      "Epoch 487: | Loss: 0.53701 | Acc: 0.724 | Val ACC: 0.681\n",
      "Epoch 488: | Loss: 0.53779 | Acc: 0.725 | Val ACC: 0.675\n",
      "Epoch 489: | Loss: 0.54109 | Acc: 0.728 | Val ACC: 0.681\n",
      "Epoch 490: | Loss: 0.53690 | Acc: 0.728 | Val ACC: 0.681\n",
      "Epoch 491: | Loss: 0.53911 | Acc: 0.728 | Val ACC: 0.671\n",
      "Epoch 492: | Loss: 0.53591 | Acc: 0.731 | Val ACC: 0.681\n",
      "Epoch 493: | Loss: 0.53501 | Acc: 0.729 | Val ACC: 0.665\n",
      "Epoch 494: | Loss: 0.53584 | Acc: 0.729 | Val ACC: 0.671\n",
      "Epoch 495: | Loss: 0.53799 | Acc: 0.730 | Val ACC: 0.673\n",
      "Epoch 496: | Loss: 0.53633 | Acc: 0.727 | Val ACC: 0.669\n",
      "Epoch 497: | Loss: 0.53889 | Acc: 0.731 | Val ACC: 0.679\n",
      "Epoch 498: | Loss: 0.53535 | Acc: 0.732 | Val ACC: 0.669\n",
      "Epoch 499: | Loss: 0.53392 | Acc: 0.727 | Val ACC: 0.685\n",
      "Epoch 500: | Loss: 0.53287 | Acc: 0.735 | Val ACC: 0.660\n",
      "Epoch 501: | Loss: 0.53856 | Acc: 0.729 | Val ACC: 0.665\n",
      "Epoch 502: | Loss: 0.53742 | Acc: 0.728 | Val ACC: 0.660\n",
      "Epoch 503: | Loss: 0.53522 | Acc: 0.730 | Val ACC: 0.658\n",
      "Epoch 504: | Loss: 0.54461 | Acc: 0.724 | Val ACC: 0.662\n",
      "Epoch 505: | Loss: 0.53733 | Acc: 0.727 | Val ACC: 0.669\n",
      "Epoch 506: | Loss: 0.54168 | Acc: 0.726 | Val ACC: 0.652\n",
      "Epoch 507: | Loss: 0.54238 | Acc: 0.720 | Val ACC: 0.665\n",
      "Epoch 508: | Loss: 0.53693 | Acc: 0.727 | Val ACC: 0.667\n",
      "Epoch 509: | Loss: 0.53777 | Acc: 0.724 | Val ACC: 0.665\n",
      "Epoch 510: | Loss: 0.52992 | Acc: 0.735 | Val ACC: 0.673\n",
      "Epoch 511: | Loss: 0.53579 | Acc: 0.731 | Val ACC: 0.667\n",
      "Epoch 512: | Loss: 0.54332 | Acc: 0.727 | Val ACC: 0.662\n",
      "Epoch 513: | Loss: 0.53350 | Acc: 0.735 | Val ACC: 0.673\n",
      "Epoch 514: | Loss: 0.53993 | Acc: 0.730 | Val ACC: 0.665\n",
      "Epoch 515: | Loss: 0.54522 | Acc: 0.724 | Val ACC: 0.663\n",
      "Epoch 516: | Loss: 0.54915 | Acc: 0.719 | Val ACC: 0.665\n",
      "Epoch 517: | Loss: 0.53828 | Acc: 0.719 | Val ACC: 0.687\n",
      "Epoch 518: | Loss: 0.52014 | Acc: 0.739 | Val ACC: 0.669\n",
      "Epoch 519: | Loss: 0.52330 | Acc: 0.733 | Val ACC: 0.669\n",
      "Epoch 520: | Loss: 0.53042 | Acc: 0.739 | Val ACC: 0.667\n",
      "Epoch 521: | Loss: 0.53007 | Acc: 0.739 | Val ACC: 0.671\n",
      "Epoch 522: | Loss: 0.52866 | Acc: 0.738 | Val ACC: 0.667\n",
      "Epoch 523: | Loss: 0.53199 | Acc: 0.729 | Val ACC: 0.671\n",
      "Epoch 524: | Loss: 0.53271 | Acc: 0.733 | Val ACC: 0.675\n",
      "Epoch 525: | Loss: 0.52924 | Acc: 0.733 | Val ACC: 0.667\n",
      "Epoch 526: | Loss: 0.52641 | Acc: 0.738 | Val ACC: 0.669\n",
      "Epoch 527: | Loss: 0.52774 | Acc: 0.734 | Val ACC: 0.677\n",
      "Epoch 528: | Loss: 0.52803 | Acc: 0.731 | Val ACC: 0.671\n",
      "Epoch 529: | Loss: 0.52249 | Acc: 0.736 | Val ACC: 0.663\n",
      "Epoch 530: | Loss: 0.52129 | Acc: 0.736 | Val ACC: 0.673\n",
      "Epoch 531: | Loss: 0.52758 | Acc: 0.733 | Val ACC: 0.665\n",
      "Epoch 532: | Loss: 0.53249 | Acc: 0.730 | Val ACC: 0.671\n",
      "Epoch 533: | Loss: 0.53435 | Acc: 0.733 | Val ACC: 0.669\n",
      "Epoch 534: | Loss: 0.53407 | Acc: 0.734 | Val ACC: 0.673\n",
      "Epoch 535: | Loss: 0.54578 | Acc: 0.717 | Val ACC: 0.675\n",
      "Epoch 536: | Loss: 0.55001 | Acc: 0.714 | Val ACC: 0.698\n",
      "Epoch 537: | Loss: 0.52686 | Acc: 0.736 | Val ACC: 0.679\n",
      "Epoch 538: | Loss: 0.52309 | Acc: 0.738 | Val ACC: 0.671\n",
      "Epoch 539: | Loss: 0.52487 | Acc: 0.739 | Val ACC: 0.673\n",
      "Epoch 540: | Loss: 0.53208 | Acc: 0.731 | Val ACC: 0.681\n",
      "Epoch 541: | Loss: 0.53238 | Acc: 0.736 | Val ACC: 0.673\n",
      "Epoch 542: | Loss: 0.53218 | Acc: 0.734 | Val ACC: 0.675\n",
      "Epoch 543: | Loss: 0.53048 | Acc: 0.734 | Val ACC: 0.677\n",
      "Epoch 544: | Loss: 0.51955 | Acc: 0.750 | Val ACC: 0.681\n",
      "Epoch 545: | Loss: 0.52225 | Acc: 0.744 | Val ACC: 0.675\n",
      "Epoch 546: | Loss: 0.52598 | Acc: 0.744 | Val ACC: 0.679\n",
      "Epoch 547: | Loss: 0.52097 | Acc: 0.745 | Val ACC: 0.681\n",
      "Epoch 548: | Loss: 0.51935 | Acc: 0.748 | Val ACC: 0.673\n",
      "Epoch 549: | Loss: 0.52662 | Acc: 0.742 | Val ACC: 0.677\n",
      "Epoch 550: | Loss: 0.52014 | Acc: 0.746 | Val ACC: 0.675\n",
      "Epoch 551: | Loss: 0.52429 | Acc: 0.744 | Val ACC: 0.679\n",
      "Epoch 552: | Loss: 0.52563 | Acc: 0.741 | Val ACC: 0.673\n",
      "Epoch 553: | Loss: 0.51954 | Acc: 0.746 | Val ACC: 0.687\n",
      "Epoch 554: | Loss: 0.52069 | Acc: 0.750 | Val ACC: 0.681\n",
      "Epoch 555: | Loss: 0.52115 | Acc: 0.746 | Val ACC: 0.679\n",
      "Epoch 556: | Loss: 0.51416 | Acc: 0.750 | Val ACC: 0.683\n",
      "Epoch 557: | Loss: 0.51539 | Acc: 0.750 | Val ACC: 0.687\n",
      "Epoch 558: | Loss: 0.51450 | Acc: 0.755 | Val ACC: 0.679\n",
      "Epoch 559: | Loss: 0.51374 | Acc: 0.751 | Val ACC: 0.685\n",
      "Epoch 560: | Loss: 0.51150 | Acc: 0.756 | Val ACC: 0.687\n",
      "Epoch 561: | Loss: 0.51032 | Acc: 0.754 | Val ACC: 0.688\n",
      "Epoch 562: | Loss: 0.50831 | Acc: 0.750 | Val ACC: 0.679\n",
      "Epoch 563: | Loss: 0.50921 | Acc: 0.748 | Val ACC: 0.679\n",
      "Epoch 564: | Loss: 0.50578 | Acc: 0.752 | Val ACC: 0.698\n",
      "Epoch 565: | Loss: 0.50553 | Acc: 0.756 | Val ACC: 0.688\n",
      "Epoch 566: | Loss: 0.50834 | Acc: 0.756 | Val ACC: 0.683\n",
      "Epoch 567: | Loss: 0.50674 | Acc: 0.759 | Val ACC: 0.698\n",
      "Epoch 568: | Loss: 0.50959 | Acc: 0.751 | Val ACC: 0.692\n",
      "Epoch 569: | Loss: 0.50264 | Acc: 0.760 | Val ACC: 0.692\n",
      "Epoch 570: | Loss: 0.50855 | Acc: 0.757 | Val ACC: 0.687\n",
      "Epoch 571: | Loss: 0.51097 | Acc: 0.749 | Val ACC: 0.694\n",
      "Epoch 572: | Loss: 0.50330 | Acc: 0.754 | Val ACC: 0.698\n",
      "Epoch 573: | Loss: 0.50405 | Acc: 0.749 | Val ACC: 0.696\n",
      "Epoch 574: | Loss: 0.50668 | Acc: 0.757 | Val ACC: 0.692\n",
      "Epoch 575: | Loss: 0.50822 | Acc: 0.753 | Val ACC: 0.692\n",
      "Epoch 576: | Loss: 0.51073 | Acc: 0.755 | Val ACC: 0.688\n",
      "Epoch 577: | Loss: 0.50950 | Acc: 0.753 | Val ACC: 0.692\n",
      "Epoch 578: | Loss: 0.50134 | Acc: 0.762 | Val ACC: 0.704\n",
      "Epoch 579: | Loss: 0.50449 | Acc: 0.756 | Val ACC: 0.687\n",
      "Epoch 580: | Loss: 0.50711 | Acc: 0.756 | Val ACC: 0.702\n",
      "Epoch 581: | Loss: 0.50277 | Acc: 0.757 | Val ACC: 0.692\n",
      "Epoch 582: | Loss: 0.50729 | Acc: 0.759 | Val ACC: 0.696\n",
      "Epoch 583: | Loss: 0.50321 | Acc: 0.758 | Val ACC: 0.704\n",
      "Epoch 584: | Loss: 0.50274 | Acc: 0.760 | Val ACC: 0.700\n",
      "Epoch 585: | Loss: 0.49712 | Acc: 0.763 | Val ACC: 0.696\n",
      "Epoch 586: | Loss: 0.50225 | Acc: 0.759 | Val ACC: 0.698\n",
      "Epoch 587: | Loss: 0.49977 | Acc: 0.758 | Val ACC: 0.681\n",
      "Epoch 588: | Loss: 0.50293 | Acc: 0.757 | Val ACC: 0.704\n",
      "Epoch 589: | Loss: 0.50178 | Acc: 0.756 | Val ACC: 0.681\n",
      "Epoch 590: | Loss: 0.50788 | Acc: 0.756 | Val ACC: 0.702\n",
      "Epoch 591: | Loss: 0.49581 | Acc: 0.765 | Val ACC: 0.698\n",
      "Epoch 592: | Loss: 0.50412 | Acc: 0.758 | Val ACC: 0.708\n",
      "Epoch 593: | Loss: 0.49593 | Acc: 0.765 | Val ACC: 0.710\n",
      "Epoch 594: | Loss: 0.51110 | Acc: 0.747 | Val ACC: 0.688\n",
      "Epoch 595: | Loss: 0.49851 | Acc: 0.755 | Val ACC: 0.706\n",
      "Epoch 596: | Loss: 0.49615 | Acc: 0.762 | Val ACC: 0.698\n",
      "Epoch 597: | Loss: 0.49791 | Acc: 0.759 | Val ACC: 0.712\n",
      "Epoch 598: | Loss: 0.49531 | Acc: 0.764 | Val ACC: 0.679\n",
      "Epoch 599: | Loss: 0.50357 | Acc: 0.756 | Val ACC: 0.723\n"
     ]
    }
   ],
   "source": [
    "print('Training Starts')\n",
    "print('-'*89)\n",
    "for epoch in range(600):\n",
    "    server.fit(epoch)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [],
   "source": [
    "history2 = server.get_history()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3 clients"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Client 0: Feature Index 0-124\n",
      "Client 1: Feature Index 125-249\n",
      "Client 2: Feature Index 250-374\n",
      "Server : Feature Index 375-499\n"
     ]
    }
   ],
   "source": [
    "client_id_list = list(range(3))\n",
    "courier = SyncLocalCourier(client_id_list)\n",
    "loader = SimpleNumpyDataLoader(\n",
    "    clients_id_list=client_id_list,\n",
    "    data_source=(X_np, y_np)\n",
    ")\n",
    "loader_dict = loader.distribute()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Starts\n",
      "-----------------------------------------------------------------------------------------\n",
      "Epoch 000: | Loss: 0.69350 | Acc: 0.499 | Val ACC: 0.506\n",
      "Epoch 001: | Loss: 0.69315 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 002: | Loss: 0.69313 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 003: | Loss: 0.69309 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 004: | Loss: 0.69306 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 005: | Loss: 0.69303 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 006: | Loss: 0.69299 | Acc: 0.502 | Val ACC: 0.512\n",
      "Epoch 007: | Loss: 0.69294 | Acc: 0.503 | Val ACC: 0.517\n",
      "Epoch 008: | Loss: 0.69288 | Acc: 0.503 | Val ACC: 0.525\n",
      "Epoch 009: | Loss: 0.69281 | Acc: 0.514 | Val ACC: 0.538\n",
      "Epoch 010: | Loss: 0.69271 | Acc: 0.512 | Val ACC: 0.571\n",
      "Epoch 011: | Loss: 0.69261 | Acc: 0.534 | Val ACC: 0.533\n",
      "Epoch 012: | Loss: 0.69246 | Acc: 0.516 | Val ACC: 0.567\n",
      "Epoch 013: | Loss: 0.69225 | Acc: 0.533 | Val ACC: 0.583\n",
      "Epoch 014: | Loss: 0.69204 | Acc: 0.541 | Val ACC: 0.558\n",
      "Epoch 015: | Loss: 0.69177 | Acc: 0.525 | Val ACC: 0.538\n",
      "Epoch 016: | Loss: 0.69146 | Acc: 0.524 | Val ACC: 0.544\n",
      "Epoch 017: | Loss: 0.69110 | Acc: 0.529 | Val ACC: 0.537\n",
      "Epoch 018: | Loss: 0.69055 | Acc: 0.538 | Val ACC: 0.546\n",
      "Epoch 019: | Loss: 0.68986 | Acc: 0.540 | Val ACC: 0.567\n",
      "Epoch 020: | Loss: 0.68899 | Acc: 0.550 | Val ACC: 0.573\n",
      "Epoch 021: | Loss: 0.68758 | Acc: 0.558 | Val ACC: 0.529\n",
      "Epoch 022: | Loss: 0.68762 | Acc: 0.557 | Val ACC: 0.538\n",
      "Epoch 023: | Loss: 0.68664 | Acc: 0.560 | Val ACC: 0.535\n",
      "Epoch 024: | Loss: 0.68613 | Acc: 0.557 | Val ACC: 0.538\n",
      "Epoch 025: | Loss: 0.68544 | Acc: 0.568 | Val ACC: 0.537\n",
      "Epoch 026: | Loss: 0.68400 | Acc: 0.567 | Val ACC: 0.531\n",
      "Epoch 027: | Loss: 0.68409 | Acc: 0.565 | Val ACC: 0.538\n",
      "Epoch 028: | Loss: 0.68343 | Acc: 0.567 | Val ACC: 0.538\n",
      "Epoch 029: | Loss: 0.68227 | Acc: 0.572 | Val ACC: 0.535\n",
      "Epoch 030: | Loss: 0.68235 | Acc: 0.569 | Val ACC: 0.544\n",
      "Epoch 031: | Loss: 0.68096 | Acc: 0.576 | Val ACC: 0.531\n",
      "Epoch 032: | Loss: 0.68177 | Acc: 0.573 | Val ACC: 0.548\n",
      "Epoch 033: | Loss: 0.67981 | Acc: 0.580 | Val ACC: 0.538\n",
      "Epoch 034: | Loss: 0.68041 | Acc: 0.576 | Val ACC: 0.546\n",
      "Epoch 035: | Loss: 0.67895 | Acc: 0.578 | Val ACC: 0.537\n",
      "Epoch 036: | Loss: 0.68009 | Acc: 0.575 | Val ACC: 0.546\n",
      "Epoch 037: | Loss: 0.67885 | Acc: 0.575 | Val ACC: 0.565\n",
      "Epoch 038: | Loss: 0.67818 | Acc: 0.582 | Val ACC: 0.540\n",
      "Epoch 039: | Loss: 0.67837 | Acc: 0.582 | Val ACC: 0.562\n",
      "Epoch 040: | Loss: 0.67694 | Acc: 0.586 | Val ACC: 0.544\n",
      "Epoch 041: | Loss: 0.67741 | Acc: 0.581 | Val ACC: 0.552\n",
      "Epoch 042: | Loss: 0.67794 | Acc: 0.580 | Val ACC: 0.560\n",
      "Epoch 043: | Loss: 0.67559 | Acc: 0.584 | Val ACC: 0.552\n",
      "Epoch 044: | Loss: 0.67650 | Acc: 0.585 | Val ACC: 0.560\n",
      "Epoch 045: | Loss: 0.67601 | Acc: 0.580 | Val ACC: 0.560\n",
      "Epoch 046: | Loss: 0.67556 | Acc: 0.585 | Val ACC: 0.558\n",
      "Epoch 047: | Loss: 0.67531 | Acc: 0.577 | Val ACC: 0.562\n",
      "Epoch 048: | Loss: 0.67529 | Acc: 0.583 | Val ACC: 0.556\n",
      "Epoch 049: | Loss: 0.67449 | Acc: 0.582 | Val ACC: 0.567\n",
      "Epoch 050: | Loss: 0.67444 | Acc: 0.588 | Val ACC: 0.565\n",
      "Epoch 051: | Loss: 0.67380 | Acc: 0.585 | Val ACC: 0.558\n",
      "Epoch 052: | Loss: 0.67427 | Acc: 0.590 | Val ACC: 0.573\n",
      "Epoch 053: | Loss: 0.67280 | Acc: 0.591 | Val ACC: 0.567\n",
      "Epoch 054: | Loss: 0.67336 | Acc: 0.591 | Val ACC: 0.567\n",
      "Epoch 055: | Loss: 0.67279 | Acc: 0.585 | Val ACC: 0.562\n",
      "Epoch 056: | Loss: 0.67251 | Acc: 0.594 | Val ACC: 0.562\n",
      "Epoch 057: | Loss: 0.67221 | Acc: 0.587 | Val ACC: 0.558\n",
      "Epoch 058: | Loss: 0.67253 | Acc: 0.593 | Val ACC: 0.560\n",
      "Epoch 059: | Loss: 0.67027 | Acc: 0.595 | Val ACC: 0.581\n",
      "Epoch 060: | Loss: 0.67272 | Acc: 0.591 | Val ACC: 0.562\n",
      "Epoch 061: | Loss: 0.67173 | Acc: 0.592 | Val ACC: 0.583\n",
      "Epoch 062: | Loss: 0.67162 | Acc: 0.594 | Val ACC: 0.562\n",
      "Epoch 063: | Loss: 0.67106 | Acc: 0.597 | Val ACC: 0.602\n",
      "Epoch 064: | Loss: 0.66963 | Acc: 0.590 | Val ACC: 0.550\n",
      "Epoch 065: | Loss: 0.67073 | Acc: 0.597 | Val ACC: 0.585\n",
      "Epoch 066: | Loss: 0.66860 | Acc: 0.591 | Val ACC: 0.562\n",
      "Epoch 067: | Loss: 0.66838 | Acc: 0.600 | Val ACC: 0.612\n",
      "Epoch 068: | Loss: 0.66885 | Acc: 0.591 | Val ACC: 0.565\n",
      "Epoch 069: | Loss: 0.66788 | Acc: 0.602 | Val ACC: 0.610\n",
      "Epoch 070: | Loss: 0.66637 | Acc: 0.593 | Val ACC: 0.567\n",
      "Epoch 071: | Loss: 0.66751 | Acc: 0.598 | Val ACC: 0.608\n",
      "Epoch 072: | Loss: 0.66565 | Acc: 0.593 | Val ACC: 0.569\n",
      "Epoch 073: | Loss: 0.66759 | Acc: 0.594 | Val ACC: 0.575\n",
      "Epoch 074: | Loss: 0.66333 | Acc: 0.603 | Val ACC: 0.590\n",
      "Epoch 075: | Loss: 0.66552 | Acc: 0.591 | Val ACC: 0.579\n",
      "Epoch 076: | Loss: 0.66155 | Acc: 0.601 | Val ACC: 0.585\n",
      "Epoch 077: | Loss: 0.66770 | Acc: 0.589 | Val ACC: 0.573\n",
      "Epoch 078: | Loss: 0.66427 | Acc: 0.603 | Val ACC: 0.590\n",
      "Epoch 079: | Loss: 0.66217 | Acc: 0.600 | Val ACC: 0.579\n",
      "Epoch 080: | Loss: 0.66175 | Acc: 0.597 | Val ACC: 0.577\n",
      "Epoch 081: | Loss: 0.66151 | Acc: 0.597 | Val ACC: 0.577\n",
      "Epoch 082: | Loss: 0.66112 | Acc: 0.600 | Val ACC: 0.592\n",
      "Epoch 083: | Loss: 0.65970 | Acc: 0.597 | Val ACC: 0.571\n",
      "Epoch 084: | Loss: 0.66116 | Acc: 0.597 | Val ACC: 0.577\n",
      "Epoch 085: | Loss: 0.65926 | Acc: 0.602 | Val ACC: 0.592\n",
      "Epoch 086: | Loss: 0.65890 | Acc: 0.596 | Val ACC: 0.571\n",
      "Epoch 087: | Loss: 0.66049 | Acc: 0.599 | Val ACC: 0.581\n",
      "Epoch 088: | Loss: 0.65687 | Acc: 0.607 | Val ACC: 0.581\n",
      "Epoch 089: | Loss: 0.65931 | Acc: 0.602 | Val ACC: 0.608\n",
      "Epoch 090: | Loss: 0.65437 | Acc: 0.608 | Val ACC: 0.577\n",
      "Epoch 091: | Loss: 0.66050 | Acc: 0.594 | Val ACC: 0.577\n",
      "Epoch 092: | Loss: 0.65559 | Acc: 0.608 | Val ACC: 0.571\n",
      "Epoch 093: | Loss: 0.65886 | Acc: 0.607 | Val ACC: 0.615\n",
      "Epoch 094: | Loss: 0.65046 | Acc: 0.613 | Val ACC: 0.575\n",
      "Epoch 095: | Loss: 0.66056 | Acc: 0.601 | Val ACC: 0.596\n",
      "Epoch 096: | Loss: 0.65068 | Acc: 0.610 | Val ACC: 0.587\n",
      "Epoch 097: | Loss: 0.65810 | Acc: 0.604 | Val ACC: 0.604\n",
      "Epoch 098: | Loss: 0.64971 | Acc: 0.614 | Val ACC: 0.598\n",
      "Epoch 099: | Loss: 0.65815 | Acc: 0.607 | Val ACC: 0.635\n",
      "Epoch 100: | Loss: 0.64499 | Acc: 0.617 | Val ACC: 0.610\n",
      "Epoch 101: | Loss: 0.65506 | Acc: 0.604 | Val ACC: 0.619\n",
      "Epoch 102: | Loss: 0.64733 | Acc: 0.621 | Val ACC: 0.596\n",
      "Epoch 103: | Loss: 0.65568 | Acc: 0.607 | Val ACC: 0.619\n",
      "Epoch 104: | Loss: 0.64724 | Acc: 0.618 | Val ACC: 0.585\n",
      "Epoch 105: | Loss: 0.65465 | Acc: 0.609 | Val ACC: 0.637\n",
      "Epoch 106: | Loss: 0.64394 | Acc: 0.617 | Val ACC: 0.604\n",
      "Epoch 107: | Loss: 0.65281 | Acc: 0.610 | Val ACC: 0.600\n",
      "Epoch 108: | Loss: 0.64876 | Acc: 0.614 | Val ACC: 0.592\n",
      "Epoch 109: | Loss: 0.65037 | Acc: 0.617 | Val ACC: 0.625\n",
      "Epoch 110: | Loss: 0.64412 | Acc: 0.619 | Val ACC: 0.600\n",
      "Epoch 111: | Loss: 0.65193 | Acc: 0.616 | Val ACC: 0.608\n",
      "Epoch 112: | Loss: 0.64500 | Acc: 0.619 | Val ACC: 0.585\n",
      "Epoch 113: | Loss: 0.64956 | Acc: 0.616 | Val ACC: 0.619\n",
      "Epoch 114: | Loss: 0.64293 | Acc: 0.622 | Val ACC: 0.615\n",
      "Epoch 115: | Loss: 0.64851 | Acc: 0.619 | Val ACC: 0.623\n",
      "Epoch 116: | Loss: 0.64192 | Acc: 0.621 | Val ACC: 0.598\n",
      "Epoch 117: | Loss: 0.64761 | Acc: 0.615 | Val ACC: 0.629\n",
      "Epoch 118: | Loss: 0.64136 | Acc: 0.622 | Val ACC: 0.617\n",
      "Epoch 119: | Loss: 0.64591 | Acc: 0.618 | Val ACC: 0.619\n",
      "Epoch 120: | Loss: 0.64187 | Acc: 0.621 | Val ACC: 0.613\n",
      "Epoch 121: | Loss: 0.64515 | Acc: 0.620 | Val ACC: 0.613\n",
      "Epoch 122: | Loss: 0.64207 | Acc: 0.620 | Val ACC: 0.606\n",
      "Epoch 123: | Loss: 0.64427 | Acc: 0.624 | Val ACC: 0.625\n",
      "Epoch 124: | Loss: 0.64026 | Acc: 0.625 | Val ACC: 0.613\n",
      "Epoch 125: | Loss: 0.64370 | Acc: 0.624 | Val ACC: 0.612\n",
      "Epoch 126: | Loss: 0.63973 | Acc: 0.622 | Val ACC: 0.615\n",
      "Epoch 127: | Loss: 0.64180 | Acc: 0.624 | Val ACC: 0.610\n",
      "Epoch 128: | Loss: 0.63945 | Acc: 0.628 | Val ACC: 0.615\n",
      "Epoch 129: | Loss: 0.63917 | Acc: 0.628 | Val ACC: 0.621\n",
      "Epoch 130: | Loss: 0.64086 | Acc: 0.629 | Val ACC: 0.625\n",
      "Epoch 131: | Loss: 0.63787 | Acc: 0.632 | Val ACC: 0.608\n",
      "Epoch 132: | Loss: 0.64107 | Acc: 0.626 | Val ACC: 0.612\n",
      "Epoch 133: | Loss: 0.63850 | Acc: 0.627 | Val ACC: 0.619\n",
      "Epoch 134: | Loss: 0.63892 | Acc: 0.628 | Val ACC: 0.604\n",
      "Epoch 135: | Loss: 0.63826 | Acc: 0.627 | Val ACC: 0.615\n",
      "Epoch 136: | Loss: 0.63996 | Acc: 0.627 | Val ACC: 0.627\n",
      "Epoch 137: | Loss: 0.63811 | Acc: 0.629 | Val ACC: 0.610\n",
      "Epoch 138: | Loss: 0.63982 | Acc: 0.628 | Val ACC: 0.621\n",
      "Epoch 139: | Loss: 0.63383 | Acc: 0.631 | Val ACC: 0.621\n",
      "Epoch 140: | Loss: 0.63721 | Acc: 0.632 | Val ACC: 0.621\n",
      "Epoch 141: | Loss: 0.63521 | Acc: 0.631 | Val ACC: 0.610\n",
      "Epoch 142: | Loss: 0.63641 | Acc: 0.633 | Val ACC: 0.625\n",
      "Epoch 143: | Loss: 0.63436 | Acc: 0.631 | Val ACC: 0.627\n",
      "Epoch 144: | Loss: 0.63533 | Acc: 0.634 | Val ACC: 0.633\n",
      "Epoch 145: | Loss: 0.63514 | Acc: 0.637 | Val ACC: 0.623\n",
      "Epoch 146: | Loss: 0.63391 | Acc: 0.632 | Val ACC: 0.633\n",
      "Epoch 147: | Loss: 0.63367 | Acc: 0.634 | Val ACC: 0.625\n",
      "Epoch 148: | Loss: 0.63387 | Acc: 0.642 | Val ACC: 0.627\n",
      "Epoch 149: | Loss: 0.63185 | Acc: 0.640 | Val ACC: 0.621\n",
      "Epoch 150: | Loss: 0.63245 | Acc: 0.643 | Val ACC: 0.623\n",
      "Epoch 151: | Loss: 0.63345 | Acc: 0.643 | Val ACC: 0.635\n",
      "Epoch 152: | Loss: 0.63389 | Acc: 0.640 | Val ACC: 0.631\n",
      "Epoch 153: | Loss: 0.63178 | Acc: 0.638 | Val ACC: 0.627\n",
      "Epoch 154: | Loss: 0.63269 | Acc: 0.638 | Val ACC: 0.621\n",
      "Epoch 155: | Loss: 0.63191 | Acc: 0.639 | Val ACC: 0.627\n",
      "Epoch 156: | Loss: 0.63039 | Acc: 0.641 | Val ACC: 0.629\n",
      "Epoch 157: | Loss: 0.63095 | Acc: 0.642 | Val ACC: 0.629\n",
      "Epoch 158: | Loss: 0.63132 | Acc: 0.645 | Val ACC: 0.625\n",
      "Epoch 159: | Loss: 0.63213 | Acc: 0.642 | Val ACC: 0.627\n",
      "Epoch 160: | Loss: 0.63029 | Acc: 0.644 | Val ACC: 0.625\n",
      "Epoch 161: | Loss: 0.63050 | Acc: 0.645 | Val ACC: 0.627\n",
      "Epoch 162: | Loss: 0.62877 | Acc: 0.647 | Val ACC: 0.623\n",
      "Epoch 163: | Loss: 0.62987 | Acc: 0.644 | Val ACC: 0.623\n",
      "Epoch 164: | Loss: 0.62786 | Acc: 0.643 | Val ACC: 0.627\n",
      "Epoch 165: | Loss: 0.62836 | Acc: 0.648 | Val ACC: 0.631\n",
      "Epoch 166: | Loss: 0.62859 | Acc: 0.647 | Val ACC: 0.627\n",
      "Epoch 167: | Loss: 0.63052 | Acc: 0.652 | Val ACC: 0.619\n",
      "Epoch 168: | Loss: 0.62854 | Acc: 0.645 | Val ACC: 0.631\n",
      "Epoch 169: | Loss: 0.62853 | Acc: 0.648 | Val ACC: 0.627\n",
      "Epoch 170: | Loss: 0.62642 | Acc: 0.652 | Val ACC: 0.623\n",
      "Epoch 171: | Loss: 0.62709 | Acc: 0.655 | Val ACC: 0.623\n",
      "Epoch 172: | Loss: 0.62809 | Acc: 0.650 | Val ACC: 0.629\n",
      "Epoch 173: | Loss: 0.62712 | Acc: 0.651 | Val ACC: 0.625\n",
      "Epoch 174: | Loss: 0.62662 | Acc: 0.657 | Val ACC: 0.623\n",
      "Epoch 175: | Loss: 0.62525 | Acc: 0.653 | Val ACC: 0.631\n",
      "Epoch 176: | Loss: 0.62724 | Acc: 0.646 | Val ACC: 0.625\n",
      "Epoch 177: | Loss: 0.62486 | Acc: 0.648 | Val ACC: 0.623\n",
      "Epoch 178: | Loss: 0.62509 | Acc: 0.650 | Val ACC: 0.621\n",
      "Epoch 179: | Loss: 0.62423 | Acc: 0.653 | Val ACC: 0.625\n",
      "Epoch 180: | Loss: 0.62103 | Acc: 0.659 | Val ACC: 0.631\n",
      "Epoch 181: | Loss: 0.62051 | Acc: 0.660 | Val ACC: 0.627\n",
      "Epoch 182: | Loss: 0.62476 | Acc: 0.659 | Val ACC: 0.623\n",
      "Epoch 183: | Loss: 0.62165 | Acc: 0.659 | Val ACC: 0.631\n",
      "Epoch 184: | Loss: 0.62253 | Acc: 0.653 | Val ACC: 0.631\n",
      "Epoch 185: | Loss: 0.61929 | Acc: 0.665 | Val ACC: 0.627\n",
      "Epoch 186: | Loss: 0.61994 | Acc: 0.662 | Val ACC: 0.623\n",
      "Epoch 187: | Loss: 0.62136 | Acc: 0.649 | Val ACC: 0.629\n",
      "Epoch 188: | Loss: 0.62551 | Acc: 0.647 | Val ACC: 0.627\n",
      "Epoch 189: | Loss: 0.61816 | Acc: 0.659 | Val ACC: 0.629\n",
      "Epoch 190: | Loss: 0.61772 | Acc: 0.663 | Val ACC: 0.633\n",
      "Epoch 191: | Loss: 0.62052 | Acc: 0.652 | Val ACC: 0.627\n",
      "Epoch 192: | Loss: 0.62192 | Acc: 0.655 | Val ACC: 0.627\n",
      "Epoch 193: | Loss: 0.61758 | Acc: 0.658 | Val ACC: 0.631\n",
      "Epoch 194: | Loss: 0.61638 | Acc: 0.665 | Val ACC: 0.631\n",
      "Epoch 195: | Loss: 0.61508 | Acc: 0.659 | Val ACC: 0.631\n",
      "Epoch 196: | Loss: 0.61877 | Acc: 0.656 | Val ACC: 0.623\n",
      "Epoch 197: | Loss: 0.61689 | Acc: 0.659 | Val ACC: 0.625\n",
      "Epoch 198: | Loss: 0.61638 | Acc: 0.662 | Val ACC: 0.627\n",
      "Epoch 199: | Loss: 0.61387 | Acc: 0.665 | Val ACC: 0.627\n",
      "Epoch 200: | Loss: 0.61454 | Acc: 0.665 | Val ACC: 0.642\n",
      "Epoch 201: | Loss: 0.61438 | Acc: 0.666 | Val ACC: 0.631\n",
      "Epoch 202: | Loss: 0.61385 | Acc: 0.665 | Val ACC: 0.623\n",
      "Epoch 203: | Loss: 0.61472 | Acc: 0.660 | Val ACC: 0.633\n",
      "Epoch 204: | Loss: 0.61507 | Acc: 0.659 | Val ACC: 0.627\n",
      "Epoch 205: | Loss: 0.61372 | Acc: 0.656 | Val ACC: 0.629\n",
      "Epoch 206: | Loss: 0.61523 | Acc: 0.660 | Val ACC: 0.627\n",
      "Epoch 207: | Loss: 0.61286 | Acc: 0.659 | Val ACC: 0.631\n",
      "Epoch 208: | Loss: 0.61156 | Acc: 0.667 | Val ACC: 0.638\n",
      "Epoch 209: | Loss: 0.60824 | Acc: 0.665 | Val ACC: 0.646\n",
      "Epoch 210: | Loss: 0.61047 | Acc: 0.665 | Val ACC: 0.637\n",
      "Epoch 211: | Loss: 0.61076 | Acc: 0.661 | Val ACC: 0.648\n",
      "Epoch 212: | Loss: 0.60989 | Acc: 0.665 | Val ACC: 0.638\n",
      "Epoch 213: | Loss: 0.61119 | Acc: 0.664 | Val ACC: 0.640\n",
      "Epoch 214: | Loss: 0.60962 | Acc: 0.659 | Val ACC: 0.637\n",
      "Epoch 215: | Loss: 0.60929 | Acc: 0.667 | Val ACC: 0.637\n",
      "Epoch 216: | Loss: 0.60888 | Acc: 0.662 | Val ACC: 0.640\n",
      "Epoch 217: | Loss: 0.61061 | Acc: 0.662 | Val ACC: 0.631\n",
      "Epoch 218: | Loss: 0.60997 | Acc: 0.665 | Val ACC: 0.638\n",
      "Epoch 219: | Loss: 0.60863 | Acc: 0.661 | Val ACC: 0.640\n",
      "Epoch 220: | Loss: 0.60818 | Acc: 0.671 | Val ACC: 0.644\n",
      "Epoch 221: | Loss: 0.60954 | Acc: 0.665 | Val ACC: 0.637\n",
      "Epoch 222: | Loss: 0.60827 | Acc: 0.669 | Val ACC: 0.640\n",
      "Epoch 223: | Loss: 0.60640 | Acc: 0.669 | Val ACC: 0.644\n",
      "Epoch 224: | Loss: 0.60529 | Acc: 0.673 | Val ACC: 0.644\n",
      "Epoch 225: | Loss: 0.60547 | Acc: 0.671 | Val ACC: 0.642\n",
      "Epoch 226: | Loss: 0.60416 | Acc: 0.671 | Val ACC: 0.648\n",
      "Epoch 227: | Loss: 0.60402 | Acc: 0.668 | Val ACC: 0.648\n",
      "Epoch 228: | Loss: 0.60432 | Acc: 0.672 | Val ACC: 0.640\n",
      "Epoch 229: | Loss: 0.60390 | Acc: 0.671 | Val ACC: 0.646\n",
      "Epoch 230: | Loss: 0.60521 | Acc: 0.670 | Val ACC: 0.642\n",
      "Epoch 231: | Loss: 0.60468 | Acc: 0.672 | Val ACC: 0.642\n",
      "Epoch 232: | Loss: 0.60153 | Acc: 0.666 | Val ACC: 0.648\n",
      "Epoch 233: | Loss: 0.60375 | Acc: 0.666 | Val ACC: 0.640\n",
      "Epoch 234: | Loss: 0.60161 | Acc: 0.671 | Val ACC: 0.652\n",
      "Epoch 235: | Loss: 0.60125 | Acc: 0.672 | Val ACC: 0.638\n",
      "Epoch 236: | Loss: 0.60030 | Acc: 0.668 | Val ACC: 0.652\n",
      "Epoch 237: | Loss: 0.60038 | Acc: 0.669 | Val ACC: 0.652\n",
      "Epoch 238: | Loss: 0.60234 | Acc: 0.669 | Val ACC: 0.642\n",
      "Epoch 239: | Loss: 0.60385 | Acc: 0.668 | Val ACC: 0.637\n",
      "Epoch 240: | Loss: 0.60520 | Acc: 0.665 | Val ACC: 0.646\n",
      "Epoch 241: | Loss: 0.60114 | Acc: 0.674 | Val ACC: 0.648\n",
      "Epoch 242: | Loss: 0.59937 | Acc: 0.681 | Val ACC: 0.648\n",
      "Epoch 243: | Loss: 0.59843 | Acc: 0.679 | Val ACC: 0.648\n",
      "Epoch 244: | Loss: 0.59796 | Acc: 0.676 | Val ACC: 0.652\n",
      "Epoch 245: | Loss: 0.59823 | Acc: 0.676 | Val ACC: 0.650\n",
      "Epoch 246: | Loss: 0.59774 | Acc: 0.674 | Val ACC: 0.646\n",
      "Epoch 247: | Loss: 0.59899 | Acc: 0.670 | Val ACC: 0.646\n",
      "Epoch 248: | Loss: 0.59829 | Acc: 0.675 | Val ACC: 0.642\n",
      "Epoch 249: | Loss: 0.59954 | Acc: 0.674 | Val ACC: 0.644\n",
      "Epoch 250: | Loss: 0.59840 | Acc: 0.678 | Val ACC: 0.644\n",
      "Epoch 251: | Loss: 0.59690 | Acc: 0.684 | Val ACC: 0.650\n",
      "Epoch 252: | Loss: 0.59390 | Acc: 0.686 | Val ACC: 0.652\n",
      "Epoch 253: | Loss: 0.59493 | Acc: 0.679 | Val ACC: 0.650\n",
      "Epoch 254: | Loss: 0.59600 | Acc: 0.676 | Val ACC: 0.646\n",
      "Epoch 255: | Loss: 0.59828 | Acc: 0.672 | Val ACC: 0.648\n",
      "Epoch 256: | Loss: 0.59686 | Acc: 0.682 | Val ACC: 0.648\n",
      "Epoch 257: | Loss: 0.59573 | Acc: 0.684 | Val ACC: 0.652\n",
      "Epoch 258: | Loss: 0.59305 | Acc: 0.685 | Val ACC: 0.654\n",
      "Epoch 259: | Loss: 0.59534 | Acc: 0.676 | Val ACC: 0.646\n",
      "Epoch 260: | Loss: 0.59502 | Acc: 0.682 | Val ACC: 0.654\n",
      "Epoch 261: | Loss: 0.59546 | Acc: 0.671 | Val ACC: 0.650\n",
      "Epoch 262: | Loss: 0.59503 | Acc: 0.681 | Val ACC: 0.646\n",
      "Epoch 263: | Loss: 0.59362 | Acc: 0.675 | Val ACC: 0.648\n",
      "Epoch 264: | Loss: 0.59297 | Acc: 0.679 | Val ACC: 0.650\n",
      "Epoch 265: | Loss: 0.59449 | Acc: 0.680 | Val ACC: 0.646\n",
      "Epoch 266: | Loss: 0.59225 | Acc: 0.678 | Val ACC: 0.652\n",
      "Epoch 267: | Loss: 0.58967 | Acc: 0.679 | Val ACC: 0.658\n",
      "Epoch 268: | Loss: 0.59242 | Acc: 0.674 | Val ACC: 0.656\n",
      "Epoch 269: | Loss: 0.59324 | Acc: 0.676 | Val ACC: 0.644\n",
      "Epoch 270: | Loss: 0.59473 | Acc: 0.678 | Val ACC: 0.644\n",
      "Epoch 271: | Loss: 0.59364 | Acc: 0.677 | Val ACC: 0.648\n",
      "Epoch 272: | Loss: 0.59380 | Acc: 0.682 | Val ACC: 0.652\n",
      "Epoch 273: | Loss: 0.59123 | Acc: 0.684 | Val ACC: 0.652\n",
      "Epoch 274: | Loss: 0.58870 | Acc: 0.679 | Val ACC: 0.658\n",
      "Epoch 275: | Loss: 0.58954 | Acc: 0.684 | Val ACC: 0.650\n",
      "Epoch 276: | Loss: 0.59149 | Acc: 0.683 | Val ACC: 0.648\n",
      "Epoch 277: | Loss: 0.59146 | Acc: 0.677 | Val ACC: 0.652\n",
      "Epoch 278: | Loss: 0.59343 | Acc: 0.677 | Val ACC: 0.650\n",
      "Epoch 279: | Loss: 0.59023 | Acc: 0.683 | Val ACC: 0.658\n",
      "Epoch 280: | Loss: 0.58643 | Acc: 0.683 | Val ACC: 0.656\n",
      "Epoch 281: | Loss: 0.58554 | Acc: 0.685 | Val ACC: 0.652\n",
      "Epoch 282: | Loss: 0.58480 | Acc: 0.686 | Val ACC: 0.660\n",
      "Epoch 283: | Loss: 0.58735 | Acc: 0.684 | Val ACC: 0.648\n",
      "Epoch 284: | Loss: 0.58827 | Acc: 0.683 | Val ACC: 0.646\n",
      "Epoch 285: | Loss: 0.58714 | Acc: 0.685 | Val ACC: 0.658\n",
      "Epoch 286: | Loss: 0.58688 | Acc: 0.685 | Val ACC: 0.652\n",
      "Epoch 287: | Loss: 0.59031 | Acc: 0.681 | Val ACC: 0.652\n",
      "Epoch 288: | Loss: 0.58777 | Acc: 0.683 | Val ACC: 0.648\n",
      "Epoch 289: | Loss: 0.58714 | Acc: 0.686 | Val ACC: 0.656\n",
      "Epoch 290: | Loss: 0.58661 | Acc: 0.687 | Val ACC: 0.650\n",
      "Epoch 291: | Loss: 0.58604 | Acc: 0.688 | Val ACC: 0.650\n",
      "Epoch 292: | Loss: 0.58330 | Acc: 0.688 | Val ACC: 0.656\n",
      "Epoch 293: | Loss: 0.58384 | Acc: 0.691 | Val ACC: 0.652\n",
      "Epoch 294: | Loss: 0.58441 | Acc: 0.686 | Val ACC: 0.648\n",
      "Epoch 295: | Loss: 0.58401 | Acc: 0.685 | Val ACC: 0.658\n",
      "Epoch 296: | Loss: 0.58283 | Acc: 0.689 | Val ACC: 0.650\n",
      "Epoch 297: | Loss: 0.58468 | Acc: 0.688 | Val ACC: 0.646\n",
      "Epoch 298: | Loss: 0.58623 | Acc: 0.686 | Val ACC: 0.648\n",
      "Epoch 299: | Loss: 0.58454 | Acc: 0.693 | Val ACC: 0.650\n",
      "Epoch 300: | Loss: 0.58272 | Acc: 0.695 | Val ACC: 0.652\n",
      "Epoch 301: | Loss: 0.57813 | Acc: 0.702 | Val ACC: 0.667\n",
      "Epoch 302: | Loss: 0.57576 | Acc: 0.701 | Val ACC: 0.658\n",
      "Epoch 303: | Loss: 0.57606 | Acc: 0.700 | Val ACC: 0.652\n",
      "Epoch 304: | Loss: 0.57772 | Acc: 0.699 | Val ACC: 0.658\n",
      "Epoch 305: | Loss: 0.57476 | Acc: 0.702 | Val ACC: 0.654\n",
      "Epoch 306: | Loss: 0.57596 | Acc: 0.701 | Val ACC: 0.662\n",
      "Epoch 307: | Loss: 0.57623 | Acc: 0.704 | Val ACC: 0.658\n",
      "Epoch 308: | Loss: 0.57701 | Acc: 0.702 | Val ACC: 0.656\n",
      "Epoch 309: | Loss: 0.57782 | Acc: 0.701 | Val ACC: 0.656\n",
      "Epoch 310: | Loss: 0.57571 | Acc: 0.704 | Val ACC: 0.660\n",
      "Epoch 311: | Loss: 0.57542 | Acc: 0.705 | Val ACC: 0.658\n",
      "Epoch 312: | Loss: 0.57453 | Acc: 0.708 | Val ACC: 0.662\n",
      "Epoch 313: | Loss: 0.57093 | Acc: 0.707 | Val ACC: 0.665\n",
      "Epoch 314: | Loss: 0.56811 | Acc: 0.712 | Val ACC: 0.662\n",
      "Epoch 315: | Loss: 0.56650 | Acc: 0.714 | Val ACC: 0.656\n",
      "Epoch 316: | Loss: 0.56698 | Acc: 0.708 | Val ACC: 0.660\n",
      "Epoch 317: | Loss: 0.56738 | Acc: 0.712 | Val ACC: 0.658\n",
      "Epoch 318: | Loss: 0.56714 | Acc: 0.705 | Val ACC: 0.656\n",
      "Epoch 319: | Loss: 0.56891 | Acc: 0.704 | Val ACC: 0.660\n",
      "Epoch 320: | Loss: 0.56770 | Acc: 0.702 | Val ACC: 0.662\n",
      "Epoch 321: | Loss: 0.57052 | Acc: 0.704 | Val ACC: 0.660\n",
      "Epoch 322: | Loss: 0.56878 | Acc: 0.708 | Val ACC: 0.656\n",
      "Epoch 323: | Loss: 0.56646 | Acc: 0.708 | Val ACC: 0.652\n",
      "Epoch 324: | Loss: 0.56613 | Acc: 0.710 | Val ACC: 0.662\n",
      "Epoch 325: | Loss: 0.56327 | Acc: 0.716 | Val ACC: 0.662\n",
      "Epoch 326: | Loss: 0.56237 | Acc: 0.718 | Val ACC: 0.658\n",
      "Epoch 327: | Loss: 0.56236 | Acc: 0.717 | Val ACC: 0.663\n",
      "Epoch 328: | Loss: 0.56088 | Acc: 0.710 | Val ACC: 0.662\n",
      "Epoch 329: | Loss: 0.56463 | Acc: 0.709 | Val ACC: 0.663\n",
      "Epoch 330: | Loss: 0.56076 | Acc: 0.712 | Val ACC: 0.654\n",
      "Epoch 331: | Loss: 0.56368 | Acc: 0.711 | Val ACC: 0.660\n",
      "Epoch 332: | Loss: 0.56013 | Acc: 0.710 | Val ACC: 0.662\n",
      "Epoch 333: | Loss: 0.56297 | Acc: 0.707 | Val ACC: 0.663\n",
      "Epoch 334: | Loss: 0.56032 | Acc: 0.710 | Val ACC: 0.660\n",
      "Epoch 335: | Loss: 0.56365 | Acc: 0.706 | Val ACC: 0.656\n",
      "Epoch 336: | Loss: 0.56163 | Acc: 0.710 | Val ACC: 0.656\n",
      "Epoch 337: | Loss: 0.56270 | Acc: 0.710 | Val ACC: 0.667\n",
      "Epoch 338: | Loss: 0.55819 | Acc: 0.720 | Val ACC: 0.667\n",
      "Epoch 339: | Loss: 0.55883 | Acc: 0.718 | Val ACC: 0.663\n",
      "Epoch 340: | Loss: 0.55846 | Acc: 0.718 | Val ACC: 0.656\n",
      "Epoch 341: | Loss: 0.55737 | Acc: 0.713 | Val ACC: 0.662\n",
      "Epoch 342: | Loss: 0.55667 | Acc: 0.719 | Val ACC: 0.663\n",
      "Epoch 343: | Loss: 0.55576 | Acc: 0.717 | Val ACC: 0.652\n",
      "Epoch 344: | Loss: 0.55482 | Acc: 0.717 | Val ACC: 0.658\n",
      "Epoch 345: | Loss: 0.55642 | Acc: 0.713 | Val ACC: 0.658\n",
      "Epoch 346: | Loss: 0.55934 | Acc: 0.711 | Val ACC: 0.660\n",
      "Epoch 347: | Loss: 0.55882 | Acc: 0.711 | Val ACC: 0.656\n",
      "Epoch 348: | Loss: 0.55371 | Acc: 0.714 | Val ACC: 0.665\n",
      "Epoch 349: | Loss: 0.55718 | Acc: 0.710 | Val ACC: 0.671\n",
      "Epoch 350: | Loss: 0.55424 | Acc: 0.718 | Val ACC: 0.662\n",
      "Epoch 351: | Loss: 0.55284 | Acc: 0.718 | Val ACC: 0.660\n",
      "Epoch 352: | Loss: 0.55331 | Acc: 0.718 | Val ACC: 0.662\n",
      "Epoch 353: | Loss: 0.55521 | Acc: 0.711 | Val ACC: 0.652\n",
      "Epoch 354: | Loss: 0.55628 | Acc: 0.711 | Val ACC: 0.667\n",
      "Epoch 355: | Loss: 0.55510 | Acc: 0.713 | Val ACC: 0.667\n",
      "Epoch 356: | Loss: 0.55342 | Acc: 0.723 | Val ACC: 0.677\n",
      "Epoch 357: | Loss: 0.55137 | Acc: 0.729 | Val ACC: 0.681\n",
      "Epoch 358: | Loss: 0.55261 | Acc: 0.724 | Val ACC: 0.663\n",
      "Epoch 359: | Loss: 0.55320 | Acc: 0.726 | Val ACC: 0.677\n",
      "Epoch 360: | Loss: 0.55295 | Acc: 0.726 | Val ACC: 0.671\n",
      "Epoch 361: | Loss: 0.55225 | Acc: 0.724 | Val ACC: 0.669\n",
      "Epoch 362: | Loss: 0.55046 | Acc: 0.726 | Val ACC: 0.679\n",
      "Epoch 363: | Loss: 0.54800 | Acc: 0.728 | Val ACC: 0.665\n",
      "Epoch 364: | Loss: 0.55181 | Acc: 0.724 | Val ACC: 0.677\n",
      "Epoch 365: | Loss: 0.55295 | Acc: 0.725 | Val ACC: 0.669\n",
      "Epoch 366: | Loss: 0.54818 | Acc: 0.729 | Val ACC: 0.673\n",
      "Epoch 367: | Loss: 0.54712 | Acc: 0.727 | Val ACC: 0.677\n",
      "Epoch 368: | Loss: 0.54891 | Acc: 0.724 | Val ACC: 0.675\n",
      "Epoch 369: | Loss: 0.54797 | Acc: 0.729 | Val ACC: 0.677\n",
      "Epoch 370: | Loss: 0.54735 | Acc: 0.727 | Val ACC: 0.669\n",
      "Epoch 371: | Loss: 0.54835 | Acc: 0.725 | Val ACC: 0.675\n",
      "Epoch 372: | Loss: 0.54577 | Acc: 0.723 | Val ACC: 0.667\n",
      "Epoch 373: | Loss: 0.54703 | Acc: 0.720 | Val ACC: 0.675\n",
      "Epoch 374: | Loss: 0.54280 | Acc: 0.722 | Val ACC: 0.675\n",
      "Epoch 375: | Loss: 0.54629 | Acc: 0.721 | Val ACC: 0.662\n",
      "Epoch 376: | Loss: 0.55056 | Acc: 0.717 | Val ACC: 0.679\n",
      "Epoch 377: | Loss: 0.54594 | Acc: 0.723 | Val ACC: 0.677\n",
      "Epoch 378: | Loss: 0.54689 | Acc: 0.724 | Val ACC: 0.665\n",
      "Epoch 379: | Loss: 0.54473 | Acc: 0.723 | Val ACC: 0.675\n",
      "Epoch 380: | Loss: 0.54395 | Acc: 0.723 | Val ACC: 0.673\n",
      "Epoch 381: | Loss: 0.54165 | Acc: 0.724 | Val ACC: 0.679\n",
      "Epoch 382: | Loss: 0.54421 | Acc: 0.724 | Val ACC: 0.667\n",
      "Epoch 383: | Loss: 0.54434 | Acc: 0.723 | Val ACC: 0.675\n",
      "Epoch 384: | Loss: 0.54346 | Acc: 0.724 | Val ACC: 0.673\n",
      "Epoch 385: | Loss: 0.54541 | Acc: 0.727 | Val ACC: 0.671\n",
      "Epoch 386: | Loss: 0.54522 | Acc: 0.727 | Val ACC: 0.667\n",
      "Epoch 387: | Loss: 0.54506 | Acc: 0.728 | Val ACC: 0.663\n",
      "Epoch 388: | Loss: 0.54760 | Acc: 0.729 | Val ACC: 0.667\n",
      "Epoch 389: | Loss: 0.54468 | Acc: 0.724 | Val ACC: 0.667\n",
      "Epoch 390: | Loss: 0.54890 | Acc: 0.727 | Val ACC: 0.675\n",
      "Epoch 391: | Loss: 0.54737 | Acc: 0.725 | Val ACC: 0.673\n",
      "Epoch 392: | Loss: 0.54889 | Acc: 0.727 | Val ACC: 0.679\n",
      "Epoch 393: | Loss: 0.54737 | Acc: 0.727 | Val ACC: 0.679\n",
      "Epoch 394: | Loss: 0.54864 | Acc: 0.729 | Val ACC: 0.677\n",
      "Epoch 395: | Loss: 0.54622 | Acc: 0.728 | Val ACC: 0.671\n",
      "Epoch 396: | Loss: 0.54657 | Acc: 0.728 | Val ACC: 0.685\n",
      "Epoch 397: | Loss: 0.54674 | Acc: 0.729 | Val ACC: 0.683\n",
      "Epoch 398: | Loss: 0.54544 | Acc: 0.728 | Val ACC: 0.679\n",
      "Epoch 399: | Loss: 0.54533 | Acc: 0.729 | Val ACC: 0.675\n",
      "Epoch 400: | Loss: 0.54732 | Acc: 0.725 | Val ACC: 0.681\n",
      "Epoch 401: | Loss: 0.54552 | Acc: 0.732 | Val ACC: 0.677\n",
      "Epoch 402: | Loss: 0.54743 | Acc: 0.729 | Val ACC: 0.681\n",
      "Epoch 403: | Loss: 0.55065 | Acc: 0.725 | Val ACC: 0.669\n",
      "Epoch 404: | Loss: 0.55064 | Acc: 0.729 | Val ACC: 0.673\n",
      "Epoch 405: | Loss: 0.54456 | Acc: 0.733 | Val ACC: 0.675\n",
      "Epoch 406: | Loss: 0.54143 | Acc: 0.733 | Val ACC: 0.683\n",
      "Epoch 407: | Loss: 0.54198 | Acc: 0.732 | Val ACC: 0.677\n",
      "Epoch 408: | Loss: 0.53851 | Acc: 0.736 | Val ACC: 0.675\n",
      "Epoch 409: | Loss: 0.53865 | Acc: 0.731 | Val ACC: 0.677\n",
      "Epoch 410: | Loss: 0.54053 | Acc: 0.731 | Val ACC: 0.673\n",
      "Epoch 411: | Loss: 0.54225 | Acc: 0.731 | Val ACC: 0.679\n",
      "Epoch 412: | Loss: 0.54227 | Acc: 0.735 | Val ACC: 0.675\n",
      "Epoch 413: | Loss: 0.54105 | Acc: 0.728 | Val ACC: 0.683\n",
      "Epoch 414: | Loss: 0.54373 | Acc: 0.730 | Val ACC: 0.681\n",
      "Epoch 415: | Loss: 0.54026 | Acc: 0.730 | Val ACC: 0.677\n",
      "Epoch 416: | Loss: 0.53954 | Acc: 0.733 | Val ACC: 0.690\n",
      "Epoch 417: | Loss: 0.53744 | Acc: 0.734 | Val ACC: 0.673\n",
      "Epoch 418: | Loss: 0.53998 | Acc: 0.734 | Val ACC: 0.688\n",
      "Epoch 419: | Loss: 0.53610 | Acc: 0.736 | Val ACC: 0.675\n",
      "Epoch 420: | Loss: 0.53646 | Acc: 0.735 | Val ACC: 0.673\n",
      "Epoch 421: | Loss: 0.53479 | Acc: 0.740 | Val ACC: 0.681\n",
      "Epoch 422: | Loss: 0.53661 | Acc: 0.736 | Val ACC: 0.696\n",
      "Epoch 423: | Loss: 0.53286 | Acc: 0.742 | Val ACC: 0.671\n",
      "Epoch 424: | Loss: 0.53281 | Acc: 0.738 | Val ACC: 0.677\n",
      "Epoch 425: | Loss: 0.53423 | Acc: 0.740 | Val ACC: 0.681\n",
      "Epoch 426: | Loss: 0.52934 | Acc: 0.740 | Val ACC: 0.679\n",
      "Epoch 427: | Loss: 0.52567 | Acc: 0.746 | Val ACC: 0.675\n",
      "Epoch 428: | Loss: 0.52612 | Acc: 0.740 | Val ACC: 0.669\n",
      "Epoch 429: | Loss: 0.52609 | Acc: 0.746 | Val ACC: 0.667\n",
      "Epoch 430: | Loss: 0.52621 | Acc: 0.743 | Val ACC: 0.679\n",
      "Epoch 431: | Loss: 0.52410 | Acc: 0.746 | Val ACC: 0.671\n",
      "Epoch 432: | Loss: 0.52527 | Acc: 0.741 | Val ACC: 0.677\n",
      "Epoch 433: | Loss: 0.53163 | Acc: 0.733 | Val ACC: 0.690\n",
      "Epoch 434: | Loss: 0.52302 | Acc: 0.743 | Val ACC: 0.669\n",
      "Epoch 435: | Loss: 0.52482 | Acc: 0.741 | Val ACC: 0.694\n",
      "Epoch 436: | Loss: 0.52354 | Acc: 0.739 | Val ACC: 0.675\n",
      "Epoch 437: | Loss: 0.52352 | Acc: 0.743 | Val ACC: 0.673\n",
      "Epoch 438: | Loss: 0.52075 | Acc: 0.742 | Val ACC: 0.679\n",
      "Epoch 439: | Loss: 0.52032 | Acc: 0.749 | Val ACC: 0.677\n",
      "Epoch 440: | Loss: 0.52006 | Acc: 0.745 | Val ACC: 0.675\n",
      "Epoch 441: | Loss: 0.51628 | Acc: 0.745 | Val ACC: 0.692\n",
      "Epoch 442: | Loss: 0.52664 | Acc: 0.735 | Val ACC: 0.681\n",
      "Epoch 443: | Loss: 0.52342 | Acc: 0.743 | Val ACC: 0.681\n",
      "Epoch 444: | Loss: 0.51787 | Acc: 0.748 | Val ACC: 0.675\n",
      "Epoch 445: | Loss: 0.51548 | Acc: 0.747 | Val ACC: 0.700\n",
      "Epoch 446: | Loss: 0.51547 | Acc: 0.744 | Val ACC: 0.683\n",
      "Epoch 447: | Loss: 0.52118 | Acc: 0.739 | Val ACC: 0.685\n",
      "Epoch 448: | Loss: 0.52953 | Acc: 0.734 | Val ACC: 0.679\n",
      "Epoch 449: | Loss: 0.51222 | Acc: 0.749 | Val ACC: 0.687\n",
      "Epoch 450: | Loss: 0.51439 | Acc: 0.750 | Val ACC: 0.667\n",
      "Epoch 451: | Loss: 0.51473 | Acc: 0.748 | Val ACC: 0.702\n",
      "Epoch 452: | Loss: 0.52269 | Acc: 0.738 | Val ACC: 0.696\n",
      "Epoch 453: | Loss: 0.51839 | Acc: 0.740 | Val ACC: 0.692\n",
      "Epoch 454: | Loss: 0.52567 | Acc: 0.736 | Val ACC: 0.694\n",
      "Epoch 455: | Loss: 0.53140 | Acc: 0.731 | Val ACC: 0.683\n",
      "Epoch 456: | Loss: 0.51658 | Acc: 0.745 | Val ACC: 0.665\n",
      "Epoch 457: | Loss: 0.51158 | Acc: 0.749 | Val ACC: 0.685\n",
      "Epoch 458: | Loss: 0.51096 | Acc: 0.749 | Val ACC: 0.696\n",
      "Epoch 459: | Loss: 0.52105 | Acc: 0.739 | Val ACC: 0.679\n",
      "Epoch 460: | Loss: 0.51654 | Acc: 0.739 | Val ACC: 0.688\n",
      "Epoch 461: | Loss: 0.51327 | Acc: 0.746 | Val ACC: 0.694\n",
      "Epoch 462: | Loss: 0.51417 | Acc: 0.737 | Val ACC: 0.687\n",
      "Epoch 463: | Loss: 0.51404 | Acc: 0.743 | Val ACC: 0.692\n",
      "Epoch 464: | Loss: 0.51448 | Acc: 0.741 | Val ACC: 0.683\n",
      "Epoch 465: | Loss: 0.51231 | Acc: 0.743 | Val ACC: 0.690\n",
      "Epoch 466: | Loss: 0.51383 | Acc: 0.739 | Val ACC: 0.704\n",
      "Epoch 467: | Loss: 0.51664 | Acc: 0.738 | Val ACC: 0.694\n",
      "Epoch 468: | Loss: 0.50954 | Acc: 0.744 | Val ACC: 0.690\n",
      "Epoch 469: | Loss: 0.51183 | Acc: 0.748 | Val ACC: 0.681\n",
      "Epoch 470: | Loss: 0.50901 | Acc: 0.745 | Val ACC: 0.704\n",
      "Epoch 471: | Loss: 0.51805 | Acc: 0.736 | Val ACC: 0.692\n",
      "Epoch 472: | Loss: 0.51604 | Acc: 0.739 | Val ACC: 0.683\n",
      "Epoch 473: | Loss: 0.51076 | Acc: 0.746 | Val ACC: 0.696\n",
      "Epoch 474: | Loss: 0.51413 | Acc: 0.742 | Val ACC: 0.700\n",
      "Epoch 475: | Loss: 0.51408 | Acc: 0.744 | Val ACC: 0.694\n",
      "Epoch 476: | Loss: 0.51436 | Acc: 0.742 | Val ACC: 0.685\n",
      "Epoch 477: | Loss: 0.51467 | Acc: 0.745 | Val ACC: 0.690\n",
      "Epoch 478: | Loss: 0.50777 | Acc: 0.752 | Val ACC: 0.683\n",
      "Epoch 479: | Loss: 0.50895 | Acc: 0.748 | Val ACC: 0.690\n",
      "Epoch 480: | Loss: 0.51119 | Acc: 0.746 | Val ACC: 0.694\n",
      "Epoch 481: | Loss: 0.50697 | Acc: 0.745 | Val ACC: 0.687\n",
      "Epoch 482: | Loss: 0.50701 | Acc: 0.754 | Val ACC: 0.700\n",
      "Epoch 483: | Loss: 0.51392 | Acc: 0.741 | Val ACC: 0.677\n",
      "Epoch 484: | Loss: 0.50873 | Acc: 0.755 | Val ACC: 0.683\n",
      "Epoch 485: | Loss: 0.51091 | Acc: 0.746 | Val ACC: 0.702\n",
      "Epoch 486: | Loss: 0.50751 | Acc: 0.754 | Val ACC: 0.696\n",
      "Epoch 487: | Loss: 0.51221 | Acc: 0.739 | Val ACC: 0.702\n",
      "Epoch 488: | Loss: 0.52075 | Acc: 0.737 | Val ACC: 0.675\n",
      "Epoch 489: | Loss: 0.51655 | Acc: 0.746 | Val ACC: 0.690\n",
      "Epoch 490: | Loss: 0.52255 | Acc: 0.743 | Val ACC: 0.677\n",
      "Epoch 491: | Loss: 0.51919 | Acc: 0.739 | Val ACC: 0.681\n",
      "Epoch 492: | Loss: 0.51569 | Acc: 0.748 | Val ACC: 0.692\n",
      "Epoch 493: | Loss: 0.52295 | Acc: 0.743 | Val ACC: 0.694\n",
      "Epoch 494: | Loss: 0.51406 | Acc: 0.752 | Val ACC: 0.690\n",
      "Epoch 495: | Loss: 0.50770 | Acc: 0.759 | Val ACC: 0.679\n",
      "Epoch 496: | Loss: 0.50049 | Acc: 0.759 | Val ACC: 0.679\n",
      "Epoch 497: | Loss: 0.50545 | Acc: 0.756 | Val ACC: 0.696\n",
      "Epoch 498: | Loss: 0.50553 | Acc: 0.755 | Val ACC: 0.692\n",
      "Epoch 499: | Loss: 0.51327 | Acc: 0.749 | Val ACC: 0.706\n",
      "Epoch 500: | Loss: 0.50938 | Acc: 0.755 | Val ACC: 0.704\n",
      "Epoch 501: | Loss: 0.50947 | Acc: 0.756 | Val ACC: 0.685\n",
      "Epoch 502: | Loss: 0.50595 | Acc: 0.762 | Val ACC: 0.702\n",
      "Epoch 503: | Loss: 0.50270 | Acc: 0.767 | Val ACC: 0.692\n",
      "Epoch 504: | Loss: 0.50410 | Acc: 0.758 | Val ACC: 0.694\n",
      "Epoch 505: | Loss: 0.50676 | Acc: 0.753 | Val ACC: 0.702\n",
      "Epoch 506: | Loss: 0.51028 | Acc: 0.760 | Val ACC: 0.706\n",
      "Epoch 507: | Loss: 0.50421 | Acc: 0.758 | Val ACC: 0.694\n",
      "Epoch 508: | Loss: 0.50695 | Acc: 0.758 | Val ACC: 0.692\n",
      "Epoch 509: | Loss: 0.52048 | Acc: 0.745 | Val ACC: 0.675\n",
      "Epoch 510: | Loss: 0.51056 | Acc: 0.755 | Val ACC: 0.715\n",
      "Epoch 511: | Loss: 0.49200 | Acc: 0.767 | Val ACC: 0.696\n",
      "Epoch 512: | Loss: 0.49324 | Acc: 0.771 | Val ACC: 0.704\n",
      "Epoch 513: | Loss: 0.50105 | Acc: 0.761 | Val ACC: 0.698\n",
      "Epoch 514: | Loss: 0.50647 | Acc: 0.752 | Val ACC: 0.698\n",
      "Epoch 515: | Loss: 0.50511 | Acc: 0.758 | Val ACC: 0.706\n",
      "Epoch 516: | Loss: 0.49742 | Acc: 0.762 | Val ACC: 0.702\n",
      "Epoch 517: | Loss: 0.49842 | Acc: 0.763 | Val ACC: 0.696\n",
      "Epoch 518: | Loss: 0.49864 | Acc: 0.764 | Val ACC: 0.704\n",
      "Epoch 519: | Loss: 0.49854 | Acc: 0.761 | Val ACC: 0.692\n",
      "Epoch 520: | Loss: 0.49269 | Acc: 0.777 | Val ACC: 0.704\n",
      "Epoch 521: | Loss: 0.49256 | Acc: 0.770 | Val ACC: 0.698\n",
      "Epoch 522: | Loss: 0.50054 | Acc: 0.761 | Val ACC: 0.708\n",
      "Epoch 523: | Loss: 0.50079 | Acc: 0.762 | Val ACC: 0.706\n",
      "Epoch 524: | Loss: 0.50503 | Acc: 0.756 | Val ACC: 0.696\n",
      "Epoch 525: | Loss: 0.49914 | Acc: 0.762 | Val ACC: 0.696\n",
      "Epoch 526: | Loss: 0.48850 | Acc: 0.773 | Val ACC: 0.704\n",
      "Epoch 527: | Loss: 0.49156 | Acc: 0.766 | Val ACC: 0.702\n",
      "Epoch 528: | Loss: 0.49143 | Acc: 0.769 | Val ACC: 0.694\n",
      "Epoch 529: | Loss: 0.49321 | Acc: 0.762 | Val ACC: 0.712\n",
      "Epoch 530: | Loss: 0.50360 | Acc: 0.757 | Val ACC: 0.702\n",
      "Epoch 531: | Loss: 0.49114 | Acc: 0.767 | Val ACC: 0.712\n",
      "Epoch 532: | Loss: 0.48967 | Acc: 0.771 | Val ACC: 0.685\n",
      "Epoch 533: | Loss: 0.48697 | Acc: 0.774 | Val ACC: 0.710\n",
      "Epoch 534: | Loss: 0.48640 | Acc: 0.773 | Val ACC: 0.692\n",
      "Epoch 535: | Loss: 0.49042 | Acc: 0.765 | Val ACC: 0.700\n",
      "Epoch 536: | Loss: 0.48887 | Acc: 0.764 | Val ACC: 0.696\n",
      "Epoch 537: | Loss: 0.49943 | Acc: 0.759 | Val ACC: 0.713\n",
      "Epoch 538: | Loss: 0.48953 | Acc: 0.763 | Val ACC: 0.706\n",
      "Epoch 539: | Loss: 0.48581 | Acc: 0.774 | Val ACC: 0.692\n",
      "Epoch 540: | Loss: 0.48374 | Acc: 0.774 | Val ACC: 0.702\n",
      "Epoch 541: | Loss: 0.48384 | Acc: 0.775 | Val ACC: 0.710\n",
      "Epoch 542: | Loss: 0.48646 | Acc: 0.770 | Val ACC: 0.708\n",
      "Epoch 543: | Loss: 0.48788 | Acc: 0.775 | Val ACC: 0.712\n",
      "Epoch 544: | Loss: 0.48742 | Acc: 0.766 | Val ACC: 0.696\n",
      "Epoch 545: | Loss: 0.49191 | Acc: 0.770 | Val ACC: 0.708\n",
      "Epoch 546: | Loss: 0.48842 | Acc: 0.761 | Val ACC: 0.706\n",
      "Epoch 547: | Loss: 0.48736 | Acc: 0.767 | Val ACC: 0.690\n",
      "Epoch 548: | Loss: 0.48555 | Acc: 0.765 | Val ACC: 0.698\n",
      "Epoch 549: | Loss: 0.49214 | Acc: 0.767 | Val ACC: 0.694\n",
      "Epoch 550: | Loss: 0.48686 | Acc: 0.766 | Val ACC: 0.704\n",
      "Epoch 551: | Loss: 0.48732 | Acc: 0.768 | Val ACC: 0.698\n",
      "Epoch 552: | Loss: 0.47902 | Acc: 0.770 | Val ACC: 0.710\n",
      "Epoch 553: | Loss: 0.48197 | Acc: 0.776 | Val ACC: 0.700\n",
      "Epoch 554: | Loss: 0.47546 | Acc: 0.778 | Val ACC: 0.690\n",
      "Epoch 555: | Loss: 0.48598 | Acc: 0.771 | Val ACC: 0.700\n",
      "Epoch 556: | Loss: 0.48247 | Acc: 0.768 | Val ACC: 0.710\n",
      "Epoch 557: | Loss: 0.48562 | Acc: 0.765 | Val ACC: 0.698\n",
      "Epoch 558: | Loss: 0.48028 | Acc: 0.769 | Val ACC: 0.702\n",
      "Epoch 559: | Loss: 0.48288 | Acc: 0.769 | Val ACC: 0.702\n",
      "Epoch 560: | Loss: 0.48094 | Acc: 0.767 | Val ACC: 0.704\n",
      "Epoch 561: | Loss: 0.48416 | Acc: 0.773 | Val ACC: 0.696\n",
      "Epoch 562: | Loss: 0.47826 | Acc: 0.769 | Val ACC: 0.704\n",
      "Epoch 563: | Loss: 0.48211 | Acc: 0.772 | Val ACC: 0.696\n",
      "Epoch 564: | Loss: 0.47565 | Acc: 0.775 | Val ACC: 0.700\n",
      "Epoch 565: | Loss: 0.48047 | Acc: 0.772 | Val ACC: 0.704\n",
      "Epoch 566: | Loss: 0.47726 | Acc: 0.771 | Val ACC: 0.698\n",
      "Epoch 567: | Loss: 0.47868 | Acc: 0.777 | Val ACC: 0.708\n",
      "Epoch 568: | Loss: 0.47748 | Acc: 0.768 | Val ACC: 0.715\n",
      "Epoch 569: | Loss: 0.48012 | Acc: 0.773 | Val ACC: 0.704\n",
      "Epoch 570: | Loss: 0.47629 | Acc: 0.772 | Val ACC: 0.704\n",
      "Epoch 571: | Loss: 0.48107 | Acc: 0.771 | Val ACC: 0.700\n",
      "Epoch 572: | Loss: 0.47485 | Acc: 0.773 | Val ACC: 0.712\n",
      "Epoch 573: | Loss: 0.48798 | Acc: 0.766 | Val ACC: 0.690\n",
      "Epoch 574: | Loss: 0.47575 | Acc: 0.770 | Val ACC: 0.704\n",
      "Epoch 575: | Loss: 0.47865 | Acc: 0.773 | Val ACC: 0.687\n",
      "Epoch 576: | Loss: 0.47269 | Acc: 0.783 | Val ACC: 0.719\n",
      "Epoch 577: | Loss: 0.47643 | Acc: 0.769 | Val ACC: 0.702\n",
      "Epoch 578: | Loss: 0.47414 | Acc: 0.771 | Val ACC: 0.710\n",
      "Epoch 579: | Loss: 0.47857 | Acc: 0.772 | Val ACC: 0.715\n",
      "Epoch 580: | Loss: 0.47838 | Acc: 0.767 | Val ACC: 0.712\n",
      "Epoch 581: | Loss: 0.47998 | Acc: 0.774 | Val ACC: 0.704\n",
      "Epoch 582: | Loss: 0.47829 | Acc: 0.774 | Val ACC: 0.698\n",
      "Epoch 583: | Loss: 0.48356 | Acc: 0.767 | Val ACC: 0.704\n",
      "Epoch 584: | Loss: 0.48042 | Acc: 0.767 | Val ACC: 0.687\n",
      "Epoch 585: | Loss: 0.48195 | Acc: 0.767 | Val ACC: 0.698\n",
      "Epoch 586: | Loss: 0.47495 | Acc: 0.782 | Val ACC: 0.694\n",
      "Epoch 587: | Loss: 0.48293 | Acc: 0.769 | Val ACC: 0.704\n",
      "Epoch 588: | Loss: 0.47625 | Acc: 0.778 | Val ACC: 0.690\n",
      "Epoch 589: | Loss: 0.48704 | Acc: 0.767 | Val ACC: 0.698\n",
      "Epoch 590: | Loss: 0.48093 | Acc: 0.767 | Val ACC: 0.700\n",
      "Epoch 591: | Loss: 0.47750 | Acc: 0.769 | Val ACC: 0.706\n",
      "Epoch 592: | Loss: 0.47924 | Acc: 0.771 | Val ACC: 0.690\n",
      "Epoch 593: | Loss: 0.47912 | Acc: 0.771 | Val ACC: 0.710\n",
      "Epoch 594: | Loss: 0.47458 | Acc: 0.773 | Val ACC: 0.704\n",
      "Epoch 595: | Loss: 0.46916 | Acc: 0.779 | Val ACC: 0.710\n",
      "Epoch 596: | Loss: 0.47058 | Acc: 0.777 | Val ACC: 0.708\n",
      "Epoch 597: | Loss: 0.47104 | Acc: 0.773 | Val ACC: 0.706\n",
      "Epoch 598: | Loss: 0.47149 | Acc: 0.781 | Val ACC: 0.702\n",
      "Epoch 599: | Loss: 0.47385 | Acc: 0.777 | Val ACC: 0.706\n"
     ]
    }
   ],
   "source": [
    "clients = [\n",
    "    SyncFNNClient(id=id, \n",
    "    model=nn.Sequential(\n",
    "        nn.Linear(125, 64),\n",
    "        nn.ReLU(),\n",
    "        nn.Linear(64, 32),\n",
    "        nn.ReLU()),\n",
    "    courier=courier,\n",
    "    train_loader=loader_dict[id]['train_loader'],\n",
    "    test_loader=loader_dict[id]['test_loader'],\n",
    "    config_dir='simple_config.ini')\n",
    "for id in client_id_list]\n",
    "strategy = SyncConcatStrategy(courier=courier, clients=clients)\n",
    "server =  SyncFNNServer(\n",
    "    strategy=strategy,\n",
    "    courier=courier,\n",
    "    top_model=nn.Sequential(\n",
    "        nn.Linear(4 * 32, 32),\n",
    "        nn.ReLU(True),\n",
    "        nn.Linear(32, 1),\n",
    "        nn.Sigmoid()),\n",
    "    emb_model=nn.Sequential( \n",
    "        nn.Linear(125, 64),\n",
    "        nn.ReLU(),\n",
    "        nn.Linear(64, 32),\n",
    "        nn.ReLU()),\n",
    "    train_loader=loader_dict['server']['train_loader'],\n",
    "    test_loader=loader_dict['server']['test_loader'],\n",
    "    config_dir='simple_config.ini')\n",
    "\n",
    "print('Training Starts')\n",
    "print('-'*89)\n",
    "for epoch in range(600):\n",
    "    server.fit(epoch)\n",
    "history3 = server.get_history()\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4 clients"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Client 0: Feature Index 0-99\n",
      "Client 1: Feature Index 100-199\n",
      "Client 2: Feature Index 200-299\n",
      "Client 3: Feature Index 300-399\n",
      "Server : Feature Index 400-499\n"
     ]
    }
   ],
   "source": [
    "client_id_list = list(range(4))\n",
    "courier = SyncLocalCourier(client_id_list)\n",
    "loader = SimpleNumpyDataLoader(\n",
    "    clients_id_list=client_id_list,\n",
    "    data_source=(X_np, y_np)\n",
    ")\n",
    "loader_dict = loader.distribute()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Starts\n",
      "-----------------------------------------------------------------------------------------\n",
      "Epoch 000: | Loss: 0.69368 | Acc: 0.478 | Val ACC: 0.492\n",
      "Epoch 001: | Loss: 0.69321 | Acc: 0.506 | Val ACC: 0.506\n",
      "Epoch 002: | Loss: 0.69315 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 003: | Loss: 0.69312 | Acc: 0.504 | Val ACC: 0.506\n",
      "Epoch 004: | Loss: 0.69310 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 005: | Loss: 0.69306 | Acc: 0.503 | Val ACC: 0.506\n",
      "Epoch 006: | Loss: 0.69303 | Acc: 0.503 | Val ACC: 0.523\n",
      "Epoch 007: | Loss: 0.69299 | Acc: 0.513 | Val ACC: 0.515\n",
      "Epoch 008: | Loss: 0.69293 | Acc: 0.503 | Val ACC: 0.523\n",
      "Epoch 009: | Loss: 0.69286 | Acc: 0.507 | Val ACC: 0.552\n",
      "Epoch 010: | Loss: 0.69278 | Acc: 0.517 | Val ACC: 0.565\n",
      "Epoch 011: | Loss: 0.69265 | Acc: 0.519 | Val ACC: 0.560\n",
      "Epoch 012: | Loss: 0.69253 | Acc: 0.549 | Val ACC: 0.563\n",
      "Epoch 013: | Loss: 0.69235 | Acc: 0.543 | Val ACC: 0.558\n",
      "Epoch 014: | Loss: 0.69214 | Acc: 0.532 | Val ACC: 0.554\n",
      "Epoch 015: | Loss: 0.69183 | Acc: 0.550 | Val ACC: 0.560\n",
      "Epoch 016: | Loss: 0.69153 | Acc: 0.545 | Val ACC: 0.558\n",
      "Epoch 017: | Loss: 0.69122 | Acc: 0.536 | Val ACC: 0.552\n",
      "Epoch 018: | Loss: 0.69098 | Acc: 0.537 | Val ACC: 0.556\n",
      "Epoch 019: | Loss: 0.69049 | Acc: 0.535 | Val ACC: 0.562\n",
      "Epoch 020: | Loss: 0.69011 | Acc: 0.540 | Val ACC: 0.542\n",
      "Epoch 021: | Loss: 0.68972 | Acc: 0.539 | Val ACC: 0.535\n",
      "Epoch 022: | Loss: 0.68963 | Acc: 0.535 | Val ACC: 0.529\n",
      "Epoch 023: | Loss: 0.68990 | Acc: 0.542 | Val ACC: 0.548\n",
      "Epoch 024: | Loss: 0.68914 | Acc: 0.535 | Val ACC: 0.560\n",
      "Epoch 025: | Loss: 0.68839 | Acc: 0.545 | Val ACC: 0.556\n",
      "Epoch 026: | Loss: 0.68761 | Acc: 0.551 | Val ACC: 0.560\n",
      "Epoch 027: | Loss: 0.68684 | Acc: 0.559 | Val ACC: 0.565\n",
      "Epoch 028: | Loss: 0.68606 | Acc: 0.558 | Val ACC: 0.565\n",
      "Epoch 029: | Loss: 0.68440 | Acc: 0.567 | Val ACC: 0.542\n",
      "Epoch 030: | Loss: 0.68398 | Acc: 0.566 | Val ACC: 0.544\n",
      "Epoch 031: | Loss: 0.68408 | Acc: 0.568 | Val ACC: 0.538\n",
      "Epoch 032: | Loss: 0.68283 | Acc: 0.566 | Val ACC: 0.550\n",
      "Epoch 033: | Loss: 0.68349 | Acc: 0.569 | Val ACC: 0.546\n",
      "Epoch 034: | Loss: 0.68178 | Acc: 0.571 | Val ACC: 0.560\n",
      "Epoch 035: | Loss: 0.68261 | Acc: 0.567 | Val ACC: 0.560\n",
      "Epoch 036: | Loss: 0.68106 | Acc: 0.575 | Val ACC: 0.552\n",
      "Epoch 037: | Loss: 0.68079 | Acc: 0.576 | Val ACC: 0.563\n",
      "Epoch 038: | Loss: 0.68105 | Acc: 0.574 | Val ACC: 0.565\n",
      "Epoch 039: | Loss: 0.67972 | Acc: 0.580 | Val ACC: 0.554\n",
      "Epoch 040: | Loss: 0.67979 | Acc: 0.581 | Val ACC: 0.571\n",
      "Epoch 041: | Loss: 0.67960 | Acc: 0.582 | Val ACC: 0.563\n",
      "Epoch 042: | Loss: 0.67885 | Acc: 0.584 | Val ACC: 0.565\n",
      "Epoch 043: | Loss: 0.67874 | Acc: 0.582 | Val ACC: 0.565\n",
      "Epoch 044: | Loss: 0.67809 | Acc: 0.582 | Val ACC: 0.562\n",
      "Epoch 045: | Loss: 0.67780 | Acc: 0.585 | Val ACC: 0.563\n",
      "Epoch 046: | Loss: 0.67723 | Acc: 0.585 | Val ACC: 0.573\n",
      "Epoch 047: | Loss: 0.67657 | Acc: 0.584 | Val ACC: 0.573\n",
      "Epoch 048: | Loss: 0.67615 | Acc: 0.586 | Val ACC: 0.573\n",
      "Epoch 049: | Loss: 0.67605 | Acc: 0.580 | Val ACC: 0.579\n",
      "Epoch 050: | Loss: 0.67542 | Acc: 0.588 | Val ACC: 0.577\n",
      "Epoch 051: | Loss: 0.67500 | Acc: 0.586 | Val ACC: 0.577\n",
      "Epoch 052: | Loss: 0.67472 | Acc: 0.590 | Val ACC: 0.575\n",
      "Epoch 053: | Loss: 0.67448 | Acc: 0.586 | Val ACC: 0.575\n",
      "Epoch 054: | Loss: 0.67381 | Acc: 0.590 | Val ACC: 0.579\n",
      "Epoch 055: | Loss: 0.67393 | Acc: 0.587 | Val ACC: 0.577\n",
      "Epoch 056: | Loss: 0.67283 | Acc: 0.589 | Val ACC: 0.583\n",
      "Epoch 057: | Loss: 0.67321 | Acc: 0.585 | Val ACC: 0.587\n",
      "Epoch 058: | Loss: 0.67222 | Acc: 0.590 | Val ACC: 0.587\n",
      "Epoch 059: | Loss: 0.67194 | Acc: 0.588 | Val ACC: 0.577\n",
      "Epoch 060: | Loss: 0.67166 | Acc: 0.589 | Val ACC: 0.588\n",
      "Epoch 061: | Loss: 0.67049 | Acc: 0.592 | Val ACC: 0.577\n",
      "Epoch 062: | Loss: 0.67089 | Acc: 0.591 | Val ACC: 0.581\n",
      "Epoch 063: | Loss: 0.66976 | Acc: 0.593 | Val ACC: 0.585\n",
      "Epoch 064: | Loss: 0.66967 | Acc: 0.599 | Val ACC: 0.573\n",
      "Epoch 065: | Loss: 0.66871 | Acc: 0.599 | Val ACC: 0.581\n",
      "Epoch 066: | Loss: 0.66853 | Acc: 0.598 | Val ACC: 0.577\n",
      "Epoch 067: | Loss: 0.66729 | Acc: 0.598 | Val ACC: 0.583\n",
      "Epoch 068: | Loss: 0.66735 | Acc: 0.602 | Val ACC: 0.577\n",
      "Epoch 069: | Loss: 0.66611 | Acc: 0.599 | Val ACC: 0.588\n",
      "Epoch 070: | Loss: 0.66627 | Acc: 0.596 | Val ACC: 0.579\n",
      "Epoch 071: | Loss: 0.66639 | Acc: 0.593 | Val ACC: 0.567\n",
      "Epoch 072: | Loss: 0.66576 | Acc: 0.600 | Val ACC: 0.594\n",
      "Epoch 073: | Loss: 0.66426 | Acc: 0.597 | Val ACC: 0.563\n",
      "Epoch 074: | Loss: 0.66495 | Acc: 0.605 | Val ACC: 0.606\n",
      "Epoch 075: | Loss: 0.66256 | Acc: 0.602 | Val ACC: 0.583\n",
      "Epoch 076: | Loss: 0.66365 | Acc: 0.600 | Val ACC: 0.596\n",
      "Epoch 077: | Loss: 0.66226 | Acc: 0.599 | Val ACC: 0.590\n",
      "Epoch 078: | Loss: 0.66242 | Acc: 0.600 | Val ACC: 0.600\n",
      "Epoch 079: | Loss: 0.66217 | Acc: 0.598 | Val ACC: 0.579\n",
      "Epoch 080: | Loss: 0.66081 | Acc: 0.599 | Val ACC: 0.585\n",
      "Epoch 081: | Loss: 0.66065 | Acc: 0.600 | Val ACC: 0.588\n",
      "Epoch 082: | Loss: 0.66058 | Acc: 0.602 | Val ACC: 0.581\n",
      "Epoch 083: | Loss: 0.66025 | Acc: 0.606 | Val ACC: 0.598\n",
      "Epoch 084: | Loss: 0.65898 | Acc: 0.605 | Val ACC: 0.583\n",
      "Epoch 085: | Loss: 0.65912 | Acc: 0.608 | Val ACC: 0.612\n",
      "Epoch 086: | Loss: 0.65740 | Acc: 0.608 | Val ACC: 0.604\n",
      "Epoch 087: | Loss: 0.65665 | Acc: 0.606 | Val ACC: 0.610\n",
      "Epoch 088: | Loss: 0.65720 | Acc: 0.601 | Val ACC: 0.590\n",
      "Epoch 089: | Loss: 0.65745 | Acc: 0.604 | Val ACC: 0.594\n",
      "Epoch 090: | Loss: 0.65707 | Acc: 0.608 | Val ACC: 0.606\n",
      "Epoch 091: | Loss: 0.65574 | Acc: 0.606 | Val ACC: 0.602\n",
      "Epoch 092: | Loss: 0.65538 | Acc: 0.607 | Val ACC: 0.602\n",
      "Epoch 093: | Loss: 0.65493 | Acc: 0.607 | Val ACC: 0.608\n",
      "Epoch 094: | Loss: 0.65417 | Acc: 0.605 | Val ACC: 0.608\n",
      "Epoch 095: | Loss: 0.65336 | Acc: 0.604 | Val ACC: 0.602\n",
      "Epoch 096: | Loss: 0.65458 | Acc: 0.606 | Val ACC: 0.606\n",
      "Epoch 097: | Loss: 0.65269 | Acc: 0.605 | Val ACC: 0.608\n",
      "Epoch 098: | Loss: 0.65322 | Acc: 0.603 | Val ACC: 0.604\n",
      "Epoch 099: | Loss: 0.65302 | Acc: 0.608 | Val ACC: 0.606\n",
      "Epoch 100: | Loss: 0.65255 | Acc: 0.607 | Val ACC: 0.606\n",
      "Epoch 101: | Loss: 0.65278 | Acc: 0.615 | Val ACC: 0.604\n",
      "Epoch 102: | Loss: 0.65175 | Acc: 0.619 | Val ACC: 0.613\n",
      "Epoch 103: | Loss: 0.65182 | Acc: 0.616 | Val ACC: 0.617\n",
      "Epoch 104: | Loss: 0.65089 | Acc: 0.616 | Val ACC: 0.617\n",
      "Epoch 105: | Loss: 0.64937 | Acc: 0.617 | Val ACC: 0.615\n",
      "Epoch 106: | Loss: 0.64943 | Acc: 0.618 | Val ACC: 0.612\n",
      "Epoch 107: | Loss: 0.65005 | Acc: 0.618 | Val ACC: 0.619\n",
      "Epoch 108: | Loss: 0.64921 | Acc: 0.618 | Val ACC: 0.617\n",
      "Epoch 109: | Loss: 0.64880 | Acc: 0.619 | Val ACC: 0.615\n",
      "Epoch 110: | Loss: 0.64797 | Acc: 0.620 | Val ACC: 0.617\n",
      "Epoch 111: | Loss: 0.64783 | Acc: 0.620 | Val ACC: 0.619\n",
      "Epoch 112: | Loss: 0.64911 | Acc: 0.620 | Val ACC: 0.617\n",
      "Epoch 113: | Loss: 0.64830 | Acc: 0.624 | Val ACC: 0.615\n",
      "Epoch 114: | Loss: 0.64903 | Acc: 0.621 | Val ACC: 0.619\n",
      "Epoch 115: | Loss: 0.64794 | Acc: 0.626 | Val ACC: 0.619\n",
      "Epoch 116: | Loss: 0.64622 | Acc: 0.628 | Val ACC: 0.615\n",
      "Epoch 117: | Loss: 0.64639 | Acc: 0.630 | Val ACC: 0.617\n",
      "Epoch 118: | Loss: 0.64625 | Acc: 0.629 | Val ACC: 0.619\n",
      "Epoch 119: | Loss: 0.64605 | Acc: 0.628 | Val ACC: 0.615\n",
      "Epoch 120: | Loss: 0.64585 | Acc: 0.627 | Val ACC: 0.623\n",
      "Epoch 121: | Loss: 0.64572 | Acc: 0.630 | Val ACC: 0.621\n",
      "Epoch 122: | Loss: 0.64488 | Acc: 0.627 | Val ACC: 0.640\n",
      "Epoch 123: | Loss: 0.64375 | Acc: 0.632 | Val ACC: 0.625\n",
      "Epoch 124: | Loss: 0.64466 | Acc: 0.630 | Val ACC: 0.623\n",
      "Epoch 125: | Loss: 0.64408 | Acc: 0.631 | Val ACC: 0.621\n",
      "Epoch 126: | Loss: 0.64386 | Acc: 0.628 | Val ACC: 0.629\n",
      "Epoch 127: | Loss: 0.64258 | Acc: 0.631 | Val ACC: 0.638\n",
      "Epoch 128: | Loss: 0.64265 | Acc: 0.634 | Val ACC: 0.625\n",
      "Epoch 129: | Loss: 0.64439 | Acc: 0.627 | Val ACC: 0.635\n",
      "Epoch 130: | Loss: 0.64268 | Acc: 0.631 | Val ACC: 0.629\n",
      "Epoch 131: | Loss: 0.64391 | Acc: 0.630 | Val ACC: 0.638\n",
      "Epoch 132: | Loss: 0.64419 | Acc: 0.631 | Val ACC: 0.631\n",
      "Epoch 133: | Loss: 0.64273 | Acc: 0.630 | Val ACC: 0.644\n",
      "Epoch 134: | Loss: 0.64137 | Acc: 0.633 | Val ACC: 0.642\n",
      "Epoch 135: | Loss: 0.64138 | Acc: 0.636 | Val ACC: 0.633\n",
      "Epoch 136: | Loss: 0.64075 | Acc: 0.632 | Val ACC: 0.638\n",
      "Epoch 137: | Loss: 0.63948 | Acc: 0.634 | Val ACC: 0.640\n",
      "Epoch 138: | Loss: 0.64129 | Acc: 0.634 | Val ACC: 0.629\n",
      "Epoch 139: | Loss: 0.64138 | Acc: 0.634 | Val ACC: 0.644\n",
      "Epoch 140: | Loss: 0.64031 | Acc: 0.638 | Val ACC: 0.640\n",
      "Epoch 141: | Loss: 0.63969 | Acc: 0.637 | Val ACC: 0.642\n",
      "Epoch 142: | Loss: 0.63795 | Acc: 0.642 | Val ACC: 0.637\n",
      "Epoch 143: | Loss: 0.63920 | Acc: 0.641 | Val ACC: 0.638\n",
      "Epoch 144: | Loss: 0.63926 | Acc: 0.641 | Val ACC: 0.642\n",
      "Epoch 145: | Loss: 0.63825 | Acc: 0.641 | Val ACC: 0.640\n",
      "Epoch 146: | Loss: 0.63746 | Acc: 0.645 | Val ACC: 0.638\n",
      "Epoch 147: | Loss: 0.63860 | Acc: 0.645 | Val ACC: 0.637\n",
      "Epoch 148: | Loss: 0.63851 | Acc: 0.643 | Val ACC: 0.640\n",
      "Epoch 149: | Loss: 0.63728 | Acc: 0.643 | Val ACC: 0.637\n",
      "Epoch 150: | Loss: 0.63642 | Acc: 0.643 | Val ACC: 0.640\n",
      "Epoch 151: | Loss: 0.63523 | Acc: 0.647 | Val ACC: 0.629\n",
      "Epoch 152: | Loss: 0.63793 | Acc: 0.648 | Val ACC: 0.631\n",
      "Epoch 153: | Loss: 0.63677 | Acc: 0.646 | Val ACC: 0.635\n",
      "Epoch 154: | Loss: 0.63644 | Acc: 0.652 | Val ACC: 0.640\n",
      "Epoch 155: | Loss: 0.63532 | Acc: 0.653 | Val ACC: 0.640\n",
      "Epoch 156: | Loss: 0.63525 | Acc: 0.650 | Val ACC: 0.633\n",
      "Epoch 157: | Loss: 0.63523 | Acc: 0.649 | Val ACC: 0.642\n",
      "Epoch 158: | Loss: 0.63408 | Acc: 0.652 | Val ACC: 0.640\n",
      "Epoch 159: | Loss: 0.63497 | Acc: 0.653 | Val ACC: 0.631\n",
      "Epoch 160: | Loss: 0.63557 | Acc: 0.651 | Val ACC: 0.638\n",
      "Epoch 161: | Loss: 0.63407 | Acc: 0.651 | Val ACC: 0.642\n",
      "Epoch 162: | Loss: 0.63326 | Acc: 0.652 | Val ACC: 0.638\n",
      "Epoch 163: | Loss: 0.63436 | Acc: 0.653 | Val ACC: 0.638\n",
      "Epoch 164: | Loss: 0.63278 | Acc: 0.654 | Val ACC: 0.640\n",
      "Epoch 165: | Loss: 0.63282 | Acc: 0.655 | Val ACC: 0.637\n",
      "Epoch 166: | Loss: 0.63209 | Acc: 0.654 | Val ACC: 0.637\n",
      "Epoch 167: | Loss: 0.63235 | Acc: 0.655 | Val ACC: 0.638\n",
      "Epoch 168: | Loss: 0.63159 | Acc: 0.656 | Val ACC: 0.637\n",
      "Epoch 169: | Loss: 0.63241 | Acc: 0.659 | Val ACC: 0.635\n",
      "Epoch 170: | Loss: 0.63394 | Acc: 0.654 | Val ACC: 0.637\n",
      "Epoch 171: | Loss: 0.63152 | Acc: 0.653 | Val ACC: 0.637\n",
      "Epoch 172: | Loss: 0.63059 | Acc: 0.655 | Val ACC: 0.638\n",
      "Epoch 173: | Loss: 0.63007 | Acc: 0.660 | Val ACC: 0.640\n",
      "Epoch 174: | Loss: 0.63024 | Acc: 0.658 | Val ACC: 0.637\n",
      "Epoch 175: | Loss: 0.63053 | Acc: 0.659 | Val ACC: 0.637\n",
      "Epoch 176: | Loss: 0.63057 | Acc: 0.659 | Val ACC: 0.637\n",
      "Epoch 177: | Loss: 0.63003 | Acc: 0.656 | Val ACC: 0.638\n",
      "Epoch 178: | Loss: 0.62927 | Acc: 0.663 | Val ACC: 0.638\n",
      "Epoch 179: | Loss: 0.62863 | Acc: 0.659 | Val ACC: 0.638\n",
      "Epoch 180: | Loss: 0.62831 | Acc: 0.661 | Val ACC: 0.638\n",
      "Epoch 181: | Loss: 0.62805 | Acc: 0.662 | Val ACC: 0.642\n",
      "Epoch 182: | Loss: 0.62829 | Acc: 0.662 | Val ACC: 0.644\n",
      "Epoch 183: | Loss: 0.62745 | Acc: 0.659 | Val ACC: 0.640\n",
      "Epoch 184: | Loss: 0.62754 | Acc: 0.663 | Val ACC: 0.642\n",
      "Epoch 185: | Loss: 0.62708 | Acc: 0.664 | Val ACC: 0.638\n",
      "Epoch 186: | Loss: 0.62714 | Acc: 0.664 | Val ACC: 0.642\n",
      "Epoch 187: | Loss: 0.62654 | Acc: 0.665 | Val ACC: 0.644\n",
      "Epoch 188: | Loss: 0.62603 | Acc: 0.663 | Val ACC: 0.644\n",
      "Epoch 189: | Loss: 0.62600 | Acc: 0.663 | Val ACC: 0.642\n",
      "Epoch 190: | Loss: 0.62619 | Acc: 0.661 | Val ACC: 0.640\n",
      "Epoch 191: | Loss: 0.62642 | Acc: 0.660 | Val ACC: 0.644\n",
      "Epoch 192: | Loss: 0.62599 | Acc: 0.662 | Val ACC: 0.644\n",
      "Epoch 193: | Loss: 0.62555 | Acc: 0.663 | Val ACC: 0.646\n",
      "Epoch 194: | Loss: 0.62537 | Acc: 0.665 | Val ACC: 0.646\n",
      "Epoch 195: | Loss: 0.62496 | Acc: 0.665 | Val ACC: 0.646\n",
      "Epoch 196: | Loss: 0.62459 | Acc: 0.667 | Val ACC: 0.648\n",
      "Epoch 197: | Loss: 0.62423 | Acc: 0.665 | Val ACC: 0.646\n",
      "Epoch 198: | Loss: 0.62440 | Acc: 0.663 | Val ACC: 0.648\n",
      "Epoch 199: | Loss: 0.62403 | Acc: 0.663 | Val ACC: 0.648\n",
      "Epoch 200: | Loss: 0.62348 | Acc: 0.666 | Val ACC: 0.644\n",
      "Epoch 201: | Loss: 0.62328 | Acc: 0.665 | Val ACC: 0.646\n",
      "Epoch 202: | Loss: 0.62336 | Acc: 0.666 | Val ACC: 0.644\n",
      "Epoch 203: | Loss: 0.62233 | Acc: 0.665 | Val ACC: 0.646\n",
      "Epoch 204: | Loss: 0.62170 | Acc: 0.667 | Val ACC: 0.648\n",
      "Epoch 205: | Loss: 0.62072 | Acc: 0.667 | Val ACC: 0.650\n",
      "Epoch 206: | Loss: 0.61991 | Acc: 0.667 | Val ACC: 0.650\n",
      "Epoch 207: | Loss: 0.61907 | Acc: 0.668 | Val ACC: 0.652\n",
      "Epoch 208: | Loss: 0.61746 | Acc: 0.667 | Val ACC: 0.650\n",
      "Epoch 209: | Loss: 0.61848 | Acc: 0.667 | Val ACC: 0.652\n",
      "Epoch 210: | Loss: 0.61870 | Acc: 0.668 | Val ACC: 0.652\n",
      "Epoch 211: | Loss: 0.61864 | Acc: 0.668 | Val ACC: 0.654\n",
      "Epoch 212: | Loss: 0.61824 | Acc: 0.667 | Val ACC: 0.654\n",
      "Epoch 213: | Loss: 0.61788 | Acc: 0.666 | Val ACC: 0.654\n",
      "Epoch 214: | Loss: 0.61779 | Acc: 0.666 | Val ACC: 0.650\n",
      "Epoch 215: | Loss: 0.61833 | Acc: 0.667 | Val ACC: 0.652\n",
      "Epoch 216: | Loss: 0.61851 | Acc: 0.666 | Val ACC: 0.652\n",
      "Epoch 217: | Loss: 0.61843 | Acc: 0.668 | Val ACC: 0.652\n",
      "Epoch 218: | Loss: 0.61873 | Acc: 0.664 | Val ACC: 0.652\n",
      "Epoch 219: | Loss: 0.61845 | Acc: 0.667 | Val ACC: 0.654\n",
      "Epoch 220: | Loss: 0.61881 | Acc: 0.665 | Val ACC: 0.650\n",
      "Epoch 221: | Loss: 0.61869 | Acc: 0.668 | Val ACC: 0.646\n",
      "Epoch 222: | Loss: 0.61797 | Acc: 0.667 | Val ACC: 0.648\n",
      "Epoch 223: | Loss: 0.61725 | Acc: 0.665 | Val ACC: 0.648\n",
      "Epoch 224: | Loss: 0.61512 | Acc: 0.669 | Val ACC: 0.654\n",
      "Epoch 225: | Loss: 0.61543 | Acc: 0.667 | Val ACC: 0.650\n",
      "Epoch 226: | Loss: 0.61557 | Acc: 0.670 | Val ACC: 0.648\n",
      "Epoch 227: | Loss: 0.61503 | Acc: 0.670 | Val ACC: 0.650\n",
      "Epoch 228: | Loss: 0.61469 | Acc: 0.668 | Val ACC: 0.650\n",
      "Epoch 229: | Loss: 0.61528 | Acc: 0.667 | Val ACC: 0.652\n",
      "Epoch 230: | Loss: 0.61533 | Acc: 0.668 | Val ACC: 0.648\n",
      "Epoch 231: | Loss: 0.61486 | Acc: 0.670 | Val ACC: 0.650\n",
      "Epoch 232: | Loss: 0.61452 | Acc: 0.671 | Val ACC: 0.650\n",
      "Epoch 233: | Loss: 0.61425 | Acc: 0.666 | Val ACC: 0.648\n",
      "Epoch 234: | Loss: 0.61451 | Acc: 0.668 | Val ACC: 0.652\n",
      "Epoch 235: | Loss: 0.61455 | Acc: 0.669 | Val ACC: 0.652\n",
      "Epoch 236: | Loss: 0.61504 | Acc: 0.668 | Val ACC: 0.654\n",
      "Epoch 237: | Loss: 0.61521 | Acc: 0.669 | Val ACC: 0.654\n",
      "Epoch 238: | Loss: 0.61517 | Acc: 0.669 | Val ACC: 0.654\n",
      "Epoch 239: | Loss: 0.61408 | Acc: 0.671 | Val ACC: 0.648\n",
      "Epoch 240: | Loss: 0.61310 | Acc: 0.673 | Val ACC: 0.650\n",
      "Epoch 241: | Loss: 0.61263 | Acc: 0.674 | Val ACC: 0.650\n",
      "Epoch 242: | Loss: 0.61201 | Acc: 0.672 | Val ACC: 0.654\n",
      "Epoch 243: | Loss: 0.61198 | Acc: 0.670 | Val ACC: 0.656\n",
      "Epoch 244: | Loss: 0.61222 | Acc: 0.671 | Val ACC: 0.652\n",
      "Epoch 245: | Loss: 0.61299 | Acc: 0.669 | Val ACC: 0.650\n",
      "Epoch 246: | Loss: 0.61216 | Acc: 0.672 | Val ACC: 0.650\n",
      "Epoch 247: | Loss: 0.61200 | Acc: 0.670 | Val ACC: 0.652\n",
      "Epoch 248: | Loss: 0.61203 | Acc: 0.669 | Val ACC: 0.652\n",
      "Epoch 249: | Loss: 0.61227 | Acc: 0.668 | Val ACC: 0.652\n",
      "Epoch 250: | Loss: 0.61315 | Acc: 0.669 | Val ACC: 0.652\n",
      "Epoch 251: | Loss: 0.61334 | Acc: 0.669 | Val ACC: 0.654\n",
      "Epoch 252: | Loss: 0.61312 | Acc: 0.671 | Val ACC: 0.654\n",
      "Epoch 253: | Loss: 0.61238 | Acc: 0.675 | Val ACC: 0.654\n",
      "Epoch 254: | Loss: 0.61177 | Acc: 0.672 | Val ACC: 0.654\n",
      "Epoch 255: | Loss: 0.61143 | Acc: 0.674 | Val ACC: 0.652\n",
      "Epoch 256: | Loss: 0.61114 | Acc: 0.675 | Val ACC: 0.654\n",
      "Epoch 257: | Loss: 0.61167 | Acc: 0.672 | Val ACC: 0.652\n",
      "Epoch 258: | Loss: 0.61107 | Acc: 0.674 | Val ACC: 0.652\n",
      "Epoch 259: | Loss: 0.61131 | Acc: 0.669 | Val ACC: 0.652\n",
      "Epoch 260: | Loss: 0.61128 | Acc: 0.670 | Val ACC: 0.646\n",
      "Epoch 261: | Loss: 0.61166 | Acc: 0.672 | Val ACC: 0.646\n",
      "Epoch 262: | Loss: 0.61182 | Acc: 0.669 | Val ACC: 0.648\n",
      "Epoch 263: | Loss: 0.61218 | Acc: 0.670 | Val ACC: 0.652\n",
      "Epoch 264: | Loss: 0.61140 | Acc: 0.670 | Val ACC: 0.650\n",
      "Epoch 265: | Loss: 0.61043 | Acc: 0.673 | Val ACC: 0.648\n",
      "Epoch 266: | Loss: 0.60958 | Acc: 0.674 | Val ACC: 0.644\n",
      "Epoch 267: | Loss: 0.60865 | Acc: 0.672 | Val ACC: 0.640\n",
      "Epoch 268: | Loss: 0.60914 | Acc: 0.672 | Val ACC: 0.638\n",
      "Epoch 269: | Loss: 0.60938 | Acc: 0.673 | Val ACC: 0.640\n",
      "Epoch 270: | Loss: 0.60882 | Acc: 0.672 | Val ACC: 0.638\n",
      "Epoch 271: | Loss: 0.60867 | Acc: 0.671 | Val ACC: 0.638\n",
      "Epoch 272: | Loss: 0.60887 | Acc: 0.670 | Val ACC: 0.648\n",
      "Epoch 273: | Loss: 0.60762 | Acc: 0.672 | Val ACC: 0.652\n",
      "Epoch 274: | Loss: 0.60723 | Acc: 0.673 | Val ACC: 0.648\n",
      "Epoch 275: | Loss: 0.60670 | Acc: 0.678 | Val ACC: 0.654\n",
      "Epoch 276: | Loss: 0.60763 | Acc: 0.675 | Val ACC: 0.650\n",
      "Epoch 277: | Loss: 0.60892 | Acc: 0.672 | Val ACC: 0.650\n",
      "Epoch 278: | Loss: 0.60892 | Acc: 0.674 | Val ACC: 0.648\n",
      "Epoch 279: | Loss: 0.60831 | Acc: 0.672 | Val ACC: 0.652\n",
      "Epoch 280: | Loss: 0.60743 | Acc: 0.675 | Val ACC: 0.648\n",
      "Epoch 281: | Loss: 0.60734 | Acc: 0.673 | Val ACC: 0.650\n",
      "Epoch 282: | Loss: 0.60645 | Acc: 0.674 | Val ACC: 0.650\n",
      "Epoch 283: | Loss: 0.60594 | Acc: 0.676 | Val ACC: 0.652\n",
      "Epoch 284: | Loss: 0.60532 | Acc: 0.677 | Val ACC: 0.654\n",
      "Epoch 285: | Loss: 0.60519 | Acc: 0.674 | Val ACC: 0.656\n",
      "Epoch 286: | Loss: 0.60585 | Acc: 0.676 | Val ACC: 0.650\n",
      "Epoch 287: | Loss: 0.60645 | Acc: 0.675 | Val ACC: 0.648\n",
      "Epoch 288: | Loss: 0.60624 | Acc: 0.678 | Val ACC: 0.650\n",
      "Epoch 289: | Loss: 0.60596 | Acc: 0.677 | Val ACC: 0.650\n",
      "Epoch 290: | Loss: 0.60657 | Acc: 0.678 | Val ACC: 0.646\n",
      "Epoch 291: | Loss: 0.60452 | Acc: 0.679 | Val ACC: 0.656\n",
      "Epoch 292: | Loss: 0.60435 | Acc: 0.675 | Val ACC: 0.648\n",
      "Epoch 293: | Loss: 0.60498 | Acc: 0.676 | Val ACC: 0.646\n",
      "Epoch 294: | Loss: 0.60427 | Acc: 0.675 | Val ACC: 0.652\n",
      "Epoch 295: | Loss: 0.60459 | Acc: 0.674 | Val ACC: 0.646\n",
      "Epoch 296: | Loss: 0.60424 | Acc: 0.673 | Val ACC: 0.652\n",
      "Epoch 297: | Loss: 0.60329 | Acc: 0.675 | Val ACC: 0.648\n",
      "Epoch 298: | Loss: 0.60316 | Acc: 0.674 | Val ACC: 0.648\n",
      "Epoch 299: | Loss: 0.60471 | Acc: 0.669 | Val ACC: 0.646\n",
      "Epoch 300: | Loss: 0.60460 | Acc: 0.676 | Val ACC: 0.648\n",
      "Epoch 301: | Loss: 0.60603 | Acc: 0.671 | Val ACC: 0.644\n",
      "Epoch 302: | Loss: 0.60696 | Acc: 0.671 | Val ACC: 0.642\n",
      "Epoch 303: | Loss: 0.60587 | Acc: 0.672 | Val ACC: 0.648\n",
      "Epoch 304: | Loss: 0.60508 | Acc: 0.674 | Val ACC: 0.644\n",
      "Epoch 305: | Loss: 0.60559 | Acc: 0.675 | Val ACC: 0.644\n",
      "Epoch 306: | Loss: 0.60607 | Acc: 0.671 | Val ACC: 0.640\n",
      "Epoch 307: | Loss: 0.60571 | Acc: 0.673 | Val ACC: 0.642\n",
      "Epoch 308: | Loss: 0.60548 | Acc: 0.674 | Val ACC: 0.642\n",
      "Epoch 309: | Loss: 0.60472 | Acc: 0.676 | Val ACC: 0.642\n",
      "Epoch 310: | Loss: 0.60386 | Acc: 0.676 | Val ACC: 0.644\n",
      "Epoch 311: | Loss: 0.60404 | Acc: 0.676 | Val ACC: 0.644\n",
      "Epoch 312: | Loss: 0.60375 | Acc: 0.675 | Val ACC: 0.640\n",
      "Epoch 313: | Loss: 0.60337 | Acc: 0.677 | Val ACC: 0.646\n",
      "Epoch 314: | Loss: 0.60339 | Acc: 0.678 | Val ACC: 0.644\n",
      "Epoch 315: | Loss: 0.60337 | Acc: 0.680 | Val ACC: 0.644\n",
      "Epoch 316: | Loss: 0.60185 | Acc: 0.682 | Val ACC: 0.650\n",
      "Epoch 317: | Loss: 0.59899 | Acc: 0.683 | Val ACC: 0.652\n",
      "Epoch 318: | Loss: 0.60034 | Acc: 0.682 | Val ACC: 0.648\n",
      "Epoch 319: | Loss: 0.60061 | Acc: 0.683 | Val ACC: 0.652\n",
      "Epoch 320: | Loss: 0.59874 | Acc: 0.683 | Val ACC: 0.656\n",
      "Epoch 321: | Loss: 0.59825 | Acc: 0.683 | Val ACC: 0.656\n",
      "Epoch 322: | Loss: 0.59935 | Acc: 0.682 | Val ACC: 0.652\n",
      "Epoch 323: | Loss: 0.59943 | Acc: 0.682 | Val ACC: 0.652\n",
      "Epoch 324: | Loss: 0.59944 | Acc: 0.682 | Val ACC: 0.650\n",
      "Epoch 325: | Loss: 0.59960 | Acc: 0.682 | Val ACC: 0.652\n",
      "Epoch 326: | Loss: 0.59965 | Acc: 0.683 | Val ACC: 0.650\n",
      "Epoch 327: | Loss: 0.60024 | Acc: 0.679 | Val ACC: 0.648\n",
      "Epoch 328: | Loss: 0.60021 | Acc: 0.681 | Val ACC: 0.652\n",
      "Epoch 329: | Loss: 0.60090 | Acc: 0.679 | Val ACC: 0.650\n",
      "Epoch 330: | Loss: 0.59942 | Acc: 0.681 | Val ACC: 0.646\n",
      "Epoch 331: | Loss: 0.59869 | Acc: 0.683 | Val ACC: 0.658\n",
      "Epoch 332: | Loss: 0.59703 | Acc: 0.686 | Val ACC: 0.660\n",
      "Epoch 333: | Loss: 0.59620 | Acc: 0.688 | Val ACC: 0.660\n",
      "Epoch 334: | Loss: 0.59610 | Acc: 0.687 | Val ACC: 0.658\n",
      "Epoch 335: | Loss: 0.59566 | Acc: 0.684 | Val ACC: 0.660\n",
      "Epoch 336: | Loss: 0.59591 | Acc: 0.687 | Val ACC: 0.662\n",
      "Epoch 337: | Loss: 0.59580 | Acc: 0.684 | Val ACC: 0.660\n",
      "Epoch 338: | Loss: 0.59507 | Acc: 0.685 | Val ACC: 0.665\n",
      "Epoch 339: | Loss: 0.59496 | Acc: 0.684 | Val ACC: 0.662\n",
      "Epoch 340: | Loss: 0.59466 | Acc: 0.681 | Val ACC: 0.663\n",
      "Epoch 341: | Loss: 0.59455 | Acc: 0.683 | Val ACC: 0.663\n",
      "Epoch 342: | Loss: 0.59530 | Acc: 0.683 | Val ACC: 0.662\n",
      "Epoch 343: | Loss: 0.59811 | Acc: 0.680 | Val ACC: 0.658\n",
      "Epoch 344: | Loss: 0.60025 | Acc: 0.677 | Val ACC: 0.648\n",
      "Epoch 345: | Loss: 0.59804 | Acc: 0.683 | Val ACC: 0.662\n",
      "Epoch 346: | Loss: 0.59425 | Acc: 0.685 | Val ACC: 0.660\n",
      "Epoch 347: | Loss: 0.59373 | Acc: 0.684 | Val ACC: 0.669\n",
      "Epoch 348: | Loss: 0.59271 | Acc: 0.684 | Val ACC: 0.667\n",
      "Epoch 349: | Loss: 0.59401 | Acc: 0.683 | Val ACC: 0.665\n",
      "Epoch 350: | Loss: 0.59566 | Acc: 0.682 | Val ACC: 0.660\n",
      "Epoch 351: | Loss: 0.59761 | Acc: 0.680 | Val ACC: 0.652\n",
      "Epoch 352: | Loss: 0.59702 | Acc: 0.682 | Val ACC: 0.654\n",
      "Epoch 353: | Loss: 0.59371 | Acc: 0.685 | Val ACC: 0.665\n",
      "Epoch 354: | Loss: 0.59219 | Acc: 0.684 | Val ACC: 0.667\n",
      "Epoch 355: | Loss: 0.59112 | Acc: 0.685 | Val ACC: 0.669\n",
      "Epoch 356: | Loss: 0.59147 | Acc: 0.684 | Val ACC: 0.671\n",
      "Epoch 357: | Loss: 0.59160 | Acc: 0.685 | Val ACC: 0.667\n",
      "Epoch 358: | Loss: 0.59102 | Acc: 0.686 | Val ACC: 0.667\n",
      "Epoch 359: | Loss: 0.59030 | Acc: 0.686 | Val ACC: 0.665\n",
      "Epoch 360: | Loss: 0.58976 | Acc: 0.686 | Val ACC: 0.669\n",
      "Epoch 361: | Loss: 0.59102 | Acc: 0.686 | Val ACC: 0.667\n",
      "Epoch 362: | Loss: 0.59785 | Acc: 0.679 | Val ACC: 0.652\n",
      "Epoch 363: | Loss: 0.59276 | Acc: 0.684 | Val ACC: 0.667\n",
      "Epoch 364: | Loss: 0.58969 | Acc: 0.690 | Val ACC: 0.671\n",
      "Epoch 365: | Loss: 0.58996 | Acc: 0.688 | Val ACC: 0.669\n",
      "Epoch 366: | Loss: 0.59228 | Acc: 0.684 | Val ACC: 0.662\n",
      "Epoch 367: | Loss: 0.59470 | Acc: 0.684 | Val ACC: 0.660\n",
      "Epoch 368: | Loss: 0.59271 | Acc: 0.687 | Val ACC: 0.663\n",
      "Epoch 369: | Loss: 0.59102 | Acc: 0.687 | Val ACC: 0.667\n",
      "Epoch 370: | Loss: 0.59063 | Acc: 0.689 | Val ACC: 0.667\n",
      "Epoch 371: | Loss: 0.58821 | Acc: 0.691 | Val ACC: 0.665\n",
      "Epoch 372: | Loss: 0.58777 | Acc: 0.691 | Val ACC: 0.671\n",
      "Epoch 373: | Loss: 0.58684 | Acc: 0.689 | Val ACC: 0.669\n",
      "Epoch 374: | Loss: 0.58742 | Acc: 0.691 | Val ACC: 0.671\n",
      "Epoch 375: | Loss: 0.58709 | Acc: 0.689 | Val ACC: 0.663\n",
      "Epoch 376: | Loss: 0.59036 | Acc: 0.689 | Val ACC: 0.654\n",
      "Epoch 377: | Loss: 0.59262 | Acc: 0.686 | Val ACC: 0.658\n",
      "Epoch 378: | Loss: 0.59302 | Acc: 0.686 | Val ACC: 0.654\n",
      "Epoch 379: | Loss: 0.59124 | Acc: 0.683 | Val ACC: 0.648\n",
      "Epoch 380: | Loss: 0.58832 | Acc: 0.691 | Val ACC: 0.654\n",
      "Epoch 381: | Loss: 0.58656 | Acc: 0.691 | Val ACC: 0.656\n",
      "Epoch 382: | Loss: 0.58602 | Acc: 0.693 | Val ACC: 0.654\n",
      "Epoch 383: | Loss: 0.58714 | Acc: 0.691 | Val ACC: 0.662\n",
      "Epoch 384: | Loss: 0.58722 | Acc: 0.692 | Val ACC: 0.656\n",
      "Epoch 385: | Loss: 0.58719 | Acc: 0.693 | Val ACC: 0.660\n",
      "Epoch 386: | Loss: 0.58585 | Acc: 0.694 | Val ACC: 0.654\n",
      "Epoch 387: | Loss: 0.58532 | Acc: 0.694 | Val ACC: 0.656\n",
      "Epoch 388: | Loss: 0.58549 | Acc: 0.697 | Val ACC: 0.656\n",
      "Epoch 389: | Loss: 0.58481 | Acc: 0.699 | Val ACC: 0.652\n",
      "Epoch 390: | Loss: 0.58284 | Acc: 0.696 | Val ACC: 0.652\n",
      "Epoch 391: | Loss: 0.58139 | Acc: 0.700 | Val ACC: 0.648\n",
      "Epoch 392: | Loss: 0.58069 | Acc: 0.699 | Val ACC: 0.650\n",
      "Epoch 393: | Loss: 0.58189 | Acc: 0.698 | Val ACC: 0.654\n",
      "Epoch 394: | Loss: 0.58201 | Acc: 0.700 | Val ACC: 0.652\n",
      "Epoch 395: | Loss: 0.58097 | Acc: 0.700 | Val ACC: 0.656\n",
      "Epoch 396: | Loss: 0.58231 | Acc: 0.702 | Val ACC: 0.652\n",
      "Epoch 397: | Loss: 0.58180 | Acc: 0.696 | Val ACC: 0.654\n",
      "Epoch 398: | Loss: 0.58307 | Acc: 0.699 | Val ACC: 0.656\n",
      "Epoch 399: | Loss: 0.58168 | Acc: 0.700 | Val ACC: 0.660\n",
      "Epoch 400: | Loss: 0.58173 | Acc: 0.704 | Val ACC: 0.658\n",
      "Epoch 401: | Loss: 0.58142 | Acc: 0.698 | Val ACC: 0.658\n",
      "Epoch 402: | Loss: 0.58183 | Acc: 0.701 | Val ACC: 0.658\n",
      "Epoch 403: | Loss: 0.58191 | Acc: 0.701 | Val ACC: 0.656\n",
      "Epoch 404: | Loss: 0.58129 | Acc: 0.703 | Val ACC: 0.656\n",
      "Epoch 405: | Loss: 0.58122 | Acc: 0.705 | Val ACC: 0.658\n",
      "Epoch 406: | Loss: 0.58053 | Acc: 0.703 | Val ACC: 0.654\n",
      "Epoch 407: | Loss: 0.57838 | Acc: 0.703 | Val ACC: 0.660\n",
      "Epoch 408: | Loss: 0.57716 | Acc: 0.705 | Val ACC: 0.658\n",
      "Epoch 409: | Loss: 0.57742 | Acc: 0.704 | Val ACC: 0.658\n",
      "Epoch 410: | Loss: 0.57711 | Acc: 0.705 | Val ACC: 0.658\n",
      "Epoch 411: | Loss: 0.57604 | Acc: 0.706 | Val ACC: 0.662\n",
      "Epoch 412: | Loss: 0.57530 | Acc: 0.706 | Val ACC: 0.658\n",
      "Epoch 413: | Loss: 0.57528 | Acc: 0.707 | Val ACC: 0.665\n",
      "Epoch 414: | Loss: 0.57543 | Acc: 0.705 | Val ACC: 0.662\n",
      "Epoch 415: | Loss: 0.57557 | Acc: 0.703 | Val ACC: 0.669\n",
      "Epoch 416: | Loss: 0.57529 | Acc: 0.708 | Val ACC: 0.658\n",
      "Epoch 417: | Loss: 0.57459 | Acc: 0.704 | Val ACC: 0.667\n",
      "Epoch 418: | Loss: 0.57446 | Acc: 0.706 | Val ACC: 0.667\n",
      "Epoch 419: | Loss: 0.57392 | Acc: 0.705 | Val ACC: 0.671\n",
      "Epoch 420: | Loss: 0.57396 | Acc: 0.707 | Val ACC: 0.669\n",
      "Epoch 421: | Loss: 0.57347 | Acc: 0.703 | Val ACC: 0.671\n",
      "Epoch 422: | Loss: 0.57268 | Acc: 0.704 | Val ACC: 0.669\n",
      "Epoch 423: | Loss: 0.57193 | Acc: 0.708 | Val ACC: 0.662\n",
      "Epoch 424: | Loss: 0.57147 | Acc: 0.709 | Val ACC: 0.671\n",
      "Epoch 425: | Loss: 0.57127 | Acc: 0.709 | Val ACC: 0.665\n",
      "Epoch 426: | Loss: 0.57113 | Acc: 0.710 | Val ACC: 0.667\n",
      "Epoch 427: | Loss: 0.57143 | Acc: 0.711 | Val ACC: 0.665\n",
      "Epoch 428: | Loss: 0.57146 | Acc: 0.708 | Val ACC: 0.671\n",
      "Epoch 429: | Loss: 0.57120 | Acc: 0.709 | Val ACC: 0.669\n",
      "Epoch 430: | Loss: 0.57108 | Acc: 0.709 | Val ACC: 0.669\n",
      "Epoch 431: | Loss: 0.57150 | Acc: 0.708 | Val ACC: 0.671\n",
      "Epoch 432: | Loss: 0.57119 | Acc: 0.707 | Val ACC: 0.669\n",
      "Epoch 433: | Loss: 0.57147 | Acc: 0.709 | Val ACC: 0.667\n",
      "Epoch 434: | Loss: 0.57148 | Acc: 0.706 | Val ACC: 0.671\n",
      "Epoch 435: | Loss: 0.57001 | Acc: 0.707 | Val ACC: 0.663\n",
      "Epoch 436: | Loss: 0.56936 | Acc: 0.706 | Val ACC: 0.663\n",
      "Epoch 437: | Loss: 0.56811 | Acc: 0.709 | Val ACC: 0.665\n",
      "Epoch 438: | Loss: 0.56880 | Acc: 0.713 | Val ACC: 0.665\n",
      "Epoch 439: | Loss: 0.56944 | Acc: 0.710 | Val ACC: 0.665\n",
      "Epoch 440: | Loss: 0.56953 | Acc: 0.709 | Val ACC: 0.665\n",
      "Epoch 441: | Loss: 0.56821 | Acc: 0.709 | Val ACC: 0.663\n",
      "Epoch 442: | Loss: 0.56821 | Acc: 0.710 | Val ACC: 0.665\n",
      "Epoch 443: | Loss: 0.56775 | Acc: 0.710 | Val ACC: 0.662\n",
      "Epoch 444: | Loss: 0.56692 | Acc: 0.710 | Val ACC: 0.665\n",
      "Epoch 445: | Loss: 0.56683 | Acc: 0.709 | Val ACC: 0.663\n",
      "Epoch 446: | Loss: 0.56721 | Acc: 0.711 | Val ACC: 0.667\n",
      "Epoch 447: | Loss: 0.56745 | Acc: 0.709 | Val ACC: 0.663\n",
      "Epoch 448: | Loss: 0.56715 | Acc: 0.708 | Val ACC: 0.665\n",
      "Epoch 449: | Loss: 0.56703 | Acc: 0.710 | Val ACC: 0.665\n",
      "Epoch 450: | Loss: 0.56691 | Acc: 0.710 | Val ACC: 0.665\n",
      "Epoch 451: | Loss: 0.56747 | Acc: 0.710 | Val ACC: 0.671\n",
      "Epoch 452: | Loss: 0.56673 | Acc: 0.712 | Val ACC: 0.662\n",
      "Epoch 453: | Loss: 0.56581 | Acc: 0.710 | Val ACC: 0.665\n",
      "Epoch 454: | Loss: 0.56516 | Acc: 0.711 | Val ACC: 0.662\n",
      "Epoch 455: | Loss: 0.56509 | Acc: 0.712 | Val ACC: 0.665\n",
      "Epoch 456: | Loss: 0.56524 | Acc: 0.714 | Val ACC: 0.663\n",
      "Epoch 457: | Loss: 0.56407 | Acc: 0.713 | Val ACC: 0.665\n",
      "Epoch 458: | Loss: 0.56356 | Acc: 0.716 | Val ACC: 0.665\n",
      "Epoch 459: | Loss: 0.56397 | Acc: 0.713 | Val ACC: 0.665\n",
      "Epoch 460: | Loss: 0.56427 | Acc: 0.711 | Val ACC: 0.669\n",
      "Epoch 461: | Loss: 0.56437 | Acc: 0.715 | Val ACC: 0.667\n",
      "Epoch 462: | Loss: 0.56466 | Acc: 0.714 | Val ACC: 0.665\n",
      "Epoch 463: | Loss: 0.56530 | Acc: 0.711 | Val ACC: 0.665\n",
      "Epoch 464: | Loss: 0.56481 | Acc: 0.715 | Val ACC: 0.660\n",
      "Epoch 465: | Loss: 0.56560 | Acc: 0.711 | Val ACC: 0.660\n",
      "Epoch 466: | Loss: 0.56505 | Acc: 0.716 | Val ACC: 0.663\n",
      "Epoch 467: | Loss: 0.56559 | Acc: 0.711 | Val ACC: 0.665\n",
      "Epoch 468: | Loss: 0.56555 | Acc: 0.709 | Val ACC: 0.662\n",
      "Epoch 469: | Loss: 0.56401 | Acc: 0.711 | Val ACC: 0.663\n",
      "Epoch 470: | Loss: 0.56258 | Acc: 0.713 | Val ACC: 0.669\n",
      "Epoch 471: | Loss: 0.56223 | Acc: 0.713 | Val ACC: 0.669\n",
      "Epoch 472: | Loss: 0.56100 | Acc: 0.713 | Val ACC: 0.667\n",
      "Epoch 473: | Loss: 0.56107 | Acc: 0.718 | Val ACC: 0.677\n",
      "Epoch 474: | Loss: 0.56113 | Acc: 0.713 | Val ACC: 0.671\n",
      "Epoch 475: | Loss: 0.56053 | Acc: 0.716 | Val ACC: 0.669\n",
      "Epoch 476: | Loss: 0.56024 | Acc: 0.716 | Val ACC: 0.663\n",
      "Epoch 477: | Loss: 0.56034 | Acc: 0.719 | Val ACC: 0.665\n",
      "Epoch 478: | Loss: 0.56019 | Acc: 0.718 | Val ACC: 0.667\n",
      "Epoch 479: | Loss: 0.55950 | Acc: 0.719 | Val ACC: 0.665\n",
      "Epoch 480: | Loss: 0.56025 | Acc: 0.717 | Val ACC: 0.665\n",
      "Epoch 481: | Loss: 0.55909 | Acc: 0.720 | Val ACC: 0.665\n",
      "Epoch 482: | Loss: 0.56075 | Acc: 0.717 | Val ACC: 0.662\n",
      "Epoch 483: | Loss: 0.55950 | Acc: 0.716 | Val ACC: 0.665\n",
      "Epoch 484: | Loss: 0.55919 | Acc: 0.720 | Val ACC: 0.662\n",
      "Epoch 485: | Loss: 0.55887 | Acc: 0.718 | Val ACC: 0.667\n",
      "Epoch 486: | Loss: 0.55978 | Acc: 0.713 | Val ACC: 0.663\n",
      "Epoch 487: | Loss: 0.55851 | Acc: 0.719 | Val ACC: 0.665\n",
      "Epoch 488: | Loss: 0.56108 | Acc: 0.713 | Val ACC: 0.662\n",
      "Epoch 489: | Loss: 0.56025 | Acc: 0.716 | Val ACC: 0.662\n",
      "Epoch 490: | Loss: 0.55878 | Acc: 0.720 | Val ACC: 0.660\n",
      "Epoch 491: | Loss: 0.55773 | Acc: 0.720 | Val ACC: 0.667\n",
      "Epoch 492: | Loss: 0.55892 | Acc: 0.717 | Val ACC: 0.662\n",
      "Epoch 493: | Loss: 0.55788 | Acc: 0.717 | Val ACC: 0.665\n",
      "Epoch 494: | Loss: 0.55769 | Acc: 0.718 | Val ACC: 0.669\n",
      "Epoch 495: | Loss: 0.55862 | Acc: 0.716 | Val ACC: 0.658\n",
      "Epoch 496: | Loss: 0.55776 | Acc: 0.718 | Val ACC: 0.663\n",
      "Epoch 497: | Loss: 0.55716 | Acc: 0.720 | Val ACC: 0.662\n",
      "Epoch 498: | Loss: 0.55829 | Acc: 0.716 | Val ACC: 0.663\n",
      "Epoch 499: | Loss: 0.55699 | Acc: 0.718 | Val ACC: 0.665\n",
      "Epoch 500: | Loss: 0.55617 | Acc: 0.721 | Val ACC: 0.667\n",
      "Epoch 501: | Loss: 0.55521 | Acc: 0.723 | Val ACC: 0.665\n",
      "Epoch 502: | Loss: 0.55741 | Acc: 0.719 | Val ACC: 0.658\n",
      "Epoch 503: | Loss: 0.55559 | Acc: 0.718 | Val ACC: 0.665\n",
      "Epoch 504: | Loss: 0.55514 | Acc: 0.721 | Val ACC: 0.663\n",
      "Epoch 505: | Loss: 0.55402 | Acc: 0.722 | Val ACC: 0.671\n",
      "Epoch 506: | Loss: 0.55375 | Acc: 0.724 | Val ACC: 0.667\n",
      "Epoch 507: | Loss: 0.55423 | Acc: 0.720 | Val ACC: 0.665\n",
      "Epoch 508: | Loss: 0.55340 | Acc: 0.721 | Val ACC: 0.675\n",
      "Epoch 509: | Loss: 0.55358 | Acc: 0.722 | Val ACC: 0.667\n",
      "Epoch 510: | Loss: 0.55399 | Acc: 0.721 | Val ACC: 0.667\n",
      "Epoch 511: | Loss: 0.55311 | Acc: 0.723 | Val ACC: 0.671\n",
      "Epoch 512: | Loss: 0.55389 | Acc: 0.719 | Val ACC: 0.667\n",
      "Epoch 513: | Loss: 0.55384 | Acc: 0.719 | Val ACC: 0.669\n",
      "Epoch 514: | Loss: 0.55172 | Acc: 0.727 | Val ACC: 0.685\n",
      "Epoch 515: | Loss: 0.55544 | Acc: 0.719 | Val ACC: 0.662\n",
      "Epoch 516: | Loss: 0.55350 | Acc: 0.721 | Val ACC: 0.669\n",
      "Epoch 517: | Loss: 0.55133 | Acc: 0.728 | Val ACC: 0.677\n",
      "Epoch 518: | Loss: 0.55341 | Acc: 0.722 | Val ACC: 0.671\n",
      "Epoch 519: | Loss: 0.55193 | Acc: 0.724 | Val ACC: 0.667\n",
      "Epoch 520: | Loss: 0.55196 | Acc: 0.723 | Val ACC: 0.662\n",
      "Epoch 521: | Loss: 0.55139 | Acc: 0.725 | Val ACC: 0.671\n",
      "Epoch 522: | Loss: 0.55185 | Acc: 0.725 | Val ACC: 0.667\n",
      "Epoch 523: | Loss: 0.55098 | Acc: 0.726 | Val ACC: 0.667\n",
      "Epoch 524: | Loss: 0.54999 | Acc: 0.727 | Val ACC: 0.671\n",
      "Epoch 525: | Loss: 0.55070 | Acc: 0.726 | Val ACC: 0.671\n",
      "Epoch 526: | Loss: 0.55053 | Acc: 0.725 | Val ACC: 0.665\n",
      "Epoch 527: | Loss: 0.54975 | Acc: 0.728 | Val ACC: 0.679\n",
      "Epoch 528: | Loss: 0.55044 | Acc: 0.726 | Val ACC: 0.675\n",
      "Epoch 529: | Loss: 0.54934 | Acc: 0.725 | Val ACC: 0.675\n",
      "Epoch 530: | Loss: 0.55089 | Acc: 0.724 | Val ACC: 0.671\n",
      "Epoch 531: | Loss: 0.54947 | Acc: 0.728 | Val ACC: 0.681\n",
      "Epoch 532: | Loss: 0.55008 | Acc: 0.727 | Val ACC: 0.663\n",
      "Epoch 533: | Loss: 0.54875 | Acc: 0.730 | Val ACC: 0.685\n",
      "Epoch 534: | Loss: 0.55005 | Acc: 0.727 | Val ACC: 0.667\n",
      "Epoch 535: | Loss: 0.54925 | Acc: 0.727 | Val ACC: 0.677\n",
      "Epoch 536: | Loss: 0.54912 | Acc: 0.726 | Val ACC: 0.673\n",
      "Epoch 537: | Loss: 0.54887 | Acc: 0.726 | Val ACC: 0.675\n",
      "Epoch 538: | Loss: 0.54947 | Acc: 0.725 | Val ACC: 0.677\n",
      "Epoch 539: | Loss: 0.54918 | Acc: 0.728 | Val ACC: 0.669\n",
      "Epoch 540: | Loss: 0.55063 | Acc: 0.726 | Val ACC: 0.671\n",
      "Epoch 541: | Loss: 0.54828 | Acc: 0.727 | Val ACC: 0.675\n",
      "Epoch 542: | Loss: 0.54864 | Acc: 0.726 | Val ACC: 0.667\n",
      "Epoch 543: | Loss: 0.54748 | Acc: 0.727 | Val ACC: 0.669\n",
      "Epoch 544: | Loss: 0.54750 | Acc: 0.729 | Val ACC: 0.679\n",
      "Epoch 545: | Loss: 0.54715 | Acc: 0.728 | Val ACC: 0.671\n",
      "Epoch 546: | Loss: 0.54829 | Acc: 0.724 | Val ACC: 0.669\n",
      "Epoch 547: | Loss: 0.54842 | Acc: 0.730 | Val ACC: 0.673\n",
      "Epoch 548: | Loss: 0.54769 | Acc: 0.729 | Val ACC: 0.675\n",
      "Epoch 549: | Loss: 0.54664 | Acc: 0.730 | Val ACC: 0.673\n",
      "Epoch 550: | Loss: 0.54633 | Acc: 0.727 | Val ACC: 0.683\n",
      "Epoch 551: | Loss: 0.54834 | Acc: 0.726 | Val ACC: 0.671\n",
      "Epoch 552: | Loss: 0.54677 | Acc: 0.728 | Val ACC: 0.667\n",
      "Epoch 553: | Loss: 0.54658 | Acc: 0.728 | Val ACC: 0.671\n",
      "Epoch 554: | Loss: 0.54718 | Acc: 0.727 | Val ACC: 0.675\n",
      "Epoch 555: | Loss: 0.54678 | Acc: 0.727 | Val ACC: 0.667\n",
      "Epoch 556: | Loss: 0.54529 | Acc: 0.731 | Val ACC: 0.673\n",
      "Epoch 557: | Loss: 0.54600 | Acc: 0.727 | Val ACC: 0.669\n",
      "Epoch 558: | Loss: 0.54520 | Acc: 0.728 | Val ACC: 0.671\n",
      "Epoch 559: | Loss: 0.54536 | Acc: 0.730 | Val ACC: 0.673\n",
      "Epoch 560: | Loss: 0.54593 | Acc: 0.728 | Val ACC: 0.669\n",
      "Epoch 561: | Loss: 0.54388 | Acc: 0.731 | Val ACC: 0.669\n",
      "Epoch 562: | Loss: 0.54579 | Acc: 0.728 | Val ACC: 0.673\n",
      "Epoch 563: | Loss: 0.54560 | Acc: 0.728 | Val ACC: 0.673\n",
      "Epoch 564: | Loss: 0.54670 | Acc: 0.728 | Val ACC: 0.671\n",
      "Epoch 565: | Loss: 0.54333 | Acc: 0.730 | Val ACC: 0.669\n",
      "Epoch 566: | Loss: 0.54364 | Acc: 0.728 | Val ACC: 0.673\n",
      "Epoch 567: | Loss: 0.54312 | Acc: 0.727 | Val ACC: 0.671\n",
      "Epoch 568: | Loss: 0.54554 | Acc: 0.732 | Val ACC: 0.671\n",
      "Epoch 569: | Loss: 0.54171 | Acc: 0.733 | Val ACC: 0.677\n",
      "Epoch 570: | Loss: 0.54123 | Acc: 0.735 | Val ACC: 0.683\n",
      "Epoch 571: | Loss: 0.54544 | Acc: 0.731 | Val ACC: 0.681\n",
      "Epoch 572: | Loss: 0.54519 | Acc: 0.731 | Val ACC: 0.667\n",
      "Epoch 573: | Loss: 0.54058 | Acc: 0.732 | Val ACC: 0.687\n",
      "Epoch 574: | Loss: 0.54391 | Acc: 0.729 | Val ACC: 0.673\n",
      "Epoch 575: | Loss: 0.54295 | Acc: 0.733 | Val ACC: 0.687\n",
      "Epoch 576: | Loss: 0.54219 | Acc: 0.733 | Val ACC: 0.677\n",
      "Epoch 577: | Loss: 0.54174 | Acc: 0.736 | Val ACC: 0.677\n",
      "Epoch 578: | Loss: 0.54093 | Acc: 0.734 | Val ACC: 0.688\n",
      "Epoch 579: | Loss: 0.54175 | Acc: 0.730 | Val ACC: 0.673\n",
      "Epoch 580: | Loss: 0.54273 | Acc: 0.731 | Val ACC: 0.677\n",
      "Epoch 581: | Loss: 0.54109 | Acc: 0.735 | Val ACC: 0.673\n",
      "Epoch 582: | Loss: 0.54012 | Acc: 0.734 | Val ACC: 0.683\n",
      "Epoch 583: | Loss: 0.53965 | Acc: 0.732 | Val ACC: 0.687\n",
      "Epoch 584: | Loss: 0.54170 | Acc: 0.733 | Val ACC: 0.675\n",
      "Epoch 585: | Loss: 0.54013 | Acc: 0.731 | Val ACC: 0.679\n",
      "Epoch 586: | Loss: 0.53747 | Acc: 0.736 | Val ACC: 0.688\n",
      "Epoch 587: | Loss: 0.54303 | Acc: 0.731 | Val ACC: 0.667\n",
      "Epoch 588: | Loss: 0.53839 | Acc: 0.736 | Val ACC: 0.683\n",
      "Epoch 589: | Loss: 0.54528 | Acc: 0.730 | Val ACC: 0.665\n",
      "Epoch 590: | Loss: 0.54100 | Acc: 0.730 | Val ACC: 0.679\n",
      "Epoch 591: | Loss: 0.53933 | Acc: 0.735 | Val ACC: 0.677\n",
      "Epoch 592: | Loss: 0.53806 | Acc: 0.735 | Val ACC: 0.685\n",
      "Epoch 593: | Loss: 0.54600 | Acc: 0.729 | Val ACC: 0.667\n",
      "Epoch 594: | Loss: 0.54644 | Acc: 0.730 | Val ACC: 0.679\n",
      "Epoch 595: | Loss: 0.54359 | Acc: 0.734 | Val ACC: 0.671\n",
      "Epoch 596: | Loss: 0.54037 | Acc: 0.733 | Val ACC: 0.679\n",
      "Epoch 597: | Loss: 0.53900 | Acc: 0.739 | Val ACC: 0.673\n",
      "Epoch 598: | Loss: 0.53729 | Acc: 0.739 | Val ACC: 0.677\n",
      "Epoch 599: | Loss: 0.53565 | Acc: 0.740 | Val ACC: 0.679\n"
     ]
    }
   ],
   "source": [
    "clients = [\n",
    "    SyncFNNClient(id=id, \n",
    "    model=nn.Sequential(\n",
    "        nn.Linear(100, 64),\n",
    "        nn.ReLU(),\n",
    "        nn.Linear(64, 32),\n",
    "        nn.ReLU()),\n",
    "    courier=courier,\n",
    "    train_loader=loader_dict[id]['train_loader'],\n",
    "    test_loader=loader_dict[id]['test_loader'],\n",
    "    config_dir='simple_config.ini')\n",
    "for id in client_id_list]\n",
    "strategy = SyncConcatStrategy(courier=courier, clients=clients)\n",
    "server =  SyncFNNServer(\n",
    "    strategy=strategy,\n",
    "    courier=courier,\n",
    "    top_model=nn.Sequential(\n",
    "        nn.Linear(5 * 32, 32),\n",
    "        nn.ReLU(True),\n",
    "        nn.Linear(32, 1),\n",
    "        nn.Sigmoid()),\n",
    "    emb_model=nn.Sequential( \n",
    "        nn.Linear(100, 64),\n",
    "        nn.ReLU(),\n",
    "        nn.Linear(64, 32),\n",
    "        nn.ReLU()),\n",
    "    train_loader=loader_dict['server']['train_loader'],\n",
    "    test_loader=loader_dict['server']['test_loader'],\n",
    "    config_dir='simple_config.ini')\n",
    "\n",
    "print('Training Starts')\n",
    "print('-'*89)\n",
    "for epoch in range(600):\n",
    "    server.fit(epoch)\n",
    "history4 = server.get_history()\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "9 clients"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Client 0: Feature Index 0-49\n",
      "Client 1: Feature Index 50-99\n",
      "Client 2: Feature Index 100-149\n",
      "Client 3: Feature Index 150-199\n",
      "Client 4: Feature Index 200-249\n",
      "Client 5: Feature Index 250-299\n",
      "Client 6: Feature Index 300-349\n",
      "Client 7: Feature Index 350-399\n",
      "Client 8: Feature Index 400-449\n",
      "Server : Feature Index 450-499\n"
     ]
    }
   ],
   "source": [
    "client_id_list = list(range(9))\n",
    "courier = SyncLocalCourier(client_id_list)\n",
    "loader = SimpleNumpyDataLoader(\n",
    "    clients_id_list=client_id_list,\n",
    "    data_source=(X_np, y_np)\n",
    ")\n",
    "loader_dict = loader.distribute()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Starts\n",
      "-----------------------------------------------------------------------------------------\n",
      "Epoch 000: | Loss: 0.69343 | Acc: 0.496 | Val ACC: 0.494\n",
      "Epoch 001: | Loss: 0.69309 | Acc: 0.504 | Val ACC: 0.508\n",
      "Epoch 002: | Loss: 0.69298 | Acc: 0.502 | Val ACC: 0.512\n",
      "Epoch 003: | Loss: 0.69289 | Acc: 0.514 | Val ACC: 0.558\n",
      "Epoch 004: | Loss: 0.69280 | Acc: 0.521 | Val ACC: 0.548\n",
      "Epoch 005: | Loss: 0.69267 | Acc: 0.511 | Val ACC: 0.579\n",
      "Epoch 006: | Loss: 0.69252 | Acc: 0.528 | Val ACC: 0.579\n",
      "Epoch 007: | Loss: 0.69231 | Acc: 0.543 | Val ACC: 0.590\n",
      "Epoch 008: | Loss: 0.69209 | Acc: 0.555 | Val ACC: 0.538\n",
      "Epoch 009: | Loss: 0.69175 | Acc: 0.544 | Val ACC: 0.577\n",
      "Epoch 010: | Loss: 0.69131 | Acc: 0.554 | Val ACC: 0.583\n",
      "Epoch 011: | Loss: 0.69080 | Acc: 0.553 | Val ACC: 0.560\n",
      "Epoch 012: | Loss: 0.69012 | Acc: 0.558 | Val ACC: 0.554\n",
      "Epoch 013: | Loss: 0.68931 | Acc: 0.562 | Val ACC: 0.594\n",
      "Epoch 014: | Loss: 0.68811 | Acc: 0.574 | Val ACC: 0.558\n",
      "Epoch 015: | Loss: 0.68696 | Acc: 0.560 | Val ACC: 0.535\n",
      "Epoch 016: | Loss: 0.68628 | Acc: 0.564 | Val ACC: 0.538\n",
      "Epoch 017: | Loss: 0.68534 | Acc: 0.561 | Val ACC: 0.535\n",
      "Epoch 018: | Loss: 0.68424 | Acc: 0.567 | Val ACC: 0.537\n",
      "Epoch 019: | Loss: 0.68331 | Acc: 0.565 | Val ACC: 0.546\n",
      "Epoch 020: | Loss: 0.68240 | Acc: 0.572 | Val ACC: 0.552\n",
      "Epoch 021: | Loss: 0.68151 | Acc: 0.572 | Val ACC: 0.554\n",
      "Epoch 022: | Loss: 0.68090 | Acc: 0.571 | Val ACC: 0.556\n",
      "Epoch 023: | Loss: 0.68015 | Acc: 0.574 | Val ACC: 0.571\n",
      "Epoch 024: | Loss: 0.67930 | Acc: 0.580 | Val ACC: 0.569\n",
      "Epoch 025: | Loss: 0.67869 | Acc: 0.581 | Val ACC: 0.565\n",
      "Epoch 026: | Loss: 0.67816 | Acc: 0.583 | Val ACC: 0.577\n",
      "Epoch 027: | Loss: 0.67742 | Acc: 0.587 | Val ACC: 0.577\n",
      "Epoch 028: | Loss: 0.67688 | Acc: 0.591 | Val ACC: 0.585\n",
      "Epoch 029: | Loss: 0.67651 | Acc: 0.590 | Val ACC: 0.583\n",
      "Epoch 030: | Loss: 0.67592 | Acc: 0.595 | Val ACC: 0.571\n",
      "Epoch 031: | Loss: 0.67560 | Acc: 0.595 | Val ACC: 0.590\n",
      "Epoch 032: | Loss: 0.67485 | Acc: 0.601 | Val ACC: 0.575\n",
      "Epoch 033: | Loss: 0.67520 | Acc: 0.596 | Val ACC: 0.600\n",
      "Epoch 034: | Loss: 0.67389 | Acc: 0.604 | Val ACC: 0.575\n",
      "Epoch 035: | Loss: 0.67440 | Acc: 0.598 | Val ACC: 0.600\n",
      "Epoch 036: | Loss: 0.67332 | Acc: 0.605 | Val ACC: 0.588\n",
      "Epoch 037: | Loss: 0.67407 | Acc: 0.600 | Val ACC: 0.608\n",
      "Epoch 038: | Loss: 0.67233 | Acc: 0.608 | Val ACC: 0.592\n",
      "Epoch 039: | Loss: 0.67310 | Acc: 0.602 | Val ACC: 0.612\n",
      "Epoch 040: | Loss: 0.67162 | Acc: 0.607 | Val ACC: 0.596\n",
      "Epoch 041: | Loss: 0.67196 | Acc: 0.606 | Val ACC: 0.608\n",
      "Epoch 042: | Loss: 0.67121 | Acc: 0.605 | Val ACC: 0.606\n",
      "Epoch 043: | Loss: 0.67070 | Acc: 0.607 | Val ACC: 0.606\n",
      "Epoch 044: | Loss: 0.67087 | Acc: 0.603 | Val ACC: 0.608\n",
      "Epoch 045: | Loss: 0.67095 | Acc: 0.608 | Val ACC: 0.608\n",
      "Epoch 046: | Loss: 0.66980 | Acc: 0.606 | Val ACC: 0.608\n",
      "Epoch 047: | Loss: 0.66989 | Acc: 0.607 | Val ACC: 0.610\n",
      "Epoch 048: | Loss: 0.66948 | Acc: 0.605 | Val ACC: 0.604\n",
      "Epoch 049: | Loss: 0.66831 | Acc: 0.608 | Val ACC: 0.604\n",
      "Epoch 050: | Loss: 0.66961 | Acc: 0.609 | Val ACC: 0.606\n",
      "Epoch 051: | Loss: 0.66880 | Acc: 0.610 | Val ACC: 0.604\n",
      "Epoch 052: | Loss: 0.66716 | Acc: 0.606 | Val ACC: 0.606\n",
      "Epoch 053: | Loss: 0.66854 | Acc: 0.609 | Val ACC: 0.604\n",
      "Epoch 054: | Loss: 0.66734 | Acc: 0.609 | Val ACC: 0.606\n",
      "Epoch 055: | Loss: 0.66674 | Acc: 0.608 | Val ACC: 0.606\n",
      "Epoch 056: | Loss: 0.66743 | Acc: 0.608 | Val ACC: 0.604\n",
      "Epoch 057: | Loss: 0.66690 | Acc: 0.608 | Val ACC: 0.604\n",
      "Epoch 058: | Loss: 0.66534 | Acc: 0.608 | Val ACC: 0.606\n",
      "Epoch 059: | Loss: 0.66552 | Acc: 0.608 | Val ACC: 0.602\n",
      "Epoch 060: | Loss: 0.66566 | Acc: 0.608 | Val ACC: 0.602\n",
      "Epoch 061: | Loss: 0.66497 | Acc: 0.609 | Val ACC: 0.600\n",
      "Epoch 062: | Loss: 0.66346 | Acc: 0.613 | Val ACC: 0.606\n",
      "Epoch 063: | Loss: 0.66353 | Acc: 0.612 | Val ACC: 0.604\n",
      "Epoch 064: | Loss: 0.66485 | Acc: 0.609 | Val ACC: 0.600\n",
      "Epoch 065: | Loss: 0.66353 | Acc: 0.612 | Val ACC: 0.604\n",
      "Epoch 066: | Loss: 0.66219 | Acc: 0.614 | Val ACC: 0.604\n",
      "Epoch 067: | Loss: 0.66311 | Acc: 0.610 | Val ACC: 0.602\n",
      "Epoch 068: | Loss: 0.66205 | Acc: 0.612 | Val ACC: 0.608\n",
      "Epoch 069: | Loss: 0.66250 | Acc: 0.611 | Val ACC: 0.604\n",
      "Epoch 070: | Loss: 0.66090 | Acc: 0.615 | Val ACC: 0.602\n",
      "Epoch 071: | Loss: 0.66089 | Acc: 0.613 | Val ACC: 0.604\n",
      "Epoch 072: | Loss: 0.66004 | Acc: 0.614 | Val ACC: 0.600\n",
      "Epoch 073: | Loss: 0.65986 | Acc: 0.615 | Val ACC: 0.598\n",
      "Epoch 074: | Loss: 0.65934 | Acc: 0.614 | Val ACC: 0.598\n",
      "Epoch 075: | Loss: 0.65980 | Acc: 0.613 | Val ACC: 0.604\n",
      "Epoch 076: | Loss: 0.65945 | Acc: 0.612 | Val ACC: 0.608\n",
      "Epoch 077: | Loss: 0.65956 | Acc: 0.613 | Val ACC: 0.606\n",
      "Epoch 078: | Loss: 0.65738 | Acc: 0.618 | Val ACC: 0.596\n",
      "Epoch 079: | Loss: 0.65755 | Acc: 0.618 | Val ACC: 0.602\n",
      "Epoch 080: | Loss: 0.65820 | Acc: 0.613 | Val ACC: 0.604\n",
      "Epoch 081: | Loss: 0.65716 | Acc: 0.621 | Val ACC: 0.610\n",
      "Epoch 082: | Loss: 0.65506 | Acc: 0.622 | Val ACC: 0.600\n",
      "Epoch 083: | Loss: 0.65560 | Acc: 0.624 | Val ACC: 0.598\n",
      "Epoch 084: | Loss: 0.65684 | Acc: 0.619 | Val ACC: 0.602\n",
      "Epoch 085: | Loss: 0.65439 | Acc: 0.620 | Val ACC: 0.598\n",
      "Epoch 086: | Loss: 0.65359 | Acc: 0.624 | Val ACC: 0.598\n",
      "Epoch 087: | Loss: 0.65448 | Acc: 0.625 | Val ACC: 0.602\n",
      "Epoch 088: | Loss: 0.65568 | Acc: 0.623 | Val ACC: 0.602\n",
      "Epoch 089: | Loss: 0.65331 | Acc: 0.625 | Val ACC: 0.602\n",
      "Epoch 090: | Loss: 0.65359 | Acc: 0.625 | Val ACC: 0.594\n",
      "Epoch 091: | Loss: 0.65254 | Acc: 0.625 | Val ACC: 0.604\n",
      "Epoch 092: | Loss: 0.65228 | Acc: 0.621 | Val ACC: 0.598\n",
      "Epoch 093: | Loss: 0.65186 | Acc: 0.622 | Val ACC: 0.598\n",
      "Epoch 094: | Loss: 0.65227 | Acc: 0.619 | Val ACC: 0.604\n",
      "Epoch 095: | Loss: 0.65185 | Acc: 0.621 | Val ACC: 0.612\n",
      "Epoch 096: | Loss: 0.65020 | Acc: 0.619 | Val ACC: 0.592\n",
      "Epoch 097: | Loss: 0.65079 | Acc: 0.617 | Val ACC: 0.608\n",
      "Epoch 098: | Loss: 0.65050 | Acc: 0.619 | Val ACC: 0.604\n",
      "Epoch 099: | Loss: 0.64895 | Acc: 0.624 | Val ACC: 0.592\n",
      "Epoch 100: | Loss: 0.64888 | Acc: 0.623 | Val ACC: 0.600\n",
      "Epoch 101: | Loss: 0.64907 | Acc: 0.620 | Val ACC: 0.594\n",
      "Epoch 102: | Loss: 0.64976 | Acc: 0.621 | Val ACC: 0.598\n",
      "Epoch 103: | Loss: 0.64998 | Acc: 0.619 | Val ACC: 0.590\n",
      "Epoch 104: | Loss: 0.64982 | Acc: 0.618 | Val ACC: 0.608\n",
      "Epoch 105: | Loss: 0.64866 | Acc: 0.621 | Val ACC: 0.592\n",
      "Epoch 106: | Loss: 0.64723 | Acc: 0.622 | Val ACC: 0.598\n",
      "Epoch 107: | Loss: 0.64748 | Acc: 0.625 | Val ACC: 0.598\n",
      "Epoch 108: | Loss: 0.64796 | Acc: 0.621 | Val ACC: 0.602\n",
      "Epoch 109: | Loss: 0.64877 | Acc: 0.620 | Val ACC: 0.600\n",
      "Epoch 110: | Loss: 0.64861 | Acc: 0.618 | Val ACC: 0.610\n",
      "Epoch 111: | Loss: 0.64768 | Acc: 0.619 | Val ACC: 0.600\n",
      "Epoch 112: | Loss: 0.64621 | Acc: 0.625 | Val ACC: 0.596\n",
      "Epoch 113: | Loss: 0.64573 | Acc: 0.627 | Val ACC: 0.604\n",
      "Epoch 114: | Loss: 0.64550 | Acc: 0.624 | Val ACC: 0.598\n",
      "Epoch 115: | Loss: 0.64527 | Acc: 0.627 | Val ACC: 0.598\n",
      "Epoch 116: | Loss: 0.64489 | Acc: 0.625 | Val ACC: 0.598\n",
      "Epoch 117: | Loss: 0.64438 | Acc: 0.628 | Val ACC: 0.602\n",
      "Epoch 118: | Loss: 0.64372 | Acc: 0.624 | Val ACC: 0.596\n",
      "Epoch 119: | Loss: 0.64416 | Acc: 0.626 | Val ACC: 0.596\n",
      "Epoch 120: | Loss: 0.64427 | Acc: 0.627 | Val ACC: 0.606\n",
      "Epoch 121: | Loss: 0.64432 | Acc: 0.624 | Val ACC: 0.604\n",
      "Epoch 122: | Loss: 0.64355 | Acc: 0.626 | Val ACC: 0.610\n",
      "Epoch 123: | Loss: 0.64224 | Acc: 0.629 | Val ACC: 0.612\n",
      "Epoch 124: | Loss: 0.64134 | Acc: 0.631 | Val ACC: 0.596\n",
      "Epoch 125: | Loss: 0.64245 | Acc: 0.624 | Val ACC: 0.598\n",
      "Epoch 126: | Loss: 0.64289 | Acc: 0.628 | Val ACC: 0.608\n",
      "Epoch 127: | Loss: 0.64234 | Acc: 0.624 | Val ACC: 0.602\n",
      "Epoch 128: | Loss: 0.64148 | Acc: 0.628 | Val ACC: 0.606\n",
      "Epoch 129: | Loss: 0.64063 | Acc: 0.626 | Val ACC: 0.596\n",
      "Epoch 130: | Loss: 0.63931 | Acc: 0.631 | Val ACC: 0.600\n",
      "Epoch 131: | Loss: 0.64111 | Acc: 0.627 | Val ACC: 0.608\n",
      "Epoch 132: | Loss: 0.64189 | Acc: 0.627 | Val ACC: 0.610\n",
      "Epoch 133: | Loss: 0.64025 | Acc: 0.627 | Val ACC: 0.608\n",
      "Epoch 134: | Loss: 0.63992 | Acc: 0.629 | Val ACC: 0.596\n",
      "Epoch 135: | Loss: 0.64033 | Acc: 0.628 | Val ACC: 0.600\n",
      "Epoch 136: | Loss: 0.63919 | Acc: 0.626 | Val ACC: 0.604\n",
      "Epoch 137: | Loss: 0.64016 | Acc: 0.629 | Val ACC: 0.608\n",
      "Epoch 138: | Loss: 0.63883 | Acc: 0.629 | Val ACC: 0.604\n",
      "Epoch 139: | Loss: 0.63753 | Acc: 0.629 | Val ACC: 0.602\n",
      "Epoch 140: | Loss: 0.63826 | Acc: 0.627 | Val ACC: 0.606\n",
      "Epoch 141: | Loss: 0.63815 | Acc: 0.627 | Val ACC: 0.602\n",
      "Epoch 142: | Loss: 0.63765 | Acc: 0.629 | Val ACC: 0.596\n",
      "Epoch 143: | Loss: 0.63726 | Acc: 0.633 | Val ACC: 0.598\n",
      "Epoch 144: | Loss: 0.63658 | Acc: 0.628 | Val ACC: 0.598\n",
      "Epoch 145: | Loss: 0.63557 | Acc: 0.630 | Val ACC: 0.608\n",
      "Epoch 146: | Loss: 0.63730 | Acc: 0.631 | Val ACC: 0.604\n",
      "Epoch 147: | Loss: 0.63544 | Acc: 0.632 | Val ACC: 0.606\n",
      "Epoch 148: | Loss: 0.63585 | Acc: 0.629 | Val ACC: 0.602\n",
      "Epoch 149: | Loss: 0.63562 | Acc: 0.631 | Val ACC: 0.606\n",
      "Epoch 150: | Loss: 0.63531 | Acc: 0.632 | Val ACC: 0.598\n",
      "Epoch 151: | Loss: 0.63528 | Acc: 0.629 | Val ACC: 0.594\n",
      "Epoch 152: | Loss: 0.63478 | Acc: 0.631 | Val ACC: 0.598\n",
      "Epoch 153: | Loss: 0.63338 | Acc: 0.635 | Val ACC: 0.600\n",
      "Epoch 154: | Loss: 0.63342 | Acc: 0.632 | Val ACC: 0.602\n",
      "Epoch 155: | Loss: 0.63434 | Acc: 0.629 | Val ACC: 0.596\n",
      "Epoch 156: | Loss: 0.63433 | Acc: 0.632 | Val ACC: 0.604\n",
      "Epoch 157: | Loss: 0.63344 | Acc: 0.632 | Val ACC: 0.604\n",
      "Epoch 158: | Loss: 0.63283 | Acc: 0.635 | Val ACC: 0.604\n",
      "Epoch 159: | Loss: 0.63428 | Acc: 0.627 | Val ACC: 0.600\n",
      "Epoch 160: | Loss: 0.63383 | Acc: 0.631 | Val ACC: 0.602\n",
      "Epoch 161: | Loss: 0.63331 | Acc: 0.634 | Val ACC: 0.608\n",
      "Epoch 162: | Loss: 0.63338 | Acc: 0.636 | Val ACC: 0.604\n",
      "Epoch 163: | Loss: 0.63306 | Acc: 0.630 | Val ACC: 0.608\n",
      "Epoch 164: | Loss: 0.63252 | Acc: 0.630 | Val ACC: 0.602\n",
      "Epoch 165: | Loss: 0.63141 | Acc: 0.632 | Val ACC: 0.600\n",
      "Epoch 166: | Loss: 0.63126 | Acc: 0.632 | Val ACC: 0.604\n",
      "Epoch 167: | Loss: 0.62945 | Acc: 0.642 | Val ACC: 0.610\n",
      "Epoch 168: | Loss: 0.62814 | Acc: 0.637 | Val ACC: 0.610\n",
      "Epoch 169: | Loss: 0.63053 | Acc: 0.634 | Val ACC: 0.596\n",
      "Epoch 170: | Loss: 0.63168 | Acc: 0.630 | Val ACC: 0.606\n",
      "Epoch 171: | Loss: 0.62898 | Acc: 0.634 | Val ACC: 0.604\n",
      "Epoch 172: | Loss: 0.62888 | Acc: 0.639 | Val ACC: 0.600\n",
      "Epoch 173: | Loss: 0.62830 | Acc: 0.639 | Val ACC: 0.606\n",
      "Epoch 174: | Loss: 0.62633 | Acc: 0.638 | Val ACC: 0.600\n",
      "Epoch 175: | Loss: 0.62805 | Acc: 0.636 | Val ACC: 0.602\n",
      "Epoch 176: | Loss: 0.62933 | Acc: 0.637 | Val ACC: 0.612\n",
      "Epoch 177: | Loss: 0.62672 | Acc: 0.637 | Val ACC: 0.602\n",
      "Epoch 178: | Loss: 0.62493 | Acc: 0.644 | Val ACC: 0.619\n",
      "Epoch 179: | Loss: 0.62395 | Acc: 0.640 | Val ACC: 0.615\n",
      "Epoch 180: | Loss: 0.62468 | Acc: 0.640 | Val ACC: 0.613\n",
      "Epoch 181: | Loss: 0.62347 | Acc: 0.645 | Val ACC: 0.612\n",
      "Epoch 182: | Loss: 0.62386 | Acc: 0.639 | Val ACC: 0.612\n",
      "Epoch 183: | Loss: 0.62573 | Acc: 0.637 | Val ACC: 0.602\n",
      "Epoch 184: | Loss: 0.62520 | Acc: 0.641 | Val ACC: 0.608\n",
      "Epoch 185: | Loss: 0.62126 | Acc: 0.642 | Val ACC: 0.612\n",
      "Epoch 186: | Loss: 0.62196 | Acc: 0.643 | Val ACC: 0.612\n",
      "Epoch 187: | Loss: 0.62448 | Acc: 0.639 | Val ACC: 0.610\n",
      "Epoch 188: | Loss: 0.62522 | Acc: 0.639 | Val ACC: 0.621\n",
      "Epoch 189: | Loss: 0.62188 | Acc: 0.642 | Val ACC: 0.610\n",
      "Epoch 190: | Loss: 0.62129 | Acc: 0.643 | Val ACC: 0.615\n",
      "Epoch 191: | Loss: 0.62251 | Acc: 0.645 | Val ACC: 0.610\n",
      "Epoch 192: | Loss: 0.62157 | Acc: 0.647 | Val ACC: 0.619\n",
      "Epoch 193: | Loss: 0.62077 | Acc: 0.646 | Val ACC: 0.610\n",
      "Epoch 194: | Loss: 0.61921 | Acc: 0.646 | Val ACC: 0.612\n",
      "Epoch 195: | Loss: 0.62074 | Acc: 0.648 | Val ACC: 0.613\n",
      "Epoch 196: | Loss: 0.62175 | Acc: 0.644 | Val ACC: 0.612\n",
      "Epoch 197: | Loss: 0.62027 | Acc: 0.647 | Val ACC: 0.612\n",
      "Epoch 198: | Loss: 0.61873 | Acc: 0.650 | Val ACC: 0.619\n",
      "Epoch 199: | Loss: 0.61964 | Acc: 0.646 | Val ACC: 0.613\n",
      "Epoch 200: | Loss: 0.61713 | Acc: 0.649 | Val ACC: 0.617\n",
      "Epoch 201: | Loss: 0.61844 | Acc: 0.648 | Val ACC: 0.615\n",
      "Epoch 202: | Loss: 0.61796 | Acc: 0.646 | Val ACC: 0.623\n",
      "Epoch 203: | Loss: 0.61815 | Acc: 0.647 | Val ACC: 0.617\n",
      "Epoch 204: | Loss: 0.61741 | Acc: 0.647 | Val ACC: 0.619\n",
      "Epoch 205: | Loss: 0.61565 | Acc: 0.651 | Val ACC: 0.613\n",
      "Epoch 206: | Loss: 0.61457 | Acc: 0.652 | Val ACC: 0.615\n",
      "Epoch 207: | Loss: 0.61485 | Acc: 0.648 | Val ACC: 0.615\n",
      "Epoch 208: | Loss: 0.61542 | Acc: 0.657 | Val ACC: 0.621\n",
      "Epoch 209: | Loss: 0.61584 | Acc: 0.657 | Val ACC: 0.621\n",
      "Epoch 210: | Loss: 0.61355 | Acc: 0.654 | Val ACC: 0.627\n",
      "Epoch 211: | Loss: 0.61487 | Acc: 0.658 | Val ACC: 0.621\n",
      "Epoch 212: | Loss: 0.61396 | Acc: 0.651 | Val ACC: 0.623\n",
      "Epoch 213: | Loss: 0.61297 | Acc: 0.656 | Val ACC: 0.619\n",
      "Epoch 214: | Loss: 0.61275 | Acc: 0.653 | Val ACC: 0.631\n",
      "Epoch 215: | Loss: 0.61367 | Acc: 0.654 | Val ACC: 0.629\n",
      "Epoch 216: | Loss: 0.60827 | Acc: 0.655 | Val ACC: 0.635\n",
      "Epoch 217: | Loss: 0.60909 | Acc: 0.653 | Val ACC: 0.633\n",
      "Epoch 218: | Loss: 0.60662 | Acc: 0.656 | Val ACC: 0.629\n",
      "Epoch 219: | Loss: 0.60821 | Acc: 0.657 | Val ACC: 0.631\n",
      "Epoch 220: | Loss: 0.61095 | Acc: 0.652 | Val ACC: 0.638\n",
      "Epoch 221: | Loss: 0.60872 | Acc: 0.653 | Val ACC: 0.640\n",
      "Epoch 222: | Loss: 0.60459 | Acc: 0.663 | Val ACC: 0.631\n",
      "Epoch 223: | Loss: 0.60400 | Acc: 0.662 | Val ACC: 0.633\n",
      "Epoch 224: | Loss: 0.60493 | Acc: 0.658 | Val ACC: 0.631\n",
      "Epoch 225: | Loss: 0.60359 | Acc: 0.659 | Val ACC: 0.633\n",
      "Epoch 226: | Loss: 0.60422 | Acc: 0.659 | Val ACC: 0.631\n",
      "Epoch 227: | Loss: 0.60664 | Acc: 0.657 | Val ACC: 0.638\n",
      "Epoch 228: | Loss: 0.60352 | Acc: 0.658 | Val ACC: 0.635\n",
      "Epoch 229: | Loss: 0.60263 | Acc: 0.661 | Val ACC: 0.644\n",
      "Epoch 230: | Loss: 0.60195 | Acc: 0.656 | Val ACC: 0.625\n",
      "Epoch 231: | Loss: 0.60190 | Acc: 0.665 | Val ACC: 0.633\n",
      "Epoch 232: | Loss: 0.59900 | Acc: 0.663 | Val ACC: 0.633\n",
      "Epoch 233: | Loss: 0.60005 | Acc: 0.658 | Val ACC: 0.629\n",
      "Epoch 234: | Loss: 0.60218 | Acc: 0.654 | Val ACC: 0.640\n",
      "Epoch 235: | Loss: 0.60305 | Acc: 0.656 | Val ACC: 0.646\n",
      "Epoch 236: | Loss: 0.59868 | Acc: 0.662 | Val ACC: 0.638\n",
      "Epoch 237: | Loss: 0.59834 | Acc: 0.667 | Val ACC: 0.640\n",
      "Epoch 238: | Loss: 0.59656 | Acc: 0.668 | Val ACC: 0.637\n",
      "Epoch 239: | Loss: 0.59697 | Acc: 0.668 | Val ACC: 0.633\n",
      "Epoch 240: | Loss: 0.59865 | Acc: 0.662 | Val ACC: 0.646\n",
      "Epoch 241: | Loss: 0.59634 | Acc: 0.663 | Val ACC: 0.633\n",
      "Epoch 242: | Loss: 0.59340 | Acc: 0.671 | Val ACC: 0.644\n",
      "Epoch 243: | Loss: 0.59453 | Acc: 0.672 | Val ACC: 0.642\n",
      "Epoch 244: | Loss: 0.59315 | Acc: 0.671 | Val ACC: 0.629\n",
      "Epoch 245: | Loss: 0.59494 | Acc: 0.669 | Val ACC: 0.644\n",
      "Epoch 246: | Loss: 0.59628 | Acc: 0.665 | Val ACC: 0.644\n",
      "Epoch 247: | Loss: 0.59399 | Acc: 0.665 | Val ACC: 0.629\n",
      "Epoch 248: | Loss: 0.59522 | Acc: 0.666 | Val ACC: 0.642\n",
      "Epoch 249: | Loss: 0.59322 | Acc: 0.665 | Val ACC: 0.635\n",
      "Epoch 250: | Loss: 0.59307 | Acc: 0.667 | Val ACC: 0.648\n",
      "Epoch 251: | Loss: 0.59150 | Acc: 0.667 | Val ACC: 0.637\n",
      "Epoch 252: | Loss: 0.59423 | Acc: 0.677 | Val ACC: 0.642\n",
      "Epoch 253: | Loss: 0.59138 | Acc: 0.677 | Val ACC: 0.633\n",
      "Epoch 254: | Loss: 0.59024 | Acc: 0.678 | Val ACC: 0.648\n",
      "Epoch 255: | Loss: 0.59168 | Acc: 0.673 | Val ACC: 0.635\n",
      "Epoch 256: | Loss: 0.59293 | Acc: 0.674 | Val ACC: 0.635\n",
      "Epoch 257: | Loss: 0.59344 | Acc: 0.682 | Val ACC: 0.633\n",
      "Epoch 258: | Loss: 0.59331 | Acc: 0.683 | Val ACC: 0.633\n",
      "Epoch 259: | Loss: 0.59231 | Acc: 0.684 | Val ACC: 0.638\n",
      "Epoch 260: | Loss: 0.59347 | Acc: 0.676 | Val ACC: 0.623\n",
      "Epoch 261: | Loss: 0.59042 | Acc: 0.683 | Val ACC: 0.633\n",
      "Epoch 262: | Loss: 0.58623 | Acc: 0.685 | Val ACC: 0.627\n",
      "Epoch 263: | Loss: 0.58839 | Acc: 0.681 | Val ACC: 0.638\n",
      "Epoch 264: | Loss: 0.58925 | Acc: 0.682 | Val ACC: 0.633\n",
      "Epoch 265: | Loss: 0.58934 | Acc: 0.672 | Val ACC: 0.637\n",
      "Epoch 266: | Loss: 0.58754 | Acc: 0.686 | Val ACC: 0.642\n",
      "Epoch 267: | Loss: 0.58727 | Acc: 0.684 | Val ACC: 0.633\n",
      "Epoch 268: | Loss: 0.58599 | Acc: 0.687 | Val ACC: 0.638\n",
      "Epoch 269: | Loss: 0.58471 | Acc: 0.682 | Val ACC: 0.637\n",
      "Epoch 270: | Loss: 0.58524 | Acc: 0.693 | Val ACC: 0.640\n",
      "Epoch 271: | Loss: 0.58321 | Acc: 0.685 | Val ACC: 0.638\n",
      "Epoch 272: | Loss: 0.58426 | Acc: 0.694 | Val ACC: 0.642\n",
      "Epoch 273: | Loss: 0.58270 | Acc: 0.688 | Val ACC: 0.648\n",
      "Epoch 274: | Loss: 0.58343 | Acc: 0.690 | Val ACC: 0.640\n",
      "Epoch 275: | Loss: 0.58466 | Acc: 0.684 | Val ACC: 0.648\n",
      "Epoch 276: | Loss: 0.58307 | Acc: 0.687 | Val ACC: 0.638\n",
      "Epoch 277: | Loss: 0.58456 | Acc: 0.691 | Val ACC: 0.640\n",
      "Epoch 278: | Loss: 0.58302 | Acc: 0.693 | Val ACC: 0.640\n",
      "Epoch 279: | Loss: 0.58019 | Acc: 0.694 | Val ACC: 0.638\n",
      "Epoch 280: | Loss: 0.58160 | Acc: 0.699 | Val ACC: 0.642\n",
      "Epoch 281: | Loss: 0.57985 | Acc: 0.691 | Val ACC: 0.648\n",
      "Epoch 282: | Loss: 0.57852 | Acc: 0.693 | Val ACC: 0.640\n",
      "Epoch 283: | Loss: 0.57898 | Acc: 0.693 | Val ACC: 0.640\n",
      "Epoch 284: | Loss: 0.57808 | Acc: 0.693 | Val ACC: 0.646\n",
      "Epoch 285: | Loss: 0.57797 | Acc: 0.693 | Val ACC: 0.648\n",
      "Epoch 286: | Loss: 0.57774 | Acc: 0.697 | Val ACC: 0.642\n",
      "Epoch 287: | Loss: 0.57857 | Acc: 0.691 | Val ACC: 0.635\n",
      "Epoch 288: | Loss: 0.57609 | Acc: 0.697 | Val ACC: 0.638\n",
      "Epoch 289: | Loss: 0.57629 | Acc: 0.692 | Val ACC: 0.638\n",
      "Epoch 290: | Loss: 0.57678 | Acc: 0.696 | Val ACC: 0.637\n",
      "Epoch 291: | Loss: 0.57622 | Acc: 0.692 | Val ACC: 0.650\n",
      "Epoch 292: | Loss: 0.57438 | Acc: 0.695 | Val ACC: 0.646\n",
      "Epoch 293: | Loss: 0.57572 | Acc: 0.691 | Val ACC: 0.640\n",
      "Epoch 294: | Loss: 0.57458 | Acc: 0.693 | Val ACC: 0.646\n",
      "Epoch 295: | Loss: 0.57364 | Acc: 0.699 | Val ACC: 0.648\n",
      "Epoch 296: | Loss: 0.57483 | Acc: 0.691 | Val ACC: 0.640\n",
      "Epoch 297: | Loss: 0.57451 | Acc: 0.695 | Val ACC: 0.642\n",
      "Epoch 298: | Loss: 0.57179 | Acc: 0.698 | Val ACC: 0.654\n",
      "Epoch 299: | Loss: 0.57020 | Acc: 0.697 | Val ACC: 0.644\n",
      "Epoch 300: | Loss: 0.57356 | Acc: 0.697 | Val ACC: 0.642\n",
      "Epoch 301: | Loss: 0.57294 | Acc: 0.692 | Val ACC: 0.642\n",
      "Epoch 302: | Loss: 0.57298 | Acc: 0.692 | Val ACC: 0.644\n",
      "Epoch 303: | Loss: 0.57420 | Acc: 0.691 | Val ACC: 0.642\n",
      "Epoch 304: | Loss: 0.57390 | Acc: 0.689 | Val ACC: 0.642\n",
      "Epoch 305: | Loss: 0.57903 | Acc: 0.684 | Val ACC: 0.642\n",
      "Epoch 306: | Loss: 0.57720 | Acc: 0.686 | Val ACC: 0.642\n",
      "Epoch 307: | Loss: 0.57308 | Acc: 0.701 | Val ACC: 0.658\n",
      "Epoch 308: | Loss: 0.57417 | Acc: 0.693 | Val ACC: 0.638\n",
      "Epoch 309: | Loss: 0.57355 | Acc: 0.697 | Val ACC: 0.648\n",
      "Epoch 310: | Loss: 0.56989 | Acc: 0.695 | Val ACC: 0.658\n",
      "Epoch 311: | Loss: 0.56673 | Acc: 0.697 | Val ACC: 0.654\n",
      "Epoch 312: | Loss: 0.56596 | Acc: 0.693 | Val ACC: 0.656\n",
      "Epoch 313: | Loss: 0.56862 | Acc: 0.690 | Val ACC: 0.644\n",
      "Epoch 314: | Loss: 0.56985 | Acc: 0.694 | Val ACC: 0.656\n",
      "Epoch 315: | Loss: 0.56503 | Acc: 0.701 | Val ACC: 0.652\n",
      "Epoch 316: | Loss: 0.56536 | Acc: 0.699 | Val ACC: 0.656\n",
      "Epoch 317: | Loss: 0.56460 | Acc: 0.699 | Val ACC: 0.648\n",
      "Epoch 318: | Loss: 0.56740 | Acc: 0.702 | Val ACC: 0.646\n",
      "Epoch 319: | Loss: 0.56427 | Acc: 0.695 | Val ACC: 0.654\n",
      "Epoch 320: | Loss: 0.56557 | Acc: 0.701 | Val ACC: 0.646\n",
      "Epoch 321: | Loss: 0.56479 | Acc: 0.703 | Val ACC: 0.660\n",
      "Epoch 322: | Loss: 0.56325 | Acc: 0.700 | Val ACC: 0.650\n",
      "Epoch 323: | Loss: 0.56868 | Acc: 0.700 | Val ACC: 0.656\n",
      "Epoch 324: | Loss: 0.56547 | Acc: 0.706 | Val ACC: 0.652\n",
      "Epoch 325: | Loss: 0.56561 | Acc: 0.700 | Val ACC: 0.656\n",
      "Epoch 326: | Loss: 0.56687 | Acc: 0.701 | Val ACC: 0.638\n",
      "Epoch 327: | Loss: 0.56997 | Acc: 0.694 | Val ACC: 0.648\n",
      "Epoch 328: | Loss: 0.56464 | Acc: 0.702 | Val ACC: 0.663\n",
      "Epoch 329: | Loss: 0.56109 | Acc: 0.709 | Val ACC: 0.663\n",
      "Epoch 330: | Loss: 0.56203 | Acc: 0.708 | Val ACC: 0.652\n",
      "Epoch 331: | Loss: 0.56334 | Acc: 0.706 | Val ACC: 0.658\n",
      "Epoch 332: | Loss: 0.56447 | Acc: 0.701 | Val ACC: 0.656\n",
      "Epoch 333: | Loss: 0.56201 | Acc: 0.709 | Val ACC: 0.667\n",
      "Epoch 334: | Loss: 0.55608 | Acc: 0.713 | Val ACC: 0.654\n",
      "Epoch 335: | Loss: 0.55880 | Acc: 0.709 | Val ACC: 0.640\n",
      "Epoch 336: | Loss: 0.56605 | Acc: 0.702 | Val ACC: 0.648\n",
      "Epoch 337: | Loss: 0.56205 | Acc: 0.707 | Val ACC: 0.658\n",
      "Epoch 338: | Loss: 0.56198 | Acc: 0.703 | Val ACC: 0.658\n",
      "Epoch 339: | Loss: 0.55889 | Acc: 0.708 | Val ACC: 0.656\n",
      "Epoch 340: | Loss: 0.55900 | Acc: 0.712 | Val ACC: 0.662\n",
      "Epoch 341: | Loss: 0.55823 | Acc: 0.707 | Val ACC: 0.663\n",
      "Epoch 342: | Loss: 0.55813 | Acc: 0.710 | Val ACC: 0.663\n",
      "Epoch 343: | Loss: 0.55876 | Acc: 0.703 | Val ACC: 0.654\n",
      "Epoch 344: | Loss: 0.55650 | Acc: 0.711 | Val ACC: 0.663\n",
      "Epoch 345: | Loss: 0.55379 | Acc: 0.711 | Val ACC: 0.654\n",
      "Epoch 346: | Loss: 0.55679 | Acc: 0.710 | Val ACC: 0.663\n",
      "Epoch 347: | Loss: 0.55246 | Acc: 0.712 | Val ACC: 0.660\n",
      "Epoch 348: | Loss: 0.55551 | Acc: 0.710 | Val ACC: 0.642\n",
      "Epoch 349: | Loss: 0.55930 | Acc: 0.710 | Val ACC: 0.654\n",
      "Epoch 350: | Loss: 0.55362 | Acc: 0.712 | Val ACC: 0.654\n",
      "Epoch 351: | Loss: 0.55614 | Acc: 0.707 | Val ACC: 0.658\n",
      "Epoch 352: | Loss: 0.55456 | Acc: 0.715 | Val ACC: 0.658\n",
      "Epoch 353: | Loss: 0.55287 | Acc: 0.715 | Val ACC: 0.652\n",
      "Epoch 354: | Loss: 0.55433 | Acc: 0.717 | Val ACC: 0.644\n",
      "Epoch 355: | Loss: 0.55563 | Acc: 0.708 | Val ACC: 0.648\n",
      "Epoch 356: | Loss: 0.55395 | Acc: 0.717 | Val ACC: 0.665\n",
      "Epoch 357: | Loss: 0.55200 | Acc: 0.717 | Val ACC: 0.656\n",
      "Epoch 358: | Loss: 0.55446 | Acc: 0.716 | Val ACC: 0.654\n",
      "Epoch 359: | Loss: 0.55355 | Acc: 0.716 | Val ACC: 0.646\n",
      "Epoch 360: | Loss: 0.55413 | Acc: 0.713 | Val ACC: 0.648\n",
      "Epoch 361: | Loss: 0.55523 | Acc: 0.716 | Val ACC: 0.642\n",
      "Epoch 362: | Loss: 0.55653 | Acc: 0.716 | Val ACC: 0.640\n",
      "Epoch 363: | Loss: 0.55644 | Acc: 0.713 | Val ACC: 0.646\n",
      "Epoch 364: | Loss: 0.55564 | Acc: 0.715 | Val ACC: 0.631\n",
      "Epoch 365: | Loss: 0.56259 | Acc: 0.716 | Val ACC: 0.635\n",
      "Epoch 366: | Loss: 0.55930 | Acc: 0.725 | Val ACC: 0.646\n",
      "Epoch 367: | Loss: 0.55318 | Acc: 0.720 | Val ACC: 0.663\n",
      "Epoch 368: | Loss: 0.55454 | Acc: 0.723 | Val ACC: 0.629\n",
      "Epoch 369: | Loss: 0.55721 | Acc: 0.722 | Val ACC: 0.648\n",
      "Epoch 370: | Loss: 0.55150 | Acc: 0.723 | Val ACC: 0.662\n",
      "Epoch 371: | Loss: 0.55272 | Acc: 0.719 | Val ACC: 0.642\n",
      "Epoch 372: | Loss: 0.55600 | Acc: 0.716 | Val ACC: 0.650\n",
      "Epoch 373: | Loss: 0.55060 | Acc: 0.727 | Val ACC: 0.660\n",
      "Epoch 374: | Loss: 0.55206 | Acc: 0.722 | Val ACC: 0.654\n",
      "Epoch 375: | Loss: 0.54881 | Acc: 0.726 | Val ACC: 0.650\n",
      "Epoch 376: | Loss: 0.54889 | Acc: 0.730 | Val ACC: 0.660\n",
      "Epoch 377: | Loss: 0.55482 | Acc: 0.723 | Val ACC: 0.642\n",
      "Epoch 378: | Loss: 0.55212 | Acc: 0.720 | Val ACC: 0.679\n",
      "Epoch 379: | Loss: 0.54584 | Acc: 0.725 | Val ACC: 0.658\n",
      "Epoch 380: | Loss: 0.54859 | Acc: 0.724 | Val ACC: 0.650\n",
      "Epoch 381: | Loss: 0.54725 | Acc: 0.729 | Val ACC: 0.652\n",
      "Epoch 382: | Loss: 0.55111 | Acc: 0.725 | Val ACC: 0.646\n",
      "Epoch 383: | Loss: 0.54657 | Acc: 0.727 | Val ACC: 0.650\n",
      "Epoch 384: | Loss: 0.54800 | Acc: 0.728 | Val ACC: 0.660\n",
      "Epoch 385: | Loss: 0.54792 | Acc: 0.731 | Val ACC: 0.662\n",
      "Epoch 386: | Loss: 0.55582 | Acc: 0.720 | Val ACC: 0.642\n",
      "Epoch 387: | Loss: 0.55241 | Acc: 0.717 | Val ACC: 0.658\n",
      "Epoch 388: | Loss: 0.55222 | Acc: 0.716 | Val ACC: 0.658\n",
      "Epoch 389: | Loss: 0.54416 | Acc: 0.722 | Val ACC: 0.658\n",
      "Epoch 390: | Loss: 0.54604 | Acc: 0.729 | Val ACC: 0.662\n",
      "Epoch 391: | Loss: 0.54651 | Acc: 0.727 | Val ACC: 0.660\n",
      "Epoch 392: | Loss: 0.54253 | Acc: 0.728 | Val ACC: 0.648\n",
      "Epoch 393: | Loss: 0.54646 | Acc: 0.722 | Val ACC: 0.656\n",
      "Epoch 394: | Loss: 0.54217 | Acc: 0.728 | Val ACC: 0.662\n",
      "Epoch 395: | Loss: 0.54183 | Acc: 0.732 | Val ACC: 0.665\n",
      "Epoch 396: | Loss: 0.54534 | Acc: 0.726 | Val ACC: 0.658\n",
      "Epoch 397: | Loss: 0.54912 | Acc: 0.727 | Val ACC: 0.642\n",
      "Epoch 398: | Loss: 0.54331 | Acc: 0.730 | Val ACC: 0.660\n",
      "Epoch 399: | Loss: 0.54907 | Acc: 0.725 | Val ACC: 0.650\n",
      "Epoch 400: | Loss: 0.54156 | Acc: 0.727 | Val ACC: 0.648\n",
      "Epoch 401: | Loss: 0.54494 | Acc: 0.724 | Val ACC: 0.658\n",
      "Epoch 402: | Loss: 0.54559 | Acc: 0.727 | Val ACC: 0.656\n",
      "Epoch 403: | Loss: 0.54156 | Acc: 0.726 | Val ACC: 0.652\n",
      "Epoch 404: | Loss: 0.54787 | Acc: 0.723 | Val ACC: 0.662\n",
      "Epoch 405: | Loss: 0.54236 | Acc: 0.726 | Val ACC: 0.660\n",
      "Epoch 406: | Loss: 0.54583 | Acc: 0.733 | Val ACC: 0.650\n",
      "Epoch 407: | Loss: 0.54047 | Acc: 0.729 | Val ACC: 0.667\n",
      "Epoch 408: | Loss: 0.54075 | Acc: 0.731 | Val ACC: 0.656\n",
      "Epoch 409: | Loss: 0.54475 | Acc: 0.727 | Val ACC: 0.663\n",
      "Epoch 410: | Loss: 0.54607 | Acc: 0.726 | Val ACC: 0.663\n",
      "Epoch 411: | Loss: 0.54588 | Acc: 0.727 | Val ACC: 0.646\n",
      "Epoch 412: | Loss: 0.54785 | Acc: 0.724 | Val ACC: 0.667\n",
      "Epoch 413: | Loss: 0.54503 | Acc: 0.718 | Val ACC: 0.642\n",
      "Epoch 414: | Loss: 0.55112 | Acc: 0.716 | Val ACC: 0.652\n",
      "Epoch 415: | Loss: 0.54090 | Acc: 0.733 | Val ACC: 0.662\n",
      "Epoch 416: | Loss: 0.53970 | Acc: 0.737 | Val ACC: 0.665\n",
      "Epoch 417: | Loss: 0.54703 | Acc: 0.722 | Val ACC: 0.652\n",
      "Epoch 418: | Loss: 0.54022 | Acc: 0.734 | Val ACC: 0.662\n",
      "Epoch 419: | Loss: 0.54988 | Acc: 0.720 | Val ACC: 0.652\n",
      "Epoch 420: | Loss: 0.54624 | Acc: 0.726 | Val ACC: 0.662\n",
      "Epoch 421: | Loss: 0.54149 | Acc: 0.734 | Val ACC: 0.652\n",
      "Epoch 422: | Loss: 0.54223 | Acc: 0.730 | Val ACC: 0.658\n",
      "Epoch 423: | Loss: 0.54054 | Acc: 0.733 | Val ACC: 0.660\n",
      "Epoch 424: | Loss: 0.54177 | Acc: 0.733 | Val ACC: 0.660\n",
      "Epoch 425: | Loss: 0.53849 | Acc: 0.736 | Val ACC: 0.656\n",
      "Epoch 426: | Loss: 0.54352 | Acc: 0.736 | Val ACC: 0.660\n",
      "Epoch 427: | Loss: 0.53724 | Acc: 0.737 | Val ACC: 0.660\n",
      "Epoch 428: | Loss: 0.54212 | Acc: 0.733 | Val ACC: 0.665\n",
      "Epoch 429: | Loss: 0.54356 | Acc: 0.729 | Val ACC: 0.660\n",
      "Epoch 430: | Loss: 0.53928 | Acc: 0.730 | Val ACC: 0.654\n",
      "Epoch 431: | Loss: 0.54313 | Acc: 0.733 | Val ACC: 0.656\n",
      "Epoch 432: | Loss: 0.54206 | Acc: 0.735 | Val ACC: 0.658\n",
      "Epoch 433: | Loss: 0.53683 | Acc: 0.739 | Val ACC: 0.662\n",
      "Epoch 434: | Loss: 0.53687 | Acc: 0.740 | Val ACC: 0.665\n",
      "Epoch 435: | Loss: 0.53396 | Acc: 0.738 | Val ACC: 0.662\n",
      "Epoch 436: | Loss: 0.53494 | Acc: 0.739 | Val ACC: 0.663\n",
      "Epoch 437: | Loss: 0.53628 | Acc: 0.737 | Val ACC: 0.642\n",
      "Epoch 438: | Loss: 0.53979 | Acc: 0.739 | Val ACC: 0.642\n",
      "Epoch 439: | Loss: 0.54009 | Acc: 0.731 | Val ACC: 0.654\n",
      "Epoch 440: | Loss: 0.53377 | Acc: 0.738 | Val ACC: 0.669\n",
      "Epoch 441: | Loss: 0.52979 | Acc: 0.742 | Val ACC: 0.667\n",
      "Epoch 442: | Loss: 0.53044 | Acc: 0.743 | Val ACC: 0.665\n",
      "Epoch 443: | Loss: 0.53147 | Acc: 0.744 | Val ACC: 0.648\n",
      "Epoch 444: | Loss: 0.53114 | Acc: 0.738 | Val ACC: 0.662\n",
      "Epoch 445: | Loss: 0.53178 | Acc: 0.744 | Val ACC: 0.662\n",
      "Epoch 446: | Loss: 0.53320 | Acc: 0.742 | Val ACC: 0.638\n",
      "Epoch 447: | Loss: 0.53437 | Acc: 0.737 | Val ACC: 0.652\n",
      "Epoch 448: | Loss: 0.53253 | Acc: 0.738 | Val ACC: 0.665\n",
      "Epoch 449: | Loss: 0.53003 | Acc: 0.740 | Val ACC: 0.658\n",
      "Epoch 450: | Loss: 0.52524 | Acc: 0.748 | Val ACC: 0.658\n",
      "Epoch 451: | Loss: 0.52406 | Acc: 0.749 | Val ACC: 0.660\n",
      "Epoch 452: | Loss: 0.52509 | Acc: 0.747 | Val ACC: 0.650\n",
      "Epoch 453: | Loss: 0.52442 | Acc: 0.744 | Val ACC: 0.650\n",
      "Epoch 454: | Loss: 0.52782 | Acc: 0.743 | Val ACC: 0.656\n",
      "Epoch 455: | Loss: 0.52566 | Acc: 0.747 | Val ACC: 0.646\n",
      "Epoch 456: | Loss: 0.52577 | Acc: 0.744 | Val ACC: 0.650\n",
      "Epoch 457: | Loss: 0.52357 | Acc: 0.748 | Val ACC: 0.656\n",
      "Epoch 458: | Loss: 0.52627 | Acc: 0.743 | Val ACC: 0.644\n",
      "Epoch 459: | Loss: 0.53004 | Acc: 0.741 | Val ACC: 0.638\n",
      "Epoch 460: | Loss: 0.52995 | Acc: 0.739 | Val ACC: 0.650\n",
      "Epoch 461: | Loss: 0.52601 | Acc: 0.744 | Val ACC: 0.640\n",
      "Epoch 462: | Loss: 0.52466 | Acc: 0.744 | Val ACC: 0.640\n",
      "Epoch 463: | Loss: 0.52579 | Acc: 0.740 | Val ACC: 0.642\n",
      "Epoch 464: | Loss: 0.52222 | Acc: 0.746 | Val ACC: 0.658\n",
      "Epoch 465: | Loss: 0.52545 | Acc: 0.744 | Val ACC: 0.646\n",
      "Epoch 466: | Loss: 0.52523 | Acc: 0.739 | Val ACC: 0.644\n",
      "Epoch 467: | Loss: 0.52859 | Acc: 0.738 | Val ACC: 0.648\n",
      "Epoch 468: | Loss: 0.52411 | Acc: 0.743 | Val ACC: 0.640\n",
      "Epoch 469: | Loss: 0.52503 | Acc: 0.745 | Val ACC: 0.646\n",
      "Epoch 470: | Loss: 0.52431 | Acc: 0.744 | Val ACC: 0.648\n",
      "Epoch 471: | Loss: 0.52436 | Acc: 0.751 | Val ACC: 0.646\n",
      "Epoch 472: | Loss: 0.52699 | Acc: 0.741 | Val ACC: 0.638\n",
      "Epoch 473: | Loss: 0.53167 | Acc: 0.741 | Val ACC: 0.658\n",
      "Epoch 474: | Loss: 0.52712 | Acc: 0.749 | Val ACC: 0.633\n",
      "Epoch 475: | Loss: 0.52767 | Acc: 0.736 | Val ACC: 0.644\n",
      "Epoch 476: | Loss: 0.52679 | Acc: 0.739 | Val ACC: 0.638\n",
      "Epoch 477: | Loss: 0.52550 | Acc: 0.739 | Val ACC: 0.656\n",
      "Epoch 478: | Loss: 0.52199 | Acc: 0.751 | Val ACC: 0.631\n",
      "Epoch 479: | Loss: 0.52483 | Acc: 0.745 | Val ACC: 0.650\n",
      "Epoch 480: | Loss: 0.52183 | Acc: 0.749 | Val ACC: 0.646\n",
      "Epoch 481: | Loss: 0.52273 | Acc: 0.743 | Val ACC: 0.646\n",
      "Epoch 482: | Loss: 0.52102 | Acc: 0.744 | Val ACC: 0.642\n",
      "Epoch 483: | Loss: 0.52184 | Acc: 0.748 | Val ACC: 0.642\n",
      "Epoch 484: | Loss: 0.52529 | Acc: 0.745 | Val ACC: 0.637\n",
      "Epoch 485: | Loss: 0.52255 | Acc: 0.743 | Val ACC: 0.644\n",
      "Epoch 486: | Loss: 0.52029 | Acc: 0.745 | Val ACC: 0.637\n",
      "Epoch 487: | Loss: 0.52266 | Acc: 0.746 | Val ACC: 0.650\n",
      "Epoch 488: | Loss: 0.52109 | Acc: 0.750 | Val ACC: 0.642\n",
      "Epoch 489: | Loss: 0.52301 | Acc: 0.743 | Val ACC: 0.629\n",
      "Epoch 490: | Loss: 0.52125 | Acc: 0.747 | Val ACC: 0.656\n",
      "Epoch 491: | Loss: 0.52146 | Acc: 0.750 | Val ACC: 0.635\n",
      "Epoch 492: | Loss: 0.52088 | Acc: 0.745 | Val ACC: 0.652\n",
      "Epoch 493: | Loss: 0.52060 | Acc: 0.750 | Val ACC: 0.662\n",
      "Epoch 494: | Loss: 0.51899 | Acc: 0.748 | Val ACC: 0.640\n",
      "Epoch 495: | Loss: 0.52066 | Acc: 0.744 | Val ACC: 0.652\n",
      "Epoch 496: | Loss: 0.51873 | Acc: 0.743 | Val ACC: 0.644\n",
      "Epoch 497: | Loss: 0.51996 | Acc: 0.747 | Val ACC: 0.656\n",
      "Epoch 498: | Loss: 0.51882 | Acc: 0.754 | Val ACC: 0.652\n",
      "Epoch 499: | Loss: 0.51678 | Acc: 0.743 | Val ACC: 0.665\n",
      "Epoch 500: | Loss: 0.51660 | Acc: 0.748 | Val ACC: 0.650\n",
      "Epoch 501: | Loss: 0.51978 | Acc: 0.747 | Val ACC: 0.650\n",
      "Epoch 502: | Loss: 0.51706 | Acc: 0.745 | Val ACC: 0.650\n",
      "Epoch 503: | Loss: 0.51660 | Acc: 0.751 | Val ACC: 0.650\n",
      "Epoch 504: | Loss: 0.51777 | Acc: 0.751 | Val ACC: 0.650\n",
      "Epoch 505: | Loss: 0.51641 | Acc: 0.750 | Val ACC: 0.654\n",
      "Epoch 506: | Loss: 0.51272 | Acc: 0.750 | Val ACC: 0.658\n",
      "Epoch 507: | Loss: 0.51333 | Acc: 0.752 | Val ACC: 0.662\n",
      "Epoch 508: | Loss: 0.51410 | Acc: 0.753 | Val ACC: 0.646\n",
      "Epoch 509: | Loss: 0.51686 | Acc: 0.753 | Val ACC: 0.637\n",
      "Epoch 510: | Loss: 0.51537 | Acc: 0.750 | Val ACC: 0.662\n",
      "Epoch 511: | Loss: 0.51366 | Acc: 0.750 | Val ACC: 0.648\n",
      "Epoch 512: | Loss: 0.51421 | Acc: 0.754 | Val ACC: 0.648\n",
      "Epoch 513: | Loss: 0.51528 | Acc: 0.750 | Val ACC: 0.658\n",
      "Epoch 514: | Loss: 0.51593 | Acc: 0.748 | Val ACC: 0.660\n",
      "Epoch 515: | Loss: 0.51426 | Acc: 0.749 | Val ACC: 0.656\n",
      "Epoch 516: | Loss: 0.51487 | Acc: 0.746 | Val ACC: 0.658\n",
      "Epoch 517: | Loss: 0.51190 | Acc: 0.756 | Val ACC: 0.642\n",
      "Epoch 518: | Loss: 0.51801 | Acc: 0.742 | Val ACC: 0.652\n",
      "Epoch 519: | Loss: 0.51387 | Acc: 0.744 | Val ACC: 0.660\n",
      "Epoch 520: | Loss: 0.51705 | Acc: 0.748 | Val ACC: 0.656\n",
      "Epoch 521: | Loss: 0.51397 | Acc: 0.746 | Val ACC: 0.656\n",
      "Epoch 522: | Loss: 0.51072 | Acc: 0.746 | Val ACC: 0.650\n",
      "Epoch 523: | Loss: 0.50785 | Acc: 0.759 | Val ACC: 0.656\n",
      "Epoch 524: | Loss: 0.51201 | Acc: 0.761 | Val ACC: 0.660\n",
      "Epoch 525: | Loss: 0.50876 | Acc: 0.751 | Val ACC: 0.663\n",
      "Epoch 526: | Loss: 0.51014 | Acc: 0.749 | Val ACC: 0.656\n",
      "Epoch 527: | Loss: 0.51330 | Acc: 0.750 | Val ACC: 0.654\n",
      "Epoch 528: | Loss: 0.51031 | Acc: 0.759 | Val ACC: 0.654\n",
      "Epoch 529: | Loss: 0.51710 | Acc: 0.755 | Val ACC: 0.621\n",
      "Epoch 530: | Loss: 0.52187 | Acc: 0.749 | Val ACC: 0.638\n",
      "Epoch 531: | Loss: 0.51705 | Acc: 0.755 | Val ACC: 0.617\n",
      "Epoch 532: | Loss: 0.52495 | Acc: 0.747 | Val ACC: 0.625\n",
      "Epoch 533: | Loss: 0.52789 | Acc: 0.746 | Val ACC: 0.619\n",
      "Epoch 534: | Loss: 0.52412 | Acc: 0.732 | Val ACC: 0.650\n",
      "Epoch 535: | Loss: 0.52113 | Acc: 0.741 | Val ACC: 0.654\n",
      "Epoch 536: | Loss: 0.51891 | Acc: 0.741 | Val ACC: 0.644\n",
      "Epoch 537: | Loss: 0.51758 | Acc: 0.743 | Val ACC: 0.644\n",
      "Epoch 538: | Loss: 0.50934 | Acc: 0.756 | Val ACC: 0.652\n",
      "Epoch 539: | Loss: 0.51642 | Acc: 0.750 | Val ACC: 0.648\n",
      "Epoch 540: | Loss: 0.50958 | Acc: 0.751 | Val ACC: 0.646\n",
      "Epoch 541: | Loss: 0.51604 | Acc: 0.751 | Val ACC: 0.650\n",
      "Epoch 542: | Loss: 0.51980 | Acc: 0.736 | Val ACC: 0.644\n",
      "Epoch 543: | Loss: 0.51667 | Acc: 0.746 | Val ACC: 0.656\n",
      "Epoch 544: | Loss: 0.51479 | Acc: 0.744 | Val ACC: 0.658\n",
      "Epoch 545: | Loss: 0.52074 | Acc: 0.738 | Val ACC: 0.660\n",
      "Epoch 546: | Loss: 0.51686 | Acc: 0.753 | Val ACC: 0.656\n",
      "Epoch 547: | Loss: 0.51210 | Acc: 0.751 | Val ACC: 0.663\n",
      "Epoch 548: | Loss: 0.51286 | Acc: 0.750 | Val ACC: 0.663\n",
      "Epoch 549: | Loss: 0.51661 | Acc: 0.749 | Val ACC: 0.654\n",
      "Epoch 550: | Loss: 0.51377 | Acc: 0.751 | Val ACC: 0.660\n",
      "Epoch 551: | Loss: 0.51377 | Acc: 0.754 | Val ACC: 0.660\n",
      "Epoch 552: | Loss: 0.51310 | Acc: 0.753 | Val ACC: 0.677\n",
      "Epoch 553: | Loss: 0.50913 | Acc: 0.749 | Val ACC: 0.650\n",
      "Epoch 554: | Loss: 0.50812 | Acc: 0.754 | Val ACC: 0.652\n",
      "Epoch 555: | Loss: 0.51099 | Acc: 0.756 | Val ACC: 0.648\n",
      "Epoch 556: | Loss: 0.50732 | Acc: 0.755 | Val ACC: 0.656\n",
      "Epoch 557: | Loss: 0.50485 | Acc: 0.757 | Val ACC: 0.660\n",
      "Epoch 558: | Loss: 0.50769 | Acc: 0.749 | Val ACC: 0.662\n",
      "Epoch 559: | Loss: 0.50928 | Acc: 0.756 | Val ACC: 0.660\n",
      "Epoch 560: | Loss: 0.50876 | Acc: 0.746 | Val ACC: 0.654\n",
      "Epoch 561: | Loss: 0.50955 | Acc: 0.754 | Val ACC: 0.658\n",
      "Epoch 562: | Loss: 0.50714 | Acc: 0.754 | Val ACC: 0.658\n",
      "Epoch 563: | Loss: 0.51161 | Acc: 0.752 | Val ACC: 0.662\n",
      "Epoch 564: | Loss: 0.50920 | Acc: 0.756 | Val ACC: 0.658\n",
      "Epoch 565: | Loss: 0.51236 | Acc: 0.747 | Val ACC: 0.648\n",
      "Epoch 566: | Loss: 0.50934 | Acc: 0.754 | Val ACC: 0.673\n",
      "Epoch 567: | Loss: 0.50605 | Acc: 0.754 | Val ACC: 0.665\n",
      "Epoch 568: | Loss: 0.51190 | Acc: 0.752 | Val ACC: 0.665\n",
      "Epoch 569: | Loss: 0.51094 | Acc: 0.751 | Val ACC: 0.665\n",
      "Epoch 570: | Loss: 0.51615 | Acc: 0.749 | Val ACC: 0.660\n",
      "Epoch 571: | Loss: 0.51663 | Acc: 0.750 | Val ACC: 0.662\n",
      "Epoch 572: | Loss: 0.50808 | Acc: 0.756 | Val ACC: 0.667\n",
      "Epoch 573: | Loss: 0.51162 | Acc: 0.744 | Val ACC: 0.671\n",
      "Epoch 574: | Loss: 0.51581 | Acc: 0.751 | Val ACC: 0.660\n",
      "Epoch 575: | Loss: 0.51279 | Acc: 0.756 | Val ACC: 0.652\n",
      "Epoch 576: | Loss: 0.51698 | Acc: 0.746 | Val ACC: 0.642\n",
      "Epoch 577: | Loss: 0.51623 | Acc: 0.742 | Val ACC: 0.681\n",
      "Epoch 578: | Loss: 0.50838 | Acc: 0.757 | Val ACC: 0.658\n",
      "Epoch 579: | Loss: 0.50999 | Acc: 0.751 | Val ACC: 0.662\n",
      "Epoch 580: | Loss: 0.51139 | Acc: 0.755 | Val ACC: 0.675\n",
      "Epoch 581: | Loss: 0.51244 | Acc: 0.750 | Val ACC: 0.665\n",
      "Epoch 582: | Loss: 0.51072 | Acc: 0.755 | Val ACC: 0.663\n",
      "Epoch 583: | Loss: 0.51666 | Acc: 0.753 | Val ACC: 0.665\n",
      "Epoch 584: | Loss: 0.51124 | Acc: 0.754 | Val ACC: 0.679\n",
      "Epoch 585: | Loss: 0.51556 | Acc: 0.751 | Val ACC: 0.640\n",
      "Epoch 586: | Loss: 0.51215 | Acc: 0.748 | Val ACC: 0.654\n",
      "Epoch 587: | Loss: 0.50865 | Acc: 0.757 | Val ACC: 0.637\n",
      "Epoch 588: | Loss: 0.50852 | Acc: 0.753 | Val ACC: 0.663\n",
      "Epoch 589: | Loss: 0.50769 | Acc: 0.754 | Val ACC: 0.644\n",
      "Epoch 590: | Loss: 0.51628 | Acc: 0.749 | Val ACC: 0.640\n",
      "Epoch 591: | Loss: 0.51625 | Acc: 0.745 | Val ACC: 0.633\n",
      "Epoch 592: | Loss: 0.51280 | Acc: 0.752 | Val ACC: 0.650\n",
      "Epoch 593: | Loss: 0.51528 | Acc: 0.750 | Val ACC: 0.650\n",
      "Epoch 594: | Loss: 0.51450 | Acc: 0.752 | Val ACC: 0.665\n",
      "Epoch 595: | Loss: 0.50819 | Acc: 0.759 | Val ACC: 0.642\n",
      "Epoch 596: | Loss: 0.50601 | Acc: 0.764 | Val ACC: 0.652\n",
      "Epoch 597: | Loss: 0.49823 | Acc: 0.771 | Val ACC: 0.675\n",
      "Epoch 598: | Loss: 0.49588 | Acc: 0.773 | Val ACC: 0.671\n",
      "Epoch 599: | Loss: 0.49871 | Acc: 0.769 | Val ACC: 0.662\n"
     ]
    }
   ],
   "source": [
    "clients = [\n",
    "    SyncFNNClient(id=id, \n",
    "    model=nn.Sequential(\n",
    "        nn.Linear(50, 64),\n",
    "        nn.ReLU(),\n",
    "        nn.Linear(64, 32),\n",
    "        nn.ReLU()),\n",
    "    courier=courier,\n",
    "    train_loader=loader_dict[id]['train_loader'],\n",
    "    test_loader=loader_dict[id]['test_loader'],\n",
    "    config_dir='simple_config.ini')\n",
    "for id in client_id_list]\n",
    "strategy = SyncConcatStrategy(courier=courier, clients=clients)\n",
    "server =  SyncFNNServer(\n",
    "    strategy=strategy,\n",
    "    courier=courier,\n",
    "    top_model=nn.Sequential(\n",
    "        nn.Linear(10 * 32, 32),\n",
    "        nn.ReLU(True),\n",
    "        nn.Linear(32, 1),\n",
    "        nn.Sigmoid()),\n",
    "    emb_model=nn.Sequential( \n",
    "        nn.Linear(50, 64),\n",
    "        nn.ReLU(),\n",
    "        nn.Linear(64, 32),\n",
    "        nn.ReLU()),\n",
    "    train_loader=loader_dict['server']['train_loader'],\n",
    "    test_loader=loader_dict['server']['test_loader'],\n",
    "    config_dir='simple_config.ini')\n",
    "\n",
    "print('Training Starts')\n",
    "print('-'*89)\n",
    "for epoch in range(600):\n",
    "    server.fit(epoch)\n",
    "history5 = server.get_history()\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x277816a3ac0>"
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABqnUlEQVR4nO2ddXgVx/rHP3M87k4SHIK7lhYpFFpqUKHu1PVXt1u5t95eqMutGy3WUgoFCqUtFHe3EKIQ9+To/P7Yk3NyIhAoFBLm8zx5sjs7MzuzhO/OvvPOO0JKiUKhUChaLrqT3QCFQqFQnFiU0CsUCkULRwm9QqFQtHCU0CsUCkULRwm9QqFQtHAMJ7sBDREZGSlbt259spuhUCgUzYZ169blSymjGrp2Sgp969atWbt27cluhkKhUDQbhBAHGrumTDcKhULRwlFCr1AoFC0cJfQKhULRwjklbfQNYbfbyczMpLq6+mQ3pdlgsVho1aoVRqPxZDdFoVCcRJqN0GdmZhIUFETr1q0RQpzs5pzySCkpKCggMzOTNm3anOzmKBSKk0izMd1UV1cTERGhRL6JCCGIiIhQX0AKhaL5CD2gRP4oUc9LoVBAMxN6hUKhaC7YXXZm75mNS7pOdlOU0B8NN954I9HR0XTr1u2oywYGBgKQnZ3NJZdccsxtmDJlCpWVlcdcXqFQ/DN8uvVTnv7raX5O/dmTlleZx4jvR7CnaE+9/LsKd+FwOU5IW5TQHwXXX389v/zyy9+qIz4+nhkzZhxzeSX0CkXz4GDFQQAq7BWetKWZS8mvyuerHV950nLKc1h8YDGX/HQJL69++YS0RQn9UXDmmWcSHh5+2DyHDh3i4osvpmfPnvTs2ZO//vrL53paWprni8DpdPLQQw/Rv39/evTowQcffADA0qVLGT58OJdccgmdO3fmqquuQkrJm2++SXZ2NiNGjGDEiBE4nU6uv/56unXrRvfu3fnvf/97YjquUCgAyK/K56d9PzUpr91lB8Cg8zo36oUegFl7ZlHt0Bwlrpp3FfctvQ+ARQcWHcfWemk27pW1efanbWzPLj2udXaJD+Zf53f92/Xcc889nHXWWcyePRun00l5eXmjeT/++GNCQkJYs2YNVquVoUOHMmbMGAA2bNjAtm3biI+PZ+jQoSxfvpx77rmHN954g99++43IyEjWrVtHVlYWW7duBaC4uPhvt1+haKksz1qOUzo5s9WZx1zHVT9fRXZFNsMShhFqCfW5Ni91HnGBcfSO7g14hb62OUYnvGPr9za9x/197yevKs+TVlBdcMxtOxzNUuhPZZYsWcIXX3wBgF6vJyQkpNG8CxcuZPPmzR5TTklJCXv27MFkMjFgwABatWoFQK9evUhLS+OMM87wKd+2bVtSU1O5++67Oe+88zwvCYVCUZ/bfr0NgC3XbTlsPpvTxkurXyKjLIOH+j9Ex7COgLY2JbsiG4BDlYc8Qp9RmsEn2z5hxu4ZPvXbnZrQl9vLsTvtlNhKPOIPUFRdBIBZb8bqtAIQ4x+DlPK4e8w1S6E/HiPvUwEpJW+99RbnnHOOT/rSpUsxm82ec71ej8NRf5ImLCyMTZs2sWDBAt555x2+//57PvnkkxPeboWiubO9YDvltnIGxA2g0l7JA78/wJ0976RzRGcmzplIWmkaAA///jA/XPQD4DvaPlR5iE7hnQB4ZsUzrD642nNtc95mekT1oMKh2ebLbGXc89s9LMtaxtjWYz359Do9NqdNE3kJKbmD+fS2N0+IW7Sy0R9nRo0axXvvvQdoNvjS0sZNTOeccw7vvfcedrv2lt+9ezcVFRWN5gcICgqirKwMgPz8fFwuFxMnTuT5559n/fr1x6kXCkXLpcpRxeVzL+emhTcBsDF3I8uzlnPlvCs5VHHII/LguxYloyzDc5xbmQvAnYvv9BF50GzuY2aMYUfBDgC2FWxjWdYyADblbfLk0ws9qSWpAMSWteWs1EmsnJV6HHvqRQn9UXDFFVcwePBgdu3aRatWrfj444/r5Zk6dSq//fYb3bt3p2/fvmzbtq3R+m6++Wa6dOlCnz596NatG7feemuDI/faTJ48mXHjxjFixAiysrIYPnw4vXr14vrrr+fFF1/8231UKFoK03ZOY/7++fXSa7xhQPuq3pLvNeWUWEt88u4t3utJSy9N96QfqjyE3Wnnj8w/Grx3TkUOhdWFAKzKWeWTXkNaaRqX/nQpAI/EPwtAVYWdE4GQUp6Qiv8O/fr1k3U3HtmxYwcpKSknqUXNF/XcFKcr3T/vDnht5jXnH47+kMmLJgMwqdMkcipy+D3zdwBePetVHvr9IZ96hiUM492z3+WZv55h3v55xPjHYHfZSQlP4df0XwG4pfstfLTlo2Nq58QOE0lZdQH5u4qJSgrissf7H1M9Qoh1Usp+DV1TI3qFQtHiyKvM8zmvPQlaM9IGmLZrGlvyt2DWa3Niuwt3A/DluC89ef7M+pPVOauZuWcmo5NHc27bc8kqz/KIPMAN3W7g8YGPA/DNud940l8c9iLR/tGNtrNXVC+eGPg02/ZqbaqusHMiBt9K6BUKRbNn/v75dP+8OwVV2oTp7qLdPtcr7d5FhjV5aiisLmRk4kifcknBST55auz5F7W/iMFxg32uLZu0jCBTEJM6TWLehHl0j+ruuTa+7XheOOMFQPsyABgSP4SL218MwFVdruJAQSUhLsFGk4MdfYNOyGRss/S6USgUitp8sElbbJhRlkGEXwT5Vfmea06X08cun16WXq98v9h+zE+bz/Ls5QAEm4K5o+cdvLvpXU+eoQlD6RvTF6fLCUC3iG58NOYjAk1aeBMhBIlBiQB0iehCtJ82kh8YN5BVV67C3+jPoYpDhJhD0Ov03EYocZ9dTka7m7DI8bQ17OL71QU8Pb4Lfib98Xw8akSvUCiaP0VWzSe91KZ5udWeVN1VtItLfvLGl/pu13cAjE4e7UnrGtGVwXGDcbgcBJmCMOgM3N7rdow6bdOe4YnDef/s99EJHUa9kR8u/IH3R7/vEXkAl8trcvlu/He8Neotz7m/0R+AmKWvYvnxbow6I/GLnkUAfjuXAHCZaS6re8477iIPSugVCkULoNhaDHjNMjXnAPf/dr9P3vCKeKLLkukc3tmbZglnaMJQQBvN11Bj239i4BM+dbQLbUeI2bsY8redubR9fB778+u4R2esgWdCIGs9SAlrPoIt32vHbsqc2sg/WJ+L0VbMiUCZbhQKRbPD5rSxMXcjXSK64Gfw84QCfvqvpym3l3tG9gDZFdmMazMOi97C7D0/cNnmRwAYeHkEb23QRt1hljDah7YH8IziAQbGDmTVwVVE+UUdtj3frNbMQat/+Yo2Z/Xmxa1B5JVZeS1igTaa3vkzlHnNR67CNM8ou9QZA0CQ/hCYuxzzMzkcakTfRDIyMhgxYgQpKSl07dqVqVOnHlV5FaZYoTh+zNwzk5sW3sTr6173iRUD8MqaV3xCA4M2Efrc0OeYMcQbkKxwsYELtt4NgMVg8Qj9uW3O9eSZOnIqP170I3qdrzmltpkGIKOwkpmmf3H53ofh49F88Hsqs9ZnkV3hHrxvmwXTrvDkXzj9Pc9xqTMak6jALCpAhSk+uRgMBl5//XV27NjBypUreeedd9i+fftR16PCFCsUf5+axUtrD64luzy73vVyezlJQV7PmZp4NX4Or019+9JDxJe1R0hNBmMCYvhhxDzOM1/myRNgDKBtSFufuqvL7bx3x2+snLENtsygsNxKu9xF9NV5Y8x3l1m8zvek7jTx7qHZHDyovSh+i7uZHyquIXrfIU/eUmcMQfpchADKc4/1kRwWJfRNJC4ujj59+gBaGIKUlBSysrLq5VNhihWK48v03dPriXmNF01aaRrvbtQ8YwS+bok1LpJB1eGseiOf8qJqbFXOevU/Y3ufaveK1MWvp/LT1E0N+rJ//lcaG1ML+fjBPwFY9+shdnz5Nfc9+wUjy2KxSxMAO6uGM7akPQeLr2NLqrZ+aXX5FTxrfYAbUkeSVTaBFWXXISWsdnWizBlNsP4Q9LgcKvLq3fd40CQbvRBiLDAV0AP/k1K+VOf6Q8BVtepMAaKklIVCiDSgDHACjsZWbh0V8x+Fg4ePQHfUxHaHcS8dOR+aWG/YsIGBAwfWu6bCFCsUxweb00bfr/p6zmdeMNMzMj9YcZAuEV3os+hydkWuonuf7riki20F2whwBRFWGk/7AyPQ2xOxGiooy7ey8c8swsP96t0nZ30Vv+p2Mv7m7tit2ovAWunAEmCkqtxGSW4VuigzC77fxVaHjtZ4zThLSu9mIHAQ+PDQd5wd8l/2VA2rd48MW28ibVB7ze3bpc/ToUcZpXnRJJo2QruR4B9xPB5dPY4o9EIIPfAOMBrIBNYIIeZIKT12Cynlq8Cr7vznA/dLKQtrVTNCSplPC6C8vJyJEycyZcoUgoOD611XYYoViqPHYXPidErMfgYcLgdPLn+SMcm+f8+3LrqV3y77DYCDlQc5M/5MAq2h9M06h5CJ2WSVZ5GVkc+kTY97yqQQyZZYLbzBH/vyGWeJa/D+uzN9gw8e3F9K624R/P7NLvatzyPmgiSGVnsnaYMM6ZQ5kupWw68l92uTqk1AV9WNygwXDqkjyFgIPSdpPyeApozoBwB7pZSpAEKIacCFQGMG6iuAb49P8xqhiSPv443dbmfixIlcddVVTJgw4W/Xp8IUK1oyTpeTtNI02oW2O2LeGa+soyCznDvfH8n+ogP4ze7Cq8nvQqg3j39FKNXldkoKKpGFJiLjYz3Xwkricf7UhkmF9ePEmCq0sOZr0ovIza+iQwP3zynX4sFLAULCz29v4owb2+Bwf5EfmuO7yEpHw6tXBU7K3F40DRGVFEReepn3vtnu+YHApr0cjpWm2OgTgIxa55nutHoIIfyBscDMWskSWCiEWCeEmNzYTYQQk4UQa4UQa/PyToyd6u8gpeSmm24iJSWFBx54oNF8Kkyx4nRkW8E2DlX4itXzK5/n1q/u489tWvTGualzeW/Tex5XyNoUZGqCmrGjkKzMXCIrWzF+xx0YHdqAx+A0cu6qu/n4wT+Z8eI6Lt34CIF5XkEVOYGUF2piXZSQQ5XBazLtVBap1SEF5WU2XNS3v1utTgrKrVQJ77Wc2f/DnO7dI7p2OX2t4xHBb3uOcw3VjT4jk0VPVHIQALFtfb/0Y0JOzM5SNTRF6Bt6dTUWded8YHkds81QKWUfYBxwpxCiwX28pJQfSin7SSn7RUUd3mf1ZLB8+XK+/PJLlixZQq9evejVqxfz5s2rl0+FKVacjkyaO4mxs8ayr3gfd/x6BwcrDrJjRxoXb72fDZ8UYK20M/3b3/hz/jbWHFzjU/bHvT96judM3cieH7wifeWGpxm+9woiKxLr3dO+xms63b+wHJ1Fz5vBVXwbuJjpPV/mt7C9PvmHVxvpbTNQJiTBXUPxj7SQcLY2Zg1wwU2fr8UlIdMfnEhCbEXYpB9GfRGdLUvY2k6bbO3l/wOjQ7yOD2EGr1PG4Au7Nfh8Jk89i+teGkqrTmEAxLbzCv1VkXcg/Bo38R4PjhimWAgxGHhGSnmO+/wxACllPVURQswGpkspv6l7zX39GaBcSvna4e6pwhQfP9RzUxxPpu2cRmpJKi7p4tKOl9IpvBOV9koGfqM5JlyVchVf7/iaYQnDsG71p8/Oc+vV8f2IZxkRNZqUFWMYflVnzl56BretnFIrh6Tu+HJ/2GbaFPU4bNucengjqApL/HcYQzbw3JDnKdydQvVPdbzjjILbpw5HCC0+zfyPt7JrzSE+DrZyS5mFPqMSWbk4HT8pCNDlE6zPY0LE4/wZNYmB+T9gdFUjzIH8FPgz6dsLueKCvXw7pz2B4Waue2EoTqeL9+9c6nPLO9/XgqZJKcncUURCp1Dec+e5I+ZixFXToePfm2M7XJjiptjo1wAdhBBtgCxgEnBlAzcJAc4Crq6VFgDopJRl7uMxwHNH3wWFQnGyqbBX8J9V//GcL0lfwpLLlvhspjFt6/fohYHtedvpUDmkwXrsxRD4Zw/yHeXMf38zZ5kur5OjvhHhSCIPUOE2NLjs2ug4wi+ci8/rxF/VejYs8trY+49JRqfz3qPv2UmkrsmlnV2HXoLZz4DR3YYKVyQRBq3ssLxpWoHYbjDxf4yytGPvukOEDR/BdYNtGM2agUSvb9xQIoQgsUu4b9qzJY3kPn4cUeillA4hxF3AAjT3yk+klNuEELe5r7/vznoxsFBKWdvIHAPMdofdNADfSCl/QaFQNCtyK3N9IkIC5FXl8cDSB1h3UJsbCrCGcM16bRyXF5BBXkA6lcYyDC4jJqfFU27Clv/zLFyqKLbRUTS80YbUgahlzj8YlEpsWVv+aPM9fvYg+meO81xbYbaz1aS5Rv5n+P0EhI3zhgWe2N4j9Fc+M5Cw2ACf+4REawHHgl2auJv8DD7CqBd1dn0a8QREp+AP9BihmZQCw8wcLVf8ayDWE7SjVF2a5EcvpZwHzKuT9n6d88+Az+qkpQI9/1YLFQrFSaXaUc2o6aMINYfWu7bowCLP8cjdN3mOoyoSMTpNVBpLWdr+W6433MOv+fMZlH6hz+pUAL00sC5hAX2zfL3Pvvez0iZ8EZkhu4g/eDYHEubTprA722OWo5N6+lVeQOuEYA7sKGS1xYFNwBUDErm0b1vAdzVrDaEx/vXSTGbNL36gVXOfjGkd7DMLaY0eBLX1OLzhumtz1bOD0Bt1fPH4X43mCY/zfeE4S0tBp0cfGNBIiWNHBTVTKBSN4pIu7llyD+AbEfLalJtZ+fsmBqVfyLRe/6Eq9xziy339ykOrYzgQvJf8gAxe4yFIgPjSDiQV1w/cdSBsG72zRqNz+4e4kKQbXGQThZEMMsP3YPLPochfMxP5HXqK11ylPNE9gVczMgH48+ERhPob69UNcNH9vZFOB8LlAL1vHlHLjBMUbiGmje/6GKvLHy58F3I2wubvIKz1EZ9bzQvlgvt6HdaUU4N0Ojn4zDNUrt9Au/nz0PnVX9j1d1AhEBQKRaNkl2ezImeF57x9aHveGfUO3y+3MXLf1fjbg7hxzUvceWBEvRAEAFkBvhOhu/21WC7podvZGO2tNzcwnU/7P4rTT/M6qxaAAEdZD6oyr8flCPKp52CRJtb/mbfDk5YY7k+QpWGhT8j/klbft4fnIyF7A1SXQkV9l8YRE2MQX030SauucEDvq2Dsy3DvZjBa6pVrjMTO4cR3CD1ivtzXXqd03nyCRo067iIPakSvUCga4ND+UvIzyyhok+pJ62zrTdSKa9mrj8FV2LA74MKOnzA2/QZc1Zro7zD6iv+Giv4Ux64lv/VX2Et6o9PZCHAFgZDYDVb0VZokTQ+wIoQ3bPstfcfzZeY8HBVtqEq/9eg7tOgp7/GCJ6AkE4oPaMJdK2Kk3+oX4NBi4C4ATAY7Qya6v0B0OvALParb1ng1Hml7QOuuXaDTEfPIw0dVf1NRI/omUl1dzYABA+jZsyddu3blX//611GVV2GKFacyj//5OONmjiOrPIsNuRuY8fJaln69i3sW3geA0WFmxIbr6VqsY+2CAyRXaYuQvun9HKWtvBORqeGb0V9lZUf/RfzY5U1KHKFUZV7tvZHQsd/WhqSALtzYdTILbD24/d4rOavVWQCUuRcs5eklASYDF/WKB+Ccjt2pyr6U6pxL67W9r9hFR0uRb2L2BsjdAdtm+2zyAUBxhibyAFN7wCdjPZf8shcD0MGiBS67ZcJaOvaPpam4qqspnTfPI/BZ995H+o03Ur1rNxWrV2NN3c+Ozins6JxCzlNPk37TzdgOHMCelUXQ6NEIk6nJ9zoa1Ii+iZjNZpYsWUJgYCB2u50zzjiDcePGMWjQoKOq53iEKb766qvx968/qaRQHAvppen8lKrFaR87UxO929D2W7hl9WsciNjKyHbD2efSvG4SnDos1YGUWPIotRRw+y29+fpfK7XKhMRaGUPXfkn8vnEusvAcXI5Qn/tF+ocx95JvWb63gHddxeQWmXjpjNfp9sw8vgmyEuQSSAFGveD1y3rx74u7U1Ztx1HiDXA2644hTHhXm+icaX4WqTNQy7MbPhzuPb6kTliQkjp7xlZ4QwNbdNpK9rNDpjAy5G3wPzpv8IPPPkfJ7NkkR0Vh6dmTsoULAdh/4YX18hZPnw5A5n33YztwgMCRI4/qXkeDGtE3ESGEZ1Rut9ux2+0Nfo6pMMWKUxVbtYPlM/Zgq3YgpWTNwTXsLdrLM1Pf4dztt3k8TQxO31FlckE39q3WRP4vs51Ql452DgMl0Tk8OuBR/EO8+Q0HXie7wMStPW6ls+sxXNVJSJs3IuPkM9sy754zEEKQHKENVrKKqthzqBKkiVKdJMug+VQa9Dr0OkGg2UB4gPced41oT+/EUADu1P8AoE2yNkZRmvc4pH4gstroheaiqRMuDCMfht7XHDZ/bZzlFZTMnq3d8rvvSbuk/tdHQ1h3aPMMuhM4eGuWI/qXV7/MzsKdx7XOzuGdeWTAI4fN43Q66du3L3v37uXOO+9UYYoVpwy2agd6o87j4VGaX8XKH1MZcU1njO7NptcvOMDGXzPwCzbh7JLPjYtvJNo/mglZ2n6oV2x4km/7/Bs/e2CD9wjoEkr6gUMM0ULKcNv4q2mTEumTJyncn6ziKjakF7NmV40d3zuefPxc7yrtqCDN5PPwzM08Nq4zdTHV8lYxG/S8dUVv+iSFkrDiX7CpD98mLGRwwSxvga2zoNuE+qaaqmIw+IHTBsMfgR/v9LksJQxxvEtpYGvfckPvAYOZsl9/xZ5zkKAxYzDGRFO1bRtF335LzIMPkv/hR/j36Y0uKJiKWoO60rlzATDExuI4eJAjIgRhV1915HzHSLMU+pOFXq9n48aNFBcXc/HFF7N161bP6LwGFaZYcTzZV7yPXYW7OLftuVidVnIrc3ltzWv0jemLWW/msk6XIYTgo/v+oE3PSEZem8KulQfZvfoguQfKqOyQhcz0J0JEsT9d27xjxax9lM0tgr7aQqgaQqxRJAUkU1Zpa7Atjh4h5GZ6V8G26hzmOR58cTsiWwWyad1+DhRUsiG9GIAuccFszymlo/1ZXr2kl099FqM3rvuL8+sP3CKDfBchnd8zHjZPh9UfavesW2DGDRDTFYLq2NQLU8Flh6H3gsHtMTNgMlZdO0whkqq1q7B8t4rYdquhP1g7TsYUG0H5H8sp+uorKv7SvIOqt20j/qUXyX7oYWypqZTM0GI3Fta2DBmNJH/6CQeuuRakJPqB+6lYuYqqTZtI/uxTHPn52A4coPCrr6hau85TLO4//8EQFsaJolkK/ZFG3iea0NBQhg8fzi+//FJP6I8GFaZYcSSu+PkKqhxVnJ18NvcuuZfl2csBWJKxBIAuEV1oa9I249i/KZ9l0/ewa6V3BPnBho8Yv+N2svF1cwyyhdExrz97Itf5pL/Z+30mz7+tXjsu/r8+PLdyH1Yd/Gax89BV3T1fCgB9TF/D0jkkx01l1f5qDpVWYzbo+PmeM0grqMTfpCcmuGG3xGiKCBKV7JPeoLiPjO3Mhe6JWAAy1sDBzfBz45FjAfjuarj0c8+prUyPbe02MuZG0baTDfPw8yn1v5SKFSaKp08l4pZbKPhIi65ZVWDCXqUj9bm5+PXsiSP/V+y1dpEr+eEHShcsQFZVNX5/ux3/fv3ouOIvCj75lKAxYwi54ALPZUNUFJaUFILOOQfsdqq2bKFq40ZCzh9/+H79TZql0J8M8vLyMBqNhIaGUlVVxa+//sojj9R/4dSEKb7vvvtwOp1UVFQ0uEEJeMMUjxw5EqPRyO7du0lIaDACtIeaMMWRkZHk5+djMpmYOHEi7dq14/rrrz8eXVWcQlQ5NFGZOGciaaVp9a7P/3kFASu9Yb0riq0+1y32xldZjtx7NSP3ahOYPUa2YvOSTH55dTcPDn2eVLxeLAajDnOcH4t3HiKMUtZbAunVu1bMdZcLfn8ZgM4JmZRUWdibW05CmB+i/BBtFt0PMd2gwxhIrBXuoFILcrvMfA8m4aR19TecnRLNrztyuX14O998H5+tHftHwk0LIawNPFdrBDziSfjt35C/G95zj/Xj+7DvjYOAFRCUrssgoO1msj5Z7ilW8NFHnmNrsZHUedEAVG3a5POs/AcNonLlSh+Rj7rvPvShIegCAtAFBJJ5xx0Y4rSNTfShoUQ/cH+jz14IASYT/n374t+3b6P5jhdK6JtITk4O1113HU6nE5fLxWWXXcb48fXfwlOnTmXy5Ml8/PHH6PV63nvvPQYPrveRCWhhitPS0ujTpw9SSqKiovjhhx8O246aMMVxcXFMmTKFG264AZdLm7xSYYpbLrVFftyOyeyKXk1qxEasW/2pLeWZO33dDAcHNRgVvB7lMWaWWGyMrDaRuty3jsBwC7PWZ+F0OllvuR0Z1xOd3v23X3YIlnmdAJI4CLRmTVoRY+Mq4PVO2oVd82D1B/BwGix7A/YshIxVTLA8gAltAlTg4sNr+uGsLoHidAhNgoJ9vqP4TmMhwv0SeDIXdAZw2rXVrr/9m9IMC0Y/J36Rdjj7GXjD+3VSMH89+XOubbD/cXdcRNkP0yjP9n51GBMSPCP62H89TeFnnxNywfn4de8OBgNC5+vL0nr6dPRhoU163v80RwxTfDJQYYqPH+q5NV+qHFUM+HoAAH4GP6ocVRicJm5e/ephy+UGpLMq6SfO33EnO8wVpFgDKNfZCHSZ+NNiZ43ZwVUuSUyZ18vjtZAqpIDRlUZ62XzHf/EdQ3muPJ+hUdW8fcjthXLTIkgcAPMe8tjMAVK73MnI9UMBmN96GikH5/g27qxHPKN/gKp+d+C3Vtvce8G43zlnYC/4/ALY/ztM+hamXeFb/uYl0Mp3BGw7cIDCr78mqvwVds/SRtTJZ+eRsbI1rvL6G/m0nT8P2/79OIuKqVi+DF1AIHFXDoDvrqLikAlbv2cJu+kOpJSUzplD8azZJH/+2WGf+anA3w1TrFAoTgKF1Zpp47khzzG+7XhKbCWU5Vczb/Xuw5brEfskxUYzduEg0q4tp8/Q60hxgd2iwwl8oRcEBlfh7xJYpOa3DlCsqz/wMxWvJ9lewdUp7aBmE6ld86FVf9g+BxAgBBj9Car2TtYmBhu0XbNBM90c2uoj8gB+9mLP8TkJ2sie/doer8yqtSHdjQshLLn+RCuQfsON2LOzkRNuBbQXy4Ffo4D6Ip/0+eeY27TB3KYNAKET3VuC7tP2og2IsRFwubagUQhByIUXEtKAD3xzQ/nRKxQnAZvTxnub3qPEWsLOwp009GVdE0QsxByCUW8k0i8SvzoxX2qY19k7qp5YUcbU3HwcwkmUS/svvtVSzJk3daFDP80G3aNVCPedn0JIfABZRju36+cQQjmltYQ+JUzzKomoWM/Hplfp6l8rbnp1CRTshfKDcP5UeDwHQhIxu7w2bEvJPu2gxyS49Q+4yCfgrcamWnsU/f4SfH8dRHfFaRfkrYOM1a05sKEPJVvysRVUk/f2O6TfeBPSbseWmUneW29jz9a8iYpnzalffy3avvsEAQMHNHzRVMul1ND0WDbNBTWiVyhOIHmVeby76V3u63MfIWavq+2G3A28u/Fd3t2omS3u6HUHt/e8HYfLwXMrnqPEWsLAuIEYnWaCdCFw4C9w2qn6axvQGaOoIsRvDfmVZ3IwMJX0sG0sbv8lMbm9PPeIjhCU5WkbVr/S/ne6/3wLL97+Fy9MOBddUSq81ZtLrlqAvdJE1Oxp3HJOX3ZZhrH+E83VMdiwBmvAWBLNG4kQZbDtG0BAYDRYy+CAe1IzeagW6MsUgNFVyXDdRobrNqIvz4GeV8DFboHvdQXsng/bvVsH1sa+6VecVh0VecHkro9zp9qAg1T+34M+eQu/+ILcV+tvVBd10+XILbMJeeJL9k24EpzaV0JElzLMXfs0/g9lqjXTYTz+QcVONkroFYoTyLc7v2XG7hnohZ4nBz3pSa8d8hfg3Y3vck7rc5BSMnvvbPQuA+FLe3JT7iusXV+CsHxNgK4AJyagM5Mi76XEEcucyjOJqdDWW1SHruC/MZXgHkh3bS+pccgZ4NDmvMTWmYhRXWGltoF96P65oNdWnYY78uid6KRmi/kk3SESO28lIcu973HmGjj3VVj/BVhLtZdPQLR3ctQUgN/+3/nM5Da9VACWUJ9+2p2hGAFG/QuWTQFrCUR0gII9pM6LxuXQobc4j/hc64p8xK23UjxtGiHX3oEx5hkA2i9ZQvlvv0FJBqFndYeY+uGRPZhrjej1DUfAbM4ooVco/g5Saj+6hq2gG3I3ALCrcJdPeo39vTaLDyxhza5AkHDxlvuJrNQE3OWAVeW+qyb9dCUEmDTvGLMxk++ycojS+2Op8m7g1qF1CStXRCPReyM0VhVD1jpY43Yr9I/Q7O0A2Ruw/PEKZvEFVhlEIFWEWrO9N+1yIQy4Bbb9oI3oD22H5CGafR58R8WehoZ6DitWrCD92UWEtg/BP7qYqrUBlO03k3hdEhZjFi6H9gyd1XrCx3SDVgOwpaVRvmSJT5Wxzz7LQXdQQV1QEOHXXUfUXXcSdfddCINX0owx0YRNqrtNYSMEH96tubmjhF5xeuFyaeaDjuO84ux0gHSBoVaMlx1zoXAfDJis2WyF0FZYbp4O2euh/y1Qmgm/PgNVRRDWmg+G3UL3mD4MSdD2SpVSsqtIE/hDldospnRJ0rbkU+gqRCCQ7gAzycHJLN6/gq1lmwixR3hEHqBLHyfb13sXJ3XxW8BBGUKiLo9rIm+F+E4ElwaAfxRUee3owZYSxsbMwmLPBId7tWt1CeTv9fbTVqEJP2gjduDaqFtJcyXhL6xYqg5RLi2sTbqR4SNu0fKZgzT7fEk69LkWKSXSakXXkNC7R/SOwkLSb7gRgOK9ARS/Od2dwUDe77nE9w/1KRZ+0WiMI7XJWOv+/aSO824yHnb5ZQSeMZTShYsIv/46T8yp2iJ/1Oj0R87TjFFCf5Q4nU769etHQkICc93xLJpCYGAg5eXlZGdnc8899xxzBMspU6YwefJkFb3yWNkxB6ZfB6Of05bEV5fAV5dooj7m37DiXW25fJ57Sf6ip6HNWdB+lHZcw27frY/3lmfx9pYPYAv8cOEPtAttR2F1IWW2MvwMfuRU5DBz9Ru0q57A8m9SKe92iKCAKIZsv5I1HebQVfQje1sBZ1ovoF1Bb5+6Q4KygUTP+cDAb9lKLFaXkfaGbMjNBXNI/RH1viW0EwvABFS6/162zoDoWu62exZqo32/cKjSvjKyLYn8WNGTh9iFsSKbdBlDbs87INzdBnMQFOzRjkMTKZk5k5wnn6L9k2diBOwuHSWV/kQGluPCj+Kvv6Z6+/ZG/0mqM8vZs6dWf8eNxFjzUgFMrbwvvYibte0KjQkJRNxwfaN1HhOXfwUHtxzfOk8RlNAfJVOnTiUlJYXS0tJjKq/CFJ9k3GLGoqfBHKwtxpHuHah/uF373W4kRLSHne4X+f7ftZ+IDlBZ4K2jhos/YOmShzynE+dMYOPExewvSwNgcEUFS8w6ntnxKaPTy2jHMHLzq0gu60frsjb47biXmPXQWFzF3zat44GYe7FKf4od8fjrSwgN7oI880WY63YPtJZqAgxe0a79MrLX2sNgyb/B6K9NOuZs0hYdtR8FW7RR9uYBL2NZMw+sEOvKJV1G0zsxFFd1Nek330x4Zyeetd4hiRR9p4U0rthXjtzjz8GNIeAUODvqKJz2kue2lq5dsXRJoXi679+/o9SK3iKI6VuIfsAV+N/8stccBAijkc47tmsmsiNs4PG3SDlf+2mBKPfKoyAzM5Off/6Zm2++udE8KkzxCaY8F2beDK+2h4zVjeezV9dPy9kEc2stS597n1fka+h1NVwzGyZ9Dee8CF0nQPuzoc2ZcPUMeGQ/XPm9lnbXWjj3NWSbs9hs9pp9nNIFn41n9cHVCCkZV1JIWGUcUeVJUOUOde2IoFWlezPqw4ROAQjT5yOExKKrwGnUXjIdkuLpkFTbriy9LoK1JxYBIjvWqVFCSCuwuL2AEvpCoDekwQWDunHX6O64nJAocqkwhJFEFVkPPkjV2nVkfbWRskwLFYdM2G0WpFULu5Dz7XoOrgsFpybGhbt929F62rfEPPkkUf/3AIkffUjEGdEEJmid90uwENK6isBBfdA1sPmGEAKh0x1xpyZFwzTLEf3BF17AuuP4hik2p3Qm9vHHD5vnvvvu45VXXqGsrKzRPCpM8QnC6dBMLrt/8U4sbpmurc6sweWCkgzY+DX89Rbc+icExcDGb6DfTbBznjdvlwu9bn4pF0Df6+CridDjMm+ewXc03JaO5+BoN5rcMivxA25hzp4f+C3A9wsrtXgPn2z+kP7VVs6pqGRf+vVg9y726ZQ/AD9dAbU1/uOgasYlhhG/3Vf5u4VbyS0NZZT1NWYGvQ72AoQpECx1YijVCLypjtAHJ2gxYABaD4O0P7UNrgv3u9PO8HjeSBcUz11M2ZxZVKyPp9Ol2QxsnUTmTTdi27vPU2XmsnAAxMpbkFVVBAwbRvXGtfiHFFJd6o+9VJt7iHvyAYwde6ILCEAYjQgg8hbNLBNY3Bvnyk0czOhO1PUToWIDdL244Weu+Fs0S6E/GcydO5fo6Gj69u3L0qVLG82nwhQfJ5x2EDrvJNmGL72mlBqKM3zPf39ZW3RTw7pPtTr+ehP8wsBRS0DjemlCH9kJLv9SS3u6qFHvmbr83/RN/Lgxm+3PncPSTM2d8IZ8JxsCHWw3mVgY4A82M5ekDsXl9y3CHlWzr4eHKleEz/mFAxO5bUwH0n7PJmvhfIYEfUqeox3dI23sKQmgDH/atoqD/Ts0M425jtCbagl9+7Nh76/u81q2+/jemtCHJmn2eYAO50DWWlwOQebyMCq+9+6qtGt6PKB5vcQ88QT+/ftRMmsGzt3LKVmZ5gnyFXXvvVjKl8H8hyEyCvtFsyieNYvgS69FVysSqw+hieiNkoSresDI64Hrj/zgFcdEsxT6I428TwTLly9nzpw5zJs3j+rqakpLS7n66qv56quvjrlOFaa4EfJ2wUejAKmFnF0+RROnGtqN0oaeu+fDMyHaLkDnv+m7yhI0zxCH24Sz4UvY/4d2fP08765Dod5JzqaKPMCPG7PR++9lTUZ7dhbu5Iz4s3hzxzh6ur7CFr2VhQH+nJ12Lqmlw/hZSs3FsQG6+c9na+U4AJ6e2A0hBDHnt4W8nyB9D7GmPVDWiejICC6IisfgcJsuAiLrT756RvQBcPVM7dmAZo+vwe3u6BJ+cPYLOFK3kXrBbRjDA7Blx3E4AoYOwdy2LZbHtfUAYRs3kvPU0wSNG4ulaxfEjn0ggMJ9mBITib733sM/xPZnw+LnfL/KFCeEJv1lCyHGCiF2CSH2CiEebeD6Q0KIje6frUIIpxAivCllmwsvvvgimZmZpKWlMW3aNEaOHNmgyNeEKQbNBn+4SduaMMV2ux2A3bt3U1FRPz5HbWrCFAPk5+fjcrmYOHEizz//POvXrz9s2VOe316ED0fAOwPAVga2cvh6olfk9Sa47EttpWVEe2+5DV9q3jQltWKuh7WGijzIdZv4akS+wxhoPdRrk24/+qibWWlzoDMfxD/5f9y17BIyyzPpETocgAJ3VEdHWT/iS7RdkxbbtL1Aa1wpi3XeeQGzyTvOF5W1JnkDorzHZQcJCQnjzSt6a30CCG/rOzHZ/2bviL5mZefYl+CST31Xeva7Ebpfxv6pa9j32HeUlHRBVldjyy4AILSTnbh/P99gv02tW/uc+/XqRduf5hB1xx2a7bzTOM3754z7Gixfj7ie8GiGZkZTnFCOOKIXQuiBd4DRQCawRggxR0rp8ZeSUr4KvOrOfz5wv5SysCllWxoqTPExkr3B1+zS80rNvr7mE2315Kin4YwHvOIW2cG3/KzJ2orG29dofu2rP9TEvfwgJA6CDPfm1Ze7X87tR8EN8yGp4X8b0AR95Gu/88KEbmzJLOXcHjH8Z/09BOgjCGi7kG45w8gJ3kdlcCkvz/Qj0il4pP0A0udN9amno1WzZxtCzTiLbXTqEM6hXcUER1rofdc9OGZsoGTPTsiogs7naYUcteLKW0u8HjUV2t6thLfVfj9TK/7Mn69rv2tWdg5yexFlrtF219ObyHr8OQzhydgOLAMg/513ADDFRxEWt4ewPhG4xoyh4MP3CAnbSXiHcgqinydw1Kh6YXnroTfCI2lH9WVUb55BcUI4YphiIcRg4Bkp5Tnu88cApJQNqooQ4hvgNynlR0dbtgYVpvj4cco/N5dTs70fWKGJs3Qvf3/ikBY/xWGD6mItvkptcjbBB3VirXe/DCa6V3z+8jis1ESMMf+Bhdq+qD7C2AA2hwub00WASc+/f97Bx8v2E2Y0UGRzIIylBHZ4Qcso4baVmqBPbT8XW/4oHio+fIyUMyd15I9pu5n4cF/8goyERLlNKg4r/LcrJPSDK6fB7gXwzWW+hWtixnx6HhxYBo9l1feu+eNVzXXykk+1vVPdVLxzJ+lvL8YTorIOrd59h6B2fvDpWIjuCnf8pU3UvtlLm+P4V1GD5RSnFn83THECUHvWKxOovyu2diN/YCxw1zGUnQxMBkhKOvxO7YoWgpSw+FlY7h4Btx6mTRJmb9REHrTVqnVFHrSwt0Kn2Xlzd2qrNHvXChPgX2v3obieh23Gxl/TqSq3M/iidjw6azOz1mfx6LjOfLxsP2YX3Jxn5C8zbDGVeyZUDS5vPBRb/giMDYyXOpwRh73UTtpmbRTeeXAcnQbFYrLU+W9nMEOPy2HF2/DfbprnEGirSquLteMas8xlX2jhfuuKPMDA2zWh7jTOk1QyZw7Zby1BM57Xx3/AAIJGjoT9bvNYTb0G9xxRPddMRXOkKULf0F9IY58B5wPLpZQ1xsYml5VSfgh8CNqIvgntUjRnClPhTd8VoLQ9C858qOH8ddHpvSPNgn2aq2Xr2iP8Wn96MV3h0s+85o86LJ+hhQQYfFE71q7K4fZKC+nfp0IQhLq0eoZYjQzJa8cG4/kska3wd3h9vW8r8aeorR+UVNNxQAy7V2vhDsZcnYKt2sFH92nzA0bzYZbZdxitCX1JrXGRf4RX6GsEOCBCe04NYQ6Ezuf6JOV/8KHPedCYMZQtXIilRw+cRUXEPP6YdiG2u3a/Ue7Vv0Fx2nH3Ol8WimZJU4Q+k9rrr6EVkN1I3knAt8dYVnE6sb1W7PCbl2hxVmqPyI+GiHYwvM48f/+bNZfKvtdrL4VG/LNdLu+YwuFwcXmFNpINdOp5qNiPUr3vmKN39tlsa1XG0MQEyNMm2oOkIGif5t3T7cwEj9AD9UfvjdH6TBj7MvxSax9iv1pfJXV945uAq7oae3q657zd+fk4r5tM2cKFRN11J4Fn1nox+oXCw6necyFg2P8d9T0VpyZN+StcA3QQQrQBstDE/Mq6mYQQIcBZwNVHW1ZxmmGr1IKB1ZDQp972cH8bv1Dof1O95Iydhaz9OY3QpEC6jU1GV2r3XHvogSW0r+OIFuz0fhmURGUTkhfPt5ecQYC/ie9Xr6lXf1CEH0MvaU9ozFGGqNDpYNBtvkLvH+49jurc5KrsOTm4qqrJuvdepN1O1KRRBJV+gynACd260mnDenR+LS/muqJxjij0UkqHEOIuYAGgBz6RUm4TQtzmvl6zbczFwEIpZcWRyh7vTiiaEXPugfWfe88H3XFi45fUvf2UjQBk7ynmhnV7+He0d7Vqe9vhvUVWhS1gTN4NZG4tpG1Pr/tjz1GJbFqsmVwCQk30Ott3junaF4ag0x9jHzuP1yar25/daBbpcIBejxACV0UFe0do7pwYDIRdeQXhF/VAN+tLT34l8qcfTfqulFLOA+bVSXu/zvlnwGdNKas4jakt8mc/A2fc32jW401dD7NYh6Bkdwm6KDOuPCtpBif9zk4i/5esBssfCNuKJcnFpl8zsFs176Bxt3XH7GfwCH1DsViCwo9xazqXEy77WAtY5p6cdhQVoQ8ORuj1VK5bx6FXXqF602bCr7uOmMceJW3SJE/x8GuuIeaRhyF91bHdX9FiUEHNjoKpU6fSrVs3unbtypQpU46qbGCgZmPNzs7mkksuOeY2TJkyhcrKyiNnPBWpHWis7w3/qMjvz6+gzWO+441ryjXx/CPYyUaTg62tDFx+USfOv7snt/z3TLo/amRL7O/81PUdMs9ehlPnoPu1IYTF+rP9T22qKSTaj9j22grUuHaNh7toMmFtvMcuhybwbq8jabezZ/AQsh96GIC8N9+ietNmAAo//5yqjRuxpWcgTCZCL72EyDvv1Oqp8blXnLYooW8iW7du5aOPPmL16tVs2rSJuXPnsqd2EO0mcjzCFDdboS8+oP2++EM4f8o/eutpq9OJdNYfbduQrMgvY5G/natHaIKY1DWCQlc+d/5+B8vbzCIreDdzK7QQvrHBMXQe7A0VYLIY0Ot1XPXcIM67s8cR25H92ONkPfAA0mZrOMMtS+C8N7Rjl++WerbMTABK582jauNGKletIvTSSwiZOAGdvz+5r7+BtFqJuvce4p5/Hn2gO0RCQOQR26Vo2SihbyI7duxg0KBB+Pv7YzAYPBEq66LCFB+Gmvgy4W0Om+1YWZqxlLQS7R7/mruG93/fwe+787j587Vk7inmhrL6JpQivQQB4/qV0S4pB9BMPM+vbDgMQKRfJOHx3hgzNS6TodH+mP0Pv9eoo6iIktmzKZ03n509elKxqoEwy/7hENVJO5ZOpJQUTZ9O+s23cOAKrx9D2qQr0AUEEPXAA8T/5z8En3celWu0yWFDZB1hV6F9T3uaZVCzP7/fTX5G4+F/j4XIxECGXdb44pBu3brxxBNPUFBQgJ+fH/PmzaNfv/qL0FSY4sNQExY37PgL/e6i3dy95G5SwrvyZJ8pLMz+P9pnnM/Gkp4U6yQjquuIcFI5pAcSHu1Hm7AsllW8xrKFEO0fjUlnIrM8kwf7PYjdZWfqem1BV6ewTvgb/Qn1hm7H5Nf0/0Jlixb5nKdfdx1Jn39OwMABOAoKyLj9DgKHDSNiXE9K9vmTPzcDxyuNb2gd+/RTGMI0F8yg0WdTPF376tDXFXrQYue38O3yFI3TLIX+ZJCSksIjjzzC6NGjCQwMpGfPnhga2KNShSluhPSVXtfB42hKmLMpm3/P3c6owdoWcFuzC7hm+7tM3v9Mo2WqUrL43v89xhbejKNvOqHOTeQXa9dyK3M9+S7ucDF+Bs1DZWzrsbQK0v49giO0L4O2vaLQ6byjZelykf/2O5T9+iuuigr8evYk4tbJFH75JbGPPUbp/PkYk5MIHjeOgve1r7f0667D3KEDfn37UL15s/azrhfVW4NwVGkxjEImTiBg0CCPbV6YzUirlcCRIz33DjzzTAzxcTiyczCE13LLrKFubCDFaUWzFPrDjbxPJDfddBM33aT5Zj/++OMeIT5WTpswxbYK+P5a7/lxMCXkVuayLtXOPd9uBGDW5i2YwkDobLSrHRCsFvpQE85iGxvCl1LlKmN29/+CFrSRh/o9hFlv5uudX7O/ZD/hlnCCTVrArZu7++4optPruPG1MzDXGc2XzJlD/rvves7tWVmUztMmgA1RUVSuWk3E5FuIvu8+ou+7j/wPPiTvv//FumcP1j17sHTtSsgF53PoxZcAPTEXdCT4sc/Qh4ZSVSv2U7tFC0FK9EG+K32TPvyQ/HffxdSu3VE/T0XLRtnoj4LcXG20l56ezqxZs7jiiivq5VFhit3k79UiLWathxfiody9WnTwXYcv1wQcLgejpo/igT+8O0DpTFrUDWEsJtDVcAQNQ1srt78znAPGXfWunZ18Npd3vpy3Rr4FwIQOE+rlqY1foAmd3ve/T+mcORiio0n65GM679hO0icfE3LJRAAK3nsfXC6Cx3lDFISMPw9DrNeP39SmDeHXXec5D7rzDQxhYQghMMR5J4CN0dEYY2rZj9yY27cn4Y03GtyKT3F60yxH9CeLiRMnUlBQgNFo5J133iEsLKxeHhWmGG0E/3ZfCIiGIK+Q8a/i4zaaBzAEpPLouUmM6tSaGxb/lxIbtCppT3RlPHadFaNL+yoqNxUTaAtlaf4cjDszKbGWgJT03yNZ214gdYIYf004k4OTmRX+GPzfR+yu/Jb4114lcNiwI7apetduKlasJPKOOwgYMgSAgCFDCBgyhMoVK7Fnab755o5eE4oxIYEOS38j+4knKJk5C0O05kYZ+/xzVK1bjzHZOzI3uq+FXXWMYSIUpzVHDFN8MlBhio8fJ+W5bf4eZt3im2b0hydyjkv1yzNXcdtizZxyd8//4+qulzHom0Fck/MI/mnayLfQL4fwKu04LWwrrYu68XvbaeyIWQHAv6pG03XKfL4cqeOngTo2X72RQy+9DE4HxTNneTa8Fn5+tP91EYaIiAZaAmWLF2NKSiLjtttxFBXRbv58jDG+0TZdFRVUbdmCPjgYS5f6k6tlixeTeeddxDz+GOHXXlvveg3O0lJt71W9mlRV1OfvhilWKJpO2rL6In/xB9oG1McBh9PFjV8twuS2ZLy16XX0es0NMSAz1hMaNTTQjyzjHgoCsgi0al9edp078FhEN4Zkd6CE+ZxVEsdZltFkP/Swx54OYIiLw5GTg6yqIvOOO4l99hmqNm+meNp3tJ45g8KPP8bcqROZd3pNUfGvvFxP5AF0AQEEDBrUaJ+CRo0i6bPP8O/b57B91werTToUx4YSesXxw14Ny2r58gfFQ0AkH5cNJPaAnvOOvJ4IKSWL0xczNGGox+OlyuYk5elfuHZwMl+sOED/6t303+6k05Mv8OSKp5iyfgoGlwnp8JqF4rbuITLpD17oms6YndooefgWQdYQyTfnfUPOU08BkLQuC9Z9Rt2ZlOgH/w8cDrIfeZSqTZvYf5E3+mXR19+Q+9rrnnO/fn3RWfwI+hteTwGDGtymQaE4LjQroZdSNhhLRNEw/7hZ7j+1Jggv/xraDueXHQU8P03z89+R055QfyM3D6u/JP/rVQf4eftWjEY7651PER+QiNEVwcWdxiKd/hiCt/N92ndEVI3luQVrqPSLxvaTnvv+1LG8q4PoIdokp3/FQSoDYonK20D0jlSe3CVwdo+gCOixXzJmlZN82zuUzZtfrw1hV16JX58++PftgyE2FiEEQWPHknHLZCpXexc3Hfr3vz3HIRddRPxLp8jciELRCM1G6C0WCwUFBURERCixbwJSSgoKCrBYjjGgVlPJ2wXbftD2YK1h4O2QMp6/9uZzm1vkAd7+Tdvgo67QL92Vy1PzFhLQ9k2c5Yno/SBgdzqZkeks/HE9Ny100X+0jq2hOp53auaZjT3uoHpPAMnWsTw0cw5ll0xkDYW0TZtLYe9IovM3AdAjTWLPfJsDSWNI8MvD0LGjZ5/Utj/PpfDzLzSf9OFnETR6NKLO2gid2UzyF59TumAhWffe600PCcFVUkLUvfccx4epUJwYmo3Qt2rViszMTPLy8k52U5oNFovlb/v6N4rTAbvmwbwHNdfJpS94r0W0Y8nOQ9z42Vr8TXoqbb4xWzKLKokTNvZW63G4XNzyvxWYA7Tt9iyGdJ79zEn7HCizQE44JObD9YtcZERJEncvYmfHCKr9tDDBBeFdaLd/DvKNbyBgLIn33sJZl4wg7803yX9Xc3M1OqoYOCaW6PtewllaysHnnseUnIy5XTvinnu2Sd01t/O+nNr+PBdjYiKO3DyMtdweFYpTlWYj9EajkTZtTkyMFMUxsPoDWPB4w9fC2zJt+QF6ymKmPTmJ2RuyeHz2Fp5ubcc6/TsOXPYWZRkH+Cu2Kwva9GLOiq8BWNNB0PqQJMptMA+qhiD3fmRJ+RBVFsbywU9gNVoxSAgNERSTiEzuyP7qVhAA4YO1Cc2Qiy7CnpWN0f2ii7z9NkCb0Ex47dWj7q7RvY9xzBNPYHYvSDK1SjjqehSKk0GzEXrFSURKWPoS9LgMNn8H3SZ6N5PGuwlwjUFtxl6J/08zeWnrHCralNN7yhRqW8SrTFBu1jPw4HaGHPTuQ9N/j++cgjUxAWdhFXlBFpIPZrOnz3icBgsGqZmjOo9ow8ofUlnZ7QGqKrSvhsBoLRy0KSmJ+JdfOm6PQGcykbJzx3GrT6H4J1FC3xLZ/weEJP79KJHbZkPmWk3gf39J+wFIXeqNRAmcG9yK+5bacEUmsiSuLznfLeaJrdqesHm14vbrunTknQlD+c2p7XbUa5eFG36rIq7I97aGqCjazJ6FLjiYma+tIze9Cjnpd/xt42B7oSdf6+6RbFiYTlWFFh5iwoN9MBiVj7lCURcVAqGlsO83+PJiqCqGz8+HT8fBjp8ge+PR1VNZCJ+MhQMrYPr1sOJt+OBMnyxlqzaSMd+GlFBRYuDfn7lI2m2g9V85OPJXc/VuzR/dmqh54aRFw1UP6Ul79VYik73L8zd2snPvbQb+dZUmzjtaCUpfmU3szLkYIiPRmUyUFGjhITr0v5PCnAoiErwhggNCzVz2RH/6nduaLsPiiWmj/MwVioZQI/qWwpcXab9Xf6j9LsuB79z7tD9TArk7weQPob77meJyauFrC/bBFxdCu5GQvgI+HdvorTL/0FaJbpoVhNnuwCLgmSt1PDTLxeTftfAEr0zUEdG7Lckzchl81f/hLHqTvWX7cegKCbOEUVTtHcbvSBI8fq2eopB4Lp2XzYZFB7ntreFk7S7C6h6tZ+0uorzISseBsRRkafGAzP4GLAFGBl6gdlBSKA6HGtG3BMpreSL9+Xr963sWwbsD4YOzYPlU+OVxbXHTgb/g5dZayILlU6Akw3dPV4AJ/wNAuiB7ZSjltu4InWZLN9s1EdZJ2J6sY8qF3j+ntR11LKhYw4yLo+h93vUkBSXx5fYvmb13NkXVRTzY70FMOm10nxKewt4EQTjaF4DT7qI0v4of3tjgqS99mxZmMioxiFHXp3D184OUm61C0UTUiL45Y6+C0mw45J3QxFFdP9/X7j1qqwph0dPacfEBzaxjLYU5d/uWazcK9i0GwNXqDDKWRBCSXEVJmj8laQV4p1011rfTzje1EcwYKtid4L1+Zqsz0ev0TOgwgTfWaVvkjU4ezaWJVxAvEnlgzb1c0+UaxrUZx941uSzeshOAL59c4XOP0nytfcGRFqKTlYlGoTga1Ii+OfPLY/BWH1j1AfiFQTe3oJ/7Wv28PSZ5j/3CYedcKM2EEU9CTfx2cwh0Hg8Xv+/JWvbnaipzzeSsCa1X5f236Llvst47kheC78/Us7Gd98+qb0xfAM5re552C7s/Y/fdxKcPL2P3m/DJwK8Y33Y8+9bks35+OgDBUX6Ndjk4svFrCoWiYZTQN2dytNWfHFgGfa6Fif+DB/dCJ2/Mc0KToc91cOE7kNBPE/Le7lC3nc6Fsx6Cu9dB+7Ph1t9h0tcQGI3zql/ITB9H9iOP+tzywZu0idPPzoiiMqYD2RGCanN9E8r4tuMBGBA7ANC26AOIK2vHnjWHPPlWTylg1Y+p/PrpdooOVoKAq54dxNk3dKFVZy0YmV+wdwLX7K8+QhWKo6VJ/2uEEGOBqYAe+J+Usp6DshBiODAFMAL5Usqz3OlpQBngBByNhdFUHAP+tbaM63eTFus9MArc8ekBuHs9CB3odHDjAu14sXs1aExXAGR4W6p7PYclNBkBSJuN7Nc+p/wv7UUihUBISWYEpEcLLr7qSvSxC+gdnERR0d4Gm/bUoKd4dsizmPRekZ59wWxy1laxeZfv6uZ1vxzwHOv0Ap1O0GlgLJ0GxpK5s5DgSD++fHIF7fpEKbu8QnEMHFHohRB64B1gNJAJrBFCzJFSbq+VJxR4FxgrpUwXQtSN1TpCSpl//Jp9GrP/D4jrCZYQKM3R7Onnvgphyd48Oh10vwxa9Qd9rX/imuN+N2o2+L7XA1AycyY5Tz5F4MiRVG3ejGjfEcfKv1jYfRRzOpfQ3eTPrV//wYK+2gdg/+5VbCop5YzkFG7qdREvr3mZrPIsn2b6G/3rNT3KEc+CmSvdTdHhdLjq5XE5fBdNteqsvcwmPTWA0Jj6dSoUiiPTlBH9AGCvlDIVQAgxDbgQ2F4rz5XALCllOoCUMrdeLYq/T/4ezUe+19Vw0TtQlg3JgyGigT1CJ37UeD1hyXDbMs9pyY/a4qbyJUvc98lnWVx3pnbrQkDr91gMrLxPT4U7PlqWbSUSSa/oXgyKG8Sg+EEM+Foz0XSL6ObZY1VKSUWxjcAwbaen2a95tzocfHE7lk3fQ79zW9PnnGRKC6qY9txqRlzducEmRyQENuUJKRSKBmiK0CcAGbXOM4G6wbM7AkYhxFIgCJgqpfzCfU0CC4UQEvhASvlhQzcRQkwGJgMkJSU1lEWxQxNkNn4FuduhqgiC4/92tbasTM+xuXNnPnclMK3jSHSGnZ70Cj9BUlASeVV55FXlYdAZ6BGpBZi36L0RMp8a/BRdIrRdlLYszeLP73bTrncUI69NoaLE5smXMjSOkrwqeo1OwmjWExEfyG3vDEevV9NGCsXxpin/qxoyitYNdG4A+gLnAecATwkhOrqvDZVS9gHGAXcKIc6kAaSUH0op+0kp+0VFRTWt9acTDhsU7veeZ7tHx6HJDedvgK9XHWD+Fm07v725ZezNLeePl97BkZ3DHwk92RmWSNSLL/F5lzFYjeDX6ltP2Ytb38jPE36mR5Qm7mOSx3jMM7Xt5iHmEJx2F2vn7efP73YDsG9DHv974A+ftpgsBs6c1BGzn3esoUReoTgxNGVEnwkk1jpvBWQ3kCdfSlkBVAgh/gB6ArullNmgmXOEELPRTEF/oGg6nj1YG3jnhjUtno3TJXlithYbft/jw7jwpYXEVhby9tK3AVgX1ZGFrQcS9F0afolfYAjc5VN+QsoIAG1jbeDKlCt9rt/d+27e2vAWhZuc/PDl0nr3r9kDpeOAGBI61d9UXaFQnDiaIvRrgA5CiDZAFjAJzSZfmx+Bt4UQBsCEZtr5rxAiANBJKcvcx2OA545b608Xtv/oPmhgx6iwpo3ot2WXeI4z77mXGStXMrfNYACWxXdnRXw3TBG/4Qr7C4OxzJNX2iIQpgK6RXYD4JnBz/DnltWwOZzpa9bSc1QrOvaPZXKPydzY9Ua+/ddq6tJxYAxGs4GEjqF06BdT77pCoTixHFHopZQOIcRdwAI098pPpJTbhBC3ua+/L6XcIYT4BdgMuNBcMLcKIdoCs92f9gbgGynlLyeqMy2W8lpz237h2gpXgJTzwT/isEUrbQ5mrs/ivd/20q44E4dOT+VKzfNl/P4V5AcJXhx2Di5rAEHRC+qV/67PF4S0NWDQaX8qXSO7snZhMcuK9gCw6OPtCAQH95cQHhdAaX41o65LISIhkOUz9jDutu6Y/Y3H4SEoFIpjpUl+9FLKecC8Omnv1zl/FXi1TloqmglH8XcoreW62OZM2P4DjH0ZBt3WYPatWSW8sWg3b17Rm4//3M9/f9Vs5fOXTvHkORBppMpsZ1VngT5gN0ivWciaN5KHzpxAl0B/lr65l/b9oml1s3fS12713TFq4cfeEAzh8QG07xeNwajnogf6/I1OKxSK44VaZngqk7kWYrppkShrCI7XolEehk+Xp7FkZy7frDrAT5u16ZQ2YV7PmEqDmZcuc1AQov3z60sPYAzeTJAxhMcGPsI5yWMxGYzkpZexkTVk7S5GSknO3hKikoOwBBqxVjoavPfAC9qqmPAKxSmGEvpTlZxN8L9R2oIoWWthkSX0iEWl25b/8+Yc8sutXDs4mTu2/EgR8HnKWOa17QMhrwEu7CW9MIZsBOCxgS9wfrvzPfVUV2ix4KvLbGxanMHyGXvpPCjWkx4UYaGsoBq9UYfTrrUxvsOR26dQKP5ZlNCfqtTEsdm3GIwBYDBrtvmQI2/2faCgEoBNmdrIP9TPSNFXXwGwOx5kj1cAuK/PfTgrOvLennvoH9PfE3ishhpBlxKydhcDsHPlQQB6j0kiZUgc3zyzCr9AI2ff0AVLgBFLgLLHKxSnGkroT0XSlkP6Ku95fC/NZLNlOnS54LBFpZSkF1bSKSYIw/bN3L/+O/L6/Fu7aDSS2umgJ2+0fzTju53B6JAFJHWMoKrEjt3qZPuybMqKqolvH+pt0uZ8QqL9KMmtAiBlSBxhsQEMmdCeVp3DiEoKOl69VygUxxkl9KcaacvgM9+RNSGJMP6/MPp5MDcuqFJKdnbtxgXtz8J1y530/G4OCRX5JDytTdraXnqYlL1/slUCQhP6zF1FzJu6lTE3d2Xh/7b51BceF+Bz3r5vNEHhFpK6RhAUrtn8e49Rq5gVilMdJfSnGqm/e48H36Xt2RrZQdsG0NRwUC97Tg6HFvxJSL/u4HJxye7feD4nn7OrvNv1VY7qzzvr1nNG9uXY2znwswdh2RvDij/3AfDn93vq1bv6p/0YTDocNs3+HpEQqPzgFYpmiBL6U410985KncfDmH9Dtwma500DSKcTV1U1ey++lCW9niPqp3l0BzITzuKuj7MIs5ZzICgG8wO3cI/tJUbs0eLQj9in/V72daqnrqpSW0O3IDo5mOw9xQDEtFY7OykUzREl9KcSUsLBzdD3Bjh/ipaW0LfBrLnvfUD+tJmYLIJCocUGKgzrjEvo2d3hMgB6b5zCt70uIXFHOd38zyTZfPhNtA1GHQ67i+AoP8oLqvELMnLhfb347cud7Fx5kKAIy2HLKxSKUxMhZQPL6k8y/fr1k2vXrj3ZzfhnObQdpl0BRWlw3hvQ/6YGs9mtTorS8vj16VkUhXVmwJp/k5kwnOz4M47qdmdd0ZHivCo2/ZrBhff3Zveqg8R3DGXxZzvoOSqRfuNaozMITBYDLpdEOiV6owo6plCcqggh1jW2sZMa0Z9EpJRICTqdgJ//TxN5gKRBjZaZ9do68jPKIUyL2766/5NNupcpRGArkYy+sQsJHcMICNVixJ9xSQcAWnUKozS/CiEgoVMYlkCvm6ROJ0CndnZSKJorSuhPArZqBy6HZO47mzi0v5To5CDalrSmr+UvAArsSZRvKyC5qxbHZt+GXA7uK6HvuNaayB+BNGMhAZfupr2lLaF729IqMZqOPeOpLrcTHh/QaLngSD+ufWGI5yWgUChaBkro/2GklHx0/x8+gShzD5SRy6V0PC+ZoP7nsfDD7RRmVzDx4b7Etg3hlw+08MKJKd49YuOtu8gN7ozDqlW0L3wj7Qp7AbClVSDzhz2mxYkf4L2Pf61NthsjMEzZ4RWKloYS+n+YvetyfUS+1+gkzOV7WbXCxBc/D+CKvm2pLNE2FZn5yjr6X5XgybvtD20nqAFr/kO5MZunbzegdxoJt8VToKtgfauF2CvaMrHnvWoTbYVC4UHNrv3DrF9wwHNsCTAydGJ72kWledK+fW6VJ/QAwJqvszD7G/APMpC6qUArV13g2ajbqbeT53cAYe9Ez6gnyCo8h/N6/P3tBRUKRctBCf0JZPraDNYd0GLHWyvtrP4plfyMcuI7hBIQambwxdqm3mHOHUyKf9SnbOsxIJxauIEY00Habfrcc23yXQ5+7u8dsQeQRJj1XF64YDDf3DSUTrEqHIFCofCiTDcniKziKh6asRmAtJfOY9/6PNb8nAZAQsdQLv4/b6x2Z2EamQ4Hr4ZW8VCxHwArtrzCeZsN7Gk3gah10wmsyEImDWR+tyWU+/uaZQ7uvIU+iREEmA0MaR/5z3RQoVA0G5TQH2ekS1JRYuWDOTu5qMLETqOT//24k852b4x2/xBfr5aS/BxyZSgAOZ38qazIJG5XOqElkv7rX2V9O8EH43QUBb4JdWzv5XsfBGkk2E9FjVQoFA2jhP44Ybc5KT5YybYV2Wz7LQupd9HBqaeDXY91fja7Qr0eL2Z/72PPKq7CUHqIfKnZ1b86lId/mw94bCWkR8LXI3RsTxJYTYLhrYZza89bCaQ1oz9+CVPkr0i75oIZbFFCr1AoGkYJ/XFi/S8HWDsvzXMe6/ROf7iQVBd7Y8kYzXpcLsninbkUp67lUlFMqtFIUIdHseaPRG85SGiFpDBIsKG9t54RSSPoFtmNeVtysBcN5vWxdzC1eDf78ioI9lP/lAqFomGUOhwDZYXVBISY0Om9Irx5U269fP6hJv6Sa7A5ouheGculd6VgrdBjTgyg7ePziKWAlZa7AdgWYAXAHLkEgNAKyKhjbu8R2ZOMwkpmrc/CoBOM6RLD//7UApOFKNONQqFoBCX0jVBZasNudRAS5RsaePVPqZ5J1S7D4mkVXYIIiqHqUBV1d0q9/PEB3PCGFlveFahDN92FvOI7HvoshwX2V/jV0Rfc65Oseq9LZWi5JLIUigO9dbkcQdz9RSa7Du7EJWFct1gsRj1Ol+aUHxGgVrMqFIqGUULfCN88sxJrpYOLH+xDeFwAf83aS5sekR6RB9j+ZzbbAUhFCEhNMMAhKxuiBL88MhyTxft4dUKL6S6+vZz/k+FULdczsngdGUP8Kf8rlNAJ6ZgDNVv8c186AXC43xz20u5YD41nh6OcyEAT1w1uzUW9tYVUVoc7VnzgkVe9KhSK0xMl9A1gq3JgrXQAMPu19Z70HctzfPJNDH8Uo6ik2JnA3eJmBvbqwpgusfxfhL8m8rbKBusPyKykuFgLZ1D+VygAD85ykR8MB0MFscVavm1JmodNddZVnrJ9ksK4e1QHz7nVob0U1IheoVA0hhL6OmRsL2TOmxsPm2dY0EfsrBpBjHEXQkCEMQM/6zkM7zScwe0ivBmrCqkqNJK1LIzk0fn813AJV/qvpmy5NjFrDXBhrvDa+SNLIbJUM8X83E+wPVlH+b7/w2LU8cjYzuSXW5nQx3dz8Gq7GtErFIrD06SVsUKIsUKIXUKIvUKIRxvJM1wIsVEIsU0I8fvRlD2VqCvyBpP3EU2KeZQxIa/RI2Ael0U+hBCwyKktfBocVsIQt8hLKXGWlUFlIVvTg7BXGti9P5ilMWlsLtF2adrfOo6f27cHYFVHwZUP6Vnc0+sjX27rhK1wEEOTUph+6xBuGNqGh87pTLuoWoZ7wGqvGdEroVcoFA1zRKEXQuiBd4BxQBfgCiFElzp5QoF3gQuklF2BS5ta9lSidoyZGkZPDOKaqMlcHP44EeYsOiTmAbBVaOaTe+134ZA6ruvkwuj2win66mt29x/ATx9fyHb3SHt3gT8ZIQdZVaV9RP3YLZCqsDQASgLAYRB8cK6e6x7QsyixH/NiJxBadTlf3TyQ7q1CGm3zPW4zTpgSeoVC0QhNMd0MAPZKKVMBhBDTgAvBPQ+pcSUwS0qZDiClzD2KsqcMXz29wue854g42v45AvRVBOvzwD8ZblrImOe+I1tGkCDyqcRChSmCSJmPlJKq9evJ+OAtTMC6A/5cvlYzrXTOggdmOTFbCyjzE6zqkkp4mWD0Bvi5v/aCuLzT5Xy1dgtv9J3EsA6RfHnTwCO2+eZhbbl52OG3CFQoFKc3TRH6BCCj1nkmUFeBOgJGIcRSIAiYKqX8oollARBCTAYmAyQlJTWl7ccda4XD53xA202wo8qb4B8BfmHslokA7JJJ/Ov8LgRvT0aU5pD7848UPPgYNWPry/90+dQ3aJcE7MweLHDqBXmhcNu1nTEE7gXgyUFPkswBMttV8ti4lBPUS4VCcbrRFKFvKLB53Y1mDUBfYBTgB6wQQqxsYlktUcoPgQ9B2zO2Ce06rricvqJ8QafvMP3+G4Qmw+1/waKnYdDtAASZDZRZtZfCDUPbQEYszp1z+WLhbs4DNrYRdMmQmGq9Nx69Xs/ZG10UBgpmDfU+Fpc9zOe+1wxKPjEdVCgUpy1NmYzNBBJrnbcCshvI84uUskJKmQ/8AfRsYtlTguy9JZ7j0d2WkVgyDcoPQftRYA6E8W9AZAdcLkmFTVPwRfefCcDG8nRWWSzEFUFaNLwwSc/0M7yP9qlr9KTGCT4cp+frlJFYK7p6rt09bOg/1EOFQnG60hShXwN0EEK0EUKYgEnAnDp5fgSGCSEMQgh/NPPMjiaWPSVYNz8NnV4wIfxROuS97r3QcZxPvreW7MUl4fVLe7Ig+3O6f96da4zF3BoXTWyRJCdcG61vbOsdte9q5T225Y2lOvNa1l+9nicHPsmtva89sR1TKBSnPUc03UgpHUKIu4AFgB74REq5TQhxm/v6+1LKHUKIX4DNgAv4n5RyK0BDZU9QX44ZKSX5GeV0GhhDXOou74XHc8DkGwLhx01Z9E0OY1yPcAZ+84En/bzVLuILYbnbp+hANBwMhT+7Nryln1Fv5PLOlwPw2lmvUe2oPq59UigUihqatGBKSjkPmFcn7f06568Crzal7ClFYSpb/iqhusJOhHOrNz2iA06DHy/9vJ3ze8ajEwKLUUdqXgWPnBvC+E/OpEOBZE+CIKJEct1izca/qpP7I0kI9DM+4BK/SKbPvdxT7YrHRnrcMGs4p/U5J7ybCoXi9EWtjH2zNysPfQ34E7f3RUjuDVfPAmBzZjEf/bmfj/7c71Mkdsa/eWdhRb2q1p/bjvSoNKafP51Iv0gi/bTwk5+c8wk3LrgRgLgQvxPbH4VCoajDaS/0Vpc/dulP/8BviTamwshZ4K/FoTEbSuvlv89vKZ1/qL8MwNyhAxe+9A0dC3fQObyzz7X+sf3pGNaxXrpCoVD8E5z2Qp9r11aWxhl3agmthwHw6MzNTN/+K8awfOxFgzFIG2NdeZzz7dwG62n9/XfoTH70j+3f4PWZF8w8/o1XKBSKJnDaC/0ht9BHG/fC0HvBoC13mrYmg6CUTzHZJWG2zbw8Yx/+Vm+5XZ0C+KVTFffOceHfvz86P2WSUSgUpyZK6O0dCdVnYtZXw6hnPOlCr9ngH5nuovuBfT5l0qKh22fTaOOoIOrScsI6dvsnm6xQKBRHRZOiV7ZUbFUOMqw9SDRvBL8w0OmwOt1b+sXOZvhmF90PaIt0rQZ46mptJ5BIEUT7sPb0jOpJZP+h6EMaDzqmUCgUJ5vTekS/9fdMnJjpaPkDjAHsK97HRT9exJThUwi3HeK2eS62JglWdhZsaCewuZ9WZJ9BJ7fhCoVCcRSc1iP63asPEm/cSqxpD5gC2FWoLZZ6buVzJBWXoZPwbfc+zOvYBb+QKygJFLx+cyTJ/3npJLdcoVAoms5pK/TSJSnOrdImYQECIimsLgSgqKqAC9dqNvoM+1hiqu7g4WGXAmDp3ROdv3+DdSoUCsWpyGljuqkut6MzCM+G3WWF1TgdklB/LcbaHos/L695GYDhmyW99mu2+QK/EBbcMZTwABNTjVMbdZ9UKBSKU5XTZkT/8YN/8u2zqzznBVnlAIQZMgHYY/K+8yYdauM5dgkdZoP2mEYmjSTIFPRPNFehUCiOG6fNiB6gvEjzqNn4azrLZ2gmm2jjPlzA1wZtw24/aSSmBDZGtOHf/a8B8Ai9QqFQNEdOSwXbslQbxYfqszAIG3+c+xybi7SJ2G9/TMa2dx/pgVEUWbSNvA360/IxKRSKFsJpMaJ32J2eYyklFcU22iQUcobtWZxAtrMSAJNdYtuhhUIoNQWcjKYqFArFcee0EHpblVfo07YU4HS4aBVbTn5uEUNbJRG4dxbjV7m4Zpke0PIG2qsaqU2hUCiaF6eFTcJaafccz3t3MwCB+74h1WgEIOFAFdcucSFs3ny/J/T6R9uoUCgUJ4rTbkRfQ4g+h1Kd9p4LrdBcKWWbdrwSPpCliX0ASAj1o22UMuEoFIrmzWkh9NYqu895n5RsIorSKdFrrpIXRYwAFlP07OssnbXXk++T6/vTKVa5UyoUiubNaWK6cficR0Vp5yU6HQYpGWDuBEC+3htqePptg5XIKxSKFsFpIfT5GWU+561iKyjW6fg4NASDwYIzN5cSkz/3Tt/iydO/dfg/3UyFQqE4IZwWppvsPcVEJATSY2QrUobEYf3tLy5oFQdAxKFqiqfPoHag4em3DT45DVUoFIoTQIsf0UspKciuIL5DKF2GxiOEYHl5KkV6LbZ8Yp42EbsnJMFTJjrIfFLaqlAoFCeCFj2iz8soQ7ok9monIdFe+3uBTTPljG09lkllQcC3PDXkFs91i1H/TzdVoVAoThhNGtELIcYKIXYJIfYKIR5t4PpwIUSJEGKj++fpWtfShBBb3Olrj2fjj8T3/1nD9Be1W4ZGe0MLF9i1EMQvDHuBRGsgGI2UmrzXLQYl9AqFouVwxBG9EEIPvAOMBjKBNUKIOVLK7XWy/imlHN9INSOklPl/r6lHh5TS5zwstpbQOysJkWDUGXHk5mKIjEQK7zvPbGzxFi2FQnEa0RRFGwDslVKmSiltwDTgwhPbrL+Pw+7yHLfvG01wpB/SpkWoLHBWE4E2anfk5WKIivIpq6JVKhSKlkRTFC0ByKh1nulOq8tgIcQmIcR8IUTXWukSWCiEWCeEmNzYTYQQk4UQa4UQa/Py8prU+MNhq+U7H5K9gcy772Znj55Y9+yhUNqJEFr4A1tWFsb4+Lpt+dv3VygUilOFpgh9Q6on65yvB5KllD2Bt4Afal0bKqXsA4wD7hRCnNnQTaSUH0op+0kp+0XVGWEfC9W14ttYf51P2aJfAShbupQCaSdCZ0E6ndizcxB1hF6hUChaEk0R+kwgsdZ5KyC7dgYpZamUstx9PA8wCiEi3efZ7t+5wGw0U9AJp/ZqWIN78hXAduAABbgIN4fgOHQI7HZkrBJ6hULRcmmK0K8BOggh2gghTMAkYE7tDEKIWOG2dwghBrjrLRBCBAghgtzpAcAYYOvx7EBj1BZ6nfQGNbNlHKBcJ4jwi8KelQVAjn/EP9EkhUKhOCkc0etGSukQQtwFLAD0wCdSym1CiNvc198HLgFuF0I4gCpgkpRSCiFigNnud4AB+EZK+csJ6osPtlqmm4AK7weIraQAgIigOBwF2vFDizMhJO6faJZCoVD84zRpwZTbHDOvTtr7tY7fBt5uoFwq0PNvtvGYsLpDE9/46hmk9fduIuIoKgIgJtdM1tP3A1Bi9oYiDjS36DVkCoXiNKTF+hHarZrpxmD0nUuWxWX03Oci/Iu/PGmlpgBahWkrZ1+79KS8lxQKheKE0WKHr7ZqJzqdQCd9QxTrrE6e+B4g3ZMWGGBh2SMjkVIq10qFQtHiaLkj+monRose7PYj5o0LsQDKf16hULRMWrDQOzBa9MgmCH376MB/oEUKhUJxcmixphu71YnJYqCgJMcnffLdOp440Itu9mjKt+3gqfizuaCtcq9UKBQtlxY7ordZnRjNet5c9ZpPenGgjuIbz6fVf/9LzptfsCY2hS5xwSeplQqFQnHiabkj+moHRrMeR3UlANsT4aeB2nstwi+CZXvyWbTjEABh/saT1k6FQqE40bRYobdVO/EPNmO0a/70Pw/Qsa6DW+gtEUx4b5Unb4ifEnqFQtFyabGmmxqvG4Ndc6+019pLJC7AdxWsEnqFQtGSablCb3ViMusx2OoLfWxArOc40GzAoG+xj0GhUChartDbrJp7pd6hmW4ces1HXi/0Pv7yJrXJiEKhaOG0SBu90+HC5ZAYzQYM9hqhhys6X0GnsE44Xd5w+naHq7FqFAqFokXQIoV+6+9a+GGjRY/BvX2g3QCPD3wcgNJq7yKqtmqxlEKhaOG0OKGXUrJs+h4ATBY9Opsm6o5aNvoKd8Cz83vG8+R5Kf94GxUKheKfpMUZqKvKvKN1o9lASG4VLqDc4s1TI/Sju8QQE2xBoVAoWjItTujLi6o9x0aLng7brGxuIygN8E7Alls1u32gWV+vvEKhULQ0WqDQWz3HJrOewHJJRp29xmtG9IFm5T+vUChaPi1a6Lf98AEmO5T5CeZePNeTvudQGQAJ7s1GFAqFoiXT4oS+usJrow/7+mctzV9HcnCyJ311WiEJoX4khCqhVygULZ8WJ/S2Su+OUka7FtDshqRBPnlS8yroEq8iVioUitODFif01ko7gWFmzs55D6OjAoDWse188pRU2QlV8W0UCsVpQosT+upKB2Z/I0Zp86TpI2N88pRW2QlWQq9QKE4TWpTQF8+cSfEfKzCZBUhvmANDgtc+73C6qLA5CbYooVcoFKcHLUroc2bMpzi0A/rSfKT0xrDRR8Z7jsuqNRt+sF+LWxSsUCgUDdIktRNCjAWmAnrgf1LKl+pcHw78COx3J82SUj7XlLLHk41mLZxBfmY5tr37POn3LyzAplsPQHGlZtJRI3qFQnG6cEShF0LogXeA0UAmsEYIMUdKub1O1j+llOOPsexxoVKn+dBH527g926CVqUutrQ7i82H7IAdu1OSXqh54qjNRhQKxelCU0b0A4C9UspUACHENOBCoCli/XfKHjV2YyiW6jzGz/0P1RX3YD60jfHtR/GY+3q13Unnp34BUJOxCoXitKEpNvoEIKPWeaY7rS6DhRCbhBDzhRBdj7IsQojJQoi1Qoi1eXl5TWiWL3aXHYfLgsVagNAJ/IJi0bUf5ZPHYtQzsU8rWoX50SYy4KjvoVAoFM2RpozoRQNpss75eiBZSlkuhDgX+AHo0MSyWqKUHwIfAvTr16/BPIdDL/RYDNH4yV2Hzff6ZT2PtmqFQqFo1jRlRJ8JJNY6bwVk184gpSyVUpa7j+cBRiFEZFPKHi8EArs0Y5FVJ6J6hUKhaLY0ZUS/BugghGgDZAGTgCtrZxBCxAKHpJRSCDEA7QVSABQfqezxQgjBeONcqioOnIjqFQqFotlyRKGXUjqEEHcBC9BcJD+RUm4TQtzmvv4+cAlwuxDCAVQBk6SUEmiw7AnqC9htGEwtammAQqFQ/G2a5EfvNsfMq5P2fq3jt4G3m1r2RCFtNnRG0z9xK4VCoWg2tKjhr7TZECYl9AqFQlGbFiX0LrsSeoVCoahLixJ6abMroVcoFIo6tDChVyN6hUKhqIsSeoVCoWjhtCyht1oRJhXDRqFQKGrTsoRejegVCoWiHi1O6HUm88luhkKhUJxStCihDxw1CkuXlJPdDIVCoTilaFH76SW8+srJboJCoVCccrSoEb1CoVAo6qOEXqFQKFo4SugVCoWihaOEXqFQKFo4SugVCoWihaOEXqFQKFo4SugVCoWihaOEXqFQKFo4Qtva9dRCCJEHHOsu35FA/nFszsmkpfSlpfQDVF9OVVRfIFlKGdXQhVNS6P8OQoi1Usp+J7sdx4OW0peW0g9QfTlVUX05PMp0o1AoFC0cJfQKhULRwmmJQv/hyW7AcaSl9KWl9ANUX05VVF8OQ4uz0SsUCoXCl5Y4olcoFApFLZTQKxQKRQunxQi9EGKsEGKXEGKvEOLRk92eIyGE+EQIkSuE2ForLVwIsUgIscf9O6zWtcfcfdslhDjn5LS6YYQQiUKI34QQO4QQ24QQ97rTm1V/hBAWIcRqIcQmdz+edac3q37URgihF0JsEELMdZ83y74IIdKEEFuEEBuFEGvdac21L6FCiBlCiJ3u/zODT3hfpJTN/gfQA/uAtoAJ2AR0OdntOkKbzwT6AFtrpb0CPOo+fhR42X3cxd0nM9DG3Vf9ye5DrXbHAX3cx0HAbnebm1V/AAEEuo+NwCpgUHPrR50+PQB8A8xt5n9jaUBknbTm2pfPgZvdxyYg9ET3paWM6AcAe6WUqVJKGzANuPAkt+mwSCn/AArrJF+I9keA+/dFtdKnSSmtUsr9wF60Pp8SSClzpJTr3cdlwA4ggWbWH6lR7j41un8kzawfNQghWgHnAf+rldws+9IIza4vQohgtEHexwBSSpuUspgT3JeWIvQJQEat80x3WnMjRkqZA5p4AtHu9GbTPyFEa6A32mi42fXHberYCOQCi6SUzbIfbqYADwOuWmnNtS8SWCiEWCeEmOxOa459aQvkAZ+6TWr/E0IEcIL70lKEXjSQ1pL8RptF/4QQgcBM4D4pZenhsjaQdkr0R0rplFL2AloBA4QQ3Q6T/ZTthxBiPJArpVzX1CINpJ0SfXEzVErZBxgH3CmEOPMweU/lvhjQTLbvSSl7AxVopprGOC59aSlCnwkk1jpvBWSfpLb8HQ4JIeIA3L9z3emnfP+EEEY0kf9aSjnLndxs++P+nF4KjKV59mMocIEQIg3NlDlSCPEVzbMvSCmz3b9zgdlo5ovm2JdMINP9pQgwA034T2hfWorQrwE6CCHaCCFMwCRgzklu07EwB7jOfXwd8GOt9ElCCLMQog3QAVh9EtrXIEIIgWZz3CGlfKPWpWbVHyFElBAi1H3sB5wN7KSZ9QNASvmYlLKVlLI12v+HJVLKq2mGfRFCBAghgmqOgTHAVpphX6SUB4EMIUQnd9IoYDsnui8newb6OM5kn4vm7bEPeOJkt6cJ7f0WyAHsaG/tm4AIYDGwx/07vFb+J9x92wWMO9ntr9OXM9A+JzcDG90/5za3/gA9gA3ufmwFnnanN6t+NNCv4Xi9bppdX9Ds2pvcP9tq/n83x76429YLWOv+O/sBCDvRfVEhEBQKhaKF01JMNwqFQqFoBCX0CoVC0cJRQq9QKBQtHCX0CoVC0cJRQq9QKBQtHCX0CoVC0cJRQq9QKBQtnP8Hsl1w74/5BcEAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "plt.plot(np.arange(600), history1['train-acc'], label='1 clients')\n",
    "plt.plot(np.arange(600), history2['train-acc'], label='2 clients')\n",
    "plt.plot(np.arange(600), history3['train-acc'], label='3 clients')\n",
    "plt.plot(np.arange(600), history4['train-acc'], label='4 clients')\n",
    "plt.plot(np.arange(600), history5['train-acc'], label='9 clients')\n",
    "plt.legend()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x2778176b670>"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAACN/0lEQVR4nO2dd3gc1dWH3zvb1WU1W5Z778Y2NjYYbMAY03vovSVAEgh8IaGH0ENLIBAIJJTQQw8d22DA4N67LRe5qXdtm7nfHzO7O7s7K8sN2/K8z6NHuzN3Zu9s+d0z5557jpBSYmNjY2PTflH2dQdsbGxsbPYuttDb2NjYtHNsobexsbFp59hCb2NjY9POsYXexsbGpp3j3NcdsCI/P1927959X3fDxsbG5oBh7ty5lVLKAqt9+6XQd+/enTlz5uzrbtjY2NgcMAghNqTaZ7tubGxsbNo5ttDb2NjYtHNsobexsbFp59hCb2NjY9POsYXexsbGpp1jC72NjY1NO8cWehsbG5t2ji30NjY2NvsDKz6B757YK6e2hd7GxsZmf2D15/Dj3/fKqW2ht7Gxsdkf0MIgHHvl1LbQ29jY2OwlKlsq295Y00DZO1lpbKG3sbGx2Qt8sf4LJr41kdnbZrftAC0Myt6RZFvobWxsbPYCCyoWALC0cmnbDpCq7bqxsbGxOZBwGm6YsAy37QBNtV03NjY2Nm3lpuk38cjsR37W11xUsYgJb06gLlAHgFPoov3kvCd54KcHdnwCLQyKbdHb2NjYtIkvN3zJy8te3unjNtVvQtXUXXrNZxY+Q5W/ioUVCwFwmET7tRWv7fgEUrOF3sbGxmZvsr1pOye8dwKPzX1sl46PDBAOw88esegjBNVg6yewwyttbGxs9iyqpvLMgmf4y+y/UOuvpSnUBMA3Zd8gpeSlpS9R1lDW9vNJXejrAnU8s+CZpP2bGjalPnjzXFj9xV7z0e+XpQRtbGxs9jY/bPmBvy/UV6JW+6u5bPBlALSEWtjYsJG/zPkLUzdO5aUpL7XpfGFNn3S976f7qA/WMzhvcNz+LY1b6JXTy/rg54/W/9uuGxsbG5s9h0RGH3+07iO+3vg1APXBek567yQAFBGTyHdXv8uoV0fxwZoPGP7ycAJqIO58keia+mB90rEAtYHapD488uaJDHlpCOd3KtI32FE3NjY2NrtHeXM5Wxu3AuBSXHH7nl34LAB+1R/dVhesiwr6Q7MeIqAGuGfmPahSpcZfE3d8xKKP4EwQ7SSh1zRe9m8EYLHXwyanA4S9YMrGxsZmtzjl/VM47r/HATGfeoTE5wCra1bzy69+CcQmWUNaCAB/2B/XNlHoE0kcGKheF/d0Wlqa7bqxsTmo2bIAvv/rvu7FfkdZQxmPz308KrL/XfVfftjyQ3R/Q7CBh2Y9RHOoGSA64VpaV5rkeklFJIWBkpCeoCncFPe8MdgY99x8/nRXejS+PsIPZTPinj+Sl8vtVCKlZE9jT8ba2BwIPHeU/n/cDSDEvu3LfsQzC5/hw7UfMihvEJO6TeLumXfH7X99xeu8uvxVCtIKuHzw5dHta2vX7jjc0YSUEiXBLo4MHhHqgvFC3hyO7S9MK6QmEG/RX7MwOYxzvmxB7IXP1xZ6G5v9jfevg6XvwW1bkvepIXC6995rN1XCI73g7Jdg0Gl773WAzY2bOf6/x/PniipuL8jjiQlPcEy3Y3bqHB3TOwK61T2q46ik/X+b/zcAHp/7OI/PfTy6/eZvbrZ01aTiuUXPJQn1K8te4fLP9cHDrbgJavEDR2ldafRxruLmyw1fcv4rY8nN78+mspngjs0RCCmRQuBjH7puhBDHCyFWCiHWCCFutdh/ixBigfG3RAihCiE6tOVYGxubBBa8CqEm630JfuE2UbUWaje2re0Mw8qct/OrSq1YX7eeLY0WAxawuGIxAA91yAVSrx6tC9Tx49Yfo+3NRNwciyoXtT1LJNb++NZ4asFTSdumbZoWfRwR+cjAk0hezWYAFmuNfFs+h1J3/ETw37ZX7FR/dpYdCr0QwgE8DUwBBgLnCSEGmttIKR+RUg6XUg4H/gB8I6WsbsuxNjY2O8GuCP3fRsATQ3bcrm4z/Pi0/jir086/jgUnv38yk/872XJfJCrFaYQ5pvKZX/f1dVz1xVWc/8n5Sf7rlnALAMuqlnHzNzcDydE0PyeXDbosaVtJKERRnfVgBzClsYlBAX2gOEfJ2Sv9aotFPxpYI6VcJ6UMAm8Ap7bS/jzg9V081sbGJsJLp8BX9+hZDSN8divM+Rd89kfY05N25kHEm7PLp3lt+WtM2zhth+0ivujIVTSFmvjTzD+xsV6/+5BS8sjsR6K5YyA5E2RE6M14HJ5d7PnuU5JZkrTt47Kt5GhaymOcEvI1jXmlGzlbZO+VfrVF6DsD5rW7Zca2JIQQacDxwH934dirhRBzhBBzKir27m2Mjc0BQek3MOs5fXl8hCX/hY9/q1ve9amtxJ0iMmCYB5SEicad4YFZD/Drab/eYbvIZKjfEPw1tWt4e9Xb3PfTfQBUtFQkJSYLqaG45+YJzwg765axInGxU1vxOXz8dsRv47Y5AEcrY7I05l5dwN6aZm/L1Vi9dqpunwx8L6Ws3tljpZTPSSlHSSlHFRQUtKFbNjYHAVLCmq+s91Wt3v3zV5fCPTmw7APQTCIaaEx5yK6QGHPOP46k5cPr9X0JYYvCkA2ruPQkiz6UbNFbWfk7S1vcP16HN2lbxhd3cHlaz6TtxeHUMfZxtr5MbfnvDm0R+jKgi+l5CZDKlDiXmNtmZ4+1sbFJRAtDzXrrfXNehPqtrR8fNvm9l76v1yU1U7FC/z//Vf21IphiwitbKpmzbU6bu2xFRUsFmtT4asNXaFKDrQtpUa3nG7xOXUD9FvvNFn1DsIHpZdNbfd3BeYMZ33l89PmADgMo8O3YkHQr1pFNhWmFAJzZ50w+PfPT6Pbh/gD/2FbOgNKZiNVfJh03pamZvgHrcE7ZyrM9RVuEfjbQRwjRQwjhRhfzDxMbCSGygaOAD3b2WBsbmxRoIWhIIebLPoC3L2n9+EBD7PHbl8Cy9+L3uzOMdo3xQm867oL/XcBlnydPMu4MWxu38v6a97lx+o28vfJtAJpTuEciqQKsrPWQ6a6jLcU87ht/H9cOuzb6/IjOR3BE5yN2eJzL4eKSgfHvbXF6MV0ydbt1bPFY8n350QnlcS0tjGsxBqZAfdL5BHBss7U7LN6i30dCL6UMA9cDnwPLgbeklEuFENcKIa41NT0d+EJK2bSjY/fkBdjYtGukpvvic7pa72+pTd5WuQbevAhCLcmiE0wI24wsuQ80xHz0whEn9Fua9JtwqxWbD89+mOmbpkP5Cnj7Ughai9kVX1zByp/0mPbnFj3Hy1mZtCjWHuny5nLCWpjfffO7pH1hLcyC8gX8YcYfqGyptDw+wh9G/4Ge2T0ZWjCUXw3/VXT7PePuYfYF1qGY/9myDdAjgm4+9GbeP/X9uNfOdGcCRP/Pv2g+i73D+GWt6X2uWmN5bk8KEdfiFkjtO4seKeUnUsq+UspeUsr7jG3PSimfNbX5t5Ty3LYca2NjsxPUlUFu9/htA07W/+dZpL195TRY/iFsngf+BKGPWO0N23TRj6wObdiqL8YC8OXovvsEYQrLsO76iQwCUvLKsle4YeoNMONRAsveI7zknbhjsj2xKJKZLfqdSXlLOY/k5VKnpLbo19auZXPj5qR9IS3ERZ9exMfrPtZdQAkcUnhI9LEjRREPIQRep5enj3mae8bdE99fVT9nxHUTcSMBPHXMU2S6DKE3/gPgTPDVly+zfN1UQh+3dV9Z9DY2NvuYsD9Z6EtGQ9dx8a6ZCHVGoNu/T4iP2AForoLK1fBoPz0HekTcmyth2fv6Y18uBOr0CJ9wzK8cCjbB1HvhgRL9daebXCdpHRjVvStXLH8u7uU005xAooS9lZWJFY3BhpQFP8yumyp/VdL+Gw65IfrYXMqve1Z3AHpk94huO7LkSM7oc0bc8WnG4DGyaCQQP+E6IG9AkkUPgDMhnNMfnwohglfT3wFXgpjHD1e20NvYHLykF8Y/d/nAm5VSVKJsnhf/vLk65vOvWBE/0bvdsESPupVNTgc/bJkZF2YZ+vw2+M5YObvma76b+4/YsU16SPS8cHx/wlqQvhm626ne0Ta5kcDzi5+33PdZ6WfRx2tqdRdJz2w9yuWqIVdxaMdDo3cRZov++O7H88qUVzip50kpX/ejTVsoUDXeOPY57hh7BwA+py+ujaXQtzFu33ncnwGIzIS4DeHfL3z0NjY2PyOpfuhpHeKfu9LAm50s9InHJ642ba7WffcRyky+aq/hZunQgxNLirlm2xcxix8ILTSlKNgyj18WZMWep4j+CYf9DNiqDyBWrpo0YR3GuLQqfiovIriJA0C+Lz8q7JFcN5FareZ88EIIhhcObzVhWHcjBHJQZrfooquI6+bW0Xr2lsH5g+mX2y/OJWUVEjnUl5wKQfXlADDar38m91XqdyTnNZhCWW2ht7E5CEhYEBTFlyj0Xl2YazfAi1Pg3WtgwevxIg7x4ZWgu2jMbRa9GXscya+jOJGGIN40887YqUwiqQUTXqemlCR++BsqkGW4b1QLke3qykg+zoKMFO08Dk90kjgizhGXTSof/Q4xxfwrQmHxJYu5YMAFAEzoMoF3TnknNoi8fx0s+E/SKf4zSJ/8zauTrPs8n3CLok/mNksuf1Vh7uLNHN/UzOIj/sq4Fj+NMuIisoXeZk8TaNAtscQJO5t9R6pcNr4cMG79Ad2i9xgW9cYfYNEb8P61yRZ+YtRNY3nyYBAhGBP6CF9u+S76OGQSan8ofm4gbBECWvvVnWhCkKFJlBSWar6Id3t4U6QKSHelW273OrzRkoARYY8sdorLH99Ybnm8JYmDo5nman3eorlan+tY8GpsQtuE6g+T3SQ57UeNQI2buvVpyOYWjp0vya1UqF5hDFw9juLfrnO5JXSN/nwvWfR2muKDmb+Pg7qNkJYP/7d2X/fGBlILvcun56Kf8Ri0VMd89IlsS8jw2JQwYdmwNXV6g0hoZIq6pSGTQd7csC3+UAtrfXw3Pe+LS0rSNUmDI7mNL0HY0qTE6h1IJfSexIlQYkIfceGw6nN47Ry48F3o3YY0yK0J/cM9oN8JsHVh3KKyKL0nwZovWXXBncQ5mYRkyKWPMTSgD2QikhNBUXje8QuK5XxAr2O7N9Ig2Bb9wci6b2DDTF3kQb+dN7N5Hqz8NPm4eS/veCWmze6RUujT9P8Rf7ArDdwW4rfw9fjn2+OFf7Zaz481K6xfI5Rs0cftjoh5eiEtjfFC72/F9+1ARqNZIpzQqL+WIyEvjdMk/IWOtGi0TKLrJhIN48WBNKz1iP894lZxCAdNP/5E8zf/0w9KjECKICXHzdVQg8Y1pBL6yDqDlZ9A/WbrifAjb4ZfzkzaLI65A2FaGSscEimh+pVXEc2NUYeN1NSkY/cEttAfjLx8Cvzr+NT7n58IrycsiajbDB/ekCwkNnuWVCLjMqI/okLvg4SIEDI7wfr48nSJXN6piKu2fmbtCY66bqx928EJf4AOvSCrE81N2+P2BVoReqeEdC3+FYcZE5IONV7YOoVjz9OEQtcsPWIn0aIvSNPTGLir1nBd6ULSHF56ZetrCiIuHIfiYOOll7LhUSMlQYo8Mr/znMSVX2hsnZ2jb0g12CYuNrPC4YKi5EzsWoLzRHFImrZ52H7ffZwz94OoHb83ygiCLfTthzVfwd3Z8PQYePNC+Ck+npn3rtVrjv4vYbVhbg9dMJoq4ZE+sHZqbF+gAf42Ejb8AJWr9G3NybHLNnuQVP7ziEUfsfhcaTHxj+DJjIY5WuKNRYqs8MTiw5/NyeJ3RYUQbMYvBOfNuNm6a70mwq/ngSuNllBCfdTWLPopD5FePCJuWwfDF5+b4JMvyuzM/zbpK3HzhCtavi9J6I18Nd5wgHEtfn467mUyjHQOEddN0mRsChE9u8dpAKh+Qw4tBtu6j//H2tPP2aELPbBuA+tOPS1pe2hjfOGXisVZNG3T3U7uoB+fWx8ItFbSGe8Oto++vfDxTfr/ihX63/KPYMBJeg6Txu2pLfFBp8F3j8PqL6GpHD6ILTihdIa+nPvre2HQ6fq25mrL0+wUkXMkhgzuSRq2p/Zj788kiEyLENQpCh2jFr0h9E5vTPwjRNp4sizzrdB5JIq6Gk0IZqVn4lPDZGoaT+fm6Pu3l7PJ5WJJ7SrLrpXWlTI4fzBuh4uWBGEPpEhnALqvPDEefVJTMzdW13AmgleNr8EtQ6/j1MotZK2axx8qq5nUfTT/Zwwo+Qkl9gq9+kGeyJ2CSdSjrpvEO5MUFn1w06b4DS3VetoJb45u2EiVLTfrg58cKRBOk9qnF+q/m67joP+J1H67isDKlbFzu9IRUhLanJzLsXqlPjA1Ky68bieE9p5Fbwt9e8FjscrwsQGtHzPmWsgs1h9HIgfqTSsStxuxzGkdYilx94RF/7CxOvHuHSz22R0e7auvJv3Nwh023a9IcBtc1qmQpR4PiyMiHrXofckWfUT4ex6lD/SJZHYit3YtVUj+ku3jL9nxx4eA+hRpCQDunnk335R9w1/VMC0J7RJ99D5Ni7ZxKk7cjvhskA7g8roGoIGi7K5sd8DFh1wLC/Vwz/MbGiHkpzagTxAXz3oR8mOGQeFGPf4/mlbAlJAtGvqYqJkWQu9fuYptd9wZ3/w9IwKmx5FQ+q2xUf+daCGBYhb6gn660KfnwbjrUeb8Ne783x3+MABTNj9OKlqcHtIModds141Nq1hNzFmRlg9DzjGOyYjdzltFYmz6Uf/vyWqb66Z+i+4e2ktf1jYTef1U6X33ZxKEfqnHiCpxpfHa8tfYFIlccfniLPoPMtL5qmYAa8J9eE5pRrvelFa4z2S+TPNxf8taakXqz6bC6eCRvJxWuzdt0zQ2qc38Mzv+TumpYVPinqebLGyH4kjO3X78g9GHLzbm8925RhinuYThio+pqdet7cR87jnVGwAQkcgXNUh1U5C/fb0aKXVZm7ku0Y0Vu3YZDlPx1NP4Fy+y2g2Af/4P1KyNv2tSQwl3Lh16RnYAECq3DuMMl1egCYXSblNQE3Ldn7H2W46cM4OqFels+KIa2Uru+l3FFvr2gjnnSWLRhIIBMN7wzfc9Ho69CzoNh1GXx1wbCeFyQMyiD9TrGRGhdaF/5wr48o7YoLCvaKnZt6+/O6SYCGxQAzww6wGu6jlAz3Pj9MZZ9LcX5LFy9cl8Xvkwf2tezaKASeTSC3g4L5fXg1toLaZjus8XG1ha4SJXDYu98e1+rJgf9/zXaX2jj52KMy4M8oIBF+hzQ9EGObGVpp1HQcmh0V11Rk76kgTxGx4IUCIdjI2kBlZD/O6tBTz65SrKqvW70398k/A9NBkgTTNnUvnUU2y98y7T/ngRL/28gG2zc+LsFi2s6JPeo6/W+znsXP1a+p8IQDiF0Kt1dWzpdDilPU5iQ9fkGrqDFs6hfEE2W8syEM4972ixXTfthYatcOiVcOKj+uKQv/TRt0fcI4uNrIL9T4DsErjmG/15JAHW909YnzPSpr4MEDH/uhqGFybBoVfAvFf0aIONP+j7WotDNuOv3zs+9D1VYm9fYHrvzH7wcW8cDsDmYA1XdhnDTy8Ppcibx2ek+BGbB3vF0Wr4Y4QH8ts2Z1LVyl1BhH6uLMZWtzDT58MpnFGL/sIBF/L70b+HNV9H24aMKyh//Akav/mGnu9/BX/uCOEW8lSV7U4nhaZonExV41B/gE/rFWgy7kTVICu36aIfUvVrFab3cvkbxRR1rWLErXqo5VdjjAHCNPnZUuVm3ef5OFySbkdXRYVfM6z4zZ3GMT34C645ryNK8dDYxf5mQfRhuDz1ZHjYqd8daCmimgCmdRrBhJR7dx1b6NsLgYaYGyYtP3l/vxP0QaBPgjXReWRyW18HfUIqwrYl+v/8vlC5Uo8M2TwXtsyDD+YlH59qGX8iDdv2jtBb3Z0cKJiibmpT+Mt/2vYTANv9VcxI82Glu/MqFzHceLxBBqh1xMQlU9Vo2EGCsSu6Hs8LGz9rtY0VmYqHBi1AUGookTVBQqHBuKxo4W6T2IWNCdWqf+hJ0qSUCKMc4Etbt7PC7Y5L8fvOlq1GrL/pwtUgW+p08U6rCBBKkzQlrFit+GEt5EMBtVTO/RqrMtyBGn0uwV8bk8ZQk97X1b3PRsNJ0/rK6NRWIuHt2y2312d2RTPmDhSLEokhZxohZxorOnSzPvFuYrtu2gNqSJ+MitzKRwSim6mSjjtNt/gdCWO7wwVDzo7flhgNE4n06GyEyDVshRWfpO5P0CJ1rhWpKiftLuYQw7beXewvmFw3rU2MRvh1UQE3dEwujff4vCf4watb0SfV/hC3b6JR6WhsS+raqmd0s15nEV1tmoKLhlwBQJ5wRuNk5m2o5+OFuksjKvSmAU1NiLHXGhpg4GkAdA6rHNPcErVIT2topDisQsLkrmpKp/zY84v4x99U3C058Z0L6wbIA67nGbj1f61eR+lnsWyhG6flASCNilgbfnUrmkW1KC0YRK2xdhvOGfl7tnUcA4CiJRtCc0f8jh8Pu4eqDp2S9u0JbKFvD0QmUs3hdn/YDBe9Z90+kcTojcQEWhEisdD1W2FlKz+UGY/qoZlmGrbDf6/UKx9FaLEI1Vw7Vff1z/13632e8Vjycv8I5uLQi95q/TytIaUeWlptkbArQvU6mP7gnpuANg1MjYmDsoks947vhL5Oj/9cncainEP9Ab7LHM3xjTGxmq7ECpj8dP5PyBTuhTxfXsrX++7c77h22LV8d+53dFG8OIz3pLIxAJGF/TJS0SoWh68mBMOEy8uZO+rhuG0C+HH9Ju6uNT7bULP+3huUbosXWKcGxwfiPzfNWJk62LEJLdz2RANqUO9zROhVVbL17rupevFf1L77Hptv+h1bX3uXL3/7L7QU5REB/F79vbOy6JvT9GyXwewd17PdFWyhbw9ErCOzYHsywGld4DiJxHhsd3rMYuoeK6xMsVG9p2q1HtGS38/6fKXfwksJeb/nvQyL39YrH0Wwmth95XRY8g589JvU/Q00wtf3wIspVvearfjVn6c+z46oXgcz/gJvXZS6zX/O0Qtw7Kl5AdMg1eC0TuELcFTJUTs81RxvfKTLJK/ub2hSBNkOL2nD9YyMmcJFniudB8sruaKuiTRXGp07H8YEf4irauuY0hhbEVqUVpT0OuM7j+exCY+R7clGCGFMrMqoRe9QNKTh715YZswZ9Tue1e6BrNY681HBlXHn82/dxpnPxRcjr5RZpF/8EUqKAWibuZSfwW3i1fgNLQEylABFshItuAvSZ4i4FA7qP/yI8ocfZusf/0j9J5/w3TtrWKP1oTp3AMKV+nMDa6GPWPkFjjb+ZncS20ffHrCy6HeGRIs+shhHDerVhiLkGxO8kWIWBYbP3gqnV69gVDBAj+6Z9ufkNru6+KrR8IMGG/XJtPs6wsQ/whG/1bdH3B9FQ+IsRwC2LIDnjoJrZujX+JQxR5GWr19fXi9YMxUOuQC+fcToZytRPNFopz1j0d9RPoMfuhTz9aYtNIVdvPVAmMdPVZg5UEFokheeVHlvrMLww4bz0bqPePrpMAX18MaRSpLZts7totSl/8R/U12Lr/tQPvVvJk2ToDhJ738KbP6MouxuEPJwYlMzGPlenC4vTx77IitP1F0xx+eG6VQD0y8McPurYerSoMUNHWsh87vfUZKfUNJQShxS8vALYdQOb/DeeD3KprLRT/db/8f71x3OHZmPsLi+DhZrfHD/V/zbOPT3T38O3Q+LO901wRs5s7wbZ+X1x701fhAA+GjuBkZtr+femS9Et5V+Hl+sRWyo5e0Vt1GaXUCgzlqMO092s/nz5GyUXZ5/Hv6jb7ecTDU+/rDTh6u4mBmFF5Hfpwgsxn8hk4Xe21JFc3pH+qZ5kw/YA9gWfXsgZAhbYu3KtpI4QDg9sbh8pxd+swgu/lCf7FWcsN00OZsC6eugT9gueBVW6QnS1LSEKknN1fqdQSR0M7Gw9JYF1ic3+/Y3z9GLa0y7P7YtYtGn58VnGNQ0+PT3+uNPbo7PC9NcCRtnwvxXoWFLTORBH/A2/gTrput//jrYMl8P44wswtmVuYDajbFrB0LbtzN3/TrKjfA6sVkXlJN/0uhXJulaARl+uGiaRt+cITjDkgLDkP3Ft9YDzWzDqs/RVM7N6M2DnSZxSmMTKK5ozdXCtEJ9TQVEE5o1z5uHvzSmUp2Mse6wD/T+ZjfrIg+gVVQTDqpsWW0aEKVEAbqXQ68VMRfLok21jChfyU1vzGPx5tiCue4rY5P6I8qTw3Ob8PHuvDKWT3yOeumjpdpFOBCTL00NcvyGnyzfg2iXjJsls8j3PCF+8tR93qOWx3p69YydJ2GewpGbi9MIA00//WzcffvRlN6JDVtSyWuy28jTVffNnzZg7/jobYu+PRB13eyiRZ84QDg9sXO5vJDbTf8DSMuLxdfn9Ul5StFgMmUM0fxV7QX8w21aIdhcBU8O0x/fXacXwTbz3FHWq2fNGTQ3GBONEbcS6Ba9w60v9DJH4Mx/JbYIbNNPqdP1gi58kUGiqRxePC62r+cEXfA7DTMJfYpEWK3xxBD9v3GNGy64kEfLVM79vQMJpK83LEcB974SHwH/5EcV5JpybK3raC0q84x49xxVw6E4OdFnCJbDRbcs/TM9q+9ZsNJYAao4CJaVseH8CxA+C8MhzQcN8YNaeovG/C83MuujUk678RA698sFqeGM873r4jayrJw7f/iEF2tPYF3fowEobqzg7p/+FW05pDI5ZfbIPl14bXUNi6s7U632oeMXlXhyg/ScrGdedROml3fnJ/c9WfHvq2vwWMt2zo4dAT3rZ8SiFy4XMhQi+/TTcU7XBznnoGEEN87UlxmnQFr48V05mVBVD6G9k+vGtujbA1HXja/1dqlIOk6YLHofmia556OlrClv0F05EVHLNwn9jcusu4ZuPQWki8+1UXymxhbDsDhhojQSrdPf5N+3cu+YLfpohI3Jog0H9MHLkxnvuqnfHH+eVJO5YJlrXAKP52azbItRfm/rwlhE0gfXtymsdOWPW1n50zY+mHor1xcVUB4Jeww2EyrTB7p+ZdCgCTI26uLY20q/Zs3h4edjdppmUU2pg6ryvwz9c8zRND2kMdJfh4vu2d2Zd+E8ju58DNMWDaZR7cDG5v4s+FrvhzQWI+UNjEVRebfXsqYT3HJhzKXnaQyghnWBWruggtC2bWz570oyG5LvMooa9O/qYdtipQILW2rj2uQEm7hx3pvREMfmShenLv8BKeGbT76nYKYuqoEaN6P8z1CxJINuW7eS79j9AjqOjHT6LVqIkqmnFOn1+Wf0W7QQYYqAksZ73W/hAvrM+JbCm39H0TWXAxDyqzSK1ifKpcVnFQmRDbTs+VWxYAt9+2B3Lfqk42RM6F1eNte28K/v13PlS3NiLorcHpBhmpgzxcMv7XZh9PGrYd1qq8cHCJoxwutyuyf3I+K6Me+rWpPczjyJG4mIMW8L+w33U0Z8qGdr74950jkFfiF4MSebi4tMUUnGghttyzzkqh1M/Kphvvr3cr761zKeX/c+3/i8zPR59fw181/Bna0PFIeu0thQk0ZaAD4fYR0dMnnVDNKNz6K0CDL8sZ/yTSNv4okJT9A9oyS6rV8gCMKBHHGJngLj8N8C4HK42LSsmmVrCvim/ho+2n4Ls2c2I9EHNndmGHdmvPg4A+ksV88j/ehjkAjC5eU4jL5sWlbF9gcfom7uds77KWYtCwkOVdKtRv88+lVvxBMOIKRGviH0vhEj+OFQPZXCcRtns/mHXDQVNk7NJ/Oj9yiUfi758Ekcm/Xz1nnTcfuDVC7JYvwPC3H5Ta/nkHhyg/jyDb96Wux9lF2sI4cKbrwRAMXtpuSvT9Lh8stxd+uG4o6fIM047Qw6v/ACCIGzoAChKAinLt615c04J59pef4IWaeeGn380+W38uLgk5CqrvTBZlvobVKxpy16KWPbTG6d5qAaC4k8+Qk9sid6jpiIflWs18uslz4Wa/okXC66hawaeUgYn5AKN9QSy/edafJTWgm9ueBDZL4gTugjFn2GbtFHQh9bE/rJ96XeZxBJxRuXqdFw3Qzr0ZV7Fj8T277sAz1tdCTfzvrv4N6YwHQpl7z1oEr9djf8qQN89wRhIzJl+DqJf4uHZo8HV9ZTLLeoitSvMub3ruycQXogZiVeNvgyjul2DEVFwwH4TdYgMqVkS2Umf79xHttHPhq3VkIY16PJ2B3CsgGXMG3C0xQMqcfRKZauoLTbCawb+jCyuSfVJ/2GaROeYtNd91H+/IsA1G5voerr76nMG8z/Ov2bJiNK5/1/fsXrD6scv0Jfie1Act8Pz3PvzH/yu3l6IrPCZ55lTXEskitY72Ll28VITe/fSx/cjkuNCWGVN4ujq5uZOuFpQs405KaY26P7pAp6Tq4kp6f+nQpnxt6fssffTno/AfKvuTr6OH3sWIr+7xbLdlfVdmfoRzX8+X/Lo9sidzRr5pQz/b0y4321PJya4tj7+ZfKfLrnT6F8gz4A2ha9TTKl3+p55K3CK3eGpONkLO2r00vAWH4eUrWYyOb3BbcpY6YpEqHKLzlfPMjxgYf4UdOLMDiF/kMIG0F3gVAIrp8DY36pH9RcHatwZMqbTkutHroY8cVDvNDXboi1W/891G6Kt+ilCmWzoWotpEoD8KsfdX/7oDP058IBp/wNbpgHV02Fc1+Hs/5lnUZAajQY2//bYuQcL5urx9YDfPRblnz7KG+++xkLZWxCr3irfkz/r3vyXs0pLC/vybbMkWjCSWEdhAIKVVn6XdXaHkczZ3AsV0zImUZzRndmF/bD/+Bfqcx34w1Z/JQj+c9C+nu+ZqM+0GxdUxvXTDGEfmMwljN+e9Fofd8Jd+E8Mzavsr7nCQBkAAu+1ScyN3Y5GnrGBDrkSqM8f7h+nh7xPm8xYhTXTbiRak8m/dVaRhoTr81ODwGnhypTFtbvSwYnXVKWaV6lZ+1Wugp9wPJ7Yq6kLkdW4T3zj4CRmwaoSo99p8Ka5I5j7uGNYyYlnd+KdQsqWDQr5j+LVAF84bvSaFphLaz/zymKGRMpsiKzcEMttx/zJ3458Xfkq/GfW6C5javKdxJb6A9U1BC8dDK8esbuh1dalS+L/OAUBy1B/RsbDGtwlBG1ktkptsr2cFPM+8hLqW4KsjV9AJ269WGbyGe7zOEd9Uh6FaSjGl+5rdWNuo+/2zj9uJbqmOumaFDsfIEGvdrVv6bEipj76/RJ4Tgk/PsE+M/Z8T560HPy/G1EfFGPTsNijwuNdM6T/qT//8WrMOJiPdSy80g9P9DgM1IIvcr6xLjpfx4N5cacxbpp/H7eLCpXHc0HDbGiLyHjfVg05Fq2BC5jat3vWNLnalb2OQt3GGhy4HcbOVukk7zxR0ePXTj0OhYM/w2rc7rRNOgQWtIclhN8p/fWawhsXa67L1oC+ufldMf7iFvLga70ORxHoX6HpWRlRV8nSypERpL13U9kqhJzE2mKG4cRF97SKz5Vdrc/3sq6nM583m00jrra6HZVKLSEVLYYdxp/HX4Wn3aJD7G0wm3EpAtjjiandxMZxQEYcQm40qJ5apY6u0ePKVYVJqr5rPDq+YNWFZeA1zpiLRxS+fTZxcx4MWa9m9/prUbaBVXV8KQ5KeiSwQ7Z2sLRah6d3V2SBDjYYpcStDETmaTcujA2OeraxfDKRB+/lHouboDaTfijFr3U49XvrotZx3fXxQTy7jo4+Ulqm0Pkprl465qxLPvT8YwJPM3NoWv5+ncTmDxUF4RgyPD1RwS7qSI2YOV218/l9Oo+dqMmKGuNRFj+OijoH+u/OcyzuSpm0Sfm6DdHxlzzLUnkdNFft/8Jlm+T3+SyWehxc3d+B7Swn/XumMuj5YvbWedycmNhPivdLi7pVIhL1a3xjEDM6jzvS0nAnYXfF5+XqMpIeyvqHLR4dN9wljObw3tO1BsoCg0ZXQBw5B9GxZpa1HRv3GSsf+MmVvzmNmo/8DHvvAVk1Y1kSfNk/IbQm63GFT9u5ccPYm6gRN59o56fvtPdCsLtQjXmJLJQ0ExLWh2mAL7ZI/+PCiOiRgnH2hTe+nt8g/VBvCohx9Ha7M58sGALS2pVnr/7NZ55415CnWKDR2RyFPS8Mcv7XYBEREvwRfLIpA/38Fntzcz6qprl6snIbN1Nst7VMXq8u1nv0x2nHsH081bzm9G/5f4bngXgsyVbOfOZH1hUVqu/l43JFrYDeOaEwZzU5GLNtgZ+/GAtq2dvx69qfLzaOntlwAkf9zcG2qAxKGkiSYADLXvHorfDKw9UzD7pyKIddxusCSsGnKznwSnor8eXI2HYeVC+HA77Jf5KXeiDqqYnnNpBJsTqpiDFOV4cisChOHjgjKHkZ+hi5zJWewZDxhe6oJ/uzCydEVuNGxlw3IaPPbe7HmlTsVJ/HqiH7IGxEMguY2KpkfP76ncGDk9ynH9kQDvLCOU7+6XotpAWQkqZVCDDjP/Up2CO7su/qTCfcqeTK2vrWOqOuVXWzfkHT+bl8pMngybhYJ7XQ796/ZwRwUd/h2lK60giIZcugDnNsMqYBFTDGsKwOIXbHU1PUOAuYssHGzn/5Gupee+p6Dm2PfwoM+vH0LywmqrNjcjm0XzDaHLQr7WhNkCwtgF3TiZf/3s5rVFXE2LRjHKOvuhqmsaMx/GOfleVHRa0mAYMrznFr1AINumWtr/XIKZ1PgRVUfjVWXpOpWcvHIH4vgkW6Sk6Pu02htf6TaLyMz180WMMnJk9uvJ2nwkcqjQw8eE7WPB/d5K9chELht5A2JVGr3XvI427SlXR36vSwGjW+g9n7adlwLlceWUVoS4rOGryeZTNGcHobpks2aYbFP6mEFk+/bs2Y7Uepvn2nDLmbqjhy2XbGVqSQ11Fcj6gWyf3Z8sXmxkQcjJ/9lac3+u/xQahUYlKP1yEkLhM8fINmsbybQ1MwRuXLeP8Q7vSMj0Wy9+pV06rn8euYlv0BypmoffX6f7yVtKftorLq2e2jEzQSamnT5jyIOR2oyUYu5184btW8r4YVDcFyU2LCeZ5o7syaaARoWO4SWoiFlZ6vi7U3z0G3zyoi34kd7nHEPLIQDb9AXigs27he7Njs10dh8RefMN3erpkp0fPuW8mHNAHh8GGL37QaTD8PL2PH5/HyFctMnmaaImsJSBW7PrPeR34T3bM2jy3c0dmeb1c/dNjdFx7KwBOTR/cFNPPTVOcBD2muQgD6YidK+jWj9PCEsWbev7l849ymDX69ujzhrCDZmMStHl9LKS0tl4/x9Lpm3nhlu9Z9uaHtJWpm4bxwzux8MViYwXtCpcu6J4U3p+axQ1s6n8Zj448D4cR6nn84E4cdZSe5jeclsFfDzmbyrSc6DEuo7hKn6IsXhx0EqU33o23Xz9qfn8vAGo0tYeCq1BfhKcZA3RNuHPc67/yVhE/eKdwaEEBq5Z0pK7XOGq3G0LfGCLHF3O7BcIqG6r1fesqmgg0h3j/sfg8+wDdcn0EGvRBrvSnmAWvCWg2UnYuc8e7YGqN7RoQMu5yFGB879gd3bBjunDcFYPYG9hCf6BiTgjmr4+fwAR+WFtJaWUbqtZbkTCL5Dfdfs9er79ueb2fL5fplsgXS7fxr+9L+cc3a3nsy1Vsq/dTmGVdwMIx8mLODNzF243DeX3WRhZsqoWIWwJ0IY7cMbgz9fw4kciaaIdq9euNmEZpefDLH/QBI4LTq2fxNBZ1BYH3GlYRNqKIllUtY3HFYjSp8d7q91hZo6dy+GjtRzQnLKT6fP3n1Ppr8ZtcP6VuF11rBrJC9qZn1bC49qrR/6LG7npX1OS7hDkD+rF48C+StmuoNKV1ZH3XyQTcOdHtitXipRRsboqJzOwvrVM2aw43Zf/+YpdK1wmXQldjkrPcmJn0agJ/ijz1JaqDmX84Om6bu29fAr//O6tui61EzVEFXUIK/pDe/98c04enzx/BxWO7A1BQnM+vJt4UXZmqKU4w1iFoRv79WjVe6APNYbaX1rPsO33AWzu/gppt+u/C3xSiZ0EG54zSXURbav1sqNL3ra1oZMNS6yI7TTVBQgG9j33C8cbVQrfKaxkBlnnihb7OEHpJbF3shL6FaGrsPXM4954c266bA5U4i742SejPf15fDr7+wRPbfs7I5Neoy+M2R354AP2KdIvz4hdnsWJbAzP/cDRXvzI36VRFWdbClO5xMVf2Y+6irXy4SI9kWP/7c2G6kcJAMX0lmytT9zXd5NtO66BP4JonaCN3BSc8Aq+cxvuZGdzbvJLGDB8XAb/4WBfZFye/yJ0/3Bk97I/f/ZHTe5/Onw7X5x0qWyq5+ZubGd1xtL6C1MQJK66JPhY9f8czRdahcU4tWegrO56PT0220lUlzNpep1GZNwRNXRFNDCYMi74tulwbEGAsg6hsJZ3Q6m5n7JKll1Hgo2FLEyEklQ7dCPAiqBFavAvHwOlx0CmhPu36RVV8/5MkbXgWoN+xXdWgf2dWGKtDc9PdnDg0FmpblOWlNLsYavXnmuJEOJxAGDp0IDc/kGTRR1gxUx/wmmoC1FfqA3bE/37i0GLemlPGnPXVhFRJptdJWU0LpQutv39VW5IX0wGkaQIpYLNTI1eNfx8We/TvhiZAGFKf5nIQNt0tK862Z9TcWWyL/kDFLPQrPo6Pad9Vsjvrk5E94zMjBkxCHzQskM01uu/y3XkJq00NUgm9ogjSEqI+yO2mhzNCfEnEVvLVN25UKZvu04UvIvDmfDMRoe81Ec56MRoD/4Mz3tJqMWWL9Pklf3xDpXL5fK753+V8d/rRPP6yviagtK40zqJXtPifjlz3KJfPeohTll5Pj22x67vip0c4vDIhkyfg0lJFSAn86UaqWiU2mAkjlUEgMaevBV03xK89KKx/3dTR3YvqED4Hufn6Z7vBqWFO/9WSoFOVikbRsDzCAZX/PjyHb19fyZxP1gMQ9OvCJ5uSJx99ZS188cLSpO2FWR5Ob4wNmr6n3qDOEO28W26jw+uraNZar5K1YUkV0nC7tTTovS8y7j6/X6ML+yFdc2nyh1k533pitXqLbvVnGe9D14H6a5pjr1pMdzdf+IJUOGKum8g3R0pJOGia0N6LFr0t9AcK4UB8OgDjcajEsMJNAqmZCjn4QyoVCblJdpYWQ+iFMEIsgTSPLmYzVuspCE4bXswNR/emc45uuRVmpq492hxUSQu10KdmE+nBFhoDYTRnLmpQgC+HcEVFfMifEWsfanLgr3EiNdh0z/M0bHQSbnZAWh5SVfFvaaCl2kWoRaFWE1S2VOpCnl5IlVFRqVmGo+fObpTUBmqjL3PoasnwUknawnVsXjqLvOVbOfJVPU2CVCXNTbH30aElZz90q16K6/tw4caYK8eluQn6k8XVKd24vArqkPjVtC7NTaNXn79wiJjQ1zcaKXLbYNK3JETyFG6JDcZpLanvkvy5rafXne4NUXByF0LG6s01LpWwSdC6dYw3NkICfOn6Obetq2fxN5v56UM9wic6oW9xOd3W+lk9O7lSk9uh0NvkKpn2yoro43BQo2Z7K7mLEujYM5vyDQ3UV7bQPS+dNLeDL+ZtAQnDu+TgleBQYaE7TNoh8YNHfZVuHLh9+t1nYXd9Al0gOPfQLrx21RiuObZ3tP3QLjnRxxqQbtz1SAkhk0Xv2EHVr93BFvoDhVfPhIdNBZWbq6l2deL8tUayrcaY9dEUjLkQ+t/xGYfe9xWrtrex6pMFfuNWOsvrIqjqX8x0IzJi7oYahnTO5olzD+F3x/Xj+MG6SKWy6CPc+dO/+es3T3L7rJcYfNfnrPv1I6x+vyPB7uewevyR1Lz8MnQ06nJOeZBw0eGs/V8hpZ8XUr0qPXqeQL0TfB2oef0NSv9VxvovCljzQUe+fG4GE9+ayEOzHoKsYioMX26LUKgP1tNvk+T5v6mEp30fPdehq3XV6dAoUQxDK/J/zNIzqX02ln3Tyh0ToUK9JOW+dEdssO4/phO/vu4hpExOi5vI5k07bhMhUegz6mKv6fGn9uV8FWh9Tme2N0xecTr5Jbr7bp1LjSs23qNzfDhrSEgyMqwHj8girTaUn40iEypRedJjbr5wSKV2m3X/e4/UPzdzsNiQibqL55NnFuF1OTiuWx7X1fuYKLx0yfXhMcR4s1Nj8CQ9nLW4Tw4AgaYwbp+Tkv76ABAReoArjujBuF75XHBYbOL+grGxxxkm15Zu0ZuE3rWPXTdCiOOFECuFEGuEELemaDNBCLFACLFUCPGNaft6IcRiY19yImmbthFJqRux6Jqr2Bz0sU4aPkyTP7spkGxBrty2O0Kv4nIIfC5HkkUfUiU9C2LC+4cp/fnixiMpzrGOEmn87nuuX/BfhhkZCodXruHqxR8Q2laB1ATBTvqcQt0nn+Af9zj1Q59BSsn8713RpfBNrgnR823NO5ftjz9F9SsvE8pVwKv3b/BGyenfa6wpW0T1+19TYUQkNSmCj1e8z3Uf6+/R9s8/4oRZGhnNkmHrDKFvgLSA/rikCk77QaNLpb7CV9H087Qm9K3hMflh3Wm6UG1038Ho2X8mXBdzV+RXLow7rqWl7Yq4oduUuOdef8yn7DXdwSQSbiWnftDYl+V1Me6MXpx9x2je/92RmF3R3gRRDwGZKe7sInedOyNt5olL0AU32vcUFr3DpeD26p9ZVn7sO9lzeAFZ+V7ChhFz2QBd+I9UvIjNLbiNlwoi6V2UyQV/OowTfxUrCO7NcHHYaT05764x5HWOff/7GHNYWT4XmvGeCcX6KqVGnOtG2YsW/Q4nY4UQDuBpYBJQBswWQnwopVxmapMD/B04Xkq5UQiRkHiciVLKVmbWbNpMsEn3xzdXUUcmVWThz+6F96jfRps0BpInBc0TqjtLS0jF63TgdioEIkLvin11uuXFvuhOh0LfosykcwBITWPTlVcSmR7+uMdYTiqdyelrY3nh/Ut1sdOamig9R6/slDNsCBlzVtHihqpMKJm3KNo+/O6XVAuByEjn5bFQlu/krtf0Pp73rcbs8lVsX7GcLsc5mTkSmjwZbH7934yo1Y+fsFhP33X0QvCGIKxAbiOkm7xd53+jMXWC/rjIWYw7Q+GSvldTbkTeKQ6RJEKJCC2MIlWWla+hY7aewTNy6z+q6yFkfDkDEdRTOwTVFpodMdHypDkJBKDak8mLg05kRPLpW0Ux+eVdoXirt0VIfIaVmVhdz+V1EDLcTp+n6b70TK8Lp9tBYecMCoG7TxvMxlf0OQG3N15ONAekpbDoI5ZsjteFx6lweO98+DHeGJGajBNJtZX3OBRUaaz2k+WppT6QE7t2h4iuBPaku8CIi3e6HHTslc22tUY6DWNFanN1gOZPy3Bn6K/r8TnJTXdDujvObaapGg6HQodO6Zb5aTxOhaAAr9T7YEmCRe9071vXzWhgjZRyndTvMd8ATk1ocz7wrpRyI4CU0noWw2bXMNdPfWoU2sov2LptMxVaBiCYPukTfcm+gaXQhzVuemsBj3+ZXNRhR/hDGh6XQ//yGkLvM02odtyBmybC5ptiKQBWnX0VS/L0VaAf9jicbv/Ry741/6Tniw+uieUkrz3xHAAeOkuhNkOA4T76sZ/+A0o/5mhK3/oTn49UWNpN4dXr/dx+kd6/ThV62wKjCESTEKhVye6LrpV6xaSFPQVD10tufF/hx0PvoDq3f1y7IwqO5OPTP+bIjhOi2zJyU89HKKrucsls2MhRM26iXsZGEI8h9Mf11lfiRtIGNLk8TC+JpXNOy/bgbwpz8Yl383XXUSlfKxXbesSikd7oEz/R/lR2bII5cZr36ieO4rpnj2bR4VmsMOLCs3zxYj5xYMymc3kTJtmdSnQwM/PiLTOoLNPvMlyKYOWfpzCuV3JGyf8+EovmWjV7G6/d9aPF1YEvy82SbzZTuqCSDv16x+1TlJjQe9Pj+6I4lOgAXVcefzdwXqP+mRbmxu4CzAsF/aa7CbfHIu2wENG7ICWFRe9vDrN0Rqxugyet9TmS3aEtQt8Z2GR6XmZsM9MXyBVCTBdCzBVCXGzaJ4EvjO1XkwIhxNVCiDlCiDkVFRWpmh0c+Ouh0fQebDaFLzZsRbx9MZ207dRI3XJesKk2zmJfV5Ec/hUIqbw7bzNPfr16p7sTCKn43Apup8K6iibWVzbFvV7H7GShk5pGYO1amufMQa2rQ21ooOGrr3Dk5pJ3zTWsG3s4P5T05KUBx/PvgVNw99RFv+mHmSn7sTlPUFir/3jKx/bh9aMUPh0pUC89k9Ktm3CqbtKCWXyS1oFq46aixAhOGr5O4sWJX/WTWR+i2VdAsy9WiLnZV8A74xRWlxQiEQQ8HWhO78iy/hfR4o2J0GitP6HNmwk1xwQyI9d6oMvs4qWzT+9AVMRNBb89aS4WldVSb7gPhJG3RQXqtZjsCo9C2dpaSGHRBrvGxKiwfF7S/oBpUZDVGV7N8JN9XHFKx41q8o1neePFyBwpkmjRKy6Fzv1yGXZsl7jtLQ0hlnyjTxD7m8I01wdxNiUbJ9tL66PROV+9uIzmeut5inGn96L/2E4MOKKYEZO7xe3TLXol2r/TbjqEk2/QJ8sVJXYnFvBbh8Z2yo+Pjjr0JH2eLGxyj6ZyzRhpdhCK4LWrxvCfK8fE7W+oil9167EYFPcUbTmz1VUkfiecwEjgGMAHzBRC/CilXAUcLqXcYrhzvhRCrJBSJiUakVI+BzwHMGrUqJ1fxdGe+Oshus89Ul0pHP8FF0ZI4GapC9Cz36xlyeY6XjW+SDe9Fe/fhVjypV0h4rqRElZub2DCX6bH7S/MTBa6qn++QMVjjwGQdcIUMo45BsJhSp5+irQRI3jxpSF4B8Ab6Fkenbm5Sef4/DAPk3+MWcB16bCoh+DYBZLfjVtHwC3413EORhYpyPt6c3LGdRQ1dqfat413B98fd67sZhjbVMy09I3kBAbz45hrAei96gWyG6qZO/IWGgtm0a9iNFuKXye7Tr+jCHpymHnYn6Lnybz3JdbU/IHmYy4A9IRsqSz6/HHFpG3pADO2RC37FkfM+tvWHODCp2ZzpNLEH4ANGXl0BVxOhWbTr27RplpKVAfjPU6m+5IFadyITkzdtIYsqdCl7GvKC+OdO02R+RKpxf2aQ8bPeKtTcsHp/Xjue9OqZ5MJGDINMImhsQ6XWeiT93l8To44qw8Lv9qEFdvW1fGv//vOch9AXXkLBV0zcbiUOH+2mf5jO9F/rCm1tSCqUPldMqMWvcOl0Llv7Htmdrmlcr317RK/PuXQE7oz++NSegzLT2qb2zF+UDAWD6M4BON66e0XmPaHEubSrO5+9hRtsejLAPOQXEJyydsy4DMpZZPhi/8WGAYgpdxi/C8H3kN3BdkkUvotrPxUn6RKXCikWodHvqxOjj7+bk3rUyBtSV2QCn9IxetyRMMsE0mMsKl54w0qn302+rz+k0/Zfv8DODp0wDdsGDPKZiSeIrrKFiDwrwe57pcOXpwQ5svHzuLCmx3cfFUJ/cvH8uIkhYfPO4oRORN448Q3AHh8rp5GN7IStUNLR4bmdI+e76PR+i8uYt0LR8zqc11xC2l3/gWAfhVGat4LrkJVrMU77NRFs3FhbOI0o0Ps+hVTXH5DUxCnSxeZYLp+XEYw5iO/63M9z8y3WgcuPu42fizS3UQetyNulWmBkcp2YNDJsECymyAn10snw8XQ+e9PJe2v6ZTJxemXoHT9LLqtx9iOnHzHoSy5ZzLz7piEECLeejNNDKqaZEjnbGbddkxSniOzRe/yOsk6rzsL3Ppg5LRwaewskYRrkfexLURcJYOP6szkqwbjNAajxDh1XeiNwjEphP6sMfF3I0IRXPrQ4Rx3ZXyqgsv/cgRn/+HQuG3Du+dGj7EiceDypO1boZ8N9BFC9BBCuIFzgcQkGR8A44UQTiFEGjAGWC6ESBdCZAIIIdKB44CE9ew2gJ5y+PVzmbbSYnpDDYEzPoplqdaN0M+0sNkf0vC5HEkTug5FML5PPnnpsQgUtb6ebX++DxFJ3WsIg1pVRcbECQiHg199/auk17jq5TkU3X47ueefx7ZOHipyBFIInq94n6BLcMLqPzBh3XlIxcWorecwbOYpDMofRJ43j9Ka5EHs1p5n8NOYHGpLspHnnoKmwKHBYiCWewYgLT0TkRlvtbkycsi69ArL9yIi9Kop+VlaZuxx+sYPCSGpVTSa0x1Ri3dOWhGlWZ34sdPAaNtyUxhsRVouYUMQfB5HdMFNAMmnafrdQLoUHNeSHO3jTXfhMoQwvXNHBo0v5oizY2UeC7K8ZGbWU5jtjIp5h2wPvTpnkeFx0sH4/C472ZQEzjSBGFI1fG6H5Z2beaLR7XHgy3ATuSq3yfofeXy3JIu3LWxcWoUa0uLuHHaEYgh6x57ZeHzOqIgnnkNRRDT6xyz0mXmx6/RZ+M3Tsz1JA48vw40rYWDzGsK9o4n6CHvTdbPDd09KGQauBz4HlgNvSSmXCiGuFUJca7RZDnwGLAJmAf+UUi4BioDvhBALje3/k1J+ZvU6NjpBU14ZAo3waH995WtGQVw7L8n+Sn9IjUYGDKwq5e2PbyfXX0/fmo387/1bGJsZxqGINi26MdMSUvG4lCShvzG8knve/zNCUyn9xS9YM+k41p1yKoTDdHn2GQasWE7+L6+Nts84/PCUryGcdRyj/YX7Dq/glm/0yj6jN57EOQvio3nTgkbMcrOT9x+fxznf3Mmg7cllAAdk9+LSl2Yy9qsf+f0JD+Lt0o1cI/zOIWM/XuEQ+BNWZ27Y3IDviPhJywibSo5m9ohbUB0xMaj6UyyhWL3WyBM5fp7PCnDP5yv4x3e6RVrj9vKro39HnSl1ckuKIHK3y0GkMuA2p8Zqt8Z6p/XdFOhCHxFcqUkmXNCfYcfELNFTh+lTakVZnqjQW73y5RP0sEkAadKssCajicYSMVv4Lq8Dr0uJpm3wmXz2h53WiyPP68euMOPt1TTVWt/VWq0mjVjwkfckbHxvnUkWvWJy3cR+d8OP7RK9K0g1kdoWIsIfsgiOaK393qBNQ4iU8hPgk4RtzyY8fwR4JGHbOgwXzkGPpsH2xfEFLyyIuzOu3RhNAyAzi7mz9kTWB7N5xf0gmSI5fer3ayrp2kG3mm4IrSQj7OfYjXMoaq5BQXL1ys9Yk3M4DZu3kVXSKen4lsVL8A4ehH/ZMrz9+yMMf7I/pFKQ6YmGVkY45sPnCAAVTz2Ff2Es5NHdowe+4cMByL38MippoiinhMzjjrN6YwAFZ8ZywjLM9LLp0T0jNidXAMoMxFYpbl5ZC0D/7RYFKpT4r7baYyCurZu5buivkB/EJqs1VdKSMMm3oaye0UFrYW1K19+3yOQqgDPcwtBFf6eeMI8MO4lbp/TnwU/1VZt6pkIHqoVeXHxEDzZUNfH1Cv0u7p6TB7Hy9TU4HIJ6RfKJL8hal96PxlZWFrl9jlg5QEvrUd82tlc+K1qKCMyvtlZ6QDGMAFUI1lc20T0/nbAmcSo7tqjdXieeJkfUekzzxH8Gvsx46zinKA1vupNt6+ppjaXfWqfZADj3zmRPcGTyNTIIqOEUFr0j3qJ3ex0MPaYLvUcW0W1wXjTVwa4SFXqLldFWpHLx7AnslbE/FzP/Bv84EjZah4hFMWeONOWzCUgnr/jHk9ZLnwB8Vz0CIM7S+nFdFZMeN+a5O+sWXd/aTfg76HcDHWdN5+Uv7mPzsfGZBAH8y5ez/uyz2XTttaw/8yyqnn8+9tphDa/LERX6YSXZODVT7c5n/xF3roJf3xC19F4qfYtzsl9j05ShCKeToJpwJ6Lolppwtu1HZS7eESGvpTi5YULFpS85iemdruGqHufFlZ0LtoRprIm3Fl1BSTiQPPFnniyrzYm5RpzhFvKrl/LMwKNZn92Ja4/qRf+OuuUesdGsbLo/nNCfyYP0lcQjuuYwepD+OeX1zwFgqUeNWvb1SmqhT8v20GdUofE43rWTVeCLLrJzKApTJujzE5FVnon4DDfUD+GW6KR7WNVwtiJCXiPNgS9Tj4mPvPPpvngLNT0hOsub7mTE8d1Tnrct7pqcwmR3kMtwGUUs+o499bvA4r45ce0UhwCpL97SVEl+l0zGnNyTtCw32QVp9BgWfxe9s3QfrE/A5nVuPQ9V4me2N7CzV/5cbDUiYWo3Qdd4CzQQVon8BJ7+bD7HR3Z8eH20zZLtugV/wfiBDH3xeRrxseju4xhz39ecuHIaa4r7UtUYi3p1G5E6HZuqGTc/eVqk+j//oWX+AjKPPRYUwfb7HwCg6Rt9oKh46mkc2dnknnceLUEVnwMuWPIJU7uMpFteMTXLViadU0lPR2tqwt29e3Tbimrdst3csJlhBcPicssAuDvMQChBhKuOLjUDSAtlcfzxhzE0fyizZ+qWt5CxH/z40rOTXteSFMVRZl59H/UdYitHI2F+EUJIXGpyRATAGTePADQu/ccMrp72b0IdhrO101haFEEuUGtyy7x5zVga/CFWfLKRlTO2oCIpyPTE5R1yORQyDfeGz+0gK9/HFY+OZ2V1I8xdG/fa610a4yy8F1c+Nh6Pz8mIyd0YfGTnuFjsq/96FIoQ8NUH0W2d++VyxaPjo+KcSHq2hxcLQ1SZ3A1hVeJMtegHuOBPhxEOanjSXHhdDiJjUlpCKGbiZGNJ/w5JaQ3MuL0OBhzdhXmfb0jZxopolI1h0XcdmGd5zZGBQFM1NFVLKrG4u/Q8pKDV9zrC2beO2qsRN2AL/c9I5IeS/MV+e04ZFxqPa6oriKp+zfpom3q/SnG2l8N753Pk0N5ceFg3srwu/n3ZoWSdcCMsgYcOeSna3mWU6utdZ33bu/3eP+vn/fhjnMWdCG9LyFseDrPtnj+Rc+65+MMqJVvXMmnV1wyoXk/TOUdQ3aBHyQiXC2lUi+r6739R+Y9/4OnVK3oaxbCsVWN1Zo2/Ju5lPAVTo49PnPkkAH1znRxSeAiz0aNzHjn0MdYYN0IuLfXipDhMRUdUk8tpoSHyfi2A1xRZ4013EfAIVjc0MzDkpLZGD0dd7VQ5ckwxOSh0KE5HCEFLTgbP9RvPlDr9Pf76hIvoNu0dtqbH4u2zfS6yfS4ieSRVAf07ZlLREGCqN4gDwXXExiOfyxHtR3pTsjBscWgEkbgTop0jwi6ESFpw40ohXDsSHuFWiEwBaZokrGk4W1me7013RdMimy36zATxMvvzC7pm0u+wjqRne+g+JI9grost325Lal/Q1XqVdUHXTEZN6W65L+qjN92FWF2zYrijNFW36FOuYN0NEl/3iHP68N1b8WtZ3D5n0hqEPY3tuvm5iGbrixf6ldsauP39xdHnuVjnulaQ3HXKIByK4KnzR3BYT11URneNRYz0mv01n75/M8dtmEXBJ++0uWtqbV3KfSsGDGTY+vl0W6kvxHFpKkfccRW3zX4FiaDPDP0OwFncCd+QIXR56imEO3Yr6jBqmUaEPtGij2B2yfzxze3MfCcWSTMye4zVIa1SuqyBp6+dyvE3fcaz109P2r9Si8+OuKmjk4e1Osqc+qCwxIj7fj89yJ1ryjj6ogFRofI4FRbn92Jq56FIJN3GDeeB0RejKo4kF0dalv5eNArJsQP0qk9zvSqzvLrF7DOSw/UqiN3ep1tMykmh92XXiHzn2iZkbpOoB8Ka4aNv27Eel0KjYdJnp1hIBnDOHw8lpzANl8fBidcNo9CqhJ6A9BzrgX3S5QPpeYi1ayXiow+HrePuI0SE/d+//x5VlXs110yEYUd3Sdrm3ImIol3Ftuh/Nqwt+ukry+klYssShivxucQrFYUahwNF1aJhcGZkMPbjHz/zIwAuX/q/VnsSGDwcz5IFlucAUNLS8I0aSdO3ukV944+v4jF88oOq1+v1TqecRtGR43Dk5NDlhX/GWfFx54pY9Frbhb6s3Mvi1WXR5y2Nev/SOgmat7YtYihyu3+4P/kr3hyqZWqHDtz/m9GEQyp15S2c8O5cECC8DjDPcws9pYSmyaiFGCmtuMQdptqh8XjfAjBSS0y7eULca42c0p3MAh+d3GFOG17CyG65rNzWQD/Dh39kn3yePHd4NOsnEFeG0UxozxuclrhN0SktIVV33bRhMhbA43Qwwxtmq0Ny8oDk3PC/uH20pbvm8DHFlC6ooG5+bF5KCEF6Cv91a26WyL5wign1CBGhDwXUvWbRt4WfY4CxLfr9gK89t0Qf3+v6d9y+k7sUc0ZJJxQ0SwHQAqYc6bJ1C0YWl7AuqxO1p54bvyMcP1XoHTKE4gceAKcukhGRd3XW5wAceXmMeOgeupx+MqCHTbo6Jhe6BnAYWSP9qu4KqfXXRvfdPfbu2GuGY4nRXAlfy0iBiImnD6GtNNUaK1EtolXezvUSdjjI7pRGxx7Z9BvTMRoVY/aVqqZBud4fSnrsV2CdS6NPkW6NnzysmC4d4icHHU6FAYd14owResje4M7ZnDmyhMGd9TsxIQSnDu+MxxkTLm+KxUGtZZdslT5G9FJXi+gkC8x9aQmpuuumjRa916WgCljpVsn0JrtL8ksyUrpjJp/QM+65EMkTuBFas4L7HKrfOXXolJ6yDcSvAdBU7WcT+g7F6T/7oGJb9D8XKVw3Vnzj87LF6eS8hkYaDUtKQe7QoncY7pHsoHUEi+c/b/Prx7/l0cEjufrkB7hoxf84a3Xy8nPhcODMy6Pv999x8fV/5/bZL9OY2YFRX31JeNs2lMwsFHfbIgUiFn2TkTWxJqD76H86/yfSXGmMLzqKe//yHPXemCWXWGi6xSjE7M1s3bdsXvreUK0PLInvtm9wDuVleshqTXMwaRGQ26MAKlsdGq9lBFAEaBKueWUu/pBKbrqbysYg1xzZkxsn9UURArdTYfV9U3CkmADeExRmenj93JF89sj8nT+497FwRyU42pY0K86iD6o7nIw1Yx4k0j07Jy+JMetCCBwuhWufmgAQ54JrLRSxz6gieh5SsMNCHuZzSO3ns+h/cbseEvrMr6b9LK8HttD/jESE3rC6NQ0UxTI45PqOeqjceQ0xf70iJNm+5B+qNFn0Tq31W1Wvz4Oq6Ctcgw4XjVkpfPPGj9WRnU3LoeP4attSJt11I0IIXJ2S4++jfTEGMfOkm2Zcb0ToawO1ZLgySHPplm9jqaR/RbylmaYJFLeCZiwRrys3Kvq0Ihy5HdOQEmoTcpK7EvzSn6+pAEPba5tDSUIfznLRe3Q+zy/fgCb0up7NQZWfSuMzXnZId8dZ3q69dPvtNjKGChGfSXGnaaPIA5g11B9SCalam6/PY05ytpOl8SJrAJwuRc8Tb/QjcVFUz+EFO0wX0JZqTY44i/7n8dHD7i3C2uXX/Nlf8WAlKn4SVn8Jf8qFipU4w20rfyaQOCy+IPEWfeuuG69TIb33/dy3/ETSejxBi8NaOIQSE7D6MCy/9Eb6HzFyh318c+WbDH15KPXB2AKYSJ3VF5e8yEWfXERtoJYcT050f8gia+AljV60oBYt6LB4uu6vd3kd5BRZL6M/9rKBFHbTXQLpA2PnL1Ljv+I1Suw9qmxMjlcc1i2XyZcP4tHL9etNJVYds9uWmnlXiUTh/Pdafd2ElKmjaPY0kYpi+mMVVbP+7lnh2Y26p5EFRhHXTqobpKPO75eUc2dXECahV/ehj/7nwLbof240FabrMeuyfDnzl+pC/aE6lm5iO8OUdZaH9S2wFrgd+ehvG3sV983UFz95XQ4Uly7CDu82WlyxCIBeX35B3QcfUvnUU1GLfvqm6VQHNiPdXvTkpDBzy0wG5g1EkxqralYRVIME1SDHdDuG5xfrr/PR2o/QpMaEkglxBbUXVCwg3ZVOrlefeJVSsmhabNI1kcRaq26vgzNvGcnWdXVIVfLpP2LRSi6PgwkX9qf/uE7c+8MaWjwhDgvErNjnM/100ATrnLH3aENVM+N66esYAEZ1y+Xmyfoy/Qzj7iGVJWuV92VPMv2WCWyv98cNNHuzMIWZ2paY8dASUglpbXfdtBaGuSNyitI45TfD8WW6ePPPs1OK+Z4SZMU0wayFtTYPZnuKi+4bS7Bl94q1txVb6H8uIis11SBU6ZE1s1aVUb21EdzwmnoM1zo+Snl4lsf6ByQDJos+wXWzedRRzCuK5RdJnONrMZzhTT6Bu0sXsk87jcqnniLnjDNZUb2CG6beAMXwXTNIeQZNoSau/vJqDu14KGEtzPzymL94xi9m0BzS704enKWnHl5etTw6CRuhvKWc4nR9JWvZ8hrKN8SqCoWQca6W3iMLmf/Fxlj/PQ4Uh0KPofnRPOURAlKS63bQpX8HMhd5+N4TjhP6Woek1hHvsV9brrvGGoxznTq8OCrskSpZvz6mD3e8n7zgrCirjfH8u0hRlpeiLC/rK2PzLZGVoqNO6M6cT9bvtdeubY5NPLcEdYve1caom92ly4AOBI2KTeZ8PWas8tvsCuYBI+RXfzbXTYSsvN1wxe0ktuvmZ0P/UjU0N9MU0H9I9VXbo6GV67ROzNH6pjw6sf7PvO3zuHfmvcigyUefYNEvvuSmuOcNoXifvNNYFbOoG7y+4nWuWvxHMmZ9QebRE9ncEL/Q6v0173Pbd7fp561YzMrq+JWx1319HY2h+DUA1f7qOIseYHXNarpndQdAVeP7u96p8Uh2C4/ktFB/Skd6jYivSGn+ISa6MYKmQh3ZPlebQhHXGSL68g/rAeKiRHLT3ax/8EQuMhV5Xnv/CXgNsd1R8fM9RcSiF0Kf+7ju2aMZc0rPHRy1e5hzGrXspOtmT+D2Obnu2aMZMqEkbnvkjmaPWfRmH/3POBm7L7At+p8L4zv07fItHK2FQcCajRvpKZppkD7KyeEFdQpFogZILvenoWGWtks+uwSAG7paZ4Ts8q9/EarTgDDLzvslkycMo6wlvnLX/O4+vjxE8MaRCg0/6YU6vtjwBef2P5fS+vjUv3f+cGf0cVALkufNo9k0v7CoclFce0UoVPurqWxJzpPfNasroYAaTTOQPjCH79dU8oM3BEL3T9f71bgQurGnx8fpC0Uw5pQe3PnVSvJUwUkeB02BMOkeJ+keh2VumQinDCumwR9irVGJ69vVeh/H9EyO+wZ48+rDmFVajUMRvHPtOL5Ytn2nI0p2FSWF++Kw03pS0t+6v3uSeuNuJ1X2yp+TM/9vFOvml+8Vi97qeXvCtuh/JjZU6aJYunkrPqFb0tlaPT3FVkplR0DQgpc7wpdHj6mTsVu7UMehcedzK3p4Y0uzdea/jLGH4Q9pZA64nWf7fUvGUUdR1hDvD9eKfuL54x00pMW+4E/Me4LDXjuM5xc9n3jK2HFSw+ts3aIdUTiC5dXLqUgYXACyqzrx3G++YcNiPaSy+xGd+DItRJPxbeyU7aXeH4r+oNOy3Ekl4gBGndCDZW6VGb4wD362gkF3fU6DP0QwrKGZfrOaKcjSqQiePHc4Q0py2FTdTCCssq6ikQsP60qnbOtb6TE987jhGD2J2eDO2dw0qbU7rz1LZFJ2RNf4ZG4jj+9OUfesvfKaXTrE3odGQ+gdP5PrpjXySzIYffKeu5tJjH5pz0JvW/Q/E1tqW+gGTFFmRbcd6VhEiajkK/UQy2M2yNgS701jr6Z6y0zGFo8FwOPwENSCBFqshX5T/aboJOO2gO5mmbE5ubJTKpp3EA1UF0gOzbxyyJVM6jaJukAdMzbPYM72OQAMzhvMkqol/KLfL5jcfTLqTx2ADaxbqA8Ch/XL59dH9+avU/W5iyyfi/qWUDRcM1WlInNVqk+X6HlSKhoCSXnzn82KuY9y0twIIehVkI4m4Q//XUy9P0zP/NYzDO4rstNcfHT9EfQu/Pn698F1R7CpuplTn/6eBmNx2M6ESk67eYJlGof9jcQ1Fj+3j/7npP1e2X5GxJnQS9ka3VYidJdBpMh3Io+qp0cfn/XJ+Vz9Zay2utuocORvTs6N8+EYwdkfn52UP35p5dKktlowdvt/XDerfPHWmEMoIxzb9VgG5g1kbPFY0l16aOQpvU6hOEOffB1ZNJJDO8bKrUVcN26vg/F9Y4NaptdJgz8czXMy5uQeSa9VXu/nqpfnJG2vaQ7iD2lxk6VNpm95RIAiK1jfna/PRfTvZP0Z7A8MKcnG9zOFVoK+RmBQsX63EFkFvDNhkz3y0/d6VNKeQCbk7m/PFr0t9Hubz29j+9S/U17TkLJJNdYi8420XvL/9YavqfLrbg95/9/i9t048XJePdpBU6iJlmB85aQWUz1TLRgJcYzd1D145IMsvmQxI4zi0qq/KGWfAUoy9MmyfF9+3H+AxqA+AB3a8VCyAwWcsuQGtIBR8SeS6934nSkOJTrZN6ZHB92i94dwe/VJub6jk9MrTF1hUXIROPOZmdT7Q3GLmdY/eCJ3nqSX8Iu4QnISFp8d2n3v+7sPJJzGZxJx3ezs4qcDAS0h587eLPyxr7FdN3sLTYOWapj5FEWAk5glu13msEDrzWSHbpHWSevbciGsY2x/O/23ALhCyekUGjN0Mc90Z9IUjLlfllYtpSZQw9Cco5i9RiBDOXg7vY9UY/H5LkUXv3P7XI5PFPL1QheOjh/hcXiY3H0yR3Q+gqqWKh6a/RAAN4+6mQ0NG5jUdRKfrPuUTC0neq7LB1+Oz+njxB4n4vmmO+saqula2QcpJVvX1ib1e2jnbK49qhdXju/Bo1+sor6l9fJrm2pSu5bmbKghN83FsZf1I8O4K4hYxJF8QeYImxuO7r3XVrYeyHicSjT0dHcWQu2vJFbjCjSHUrQ88Gl/n97+wnePwSOxSBGnKQ7kquDvqE7hronDQug1UwhlpinL4pzeujVSn6ULYMf0jvTtGBOzcz8+l4ZgA4M7dqZEnokW0i3YcG1sxauqSUorm/jlPxtx15yPFtAt6f65A7nviPuY0mMKFw68kDEd9bTBR5YcyeWDL6dLVheGlh7Nv275Plp/tSCtgF+P+DUuhwuvx2tcjmDbuvq42Pno++NQuHVKf/IzPGT5nHFJxKxosigMEqGiIYDX5aDfmI507pdrvG/6j7o4R59ozDTl/754bPdWX+tgxSz07nY4EGZ2iHcvRXIqtUfa36e3v7AiPlVwjogtfFknO9FE7EvmFtZfMHe+XpSjpEIyZoVG50rJmjdfZPIcjaMXaJw8Sxf9zybn8egZCrX/vJezjtZ93QIBOdOTzul1enn1yjG8ct5FhDb8llDdoXRpeJjG1X+kORhme70+cfnhwi1Rt44q4/t3/7BHeKbba7hM+VPWzNVdKbXlzSydsRkpJcGWMIunl0Vzisz5ZD012/T3YfJVg1O+dVleF8GwxgvflVJlkaYAYoucUuF1xvu0t9bq11Wco7/vZtdO5l4u+nCg4nE6ogNue3TdFHTNZOJF/aPPI1lS2yP2N3xvIeMtzh4iNgnbSBpBYiL5gWodC+/O1SN0bn9DpUMjLOkqUDc+yhUJ7dZ1daM6BM1dcwlt0ksWrqpZxaqa5Hh8r9NLcY6P4hwfF44Yy3vzNzO2e2eWlWm0BNW4dLRaoCNS9XLdIdfFneOrZ1dSs62Z0YeFkyrjfPnCUuor/XTsmc2i6WUsm7GFzDxdXJvqgqz8UY+O6T4kj/7jOllm88wyznnvx8vYUNXEn05NHhSajFJ3mR4nDcbjYSXZLCzTo4E8CWlsTx1ezL++L+XskcmrLVOlBT7Y8bhMFn07FHogLndSqpW47YH2+entJHd+sIRb/7toxw13Bi0+4iVPxLsrIvL2SOgcSqV1RsjJczX+/lSYDkZgzeCN1imOtxiFtZtDzVQ0J8etm/GZEpndftJA5t4xKRphceNbC+ISWqF5aFx1N0d0PiLuHJFomcRMkQD1lbrlHA5qBBp1a7ChKhbeWLOtCU+aE6fbwTEXD+CYSwYmnSPLNFGa5o4fSD5etIXTnv6epmCY4V1ymHfnpOi+D64/gmuP0t1liRFHfYoyWfqn4+maZ50zyCYZj1MxRd2008HQMDR6jyykc9/kwvPthYND6DUVNv4E/jr9g63WE4dVNwWpaQry8swNvDF70559TWntQz4rcGfccxENPWlBOOLzyI9ZKclPHawTZZtLP660rpSNDRtbbWu10CnNmKj8fk0VLQkx6I+ePSypfcRCtxL6COGQimJhBbY0hFKWh4uQZZoolVLSHAyzrqKRWaXVXP/afBZsqmV9VRMZHmfSJOqkgXqk0KyEtMI2O4/b2f4t+k69cxh7Ri8mXNBvx40PYA4O183KT+HNC2DgadD7GPjwBrjsU0Y8U0OeRTGPPYJFbviv1UOYI/tbNIaMvn9CCEnD8gej2zpVx1vw9T6oLs6g+9r42PlGw0iPZI9sDY8jWWTNrguz0LscgjNGdE5qHxHqik2N9B1t/TrBlnDcysP8LhlUbtL7nZG7A6E3VXmq94e56c2FfLY0vnD0puoWBnaKrQwdbYRHHtIlB4DDe+exIyK53m2sMVvx7XEyFvT8QSOOS1513d44OIS+yYi5btgKZbMBqN24FCimqmkvTcBYpAwOtfJ2i4SSd96AJK8BmjyQHoCf+gpePE6hV9d+rN24AEVCKN2Lo9mPthPxv1YWfb1pYrPBFO2S5XVZpoqNXNr6RZUcfmZvy9dZNK2MyrLYgHT670bw/G/1QuLHXprsrjFjrk9a7w8xe721dR7JNzPrtmOidwGKIphz+7HRu5TWmHP7sVYfk43B7hQRsdm/ODg+vUg8udNLJLvYltq2FfzYZSwUJGgS+tfVo9moFfCuOt7y8IhfflEPvb+eENRkCppEkJpMQVWWwJmeTqNF8eRMd+rQTUGycI/vHVvoFIlOgdTRKGoo5qOPRNEkUraiBr/hox93Rm/cXicjJnfj0JN64Mts/S6qb1EmQ4yaqvUtIYYbVvrIbvE+1ExD6AszvXF3JfkZniTfvhVZXhfZaW2vvHSwYRb69hhHfzBxcHx6Rp50XL5o2ZrmQOvheS2NQZ6+diqblu+8r/eEd0/gb57kkMlGU5KyS378lE++H8UWYiJ71NpzOatRF8F0Q2/XF+r9bTIMcfPqVoA7x95Jv/IxnDf/doTUP84umamjB1SLuYPcdDdPna/n23lq2prodqsJuIVfb2L94qqo++Wt+2fzzkNzopOwifQ8pIBDjusK6BkoR5+UnM4gEZ/bwUc3HMGRfQto8IcJqhrDu+Rw5oj4tLU/VwbJgxXzPaZt0R/YHByfXqRYtogJV3OwdaGPLOqZ/2VscrOyrBE1tON7/U0Nm3jOC3UJLpVGfGRqgjQNjti6mKM2LyTXH8sZM6B8LD3Ceh/T/PrPbGk3wd9PVHjhOIXc5o4ETKUDpZTkenKZuPZ8sv0FuMP6aFCUVkRWSz6eUCzCxKnooqil8FWYJ0AjuJyCrWvrqN4Ss9q/e3s1oE/IFnbPIhzU2F5qnVhtxPHdGDWlu/Wb1AayvPrCqaZAmAyPM6nYR2Txk83eYUttzKiwLfoDm4Pj04sIvcma3ZHQR1zT0siHUbOtiTf/PItZH5e2clQ8Z3bWwyZrpZ7gq0l6ubbey3X1MYF67bM/WR6bZqwTavLC9KEKYVcGv1j4BwYvPzbaRiLJ9mRHn1/WX4+wV6SD8xfcwWlLfhvbN+gyAHrnWPvUO1sUnk7XBO8+MpfX7/0paZ+mSgYfmTxRG6GwWyZjT+sVrf+5K2T7XNQ1h2gKqKR7HEnFPnoWpO/yuW12zBaTG8+26A9sDo5734jrRmpEfPQtgTBFWR6GluTEpbsF0FSNH9/XQzDLVtSw/IetLPlWz3JYV7Fj336mvwMDyscyu8snzPN4yG0qYGnzWTR6NaxSZzkzl3DoypilfcrcUTSJuUgkPWtPpDbze5ya7tIpqOkOhvdDkxpp4SxAdy8NKTuK98eexN/nPANArr+IjEAu9+U8w1FDB3DJoEvwael8985qxpzSM65KU6+CDC48rCuvzdzIUX4XXg1E5HduEb4fDml40mJfn3P+eGg07BKpF/LeXQozvVQ1BalqCjK4czaFmfEWfe+C/TO1cHuh0eTebK9RNwcLB8enF7Xotaip3hRSyU1zW9b+3Li0moqNsQD2qS8vj046mqvbaKq1G+TIdb9gxObjyG/qwiXFRSxrOokFzach/NYFK3wlr3Lh1Ni5ioMXkdnooT6rG4MqJjNxzYW41Fg/FaP+rETi9scs8cXTN/PZYyuYknVadNukVZeyfOp2Ni6rJtuTzaJpZSz8ahOLLYpyd+uQTpewwqiAk8EhJ4M3xuYZZEKmPy2s4TaFQeZ1Tseb7tL/Mlx7pAqQ+bPxuhTyMzycNLQTNx7blzMO6UxB5t6t23qw88Ilo6KPd6fot82+5yC06HVaAmFyM92WWQsT05cCDD+2K2XLq1k1azurZsXuADpkt9DUoDL+olF89dIKrur3RxyaXiUqI5BD163rCa1xQWdwB4stu/fWA2HCDi/LTdvO+TFMY7reN5fmxq3GBN3n9NEUagIN/vfwchJZ+XGsKEhmQL+H+OTvi+g7uijqSpn53lpmvreWc+8YTV5n3TLO8jnjvhDmfN1//9W0uNdQwxoek9DvjaINZldNTXMQRRE8df6IPf46NtYcM6AoLsWEzYHLwTFMmy16gwZ/iMIsj+UtaWJe6m6D8xg6sYS07GQLsrrOR0DLYMabKwCYUzaKjg26b6Vn1XCOWZhJs09PNOZVrUu/tXg6UG8UzI7QnFYUWzULeMI+438aRc36Ag9F27F7xBOOTciumrU9KS5+41Ld7VNf2YIobaJHqG0uFzUs41w3ewOzxb693jq5mc3e5YubjuTVK8bs627Y7CYHh0UfEXpNJeKjb/CHKMryWlr0iWuERp/cA2+6iw7F6aydF9vuVIKEDd+5FmgBfMxvOiM6evapGkljYRfcYX2Lw1TkQxMKijHwzBx7b1IfWnz5SK02+jxi0XtUH8fNuYZnD/sNTnXHrgvza0Jyzu2IL/27t1dTsbCSQ9r4lVATXDd7g86mqJoThljnA7LZu3TK9qWspWtz4NCmX6oQ4njgScAB/FNK+aBFmwnAE4ALqJRSHtXWY/c6UdeNjKp4WJN0zvRY5zVP8NxEMjQeOqUrA8Ov4KmcA5tm4iREdbgLb1Q9SUha/xg0VyFBR3JM/YLB15Bfs4qqDgMsj3vq6N60ZC/mhJV6FM3EtefH7e9aO5BBFXrWy+GTurLAFAYK4M1wRRcsmane0oTDpUTDRAPNIVb+tI3ShZXRNmEH9BqQx4YlVZZ9g2Qf/d4gN93NinuPRwh7MtDGZnfY4a9HCOEAngamAAOB84QQAxPa5AB/B06RUg4Czm7rsT8LISN8xOS6EUg6Znu59PDuAJi9NWpC/pOI1Stq15Ox4FFcZd/gEkGEkOS5NlLkWhnXfltGKTO7fhB9rinJMeq1eYNZ0/sMalIIvQsnAv1uIbc5uaTfCSuuoVuVnr43Oz85rUFWXmzbuDN6483Q+1BR1ogvw8VR5+kTwy2NIb7617K4YzMz3OR30f32Vu4Zb4aLU357CIoiGD6pK6f/zrq4+Z7A63LgcTosUzHY2Ni0jbaYSaOBNVLKdVLKIPAGcGpCm/OBd6WUGwGklOU7cezeRzMsW1N4JUBBhodsn4tLxnYj0+uiZlsTT187lc+eWxJ3uCtS0T5gnUrypNw/xz3/ot+LLOw8lW97vBXbGNKPraVtVWyO2DaOvhV6+UGnbD1lgHnuIDLZmpUfu8PoOqgDp9+kT2LWV7Tg9jkZfFQJGbkeykwrfwu76cf2HlFI14F6UjBzvu4IF/7pMIq66/MNh5/Zm+I+7Te9q41Ne6At996dAXMO3zIgcXamL+ASQkwHMoEnpZQvt/FYAIQQVwNXA3Tt2rUtfW87qrGaNM6ij2VtFEKgScnWtXUWB5uE3m+936s0ckaHP3BW1zQygx1odusrRbdkr462yWqpot6VSVOwghx3LPrGGWok7LKOB+9VPTzu+YacpXSrHZTUzpz2N2K553VOZ81c/UKzC31xK3ojVro3wxXNKAlQ3CeHUSf2oOvADjicCiddPwyX18F7f5mHw6Vw3BWD8GW68dj5YWxsDijaYtFb3TMnxh86gZHAicBk4A4hRN82HqtvlPI5KeUoKeWogoKCNnRrJ1CN8DCpRn30AoliPHYowjKkMkLUbdCSOu9NR/cK6n2VbM6OVXWq9ZVDeAuKGsQTqAWgSYRJb9oSbZNbuzrxVJaERZBN3RZa7ks3JTaLxLvnFKXj8jjIyvPidDni/OmRhU1pWfF3Cqoq6TE0PxoD321wHt50XdQzcjz0HF5Ap17Z2NjYHFi0RejLAHOWrBJgi0Wbz6SUTVLKSuBbYFgbj937WFj0EIuucSgCVcqkaJs4Vn8JH/4m5e5UGXBE3YOM//7/cIV0yzkoYPTs+5g4/XoGzr6bjEZ9xe3CTtP4ss+/U57/y77/ptqtx+83pVdz8f3jovs86ckWdnqOB5fXQU6RnibA7OM+1qjqlF0Y75axuv7Itj2x0tXGxmbf0BbXzWygjxCiB7AZOBfdJ2/mA+ApIYQTcKO7Zx4HVrTh2L2PGiQsXThNPnqzRa8IkVj5L8r4X/TVJ3P/c5Z1gxEXw7yXsawnJSVOVcOhqfQs/RghNb7JzzR6IAkoCsIYfDShkuJmB4AGTw2NrioWFH9NS89tZOSeybCjuyCROF0KY07pQXGfXNKy3Sz5ZjNFPbIYeXy3ODGfcEE/3D5ndJ2Az3DzDBpfDEIw6oTuSa+bU5jG8GO7MKiVvDY2Njb7NzsUeillWAhxPfA5eojki1LKpUKIa439z0oplwshPgMWoRu3/5RSLgGwOnYvXYs1msbypqOYWn8DF3Z4HLPjIVLfwqGAZlGkGmDoKCfclxz1EuWkJ2Dey4QtzOFJ8yXjl+nndYca6Lf6TRy5l0b3tzjddGjWrfQ6bwXWni6dZncdQRngx24f0iujF0IIjjinT3T/qBNi6X+POFvfPnRifLriQePjxTq3o27tF3bPYuDh1qt2hSI4/Kw+lvtsbGwODNoUCC2l/AT4JGHbswnPHwEeacuxPytaiLWBsQDU+PPJjljxCRa9mkLoqW2lluzkB0DRXRpWFv245cm3CS4tlma4xS0o2L6A4tLHWHHYBnpXjkz5Un5nLFWwVU75XaHXiAJO/90hdOqds0fOZ2Njs3/S/lehqKZSgVrMdeNAjcbOK0IgZbQgfJTsAp+14zpC51jeFdWinbAYO3zpsdBNkbYJATS51oGAbVl6xsxaVxO5HWMul4CnCQRcMOACQE9mticQQlDcJ9eOUbexaee0W6HfVlrHe4/OQw3EcqSYhdyBJCr6isCrwbRXVsSd48J7x7b+Io7YJGhbbWw3tdHH3pDeoc1GHetGTw3Pjv0NL/f4jPPvPizabukJ77P4ksWc1vs0IHXxEBsbGxsr2q3QT3tlBVtW11K7rcmUHCwmkApa1KJ3KIJ+KZN5tWLtOvTwRKlBKBj/Vvr8EpdF0r+C0OZYH4cqfHCY4O0jEj4GoR/Y/zIf03r9J2rBu4wVtrbQ29jY7AztVujDQd3GdjpMamvKdeNEjfrohWgl3sUsqiWj4/cZQr99XjY17xTiCsfO8tLjKn22Jp/u7B9itn/YAf+Z6CDgThhMhN4mq5uTlYWzosLuVoyBJdV8go2NjY0F7VboQ0FDoGWC0BsiqQgttmBKiCSh/8XthqibffyOhFQEDhc1b75FzRo9esUbZKdwpDDMhYi/FYgIe7Tua8qofRsbG5tk2q3Qh42slDJsFnotaqE70OIWTCUKfVYkUVic0McHKaktIbbddVf0uc+YDhBttLgdKRz7wRrdPx+pJBUhMmlqu25sbGx2hvYr9MGI0IeIOWZktEC4Aw3FcNKrS2o5piV+dWm0FJ5mGigSLPpg2ba45w/+W2X8Yg1PGy17K4s+3NgHza/Hv3sceg6bDLeeCydi0ZsLgtvY2NjsiHZbeCRiVDfVBaK54qWURvERQ+gNiz68sBZ3wqSr4jCet+K6qVwdnwM+ww83fKzx+0vbli7ASuhHdM3jsol6rc5BeYO4ZdQtnNzrZADyffncPuZ2jupyVJvOb2NjYwPt2KKP8MErTWwODgFgqyDeom8lfjwaW24WesUJ3cdHn77w9syk41rc8NC/k30y1RYJKjdZ5G7rkObl2IFF0T5cPOhicr2xNMC/6P8LOqZ3TNlvGxsbm0TavdCbqUFEffSKyUffKmqC6+aCt6NPc1uS89P7UrhtfnNNvJX/33GCJd2T336HYicPs7Gx2bMcVEIvpCCSvSwSXqmqO5jYTHTduGIFPXL9DfgdbcvNbg6h3J4Dn42yfusTJ2BtbGxsdpd2qSqbllvnjVekSHLdWNVVjUOL7ZdSYe2UE6jb0oG69T4OLV9JWcbO586/4ZdO6tJF9JxmHMK26G1sbPYs7XIytnpLk+V2qSnRydjIytjKskbLtlHUmNCH6lWCpaVsKfUCevhletjf5n7dc55CsyfBX6R5wNESfWpb9DY2Nnuadin0mmodx65KkRBHL5j9cWnrJzO5bsKNyTkNvi0ZyC9Wftemfi218MlroRwcJqGPhFDa2BwMhEIhysrK8PvbbjAd7Hi9XkpKSnC52l7Ss12qipaiikhYOmKuG6FPxqphDWcHN+HqhFnUhW/AoDPg89uim0IN8W6eqSUjeGXQBD45YSYODe59RaVzNTxyhoI3BDd8lNyPprU3kd7rsehz1d8Z4WhEcTUY/bJdNzYHD2VlZWRmZtK9e3c7i2obkFJSVVVFWVkZPXr02PEBBu3ST5DSokdJiKMXBP0qbqOGKkAAySHp78F718CS/2LOghOu0weDxw45h7VZxXzR7VA0h0qLV9CYJqJpibd2EJTnWH9ptWBhwgYXqj9WIOSqoVft7OXa2Byw+P1+8vLybJFvI0II8vLydvoOqH1a9CmEPiwVkBpnFnfEFapkkoCQP4zT5Ym2GVRwP+Mcc/QnG2IumaoV6ZQvmIfw+fjmkHp+PL6chuV9UER5tM32XEFxjaTJ22rOy3ikExnWg+xvH3M7XTK77OAAG5v2hS3yO8euvF/tVOitXTcqutCv8rjBHWDb6lpCfhW3O3ZjMzki8kB4+Q+odU482WH8Nbo/LOu+B/FU/E5voASiKYUB/nqKwsBNkppMgaLFDzZN638Jqo9EpHRBWD93lb9q1y7YxsbGphXapetGTTkZq6AZC6B6VA/l078tIhzScHoc1CgawYTUZmtebWbdp7qrRQspeLrm4Rp/FFLTx0fhrI8T+iafYHZf/S1t8cSdCq2lW9Rto4Vj1aPUxr54goMAGF44fBev2MbGZle5/PLLKSwsZPDgwTt9bEaGfje+ZcsWzjrrrF3uwxNPPEFzc/MuH78j2qXQt+ajr5G6n10xxa87PAovZAb4a3a830uGY23UkMDhCKNJidR0y1xx1iMUi+oi6KkQUtG0+jYalt9Hw4o/o7Z0Z8GtVzH3wrmMKx7XpuuzsbHZc1x66aV89tlnu3WO4uJi3nnnnV0+fm8LfTt13aSy6B1UWAm924Fsxe2lhUFzdMBV0o+gqiJVLzgbEK5qZNg6k6RUWvOjGZE1RjcdisBBKyODjc1BwD0fLWXZlvo9es6BxVncdfKgVtsceeSRrF+/vtU227dv59prr2XdOr2u8zPPPMO4cTHDbP369Zx00kksWbIEVVW59dZbmT59OoFAgOuuu45rrrmG6dOnc/fdd5Ofn8+SJUsYOXIkr776Kn/729/YsmULEydOJD8/n6+++oorrriCOXPmIITg8ssv58Ybb9yt96GdCn0qH72DCi0ECjjVmLA6Xa2HNKpBBVVk4crryAkfHoaRPRhf8X9p3nTxHuu3jY3N/smvf/1rjjrqKN577z1UVaWxMfVCyxdeeIHs7Gxmz55NIBDg8MMP57jjjgNg/vz5LF26lOLiYg4//HC+//57fv3rX/PYY48xbdo08vPzmTt3Lps3b2bJkiUA1NbW7nb/26nQW1v0Gg5apB4L79FiQq8sXESv2irW5nSObvsunIZRs5tKeQHhrV8TSku2uhVn6g988cOXMOT/XtqFK7CxOfjYkeW9L5k6dSovv/wyAA6Hg+zs1DUhvvjiCxYtWhR15dTV1bF69WrcbjejR4+mpKQEgOHDh7N+/XqOOOKIuON79uzJunXruOGGGzjxxBOjg8TucFD56DXpIGgUEnGpsVVlhR8+y22zXo4+n+vxkPdOTvR57UdfAxBOT5hhBYTLOq8OwOTJv9ypftvY2Bz4SCn529/+xoIFC1iwYAGlpaVRsfZ4YhricDgIh5Pn+HJzc1m4cCETJkzg6aef5sorr9ztPrVToddihUNMqCgEjZWxLs0FAs6fXE9mYxnZwZhlXhO2duUEvbEbIKnqi6wUt7XQf37m53YlKBubdsIxxxzDM888A4CqqtTXp55LmDx5Ms888wyhkO49WLVqFU1N1vm3ImRmZtLQoK+Or6ysRNM0zjzzTO69917mzZu32/1vp0Ivcbgs8srgICg1XCGJS3XjdCmoFRUA1LnTo+2URuu3paFTVvRxROgdni2WbVVjBW7RHbfT5Z//3LULsbGx2eucd955jB07lpUrV1JSUsILL7yQ1ObJJ59k2rRpDBkyhJEjR7J06dKU57vyyisZOHAgI0aMYPDgwVxzzTWWlruZq6++milTpjBx4kQ2b97MhAkTGD58OJdeeikPPPDAbl+jkG0sZP1zMmrUKDlnzpwdN0zBx08tZPv6+qQUxIGc6fQJv0fvj8O8evK59JRHc2LOt9S8+ipl6flcNelW1nvPZ1p9Jh0/yQTg6z7dOEF4CKxaxfm3OAg7jTqz/k44vFtT9mHq2VMpSIulMO5+6/9Stl3/4Im7fK02Ngcyy5cvZ8CAAfu6GwccVu+bEGKulHKUVft2atFrOC0s+vQ6J7nLdUs7w+/GgUrjN98A4DIscH+Nk+z5sdw3insbC24/jTduOywq8jqSYM3o6LPmjZdHH/9h9B/iRN7GxsZmX9I+hV6TOJzJl5a/1UXeWv0ORgoXcvsWQps2AeAyJmm3zc3BtzU2UeskyF3LH+NdLf4OQyghQnUjo8/Vpt50y9SzyQ0rHLZnL8jGxsZmN2h3Qr91bR2bV9YiLBYsaaYUwIp049BiqYkjQq+F4o9TUlQalNKBFjBb7QrSWAHlFO0yatXGxuYApd0p0tSXlwPQWBtI2icVs9C7UExFRZyG60Yo8XMWDguhD5RPJlQ/BLQ0AuXHE27sq58/IvQ7UTzknxdbutRsbGxs9hjtzqJ3e3Uxt1odK00WvVPLxKkag0GXrlGLXmrxFn1E6AfnDaZ3Tm8AglUTkaF84/EEtEAxAMIQ+p0pB3jswKI2t7WxsbHZFdqd0LuiQp8cTaQZlrbfk4NHKyanfi3pR45HOXYyTqkhpIYWthZ6IQSvTHmFP4141fJ1P/3N+KhFb9d9tbGx2Z9okyIJIY4XQqwUQqwRQtxqsX+CEKJOCLHA+LvTtG+9EGKxsX3XYybbiMtjuE1MOl/j0evCRoS+Mm8oAPkVC0gbdSiKW09t4NTUOKEPK/DhmNjzDHcGue5OSa+Zk+aiX1EmkVBVsYOyIxP76b79k4cV78yl2djY7GE2bdrExIkTGTBgAIMGDeLJJ5/cqeMPlDTFO3QmCyEcwNPAJKAMmC2E+FBKuSyh6Qwp5UkpTjNRSlm5e11tGy5P8qrWaT0eZ/TW6xBqROgHo2nbSW/ejpKehhLSBdqVIPTn/z729jQ2e7jwnz9x2eHd41/PIVhwp768OWLR76gCzL8uG93qfhsbm58Hp9PJo48+yogRI2hoaGDkyJFMmjSJgQMH7tR59kSa4gsvvJC0tLQdN94F2jJrOBpYI6VcByCEeAM4FUgU+v2CiOvGTMgJqghTn+5j9miVkChCCelWvpKejmjSJ2VzAg1IVaGii8q/RsdCLIfkD2HmD8cj1UomteJTb6tFb2NjY8Gnt8K2xXv2nB2HwJQHU+7u1KkTnTrpd+mZmZkMGDCAzZs3Jwn9wZCmuDOwyfS8DBhj0W6sEGIhsAW4WUoZWSMsgS+EEBL4h5TyOasXEUJcDVwN0LVr1zZ2Pxkriz7sAFUJI4WD78doDF+YjbupFtCFXvHoeShe+OohAMpLNOb0jXm1bhj+G36Yod+Q3PVh6qXP40vG8+bKN8l0Z+5y/21sbPYN69evZ/78+YwZkyxvB0OaYivzNHGmcx7QTUrZKIQ4AXgf6GPsO1xKuUUIUQh8KYRYIaX8NumE+gDwHOgpENp6AYkoFvHzqgJSqAjNiSoz0BQXPn8dAI70dBSXK659wBX/8pnuHGDHnqffj/49Vw650jKZ2fw7JnHIvV+2/UJsbA42WrG89zaNjY2ceeaZPPHEE2RlZSXtPxjSFJcBXUzPS9Ct9ihSynopZaPx+BPAJYTIN55vMf6XA++hu4L2GtIoyn3U+f2i21QFcGgo0okI5wCQ3lwLgPD5UDzx6YeVQPxgkelK/uCtcCkuOqZ3tNyXm25XkLKx2R8JhUKceeaZXHDBBZxxxhm7fb4DNU3xbKCPEKKHEMINnAt8aG4ghOgojBlIIcRo47xVQoh0IUSmsT0dOA5Ystu9bgWp6bH0g4+MFREJO0BRJEI6UQyhz2jW04wqXi+KO96iX+mOF/50p51u2MamPSKl5IorrmDAgAHcdNNNKdu1+zTFUsowcD3wObAceEtKuVQIca0Q4lqj2VnAEsNH/1fgXKnPTBYB3xnbZwH/k1LuXhXeHaBpMin9gaqAwylQNCcNmm6dZzTrb6rweHAkCLs5pBLAYac0sLFpl3z//fe88sorTJ06leHDhzN8+HA++eSTpHYHepriNimY4Y75JGHbs6bHTwFPWRy3DvhZM3xJC6FHCBxOBYd0okk977wv4Nd3uT0onphb5aPRgpAr/viwlnrK4IxDSvZQz21sbH5ujjjiCNqSqr2oqIgPPvggaXtkUrZ79+7RyVNFUbj//vu5//7749pOmDCBCRMmRJ8/9VRMMm+44QZuuOGG6PM9YcWbaXdLODUpLSdkHS4ninTiNgqGOMP64gTF44ZALC9Osyf+2IaV96Cl+CJk+1zcd/rgPdV1Gxsbm71CuxN6qVpY9ICqSBzSgSeUBloIh5HbRrjdpI0cEW0XdCUcqHlQU1j0PpcDp6PdvYU2NjbtjHanUrrrJnl7i9QteF84E0W2RLcLjwdXTg7vjtMHB3co+dhUQt+aS8fGxsZmf6HdzTKmct0069Gf5DeVgIzllCjzb2dl2Xe0uAUg8QV18S72u1i9TQ9rSi30KZLVp+C9X40jw9Pu3nIbG5v9nHanOomum4x6PT99j1y9+lNeczFS6OkPwgqc+IGenqdnd/2Yxd30//khJytbuuntUgj9Lw7tYrk9FYd0zd2p9jY2NjZ7gnYn9JoWWx07NPAEwdI1gIMBHfpThpFzXugRN+YaI+s6CS64xUGmSLbS//DuoqRtC+86jixvu3v7bGxs2iHtz0cvYxa9DAXQDDUPmdY3uBy6GyehmBQhp+C4pmamNDaxetsl0e2z19ckvY7XpewwS6WNjc3+jd/vZ/To0QwbNoxBgwZx11137dTxB0qa4vYn9JpZ6EOEjRxninlNlLss5fFuKcnfcizl4e6tvo7DFnkbmwMej8fD1KlTWbhwIQsWLOCzzz7jxx9/3Onz7Ik0xfs0H/2BhqbFJmO1cFjPcwN0PNzF8H4DAPj4nYeSjuvT4mK1L8Tn6khK1VRp9WM4LCZ8bWxsdp2HZj3EiuoVe/Sc/Tv05/ejf59yvxAiapWHQiFCoZDlnfrBkKb4gEJqkujnFA5HLfreHXpRWKJXdqr9PDmV8eAmL6t9ITZRgEry/kRst42NTftAVVVGjhzJmjVruO666w7aNMUHFHGum3AYl5AsPuKvkFYYbVOTbiXSrYv7C5eM4pgBRXS/9X97srs2NjYGrVneexOHw8GCBQuora3l9NNPZ8mSJQweHL/i/WBIU3xAoWkSxWEIeVhDKoASP575HbFZ2AfKK7m/vJIRjZkEa0YTqDjW8rxuZ7t7q2xsbEzk5OQwYcIEPvts9/IuHqhpig8opGZyq6iqHlrjiM9rkKV5o49Pamrm5KZmFNwEtp0BmnXNRpeR6qBHfvre6biNjc3PTkVFRdQ10tLSwldffUX//v2T2h3oaYrbpetGMaxvEdbATZJFP7ylmOYzf2BoMBjdpu5gzHMZdwnv/Woc2+r9e7bTNjY2+4StW7dyySWXoKoqmqZxzjnncNJJycEYTz75JFdffTUvvPACDoeDZ555hrFjx1qe88orr2T9+vWMGDECKSUFBQW8//77rfYjkqa4U6dOPPHEE1x22WVoxsr7PZGmWLQlRefPzahRo+ScOXN26dh3HpqD2+fklF8PZ/r4ITRlBTnx+TeheHi0zX/+8hsuaPx33HFvi8nc0nIJqfjw+sMZWpKzS32ysbGxZvny5QwYMGBfd+OAw+p9E0LMlVKOsmrfDl03Muq6EaqGosgki15alMEN72Ay1mVnqbSxsTlAaXfqpcfRg9rYiCsocYhkoVelRRrjHQq9HU5pY2NzYNIOffQgFMGqUYeSC/gtJmM1i/HNtuhtbGzaK+1OvcwrYwEcbg2UeBFXLVw3qrCF3sbGpn3S7tQrsWasy62CEm/Rh7Vkobfy25tx2q4bGxubA5T2KfQmTfZ6NHD54toELeuFtC7kbtuit7GxOUBpd+olpUTImJKnuVRweuPaBNXk4wSth5natWFtbNovqqpyyCGHWMbQt4adpngfoalSn5E1yLQQek2zUPodYEfd2Ni0X5588sndiue30xT/zEhNImRMyLMdCijx45mmqhaemtaF3KW0uzHRxma/Ytv99xNYvmfTFHsG9KfjH//YapuysjL+97//cdttt/HYY49ZtrHTFO9naBKEqWh3Wp4ruY0WTkpWqYjWXTdWBcdtbGwOfH7729/y8MMPR3PNWGGnKd7PkGrMon/zrHTu9sW7aVRNgqbuKCuxjY3Nz8yOLO+9wccff0xhYSEjR45k+vTpKdvZaYr3M6SUCMMHn+5yJPnn/SEVh8XEq22v29gcfHz//fd8+OGHdO/enXPPPZepU6dy4YUX7tY57TTFexmpScJBLRp1k+lwJIVW+kMqCpbxlTY2NgcZDzzwAGVlZaxfv5433niDo48+mldffTWp3YGeprhdCX1jbQA1rJHm09/gLKcApyeujT+s4bCF3sbGZid48sknmTZtGkOGDGHkyJEsXbo0Zdsrr7ySgQMHMmLECAYPHsw111xjabmbiaQpnjhxIps3b2bChAkMHz6cSy+91E5TnMimZdV8+NcFHDamhrSHbqf0ig6c0CEDrvg82mZdRSMfPXkDv3G+G3fsS44zuavpzKRzds9LY3yfAu49bXDSPhsbm93DTlO8a+xsmuJ2NRlbW67HoWqyHIBcJLgSffSapesmVa3vM0aU8Otj+uzZjtrY2Nj8jLQrof925gfAADZ9+ToDgA41a9iaUcDMeWXRNmsrGkm3EPpUScvCqu3msbGxObBpV0LfsHETWVpvBsyvIOCEnkqIrVvXcNNbC+Pa3epMFu816SOgLvmcIW3/c23Z2NjY7AztSuj7evpS0RCk6W93kpHuJeuLS8mijG9umQBAMKwx6fFvkydjb9vOpteXwJbtTOhXwPSVFdFdtkVvY2NzoNMmoRdCHA88ib7M6J9SygcT9k8APgBKjU3vSin/1JZj9yRuZwYOLcioSefpG5oXQckouuWlR9v89tg+dF3oAfMiOJeXbJ++gvbIPgV4nQ5Kcn3887tSQqpt0dvY2BzY7FDohRAO4GlgElAGzBZCfCilXJbQdIaU8qRdPHaPEA5LnNIUxnTsXUltfntsX0gbBV+8F7c9J00X+qCq8exFI3nph/X6OTXborexsTmwaUsc/WhgjZRynZQyCLwBnNrG8+/OsTtNOAwOWo9XBeCwX8J5b8RtyklzA1DXosfgRwqNhG2L3samXfPkk08yePBgBg0axBNPPLFTx7anNMWdgU2m52XGtkTGCiEWCiE+FUIM2sljEUJcLYSYI4SYU1FRYdVkh4RVcIo2pCBWHNBvStymif0KAd11A7EoHNt1Y2PTflmyZAnPP/88s2bNYuHChXz88cesXr16p8/THtIUW0WYJ6rfPKCblLJRCHEC8D7Qp43H6hulfA54DvQFU23oVxJhVcGt7HyueYCBxVmsf/DE6PNI/nnbdWNj8/Mw461VVG5KnRVyV8jvksH4c/qm3L98+XIOO+ww0tLSAKIZKv/v//4vrt3BkKa4DOhiel4CbDE3kFLWmx5/IoT4uxAivy3H7knC0kGashNjxOn/AMX6LXAa+efDdniljU27ZfDgwdx2221UVVXh8/n45JNPGDUqeXHpwZCmeDbQRwjRA9gMnAucb24ghOgIbJdSSiHEaHSXUBVQu6Nj9yQqDpw7k3542LkpdzmViI/etuhtbH4OWrO89xYDBgzg97//PZMmTSIjI4Nhw4bhdCbLYrtPUyylDAPXA58Dy4G3pJRLhRDXCiGuNZqdBSwRQiwE/gqcK3Usj93tXqdAxYnFZ7RLOG0fvY3NQcEVV1zBvHnz+Pbbb+nQoQN9+uxeypMDNk2xlPITKWVfKWUvKeV9xrZnpZTPGo+fklIOklIOk1IeJqX8obVj9xaqcOJy7ZnM8sNK9BH7/NFd98j5bGxs9k/Ky/XcWBs3buTdd9/lvPPOS2pzoKcpblcrYyfOu4PsM87YI+cqzPLGTc7a2Ni0T84880yqqqpwuVw8/fTT5ObmJrV58sknufrqq3nhhRdwOBw888wzjB071vJ8V155JevXr2fEiBFIKSkoKOD9999vtQ+RNMWdOnXiiSee4LLLLkMzAkHsNMUJbL7l/8gYfwTZp5yyF3plY2Ozp7HTFO8aB3Wa4s6PPLyvu2BjY2Oz39GuKkzZ2NjY2CRjC72Njc0+ZX90H+/P7Mr7ZQu9jY3NPsPr9VJVVWWLfRuRUlJVVYXX691xYxPtykdvY2NzYFFSUkJZWRm7mt/qYMTr9UYXXbUVW+htbGz2GS6Xix49euzrbrR7bNeNjY2NTTvHFnobGxubdo4t9DY2NjbtnP1yZawQogLYsIuH5wOVe7A7+5L2ci3t5TrAvpb9Ffta9JogBVY79kuh3x2EEHNSLQM+0Ggv19JergPsa9lfsa+ldWzXjY2NjU07xxZ6Gxsbm3ZOexT65/Z1B/Yg7eVa2st1gH0t+yv2tbRCu/PR29jY2NjE0x4tehsbGxsbE7bQ29jY2LRz2o3QCyGOF0KsFEKsEULcuq/7syOEEC8KIcqFEEtM2zoIIb4UQqw2/uea9v3BuLaVQojJ+6bX1gghugghpgkhlgshlgohfmNsP6CuRwjhFULMEkIsNK7jHmP7AXUdZoQQDiHEfCHEx8bzA/JahBDrhRCLhRALhBBzjG0H6rXkCCHeEUKsMH4zY/f6tUgpD/g/wAGsBXoCbmAhMHBf92sHfT4SGAEsMW17GLjVeHwr8JDxeKBxTR6gh3Gtjn19DaZ+dwJGGI8zgVVGnw+o6wEEkGE8dgE/AYcdaNeRcE03Aa8BHx/g37H1QH7CtgP1Wl4CrjQeu4GcvX0t7cWiHw2skVKuk1IGgTeAU/dxn1pFSvktUJ2w+VT0LwHG/9NM29+QUgaklKXAGvRr3i+QUm6VUs4zHjcAy4HOHGDXI3Uajacu409ygF1HBCFECXAi8E/T5gPyWlJwwF2LECIL3ch7AUBKGZRS1rKXr6W9CH1nYJPpeZmx7UCjSEq5FXTxBAqN7QfM9QkhugOHoFvDB9z1GK6OBUA58KWU8oC8DoMngP8DNNO2A/VaJPCFEGKuEOJqY9uBeC09gQrgX4ZL7Z9CiHT28rW0F6EXFtvaU9zoAXF9QogM4L/Ab6WU9a01tdi2X1yPlFKVUg4HSoDRQojBrTTfb69DCHESUC6lnNvWQyy27RfXYnC4lHIEMAW4TghxZCtt9+drcaK7bJ+RUh4CNKG7alKxR66lvQh9GdDF9LwE2LKP+rI7bBdCdAIw/pcb2/f76xNCuNBF/j9SyneNzQfs9Ri309OB4zkwr+Nw4BQhxHp0V+bRQohXOTCvBSnlFuN/OfAeuvviQLyWMqDMuFMEeAdd+PfqtbQXoZ8N9BFC9BBCuIFzgQ/3cZ92hQ+BS4zHlwAfmLafK4TwCCF6AH2AWfugf5YIIQS6z3G5lPIx064D6nqEEAVCiBzjsQ84FljBAXYdAFLKP0gpS6SU3dF/D1OllBdyAF6LECJdCJEZeQwcByzhALwWKeU2YJMQop+x6RhgGXv7Wvb1DPQenMk+AT3aYy1w277uTxv6+zqwFQihj9pXAHnA18Bq438HU/vbjGtbCUzZ1/1PuJYj0G8nFwELjL8TDrTrAYYC843rWALcaWw/oK7D4romEIu6OeCuBd2vvdD4Wxr5fR+I12L0bTgwx/ievQ/k7u1rsVMg2NjY2LRz2ovrxsbGxsYmBbbQ29jY2LRzbKG3sbGxaefYQm9jY2PTzrGF3sbGxqadYwu9jY2NTTvHFnobGxubds7/AymJkcXD0P/TAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(np.arange(600), history1['test-acc'], label='1 clients')\n",
    "plt.plot(np.arange(600), history2['test-acc'], label='2 clients')\n",
    "plt.plot(np.arange(600), history3['test-acc'], label='3 clients')\n",
    "plt.plot(np.arange(600), history4['test-acc'], label='4 clients')\n",
    "plt.plot(np.arange(600), history5['test-acc'], label='9 clients')\n",
    "plt.legend()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "1579d4a348c2ef16482c05d3cfac916f73c8945ddf1938a1e045b3bdea82eece"
  },
  "kernelspec": {
   "display_name": "Python 3.9.7 ('VFL')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}