Baseline_code.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Baseline_code.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python2",
      "display_name": "Python 2"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/MatchLab-Imperial/keras_triplet_descriptor/blob/master/Baseline_code.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "metadata": {
        "id": "scylC1qNuBY0",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Baseline Code\n",
        "\n",
        "This code introduces a two-step training for the N-HPatches problem. In N-HPatches problem, we aim to generate a patch descriptor that is able to perform successfully tasks such as matching, retrieval or verification. \n",
        "\n",
        "Contrary to classical HPatches dataset, in N-HPatches, images contain random non-smooth perturbations produced by a synthetic noise. This noise could be critical when training the descriptor, therefore, we introduce a denoising model that could help us to deal with those perturbations. Denoising models have been already introduced in the course [tutorials](https://github.com/MatchLab-Imperial/deep-learning-course) and lectures, their objective is to generate a clean/denoised version of the input image.  We will refer in this code to the images with noise as `noisy`, to the images after applying the denoise model as `denoised` and the original patches from HPatches (so no extra noise added) which are used as ground-truth for the denoising step as `clean`. \n",
        "\n",
        "\n",
        "Thus, we aim to minimize the noise in images before the second step, which is computing a feature vector, also called descriptor. Those descriptions must be a powerful representation of the input patches. The idea behind is that if two descriptors belong two similar patches, they should be close to each other, i.e. have a low Euclidean distance. See figure below:\n",
        "\n",
        "![](https://i.ibb.co/4tvm3Vh/descriptorspace.png)\n",
        "\n",
        "This baseline code gives a method you can use to compare to whatever another approach you develop.  There are several other approaches you can test to see if there is any improvement, e.g. train the descriptor directly with noisy patches, without the denoising model. However, this code provides some guidance about how to implement the different blocks, how to stack them if desired, how to read the data and how to evaluate the method.\n",
        "\n",
        "The values given can be improved without changing the core method, only by tuning correctly the hyperparameters or giving it more training time, among others.\n",
        "\n",
        "As a first step of the project, you should get familiar with the problem and the provided code, so you can develop more complex and robust algorithms afterward. "
      ]
    },
    {
      "metadata": {
        "id": "iamuRgeiNLjW",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Safety Check\n",
        "\n",
        "As Google Colab is an external platform, we cannot guarantee that everytime you connect to a remote server, you will have the same amount of RAM or video RAM. For that reason, we will first check the amount of memory we have in the notebook. RAM should be around 12.9 GB, which is enough to load the datasets in memory. Also, usually, we have available 11.4 GB of GPU memory, which is more than enough to run this code. However, some users reported having only 500 MB of GPU memory. If you have that amount, restart the environment to see if you get the corresponding 11.4 GB."
      ]
    },
    {
      "metadata": {
        "id": "ZZG4BqkENEyd",
        "colab_type": "code",
        "outputId": "1dab350e-9179-4631-cae3-b9d43090e702",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 187
        }
      },
      "cell_type": "code",
      "source": [
        "# Taken from\n",
        "# https://stackoverflow.com/questions/48750199/google-colaboratory-misleading-information-about-its-gpu-only-5-ram-available\n",
        "# memory footprint support libraries/code\n",
        "!ln -sf /opt/bin/nvidia-smi /usr/bin/nvidia-smi\n",
        "!pip install gputil\n",
        "!pip install psutil\n",
        "!pip install humanize\n",
        "import psutil\n",
        "import humanize\n",
        "import os\n",
        "import GPUtil as GPU\n",
        "GPUs = GPU.getGPUs()\n",
        "# Colab only provides one GPU and it is not always guaranteed\n",
        "gpu = GPUs[0]\n",
        "def printm():\n",
        "  process = psutil.Process(os.getpid())\n",
        "  print(\"RAM Free: \" + humanize.naturalsize( psutil.virtual_memory().available ), \" | Proc size: \" + humanize.naturalsize( process.memory_info().rss))\n",
        "  print(\"GPU RAM Free: {0:.0f}MB | Used: {1:.0f}MB | Util {2:3.0f}% | Total {3:.0f}MB\".format(gpu.memoryFree, gpu.memoryUsed, gpu.memoryUtil*100, gpu.memoryTotal))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting gputil\n",
            "  Downloading https://files.pythonhosted.org/packages/ed/0e/5c61eedde9f6c87713e89d794f01e378cfd9565847d4576fa627d758c554/GPUtil-1.4.0.tar.gz\n",
            "Building wheels for collected packages: gputil\n",
            "  Building wheel for gputil (setup.py) ... \u001b[?25ldone\n",
            "\u001b[?25h  Stored in directory: /root/.cache/pip/wheels/3d/77/07/80562de4bb0786e5ea186911a2c831fdd0018bda69beab71fd\n",
            "Successfully built gputil\n",
            "Installing collected packages: gputil\n",
            "Successfully installed gputil-1.4.0\n",
            "Requirement already satisfied: psutil in /usr/local/lib/python2.7/dist-packages (5.4.8)\n",
            "Requirement already satisfied: humanize in /usr/local/lib/python2.7/dist-packages (0.5.1)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "BBvIvBoyg68g",
        "colab_type": "code",
        "outputId": "bec85b71-f8d2-483b-9372-352868b5bbf9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "cell_type": "code",
      "source": [
        "printm()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "('RAM Free: 12.9 GB', ' | Proc size: 151.4 MB')\n",
            "GPU RAM Free: 11441MB | Used: 0MB | Util   0% | Total 11441MB\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "OMiynJ7p-zI8",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Downloading Functions and Data\n",
        "\n",
        "The first step is to clone the GitHub repository of the course, which contains already implemented functions. You can use your own function and import them here doing the same. In addition, we are going to download and extract the N-HPatches data. \n",
        "\n",
        "As a note, in colab, we can run terminal commands by using ```!```. Also, by using ```%``` we have access to the [built-in IPython magic commands](https://ipython.readthedocs.io/en/stable/interactive/magics.html#magic-cd), which we will use to move through directories (`cd` command). It takes around 5 minutes to download and unzip the dataset. \n"
      ]
    },
    {
      "metadata": {
        "id": "yV1m-9ZGuKGj",
        "colab_type": "code",
        "outputId": "e97fb238-4869-42d3-e83a-4268d81e5eaf",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 153
        }
      },
      "cell_type": "code",
      "source": [
        "# Clone repo\n",
        "!git clone https://github.com/MatchLab-Imperial/keras_triplet_descriptor"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Cloning into 'keras_triplet_descriptor'...\n",
            "remote: Enumerating objects: 34, done.\u001b[K\n",
            "remote: Counting objects: 100% (34/34), done.\u001b[K\n",
            "remote: Compressing objects: 100% (24/24), done.\u001b[K\n",
            "remote: Total 162 (delta 15), reused 22 (delta 10), pack-reused 128\u001b[K\n",
            "Receiving objects: 100% (162/162), 149.81 MiB | 19.85 MiB/s, done.\n",
            "Resolving deltas: 100% (48/48), done.\n",
            "Checking out files: 100% (69/69), done.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "pyZSqhZ5LACT",
        "colab_type": "code",
        "outputId": "d97dd58d-e75d-4151-f3d7-82dd56447476",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "# Change directory\n",
        "%cd /content/keras_triplet_descriptor    \n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/keras_triplet_descriptor\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "307CBCL-FjX4",
        "colab_type": "code",
        "outputId": "7fe3901b-992e-4545-a07f-3b4fa40cb77a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 462
        }
      },
      "cell_type": "code",
      "source": [
        "# Download data\n",
        "!wget -O hpatches_data.zip https://imperialcollegelondon.box.com/shared/static/ah40eq7cxpwq4a6l4f62efzdyt8rm3ha.zip\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "--2019-02-15 17:53:32--  https://imperialcollegelondon.box.com/shared/static/ah40eq7cxpwq4a6l4f62efzdyt8rm3ha.zip\n",
            "Resolving imperialcollegelondon.box.com (imperialcollegelondon.box.com)... 185.235.236.197\n",
            "Connecting to imperialcollegelondon.box.com (imperialcollegelondon.box.com)|185.235.236.197|:443... connected.\n",
            "HTTP request sent, awaiting response... 301 Moved Permanently\n",
            "Location: /public/static/ah40eq7cxpwq4a6l4f62efzdyt8rm3ha.zip [following]\n",
            "--2019-02-15 17:53:33--  https://imperialcollegelondon.box.com/public/static/ah40eq7cxpwq4a6l4f62efzdyt8rm3ha.zip\n",
            "Reusing existing connection to imperialcollegelondon.box.com:443.\n",
            "HTTP request sent, awaiting response... 301 Moved Permanently\n",
            "Location: https://imperialcollegelondon.app.box.com/public/static/ah40eq7cxpwq4a6l4f62efzdyt8rm3ha.zip [following]\n",
            "--2019-02-15 17:53:33--  https://imperialcollegelondon.app.box.com/public/static/ah40eq7cxpwq4a6l4f62efzdyt8rm3ha.zip\n",
            "Resolving imperialcollegelondon.app.box.com (imperialcollegelondon.app.box.com)... 185.235.236.199\n",
            "Connecting to imperialcollegelondon.app.box.com (imperialcollegelondon.app.box.com)|185.235.236.199|:443... connected.\n",
            "HTTP request sent, awaiting response... 302 Found\n",
            "Location: https://public.boxcloud.com/d/1/b1!qIvxgrCJhvYo1tBTRI7ZmcgqH8WDb0Lem7Ro-ejdXdZSLhyNpg-cOFK2MpqZMSkqmCdJMgisWYNEUSQcJ_3VOjxU7rOWvoMf3zKviRd-UkA4D_naP_SjcMYsSe--SI0jwvCutRZkthVzt3KpCRQjDkzSj4wxhZdkozRpIBHaMYOrLJ2SGoDBxvKtKfyVnRahc_9ZxqX9gEgH1yAyTh93LPfFeQB2Ky5wLNukTYWctSEZTeA9Ra-d1mJ4yX5Sy9MyyliGcGDTIpiRBXwijD1xcqueyj3lf6cWYS7FcoCWtdf2pyt547mTHqEjQstVVAchjminPfwgIh5dOF9fvtpLgVGaGGOYLrsJg56vmaoeqf17c86hlI4vmL__WSSlPZ4iIyChfgtEnP0o52CnGvmozY5Y_P1tsBqmQGU6PsDAYKhpxVj3cN8-xcKtzMNIb9h4gvc2fCiUnWeWev2cHEgK5nSRrCgwIcVFExunncgug8wyo6WGYWqnDdccFr74FPnmkOjW-IZByV27i16nmEh3C--aghE-WKMaWsZzhoz2YQny0l0eH1Xm2ZD_6A9_qVT9uXJC1SipvTbeRTJ4MGxLL642phZ_sfxL3llYCmJn21Xitk23GbYTXf7j7R8ui1jDB2spIfV3CkmdJxnBCblshH4aqsSV3sAkDaQQQdwKVEGcRUZGW-UM-yc2niGWnDahakoRPVzl-syGAC3aHFFJrnLxLjlUtg5Ihh1d_ADZih5U-a2T5AxCAfi1K3VwapK_c8K1wfcc9tgiPTC8tfwf-UO44N1UgXloYFKpxCImdjfWpVdR_gnCPDrc0X2EPygxDqayloAnSfItrlSoouNAhPyLfkOe8bHHn-trhxOkfEDlFrHmHu_Uqz5zuBzhgbW5rVPlwADUKAKPJqwqbRooW5F3MQ3QvgdugmGQfQZW6yUPUmH_1YT_5o1lcev2bO9ST3NCDrzmGgspH-JXo9uu2PYzFtsUQgaVC8W_Ld8O2vfgjfIH57nG5qhQNlaHNJlGv11nQqDoz9cl9zoEUISZoARZJtEt436Ak45_kjwAwfqoRTa5FaR9JS43opAcSSQxKAwpJPeSErl230Yi7BckoyHmZe-wOXvYbcCAyJDOBwyFg6y9_NypqYflBn_--Ms-ZVgFR8ytfvzstMzlxDM6BtRU0qogL79w_63MiJlJ6BTRybfeN5trx5ip8GnSfcGojEIXmI7dGlyL1IAfdc_5kSFEXk9-vif3m0xchXtOeCWEnbydK-iSXiNEt93FFMy6muCVJXYGNOSwTpPwJgbt41EFWrn9SNM_y5Zc5S9wH98IxoLW0Qf6n1Y8XEBoSZpxxOE9v-1QY-z8zv8k2o-NSvM6Zg3pue3ri45Q_AR1iDxhtlY4pJmASYpbeJPxeruj80e_nvvC6kZnTha1T6bl23RRgbBBg0IS6punHRfd3b9Ts3rWvMc./download [following]\n",
            "--2019-02-15 17:53:33--  https://public.boxcloud.com/d/1/b1!qIvxgrCJhvYo1tBTRI7ZmcgqH8WDb0Lem7Ro-ejdXdZSLhyNpg-cOFK2MpqZMSkqmCdJMgisWYNEUSQcJ_3VOjxU7rOWvoMf3zKviRd-UkA4D_naP_SjcMYsSe--SI0jwvCutRZkthVzt3KpCRQjDkzSj4wxhZdkozRpIBHaMYOrLJ2SGoDBxvKtKfyVnRahc_9ZxqX9gEgH1yAyTh93LPfFeQB2Ky5wLNukTYWctSEZTeA9Ra-d1mJ4yX5Sy9MyyliGcGDTIpiRBXwijD1xcqueyj3lf6cWYS7FcoCWtdf2pyt547mTHqEjQstVVAchjminPfwgIh5dOF9fvtpLgVGaGGOYLrsJg56vmaoeqf17c86hlI4vmL__WSSlPZ4iIyChfgtEnP0o52CnGvmozY5Y_P1tsBqmQGU6PsDAYKhpxVj3cN8-xcKtzMNIb9h4gvc2fCiUnWeWev2cHEgK5nSRrCgwIcVFExunncgug8wyo6WGYWqnDdccFr74FPnmkOjW-IZByV27i16nmEh3C--aghE-WKMaWsZzhoz2YQny0l0eH1Xm2ZD_6A9_qVT9uXJC1SipvTbeRTJ4MGxLL642phZ_sfxL3llYCmJn21Xitk23GbYTXf7j7R8ui1jDB2spIfV3CkmdJxnBCblshH4aqsSV3sAkDaQQQdwKVEGcRUZGW-UM-yc2niGWnDahakoRPVzl-syGAC3aHFFJrnLxLjlUtg5Ihh1d_ADZih5U-a2T5AxCAfi1K3VwapK_c8K1wfcc9tgiPTC8tfwf-UO44N1UgXloYFKpxCImdjfWpVdR_gnCPDrc0X2EPygxDqayloAnSfItrlSoouNAhPyLfkOe8bHHn-trhxOkfEDlFrHmHu_Uqz5zuBzhgbW5rVPlwADUKAKPJqwqbRooW5F3MQ3QvgdugmGQfQZW6yUPUmH_1YT_5o1lcev2bO9ST3NCDrzmGgspH-JXo9uu2PYzFtsUQgaVC8W_Ld8O2vfgjfIH57nG5qhQNlaHNJlGv11nQqDoz9cl9zoEUISZoARZJtEt436Ak45_kjwAwfqoRTa5FaR9JS43opAcSSQxKAwpJPeSErl230Yi7BckoyHmZe-wOXvYbcCAyJDOBwyFg6y9_NypqYflBn_--Ms-ZVgFR8ytfvzstMzlxDM6BtRU0qogL79w_63MiJlJ6BTRybfeN5trx5ip8GnSfcGojEIXmI7dGlyL1IAfdc_5kSFEXk9-vif3m0xchXtOeCWEnbydK-iSXiNEt93FFMy6muCVJXYGNOSwTpPwJgbt41EFWrn9SNM_y5Zc5S9wH98IxoLW0Qf6n1Y8XEBoSZpxxOE9v-1QY-z8zv8k2o-NSvM6Zg3pue3ri45Q_AR1iDxhtlY4pJmASYpbeJPxeruj80e_nvvC6kZnTha1T6bl23RRgbBBg0IS6punHRfd3b9Ts3rWvMc./download\n",
            "Resolving public.boxcloud.com (public.boxcloud.com)... 185.235.236.200\n",
            "Connecting to public.boxcloud.com (public.boxcloud.com)|185.235.236.200|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 4088106554 (3.8G) [application/zip]\n",
            "Saving to: ‘hpatches_data.zip’\n",
            "\n",
            "hpatches_data.zip   100%[===================>]   3.81G  21.0MB/s    in 3m 13s  \n",
            "\n",
            "2019-02-15 17:56:47 (20.2 MB/s) - ‘hpatches_data.zip’ saved [4088106554/4088106554]\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "36mBTFvPCxY9",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Extract data\n",
        "!unzip -q ./hpatches_data.zip\n",
        "!rm ./hpatches_data.zip"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "Rjyr96hR_4wS",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Importing Necessary Modules\n",
        "\n",
        "We now import the modules we will use in this baseline code. "
      ]
    },
    {
      "metadata": {
        "id": "o0KYfe-at9KN",
        "colab_type": "code",
        "outputId": "b9c623a5-354a-47fa-ada7-d7e22beafe23",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "import sys\n",
        "import json\n",
        "import os\n",
        "import glob\n",
        "import time\n",
        "import tensorflow as tf\n",
        "import numpy as np\n",
        "import cv2\n",
        "import random\n",
        "\n",
        "import keras\n",
        "from keras import backend as K\n",
        "from keras.models import Sequential, Model\n",
        "from keras.layers import Dense, Dropout, Activation, Flatten, Input, Lambda, Reshape\n",
        "from keras.layers import Conv2D, MaxPooling2D, BatchNormalization \n",
        "from keras.layers import Input, UpSampling2D, concatenate  \n",
        "\n",
        "from read_data import HPatches, DataGeneratorDesc, hpatches_sequence_folder, DenoiseHPatches, tps\n",
        "from utils import generate_desc_csv, plot_denoise, plot_triplet"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Using TensorFlow backend.\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "metadata": {
        "id": "AFG0LyAct_-l",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "The `read_data` and `utils` imports are functions provided in the repository we just cloned. You can navigate through the *File tab* and check what those functions do for a better understanding.\n",
        "\n",
        "![texto del enlace](https://i.ibb.co/HnfSvfT/filetab.png)\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "2Y61ZKWZ7o5k",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "We also fix the seeds of the pseudo-random number generators to have reproducible results. The idea of fixing the seed is having the same results every time the algorithm is run if there are no changes in the code."
      ]
    },
    {
      "metadata": {
        "id": "NXL31ez-AT5h",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "random.seed(1234)\n",
        "np.random.seed(1234)\n",
        "tf.set_random_seed(1234)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "_OqFkNujBGzf",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Now we load the data. The original HPatches dataset has several splits, which are used to separate the available sequences in train sequences and test sequences. For our experiments in N-HPatches we use the same splits as in HPatches. Specifically, we load (and report results) using the split `'a'`:\n"
      ]
    },
    {
      "metadata": {
        "id": "ABKDHB9RApZk",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "hpatches_dir = './hpatches'\n",
        "splits_path = './splits.json'\n",
        "\n",
        "splits_json = json.load(open(splits_path, 'rb'))\n",
        "split = splits_json['a']\n",
        "\n",
        "train_fnames = split['train']\n",
        "test_fnames = split['test']\n",
        "\n",
        "seqs = glob.glob(hpatches_dir+'/*')\n",
        "seqs = [os.path.abspath(p) for p in seqs]   \n",
        "seqs_train = list(filter(lambda x: x.split('/')[-1] in train_fnames, seqs)) \n",
        "seqs_test = list(filter(lambda x: x.split('/')[-1] in split['test'], seqs)) \n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "qeWik0vMEtuC",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Models and loss"
      ]
    },
    {
      "metadata": {
        "id": "LYJz8BDzBkIx",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "We now define three functions that define the main modules of our baseline. \n",
        "\n",
        "*   **get_denoise_model(..)** returns the denoising model. The input for the function is the size of the patch, which will be *1x32x32*, and it outputs a keras denoising model. \n",
        "*   **get_descriptor_model(..)** builts the descriptor model. The input for the function is the size of the patch, which will be *1x32x32*, and it outputs a keras descriptor model. The model we use as baseline returns a descriptor of dimension *128x1*.\n",
        "*   **triplet_loss(..)** defines the loss function which is used to train the descriptor model. \n",
        "\n",
        "You can modify the models in these functions and run the training code again. For example, the given denoising model is quite shallow, maybe using a deeper network can improve results. Or testing new initializations for the weights. Or maybe adding dropout. Or modifying the loss function somehow..."
      ]
    },
    {
      "metadata": {
        "id": "W6QbkHnbuIUD",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def get_denoise_model(shape):\n",
        "    \n",
        "  inputs = Input(shape)\n",
        "  \n",
        "  ## Encoder starts\n",
        "  conv1 = Conv2D(16, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(inputs)\n",
        "  pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)\n",
        "  \n",
        "  ## Bottleneck\n",
        "  conv2 = Conv2D(32, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool1)\n",
        "\n",
        "  ## Now the decoder starts\n",
        "  up3 = Conv2D(64, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(conv2))\n",
        "  merge3 = concatenate([conv1,up3], axis = -1)\n",
        "  conv3 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge3)\n",
        "    \n",
        "  conv4 = Conv2D(1, 3,  padding = 'same')(conv3)\n",
        "\n",
        "  shallow_net = Model(inputs = inputs, outputs = conv4)\n",
        "  \n",
        "  return shallow_net\n",
        "\n",
        "\n",
        "\n",
        "def get_descriptor_model(shape):\n",
        "  \n",
        "  '''Architecture copies HardNet architecture'''\n",
        "  \n",
        "  init_weights = keras.initializers.he_normal()\n",
        "  \n",
        "  descriptor_model = Sequential()\n",
        "  descriptor_model.add(Conv2D(32, 3, padding='same', input_shape=shape, use_bias = True, kernel_initializer=init_weights))\n",
        "  descriptor_model.add(BatchNormalization(axis = -1))\n",
        "  descriptor_model.add(Activation('relu'))\n",
        "\n",
        "  descriptor_model.add(Conv2D(32, 3, padding='same', use_bias = True, kernel_initializer=init_weights))\n",
        "  descriptor_model.add(BatchNormalization(axis = -1))\n",
        "  descriptor_model.add(Activation('relu'))\n",
        "\n",
        "  descriptor_model.add(Conv2D(64, 3, padding='same', strides=2, use_bias = True, kernel_initializer=init_weights))\n",
        "  descriptor_model.add(BatchNormalization(axis = -1))\n",
        "  descriptor_model.add(Activation('relu'))\n",
        "\n",
        "  descriptor_model.add(Conv2D(64, 3, padding='same', use_bias = True, kernel_initializer=init_weights))\n",
        "  descriptor_model.add(BatchNormalization(axis = -1))\n",
        "  descriptor_model.add(Activation('relu'))\n",
        "\n",
        "  descriptor_model.add(Conv2D(128, 3, padding='same', strides=2,  use_bias = True, kernel_initializer=init_weights))\n",
        "  descriptor_model.add(BatchNormalization(axis = -1))\n",
        "  descriptor_model.add(Activation('relu'))\n",
        "\n",
        "  descriptor_model.add(Conv2D(128, 3, padding='same', use_bias = True, kernel_initializer=init_weights))\n",
        "  descriptor_model.add(BatchNormalization(axis = -1))\n",
        "  descriptor_model.add(Activation('relu'))\n",
        "  descriptor_model.add(Dropout(0.3))\n",
        "\n",
        "  descriptor_model.add(Conv2D(128, 8, padding='valid', use_bias = True, kernel_initializer=init_weights))\n",
        "  \n",
        "  # Final descriptor reshape\n",
        "  descriptor_model.add(Reshape((128,)))\n",
        "  \n",
        "  return descriptor_model\n",
        "  \n",
        "  \n",
        "def triplet_loss(x):\n",
        "  \n",
        "  output_dim = 128\n",
        "  a, p, n = x\n",
        "  _alpha = 1.0\n",
        "  positive_distance = K.mean(K.square(a - p), axis=-1)\n",
        "  negative_distance = K.mean(K.square(a - n), axis=-1)\n",
        "  \n",
        "  return K.expand_dims(K.maximum(0.0, positive_distance - negative_distance + _alpha), axis = 1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "RlS5zcV7EJgp",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Denoising Image Patches\n"
      ]
    },
    {
      "metadata": {
        "id": "wHxHwjUd3-pY",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "We use the *DenoiseHPatches* class implemented in the read_data.py file, which takes as input the list of sequences to load and the size of batches. \n",
        "\n",
        "*DenoiseHPatches* outputs batches where the input data is the noisy image and the label is the clean image, so we can use a mean absolute error (MAE) metric as loss function. You can try to use different metrics here to see if that improves results. \n",
        "\n",
        "Afterward, we take a subset of training and validation sequences by using *random.sample* (3 sequences for training and 1 for validation data). The purpose of doing so is just to speed-up training when trying different setups, but you should use the whole dataset when training your final model. Remove the random.sample function to give the generator all the training data.\n",
        "\n",
        "In addition, note that we are using the test set as validation. We will provide you with a new test set that will be used to evaluate your final model, and from which you will not have the clean images. \n",
        "\n",
        "**Updated**: Training should be quite faster now (1 epoch around 15 minutes)."
      ]
    },
    {
      "metadata": {
        "id": "m_VPSHmSK0dS",
        "colab_type": "code",
        "outputId": "d4680eaf-3eb7-4758-cfab-6b6d683cd75e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "cell_type": "code",
      "source": [
        "denoise_generator = DenoiseHPatches(random.sample(seqs_train, 3), batch_size=50)\n",
        "denoise_generator_val = DenoiseHPatches(random.sample(seqs_test, 1), batch_size=50)\n",
        "\n",
        "# Uncomment following lines for using all the data to train the denoising model\n",
        "# denoise_generator = DenoiseHPatches(seqs_train, batch_size=50)\n",
        "# denoise_generator_val = DenoiseHPatches(seqs_test, batch_size=50)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "100%|██████████| 3/3 [00:03<00:00,  1.27s/it]\n",
            "100%|██████████| 1/1 [00:00<00:00,  1.38it/s]\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "metadata": {
        "id": "-eUSba93Dttj",
        "colab_type": "code",
        "outputId": "3c7a8e9b-876d-4768-bc33-9a074dd42ffc",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 88
        }
      },
      "cell_type": "code",
      "source": [
        "shape = (32, 32, 1)\n",
        "denoise_model = get_denoise_model(shape)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Colocations handled automatically by placer.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "H3wkjkpk4bRh",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "We set number of epochs to 1, tweak it, along with other hyperparameters, to improve the performance of the model."
      ]
    },
    {
      "metadata": {
        "id": "edwbgE6yKqcD",
        "colab_type": "code",
        "outputId": "8dd274f2-0c14-4124-a36b-d904d050ead3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 139
        }
      },
      "cell_type": "code",
      "source": [
        "sgd = keras.optimizers.SGD(lr=0.00001, momentum=0.9, nesterov=True)\n",
        "denoise_model.compile(loss='mean_absolute_error', optimizer=sgd, metrics=['mae'])\n",
        "epochs = 1\n",
        "### Use a loop to save for each epoch the weights in an external website in\n",
        "### case colab stops. Every time you call fit/fit_generator the weigths are NOT\n",
        "### reset, so e.g. calling 5 times fit(epochs=1) behave as fit(epochs=5)\n",
        "for e in range(epochs):\n",
        "  denoise_history = denoise_model.fit_generator(generator=denoise_generator, \n",
        "                                                epochs=1, verbose=1, \n",
        "                                                validation_data=denoise_generator_val)\n",
        "  ### Saves optimizer and weights\n",
        "  denoise_model.save('denoise.h5') \n",
        "  ### Uploads files to external hosting\n",
        "  !curl -F \"file=@denoise.h5\" https://file.io\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.cast instead.\n",
            "Epoch 1/1\n",
            "1797/1797 [==============================] - 50s 28ms/step - loss: 11.4135 - mean_absolute_error: 11.4135 - val_loss: 7.6019 - val_mean_absolute_error: 7.6019\n",
            "{\"success\":true,\"key\":\"Hh7YWI\",\"link\":\"https://file.io/Hh7YWI\",\"expiry\":\"14 days\"}"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "2dKBM4qA8GTw",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "After every epoch, the code will generate an external link, this link saves your weights in case of colab disconnecting during training. Example of an epoch:\n",
        "\n",
        "**Epoch 1/1**\n",
        "1797/1797 [==============================] - 48s 27ms/step - loss: 11.4135 - \n",
        "mean_absolute_error: 11.4135 - val_loss: 7.6013 - val_mean_absolute_error: 7.6013 \n",
        "{\"success\":true,\"key\":\"fv9vjj\"\n",
        "\n",
        "\"link\":\"https://file.io/fv9vjj\",\"expiry\":\"14 days\"} **Epoch 1/1**"
      ]
    },
    {
      "metadata": {
        "id": "Ohb6Q94z4yya",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "If colab did not disconnect, and you want to save the weights in your local disk, you also can use:\n"
      ]
    },
    {
      "metadata": {
        "id": "GjAQRnPV47BI",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "from google.colab import files\n",
        "files.download('denoise.h5')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "vCfE3xnF8Nfc",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Moreover, if you have a model saved from a previous training session, you can upload it to colab and initialize the model's weights with it. \n",
        "\n",
        "You either can use `!wget download_link` or upload the weights from your local disk by using the left panel ('Files' section) in colab.\n",
        "\n",
        "Once the weights are uploaded, you can use\n",
        "\n",
        "> ``denoise_model = keras.models.load_model('./denoise.h5')\n",
        "``\n",
        "\n",
        "to load the weights."
      ]
    },
    {
      "metadata": {
        "id": "e9FzSZzMEcs4",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Visualization of Denoising Results\n",
        "To visualize how the denoised patches look, you can run the following function. It returns the noisy patch, the denoised patch in the middle, and the clean patch in the right side. "
      ]
    },
    {
      "metadata": {
        "id": "XFA_8uN4Eb3B",
        "colab_type": "code",
        "outputId": "9b0d86ca-15c3-428d-80a4-930609c4b7b6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 200
        }
      },
      "cell_type": "code",
      "source": [
        "plot_denoise(denoise_model)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "100%|██████████| 1/1 [00:00<00:00,  1.56it/s]\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcwAAACmCAYAAABXw78OAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJztnXd4VVXaxRdFVMrQlAGRIsgNvQak\nQ0ITUCmCKEV0YCCKgsJYUD8rqAPMKM1ehiIISFEUkBoglCggCkoVhFAEgYCKDeR8f/DcTPY6O3ef\nCyFRZv2ehz9W7in7nLPP3dy99vu+OTzP8yCEEEKIiOTM7gYIIYQQfwY0YAohhBAB0IAphBBCBEAD\nphBCCBEADZhCCCFEADRgCiGEEAHQgBmAffv2ISYmBg8//HB2N0UIxMfHIz4+PtvOP2vWLMTExGDW\nrFnZ1gYRnOTkZMTExGDs2LHZ3ZQ/PbmzuwGZwaxZszB06FBceuml+Oijj1CqVCnrdvHx8ShZsiQm\nTZoU1fGLFi2K0aNHo2TJkpnRXPEHINxn0nPJJZegUKFCiImJQbNmzdC5c2fkz58/m1qYMU888UR2\nN0FkI57nYf78+fjggw+wefNmHD9+HAUKFECJEiUQHx+Prl274q9//Wt2N/Oi5KIYMMP8+uuveOaZ\nZ/Daa69l6nEvv/xyXH/99Zl6TPHH4MYbb0TLli0BAKdOncKhQ4ewZs0aPPvss3jttdcwcuRINGjQ\nIJtbadKsWbPsboLIJk6cOIF7770XycnJqFy5Mm6//XYUL14cR48eRXJyMsaNG4dJkyZh9OjRqF+/\nfnY396Ljohowr7vuOixfvhwLFy5E69ats7s54k9AKBTy/Weob9+++PzzzzFgwAAkJCRgypQpqFKl\nSja1UIizeJ6HwYMHIzk5Gffffz/69++PHDlypH1+5513YuXKlbjnnnswcOBAzJ8/H0WLFs3GFl98\nXFQe5t///neUKVMGw4cPx08//eTc/syZM5g4cSI6duyIGjVqoEaNGrjpppvw5ptv4vTp02nb2TzM\nU6dO4T//+Q86duyI2NhY1KpVC+3atcOYMWPw22+/AQAGDRqEmJgYbN682Xfu3377DbGxsWjRogWU\nnfCPR40aNTB69Gj88ssvGD58uPHZe++9hy5duqBGjRqoVasWOnXqhEmTJuHMmTNp24T7zKOPPood\nO3agb9++iI2NRfXq1dGjRw9rn1ixYgV69+6N2NhYVK1aFfHx8Rg2bBiOHTtmbGfzMBcvXoxevXqh\nYcOGqFatGuLi4vDYY4/hwIEDvvMEaT8A/PTTTxg2bBgaN26MatWq4YYbbpBvmY0kJiYiKSkJrVu3\nRkJCgjFYhmnSpAkGDx6Mtm3b4scff4x4vH379mHo0KFo3LgxqlatikaNGmHIkCH4+uuvfdt+8cUX\nGDhwIOrXr4+qVasiLi4OgwYNwq5du4ztwv72+++/jyVLlqBz586oUaMG6tati/vuu8/Xl/9sXFS/\nMPPkyYP/+7//Q9++fTF27Fg89NBDEbd/7LHHMHPmTDRp0gRdu3ZFrly5sHz5cowYMQJbtmzBqFGj\nMtx32LBhePfdd9G+fXv06tULuXLlwqeffoqXXnoJ27dvx7hx49ClSxcsWLAAs2fPRtWqVY39V65c\niR9++AF33HGHteOL7KdOnTqoW7cuPv30U6SkpKBUqVJ4/vnn8fbbb6NFixbo1q0bTp8+jWXLlmHY\nsGHYunWrb3A9fPgw7rzzTrRv3x7t27fH9u3bMXHiRCQkJGDp0qXIkycPAGD27NkYOnQoypUrh4SE\nBBQpUgRffvklpk6diqSkJMyaNQt58+a1tnPevHm4//77UaNGDdxzzz0oUKAAdu3ahYkTJyIpKQkf\nffQR8uXLBwBRtf/BBx/EokWL0KJFC8TFxeHEiRN4/fXX5Y9lE3PmzAFw9pdkJHr37u08VkpKCrp2\n7YrcuXPj1ltvxdVXX429e/finXfeQWJiIt59911UqFABALBlyxb06tULhQsXRv/+/XHFFVdgz549\nmDhxIlatWoW5c+eiRIkSxvFXrlyJtWvXomfPnihWrBgSExMxf/58nDp1CuPHjz/HO/AHwLsImDlz\nphcKhby1a9d6nud59957r1e5cmVv27ZtxnZxcXFez549Pc/zvI0bN3qhUMj729/+5p05c8bYrl+/\nfl4oFPI2btzoeZ7npaSkeKFQyHvooYfStqldu7bXvn17X1teffVVb8CAAd7Jkye933//3WvevLlX\nr14979dffzW2GzJkiBcTE+OlpKSc/w0QURPuM6+++mrE7caMGeOFQiFv7ty53pYtW7xQKOQ9+eST\nvu3uvfdeLxQKeV9++aXnef/tM6FQyJs3b56x7dChQ71QKOStXr3a8zzP+/nnn726det6DRo08E6c\nOGFs+/rrr/vaGRcX58XFxaXphIQELxQKeUePHjX2XbFihdenTx9v06ZNnud5UbU/vG2PHj2M9+PY\nsWNe/fr1vVAo5M2cOTPivROZS/Pmzb3q1at7p06dimq/tWvXeqFQyBszZkza3wYMGODVqlXL27Nn\nj7Htli1bvEqVKnkJCQlpf5szZ47Xs2dPLzk52dh26tSpXigU8saPH5/2t/B7Vb16dW/fvn1pfz9z\n5ozXqlUrr3Llyr7vwj8TF9WUbJhHHnkEefLkwVNPPZXhdOeiRYsAALfeeqvvF17nzp0BAMuWLcvw\nHLlz58ahQ4ewb98+4+/9+vXDuHHjkDdvXuTMmROdO3fG8ePHjWP99ttvWLp0KerVq4err776nK5R\nZA1XXnklAODo0aOYP38+AKBdu3b4/vvvjX9t2rQBAHzyySfG/sWLF0fbtm2Nv1WrVg0A8N1336Xt\nc+LECbRr1w5/+ctfjG3DfTExMTHDNubOfXaiaMOGDcbfmzRpgjfeeCNtdiOa9q9duzZt2/TvR+HC\nhbUALps4cuQIihYtmva8z5Wff/4ZiYmJqFOnDgoVKmT0g6uuugoVKlQw+nGHDh0wadIk1KtXDwDw\n448/4vvvv0+LGti/f7/vHK1btzaiCnLkyIEqVarg9OnTSE1NPa/2ZycX1ZRsmOLFi+Oee+7BiBEj\nMHv27LQvnfSE597D0w7pueaaawAA33zzTYbnGDBgAIYPH462bduiadOmaNiwIRo3bowyZcoY23Xu\n3BkvvfQSZs+enfaltGLFCpw8edLaLvHHIuxl586dGzt37gQA9OzZM8Pt2TMsXbq0b5tLL73UOHa4\nL4ZCId+2RYoUQaFChSL2xT59+qQt9qhduzaaNGmChg0bonr16sZgF037U1JSAABly5b1bVO+fPkM\n9xcXjpw5c2bKeoc9e/bg1KlTWLFiBerWrZvhdj/88AMKFCgAz/MwZcoUTJ8+Hbt378avv/5qbPf7\n77/79o3U70+dOnWeV5B9XJQDJnB2Hn/27NkYOXIkWrRogYIFCxqfhxcFXX755b59L7vsMgBn/yeW\nEbfffjvKly+PiRMnYuXKlVi8eDEAoHbt2njyyScRExMDAChZsiQaNmyIlStX4siRI7jiiiswf/58\n5MuXL20AFX9cwjMIxYoVw8mTJwEA//73v3HFFVdYtw//Ig0T9igjEakvAmf74/fff5/h/jVr1sTM\nmTPx1ltvYfHixVi/fj1efPFFXH311XjooYfSVoxH0/5w3w+/C+kJf/GJrKVYsWI4ePAgfvvtt0D9\nKiPCi4EaN26Mfv36Zbhd+DmPHj0aL7/8MsqXL4+HHnoIpUuXRp48ebBz5048/fTTEfe92LhoB8zc\nuXPjiSeeQM+ePfGvf/3L92DDCyhsq2nDfwsvlMiIRo0aoVGjRvjll1/wySef4MMPP8QHH3yA3r17\nY+HChWnTazfffDOSkpKwYMEC3HzzzVi6dCnatWuX4Rek+OOQlJSEHDlyoE6dOmmLLkqVKoXq1atn\n2jki9UXg7ODl6ovly5fH8OHD8cwzz2Dz5s1YtGgR3nnnHQwcOBCTJ09GbGxs2jGCtD88UPKviUjt\nFBeWWrVqYe/evfjkk0/QuHHjiNumpqaicOHC1s/CyThy5syJ6667LuJxTp8+jYkTJ6JgwYKYPHky\nihQpkvZZOBrgf4mL0sMMU7duXXTs2BEzZszAF198YXx27bXXAgC2b9/u2y+8rLpcuXKBznPZZZeh\nadOmGDFiBHr37o3U1FTDA2jZsiUKFSqEefPmYdmyZfjpp5/QqVOnc70skUUsWrQIO3bsQKtWrVCk\nSJG0PsNeIXD215ttcAlCpL743Xff4cSJE4H7Ys6cOVG9enUMGTIEI0eOhOd5WLhwoXGeIO2/6qqr\nAMDn0QPAjh07ArVFZC7h74xXXnkl4tTszJkzER8fn7ZOgylbtiwuueQSbNq0yTo9mj70IzU1FSdP\nnkRMTIwxWALAunXrzuUy/tRc1AMmcHZpfP78+fHEE08Yc+3h6dBp06YZnc/zPEyfPh0AMkx+sHnz\nZrRp0yZtu/SE//eWfsokT5486NChAzZs2IAJEyagbNmyiI2NPf+LExeMdevW4dFHH8Vf/vIXPPDA\nAwCQtnhn6tSp+OWXX4ztR44cifr162Pv3r1Rn6tevXooUqQI5s2bhxMnThifTZs2DQAynL7/5Zdf\ncMstt1hDqLgvRtP+8AKPBQsWGNsdO3Yswy9icWFp0KABWrZsiU8//RRPP/20dbBbvnw5nn76aeTN\nmzfD75jLLrsMzZs3R2pqatqsSZiUlBTEx8enpV8sVKgQcuXKhYMHDxrfk9u2bcMHH3wAAL6+dDFz\n0U7JhilatCjuu+++tCnZsBldpUoVdO/eHVOmTEFCQgLi4+Nx+vRpLF26FGvXrsWdd95pXYQBABUr\nVsSll16Kp59+Glu3bkXVqlWRK1cubN26FZMnT0aFChV8aam6dOmCCRMmYOPGjbjvvvsu7EWLwGzf\nvj1tUPA8D0ePHsWqVauwbNkyFC1aFGPHjk3rMxUrVkTv3r0xYcIE3HbbbejWrRty586dll3qpptu\nsi52cBGOHx48eDB69OiBLl26oECBAvj8888xY8YM1KxZE127drXue9lll6FKlSqYMmUKvv/+ezRv\n3hz58uXD/v37MWXKFOTNmzdtcVk07a9evToaNGiApKQkDBo0CI0bN8aJEyfw3nvvoWbNmhFX7YoL\nx4gRIzB48GBMmTIFq1evxo033ojSpUvj2LFjWLNmDRITE1G6dGm88sorGU7JAmd/SKxbtw5PPfUU\ndu3ahUqVKmH//v145513kCNHDnTr1g3A2fzKrVq1woIFC/CPf/wDTZs2xZ49ezBlyhSMGjUK/fv3\nx5o1azBr1qxsLQiQVVz0AyYA3HbbbZg1a5Yvu8rjjz+O8uXLY/r06Rg+fDhy5syJa6+9FsOGDcvw\nCwo464++8847ePnll7FkyRLMnj0bp06dQsmSJdGjRw8kJCT4TPlQKIQqVapgy5Yt6Nix4wW5ThE9\nc+fOxdy5c9N0/vz5Ua5cOQwaNAg9evTwhXk88sgjqFChAqZNm4bnnnsOZ86cQdmyZfHAAw/gjjvu\nOOd2tGvXDgULFsSrr76ali2qZMmS6NevH/r37x9xkcfjjz+OcuXKYc6cORg1ahR++uknFClSBPXq\n1cNdd91lTOdG0/4xY8Zg5MiRWLx4MZYsWYIyZcqgT58+uPLKKzVgZhP58uXDq6++ikWLFmHOnDl4\n9913cfz4ceTJkwcVKlTAk08+iY4dOzrXR5QuXRozZszA+PHjMXfuXEycOBEFChRAvXr1cPfdd6Ni\nxYpp2z755JPIkycPVq1ahcTERFSpUgXjxo1DbGws7r77brz55psYOXIk6tSpc6EvP9vJ4WXGOmXh\n5OTJk4iPj0dsbOyfO9OFEEL8j3LRe5h/FF5++WUcP34cffr0ye6mCCGEOAf+J6Zks4sjR44gOTkZ\nq1evTkt4Xbt27exulhBCiHNAU7IXkPXr16Nnz57Inz8/OnTogAcffPC8Ao6FEEJkHxowhRBCiADI\nwxRCCCECENHDvOuuuwzNBWY5ITRXpf/22299x+RaeumXLwP/DdQOkzOnOaaHKz2E4WwTfDxbGzhB\nOk+Tpi8eDZyt7ZaeWrVqGTqcQSVM5cqVDW3Lx1msWDFDr1+/PuIxV61a5TtGpHPy8W1prDjIngOQ\nuf7i7bffHrENmUWhQoUMzbX2ePk690tb3lWuPF+8eHFDh/Oshtm0aZOheZn+xo0bDc1B4rak5VzQ\nl4vpchv5eXBcXaVKlQzNhQT4fQT+m8wgzKFDhwzNae/4Pv3www+G5vvGxw9iQfB94IxJDRo0cB7j\nfAlnNgrDSQFcaeBsCchd8L3JlSuXoS+55JKI2/PntvytrmO4jsnfXfxu8neprd+XKlXK0Fyhid8t\nfn+5f/DnnPObMxLZzsn9lN/NjFJH6hemEEIIEQANmEIIIUQAIk7J8nQmT5Xw1NjRo0cNzQnPAfiy\n7Ifr7oW5++67Db18+XJDc/kknjrh4qovvPCCrw0PP/ywoXk6huECwDxdx1MOnJyap7UA/3TnkSNH\nDL17925Dd+/e3dA8dcb3nqd4bW3gdvPUxrlMM2UGPOXNUzB8rdwveZoR8D9jnori6c/0xW8B/7Qv\nT3/yvbJNC/E5OI0eT/txYXOGn9fBgwcNbStOzudg+4Gnx/hznpLjTEj8bGzFjgsUKGBotl2yo9/x\n83W1gT+3bc9TrKxduKZo+TuE72OQc7r6A98X1q7vTsDfbp4O5T7En3MpOs63zLaAzQbgv51rH9Mv\nTCGEECIAGjCFEEKIAGjAFEIIIQIQ0cPkuWGeD9+6dauhU1NTDW2r+MHzzzyHPmXKFEPzUmmez+Y5\n9K+++srQAwcO9LWB09NxG9i/4soMfMxwXbgw7NHYQkJ4GTOHw7AH9uWXXxqa70O46HUYflY2D5M9\nZr7u7IKXkTPsrXOog+1a2Zvh+88+C/cJDvngvs5euq3IMvcL9ihdfhFrDgvi9QC2+xgTE2NoDj/i\nMAEOK+D3kb137rccEgT4+x3f+0hlqS4U/L3EXIiwkWg9Tb73tnA1hn1Nl1fLmvs1f87vkS38xhWi\nw8fk68qXL5+h2fPk49nqc/I5eB++LxmhX5hCCCFEADRgCiGEEAHQgCmEEEIEIKKHyV4C+2QcU3n8\n+HFDT5482XfMRx55xNDsR3GKN55b5hi7hg0bGpo9zCVLlvja0KtXL0O/8sorhma/in2eZcuWGZr9\nD44d5Xl+wJ8ikDXHIrHfUbBgQUPXrFnT0OwD8H0B/Pd6y5YthuZ7m1VwLCDfX5d3Y/Oj2P9xpWPj\n+EH2PLiN7GnaYkHZs+S4SYb7BL8rHIfJ/Zbj9AB/ykX2VbkWA/tPfEz2i9kr2rNnj68N3Hc5ttMV\nf3ohcKW+cxHEj3TFAkarz7fNNlyeJcPrXFgD/n7KHiOf05W+z5VukWPUbe0KEkdrQ78whRBCiABo\nwBRCCCECoAFTCCGECEBED5PztrLvw3FgjC225fPPPzc0+4E8/8z5MLmMEpcD4/y2ttJACxYsMDRf\nF8eFsefC5b7YH3z00UcNbfNx6tata2i+7n379hn6wIEDhuZ4OY5NYk+NrwEA9u/fb+jvvvvO0Lbc\nlFkB+w3sJ7KPxvfG5mFwOS6Oy2TvLhQKGZp9FvZJDx8+bGhbLCi/L3ydfE6OL+U8rbxmgH009qgB\n/73j6+Z3gfsAn4PXObD/yPcZ8L8P7GFyLKet72Y2QePwotk/2jhL1zHZswxSOs0Fe3fs5XO/52vi\n98Dm3XMf4e8qhj1P3p77gytWGPDfS5enmRH6hSmEEEIEQAOmEEIIEQANmEIIIUQANGAKIYQQAYi4\n6Me2eCI9rgTgvDAF8C+QadasmaHXrFljaFeRY14UNGfOHEN36tTJ1wZOjF2tWrWIbeJEBHFxcYbm\nBTpsfNsWBLCBz8nVOeEwm+0ff/yxodu0aRNxf15AAvgXofTu3dvQ48aNMzQnR7hQ8CIQXjjCC1E4\niYDNwOfkEbyQiJ8RL37hhSicRCBIcDU/k2uuucbQvMiO28yLsnixA2tbv+NjsnYlDeDP+T7y57aF\nY7yIg7fhguFZAS/048Uu50K0wfH8jvP2/N3HcLITIPrr4s95cQx/b3Gf5ncR8D9v1txP+ZjcBu4f\nfN22+8zt5mPYksvY0C9MIYQQIgAaMIUQQogAaMAUQgghAhDRw2Sfhr295ORkQzdp0sTQtoTf7O1s\n27bN0GXLljU0e0euBNbs5XEgLgDcfPPNhuZAap6HZ4+Tk5RzGznBNRf2BfwJFzghPAfa871lD5P9\nDU4QYUtYfv311xt6xYoVhuYkEVkF+y6cIJw9TQ6MtvkR7P9xsWz2ytkX4fvHidG5z9gKJ3NANR+D\nr5P9JPb6uO/z8W2eFyc/4HPy+8nvG7eJ/SZe98AJHQD/veZ9XF7dhYCv2+X18bOwedbcJ1zJ1xlX\nwWluA783Gf0tEq4C0UyQ5AzcR/jdYo/StT2/i677bGunKyF8RugXphBCCBEADZhCCCFEADRgCiGE\nEAGI6GHyvC77gfw5e57s/QHAqlWrDM3+H3tL7LFw7Cd7UzExMYbmGEnA7+9xoWQuOt28efOI5+D4\nRPaWOKYPABo1amRojj3kOXdOtO3yJjjWlI8P+OOfOA7QlVz/QsEeBXteGzZsMDTHAnJsLuD3yfha\nufgye3fss/D+nBCajwf4vRaOR2PfjJ9ZpUqVDM3XyUnouc/Y/saela0AcHo42TZ7QRzvG6Tf8fvC\nvlnt2rUjtikz4PfN9X7x9rb4Q5fX5orTdMX2stfH36WA/92wxWOnxxbLGQlXMXfAH/PIcerch/i6\n+Lpd9573B/zX5Yp5zQj9whRCCCECoAFTCCGECIAGTCGEECIAET1M9u527dplaJeHYssJ2aJFC0Oz\nZ8m+KMcjsifJsUiuGB7AH6s5fvx4Q7du3drQHPPIXhJ7tW+88Yahd+/e7WsDz7vzfeAYyOnTpxua\n8/RyDl2+bltcIMdhrl692tAcI5tVsJfHeXaZILFg7IOWLl3a0BzD+M033xiaPQ721tmr41hcwO8P\nHjlyJOMGwx0byB4oxzza/Cr2MHkNAF8Xb885OdmPYu+W1xwA/utgH4wLm2cFrmLM3Md4e1vOXNc2\nruLcjK0Yt6sN3K/ZP+Zjcj9mzZ4of4/ZPE2Xd+sqjM2ajxck1pTPca7oF6YQQggRAA2YQgghRAA0\nYAohhBABiCoOc+vWrYbOmzevodkXsvkC7IlwPk32QV05/thn5Ri9cuXK+fbhbT777DNDV6lSxdC3\n3367oblOJHsHHJdpiwtiz3LChAmGbteunaFdtQ85ry/XtrTFia1cudLQnPu3bt26vn2yAo7/5Lg8\n9izYn7Xdb4Zjv9hfZP+P28BeH8dt2mIg2S9i/4+9cH7G3CaOi+Y4PFsbOJfs9u3bDc3xpQxfJ/tu\n/J3AnhfgjsPjWM+swOVRMkHi9tg3c8VZus7B98nlJwLunLech5mP6fITuY+6rglwe5j8nvAxeczg\nNtjyAPM5XM8mI/QLUwghhAiABkwhhBAiABowhRBCiABE9DBnzJhh6MaNGxt64cKFhuYYLFstSvYn\neI6d5+E5toy9JI4FZc/Tlg+Vj8FxlZUrVzY0x9TxNbCvw9tz7BMALF++3NDs63C7uU0cI9m/f39D\n87y+7VnwXD/H2bJPl1V8/fXXhmZ/gZ8554m13W/25tiz5Jhhjp3lNnAMJPuqtpg5V8wbrwFITU01\nNPcR3p6PZ8thzHGX3C88zzM0+6J8nXxf+f3lawD87x+3IdoajlmBy/MK4oFFG+vp0tyfbLGG/H3I\n3wvsObP/z8+T47v5PbDBz5vfTz4Hx45Ge99ctUwB//NSPUwhhBAiE9GAKYQQQgRAA6YQQggRgIge\nJue65JitDh06GJpjHqdNm+Y7JvuFHAvYrVs3Q7OfNWLECEO/9dZbhm7SpImh2V8EgAoVKhiavTrO\nXcpz4hwHVKdOHUOzP2nzceLj4w3NdUKLFStmaI4V5djQdevWGbply5aGfumll3xtYF+0ffv2hg4a\nm5TZ8DPjOEv2TdgDscX+sSfJz4R9EvZFuQ+w58l+EvuLgD+vKns17I1zm9m3Zi+Xj1e2bFlfG9jD\n4uvge8t1Pl01GTmuz5YXlu+tKy9vVsA+W7T1EoPkM4429s+VQ5W9QJuHycfgfWx+f3r4ulzxjLb7\nwG3gfdiz5j7q8rTPJU8s78MxrhmhX5hCCCFEADRgCiGEEAHQgCmEEEIEIKKH2aNHD0NzLcqlS5ca\nmn0h9ukAfx5X9i9SUlIMzflM77vvPkNz/OELL7xgaK75CPi9H87hyXFhL7/8sqE5XrFTp06GDoVC\nhq5ataqvDVxjk/0nnsfnOqKTJ082NOfU5WfFHg3gjme01fHMCtiz5NhAzqnK3qDN8+Acqpxnlb07\nV05N7gPs5dl8OI5p5L6/Z88eQ3MtWH4+/Ez5vnFcNOC/N3xf+D6wl+7yetgT5TbbjsGxgPwuZAXn\n69fbYgX5b67csEGOGQmbH8lencsH5Tayd89tZs8ySAwtt4F9cvbBefsgfjHjyhUcJHYT0C9MIYQQ\nIhAaMIUQQogAaMAUQgghAqABUwghhAhAxEU/SUlJhuZFPW3bto24vS0xLy/qYcOXExn06dPH0LzA\ngxdGcOFlW9JxLt7MSeQ5yfyHH35o6Mcee8zQK1asMDQv8lm8eLGvDbyQhRc68EIINtvLly9vaF4w\nwgtCOFkDAKxZs8bQ/Gz69u3r2ycrYNOfEyPzQhTuQ3wdtmPw/eFkCbwgipNO8/6czN22yIoXFnCb\neCES911uAy+o+fbbbw1tSwDPizp4cZIruQIvuOLvBF6sZktcwN8LfJ3ZsejnfBeW2Ba7uBb9uLQr\n+ToTZOESF7vgxW2uc0SbIB5wJ2Bg7SpazdfAuD63HTMo+oUphBBCBEADphBCCBEADZhCCCFEACJ6\nmJxcnee72SO54447DG0LBl29erWhXcme2efhAHFObMAJAUaNGuVrA3uWHORfvHhxQzdo0MDQ7Muw\nt/T8888bmhPGA8AXX3xhaL4uDjjmhPDXXXedofm+sg/IhYMBv9fKxYBtScyzAg64Z7+BA/65newP\nA/5r42fGvgefg+8n788eaKkL0CrUAAAR+klEQVRSpXxt4OTq7HuyN8v3gRO6s1+4efNmQ5cpU8bX\nBr4OPidrfofZo+LEB+y1c78G/F4bf4/Ykh1caPheRgv7cLa/8b3he+vyMLmPsg7io7J2FWvmY7r8\nxCD3wZXcwOXtupIM2LxcV+L7oAnc9QtTCCGECIAGTCGEECIAGjCFEEKIAEQ0qV555RVDX3vttYbm\nIsbPPPOModkvAfyxmxx7tnPnTkPPmTPH0BxPyF4gJ2cfP368rw2ccJgTtHPRap7fZj+ME6NzfCNf\nk+2YDz/8sKHZh+PE3Zzkms/BcYSxsbG+NrB3tHfvXkOzv9G0aVPfMS4E7KOxB8LeH8cG2oqGc9Fv\nhv1F9o/4HNwmju3lfg34k9mzx8j9kmMi2Te1nSM9HMsL+NcIsNfK7yx7PdwvOfaT76PtWbBHyR70\nuRQEPl9cBaNdhZZtbT7feEH+PFr/EXB7kKzZy3Vp7pM2z9oVd8ltdD0Ll7YltXclkZeHKYQQQmQi\nGjCFEEKIAGjAFEIIIQIQ0cPs3r27odl74ALS9erVMzR7fQBw1VVXGXr79u2G5tyWMTExEY9Zv359\nQ7NnybGkADB8+HBD85x5nTp1DN2+fXtDc0zkunXrDM2Fs/kaAH++U44N5bjJVq1aGZrz07J3m5yc\nbOiRI0f62tCmTRtDcyFt9mKzCvbJONfokSNHDM39kr0/wO+tuPw/jv9lT5i9H/ZAbJ4Ix3Zyvln+\n3BZPmp5jx44ZmuM2bbGF7OVyHDR7Unwd7IHyveY22eJ/uZ0c95wdhctdsX2uOD4bLi+OcXl5/LnL\nnwTcsfOuGEfuQ9wmlz8J+PsUH5PbHa2HGeTZuN7PoAXE9QtTCCGECIAGTCGEECIAGjCFEEKIAET0\nMLnGYo8ePQzNdSJdvhAAfPbZZ4ZmP8rlHa1du9bQlSpVMjTHtrGnAgC1atUydM2aNQ09Y8YMQ3OO\nzhtuuMHQM2fONDR7oBUrVvS1gX0b9jT5Ojj2kOf9ExMTDc0xeOxX2s7BsZ3sSWcVXOuTY/04dnDX\nrl2GtvkRHLPG95O9G/bS2Vfhfsp+ki0Gkr1W9pPYC+fP+Zgc88jxq6FQyNcG9jBdtSe5n7lyy3K8\nqi1vKB+Dn2925JLle8m46j7aPGvuhy6P0qVduZ2D1PDkeFK+1y4vj/sLxzPaaiDz+gH2NF33he8t\nv3tBYmaDepQu9AtTCCGECIAGTCGEECIAGjCFEEKIAETlYb7//vuGZn+QvQiOKwP8MVjsqXA+U45V\n69mzp6GTkpIMzf4Ie56Av27nhg0bDM0xjE899ZSht2zZYmj2YdkTXbJkia8NEyZMiHjO5cuXG7ps\n2bKG5py67MGwrl27tq8N7P3xPuw1ZBWcy5c9De5n7KOw9wf4YzltHmN6OI8n+yLctw8cOGBoWwwl\n17Pk62APmT8vUaKEoTlPK/cRzv0M+P1fvi6X18PeO3uW3C9t3h57UDt27DC0K/70QuCqRenyB88l\nLtMVf8h9kPs130ebd8f9nOMko/VF+fuY/UdbDDSf0+Vp8nXzdUYbp2njXD1N/cIUQgghAqABUwgh\nhAiABkwhhBAiABEnsHlumf0L9mTmz59vaFtOSPYYOW5n//79hu7YsaOhFyxYYGj2QPft22foli1b\n+trAx+BYz6FDhxr6n//8p6E//vhjQ3MMZOvWrSMeHwA++eQTQ69atcrQXbp0MfTq1asNzX7VLbfc\nYmj2vzhWFAA6d+5saPZq+d5nFewHcuxXamqqoQ8fPmzoa665xndMV45U9iT5mR06dMjQ7Pfy/hzH\nCfjzqvI+XFOTa6Cy/8/XwO+jLcbSVQfQFTvI/hL3Q34Wtths9rC4pi3nq80KXP6ia3ubx+nyJF2w\nJ+nyNG3w83N5lgzHabq8XFfdWcCdC9blUUbrBWf0t3NBvzCFEEKIAGjAFEIIIQKgAVMIIYQIQMQJ\nbZ5DHzt2rKHZ42IPxRaTc/ToUUNzDBbXSuMcqM8995yhGzZsaGjOQ2rLEcmeI9eBrFKliqHvvPNO\nQz/77LOG5vqaL774oqE5LhPwe618Tq6PyR4a+8mTJk0yNHvDDz/8sK8NU6dONTQ/G1stw6yAvRn2\nY9mPZL/RFsfH95u9co4f5Bhjjlnl2DCOc2WvEPD7e+xRsu/JtWO5D7BXy54oXzPgX0PAMXIun429\nII4dDRK7y/eWnxd7tdmByzdjbJ/zvYw2ltO1PcdY2vL28vPgfaKNw3TFPNrqivK7EK1Hydhie9Nj\n85+5XUF8Txv6hSmEEEIEQAOmEEIIEQANmEIIIUQANGAKIYQQAYjo+NatW9fQvCDHZjKnhxcpAED1\n6tUNPWfOHENzQei77ror4udsWnOy9f79+/vacN111xmaFz7wYoz33nvP0AkJCYbetGmToTnBtc0I\n54UPvOCGA4B5wciKFSsMXbhwYUPzgquVK1f62nDllVca+vXXXzc0J9bOKnixGbeDF8Pw8wtSzJcX\nmnA/4r7L958ToXMbbImwXUXAuaA3LyziQuTc5iCFdHlBFS+KcxWI5uvkBR38ncDfGYB/4Unjxo0N\nzf0yK+A+E23igiC4FshwQgduA99L/pz3B/yLfs63oIIraQAnZwf87xK/B67rcj0LV/INwL8Ai/ux\nFv0IIYQQmYgGTCGEECIAGjCFEEKIAET0MLdv327ob775xtADBgww9KxZswxtK9K7c+dOQ3PC6aZN\nmxqaizNzomb2ee6++25D24pYs/fDybyjDSBPSUkxNM+hx8XF+dpQuXJlQy9evNjQPOfeoEEDQ7OX\nyx7bhx9+aGhOtA4ABw8eNDT7yS1atPDtkxXw82GPgpM28L2yecbcjzhpOHua7KNx3+dzcD9jHxzw\nJxLgxBCcgIF9UvZZuIA0e0G2BA7sH7F2JaVn74fPyX6z7VnwdXOyEU66kRW4AtkzI3m37V6kh783\nuE3sP7oKM9v24efl8lXPN/kC4Pe5XcnXXd4tt4GPz2OC7Zguvzgj9AtTCCGECIAGTCGEECIAGjCF\nEEKIAET0MHlet3fv3oZmD/O2225znpDnxNmLW7NmjaE5STbHK9apU8fQycnJhrbFonGsGPtT5cqV\nM3SnTp0MzQWOOY6MPUtO9g74/aUbbrjB0BxP+tVXX0Xcn70jvgYuWG3bplixYobmhONZ5WmyB8Ge\nBT8v9sFtsC/N/ZB9FU74/uuvvxqaPU7ul6wBv9fC8brsYbL/xO8jrxFgX8a2hoC3cflJXECYPS7u\nh3zfbHCMMRe6Zm8uK4j2nNwnXTHpgP+7KNp7z+cIck5X/KErkbkL7pO2+8h/c8VZut5N/n5wFUUH\n3J60PEwhhBAiE9GAKYQQQgRAA6YQQggRgIgeJnsLzz//vKG7du1qaI4z4zgxANi6dauhuQguF49l\nH4ZjtNhDOX78uKG5QDXgL9Y8ZMgQQ3OcJcdIcrFgji3lfJ22orwbNmwwNHuxHKfJfgXHUHKO3s2b\nNxuaPVHAH18aGxtraJvvmRXYvLf0sE/GcVrs/QD+eEL2NPmcnGOVfRiOX2RfhQsrA36Pkp8592VX\nXlfOH+zyfmzH4Ovmd5a3d+UBZY/U5uXyO8zPxnbv/ugEudf8DvPzYo+T77ULWzFoV+5YV5yly+vj\n523zMKMt1sz3gTV7lqxt8a6ZFVerX5hCCCFEADRgCiGEEAHQgCmEEEIEIKKHybF9zZs3NzTndWVv\n8LPPPvMds23btobmvJLsxfE8PnsFnGOVsdXk5H3Yyxs9erSh2WviPKGcp5XjU7t37+5rA3tHrDlG\n8u233zZ0w4YNDc116EqXLm3o+vXr+9rA18H+MsceZhV8Lew3sm/GeWJtnjF743wM7gPs/fAxXbFj\ntlyy1apVi7gN10h1nZM9T97fVleSPUr2oPi+8JoAlw/HMXC2WEFXXJ0r5+qF4FxypKbHFu/Nz4vv\nBXuOrpyrNo8yPbZ+H20uWIbXsfA1uHIL287puk5XnOa51CqNNtYzI/QLUwghhAiABkwhhBAiABow\nhRBCiABEnBRnP5Fj/9q3b2/oN954w9CcgxXweyicF7RGjRqG7tKli6HHjh1r6BdffNHQ7KPeeOON\nvjawz8Jebbdu3QzNnsr7779vaPbQ2OvlmD7An/+U40U5J26ZMmUMzZ7nunXrDM3+cuvWrX1t4Hg4\njid1eSYXCq5VyZ4mt4vjf23+IXsx7Fmw9+I6Bz9zhmNaAX8OVfYP2ZPkc7BHyW1mP8nmQXPf5/eR\nNW/v8n54e5unxXHK/HyD5EjNbLid/M7zdfF123wzvld8XbwPPz/2+qKN68yoXZE+j1afr0cKuOth\nuq6bdZB6qucal6lfmEIIIUQANGAKIYQQAdCAKYQQQgQgokm1ZcsWQxcsWNDQK1euNDTPZ3O+VMA/\n38z+Bddg5PqYPM/P+TQ3bdpk6L59+/rawDGK48aNMzT7V9xmjpEsXry4oUuUKGFoW05W9kD4GLVq\n1TL09u3bDc2eZaNGjQx97bXXGjo1NdXXBvbEeBuOG8wq2HtjP4nj9tgTs+WSdXlvKSkphmZfhT1M\njk9kj5mfJ+D3LPk6+brY+2aPk/sp+9r8bgF+vz7aeFL21Rj+jrDlheW/8Tn4OrMD7h/RatvfOFaT\n+7Xru83lWdvWHLhqy7pqVdp80fPFlZ+WNV/3ufikyiUrhBBCZCEaMIUQQogAaMAUQgghAhDRw2QP\nhL25L7/80tA812zzzeLi4gzN9TDZA2HvqFSpUobmPKO33Xabod977z1fG9iTZG+JPUqO4Vu/fr2h\n2SdITEw0NMerAsCbb75paK5/yW3guM2aNWsamj01rhl54MABXxvYS+K6hNmVS5Z9NvZa2athr489\nTcDv7/Ezd9X147ys7Ltxrlqbj8oxx644PW4z9zOGvR9bbUJ+J/kcfF2cU5ffBVfNTt4e8L+zX3/9\ntaGvuOIK3z4XGs5ZzR62Kw7Thsvv5efv8hOD5G11tcHlvfL3ryufMb+LtvvC945xxV1GW0/T9jm/\nW3xM5ZIVQgghMhENmEIIIUQANGAKIYQQAdCAKYQQQgQg4qIfXrjAxigvCmrVqpWhV61a5TsmJyZg\nKlWqZGhOQM3mPC9S4O1tibh5EQkbwLzYhRcl8GKnjh07GppN540bN/rawAsb2MBPSkoyNN97voYK\nFSoYukmTJobes2ePrw21a9c2NC9m4uLeWQX3O154wveCF+zwQhbAvVCAF+0cPXrU0Py8XOfkRAiA\nf/EDB6ZzcnYOdOf9eZEWY0sQzwukbAuD0sN9gBeF8HVzwgZejAb4nx8v5AuymCWzcS3qcSUhOJdz\n8PeOa6ERL8DJjOII55ucnRfYuRb4ANEXhOaFS67FULbk/dxvXfc+I/QLUwghhAiABkwhhBAiABow\nhRBCiABEnATnRAUc6F6nTh1DcxFkW/A2+xNclHr37t2G5jlzDnrmef3Dhw8bmgOrbezdu9fQ7Muw\n19ShQwdD85z5p59+amj2aAD/3P+MGTMM3axZM0Nz4gI+Jxf+5TZcf/31vjawT8qFV9mnyyq433Cf\n4X7IfcKW8Jt9Dk7swIkJOOCe7wX7R5ww3Ob/8vvE+7Dfxwkc+Dr3799vaPbVbG3gNQDszXK/Ys+T\nn4XLf2L/GfBfJ79fQd7ZzCZaL8913YA7EYErmborSQB/hwQpIM3n5OfjeuddBcJtzy7aotTsWfJa\nGZfvbvPA+V7yd508TCGEECIT0YAphBBCBEADphBCCBGAiB4mx5LVqFHD0JwYnb0/jgUE/EWnec7b\nFQ/H/uG2bdsMzR5mfHy8rw3z5883dL9+/QzNxZk3b95saPaBOCk2x6JxIW7AX8R6/PjxEc/JfsUH\nH3xgaE46z14EXzPg9/p27txpaNu9ywq4X4VCIUMXK1bM0OyBcYwq4I9JZB+Ek9Ozt8eeJj9z9l1s\nPgv7zOxR8rvAnpQrPpU9TVsMMsc5s0fJ/iL7rNwGvg98DbZYUe53vI6BC67Hxsb6jpHZsMfFz/Nc\nEoKzT+byC6P1MIPEYbqSirs8ThfcBtt9iNb/Zc+SCwa44jB5rYCtDdxuW5EAG/qFKYQQQgRAA6YQ\nQggRAA2YQgghRAByeGxSCCGEEMKHfmEKIYQQAdCAKYQQQgRAA6YQQggRAA2YQgghRAA0YAohhBAB\n0IAphBBCBOD/ATl/gSbn/MeDAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 576x396 with 3 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "SyABaCvkEPDR",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Training a Descriptor Network\n",
        "In the last section we trained a model that given a noisy patch, outputs a denoised version of it. We hoped that by doing so, we will improve the performance of the second part, which is training a network that outputs the descriptor. As we mentioned, a descriptor is a numerical vector that represents the small images we have. The dataset consists of a large number of small images, which are cropped patches from other larger images. Hence, they represent some local part of a scene. That is why there are no objects represented, only corners or textures. Each of these patches is related to a subset of other patches of the dataset by some kind of geometric transformation (e.g. rotation).  For a given patch, we want the network to output a vector that is close to the vectors of the patches that represent the same local part of a scene, while being far from patches do not represent that local part of a scene.\n",
        "\n",
        "To do so, we will build a convolutional neural network that takes the input of $32\\times32$ and outputs a descriptor of size $128$. For the loss, we use the triplet loss, which takes an anchor patch, a negative patch and a positive patch. The idea is to train the network so the descriptors from the anchor and positive patch have a low distance between them, and the negative and anchor patch have a large distance between them. \n",
        "\n",
        "In this cell we generate a triplet network, which is a network formed by three copies of the same network. That means that the descriptor model will compute the descriptor for the input `'a'` (anchor), the same descriptor model (with the same weights) will compute the descriptor for the input `'p'` (positive), and again the same model will compute the descriptor for the input `'n'` (negative). \n",
        "\n",
        "**Updated explanation**: Due to the way Keras handles the compile method, it needs a loss as an argument in that compile method. However, our loss is computed in the lambda layer, so we want to minimize the output of that layer. As we want to minimize the output of the Lambda function (in this case the triplet loss), we output as the label in the training_generator a vector of zeros and we compute the mean absolute error of the triplet loss and this vector of zeros. To give you an intuition, what we aim to minimize is\n",
        "$$  |\\text{triplet_loss} - 0| =  |\\text{triplet_loss}| = \\text{triplet_loss} $$\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "DVmDZIRTHPDa",
        "colab_type": "code",
        "outputId": "94bda2ad-e40e-4c44-e74f-be30d956eed6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 88
        }
      },
      "cell_type": "code",
      "source": [
        "from keras.layers import Lambda\n",
        "shape = (32, 32, 1)\n",
        "xa = Input(shape=shape, name='a')\n",
        "xp = Input(shape=shape, name='p')\n",
        "xn = Input(shape=shape, name='n')\n",
        "descriptor_model = get_descriptor_model(shape)\n",
        "ea = descriptor_model(xa)\n",
        "ep = descriptor_model(xp)\n",
        "en = descriptor_model(xn)\n",
        "\n",
        "loss = Lambda(triplet_loss)([ea, ep, en])\n",
        "\n",
        "descriptor_model_trip = Model(inputs=[xa, xp, xn], outputs=loss)\n",
        "sgd = keras.optimizers.SGD(lr=0.1)\n",
        "descriptor_model_trip.compile(loss='mean_absolute_error', optimizer=sgd)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "BllXKocHCwZ7",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Here we use the class HPatches, which loads the corresponding files by using the method `read_image_file`. It reads the patches. The output of read_image_file is a tuple of the form (images, labels), which is passed to the class `DataGeneratorDesc`. This class is a generator that creates batches of triplets, and each epoch is defined by the number of triplets in the argument `num_triplets`.\n",
        "\n",
        "**Updated**: In the previous version of the baseline code, we were training the descriptor model with the noisy patches, not with the denoised ones. By adding the argument `denoise_model=denoise_model` to the class HPatches we can use the denoised images instead to train this descriptor model (if `denoise_model=None`, the noisy patches will be used). However, as it has to compute the denoised patch first, the loading of the data will be slower (6/7 extra min).  If you want to train the model with the clean patches instead, you can set the argument `use_clean=True`. In this last case, even if a denoise model is given, it will not be used. When running this piece of code **the type of patches (denoised, noisy or clean) used is printed**."
      ]
    },
    {
      "metadata": {
        "id": "YIR1cH4fDwKj",
        "colab_type": "code",
        "outputId": "b17c0b36-e90e-4df0-e969-0ab13136786d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 187
        }
      },
      "cell_type": "code",
      "source": [
        "### Descriptor loading and training\n",
        "# Loading images\n",
        "hPatches = HPatches(train_fnames=train_fnames, test_fnames=test_fnames,\n",
        "                    denoise_model=denoise_model, use_clean=False)\n",
        "# Creating training generator\n",
        "training_generator = DataGeneratorDesc(*hPatches.read_image_file(hpatches_dir, train=1), num_triplets=100000)\n",
        "# Creating validation generator\n",
        "val_generator = DataGeneratorDesc(*hPatches.read_image_file(hpatches_dir, train=0), num_triplets=10000)\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Using denoised patches\n",
            "100%|██████████| 116/116 [00:33<00:00,  3.44it/s]\n",
            "Denoising patches...\n",
            "100%|██████████| 15589/15589 [04:21<00:00, 59.52it/s]\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "100%|██████████| 100000/100000 [00:01<00:00, 66375.95it/s]\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Using denoised patches\n",
            "100%|██████████| 116/116 [00:20<00:00,  2.21it/s]\n",
            "Denoising patches...\n",
            "100%|██████████| 9525/9525 [02:37<00:00, 60.37it/s]\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "100%|██████████| 10000/10000 [00:00<00:00, 59634.17it/s]\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "metadata": {
        "id": "GoQYyuD7_4PS",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "We plot a random triplet in the form of anchor, positive and negative sample. The positive and anchor patches are similar between them (the difference is a geometric transformation, for example rotation), whereas the negative sample should be quite dissimilar to any of the other two."
      ]
    },
    {
      "metadata": {
        "id": "3RQmOMU92csu",
        "colab_type": "code",
        "outputId": "648c64db-a797-4bb6-c7de-5eb45d27da25",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 183
        }
      },
      "cell_type": "code",
      "source": [
        "plot_triplet(training_generator)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcwAAACmCAYAAABXw78OAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJztnXu4VlWdx78oF4+RkSTiJCKg7zki\nlw7XJBXHSEEBzXQUIUUlzSaeQksbxUtJpjkzPXhpsB6nKZuSiyA5eCshsdQEjTATUkAC9aCdBEEE\nUfb84fOezv7sxbvWAQ6Efj/Pc/5Y77v22uvy23udd33X77daZFmWyRhjjDEV2Wt3V8AYY4zZE/CE\naYwxxiTgCdMYY4xJwBOmMcYYk4AnTGOMMSYBT5jGGGNMAp4wm8Dq1atVXV2tq6++endXxbzPKdva\nN77xjZ2Sz5jm5INih3vshHnjjTequrpa/fr106ZNm3Z3dcz7iJkzZ6q6urrwd8QRR2jQoEH613/9\nVy1cuLBZ69C+fXtNnjxZo0ePzn3+/e9/X6tXr47mM//YlG2sV69eWrVq1TbzHX/88fr85z+/C2uW\nxgfVDlvu7gpsD2+//bZmzZqlvfbaS+vXr9cDDzygU089dXdXy7zPGDFihIYMGdKQ3rRpk5YvX66p\nU6dq7ty5uvHGGzVy5MhmuXdVVZWGDh2a+2zVqlWaPHmy+vbtq4MPPnib+cyew+bNm3XdddfpBz/4\nwe6uSjIfZDvcI39hPvTQQ3r99dd11llnqUWLFpo2bdrurpJ5H1IqlTR06NCGv1NPPVWXXHKJZsyY\noaqqKn3729/Wli1bdll9nnnmmV12L7NrGDhwoB555BE99NBDu7sqyXyQ7XCPnDCnTp0qSTr33HPV\nt29fPfXUU1q2bFkuz+9+9ztVV1fr+9//vp566imNHj1atbW1qq2t1bhx44LLIPfdd5/OOuushnwX\nXXSRnn322WAdnn/+eZ1//vnq06ePamtrdcEFF+gvf/lLId8vfvGLhjJ79uypoUOH6nvf+542btyY\ny1ddXa2xY8fq0Ucf1YknnqhPfepT29s9ppnp1KmTBgwYoLVr1+r555+X9N4vhVtvvVUnnXSSevXq\npdraWp1xxhmaPn164fonn3xSF154oY455hj17NlTxx57rCZMmKA///nPDXmoCX3+85/XhAkTJEnn\nnHOOqqurtXr16kK+s88+WzU1NVqzZk3hvnV1daqpqcktm/3tb3/TpEmTdPzxx6tHjx4aOHCgLr74\nYv3hD3/YeR1mtskXvvAFde7cWd/+9rcL74RtMWPGDJ1++unq3bu3amtr9dnPflZ33nmntm7dmsu3\nYcMGXXfddTr66KPVq1cvnX766frtb3+r6dOnq7q6WjNnzszlv++++zR69Gj16dNHPXv21Iknnqgb\nb7xRb7zxRkOeD7od7nET5ooVK/Tkk0+qtrZWhx56aMNSbOjFJElLly7Vl7/8ZfXv319XX321Tj75\nZD366KMaP358Lt8dd9yhCRMmqF27drrmmms0YcIEPf/88xo1apQWL16cy1tfX6+LLrpIPXv21LXX\nXqsRI0boN7/5jS677LJcvttuu01f//rXlWWZvvKVr+jqq6/WJz7xCU2ZMkUXXnhhwcA3bdqkb37z\nmzr77LN1xRVX7GhXmWZkn332kSS988472rp1q774xS/qlltuUU1NjSZOnKhLL71Ubdq00cSJE/W9\n732v4bqnn35aY8eO1erVqzVu3Dhdf/31Ovvss7VgwQKNHj1aL7/8cvB+48ePb1jyGj9+vCZPnqz2\n7dsX8p188snKsky//OUvC9898MADyrKsYRl53bp1Ouuss3TPPfdo2LBhmjRpki644AItWbJEo0eP\n1uOPP77D/WQq07p1a1111VWqq6vTLbfcEs1/ww036Morr1SHDh00ceJEXXbZZTrggAM0adIkXXXV\nVbm8X/va1/TTn/5UvXr10sSJE3XMMcfokksu0WOPPVYo96677tKECRP07rvv6vLLL9d1112nQYMG\n6X/+53903nnnNbyrPvB2mO1h3HDDDVmpVMqmTZuWZVmWrV+/Puvdu3c2cODAbPPmzQ35nnjiiaxU\nKmXV1dXZokWLcmWcc845WalUyv7yl79kWZZl9fX12ZFHHpmNGTMm27p1a0O+ZcuWZdXV1dn555+f\nZVmWrVq1qqHMhQsX5so8//zzs1KplL388stZlmVZXV1d1r1792z48OG5emVZll177bVZqVTK5syZ\n0/BZudzZs2fvaBeZHeTuu+/OSqVSdvvttwe/37hxY3bMMcdkvXr1yjZu3JjNmTMnK5VK2VVXXZXL\nt2XLlmzkyJHZEUcckdXV1WVZlmXXXXddViqVssWLF+fyPvfcc9nYsWOzRx55JMuyv9va5Zdf3pDn\n5ptvzkqlUvbEE080fMZ89fX1Wffu3bMxY8YU6n3mmWdmPXr0yNauXZtlWZZdf/31WU1NTeH5qKur\ny/r27ZuNGDEiqb9M0ynbWHksx48fn3Xv3j1bunRpLt8///M/N4zlc889l5VKpezaa68tlDd+/Pis\nVCplzz77bJZlWfbHP/4xK5VK2ejRo3P5FixYkFVXV2elUim7++67Gz6/8cYbs1GjRmXr16/P5Z8w\nYUJWKpWyBQsWNHz2QbbDPeoXZnmzT1VVlYYNGyZJatu2rU444QS9/vrr+tWvflW4pk+fPurdu3fu\ns549e0qSXn31VUnSL3/5S23ZskUjR45UixYtGvJ17dpVP//5z/Vv//Zvuet79Oihvn375j6rrq7O\nlTl37ly98847+tznPqfWrVvn8n7uc5+TJM2bNy/3+d57753bZGJ2L5s3b9Ybb7zR8Pfaa69pwYIF\n+uIXv6g1a9boC1/4gqqqqhr+iz7rrLNy17ds2VKnnHKK3n33Xc2fP7/hM0l66qmncnlramr0ox/9\nSMcee+wO1Xn//ffXUUcdpaeeekr19fUNn9fV1WnRokUaPHiwPvKRj0h6bwmuW7du6tKlS66dVVVV\n6tevn5YuXap169btUH1MGldccYVat26tb37zm8q2cYDU/fffL0k66aSTcuP1xhtv6MQTT5T03nK/\n9J4kJUnDhw/PldGvXz/16dOnUPZll12mn/3sZ2rbtq22bt2q9evX64033tAhhxwiSXrppZea1J73\nqx3uUbtky5t9Ro4cqbZt2zZ8ftppp2n27NmaNm2aTjrppNw15QFvTJs2bSS9t5wmqUGH6tSpUyFv\nbW1t4bPOnTsXPquqqpKkBheX5cuXS5IOP/zwQt4uXbpIkl588cXc5/vvv7/23XffQn6ze7j11lt1\n6623Fj5v166dLr/8cp133nmS/j7Whx12WCEvx3rUqFGaPXu2vvOd72j27Nk69thjNWjQIPXt27dh\nMt1Rhg8frkcffVS/+tWvdOaZZ0oqLoOtX79er776ql599VX1799/m2W98sorDS8203x07NhRX/7y\nl/Xd735Xs2bN0mmnnVbI88ILL0iSxowZs81yykv65Qku9K7q3bt34R+2DRs26LbbbtNDDz2kurq6\nhndjmXfffbdpDdL70w73qAmzvNlnwIABWrlyZcPnHTt21Mc+9jE98cQTWrVqVW7i46+7EOVJrlWr\nVkn1SMlXFvDLE2ljyvrXW2+9lfv8Qx/6UNL9za7hX/7lX3L/oe+1115q166dunbtqr333rvh840b\nN6pVq1ZBW+NYd+7cWbNmzdIdd9yhBx98UFOmTNGUKVPUvn17jR8/XqNGjdrheg8ZMkRt2rTRQw89\nlHtR7bfffjruuOMkSW+++aak937ZVtLLP/7xj+9wfUwa5557rmbNmqWbbrpJn/70pwsTRHnM/vM/\n/1Mf+9jHgmUccMABkv5ub2X7a8yHP/zhXDrLMl100UVauHChjj76aI0fP14dOnTQ3nvvrf/7v//b\nbi+E96Md7jET5vLlyxuWGyZOnLjNfDNmzGjYxZVKWbRev3799lcQlH8phna+lY3ZE+Q/Np06ddLA\ngQOj+fbdd19t2bJFb7/9dmHSLI9/47Hu2LGjrrzySl155ZVasmSJ5s2bp5/+9Ke69tprte++++qU\nU07ZoXq3bdtWxx13nObOnat169bprbfe0qJFi3TGGWc01K9cny1btiS10TQ/LVu21DXXXKMxY8bo\nP/7jP/Stb30r9315zDp16qRevXpVLKs8zps3by58t2HDhlx68eLFWrhwoQYMGKAf/vCH2muvvyt1\nv/nNb7arLdL70w73GA2z/F/OGWecocmTJxf+vvvd72rvvffW3XffXVhOiFH+76W8NNuYhx9+WL/4\nxS+aXN/y8lxjV4Ey5aWVrl27Nrlc849HpbEuuzt169YteG1NTY0uvvhi3XHHHZK00/zxRowYoS1b\ntujXv/51YRlMeu9XxoEHHqiVK1fmNKYyf/vb33ZKPUzT6N+/v0499VRNnz69sDu/bGdPP/104bo3\n33wzNzkeeOCBkhTcdU1XjXLEnoEDB+YmS0lasGDBdrTi77zf7HCPmDDLm31at26tSy65JOdMXv47\n5ZRTNGTIEL322mv69a9/3aTyBw8erFatWumee+7JOaKvWbNGX/nKVzRjxowm1/n4449Xq1atdPfd\nd+vtt9/OfVdeWi4L9WbPprzN/q677sp9XrbbNm3aaPDgwZKkCy+8MLdNv0xZk68kIZRfZqFfDWTw\n4MH68Ic/rPnz5+vhhx/Wxz/+cfXr1y+XZ9iwYXrnnXf0k5/8JPf5unXrdOqpp2rcuHHR+5idz2WX\nXaa2bdvqmmuuyWmH5Y2OP//5zwvhQG+66SZ98pOfbPAFL++9KG8UKrNw4cLChFteYePGnpkzZzbo\n843v90G2wz1iSfbBBx/U2rVrddppp2n//fffZr4xY8bowQcf1PTp03X++ecnl3/ggQfqS1/6kiZP\nnqzzzjtPn/3sZ7Vx40bdeeedklTwr0zhgAMO0Fe/+lXddNNNOuecczRixAi1atVKjz/+uO677z6d\ncMIJDev4Zs9myJAhOu644zR9+nRt3rxZAwcO1Jtvvqk5c+Zo+fLluvLKK/XRj35U0nv6e9kmhg0b\npo985CP661//qmnTpqlly5aFnbaNKYchmzJlipYtW6Zjjz22YQMbad26tT7zmc9o7ty52rBhgy64\n4ILcDnBJuvjii/Xwww/r9ttvV319vfr376/6+nrdddddqq+v1znnnLOTesg0hfbt2+urX/1qw5Js\neeNiTU2Nzj33XP34xz/WqFGjdOaZZ6ply5YNkYJGjhzZkHfgwIHq0aOH5s+fr0svvVSDBg3SSy+9\npGnTpunkk0/Wvffe23C/2tpaHXTQQbr33nt14IEHqkuXLnryySf1+OOP65prrtEll1yiWbNm6aMf\n/aiGDRv2gbbDPWLCbBzZpxIDBgxQqVTSo48+WtgtG+NLX/qSDjroIN1555361re+pb322kt9+/bV\nzTffrJqamu2q97hx43TQQQfpxz/+sf793/9d7777rjp37qyvf/3rGjt27HaVaf7xaNGihW655Rb9\n8Ic/1L333qv7779frVu3Vvfu3XXbbbflXIXGjRunDh06aOrUqbr55pu1YcMG7bfffvrEJz6hSZMm\nBbf8lxk6dKjuv/9+PfbYY1q+fLl69uypjh07bjP/8OHDG6K5hGLetmvXTtOmTdNtt92mefPm6Z57\n7lFVVZV69+6tSZMmacCAATvQK2ZHGDVqlGbOnKk//vGPuc+vuOIKHX744Zo6daq+853vaOvWrTr0\n0EML75QWLVpoypQpuv766/XII49o3rx56tGjh2699dYGl5PyL8U2bdro9ttv16RJk/STn/xE++yz\nj4466ij97//+rzp06KB7771Xjz32mKZMmaJhw4Z9oO2wRbYtpx9jjDHvO2644Qb96Ec/0g9+8IMG\nqcCksUdomMYYY9LZtGmTLr300kLQlc2bN+uBBx5Qq1atGgK4mHT2iCVZY4wx6ZT9L2fOnKl169Zp\nyJAh2rRpk2bMmKFXXnlF48aNq7gfxITxkqwxxrwP2bJli/77v/9bs2fP1iuvvKKtW7eqa9euOv30\n03X22WcXNt+YOJ4wjTHGmASsYRpjjDEJVNQwy/42ZRjI96CDDsqlf//73+fSjeO9lqG/DrcjM/h4\nOTbittJl/7Yyr7/+esV0CLaLadaZZTKy0H777VcxLf09huK27sGweSyj8aGuoe/ZhlDEDN6D7WCd\ndvQkjVQYhJlBycsBzcukxPalXXXo0CGXphN24+D+UjH+Ju2O+ekOIKkQReXTn/50hRoXA15zzHlo\nOunevXvhs1gAa54KUT59pwwPSWe/biuiUWPYd0cddVQuzXBsRx99dLTMHYVuaHw2GLu1cYCTbUG7\n5GELPEeSz/Bzzz2XSzPMJt+/oTB2jJ7DAwLatWuXSzOoAA+P4HPwzDPP5NKhSFXsK443YcQ1aq09\nevTIpbcVV7cxsee3fJpQmf/6r/8KluNfmMYYY0wCnjCNMcaYBCouyfLnOZcMuIzFpbOUM9RiS4ks\nk99zGZHLwFzGkopBz9kOLn1waZL3ZLrx0U+S9NprrxXqwKUsxhDlkgHryDrFliVCW8hjy2mheu8K\nuGxIG2EcTaZDxJZtubxFuJzKNO1s7dq1hTJoF6tWrcqluYRaDopdhkt6f/3rX3Np2kDopJxyUO4y\n7Dv2PU+2oI3Qhl555ZVcmkthUnEsuGzIftoVsF/Yd0yzDaGTjvjMUQbgkizhO4D9wqXLkF8l67li\nxYpcmu1as2ZNLt27d+9cmjbJuLRcwpeK8wDbwb6L2VR1dXUuzSXfkN3znnxWUo8P8y9MY4wxJgFP\nmMYYY0wCnjCNMcaYBCpqmNTJuDZMXY2uC6Gz/eiqQjcRrnezDiFNsjFc9w+5dFD7YR66fFCjZLu5\nPk63k9DRN9y2Tg2N38fuwX5jG0IaHfPQLSjUd7sCahjkrbfeyqWp01ArCuWhbsL+ZX72H22CWiC3\n60tF1xNqWHy+qHGyX6hZ8ftQP/IZpRZLzZHaDt1OqB/x+Qsd5s7ngXVK2fuws2G9S6VSLk09n+Md\n0sg5njFNkuNF96olS5bk0uUzL8uEXIZ4gDTfXXwX8v0c0gMbw2ctpD/TFYkHY3fq1KlimXRtod3z\nbNmQCx21V/ZV6kHV/oVpjDHGJOAJ0xhjjEnAE6YxxhiTQEUNkzoZ1+AJ/Y6ow0nFdfqm6oGEOhv1\nj5AewmvYLl4Tuwd1HWo01GlD9+Q17Af2E+sQC2sX8qlkHqZ3h5YkFbU++gLye6ZD2iv1dLaNWhz9\ntOgPzDrR/4zaoFTUZqij0I6oi8U0ypTxo/bGdlJX5dkM1IuoDcWeZ6mok6Zc09zEwmHSF7Curi6X\nDvkC07eT7aTN0I5pc/SJZD8eeuihhTrQ5n7729/m0i+88EIuHXpnN+ZPf/pTLp3iA8060G6rqqpy\n6UMOOSSX5rxCXZ02GdozwmeJ9Q69o0P4F6YxxhiTgCdMY4wxJgFPmMYYY0wCFTVMrjXHYskyf0jH\noT5BX0CmuaYe821iHUJ+RDENM6ZB0mdnn332yaWpPYQ0NdYzFoeX38d8uqhphvSsWMzb1HX9nQ31\nBdoR+45aDsdDKuog1CyZ5j1ivri0s1BsX5ZJHTSmWcbi2RJqO6F7MM3njboq02x3it0xnik1rpiO\n1hzQL5f2QH9Fankpuis1TY4/+4p2zP0c9A0O7TFhu6iTDhs2LJdmLNnly5fn0vSZpO92yJ+R766Y\nRsl4xYyxTO2fdh7yOacPK8c35Vg6yb8wjTHGmCQ8YRpjjDEJeMI0xhhjEvCEaYwxxiRQcdMPRWhu\nCokF5qXYGyqTcNMAN8zwewZjT3G256Ye1jMW8J35KfjTST7kSBv6rFKZsU0/bDfF91DQem4q6dix\nYy4dCxrRXLAeDLZOR+dYsHapaKvcGMBNANwswY0FsY1JoQOFWQduduA9aPvcoBML3h1yKmc7eU9u\n6uHGk1iwBZYX2pjEDVO0TW6O2RWwnbF3Gzfs0CalYgB+9l1s4xjrxI0rLC+08WjevHmFzxrTr1+/\nXJqHUnNzEw8E4HuJ9hXKw3ozEAG/jx3+TkLvAwZ14PjF3sdl/AvTGGOMScATpjHGGJOAJ0xjjDEm\ngYoaZswZl0EGYppKCK7bv/322xW/J9TZqPuE6hA7/JfE1rdjQa9Dh+gS9iXLpDM328V+o24Q6oeY\n3pu6rr+zSdFFKuUPaeexA3+ptVGLox3yemoiKbbPOvEaOplT84ppQ7xekl566aWKdWCZsaD+vCdt\nKvT8sgz2/e7QzmMBGlhHBvVgkHqpeHDy2rVrc2k+s9z7wOtpk3Pnzs2la2pqCnUgDCI/derUXJrj\nT02TY9OlS5dcOvSsUpOmZsnAI7EDOGhTvD4En61YQPdt4V+YxhhjTAKeMI0xxpgEPGEaY4wxCVTU\nMKk1UEOJaVzMLxXXjukvGNNMYoHNY0HNpaKOw3seccQRhWsq3ZN+ZCmBmOmnFwtazX4LtasSKfnp\n27m7gq/HfEhZz5jOLRW1OmrCsQNmY/egdhN6NqhBUffk99QgeQ/WOab1hvIwqDj1oFiA91j5If9f\n2iL7NkX/3dnEAtszYDjHjnqkFPcZX7ZsWS5NPZj6YIsWLXJp+ieH3js8BIDvFR6EzbHp379/Lh3z\n6zzyyCMLn9GOWU+2m9A+Ytp/qDw+7xyvlGdH8i9MY4wxJglPmMYYY0wCnjCNMcaYBCoKW1xjpybJ\ndWGuTYd0HJZB3Ya+SLHDnZmmvsj4t1JRW2A7Yjpp7J4kpOXGNEVqDbG4k/w+duCxVKw31/Xp/9S1\na9cKNd55UB+K6Ynsq9DBySSmg9BfjRoH7ZL9G4olyzKpQfIgZbaTdWCsUepu7EdJevXVVwufVaoT\n9SDWie8I9n3Iv43toMbV1NihOwP2JfcM0MaoaVILDn1GG6O+yH7o3bt3Ls3DnbkXI3RwOmPFsh08\nAJp2y1iyQ4cOzaXprxqyL8Zx5R4FxnWO+bXTxjjvhPyPaXPUf0PvxxD+hWmMMcYk4AnTGGOMScAT\npjHGGJNAk5z5Yr6DXEMPaZhcj2YM1ZhmSVgHaizUJ0Of8R48j7GpZ0+SkMYZO/czlub17MeYDptC\nU309dxYcQ2oW1Ek4PiENM9YW2ja1GdpITIcLQQ2TMTY5pqwDNTDaITXLkA8ly6DOxnZxLNgGfk89\nKeS/Sq2WpPjV7myoJ7JfaGMxf3GpaHP0w+Q96f99+OGH59LcY5Ci3bEvY3F8Gb/26aefzqX5nNCG\nQ1ogbSbmRx07u5bfx3R2qfjO4LPDe2wL/8I0xhhjEvCEaYwxxiTgCdMYY4xJoEmxZKmTVVVVVSw8\ntJ5N/7fYGX/0BaS+GItvmxIPlfEzqfdRK+D6N/UO1oE+XVLaeZWNYTuZn7os81M3CNWTfparV6+u\nWKfmgppETH+gFkTtTyr2B22ZxOIoU/vheIR0FPoXsh3Uh7gngHXgs0HtNhQfk88f2xmLd0vYbo5V\nSK9kvWNnke4KYrbOvkw545bvFT6jfDcxTT2RpMQS5vjQxqjd0W+d31OzZB1Cfre081jsWLaDNsX3\nFN9t1Eylos3RN5Q+rdXV1cG6+RemMcYYk4AnTGOMMSYBT5jGGGNMAhU1zJUrV+bS1PZi8RZDWhL1\nidh5ltRYQufrNYbr2e3bty/k4Ro69Quud1NvpKbJNNsd0oHYLmpeMb/LWOzDmEYnFeNhsg6heKi7\nghdffDGXjvmssp60Q6k4howlSbvh9xxDfk9COhw1SfY39SCWwWeD37MfQs8KNcxYPFrqcDEtnb6A\n1Iak4j4EvkdC743mhv1ALY/pWGzhUB7eo127drk047byPRXTyakvSsVngePJOg0fPjyXZrtjsYZ5\nZqdUfBexDmwXdW8+e7yehJ497jPh+yD17F//wjTGGGMS8IRpjDHGJOAJ0xhjjEmgoobJdXlqR/Qr\nSvFN4no2z79kmYzTyjV56iGE8RelomZJLYBr6tQoqXFS1+G6fuicOhLzE4z5WTJN/SqkeXK8qDdt\nT/zZnQG1H445dThqYPxekurr63Np9ge/J9Q8qLPxnvQNleL6O+2O7aJ+FPNfDGm51Bhpm7R13oM2\nwX7h2PFZkYp2xnvGfD+bgxUrVuTS8+bNy6XZT7GxkoqxYFPjlW4r/4ABA3Lp+fPn59I8L1Mqvh+p\nk3Is2C6OH+2+trY2lw6962i3fP/y3E/WgWXy/Zrif0wWL16cS9Mv036YxhhjzA7gCdMYY4xJwBOm\nMcYYk4AnTGOMMSaBipt+KDpzI0psk09o00hsI0ls8wqds2OHO4c2u3CTz8EHH5xLx4Kts8zu3bvn\n0uwXli/Fg60T9kMsODg3Y4SCELBdrNPu2vTDzQt1dXW5dL9+/XJpBlsO9S03K9AGaEdMc/NL7FAA\n2pAUP+ycZaQ4xzeGG49CQa5jB0TzHrHnkZuIYhuTQp/9IwTM4EYUtpvBNLgRhe8AqWjHHA8+fwxc\n3q1bt1yaNhULPBKqJzdU8dmJbcqkfTBYAgMEhMpgms8S7xF7l7GNtEGpaHN81kKb9EL4F6YxxhiT\ngCdMY4wxJgFPmMYYY0wCFTXM2EHKMR2NzvYhYtocHYKpA8WCs4d0Vl7DdlFzZLtiB06TUD/E2kF4\nD+oVXOdnOtQPbDf7OuZo31xQg+CBsdQjQsEpYlA/4iG11HpCukhjUmyd/Us7ow3wntR6mD+mkUlF\nu4kFDYhpntTpWOeQIzu1OupeseDazUFNTU0uzcDo99xzTy4de/dJ8QOgqbUPGzYsl2ZfU6OmfhjS\nMJ9++ulcmuO1aNGiXJrBFtgvhGMZ0s2fffbZXJoHF1CT5H6DWKACtikUuIDvPwYq6NOnT+GaEP6F\naYwxxiTgCdMYY4xJwBOmMcYYk0BFDZM+OfSPoW7D/KHAy9RVYgcjU0fj2nPs4M+Qdkd9j9oAdZ2Y\nzyT1RR66GtI3YpoX60ioV/AeMS1YKvYd9Qi2Y1cxcODAXJo+ctQb2TYeQCvFA5Xze9oEDwGgjzJt\nJGQztH36ftGnkc8PtT5qWBzz0PPHdlNzDOmejaE+FDs8vUuXLoUy2Df0BQz5sDY39A3k2Jx88sm5\nNIOYP/PMM4UyY9obtVqO7wsILJ3WAAAO3klEQVQvvFAxP8fuhBNOKNSB7+zHH388l6aOzjTvyWdv\nyZIluXTID5MB3qmT0k6beiAACb1b+WzQr5Za7bb2pfgXpjHGGJOAJ0xjjDEmAU+YxhhjTAJN8sOk\nv1vsMNnQOnBMo4yV0blz51x6e3wHqaFQO4rVIRZvk2vyIR2V9eQ6PesQi/vKe1BPDulCjKfIe65c\nuTKXLpVKhTKaA+oN1FWoL1LPDfmjMe4mdRWOKctoqq9nyB+N/ct2UcujHx+1HWo5tMOQHslrqDny\nWaBmye/pt0nNK+RfTL+72IHBuwLaFNtJfZ/9yPxS3Lc3ZCONoS/o9OnTc+ljjjkml54zZ06hDOrc\n3APCZ5rjxf0AS5cuzaXpnxp6z8QOpaaNUZtlP7GOLD8UizhWJp89a5jGGGPMDuAJ0xhjjEnAE6Yx\nxhiTQEUNk/5v9G+hrxL1xZD/IbW7WExU6mwxv6GUGKCsF6+hzx3X5WPnZfL7lDM5uWbOa2KxZ2M+\neCHfJOo2HJuYj2tzcfTRR+fS9Jmitkc9KaSBcYzZX7EYq9SoqBGH/PAI9SNqL/yedkW9idezX0J6\nPvWdkG02JnZO4PacbUmdM+aPuiuInXfKZ4V9H+rH2DmO1I/pw8g6cXwfeeSRXHrw4MGFOtD3M6YP\nU7Pk+5Nt4N6AZcuWFcpkO9hO+mWyr9lP//RP/5RL055CNkc9meeX0ue8urq6UIbkX5jGGGNMEp4w\njTHGmAQ8YRpjjDEJVNQwP/WpT+XSMf+XWBxKKe7jSG2I38fiuh522GG5dEjHoTbLetNHj7oMffZi\ncV9D+gY/i/Ul/U2ps1IXYJ1D/UBNJKab7irYVupJ1B+oN4Z0N/pRUpuhbsLvqdVwzFlH6i5S0W44\n5hzDbt265dLU9pg/5qsrFcecdkK9if5qMX/VmH+wFD9jkz5xuwJqe7///e9z6ZivYCh+MZ8nnttI\nm/zzn/+cS1N/pF1Tq6PfplR8nxLaA+PX0o+Tds7xpr+qVBzPmN80x4I2yP0IbHfo2WM7+bzHzv0s\n41+YxhhjTAKeMI0xxpgEPGEaY4wxCVQU36jTUFOhPlFXV5dLh87j45o6z6GLnYUX0wtjGqhU9Emk\nD07sjM5QfMzGcF0/1CZqQTG/StaR6Vhs2lAdYuddxvTi5qJ9+/a5NDUJ6keLFi3KpUN67VtvvZVL\nUx+i/kfdhPoRdRbqKKEz+6iDEWqUzz//fMU6URuKxSaVin1HW4/5SMb8EWNxmqXi88S+3B2xZNnX\n7Ieqqqpcmu+6kG5Om6JvH+Oyrlq1KpfmO4DPAcc75ANJTZF9S5vk+FJ3pW7K54jPmVR81/H5ZDti\nNkTNk/s5Qs8/x2fx4sW5NM+33Rb+hWmMMcYk4AnTGGOMScATpjHGGJNARUGQa8/UwejPyHX9lPiK\n1BNj51lynZ7XM95m6BzD2JmcMX9TxiWk5sZ1fuohUlFbpd4Ri/EZi7lLbSqk/TJWbMo1uwL2L3U3\najnUMHi9VNRaCMeD+ambUHeh1hPS72NQ/6G/GW2EdsU6U4+Smn7eZUyjjPlhhs4mpb5L26V/6a4g\nFlt07dq1ufSRRx6ZS4eeFepisZiqtBnqjbwn330pNsc68dmqr6/Ppfk+ZZ3o3xjyxY+9y2iDtKGY\nVs/nJDQWfL4/+clP5tJ8h28L/8I0xhhjEvCEaYwxxiTgCdMYY4xJwBOmMcYYk0DFXR0x53gGCWA6\nJL7GNtRQnOXGBW7iYXncABIiFkydm2EoZMfaHTucOATvwTLYl7GgArFABiEozscCGzQX3IgQcwDn\n5omUw5y5cYhpbm6gjXCDRSyguFTcnMANFiyD7eaYxg7zDcF6xZ4nprnRKBZIO2SndCJv6oa35oA2\nF9tUl3K4OstkXzKwOe2B78KYDfL6EBw/bkSKBTLg2HATZmxznVS0Cd6TzzPbzU1ArFPXrl0L9+RG\nIW6wSz1owr8wjTHGmAQ8YRpjjDEJeMI0xhhjEqioYVKPoKZF7YH5Qw77JBaonIcJx0g5xJqBsjt1\n6lSxTjxQmjoptQSuuYegntHUw5t5fUq7CbXX2KHVuwrqINRmYhp0SMtbsWJFLh0LIh4LfE3th+MX\nCoTOMpYsWZJLx7Rb6kV0tk7Rj+hozmeU9eb3MbtjOqSdU3slsSD1zQHryXbyWaGOFtJq+Z5g0AB+\nz77n80ebix2CLhXf2YcddlguzcMvON7sF44NbS506EAsIArLTNmH0hi2O6TlcjwZoIPP1nHHHRe8\nl39hGmOMMQl4wjTGGGMS8IRpjDHGJFBRw4z5GsV0t5AvGtf6eU2qP0wZrodzzT3kC8r1a15DTZJ1\niuk4DIqccngzr6Eux36jvkEfPbY71A8cP7Y7dPj2riAWfJs6CTUM6nRS0e+KvoD0iYtpedQjSeiQ\ncdabeagHMU3NkjZDXY2HB0vxQ6ZjhzfzmaYemeJDGQvQHtLBmhvqZvSZjfkjhuAhALRr2mSHDh0q\npqn1sZ9COiqfe47vwQcfnEtTF+XYdOvWLZfm+LPfpOLh3CyTzx7fQzF74fU87FuSampqcmn2Q8yv\nvYx/YRpjjDEJeMI0xhhjEvCEaYwxxiRQUcPs3LlzLk0dh2vy1OpSfANjflwx30CuPdOviGvyoWu4\nDh/TUWOHYJOQlhs7tJoaCfXEmN8Y9efYwdyhe6b4kzYHHDNqNbQJjhf1SamoWTA2KPuTMVapH7Fv\nQnZGYnliPskxbXd7/J5j8U5j+iLtihpWSAen/suD57nHYFdA/TfmC8h3QMi3NOZPyrE49NBDc2n6\nE9MfnGm2QYq/Fwj9EWP9wjTfvylQs6SNsUzaVEosYvYV9wOE3hkh/AvTGGOMScATpjHGGJOAJ0xj\njDEmgYoaJteCqWny+5gvYApcr2YZ/D7my5ai3XFNPFZvlsl+SPHRou7JdlC/it0zdn0I6lPsB2oL\nuwrqItSCYvptyAeSehH1vpguGoubTF+wkH4V07RicToZi5S+gdQPQ76izNO9e/eKdWC/xe7B60Nj\nEYuZuzv8MNku+kjSt4/tDvnxcfx4DTVLji/LpA8k+/HZZ58t1CEWC5ZaPP10Y7G9WWfG3A3Besb8\nSfnOZ37uTwjZHH1au3Tpkkun7EGQ/AvTGGOMScITpjHGGJOAJ0xjjDEmgYpiXSyWKLUH5k+JC0st\nLua7Se2P69n0hwr5QMY0Rq6Bx85zo9bH60P6RkynCdW7EqF1+8aE2hy7x+6KJRvTXWgj1E1CegTL\niMXAZBn004qdFRqyY/Z3LD4p9SSWGYuxGeqHmEZFnY3pWCxaErJz9jVtN9UnbmcS86ulD2xK7FHq\nZiyT37Pd9BVmbFrGCqbGKUlLly7NpWlzLJP2QJ9Ytpt1YHlS8QxO+jjzfcp+4f4N6qasI7VhqWi3\nfKezr7eFf2EaY4wxCXjCNMYYYxLwhGmMMcYkUFHDpDb0pz/9KX8x1p6pRYRiW1KHYRxJ3pPaAtee\n6aPD9fEQXKfnPekHGDv/MkZIK2QZsdiyzM9+YZt4z5CGGYtX29SzSZsLaj/0byOh72M+w9Q4O3Xq\nlEtTA2GdeH3oXEDaJvOwv3kPajX8ns9bKLYo89COGHMzFt825r8WGouY313sTM7mIBY3+cUXX8yl\nqQWm1Pnwww/PpamD04b4bmSc1z/84Q+59IoVKwr3pPbKsykZW5bt4nPCdrLfeL1U1DVj+1BCz05j\n2G+sw8svv1y4hs8W+5JzQHV1dfDe/oVpjDHGJOAJ0xhjjEnAE6YxxhiTQEUNk/6FsfilKX57XCum\nJhKL40rNkmvqPOeMMSFDNPU8zNj5a9sTS5br8CwjFns2ds+Q7hqL2ZjSjuaAmgb1w9h4hTQQarrU\nA2m7HI+Y310s1mzoM2pWtH3qgzE7C+lHhGVSX4rVkWMRO/8y5B9MXS3mQ7w7YD/R35A2GtPVpaKd\n0seVeydYB2qWHKtQ/GiOF22G78v6+vpCGZXg+Ie0XJZZU1OTS1NrZ7t69uyZS7PO9MsMaffMQxtk\nfFtrmMYYY8wO4AnTGGOMScATpjHGGJOAJ0xjjDEmgYo7bGIHQlNA5oYeblSRipsjYkEE6FhLwZf5\n+X1o48rq1atz6WeeeaZinWIbQmKHDYeCIrPMWBBswvyxdMrGipUrV+bS3AS0q+AGCtadm1u4gScE\nN+3EAubzgFnaEctj4OzQph9ewzHmBjXWiRs41qxZk0tzgw4PtZaKARl4T26GiG364WaYWAANqdj3\nDJadMp47m9hByRxfbsh57bXXCmXGNkGybzmetDn2S9euXXPp0IHvU6dOzaW5YY4HY3NzDG20RYsW\nFcsLbX7KsiyXjgV+iW2gYt9zrEIbjzg+tFsG7NgW/oVpjDHGJOAJ0xhjjEnAE6YxxhiTQEUNk2vN\nTXUwTjm0mPegZhkLIsDyqLOGtAW2g1otob5BDZNaLbWqUGBntpPENMzYwdvUXEIBIdhuaq0hTWRX\nQAfumPZH/SFkM9RRCO0kdmAw7Y46XCgoOR2qqb0ccsghuTTHkHXgmLNfQodcU9+hU3nMjvgsxA69\npsYlFTVo9uWSJUty6e7duxfK2Nmw3dR/aQ8kpPfTBphmcHWOBceTwdv5/LI8Ka69Mhh/LGgI4diF\n9EdqubwmFlSENst+4fXc1yIV39Hsh9SD0f0L0xhjjEnAE6YxxhiTgCdMY4wxJoGKGubcuXNz6UGD\nBuXS9AOiT1dIw6QGEgtkzvzUmmL5Q/ohtTmu89MfkVAboiaacuA09QfWm+v21H1i+nJKQHm2k2X8\n7Gc/y6XHjh1bKKM5ePLJJ3PpmBZEvSmkvcYCVVNnoY8cNSzqjSSkYdLHjfegrcYC8lMv6tChQ8X8\noWt4D9pNLMA79SamQ9peTJOeP39+Ln3aaacVytjZUD9s165dLk2NK0W7I9QHqavRJ5JafuzgCY6/\nVBxPvus4niwjdoA438ehwy74rqMmzWeF9+BzwgMD2KaQHskyqHPOmTMnl/7MZz5TKEPyL0xjjDEm\nCU+YxhhjTAKeMI0xxpgEWmQxBzVjjDHG+BemMcYYk4InTGOMMSYBT5jGGGNMAp4wjTHGmAQ8YRpj\njDEJeMI0xhhjEvh/bIHItqmLRW4AAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 576x396 with 3 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "UaE2_6HUCAOw",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "We now train the descriptor model and save the weights afterward."
      ]
    },
    {
      "metadata": {
        "id": "QPyc8as42WTQ",
        "colab_type": "code",
        "outputId": "69ea09ee-8198-42d5-a654-2bd3e339e5b2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 122
        }
      },
      "cell_type": "code",
      "source": [
        "epochs = 1\n",
        "### As with the denoising model, we use a loop to save for each epoch \n",
        "## #the weights in an external website in case colab stops. \n",
        "### reset, so e.g. calling 5 times fit(epochs=1) behave as fit(epochs=5)\n",
        "\n",
        "### If you have a model saved from a previous training session\n",
        "### Load it in the next line\n",
        "# descriptor_model_trip.set_weights(keras.models.load_model('./descriptor.h5').get_weights())\n",
        "# descriptor_model_trip.optimizer = keras.models.load_model('./descriptor.h5').optimizer\n",
        "\n",
        "for e in range(epochs):\n",
        "  \n",
        "  descriptor_history = descriptor_model_trip.fit_generator(generator=training_generator, epochs=1, verbose=1, validation_data=val_generator)\n",
        "  \n",
        "  ### Saves optimizer and weights\n",
        "  descriptor_model_trip.save('descriptor.h5') \n",
        "  ### Uploads files to external hosting\n",
        "  !curl -F \"file=@descriptor.h5\" https://file.io\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/1\n",
            "1999/2000 [============================>.] - ETA: 0s - loss: 0.2180"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            " 93%|█████████▎| 93058/100000 [00:03<00:00, 31816.45it/s]"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r2000/2000 [==============================] - 162s 81ms/step - loss: 0.2180 - val_loss: 0.2352\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "100%|██████████| 100000/100000 [00:03<00:00, 25025.97it/s]\n",
            "100%|██████████| 10000/10000 [00:00<00:00, 61048.19it/s]\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "{\"success\":true,\"key\":\"4TMbf9\",\"link\":\"https://file.io/4TMbf9\",\"expiry\":\"14 days\"}"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "NJ-r9D4hDxij",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Generating descriptors files for test data \n",
        "\n",
        "To evaluate the performance of out model we will use an existing evaluation code, which is called HPatches benchmark. HPatches benchmark takes as input the descriptors for the test data in a CSV form. So the whole pipeline is represented in the following image.\n",
        "\n",
        "![](https://i.ibb.co/WcDDf3q/Screenshot-from-2019-02-15-11-17-24.png)\n",
        "\n",
        "This function generates those files by passing it a descriptor model and a denoising model. It performs a first step of denoising the patches, and a second one of computing the descriptor of the denoised patch. If no denoising model is given (variable set to `None`), the descriptor is computed directly in the noisy patch.\n",
        "\n",
        "Similarly to the loading data part, you have the denoise_model variable and `use_clean` variable. If `use_clean` is set to True, the CSV generated will be those of the clean patches, even if a denoising model is given. If set to False, then depends on the variable `denoise_model`. If there is no denoise model (`denoise_model=None`), then it will use the noisy patches. If you give a denoising model, then it will compute the CSV for the denoised patches. This can be useful to explore different scenarios (for example, the Upper Bound can be training the descriptor network with clean patches, and testing with clean patches), however you should always report the score when using noisy patches (depending on the approach you develop, you may want to denoise them or not). The official baseline uses the denoised patches. "
      ]
    },
    {
      "metadata": {
        "id": "kiJb2XDG9bsJ",
        "colab_type": "code",
        "outputId": "44209aa0-be20-4c05-8a39-2264b61cb08e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "generate_desc_csv(descriptor_model, seqs_test, denoise_model=denoise_model, use_clean=False)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "100%|██████████| 40/40 [06:35<00:00, 11.25s/it]\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "metadata": {
        "id": "s0jFr05rE1oI",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Evaluating descriptors in HPatches Benchmark\n",
        "We use HPatches benchmark code to compute the results for our model. \n",
        "\n",
        "**Updated**: The necessary code is included in the repository we cloned at the beginning of the code, so we do not need to download any extra data. Also, we simplified the results, so now they only return one value for each of the three tasks."
      ]
    },
    {
      "metadata": {
        "id": "YvOGRh3sc9Wo",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Now we will perform the evaluation of three different tasks (Verification, Matching and Evaluation) using the CSV files we generated as input and the `hpatches_eval.py` script. We also print the results using the `hpatches_results.py` script. The scripts will return a score for each of the tasks. The metric used is called mean Average Precision, which it uses the Precision of the model. The Precision is defined, for a given number of retrieved elements, as the ratio of correct retrieved elements / number of retrieved elements. [Link to Wikipedia with Precision explanation](https://en.wikipedia.org/wiki/Precision_and_recall). The definition of the three different tasks is taken from the [HPatches paper](https://arxiv.org/pdf/1704.05939.pdf).\n",
        "\n",
        "In all of the tasks if you use the optional argument `--more_info` in `hpatches_results.py` you can see extra mAP information. However, the important score is the mAP score reported without this flag.\n",
        "\n",
        "### Verification\n",
        "\n",
        "Patch verification measures the ability of a descriptor to classify whether two patches are extracted from the same measurement. Now we compute the score of our architecture in this task.\n",
        "\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "Awnyv4xTYSFH",
        "colab_type": "code",
        "outputId": "be06d7d6-203a-44ac-9d00-bf10fe2c997e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 238
        }
      },
      "cell_type": "code",
      "source": [
        "!python ./hpatches-benchmark/hpatches_eval.py --descr-name=custom --descr-dir=/content/keras_triplet_descriptor/out/ --task=verification --delimiter=\";\"\n",
        "!python ./hpatches-benchmark/hpatches_results.py --descr=custom --results-dir=./hpatches-benchmark/results/ --task=verification\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\n",
            ">> Running HPatch evaluation for \u001b[34mcustom\u001b[0m\n",
            ">> Please wait, loading the descriptor files...\n",
            ">> Descriptor files loaded.\n",
            ">> Evaluating \u001b[32mverification\u001b[0m task\n",
            "Processing verification task 1/3 : 100% 1000000/1000000 [01:32<00:00, 10865.68it/s]\n",
            "Processing verification task 2/3 : 100% 1000000/1000000 [01:30<00:00, 10992.03it/s]\n",
            "Processing verification task 3/3 : 100% 1000000/1000000 [01:31<00:00, 10900.89it/s]\n",
            ">> \u001b[32mVerification\u001b[0m task finished in 284 secs  \n",
            "\u001b[32mVerification\u001b[0m task results:\n",
            "Mean Average Precision is 0.714930\n",
            "\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "5290Bw-udJdr",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Matching\n",
        "Image matching, tests to what extent a descriptor can correctly identify correspondences in two images."
      ]
    },
    {
      "metadata": {
        "id": "EUqpwi87ckJv",
        "colab_type": "code",
        "outputId": "bae8823a-85dd-4c72-d162-bc294b20acb8",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 238
        }
      },
      "cell_type": "code",
      "source": [
        "!python ./hpatches-benchmark/hpatches_eval.py --descr-name=custom --descr-dir=/content/keras_triplet_descriptor/out/ --task=matching --delimiter=\";\"\n",
        "!python ./hpatches-benchmark/hpatches_results.py --descr=custom --results-dir=./hpatches-benchmark/results/ --task=matching\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\n",
            ">> Running HPatch evaluation for \u001b[34mcustom\u001b[0m\n",
            ">> Please wait, loading the descriptor files...\n",
            ">> Descriptor files loaded.\n",
            ">> Evaluating \u001b[32mmatching\u001b[0m task\n",
            "100% 40/40 [02:07<00:00,  4.29s/it]\n",
            ">> \u001b[32mMatching\u001b[0m task finished in 127 secs  \n",
            "\u001b[32mMatching\u001b[0m task results:\n",
            "Mean Average Precision is 0.104292\n",
            "\n",
            "\n",
            "\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "RXXgbN7DdMnx",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Retrieval\n",
        "Retrieval tests how well a descriptor can match a query patch to a pool of patches extracted from many images."
      ]
    },
    {
      "metadata": {
        "id": "ZNmKIat1cn_M",
        "colab_type": "code",
        "outputId": "7e970d03-b631-4ab9-9a84-452031fc6ded",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 275
        }
      },
      "cell_type": "code",
      "source": [
        "!python ./hpatches-benchmark/hpatches_eval.py --descr-name=custom --descr-dir=/content/keras_triplet_descriptor/out/ --task=retrieval --delimiter=\";\"\n",
        "!python ./hpatches-benchmark/hpatches_results.py --descr=custom --results-dir=./hpatches-benchmark/results/ --task=retrieval"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\n",
            ">> Running HPatch evaluation for \u001b[34mcustom\u001b[0m\n",
            ">> Please wait, loading the descriptor files...\n",
            ">> Descriptor files loaded.\n",
            ">> Evaluating \u001b[32mretrieval\u001b[0m task\n",
            ">> Please wait, computing distance matrix...\n",
            "tcmalloc: large alloc 1600004096 bytes == 0x56234abbe000 @  0x7fd73548d1e7 0x7fd72b08ecf1 0x7fd72b0f13a2 0x7fd72b0f30de 0x7fd72b18a0e8 0x562323519fe5 0x56232350fd0a 0x5623235175fe 0x562323517232 0x56232350fd0a 0x562323517c38 0x56232350fd0a 0x56232350f629 0x56232354061f 0x56232353b322 0x56232353a67d 0x5623234e91ab 0x7fd73508ab97 0x5623234e8a2a\n",
            ">> Distance matrix done.\n",
            "Processing retrieval task: 100% 10000/10000 [03:57<00:00, 42.23it/s]\n",
            ">> \u001b[32mRetrieval\u001b[0m task finished in 276 secs  \n",
            "\u001b[32mRetrieval\u001b[0m task results:\n",
            "Mean Average Precision is 0.358950\n",
            "\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "8_2fBzUB5RF2",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Compressing and saving the CSV files \n",
        "\n",
        "This is not necessary for the analysis of the baseline code included in the report. However, we will be hosting a competition in an external website to see who can achieve the highest score. In that case, you will need to submit the CSV files, as the scoring script will be performed in an external server. With that aim, we include here a way to save the files either in your local disc or in your google drive account.\n",
        "\n",
        "We first compress the directory with all the CSV by using the following command. Remove the `q` option if you want it to output the progress."
      ]
    },
    {
      "metadata": {
        "id": "Lh_svT3p5Ww-",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "!zip -rq descriptors.zip ./out/custom"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "svoL779J8AJK",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "The generated .zip is quite large, the method we used for the weights does not work. We have two other methods. First, in the file explorer in the left column we can right-click in the file and then click download. Then, we will see a circle next to the file showing the download progress.\n",
        "\n",
        "The second way does not require for you to download the files, it save the zip file in your Google Drive account, and you can download it later to your machine if you want. To do so, follow this method (found [here](https://stackoverflow.com/questions/49428332/how-to-download-large-files-like-weights-of-a-model-from-colaboratory)). First run the next cell, and the output will be a link for authentication purposes, and just follow the instructions"
      ]
    },
    {
      "metadata": {
        "id": "RjOmPv5z7Opx",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "from google.colab import auth\n",
        "from googleapiclient.http import MediaFileUpload\n",
        "from googleapiclient.discovery import build\n",
        "\n",
        "auth.authenticate_user()\n",
        "drive_service = build('drive', 'v3')\n",
        "\n",
        "def save_file_to_drive(name, path):\n",
        "  file_metadata = {\n",
        "    'name': name,\n",
        "    'mimeType': 'application/octet-stream'\n",
        "  }\n",
        "\n",
        "  media = MediaFileUpload(path, \n",
        "                          mimetype='application/octet-stream',\n",
        "                          resumable=True)\n",
        "\n",
        "  created = drive_service.files().create(body=file_metadata,\n",
        "                                  media_body=media,\n",
        "                                  fields='id').execute()\n",
        "\n",
        "  print('File ID: {}'.format(created.get('id')))\n",
        "\n",
        "  return created\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "YfzjfMc59NKm",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Now we can use the following function to save the file to your drive account. The second argument is the name of the file we want to save, and the first argument the name that will have in your Drive."
      ]
    },
    {
      "metadata": {
        "id": "UwrqWr_c7pAi",
        "colab_type": "code",
        "outputId": "005994a0-bb3c-4c8d-dc93-91fdd94a4e98",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "cell_type": "code",
      "source": [
        "save_file_to_drive('descriptors_save.zip', 'descriptors.zip')"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "File ID: 1dHq0eDqoLMMmr6YOexZOU6cLwhFJYgH8\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{u'id': u'1dHq0eDqoLMMmr6YOexZOU6cLwhFJYgH8'}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 26
        }
      ]
    }
  ]
}