notebooks/239-image-bind/239-image-bind-quantize.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "4ad832c5-71db-456f-9702-fb8f3b489e46",
   "metadata": {},
   "source": [
    "# Post-Training Quantization of ImageBind model with NNCF\n",
    "\n",
    "The goal of this tutorial is to demonstrate how to speed up the model by applying 8-bit post-training quantization from [NNCF](https://github.com/openvinotoolkit/nncf/) (Neural Network Compression Framework) and infer quantized model via OpenVINO™ Toolkit. \n",
    "\n",
    "The optimization process contains the following steps:\n",
    "1. Quantize converted OpenVINO model from [notebook](239-image-bind-convert.ipynb) with NNCF.\n",
    "2. Compare probability matrices between converted and quantized models on input data examples.\n",
    "3. Compare model size of converted and quantized models.\n",
    "4. Compare performance of converted and quantized models.\n",
    "\n",
    "> **NOTE**: you should run [239-image-bind-convert](239-image-bind-convert.ipynb) notebook first to generate OpenVINO IR models that are used for quantization.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57ec0370",
   "metadata": {},
   "source": [
    "#### Table of contents:\n",
    "- [Prerequisites](#Prerequisites)\n",
    "- [Create and initialize quantization](#Create-and-initialize-quantization)\n",
    "    - [Prepare datasets](#Prepare-datasets)\n",
    "- [Compare results for the OpenVINO FP16 model and the quantized model](#Compare-results-for-the-OpenVINO-FP16-model-and-the-quantized-model)\n",
    "    - [Select inference device](#Select-inference-device)\n",
    "        - [Compare File Size](#Compare-File-Size)\n",
    "        - [Compare inference time of the FP16 IR and quantized models](#Compare-inference-time-of-the-FP16-IR-and-quantized-models)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5487fe23-6122-47ca-9c29-3beb7802b292",
   "metadata": {},
   "source": [
    "## Prerequisites\n",
    "[back to top ⬆️](#Table-of-contents:)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "13473357",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33mDEPRECATION: omegaconf 2.0.6 has a non-standard dependency specifier PyYAML>=5.1.*. pip 23.3 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of omegaconf or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063\u001b[0m\u001b[33m\n",
      "\u001b[0m\u001b[33mDEPRECATION: pytorch-lightning 1.6.5 has a non-standard dependency specifier torch>=1.8.*. pip 23.3 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pytorch-lightning or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063\u001b[0m\u001b[33m\n",
      "\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.2.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.3.1\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install -q datasets librosa soundfile \"openvino>=2023.1.0\" \"nncf\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "e1a60d45",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/ea/work/openvino_notebooks/notebooks/239-image-bind/ImageBind\n"
     ]
    }
   ],
   "source": [
    "from pathlib import Path\n",
    "\n",
    "repo_dir = Path(\"ImageBind\")\n",
    "\n",
    "if not repo_dir.exists():\n",
    "    raise RuntimeError('This notebook should be run after 239-image-bind-convert notebook')\n",
    "\n",
    "%cd {repo_dir}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "488397fd",
   "metadata": {},
   "source": [
    "## Create and initialize quantization\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "[NNCF](https://github.com/openvinotoolkit/nncf/) enables post-training quantization by adding the quantization layers into the model graph and then using a subset of the training dataset to initialize the parameters of these additional quantization layers. The framework is designed so that modifications to your original training code are minor. Quantization is the simplest scenario and requires a few modifications.\n",
    "\n",
    "The optimization process contains the following steps:\n",
    "1. Create a Dataset for quantization.\n",
    "2. Run `nncf.quantize` for getting a quantized model.\n",
    "3. Serialize the `INT8` model using `openvino.save_model` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "810e2dd7",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ea/work/ov_venv/lib/python3.8/site-packages/torchvision/transforms/functional_tensor.py:5: UserWarning: The torchvision.transforms.functional_tensor module is deprecated in 0.15 and will be **removed in 0.17**. Please don't rely on it. You probably just need to use APIs in torchvision.transforms.functional or in torchvision.transforms.v2.functional.\n",
      "  warnings.warn(\n",
      "/home/ea/work/ov_venv/lib/python3.8/site-packages/torchvision/transforms/_functional_video.py:6: UserWarning: The 'torchvision.transforms._functional_video' module is deprecated since 0.12 and will be removed in the future. Please use the 'torchvision.transforms.functional' module instead.\n",
      "  warnings.warn(\n",
      "/home/ea/work/ov_venv/lib/python3.8/site-packages/torchvision/transforms/_transforms_video.py:22: UserWarning: The 'torchvision.transforms._transforms_video' module is deprecated since 0.12 and will be removed in the future. Please use the 'torchvision.transforms' module instead.\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "from pathlib import Path\n",
    "from imagebind.models.imagebind_model import ModalityType\n",
    "\n",
    "modalities = [ModalityType.TEXT, ModalityType.VISION, ModalityType.AUDIO]\n",
    "fp_model_paths = {modality: Path(f\"image-bind-{modality}\") / f\"image-bind-{modality}.xml\" for modality in modalities}\n",
    "int8_model_paths = {modality: Path(f\"image-bind-{modality}\") / f\"image-bind-{modality}_int8.xml\" for modality in modalities}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f261e6a4",
   "metadata": {},
   "source": [
    "### Prepare datasets\n",
    "[back to top ⬆️](#Table-of-contents:)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65d8d559",
   "metadata": {},
   "source": [
    "The [Conceptual Captions](https://ai.google.com/research/ConceptualCaptions/) dataset consisting of ~3.3M images annotated with captions. Dataset is used to quantize image and text models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "40d71747",
   "metadata": {},
   "outputs": [],
   "source": [
    "import imagebind.data as data\n",
    "import os\n",
    "import requests\n",
    "import tempfile\n",
    "\n",
    "from requests.packages.urllib3.exceptions import InsecureRequestWarning\n",
    "requests.packages.urllib3.disable_warnings(InsecureRequestWarning)\n",
    "\n",
    "def check_text_data(data):\n",
    "    \"\"\"\n",
    "    Check if the given data is text-based.\n",
    "    \"\"\"\n",
    "    if isinstance(data, str):\n",
    "        return True\n",
    "    if isinstance(data, list):\n",
    "        return all(isinstance(x, str) for x in data)\n",
    "    return False\n",
    "\n",
    "def collate_fn(examples, image_column=\"image_url\", text_column=\"caption\"):\n",
    "    \"\"\"\n",
    "    Collates examples into a batch for processing.\n",
    "    Preprocesses each example by loading and transforming image and text data.\n",
    "    Checks if the text data in the example is valid by calling the `check_text_data` function.\n",
    "    Downloads the image specified by the URL in the image_column of the example dictionary.\n",
    "    Constructs and returns a dictionary representing the collated batch with the following keys:\n",
    "       - \"pixel_values\": The pixel values of the preprocessed example.\n",
    "       - \"input_ids\": The transformed text data of the preprocessed example.\n",
    "    \"\"\"\n",
    "    assert len(examples) == 1\n",
    "    example = examples[0]\n",
    "    if not check_text_data(example[text_column]):\n",
    "        raise ValueError(\"Text data is not valid\")\n",
    "\n",
    "    url = example[image_column]\n",
    "    with tempfile.TemporaryDirectory() as tempdir:\n",
    "        f_name = os.path.join(tempdir, 'image.jpg')\n",
    "        try:\n",
    "            response = requests.get(url, verify=False, timeout=20)\n",
    "            with open(f_name, \"wb\") as file:\n",
    "                file.write(response.content)\n",
    "            pixel_values = data.load_and_transform_vision_data([f_name], \"cpu\")\n",
    "        except Exception:\n",
    "            print(f\"Can't load image from url: {url}\")\n",
    "            return None\n",
    "\n",
    "    text = data.load_and_transform_text([example[text_column]], \"cpu\")\n",
    "\n",
    "    return {\n",
    "        \"pixel_values\": pixel_values,\n",
    "        \"input_ids\": text\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "befc2204",
   "metadata": {},
   "outputs": [],
   "source": [
    "from datasets import load_dataset\n",
    "import itertools\n",
    "import torch\n",
    "from tqdm.notebook import tqdm\n",
    "\n",
    "def collect_vision_text_data(dataloader, init_steps):\n",
    "    \"\"\"\n",
    "    This function collects vision and text data from a dataloader for a specified number of initialization steps.\n",
    "    It iterates over the dataloader, fetching batches and storing the relevant vision and text data.\n",
    "    Returns a tuple containing the collected vision_data and text_data lists.\n",
    "    \"\"\"\n",
    "    text_data = []\n",
    "    vision_data = []\n",
    "    print(f\"Fetching {init_steps} for the initialization...\")\n",
    "    counter = 0\n",
    "    for batch in tqdm(dataloader):\n",
    "        if counter == init_steps:\n",
    "            break\n",
    "        with torch.no_grad():\n",
    "            if batch:\n",
    "                counter += 1\n",
    "                text_data.append(batch[\"input_ids\"].to(\"cpu\"))\n",
    "                vision_data.append(batch[\"pixel_values\"].to(\"cpu\"))\n",
    "    return vision_data, text_data\n",
    "\n",
    "def prepare_vision_text_dataset(opt_init_steps=300):\n",
    "    \"\"\"\n",
    "    Prepares a vision-text dataset for quantization by collecting vision and text data.\n",
    "    \"\"\"\n",
    "    dataset = load_dataset(\"conceptual_captions\", streaming=True)\n",
    "    train_dataset = dataset[\"train\"].shuffle(seed=0)\n",
    "    dataloader = torch.utils.data.DataLoader(train_dataset, collate_fn=collate_fn, batch_size=1)\n",
    "    vision_data, text_data = collect_vision_text_data(dataloader, opt_init_steps)\n",
    "    return vision_data, text_data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c0f6587",
   "metadata": {},
   "source": [
    "The [`ESC-50`](https://github.com/karolpiczak/ESC-50) dataset is used to quantize the audio modality of the ImageBind model.\n",
    "Dataset is a labeled collection of 2000 environmental audio recordings suitable for benchmarking methods of environmental sound classification.\n",
    "The dataset consists of 5-second-long recordings organized into 50 semantic classes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "2a2d4357",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import torchaudio\n",
    "from io import BytesIO\n",
    "\n",
    "def collect_audio_data(dataloader, init_steps=300):\n",
    "    \"\"\"\n",
    "    This function collects audio data from a dataloader for a specified number of initialization steps.\n",
    "    It iterates over the dataloader, fetching batches and storing them in a list.\n",
    "    \"\"\"\n",
    "    audio_data = []\n",
    "    for _, batch in tqdm(zip(range(init_steps), itertools.islice(dataloader, 0, init_steps))):\n",
    "        with torch.no_grad():\n",
    "            audio_data.append(batch)\n",
    "    return audio_data\n",
    "\n",
    "def prepare_audio_dataset():\n",
    "    \"\"\"\n",
    "    Prepares an \"ashraq/esc50\" audio dataset for quantization by collecting audio data.\n",
    "    Collects audio data from the dataloader by calling the `collect_audio_data` function.\n",
    "    Returns a list containing the collected calibration audio data batches.\n",
    "    \"\"\"\n",
    "    audio_dataset = load_dataset(\"ashraq/esc50\", \"clean\", streaming=True)\n",
    "    train_dataset = audio_dataset[\"train\"].shuffle(seed=42, buffer_size=1000)\n",
    "\n",
    "    def collate_fn(examples):\n",
    "        assert len(examples) == 1\n",
    "        with tempfile.TemporaryDirectory() as tempdir:\n",
    "            f_name = os.path.join(tempdir, 'audio.wav')\n",
    "            audio_data, sample_rate = torchaudio.load(BytesIO(examples[0]['audio']['bytes']))\n",
    "            audio_data.to(torch.float32).unsqueeze(0)\n",
    "            torchaudio.save(f_name, audio_data, sample_rate)\n",
    "            return data.load_and_transform_audio_data([f_name], \"cpu\")\n",
    "\n",
    "    dataloader = torch.utils.data.DataLoader(train_dataset, collate_fn=collate_fn, batch_size=1)\n",
    "    calibration_data = collect_audio_data(dataloader)\n",
    "    return calibration_data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "768c5f5f",
   "metadata": {},
   "source": [
    "Create a quantized model from the pre-trained `FP16` model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "4137cd4b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fetching 300 for the initialization...\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "242df870a8da438a9a3b488455ea73f4",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Can't load image from url: http://homeklondike.org/wp-content/uploads/2015/06/2-Bright-living-room-in-the-attic1.jpg\n",
      "Can't load image from url: http://www.lovemeinitaly.com/wp-content/uploads/cache/images/2018/01/4A-e1491723576743/4A-e1491723576743-1964759082.jpg\n",
      "Can't load image from url: https://i0.wp.com/childphotocompetition.com/wp-content/uploads/2016/02/Agnieszka_He%E2%80%8E_childphotocompetition.jpg\n",
      "Can't load image from url: https://magankonoski.com/wp-content/uploads/2016/05/MaganKonoskiFineArtWeddingandLifestylePhotographer-25-683x1024.jpg\n",
      "Can't load image from url: http://www.huahin-home-property.com/wp-content/uploads/2016/11/2immobilier-real-eatate-huahin-maison-a-vendre-condo-for-salerent-The-Autumm-Khao-takibe.jpg\n",
      "Can't load image from url: http://www.americanclassichomes.com/blog/wp-content/uploads/2015/04/Alki_SB_Kitchen_internet.jpg\n",
      "Can't load image from url: http://assets.nydailynews.com/polopoly_fs/1.110031.1313943805!/img/httpImage/image.jpg_gen/derivatives/article_750/alg-fencer-sara-harvey-browne-2-jpg.jpg\n",
      "Can't load image from url: http://static.panoramio.com/photos/large/34107183.jpg\n",
      "Can't load image from url: https://odis.homeaway.com/odis/listing/2f9f1d46-0559-4811-95ed-c97cc8608793.c10.jpg\n",
      "Can't load image from url: https://odis.homeaway.com/odis/listing/75953842-3278-42a1-91ef-2bb2be2ecb05.c10.jpg\n",
      "Can't load image from url: https://ak6.picdn.net/shutterstock/videos/2504486/thumb/1.jpg\n",
      "Can't load image from url: http://www.buro247.my/thumb/625x960_0/galleries/2017/10/lady-dior-art-2-19.jpg\n",
      "Can't load image from url: http://oneindiaonepeople.com/wp-content/uploads/2014/02/13.jpg\n",
      "Can't load image from url: http://www.johnsoncitypress.com/image/2016/10/27/640x_cCM_q30/XC-Region-A-AA-JPG.jpg\n",
      "Can't load image from url: http://fromthedeckchair.com/wp-content/uploads/2013/06/ftdc_norwegianpearl-0737.jpg\n",
      "Can't load image from url: http://thedailyquotes.com/wp-content/uploads/2015/04/could-be-another-broken-heart-love-daily-quotes-sayings-pictures.jpg\n",
      "Can't load image from url: https://www.popsci.com/sites/popsci.com/files/styles/1000_1x_/public/vizdata_map_key.jpg?itok=7myhqx2P\n",
      "Can't load image from url: https://www.interlatesystems.com/img/1166/183.jpg\n",
      "Can't load image from url: https://i1.wp.com/dailynexus.com/wp-content/uploads/2016/10/HalloweenWeekend_KennethSong-4-1024x671.jpg?resize=1024%2C671\n",
      "Can't load image from url: https://odis.homeaway.com/odis/listing/d81ed29b-f448-444a-9048-ed9cc9fe666a.c10.jpg\n",
      "Can't load image from url: http://exploresrilanka.lk/wp-content/uploads/2016/04/BTI37666.jpg\n",
      "Can't load image from url: http://www.tampabay.com/storyimage/HI/20170528/ARTICLE/305289727/AR/0/AR-305289727.jpg\n",
      "Can't load image from url: http://wewegombel.me/photo/558689/IMG_7994.jpg\n",
      "Can't load image from url: http://www.thedonkeysanctuary.ie/sites/ireland/files/styles/large/public/press/259-1445414098.jpg?itok=dwa9kRh_\n",
      "Can't load image from url: https://thumb1.shutterstock.com/display_pic_with_logo/3816881/478955293/stock-vector-abstract-pattern-in-the-memphis-style-of-large-white-spots-and-little-green-with-black-dots-on-a-478955293.jpg\n",
      "Can't load image from url: http://media.santabanta.com/images/picsms/2016/sms-16401.jpg\n",
      "Can't load image from url: https://lookaside.fbsbx.com/lookaside/crawler/media/?media_id=657209177718359\n",
      "Can't load image from url: http://www.blogbeen.com/wp-content/uploads/2017/09/-mesmerizing-bathroom-tiles-11-jpg-bathroom-full-version-helulis-.jpg\n",
      "Can't load image from url: https://6e58e2e225bb143c019e-e234a4d870c026b5f56b4446f6e62d64.ssl.cf1.rackcdn.com/a9ad7fa8-cf6c-4d2b-bbc6-591e0fd0cb2f.jpg\n",
      "Can't load image from url: http://wewegombel.me/photo/487654/img_8173.jpg\n",
      "Can't load image from url: http://s1.ibtimes.com/sites/www.ibtimes.com/files/styles/lg/public/2011/06/04/109074-an-african-giant-pouch-rat-is-watched-by-his-handler-at-a-laboratory-i.jpg\n",
      "Can't load image from url: http://nnimgt-a.akamaihd.net/transform/v1/crop/frm/w9qsSAVumVxqyCiyw3G2iR/d9d78dda-7d5d-4420-9f3d-a1d44813c251.jpg/r0_64_960_604_w1200_h678_fmax.jpg\n",
      "Can't load image from url: https://www.thenational.ae/image/policy:1.197226:1499310330/image/jpeg.jpg?f=16x9&w=1024&$p$f$w=2589da4\n",
      "Can't load image from url: https://ak4.picdn.net/shutterstock/videos/14101994/thumb/1.jpg?i10c=img.resize(height:160)\n",
      "Can't load image from url: http://sanpancholife.com/photos/home/2386/super/5005683111355530342.jpeg\n",
      "Can't load image from url: https://media.gettyimages.com/photos/two-bottles-of-pills-one-knocked-over-with-contents-spilling-out-and-picture-id73740799?s=612x612\n",
      "Can't load image from url: https://www.thestar.com/content/dam/thestar/entertainment/music/2017/04/17/prince-was-prescribed-oxycodone-under-another-name-court-document/prince-07.jpg.size.custom.crop.891x650.jpg\n",
      "Can't load image from url: http://photos.mycapture.com/TWCM/1473481/41921058E.jpg\n",
      "Can't load image from url: http://xboxhut.com/wp-content/uploads/2016/05/simple-bathroom-designs-grey-modern-double-sink-bathroom-vanities60-37.jpg\n",
      "Can't load image from url: http://seanverret.com/wp-content/uploads/2012/07/20120710_104349.jpg\n",
      "Can't load image from url: http://neveradulldayinpoland.com/wp-content/uploads/2014/04/DSC_3434-1024x682.jpg\n",
      "Can't load image from url: http://wewegombel.me/photo/687156/watercolor-christmas-tree-isolated-white-background-texture-paper-new-year-christmas-card-template-62641882.jpg\n",
      "Can't load image from url: http://expatedna.com/wp-content/uploads/2015/06/City-in-the-sky-by-Expat-Edna.jpg\n",
      "Can't load image from url: https://lookaside.fbsbx.com/lookaside/crawler/media/?media_id=1291121264312721\n",
      "Can't load image from url: https://i0.wp.com/cindi-keller.com/wp-content/uploads/2014/09/cindi-keller_2014-08-15_15.07.29_ronda-spain.jpg?w=400&h=533&crop&ssl=1\n",
      "Can't load image from url: http://www.robinhoodshow.com/clients/17668/8642054_org.jpg\n",
      "Can't load image from url: https://www.101india.com/sites/default/files/image-upload/blogs/TravelandFood/29NovSecretDevkundWaterfalls/Inline%204%20%3C%20Sunrise%20at%20the%20river%20behind%20the%20farmhouse%20%3E.jpg\n",
      "Can't load image from url: http://www.nextavenue.org/wp-content/uploads/2017/05/image-3-w1024-750x485.jpg\n",
      "Can't load image from url: http://nnimgt-a.akamaihd.net/transform/v1/crop/frm/342N54ExNnUCDyWzghgYbSC/cd538c73-466c-4e05-8202-0892dceb8a44.jpg/r401_321_5388_3369_w1200_h678_fmax.jpg\n",
      "Can't load image from url: https://www.universetoday.com/wp-content/uploads/2016/05/Earth-magnetosphere-ESA-Medialab.jpg\n",
      "Can't load image from url: https://c5eeb468edc90bcfda59-8477d1500ace5389b08f6bb1cc2fee82.ssl.cf5.rackcdn.com/837712-residential-x722qn-o.jpg\n",
      "Can't load image from url: https://ak3.picdn.net/shutterstock/videos/7414963/thumb/1.jpg\n"
     ]
    }
   ],
   "source": [
    "vision_data, text_data = prepare_vision_text_dataset()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "1d6e9780",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:nncf:NNCF initialized successfully. Supported frameworks detected: torch, tensorflow, onnx, openvino\n"
     ]
    }
   ],
   "source": [
    "import logging\n",
    "import nncf\n",
    "import openvino as ov\n",
    "\n",
    "nncf.set_log_level(logging.ERROR)\n",
    "\n",
    "core = ov.Core()\n",
    "\n",
    "def quantize_openvino_model(modality, calibration_data):\n",
    "    model_path = fp_model_paths[modality]\n",
    "    if not os.path.exists(model_path):\n",
    "        raise RuntimeError(f\"Model: {model_path} not found. \\\n",
    "                             First run 239-image-bind-convert notebook to convert model to OpenVINO IR.\")\n",
    "\n",
    "    model = core.read_model(model_path)\n",
    "    quantized_model = nncf.quantize(\n",
    "        model=model,\n",
    "        calibration_dataset=calibration_data,\n",
    "        model_type=nncf.ModelType.TRANSFORMER,\n",
    "        # remove ignored_scope for nncf>=2.6.0 (PR with fix https://github.com/openvinotoolkit/nncf/pull/1953)\n",
    "        ignored_scope=nncf.IgnoredScope(types=[\"ReduceL2\"])\n",
    "    )\n",
    "    ov.save_model(quantized_model, int8_model_paths[modality])\n",
    "    return quantized_model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e56bc39",
   "metadata": {},
   "source": [
    "Quantize ImageBind model for vision modality.\n",
    "\n",
    "> **NOTE**: Quantization is time and memory consuming operation. Running quantization code below may take a long time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "3213f8b7",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-10-26 13:34:25.166422: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.\n",
      "2023-10-26 13:34:25.203294: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n",
      "To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
      "2023-10-26 13:34:26.097309: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT\n",
      "Statistics collection: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 300/300 [01:18<00:00,  3.81it/s]\n",
      "Applying Smooth Quant: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 129/129 [00:13<00:00,  9.69it/s]\n",
      "Statistics collection: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 300/300 [03:03<00:00,  1.64it/s]\n",
      "Applying Fast Bias correction: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 128/128 [00:23<00:00,  5.54it/s]\n"
     ]
    }
   ],
   "source": [
    "if len(vision_data) == 0:\n",
    "    raise RuntimeError(\n",
    "        'Calibration dataset is empty. Please check internet connection and try to download images manually from the URLs above.'\n",
    "    )\n",
    "\n",
    "vision_dataset = nncf.Dataset(vision_data)\n",
    "vision_quantized_model = quantize_openvino_model(modality=ModalityType.VISION, calibration_data=vision_dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9db7cc5e",
   "metadata": {},
   "source": [
    "Quantize ImageBind model for text modality"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "7d5eba00",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Statistics collection: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 300/300 [00:17<00:00, 16.82it/s]\n",
      "Applying Smooth Quant: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 97/97 [00:06<00:00, 15.41it/s]\n",
      "Statistics collection: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 300/300 [00:50<00:00,  5.97it/s]\n",
      "Applying Fast Bias correction: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 120/120 [00:10<00:00, 11.06it/s]\n"
     ]
    }
   ],
   "source": [
    "text_dataset = nncf.Dataset(text_data)\n",
    "text_quantized_model = quantize_openvino_model(modality=ModalityType.TEXT, calibration_data=text_dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a27983fe",
   "metadata": {},
   "source": [
    "Quantize ImageBind model for audio modality"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "95540877",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Repo card metadata block was not found. Setting CardData to empty.\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "91de671970d5430fa5051869b1a09464",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Statistics collection: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 300/300 [01:10<00:00,  4.26it/s]\n",
      "Applying Smooth Quant: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 49/49 [00:01<00:00, 27.79it/s]\n",
      "Statistics collection: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 300/300 [01:21<00:00,  3.69it/s]\n",
      "Applying Fast Bias correction: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 48/48 [00:03<00:00, 12.18it/s]\n"
     ]
    }
   ],
   "source": [
    "audio_calibration_data = prepare_audio_dataset()\n",
    "audio_dataset = nncf.Dataset(audio_calibration_data)\n",
    "audio_quantized_model = quantize_openvino_model(modality=ModalityType.AUDIO, calibration_data=audio_dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8af6d642",
   "metadata": {},
   "source": [
    "NNCF also supports quantization-aware training, and other algorithms than quantization.\n",
    "See the [NNCF documentation](https://github.com/openvinotoolkit/nncf/#documentation) in the NNCF repository for more information."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0508c15",
   "metadata": {},
   "source": [
    "## Compare results for the OpenVINO FP16 model and the quantized model\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "Compare the probability matrices for `FP16` and `INT8` models.\n",
    "More details about probability matrix can be found in [notebook](239-image-bind-convert.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "06d49203",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Prepare inputs\n",
    "from imagebind.models.imagebind_model import ModalityType\n",
    "\n",
    "text_list = [\"A car\", \"A bird\", \"A dog\"]\n",
    "image_paths = [\".assets/dog_image.jpg\", \".assets/car_image.jpg\", \".assets/bird_image.jpg\"]\n",
    "audio_paths = [\".assets/dog_audio.wav\", \".assets/bird_audio.wav\", \".assets/car_audio.wav\"]\n",
    "\n",
    "inputs = {\n",
    "    ModalityType.TEXT: data.load_and_transform_text(text_list, \"cpu\"),\n",
    "    ModalityType.VISION: data.load_and_transform_vision_data(image_paths, \"cpu\"),\n",
    "    ModalityType.AUDIO: data.load_and_transform_audio_data(audio_paths, \"cpu\"),\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cfab93b3-f01f-4fc7-ac64-2ffc68c070a9",
   "metadata": {},
   "source": [
    "### Select inference device\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "select device from dropdown list for running inference using OpenVINO"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "059788af-9ec0-4808-aa60-dac04216f4a3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0f65e2986e2a4c73ac0828c503cff91a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Dropdown(description='Device:', index=2, options=('CPU', 'GPU', 'AUTO'), value='AUTO')"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import ipywidgets as widgets\n",
    "\n",
    "device = widgets.Dropdown(\n",
    "    options=core.available_devices + [\"AUTO\"],\n",
    "    value='AUTO',\n",
    "    description='Device:',\n",
    "    disabled=False,\n",
    ")\n",
    "\n",
    "device"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1de57780",
   "metadata": {},
   "source": [
    "Get embeddings of original models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "82e8b0d3",
   "metadata": {},
   "outputs": [],
   "source": [
    "embeddings = {}\n",
    "for modality in modalities:\n",
    "    ov_model = core.compile_model(fp_model_paths[modality], device.value)\n",
    "    embeddings[modality] = ov_model(inputs[modality])[ov_model.output(0)]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1dcf3317",
   "metadata": {},
   "source": [
    "Get embeddings of quantized models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "995c089a",
   "metadata": {},
   "outputs": [],
   "source": [
    "quantized_embeddings = {}\n",
    "for modality in modalities:\n",
    "    model = core.compile_model(int8_model_paths[modality], device.value)\n",
    "    quantized_embeddings[modality] = model(inputs[modality])[model.output(0)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "7668461c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from scipy.special import softmax\n",
    "\n",
    "def visualize_prob_matrices(fp_matrix, int_matrix, x_label, y_label):\n",
    "    fig, ax = plt.subplots(1, 2)\n",
    "    for i, matrix in enumerate([fp_matrix, int_matrix]):\n",
    "        ax[i].matshow(matrix, cmap='winter')\n",
    "\n",
    "        for (k, j), z in np.ndenumerate(matrix):\n",
    "            ax[i].title.set_text('FP16 probs' if i == 0 else 'INT8 probs')\n",
    "            ax[i].text(j, k, '{:0.3f}'.format(z), ha='center', va='center')\n",
    "            ax[i].set_xticks(range(len(x_label)), x_label)\n",
    "            ax[i].set_yticks(range(len(y_label)), y_label)\n",
    "    fig.tight_layout()\n",
    "\n",
    "image_list = [img.split('/')[-1] for img in image_paths]\n",
    "audio_list = [audio.split('/')[-1] for audio in audio_paths]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "ca8620d1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnYAAAD/CAYAAAB8dJvAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABOaklEQVR4nO3dd3gU5dfG8e8mgfSEnkCIhGYABYIgEEDAF5AmxQIWlCIKKqF36QKiIE2qilIEBRtYUJoIP6UjoFKMUiQoVcWEIiFl3j8m2WTTSEKShd37c11zwcyceeZ5srsnZ6fFYhiGgYiIiIjc9lzs3QERERERyRsq7EREREQchAo7EREREQehwk5ERETEQaiwExEREXEQKuxEREREHIQKOxEREREHocJORERExEGosBMRERFxECrsRBzEkiVLsFgs7N27195dERHJUxaLhYiICHt347agwk5uC8lFS0bTiBEjrHEhISE260qVKsV9993H6tWrbdrbvXs3L774IrVr16ZQoUJYLJYs93/u3Dl69+5NUFAQHh4ehISE0LNnz3wZq4g4p4y+nI0fPx6LxUJAQABXr15Nt01ISAgPPvggAN27d880T6aeunfvbt3+ww8/pH79+hQpUoTixYvTpEkT1q5dm+9jlfzjZu8OiOTEyy+/TPny5W2W3X333TbzYWFhDB48GIDTp0/z5ptv8vDDD7NgwQKef/55AL766isWLVpEjRo1qFChAr/++mum+zx16hQNGzYE4PnnnycoKIjTp0+ze/fuvByaiEimzp8/z4IFC6y5LSO9e/emefPm1vkTJ04wduxYevXqxX333WddXrFiRQDmzJlDv379aNu2La+++irXrl1jyZIlPPjgg3zyySc8/PDD+TcgyTcq7OS20rp1a+rUqZNlTFBQEE899ZR1vmvXrlSqVImZM2daC7sXXniB4cOH4+npSURERJaFXe/evXFzc2PPnj0UL148bwaSDYmJiVy/fh0PD48C26eI3JrCwsKYNm0aL774Ip6enhnGhIeHEx4ebp3fu3cvY8eOJTw83CYnJpszZw733nsvX3zxhfWsxTPPPENQUBBLly7N18LuypUreHt751v7zkynYsXhBQYGUrVqVU6cOGFdFhAQkGlyTO2XX37h66+/ZujQoRQvXpxr164RFxeX7X0nn0b55Zdf6Ny5M35+fhQvXpz+/ftz7do1m9jka0hWrFjBXXfdhbu7O+vWrQNg//79tG7dGj8/P3x8fGjWrBk7d+7McJ9Xr16ld+/eFC9eHD8/P7p27crFixdtYvbu3UvLli0pUaIEnp6elC9fnmeeeSbb4xKRgjV27FjOnTvHggUL8qzNmJgYSpUqZXMpSnKOuVF+/P3337FYLLz++uvMnDmTcuXK4enpSZMmTTh48KBNbPfu3fHx8eHYsWO0adMGX19funTpApgF3uDBgwkODsbd3Z3Q0FBef/11DMPIcL8rVqwgNDQUDw8Pateuzf/+9z+b9ZcuXWLAgAGEhITg7u5OqVKlaNGiBfv27cvNj+i2pCN2cluJjo7mr7/+sllWokSJLLeJi4vj1KlTuTratmnTJsAsBJs1a8bmzZtxdXWlRYsWLFiwgJCQkGy107lzZ0JCQpgyZQo7d+7kjTfe4OLFiyxbtswmbvPmzXz44YdERERQokQJQkJCOHToEPfddx9+fn4MGzaMQoUK8eabb9K0aVO2bt1KvXr1bNqIiIigSJEijB8/nsjISBYsWMDJkyfZsmULFouF8+fP88ADD1CyZElGjBhBkSJF+P333/n0009z/PMRkYJx33338X//939MnTqVF154IVtfTG+kadOmfPzxx8yZM4d27dpx7do15syZQ3R0NP37989WG8uWLePSpUv06dOHa9euMXv2bP7v//6Pn3/+mYCAAGtcfHw8LVu2pFGjRrz++ut4eXlhGAbt27fn22+/pWfPnoSFhbF+/XqGDh3Kn3/+ycyZM232tXXrVlatWkW/fv1wd3dn/vz5tGrVit27d1svyXn++ef5+OOPiYiIoFq1avz99998//33HDlyhHvuueemf2a3BUPkNrB48WIDyHBKrVy5csYDDzxgXLhwwbhw4YLx448/Go8//rgBGH379s2w7T59+qRrJ1m/fv0MwChevLjRqlUrY9WqVca0adMMHx8fo2LFisaVK1ey7Pe4ceMMwGjfvr3N8hdffNEAjB9//NG6DDBcXFyMQ4cO2cR27NjRKFy4sHHs2DHrstOnTxu+vr5G48aN0/2MateubVy/ft26fOrUqQZgfPbZZ4ZhGMbq1asNwNizZ0+WfReRgpX8GU792UzOIRcuXDC2bt1qAMaMGTOs68uVK2e0bds2w/b27NljAMbixYszXH/u3DmjWbNmNvm0RIkSxvbt22/Y1xMnThiA4enpafzxxx/W5bt27TIAY+DAgdZl3bp1MwBjxIgRNm2sWbPGAIxJkybZLH/00UcNi8ViHD161LosuX979+61Ljt58qTh4eFhPPTQQ9Zl/v7+Rp8+fW7Yf0emU7FyW5k3bx4bN260mdLasGEDJUuWpGTJktSsWZOPPvqIp59+mtdeey3H+7t8+TJgns5du3YtnTt3ZsiQIbz99tscO3aM999/P1vt9OnTx2a+b9++gHkTR2pNmjShWrVq1vmEhAQ2bNhAx44dqVChgnV56dKlefLJJ/n++++JiYmxaaNXr14UKlTIOv/CCy/g5uZm3VeRIkUA+PLLL3N0WllE7Ktx48bcf//9TJ06lf/++++m2/Py8iI0NJRu3brx0Ucf8e6771K6dGkefvhhjh49mq02OnbsSFBQkHW+bt261KtXL11uAzMXpfbVV1/h6upKv379bJYPHjwYwzD4+uuvbZaHh4dTu3Zt6/wdd9xBhw4dWL9+PQkJCYCZ33bt2sXp06ez1X9HpMJObit169alefPmNlNa9erVY+PGjWzatInt27fz119/sWzZslydukjepnPnzri4pHxcOnXqhJubG9u3b89WO5UrV7aZr1ixIi4uLvz+++82y9Pe8XvhwgWuXr1KaGhoujarVq1KYmIip06dynJfPj4+lC5d2rqvJk2a8MgjjzBhwgRKlChBhw4dWLx4MbGxsdkai4jYz/jx4zl79iwLFy686bY6depEVFQUS5Ys4dFHH6VHjx5s2bKF69evM2rUqGy1kTbfANx5553pcpubmxtly5a1WXby5EnKlCmDr6+vzfKqVata12dnX1evXuXChQsATJ06lYMHDxIcHEzdunUZP348x48fz9ZYHIUKO3E4JUqUoHnz5jRr1ozw8HDrEarcKFOmDIDNtSIArq6uFC9ePN1NCdmV2XPz8uK6mezs++OPP2bHjh1ERETw559/8swzz1C7dm3rEUoRuTU1btyYpk2b3vRRu+PHj7Nu3Trat29vs7xYsWI0atSIbdu23WxXbbi7u9t8Oc4vnTt35vjx48yZM4cyZcowbdo07rrrrnRH/xyZCjuRLCQf9v/zzz9tll+/fp2//vqLkiVLZqud3377zWb+6NGjJCYm3vDmi5IlS+Ll5UVkZGS6db/88gsuLi4EBwdnua/Lly9z5syZdPuqX78+kydPZu/evaxYsYJDhw6xcuXKbI1HROwn+ajdm2++mes2zp07B2A9hZlaXFwc8fHx2Wonbb4B+PXXX7N1Y1m5cuU4ffo0ly5dsln+yy+/WNdnZ19eXl42ubh06dK8+OKLrFmzhhMnTlC8eHEmT56cneE4BBV2Illo2rQppUqVYsWKFTaPJ1myZAkJCQm0aNEiW+3MmzfPZn7OnDmA+Vy+rLi6uvLAAw/w2Wef2ZzaOHfuHO+//z6NGjXCz8/PZpu33nrL5tq5BQsWEB8fb93XxYsX0z1KICwsDECnY0VuA02aNKFp06a89tpr6R6blF2VKlXCxcWFVatW2eSDP/74g++++45atWplq501a9bYfPHdvXs3u3btumFuA2jTpg0JCQnMnTvXZvnMmTOxWCzp2tixY4fNY0tOnTrFZ599xgMPPICrqysJCQlER0fbbFOqVCnKlCnjVLlNjzsRp3Ty5Enee+89AOuf75k0aRJgfkt8+umnAfP0wbRp0+jWrRuNGzfm6aefJioqitmzZ3Pfffdl+wGeJ06coH379rRq1YodO3awfPlynnzySWrWrHnDbSdNmsTGjRtp1KgRL774Im5ubrz55pvExsYyderUdPHXr1+nWbNmdO7cmcjISObPn0+jRo2sp1yWLl3K/Pnzeeihh6hYsSKXLl3i7bffxs/PjzZt2mRrPCJiX+PGjeP+++/P9fYlS5bkmWeeYdGiRTRr1oyHH36YS5cuMX/+fP777z9GjhyZrXYqVapEo0aNeOGFF4iNjWXWrFkUL16cYcOG3XDbdu3acf/99zNq1Ch+//13atasyYYNG/jss88YMGCA9S9kJLv77rtp2bKlzeNOACZMmACYz7ArW7Ysjz76KDVr1sTHx4dNmzaxZ88epk+fnsOf0G3MznflimRLRo8ByEhWt/6n9u2332b6+JQmTZqki//ggw+MmjVrGu7u7kZAQIARERFhxMTE3HA/yY8qOHz4sPHoo48avr6+RtGiRY2IiAjjv//+s4kFMr1Nf9++fUbLli0NHx8fw8vLy7j//vvTPZIg+We0detWo1evXkbRokUNHx8fo0uXLsbff/9t09YTTzxh3HHHHYa7u7tRqlQp48EHH7R5jICIFLwbPe4krSZNmhhArh93EhcXZ8yZM8cICwszfHx8DB8fH+P+++83Nm/efMO+Jj/uZNq0acb06dON4OBgw93d3bjvvvtsHuNkGObjTry9vTNs59KlS8bAgQONMmXKGIUKFTIqV65sTJs2zUhMTLSJS86Py5cvNypXrmy4u7sbtWrVMr799ltrTGxsrDF06FCjZs2ahq+vr+Ht7W3UrFnTmD9//g3H40gshpHJ451F5KaNHz+eCRMmcOHChRs+SFlE5Hbx+++/U758eaZNm8aQIUPs3R1JRdfYiYiIiDgIFXYiIiIiDkKFnYiIiIiD0DV2IiIiIg5CR+xEREREHIQKOxEREREHocJORERExEGosBPJAyEhIcyaNSvLGIvFwpo1a3Lcdvfu3enYsWOu+mUP2flZiMjtQ/ktxe2Q31TYSbbt2LEDV1dX2rZta++u5Lv8GOuZM2ey9fcT7cmZXmORtJzl/a/85tivrwo7ybZ33nmHvn378r///Y/Tp08X+P6vX79eYPvKj7EGBgbi7u6e6fq4uLg82c/NsPdrLGJP9n7/F1SOU35z7Pymwk6y5fLly6xatYoXXniBtm3bsmTJkhtuExsby/DhwwkODsbd3Z1KlSrxzjvvAJCQkEDPnj0pX748np6ehIaGMnv2bJvtkw/RT548mTJlyhAaGpofQ0snN2MF8w9QP/HEE3h7exMUFMS8efNs1qc+VfH7779jsVhYtWoVTZo0wcPDgxUrVpCQkMCgQYMoUqSI9Q9pF9QTiXIz7vPnz9OuXTs8PT0pX748K1asSBcTFRVFhw4d8PHxwc/Pj86dO3Pu3DmbmEmTJlGqVCl8fX159tlnGTFiBGFhYXk0MpEbc5Ycp/zmBPnNnn+oVm4f77zzjlGnTh3DMAzjiy++MCpWrJjujzSn1blzZyM4ONj49NNPjWPHjhmbNm0yVq5caRiGYVy/ft0YO3assWfPHuP48ePG8uXLDS8vL2PVqlXW7bt162b4+PgYTz/9tHHw4EHj4MGD+TfAVHIz1nLlyhm+vr7GlClTjMjISOONN94wXF1djQ0bNlhjAGP16tWGYaT8Ae2QkBDjk08+MY4fP26cPn3aeO2114yiRYsan3zyiXH48GGjZ8+ehq+vr9GhQ4f8Gq5VbsbdunVro2bNmsaOHTuMvXv3Gg0aNDA8PT2NmTNnGoZhGAkJCUZYWJjRqFEjY+/evcbOnTuN2rVrG02aNLG2sXz5csPDw8N49913jcjISGPChAmGn5+fUbNmzXwaqUh6zpLjlN8cP7+psJNsadCggTFr1izDMAwjLi7OKFGihPHtt99mGh8ZGWkAxsaNG7O9jz59+hiPPPKIdb5bt25GQECAERsbm+t+50ZOx2oYZuJr1aqVzbLHHnvMaN26tXU+o8SXvJ9kpUuXNqZOnWqdj4uLM8qWLVsgiS+3r/Hu3buty44cOWIA1sS3YcMGw9XV1YiKirLGHDp0yGa7evXqGX369LFpu2HDhirspEA5S45TfnP8/KZTsXJDkZGR7N69myeeeAIANzc3HnvsMesph4wcOHAAV1dXmjRpkmnMvHnzqF27NiVLlsTHx4e33nqLqKgom5jq1atTuHDhvBlINuRmrMnCw8PTzR85ciTLberUqWP9f3R0NGfOnKFevXrWZW5ubjYx+SU34z5y5Ahubm7Url3buqxKlSoUKVLEJiY4OJjg4GDrsmrVqlGkSBHrzyYyMpK6devatJ12XiQ/OUuOU35zjvzmlq+ti0N45513iI+Pp0yZMtZlhmHg7u7O3Llz8ff3T7eNp6dnlm2uXLmSIUOGMH36dMLDw/H19WXatGns2rXLJs7b2ztvBpFNuRnrzSjo8WWmoMctcitxlhyn/OYc+U1H7CRL8fHxLFu2jOnTp3PgwAHr9OOPP1KmTBk++OCDDLerXr06iYmJbN26NcP127Zto0GDBrz44ovUqlWLSpUqcezYsfwcyg3ldqzJdu7cmW6+atWq2d6/v78/pUuXtkn88fHx/PDDDzkbSA7ldtxVqlRJ17/IyEj+/fdf63zVqlU5deoUp06dsi47fPgw//77L9WqVQMgNDSUPXv22LSddl4kvzhLjlN+c6L8lm8necUhrF692ihcuLDx77//pls3bNgw68WoGenevbsRHBxsrF692jh+/Ljx7bffWi8cnj17tuHn52esW7fOiIyMNEaPHp3ugtJu3boVyLUXyW5mrOXKlTP8/PyM1157zYiMjDTmzp1ruLq6GuvWrbPGkME1KPv377dp59VXXzWKFStmrF692jhy5Ijx3HPP5fvFxTcz7latWhm1atUydu7caezdu9do1KiRzcXFiYmJRlhYmHHfffcZP/zwg7Fr164MLy729PQ0lixZYvz666/GxIkTDT8/PyMsLCyvhyqSjrPkOOU358lvKuwkSw8++KDRpk2bDNft2rXLAIwff/wxw/X//fefMXDgQKN06dJG4cKFjUqVKhnvvvuuYRiGce3aNaN79+6Gv7+/UaRIEeOFF14wRowYYdfC7mbGWq5cOWPChAlGp06dDC8vLyMwMNCYPXu2TUx2El9cXJzRv39/w8/PzyhSpIgxaNAgo2vXrvn6c7iZcZ85c8Zo27at4e7ubtxxxx3GsmXLjHLlylkTn2EYxsmTJ4327dsb3t7ehq+vr9GpUyfj7NmzNu28/PLLRokSJQwfHx/jmWeeMfr162fUr18/z8YokhlnyXHKb+k5an6zGEYBPURGRCSbWrRoQWBgIO+99569uyIikqfyO7/p5gkRsaurV6+ycOFCWrZsiaurKx988AGbNm1i48aN9u6aiMhNsUd+0xE7EbGr//77j3bt2rF//36uXbtGaGgoo0eP5uGHH7Z310REboo98psKOxEREREHocediIiIiDgIFXYiIiIiDkKFnYiIiIiDUGEnIiIi4iBU2Em+i42NZfz48cTGxtq7KwXGGccMzjtucW7O+L7XmG9duitW8l1MTAz+/v5ER0fj5+dn7+4UCGccMzjvuMW5OeP7XmO+dcesI3YiIiIiDkKFnYiIiIiD0J8UcxKJiYmcPn0aX19fLBZLge47JibG5l9n4IxjBvuO2zAMLl26RJkyZXBx0XdWZ6McV7A05oKVk/yma+ycxB9//EFwcLC9uyGS706dOkXZsmXt3Q0pYMpx4gyyk990xM5J+Pr6mv9xOQmWW/eiz3yR4IRHb/5+1d49KHgxsVB+Rsp7XZxKyut+CnCyHOeMoqfYuwcFKyYWgmdmK7+psHMS1lMTFj/nK+yc8VJSPw9798BuCvo0nNwaUl53P1TYOQE/d3v3wC6yk9+c8DeeiIiIiGNSYSciIiLiIFTYiYiIiDgIFXYiIiIiDkKFnYiIiIiDUGEnIiIi4iBU2ImIiIg4CBV2IiIiIg5ChZ2IiIiIg1BhJyIiIuIgVNiJiIiIOAgVdiIiIiIOQoWdiIiIiINQYSciIiLiIFTYiYiIiDgIFXYiIiIiDkKFnYiIiIiDUGEnIiIi4iBU2ImIiIg4CBV2IiIiIg5ChZ2IiIiIg1BhJyIiIuIg8qywa9q0KQMGDMir5rI0fvx4wsLCCmRf+aF79+507NjR3t3IvsT5EF8B4r0gPhyM3TeI/wjiqyXF14TEr9Ks/xQSWkJ8SYh3BeNAxu0YOyChOcT7QnwRiG8Kxn95MKDsmgeEAB5APeAG4+YjoEpSfHUgzbgxgLFAacATaA78libmH6AL4AcUAXoCl3PZ/1yYvwsqzQSfidDgLdj9R+axcQkwaQuEzjLj75kP69OMJyERxn0DlWeC70QzdvIWMIyM23zxCyg0DmbvyJvx5BHlt+y77fIb4JSfdWcb8/9OQrsPoMx0sEyANb/ceJstv8M9b4L7JKj0Biw5kD5m3m4ImQUek6DeItj9p+36a/HQZy0Unwo+r8AjH8K5/B3zbXnEbsiQIXzzzTf27kauzZ49myVLlti7G9mTuAoSB4PLGHDdC5YakNAajPMZxxvbIbELuDwDrj+ApQMkPgzGwVRBV8DSCFymZL5fYwcktAFLC3DdCa67wOVFCu4tuwoYBIwD9gE1gZZAJuNmO/AEZqLaD3RMmlKPeyrwBrAQ2AV4J7V5LVVMF+AQsBH4Evgf0Ovmh5MdHx6EoethdFPY3RtqBELb9+B8Jklo7Dfw9l6Y1QZ+6gO96sCjK2H/mZSYad/Dm3thdlv4OQJeaQGvb4O5u9K3t+YI7PoDyvjmy/BuF8pvBc0JP+vOOOYr16FmAMxrk734Exeh7ftwfwgc6A0D6sOzn8P6oykxqw7CoA0wrgns622233I5nL+SEjNwHXzxK3zUCbZ2h9OX4OEP83Jk6dyWhZ2Pjw/Fixe3dzdyzd/fnyJFiti7G9mTOAssz4JLD7BUA5cFgBcYizOJfwMsLcFlCFiqguvLwD2QOC8lxuVps1C0NM98vwmDwdIXXIaD5S6whIJLZ7C45+HgsjIDeA7oAVTDTFZewLuZxM8GWgFDgarAROAeYG7SegOYBYwGOgA1gGXAaWBNUswRYB2wCPMbdCNgDrAyKS6fzdoOPWtD91pQrRTMfxC8CsGS/RnHr/gJht8Hre+ECsXg+brQujLM3J4Ss+MUtAuFNndCSFF45C5oURH2pPlW+2cMDPgKlj0ChVzzb4y3AeW3guaEn3VnHHPryjDp/+ChqtmLX7gXyheB6S2hakmIqAuPVoOZO1NiZuyE5+6BHrWgWklYmJQz303KmdHX4J39MKMl/F95qF0GFneA7adgZxZnQ25Srgq7K1eu0LVrV3x8fChdujTTp0+3WX/x4kW6du1K0aJF8fLyonXr1vz2m+0h2bfffpvg4GC8vLx46KGHmDFjRraTQdpTFcmH/l955RUCAgIoUqQIL7/8MvHx8QwdOpRixYpRtmxZFi+2LUaGDx/OnXfeiZeXFxUqVGDMmDHExcXZxEyaNIlSpUrh6+vLs88+y4gRI9KdJlm0aBFVq1bFw8ODKlWqMH/+/Cz7n/ZURdOmTYmIiCAiIgJ/f39KlCjBmDFjMFKdrjpz5gxt27bF09OT8uXL8/777xMSEsKsWbOy9TPLFeM68ANYmqUss7iY80Ymp8qMnekLNssD5vJs7/c8sAsspSC+EcSXhvj7wfg+pyPIpaRxk3ocLknzmZ0i3JEmHsxvq8nxJ4CzaWL8MRNccswOzNMTdVLFNE/adwZHuPLS9XjYdwaaVUhZ5uIC/1cBdp7KeJvYePBws13mUQi2R6XMhwfDtyfg17/M+R/PwrYoaFU5JSYxEbp/CoMawF2l8mY8N0H5LcwmxmHzG+CUn3WnHHMu7PgDmlewXdayorkc4HoC/HDaNsbFYs4nx/xwBuISbWOqlIA7/M0vvfkkV4Xd0KFD2bp1K5999hkbNmxgy5Yt7Nu3z7q+e/fu7N27l88//5wdO3ZgGAZt2rSxJpVt27bx/PPP079/fw4cOECLFi2YPHnyTQ1k8+bNnD59mv/973/MmDGDcePG8eCDD1K0aFF27drF888/T+/evfnjj5Qq2dfXlyVLlnD48GFmz57N22+/zcyZM63rV6xYweTJk3nttdf44YcfuOOOO1iwYIHNflesWMHYsWOZPHkyR44c4ZVXXmHMmDEsXbo0R/1funQpbm5u7N69m9mzZzNjxgwWLVpkXd+1a1dOnz7Nli1b+OSTT3jrrbc4fz6zw+Z55S8gASwBaZYHgHEuk23OAml+OVsCkpZn13Hzn8QJ4NITXL8CSy1IaAFG2ms28kPSuMlg3JmO4+wN4s+mWpZVTNrCxg0olsV+88hfV83r4Ur5pOmeD5zN5FTsA5XMa+F++9sszjYdM0+nnrmUEjOsEXS+G+6eC54T4N6F0K8+PFkjJWba9+DmAn3r5/24ckH5DZsYx81v4JSfdacccy6cvQwB3rbLAnwgJhb+i0vKmUYGMd4pOfPsZSjsCkU8Mo/JB243DrF1+fJl3nnnHZYvX06zZuaRnKVLl1K2bFkAfvvtNz7//HO2bdtGgwYNADM5BAcHs2bNGjp16sScOXNo3bo1Q4YMAeDOO+9k+/btfPnll7keSLFixXjjjTdwcXEhNDSUqVOncvXqVV566SUARo4cyauvvsr333/P448/DsDo0aOt24eEhDBkyBBWrlzJsGHDAJgzZw49e/akR48eAIwdO5YNGzZw+XLKCzJu3DimT5/Oww8/DED58uU5fPgwb775Jt26dct2/4ODg5k5cyYWi4XQ0FB+/vlnZs6cyXPPPccvv/zCpk2b2LNnD3XqmN92Fi1aROXKlTNtLzY2ltjYWOt8TExMtvtif4nmP5Ze5ilgANdaEL8ZEheD6yv265qkmNEanv8c7p4DFgtULArdwmxP3X50CD74Cd57xDy9++NZGPw1lPaDrmHmN945u8xr+iwWe43ESvnt9shvcLvnOJH8k+MjdseOHeP69evUq1fPuqxYsWKEhoYCcOTIEdzc3GzWFy9enNDQUI4cOQJAZGQkdevWtWk37XxO3XXXXbi4pAwnICCA6tWrW+ddXV0pXry4zbfAVatW0bBhQwIDA/Hx8WH06NFERaWcRrpRP69cucKxY8fo2bMnPj4+1mnSpEkcO3YsR/2vX78+llS/2MLDw/ntt99ISEggMjISNzc37rnnHuv6SpUqUbRo0UzbmzJlCv7+/tYpODg4R/0xlQBcMzg6dy6Do3jJAkl3Aa5xLml5dpU2/7GkuRbCUgWIShed95LGTQbjznQcgTeID0y1LKuYtEcp4jHvJMvJzy8XSniBq0v6GyXOXYZAn4y3KekNnzwB0aPg2EA42Bd8CkOFVO/LERtgaCN4rDpUD4CnakL/cJj6nbn++5PmhcYVZoLHBHM6+S8MW2/enVvAlN9uj/wGeZzjnOmz7pRjzoVAHzh3xXbZucvg5w6ehZJypiWDmCspOTPQxzxl+++1zGPywW1580RGChUqZDNvsVgyXJaYaB4N2rFjB126dKFNmzZ8+eWX7N+/n1GjRnH9+vVs7zP5m+3bb7/NgQMHrNPBgwfZuTMH15Tlg5EjRxIdHW2dTp3Kxfl8S2GgNhibU5YZiea8JTyTbeqDkeaOPmOTuTzbQoAyYPyapp3fgHI5aCe3ksZN6nEkJs1nMm7C08SDeedXcnx5zOSVOiYG89qS5Jhw4F/M61+SbU7adz3yVWE3uKc0bD6esiwx0bw+rv4NfmF6FIIgP4hPhNVHoF2VlHVX48zrTlJztUBi0vVVT9WEfS/A3udTpjK+MLghrH06b8bmAJTf0suTHOeMn3WnHHMuhJeFb07YLtt43FwO5inW2mXgm9Q50zDnk2Nql4ZCLrYxkX9BVLR5/XE+yfGp2IoVK1KoUCF27drFHXfcAZgXE//66680adKEqlWrEh8fz65du6ynKv7++28iIyOpVq0aAKGhoezZs8em3bTz+W379u2UK1eOUaNGWZedPHnSJia5n127drUuS93PgIAAypQpw/Hjx+nSpctN9WfXLtuLR3fu3EnlypVxdXUlNDSU+Ph49u/fT+3atQE4evQoFy9ezLQ9d3d33N3z4A5SlwGQ2AMSa4OlLiTOxnxcSXdzfUI3ICjl9KhLP0i4HxJngKWN+bgU9oLLwpQ2jX+AKDCS7oQyIpNWBIIl0Dwl5zIEEsdDYg2whEHiMuAXcMnf28RTDAK6YV7oWxfzjq8rmHeRAXQFgoDkR7b0B5oA04G2mHd67QXeSlpvAQYAk4DKmIlwDFAG87EBYN5t1grzbrWFQBwQATyeFJfPBjSAZ1ZD7SC4Nwje2GE+IqBbLXN9908hyBcmtzDnd/0Bp2OgZqB5C//L35qJbUjDlDbbhsKr38EdRcy7xg6chVk7zDtvAYp7mVNqhVzNa1lCS+T7kNNSfrs98hvkYY5zxs+6M4758nU4+k/K/ImLZj4q5mnezDByE/x5CZY9ZK5/vg7M3QPDNsIztWDzCfjwEKx9MqWNQfWh2xqoUwbqBsGsnXAlDnqEmev9PaBnLfORKMU8zaN9fb82C7/6ZfNtqDku7Hx8fOjZsydDhw6lePHilCpVilGjRllPE1SuXJkOHTrw3HPP8eabb+Lr68uIESMICgqiQ4cOAPTt25fGjRszY8YM2rVrx+bNm/n6669tDtXnt8qVKxMVFcXKlSu59957Wbt2LatXr7aJ6du3L8899xx16tShQYMGrFq1ip9++okKFVLucJkwYQL9+vXD39+fVq1aERsby969e7l48SKDBg0CoFmzZjz00ENERERk2p+oqCgGDRpE79692bdvH3PmzLHejVelShWaN29Or169WLBgAYUKFWLw4MF4enrm/8/M5THgL7PI4iwQlnQzQ9KpWOOUeadsMksDcFkOiWOBUUBlcPkULHenxBifQ2LPlPnEpA+KZSy4jkvab3/gmvkMPf4BaoLrerBUzI9RZuAx4ALmAzeTxs06Ui4OjsL2gHcD4H3M2/1fwkxua4BU42YYZvLshfnNtVFSm6kvrF2BmeyaJbX/COazoQpA57vhwhWYsNm8sLdmIHz5tFlkAZyKtj36FhsP4zbD8YvmKdhWlWHJw1DEMyVmdhszpu+X5inXMr7wXB0Y3aRgxpRDym9Olt8Ap/ysO+OY956G+1Pd9DNog/lvt5qwpCOcuWweSUtWvqhZxA1cD7N3QVk/WNQeWlZKiXnsbrhwFcZuMXNmWCCs65KSMwFmtgKX9eaDiWMTzDtr57fNz5HmvLADmDZtGpcvX6Zdu3b4+voyePBgoqNTfiCLFy+mf//+PPjgg1y/fp3GjRvz1VdfWU8dNGzYkIULFzJhwgRGjx5Ny5YtGThwIHPnzs1sl3muffv2DBw4kIiICGJjY2nbti1jxoxh/Pjx1pguXbpw/PhxhgwZwrVr1+jcuTPdu3dn9+6UJ3Q/++yzeHl5MW3aNIYOHYq3tzfVq1e3eUr9sWPH+Ouvv7LsT9euXfnvv/+oW7curq6u9O/fn169Uh7cuGzZMnr27Enjxo0JDAxkypQpHDp0CA8PjyxazSMufcwpI26b0y9z6WROmbbX3ZxuuN/h5mQ3EUlTRrZksKxT0pQZC/By0pSZYpgJ1E761DOnjHzTw3a+cQj8lPkvcwB83c2bLGa0zn4fjg7Mfmw+UH5zsvwGOOVn3dnG3DQEjHGZr1/SMeNt9vfOut2IuuaUGQ83mNfWnAqIxTAy+9s+BSv57qjvvvvO3l3JUosWLQgMDOS9997LdRtPPPEErq6uLF++HDCf8xQWFpajZzb98ccfBAcHs2nTJuvde1mJiYnB398fXC+CxS+3Xb89xTvMpaTZF5dVcnVQMdeg+BSio6Px87u13uPKb/mb3yBVjiMa809WiUMzxtu7BwUrJhb8X81WfsvVEbu88Prrr9OiRQu8vb35+uuvWbp06Q0ffFnQrl69ysKFC2nZsiWurq588MEHbNq0iY0bN+aqvfj4eH799Vd27NhB7943+BaQxubNm7l8+TLVq1fnzJkzDBs2jJCQEBo3bpyrvohI/lF+U34TsRe7HcrYvXs3LVq0oHr16ixcuJA33niDZ599FjBv7U99e33qacWKFQXWR4vFwldffUXjxo2pXbs2X3zxBZ988gnNm2fxp7CycPDgQerUqcNdd93F888/n6Nt4+LieOmll7jrrrt46KGHKFmyJFu2bEl3Z5yI2J/ym/KbiL3cMqdiUzt58mS6P32TLCAgAF9f5/4j4bmhU7FORqdi7d2bTCm/5Q+dinUyOhWbKbudis1KuXIF8awyEZGCp/wmIvnJCQ9liIiIiDgmFXYiIiIiDkKFnYiIiIiDUGEnIiIi4iBU2ImIiIg4CBV2IiIiIg5ChZ2IiIiIg1BhJyIiIuIgVNiJiIiIOAgVdiIiIiIOQoWdiIiIiINQYSciIiLiIFTYiYiIiDgIFXYiIiIiDkKFnYiIiIiDUGEnIiIi4iBU2ImIiIg4CBV2IiIiIg5ChZ2IiIiIg1BhJyIiIuIgLIZhGPbuhOS/mJgY/P39IXoE+LnbuzsFyzLe3j2QAhED+BMdHY2fn5+9OyMFTDlOHFv285uO2ImIiIg4CBV2IiIiIg5ChZ2IiIiIg1BhJyIiIuIgVNiJiIiIOAgVdiIiIiIOQoWdiIiIiINQYSciIiLiIFTYiYiIiDgIFXYiIiIiDkKFnYiIiIiDUGEnIiIi4iBU2ImIiIg4CBV2IiIiIg5ChZ2IiIiIg1BhJyIiIuIgVNiJiIiIOAgVdiIiIiIOQoWdiIiIiINQYSciIiLiIFTYiYiIiDgIFXYiIiIiDkKFnYiIiIiDUGEnIiIi4iBU2EnW/ncS2n0AZaaDZQKs+eXG22z5He55E9wnQaU3YMmB9DHzdkPILPCYBPUWwe4/bddfi4c+a6H4VPB5BR75EM5dzoMB5cQ8IATwAOoBu28Q/xFQJSm+OvBVmvUGMBYoDXgCzYHf0sT8A3QB/IAiQE+gIMftjGMWp6X8hnN91p1jzLdlYde0aVMGDBhg727kWkhICLNmzbJ3N7LnynWoGQDz2mQv/sRFaPs+3B8CB3rDgPrw7Oew/mhKzKqDMGgDjGsC+3qb7bdcDuevpMQMXAdf/AofdYKt3eH0JXj4w7wc2Q2sAgYB44B9QE2gJXA+k/jtwBOYH9r9QMek6WCqmKnAG8BCYBfgndTmtVQxXYBDwEbgS+B/QK+bH062OOOYbz3KbwVI+Q3n+aw7z5gthmEY+bqHfPDPP/9QqFAhfH197d2VXLlw4QLe3t54eXkV2D5jYmLw9/eH6BHg5567RiwTYPVj0LFK5jHDN8La3+DgiynLHv8Y/r0G654y5+stgnvLwNykZJpoQPBM6FsXRjSC6GtQchq8/wg8Ws2M+eUvqDoPdvSE+mVz2O/xOYs3OwncC8xNmk8EgoG+wIgM4h8DrmB+cJPVB8IwP/QGUAYYDAxJWh8NBABLgMeBI0A1YA9QJylmHdAG+CNp+/x0u485BvAnOjoaPz+/HGx3a1F+y52bznG3a36DXOS42/2znhu3+5izn99uuSN2169fv2FMsWLFbtukB1CyZMkCT3oFZscf0LyC7bKWFc3lANcT4IfTtjEuFnM+OeaHMxCXaBtTpQTc4Q87TuVv/81OAj9gHla3djJpfkcm2+xIEw/mN7fk+BPA2TQx/pjJJjlmB+ah+jqpYpon7XtXTgaQC8445oKn/HabU35L5Xb6rDvXmPOksEtMTGTq1KlUqlQJd3d37rjjDiZPngzA8OHDufPOO/Hy8qJChQqMGTOGuLg467bjx48nLCyMRYsWUb58eTw8PG64v7SnKkJCQpg0aRJdu3bFx8eHcuXK8fnnn3PhwgU6dOiAj48PNWrUYO/evdZt/v77b5544gmCgoLw8vKievXqfPDBBzb7uXTpEl26dMHb25vSpUszc+bMdPuOjY1lyJAhBAUF4e3tTb169diyZUuW/U97qsJisbBgwQJat26Np6cnFSpU4OOPP7bZZvv27YSFheHh4UGdOnVYs2YNFouFAwcO3PDnVaDOXoYAb9tlAT4QEwv/xcFfVyHByCDG29w2uY3CrlDEI/OYfPUXkID5zcumA5gf5IycvUH82VTLsooplWa9G1Asi/3mFWccc/Yovym/WSm/ZRJ/q3/WnWvMeVLYjRw5kldffZUxY8Zw+PBh3n//fQICzMH6+vqyZMkSDh8+zOzZs3n77beZOXOmzfZHjx7lk08+4dNPP831B3nmzJk0bNiQ/fv307ZtW55++mm6du3KU089xb59+6hYsSJdu3Yl+czztWvXqF27NmvXruXgwYP06tWLp59+mt27Uy6mHDRoENu2bePzzz9n48aNfPfdd+zbt89mvxEREezYsYOVK1fy008/0alTJ1q1asVvv6W9gDJrY8aM4ZFHHuHHH3+kS5cuPP744xw5cgQwTzG0a9eO6tWrs2/fPiZOnMjw4cOzbC82NpaYmBibSURyTvnt1stvoBwnkhm3m23g0qVLzJ49m7lz59KtWzcAKlasSKNGjQAYPXq0NTYkJIQhQ4awcuVKhg0bZl1+/fp1li1bRsmSJXPdjzZt2tC7d28Axo4dy4IFC7j33nvp1KkTYH6zDg8P59y5cwQGBhIUFMSQIUOs2/ft25f169fz4YcfUrduXS5dusTSpUt5//33adasGQCLFy+mTJmUc+JRUVEsXryYqKgo6/IhQ4awbt06Fi9ezCuvvJLt/nfq1Ilnn30WgIkTJ7Jx40bmzJnD/Pnzef/997FYLLz99tt4eHhQrVo1/vzzT5577rlM25syZQoTJkzI9v7zTKAPnLtiu+zcZfOaF89C4OoCrpYMYq6Y2ya3cT3BvG4l9bfa1DH5qgTgCpxLs/wcEJjJNoE3iA9Mtax0mpiwVDFpL+SNx7yrKrP95hVnHPONKb/dmvkN7JTjlN8yib/VP+vONeabPmJ35MgRYmNjrckhrVWrVtGwYUMCAwPx8fFh9OjRREVF2cSUK1fuppIeQI0aNaz/T/42Xb169XTLzp83f8gJCQlMnDiR6tWrU6xYMXx8fFi/fr21b8ePHycuLo66deta2/D39yc0NNQ6//PPP5OQkMCdd96Jj4+Pddq6dSvHjh3LUf/Dw8PTzSd/o42MjKRGjRo2p3FS9ysjI0eOJDo62jqdOlUQ124A4WXhmxO2yzYeN5eDeQqidhn45njK+kTDnE+OqV0aCrnYxkT+BVHREB6cv/03OwnUBr5JtSwxaT48wy3M5d+kWbYxVXx5zA9y6pgYzOsskmPCgX8xrwVJtjlp3/VyMoBccMYx35jy262Z38BOOU75LZXb6bPuXGO+6SN2np6ema7bsWMHXbp0YcKECbRs2RJ/f39WrlzJ9OnTbeK8vb0zaSH7ChUqZP2/xWLJdFliYiIA06ZNY/bs2cyaNYvq1avj7e3NgAEDsnVxc7LLly/j6urKDz/8gKurq806H5+C+OaVOXd3d9zdc3n3a2qXr8PRf1LmT1yEA2ehmKd5se/ITfDnJVj2kLn++Towdw8M2wjP1ILNJ+DDQ7D2yZQ2BtWHbmugThmoGwSzdsKVOOgRZq7394CetcxHBhTzNL8N9/3aTIy5uWMsVwYB3TAveq0LzMK8Q6pH0vquQBAwJWm+P9AEmA60BVYCe4G3ktZbgAHAJKAyZlIYg3lXVMekmKpAK+A5zLuu4oAIzLur8vuOMXDOMWdN+e3WzG+QRzlO+Q3n+aw7z5hvurCrXLkynp6efPPNN9ZD7cm2b99OuXLlGDVqlHXZyZMnb3aXeWLbtm106NCBp54yb1FPTEzk119/pVo18/bzChUqUKhQIfbs2cMdd9wBQHR0NL/++iuNGzcGoFatWiQkJHD+/Hnuu+++m+rPzp076dq1q818rVq1AAgNDWX58uXExsZaE9mePXtuan/Ztvc03L80ZX7QBvPfbjVhSUc4c9n8ppmsfFEzyQ1cD7N3QVk/WNQeWlZKiXnsbrhwFcZuMS8WDguEdV3Mi5CTzWwFLuvNB3fGJph3ns1vm58jTeMx4ALmwyfPYh5aX0fKhbJR2B7wbgC8D4wGXsL8oK8B7k4VMwwzkfTC/BbXKKnN1BdRr8D84DdLav8RzOckFQRnHHPWlN+U35TfwDE+684z5psu7Dw8PBg+fDjDhg2jcOHCNGzYkAsXLnDo0CEqV65MVFQUK1eu5N5772Xt2rWsXr06L/p90ypXrszHH3/M9u3bKVq0KDNmzODcuXPWxOfr60u3bt0YOnQoxYoVo1SpUowbNw4XFxfrt+M777yTLl260LVrV6ZPn06tWrW4cOEC33zzDTVq1KBt27b8+eefNGvWjGXLlmV5euGjjz6iTp06NGrUiBUrVrB7927eeecdAJ588klGjRpFr169GDFiBFFRUbz++utAyjf1fNM0BIxxma9f0jHjbfb3zrrdiLrmlBkPN5jX1pzsJiJpysiWDJZ1SpoyYwFeTpoyUwwzmdiLM445c8pvym8ZbqP8loHb4bPuHGPOk7tix4wZw+DBgxk7dixVq1blscce4/z587Rv356BAwcSERFBWFgY27dvZ8yYMXmxy5s2evRo7rnnHlq2bEnTpk0JDAykY8eONjEzZswgPDycBx98kObNm9OwYUOqVq1qcy3I4sWL6dq1K4MHDyY0NJSOHTvafAuOi4sjMjKSq1evZtmfCRMmsHLlSmrUqMGyZcv44IMPrEnYz8+PL774ggMHDhAWFsaoUaMYO3YsQLYenyAiuaf8pvwmcju5Lf/yhL1cuXKFoKAgpk+fTs+ePXPdTunSpZk4caL11I7FYmH16tXpEm9WVqxYQY8ePYiOjs7yOqBkefKXJ25XufrLE3L7cYy/PGEvt3N+A+U4cXTZz283fSrWke3fv59ffvmFunXrEh0dzcsvm4dbO3TokKv2rl69yrZt2zh37hx33XVXjrZdtmwZFSpUICgoiB9//JHhw4fTuXPnbCc9EZHUlN9EHNMtV9hFRUVZD9Fn5PDhw9bTAAXh9ddfJzIyksKFC1O7dm2+++47SpQokau23nrrLSZOnMiAAQPS3f5/I2fPnmXs2LGcPXuW0qVL06lTJ+vT70Xk9qD8ljHlN5G8c8udio2Pj+f333/PdH1ISAhubrdcPXrL02kKcXy3/qlY5bf8oxwnju02PhXr5uZGpUqVbhwoInKbUX4TkfyWJ3fFioiIiIj9qbATERERcRAq7EREREQchAo7EREREQehwk5ERETEQaiwExEREXEQKuxEREREHIQKOxEREREHocJORERExEGosBMRERFxECrsRERERByECjsRERERB6HCTkRERMRBqLATERERcRAq7EREREQchAo7EREREQehwk5ERETEQaiwExEREXEQKuxEREREHISbvTsgBazocLD42bsXBSv+ZXv3oOC5jbV3D0Tsw38k4GQ5zhhv7x4UPMt4e/fglqUjdiIiIiIOQoWdiIiIiINQYSciIiLiIFTYiYiIiDgIFXYiIiIiDkKFnYiIiIiDUGEnIiIi4iBU2ImIiIg4CBV2IiIiIg5ChZ2IiIiIg1BhJyIiIuIgVNiJiIiIOAgVdiIiIiIOQoWdiIiIiINQYSciIiLiIFTYiYiIiDgIFXYiIiIiDkKFnYiIiIiDUGEnIiIi4iBU2ImIiIg4CBV2IiIiIg5ChZ2IiIiIg1BhJyIiIuIgVNiJiIiIOAgVdiIiIiIOIkeFXdOmTRkwYECm60NCQpg1a1aOOzF+/HjCwsKyFdu9e3c6duyY433cKm70M7wlJc6HhAqQ4AUJ4WDszjre+AgSqiXF1wTjqzTrP4WElpBQEhJcwTiQwT6fh4TKkOANCQGQ0BGMX/JqRDc2fxdUnAneEyH8Ldj9R+axcQkwcQvcOcuMv2c+rPvNNiYhEcZ+A5Vmgs9EM3bSFjAM27gjF6Dj+1DsFfCbBPXfhKh/83JkNzAPCAE8gHrADV5rPgKqJMVXB9K81hjAWKA04Ak0B9L8bPgH6AL4AUWAnsDlXPb/5ijH3bzbMsfZ5X0/GWgAeGG+7wvYvN0QMgs8JkG9RbD7z8xj4xLg5a1Q8Q0zvuZCWHfUNuZSLAxYB+VmgedkaPAO7EnT5rnL0H0NlJkOXpOh1XL47e88HlhWnCO/ueVlY3v27MHb2zsvm0xn9uzZGGl/Gd5GPv30UwoVKmTvbmRf4iowBoNlPljqgTEbEluDyxGwlEofb2yHxC5geQUsbcH4ABIfBpe9YLk7KeYKWBoBncDoncmO7wGXJ4E7gH8gcQIktgKXY2BxzZ+xJvvwIAxZD/PbQd0geGMntHkPDveFUj7p48d8A+//BAvbQ5USsOEoPLoSvnsWapU2Y6Z+D2/uhXcfgrtKwg+noeca8PeAvvXNmGP/QJN3oMc9MO5+8HOHw+fBI08/pllYBQwCFmImvVlASyASyOC1ZjvwBDAFeBB4H+gI7AOSXmumAm8AS4HywJikNg9jJkswk94ZYCMQB/QAeiW1d2tRjrux2y7H2e19fx3oBIQD7+TpiG5o1UEYtAEWtoV6ZWHWTmi5HCIjoFQG7+/Rm2H5z/B2OzPHrT8KD62C7c+k5Lhnv4CD5+G9h6CMLyz/CZq/B4dfhCA/80tsx1VQyAU+e9zMbzN2pMR4F87vQeMs+S1PT8WWLFkSLy+vTNfHxcXd9D78/f0pUqTITbdjL8WKFcPX19fe3cg+YxZYngWXHmCpBpYFgBcYizOJfwNoCS5DwFIVXF4G7gFjXkqMy9PgMgYszTPfr0svsDQGSwhY7gGXicAp4Pe8GVdWZm6HZ2tD91pQrRTMfxC8CsHi/RnHr/gJRtwHbe6ECsXg+brQurLZTrIdp6B9KLS9E0KKwiN3QYuKtt9ox3xjbvfaA2ayrFgM2lXJuJjMFzOA5zATTzXMBOgFvJtJ/GygFTAUqApMBO4B5iatNzCT52igA1ADWAacBtYkxRwB1gGLMJNtI2AOsDIp7taiHHdjt12Os8v7HmACMBDzSFABm7ETnrsHetSCaiVhYVKOezeTHPfeT/BSI2hTGSoUhRfuNf8/fYe5/r84+OQwTG0OjctBpWIwvqn574K9Zsxv/8DOP2BBW7g3CEJLwIIHzW0/OFgQg8ZZ8luOC7v4+HgiIiLw9/enRIkSjBkzxvrtMu1pCovFwoIFC2jfvj3e3t5MnjwZgFdffZWAgAB8fX3p2bMn165dy/b+056maNq0KX379mXAgAEULVqUgIAA3n77ba5cuUKPHj3w9fWlUqVKfP3119ZtEhIS6NmzJ+XLl8fT05PQ0FBmz56dbpz9+vWjSJEiFC9enOHDh9OtWzebfScmJjJlyhRrOzVr1uTjjz/Osv9pT1OEhIQwceJEnnjiCby9vQkKCmLevHk22/zyyy80atQIDw8PqlWrxqZNm7BYLKxZsybbP7dcMa4DP4ClWcoyi4s5b+zIZJud6Qs2ywPm8lz34woYSzC/EQXnvp3suB4P+85Aswopy1xczPmdpzLeJjY+/VE1z0KwLSplPjwYNp+AX/8y5388a65vVdmcT0yEr36FysWh9TIoPdU8BfzZkbwbW5aSXmtSv3YuSfOZvNbsSBMP5rfV5PgTwNk0Mf6YCS45Zgfm6Yk6qWKaJ+17V04GkGeU41L27fA5zm7vezu6nmCeMWieOsdZzPkdmVxyEpuQQY5zg++Tclx8IiQYWcfExpv/po5xsYB7qph841z5LceF3dKlS3Fzc2P37t3Mnj2bGTNmsGjRokzjx48fz0MPPcTPP//MM888w4cffsj48eN55ZVX2Lt3L6VLl2b+/Pk3NYilS5dSokQJdu/eTd++fXnhhRfo1KkTDRo0YN++fTzwwAM8/fTTXL16FTCTVdmyZfnoo484fPgwY8eO5aWXXuLDDz+0tvnaa6+xYsUKFi9ezLZt24iJiUmXZKZMmcKyZctYuHAhhw4dYuDAgTz11FNs3bo1R/2fNm0aNWvWZP/+/YwYMYL+/fuzceNGwEzQHTt2xMvLi127dvHWW28xatSoG7YZGxtLTEyMzZRzfwEJQECa5QHAuUy2OUv6w9oBSctzKHEBJPhBoh8Y68BlPVjy+XD9X1fN6+HSHiUr5QNnM7ku4oFKMGuHea1IYiJsPAarj8CZSykxwxtB57vhrrngMQHqLIR+9eHJGub681fg8nXzlG3LSvD109CxKjy6Crb+ni9DtZXVa53Za3f2BvFnUy3LKibt+8UNKJbFfvOXclwK585x+fm+t6O/rppFWECaU64B3pnnuJYVzaN8v/0NiYaZ4z49AmeS4n3dIbwsTPwfnL5k5tDlP5mFYnJMlRJwhz+M/AYu/mcWmK99D3/EpMTkG+fKbzm+eCc4OJiZM2disVgIDQ3l559/ZubMmTz33HMZxj/55JP06NHDOv/444/Ts2dPevbsCcCkSZPYtGlTjr7RplWzZk1Gjx4NwMiRI3n11VcpUaKEtU9jx45lwYIF/PTTT9SvX59ChQoxYcIE6/bly5dnx44dfPjhh3Tu3BmAOXPmMHLkSB566CEA5s6dy1dfpVw4GRsbyyuvvMKmTZsIDw8HoEKFCnz//fe8+eabNGnSJNv9b9iwISNGjADgzjvvZNu2bcycOZMWLVqwceNGjh07xpYtWwgMDARg8uTJtGjRIss2p0yZYjPG25LlyaSjf2cgcTokPg4u34HF44abFqiZraH353DXHLBYoGJR6B5me+r2o0PwwU+w/BHz9O6PZ2HQ11DGD7qGmckSoH0VGNDA/H9YafMU7lt7oElIAQ/KeSnHmZTjxGp2K3juC6gyDyyYl4n0CIN3D6TEvPcQPPM5BM0AVwvcUxqeuBt+OGOuL+QKn3aGnp9DsalmTPMK0LqSeVZT8kyOj9jVr18fi8VinQ8PD+e3334jISEhw/g6derYzB85coR69erZLEtOGrlVo0YN6/9dXV0pXrw41aunXLcQEGBW1OfPn7cumzdvHrVr16ZkyZL4+Pjw1ltvERVlHg6Ojo7m3Llz1K1b16bd2rVrW+ePHj3K1atXadGiBT4+PtZp2bJlHDt2LEf9Tzv+8PBwjhwxT8FFRkYSHBxsTXiATb8yM3LkSKKjo63TqVOZnEbMUgnAlfRH586R/ltKskDgfJpl55KW55DFHyyVzWvtXD4CfgFjdc7byYkSXuDqAufTfIM8fxkCM7nWraQ3fPoExIyC4wPhUF/zQuAKRVNihm+AYY3gsepQPQCeqgn9w+G171L26+YCVUvatl2lBERF5934MpXVa53Zaxd4g/jAVMuyikn7fonHvJMsF++ZPKAcZ1KOy8/3vR2V8DKLqnNXbJefu5J1jlvzOFx5CU4OgF/6gE+aHFexGGztDpdHwqmBsPs5iEu0jaldBg48D/8OhzODYd1T8Pd/UKFIHg8yLefKb/n+HLv8voMMSHcHlsVisVmWnKQTExMBWLlyJUOGDKFnz55s2LCBAwcO0KNHD65fv57tfV6+bP7iX7t2LQcOHLBOhw8fvuE1KAXB3d0dPz8/mynHLIWB2mBsTllmJJrzlkx+UVnqg/GN7TJjk7n8phhJU+xNtnMDhd3Mb5qbj6csS0w0r4+rf4Pr+zwKmXd/xSeap2LbVUlZdzXOvJ4kNVdLypG6wm5QJyjlGrxkv/0N5YrkejjZl/Rak/q1S0yaz6woCU8TD+adX8nx5TGTV+qYGMxrS5JjwoF/Ma9/SbY5ad+2xdGtSjnOPvIkx9ntfW9HhV3NAuub1DnOMOfDy2a9rYdbSo775Ah0CE0f410YSvuap1vXH804xt/DLBZ/+xv2noYOVdLH5Cnnym85PhW7a5ftBX87d+6kcuXKuLpm7xEUVatWZdeuXXTt2tWmjYK0bds2GjRowIsvvmhdlvobqL+/PwEBAezZs4fGjRsD5nUg+/btsz6Lqlq1ari7uxMVFZWjUxIZSTv+nTt3UrVqVQBCQ0M5deoU586ds34r37Nnz03tL0csA8DoAYm1wVLXfNwJV8DS3Vyf2A0IApdXkuL7gXE/JM4ASxswVgF7wbIwpU3jHyAK611BRmTSikCwBIJxHIwPwdICKAn8AYmvAZ5mm/ltYAPosRpqB5l3b72xA65cN++SBej+qXk7/ytJp4p2/QGnY6BmIPx5CV7+1kyUQxumtPlgKEz5DoKLmI87OXDWvC4vuU2AIQ3hiY/gvnLQtLyZFL/8Fb7pnv9jBsxHAXTDvNC3LuYdX1cw7yID6AoEYd7+D9AfaAJMB9pi3um1F3grab0FGABMAiqT8jiAMpiPDQDzbrNWmHerLcR8HEAE8HhSXMFTjgsDnCjH2eV9D2YOTM6FCcCBpOWVgHy+E35Qfei2BuqUMR/pNGsnXIkzT68CdF0NQb4wJenGgF1/mLktLBD+jIHxW80cNyxVjlt/1PzuHVocjv4DQzeaZxyS2wTzkpSS3ua1dj+fg/7roGMVeKBi/o7XHDTOkt9yXNhFRUUxaNAgevfuzb59+5gzZw7Tp0/P9vb9+/ene/fu1KlTh4YNG7JixQoOHTpEhQoVbrxxHqlcuTLLli1j/fr1lC9fnvfee489e/ZQvnx5a0zfvn2ZMmUKlSpVokqVKsyZM4eLFy9avxn7+voyZMgQBg4cSGJiIo0aNSI6Oppt27bh5+dHt27dAKhSpQpTpkyxXseSkW3btjF16lQ6duzIxo0b+eijj1i7di0ALVq0oGLFinTr1o2pU6dy6dIl67U2qU8X5RuXxyDxLzDGg3EWCAOXr8CSdCrWOGXeKZvM0gBclkPiWDBGAZXB5dOUZ9gBGJ+D0TPV/JNmQrCMBcs4wAOM75KKyItAAFjuA8v3GT87L691vhsuXIHxm82LiWsGwtqnISAp2UZF2x59uxYPYzfD8Yvm6YnWlWHpw1DEMyVmdhsYtxn6fmneKFHGF56rA2NS/cLsWNV8tMpr38GAr83HAXz0GDQql/9jBuAx4ALmAzeTXmvWkXLaPQrbg/wNMJ/FNBp4CTO5rSHlGU8AwzCTZy/Mb66NktpMfZ3kCsxk1yyp/Ucwnw1lH8pxTpbj7Pa+H4v5/LNkyV/yvgWa3sR4suGxu+HCVRi7xcxxYYGwrkvWOW50qhzXprJ5TV2RVOOJjjVvjPgjBop5wiNVYfL/mdfWJTtz2Xx+3rnL5lG9rjVsc2C+cp78luPCrmvXrvz333/UrVsXV1dX+vfvT69evbK9/WOPPcaxY8cYNmwY165d45FHHuGFF15g/fr1Oe1KrvXu3Zv9+/fz2GOPYbFYeOKJJ3jxxRdtHhcwfPhwzp49S9euXXF1daVXr160bNnS5lv7xIkTKVmyJFOmTOH48eMUKVKEe+65h5deeskaExkZSXR01tdIDR48mL179zJhwgT8/PyYMWMGLVu2BMzrXtasWcOzzz7LvffeS4UKFZg2bRrt2rXDw6OAbiJw6QP0yXid6+b0yyydwLVTFu11B7pnvt5SBlzXZr9/+aFPPXPKyOYetvNNQuDniKzb83WHGa3NKSs97jEnu4lImjKyJYNlnZKmzFiAl5OmzBTjVnoYsXKcE+Y4u7zvlyRNdhJR15wysqW77XyTEDicye+AZJ3vMqes9KtnTnbjHPnNYtzOjzgvQImJiVStWpXOnTszceLEXLcTHh5Os2bNmDRpEmA+42nAgAE5+hM827Zto1GjRhw9epSKFbN3CDsmJgZ/f39wuQiW3FyLchuLnWTvHhQ8t7H27oEdxAD+REdH5/J6K+fmMDmOaMw/3+REjPH27kHBs4y3dw8KWPbzW0H9raLbzsmTJ9mwYQNNmjQhNjaWuXPncuLECZ588slctRcbG8vPP//MoUOH6NevX462Xb16NT4+PlSuXJmjR4/Sv39/GjZsmO2EJyKSlnKciGPK97ticyr1bfVpp++++67A+uHi4sKSJUu49957adiwIT///DObNm2yXvCbU19//TX/93//R/v27Xn00UdztO2lS5fo06cPVapUoXv37tx777189tlnueqHiNiXclx6ynEieeeWOxV79OjRTNcFBQXh6emZ6XrJnE7FOhmdirV3ZzKlHJc/dCrWyehUbKZuuVOxlSpVsncXRETyjXKciOSnW+5UrIiIiIjkjgo7EREREQehwk5ERETEQaiwExEREXEQKuxEREREHIQKOxEREREHocJORERExEGosBMRERFxECrsRERERByECjsRERERB6HCTkRERMRBqLATERERcRAq7EREREQchAo7EREREQehwk5ERETEQaiwExEREXEQKuxEREREHIQKOxEREREHocJORERExEG42bsDUjAMw0j6T4x9O2IPMdfs3QM7cMLXOWnM1ve6OJWU190J3/sxsfbugR042+uc/fymws5JXLp0yfyPUQ6c7fdeMXt3wB6m2LsDdnPp0iX8/f3t3Q0pYNYcR7Bd+2EXTvl2f9XeHbCL7OQ3i6Gvt04hMTGR06dP4+vri8ViKdB9x8TEEBwczKlTp/Dz8yvQfduLM44Z7DtuwzC4dOkSZcqUwcVFV5k4G+W4gqUx37r5TUfsnISLiwtly5a1ax/8/PycJgEkc8Yxg/3GrSN1zks5zj405oKT3fymr7UiIiIiDkKFnYiIiIiDUGEn+c7d3Z1x48bh7u5u764UGGccMzjvuMW5OeP7XmO+denmCREREREHoSN2IiIiIg5ChZ2IiIiIg1BhJyIiIuIgVNiJiIiIOAgVdiIiIiIOQoWdiIiIiINQYSciIiLiIFTYiYiIiDiI/we1HPqH9d3LPQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fp_text_vision_scores = softmax(embeddings[ModalityType.VISION] @ embeddings[ModalityType.TEXT].T, axis=-1)\n",
    "int_text_vision_scores = softmax(quantized_embeddings[ModalityType.VISION] @ quantized_embeddings[ModalityType.TEXT].T, axis=-1)\n",
    "\n",
    "visualize_prob_matrices(fp_text_vision_scores, int_text_vision_scores, text_list, image_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "5519db85",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnYAAAD/CAYAAAB8dJvAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABDEElEQVR4nO3dd3gU5drH8e+SYHqhBgKR0AzgQYLUCAIeQBCkWAAVDWAUBIP0Jl1AFISANBU8FAUB9YAFpamgQmgCHmlRigRFir6YUCQJybx/TLLJhnRSyO7vc117yczc88zzZDO399RYDMMwEBEREZFir0RRd0BERERE8ocKOxERERE7ocJORERExE6osBMRERGxEyrsREREROyECjsRERERO6HCTkRERMROqLATERERsRMq7ERERETshAo7ETuxbNkyLBYL+/btK+quiIjkK4vFQnh4eFF3o1hQYSfFQkrRktFn9OjR1rjAwECbZeXLl+f+++9n3bp1Nu3t2bOHAQMG0KBBA0qWLInFYsly++fPn6dfv35UqlQJV1dXAgMDCQsLK5CxiohjyujgbNKkSVgsFvz8/Lh27dpN6wQGBvLwww8D0Lt370zzZNpP7969reuvXbuWpk2b4uvrS5kyZWjZsiUbNmwo8LFKwXEu6g6I5MYrr7xC1apVbeb961//spkODg5m2LBhAJw9e5a3336bRx99lEWLFvHCCy8A8MUXX7BkyRLuueceqlWrxs8//5zpNs+cOUOzZs0AeOGFF6hUqRJnz55lz549+Tk0EZFMXbhwgUWLFllzW0b69etHmzZtrNOnTp1iwoQJ9O3bl/vvv986v3r16gDMmzePl156iY4dO/Laa69x/fp1li1bxsMPP8zHH3/Mo48+WnADkgKjwk6KlYceeoiGDRtmGVOpUiWefvpp63RoaCg1atQgIiLCWtj179+fUaNG4ebmRnh4eJaFXb9+/XB2dmbv3r2UKVMmfwaSA0lJScTHx+Pq6lpo2xSR21NwcDAzZ85kwIABuLm5ZRgTEhJCSEiIdXrfvn1MmDCBkJAQm5yYYt68eTRq1IjPPvvMetXi2WefpVKlSixfvrxAC7urV6/i4eFRYO07Ml2KFbtXoUIFateuzalTp6zz/Pz8Mk2OaR07dowvv/ySESNGUKZMGa5fv05CQkKOt51yGeXYsWN0794db29vypQpw6BBg7h+/bpNbMo9JCtXruTuu+/GxcWFjRs3AnDgwAEeeughvL298fT0pHXr1uzatSvDbV67do1+/fpRpkwZvL29CQ0N5dKlSzYx+/bto127dpQtWxY3NzeqVq3Ks88+m+NxiUjhmjBhAufPn2fRokX51mZsbCzly5e3uRUlJcdklx9//fVXLBYLb7zxBhEREVSpUgU3NzdatmzJoUOHbGJ79+6Np6cnJ06coEOHDnh5edGzZ0/ALPCGDRtGQEAALi4uBAUF8cYbb2AYRobbXblyJUFBQbi6utKgQQO+/fZbm+WXL19m8ODBBAYG4uLiQvny5Wnbti379+/Py4+oWNIZOylWYmJi+PPPP23mlS1bNst1EhISOHPmTJ7Otm3duhUwC8HWrVvz9ddf4+TkRNu2bVm0aBGBgYE5aqd79+4EBgYyffp0du3axZtvvsmlS5dYsWKFTdzXX3/N2rVrCQ8Pp2zZsgQGBnL48GHuv/9+vL29GTlyJCVLluTtt9+mVatWbN++nSZNmti0ER4ejq+vL5MmTSIqKopFixZx+vRptm3bhsVi4cKFCzz44IOUK1eO0aNH4+vry6+//sp///vfXP98RKRw3H///fz73/9mxowZ9O/fP0cHptlp1aoVH330EfPmzaNTp05cv36defPmERMTw6BBg3LUxooVK7h8+TIvvvgi169fZ+7cufz73//mp59+ws/Pzxp348YN2rVrR/PmzXnjjTdwd3fHMAw6d+7MN998Q1hYGMHBwWzatIkRI0bw+++/ExERYbOt7du3s2bNGl566SVcXFxYuHAh7du3Z8+ePdZbcl544QU++ugjwsPDqVOnDn/99Rfff/89R48e5d57773ln1mxYIgUA0uXLjWADD9pValSxXjwwQeNixcvGhcvXjR+/PFH44knnjAAY+DAgRm2/eKLL97UToqXXnrJAIwyZcoY7du3N9asWWPMnDnT8PT0NKpXr25cvXo1y35PnDjRAIzOnTvbzB8wYIABGD/++KN1HmCUKFHCOHz4sE1s165djTvuuMM4ceKEdd7Zs2cNLy8vo0WLFjf9jBo0aGDEx8db58+YMcMAjE8++cQwDMNYt26dARh79+7Nsu8iUrhS9uG0+2ZKDrl48aKxfft2AzBmz55tXV6lShWjY8eOGba3d+9eAzCWLl2a4fLz588brVu3tsmnZcuWNXbu3JltX0+dOmUAhpubm/Hbb79Z5+/evdsAjCFDhljn9erVywCM0aNH27Sxfv16AzCmTp1qM//xxx83LBaLcfz4ceu8lP7t27fPOu/06dOGq6ur8cgjj1jn+fj4GC+++GK2/bdnuhQrxcqCBQvYsmWLzSe9zZs3U65cOcqVK0e9evX48MMPeeaZZ3j99ddzvb0rV64A5uXcDRs20L17d4YPH87ixYs5ceIEq1atylE7L774os30wIEDAfMhjrRatmxJnTp1rNOJiYls3ryZrl27Uq1aNev8ihUr8tRTT/H9998TGxtr00bfvn0pWbKkdbp///44Oztbt+Xr6wvA559/nqvLyiJStFq0aMEDDzzAjBkz+Oeff265PXd3d4KCgujVqxcffvgh//nPf6hYsSKPPvoox48fz1EbXbt2pVKlStbpxo0b06RJk5tyG5i5KK0vvvgCJycnXnrpJZv5w4YNwzAMvvzyS5v5ISEhNGjQwDp955130qVLFzZt2kRiYiJg5rfdu3dz9uzZHPXfHqmwk2KlcePGtGnTxuaTXpMmTdiyZQtbt25l586d/Pnnn6xYsSJPly5S1unevTslSqTuLt26dcPZ2ZmdO3fmqJ2aNWvaTFevXp0SJUrw66+/2sxP/8TvxYsXuXbtGkFBQTe1Wbt2bZKSkjhz5kyW2/L09KRixYrWbbVs2ZLHHnuMyZMnU7ZsWbp06cLSpUuJi4vL0VhEpOhMmjSJc+fO8dZbb91yW926dSM6Opply5bx+OOP06dPH7Zt20Z8fDxjx47NURvp8w3AXXfddVNuc3Z2pnLlyjbzTp8+jb+/P15eXjbza9eubV2ek21du3aNixcvAjBjxgwOHTpEQEAAjRs3ZtKkSZw8eTJHY7EXKuzE7pQtW5Y2bdrQunVrQkJCrGeo8sLf3x/A5l4RACcnJ8qUKXPTQwk5ldl78/LjvpmcbPujjz4iMjKS8PBwfv/9d5599lkaNGhgPUMpIrenFi1a0KpVq1s+a3fy5Ek2btxI586dbeaXLl2a5s2bs2PHjlvtqg0XFxebg+OC0r17d06ePMm8efPw9/dn5syZ3H333Ted/bNnKuxEspBy2v/333+3mR8fH8+ff/5JuXLlctTOL7/8YjN9/PhxkpKSsn34oly5cri7uxMVFXXTsmPHjlGiRAkCAgKy3NaVK1f4448/btpW06ZNmTZtGvv27WPlypUcPnyY1atX52g8IlJ0Us7avf3223lu4/z58wDWS5hpJSQkcOPGjRy1kz7fAPz88885erCsSpUqnD17lsuXL9vMP3bsmHV5Trbl7u5uk4srVqzIgAEDWL9+PadOnaJMmTJMmzYtJ8OxCyrsRLLQqlUrypcvz8qVK21eT7Js2TISExNp27ZtjtpZsGCBzfS8efMA8718WXFycuLBBx/kk08+sbm0cf78eVatWkXz5s3x9va2Weedd96xuXdu0aJF3Lhxw7qtS5cu3fQqgeDgYABdjhUpBlq2bEmrVq14/fXXb3ptUk7VqFGDEiVKsGbNGpt88Ntvv/Hdd99Rv379HLWzfv16mwPfPXv2sHv37mxzG0CHDh1ITExk/vz5NvMjIiKwWCw3tREZGWnz2pIzZ87wySef8OCDD+Lk5ERiYiIxMTE265QvXx5/f3+Hym163Yk4pNOnT/Pee+8BWP98z9SpUwHzKPGZZ54BzMsHM2fOpFevXrRo0YJnnnmG6Oho5s6dy/3335/jF3ieOnWKzp070759eyIjI3n//fd56qmnqFevXrbrTp06lS1bttC8eXMGDBiAs7Mzb7/9NnFxccyYMeOm+Pj4eFq3bk337t2Jiopi4cKFNG/e3HrJZfny5SxcuJBHHnmE6tWrc/nyZRYvXoy3tzcdOnTI0XhEpGhNnDiRBx54IM/rlytXjmeffZYlS5bQunVrHn30US5fvszChQv5559/GDNmTI7aqVGjBs2bN6d///7ExcUxZ84cypQpw8iRI7Ndt1OnTjzwwAOMHTuWX3/9lXr16rF582Y++eQTBg8ebP0LGSn+9a9/0a5dO5vXnQBMnjwZMN9hV7lyZR5//HHq1auHp6cnW7duZe/evcyaNSuXP6FirIifyhXJkYxeA5CRrB79T+ubb77J9PUpLVu2vCn+gw8+MOrVq2e4uLgYfn5+Rnh4uBEbG5vtdlJeVXDkyBHj8ccfN7y8vIxSpUoZ4eHhxj///GMTC2T6mP7+/fuNdu3aGZ6enoa7u7vxwAMP3PRKgpSf0fbt242+ffsapUqVMjw9PY2ePXsaf/31l01bTz75pHHnnXcaLi4uRvny5Y2HH37Y5jUCIlL4snvdSXotW7Y0gDy/7iQhIcGYN2+eERwcbHh6ehqenp7GAw88YHz99dfZ9jXldSczZ840Zs2aZQQEBBguLi7G/fffb/MaJ8MwX3fi4eGRYTuXL182hgwZYvj7+xslS5Y0atasacycOdNISkqyiUvJj++//75Rs2ZNw8XFxahfv77xzTffWGPi4uKMESNGGPXq1TO8vLwMDw8Po169esbChQuzHY89sRhGJq93FpFbNmnSJCZPnszFixezfZGyiEhx8euvv1K1alVmzpzJ8OHDi7o7kobusRMRERGxEyrsREREROyECjsRERERO6F77ERERETshM7YiYiIiNgJFXYiIiIidkKFnYiIiIidUGEnkg8CAwOZM2dOljEWi4X169fnuu3evXvTtWvXPPWrKOTkZyEixYfyW6rikN9U2EmORUZG4uTkRMeOHYu6KwWuIMb6xx9/5OjvJxYlR/qORdJzlN9/5Tf7/n5V2EmOvfvuuwwcOJBvv/2Ws2fPFvr24+PjC21bBTHWChUq4OLikunyhISEfNnOrSjq71ikKBX1739h5TjlN/vObyrsJEeuXLnCmjVr6N+/Px07dmTZsmXZrhMXF8eoUaMICAjAxcWFGjVq8O677wKQmJhIWFgYVatWxc3NjaCgIObOnWuzfsop+mnTpuHv709QUFBBDO0meRkrmH+A+sknn8TDw4NKlSqxYMECm+VpL1X8+uuvWCwW1qxZQ8uWLXF1dWXlypUkJiYydOhQfH19rX9Iu7DeSJSXcV+4cIFOnTrh5uZG1apVWbly5U0x0dHRdOnSBU9PT7y9venevTvnz5+3iZk6dSrly5fHy8uL5557jtGjRxMcHJxPIxPJnqPkOOU3B8hvRfmHaqX4ePfdd42GDRsahmEYn332mVG9evWb/khzet27dzcCAgKM//73v8aJEyeMrVu3GqtXrzYMwzDi4+ONCRMmGHv37jVOnjxpvP/++4a7u7uxZs0a6/q9evUyPD09jWeeecY4dOiQcejQoYIbYBp5GWuVKlUMLy8vY/r06UZUVJTx5ptvGk5OTsbmzZutMYCxbt06wzBS/4B2YGCg8fHHHxsnT540zp49a7z++utGqVKljI8//tg4cuSIERYWZnh5eRldunQpqOFa5WXcDz30kFGvXj0jMjLS2Ldvn3HfffcZbm5uRkREhGEYhpGYmGgEBwcbzZs3N/bt22fs2rXLaNCggdGyZUtrG++//77h6upq/Oc//zGioqKMyZMnG97e3ka9evUKaKQiN3OUHKf8Zv/5TYWd5Mh9991nzJkzxzAMw0hISDDKli1rfPPNN5nGR0VFGYCxZcuWHG/jxRdfNB577DHrdK9evQw/Pz8jLi4uz/3Oi9yO1TDMxNe+fXubeT169DAeeugh63RGiS9lOykqVqxozJgxwzqdkJBgVK5cuVASX16/4z179ljnHT161ACsiW/z5s2Gk5OTER0dbY05fPiwzXpNmjQxXnzxRZu2mzVrpsJOCpWj5DjlN/vPb7oUK9mKiopiz549PPnkkwA4OzvTo0cP6yWHjBw8eBAnJydatmyZacyCBQto0KAB5cqVw9PTk3feeYfo6GibmLp163LHHXfkz0ByIC9jTRESEnLT9NGjR7Ncp2HDhtZ/x8TE8Mcff9CkSRPrPGdnZ5uYgpKXcR89ehRnZ2caNGhgnVerVi18fX1tYgICAggICLDOq1OnDr6+vtafTVRUFI0bN7ZpO/20SEFylByn/OYY+c25QFsXu/Duu+9y48YN/P39rfMMw8DFxYX58+fj4+Nz0zpubm5Ztrl69WqGDx/OrFmzCAkJwcvLi5kzZ7J7926bOA8Pj/wZRA7lZay3orDHl5nCHrfI7cRRcpzym2PkN52xkyzduHGDFStWMGvWLA4ePGj9/Pjjj/j7+/PBBx9kuF7dunVJSkpi+/btGS7fsWMH9913HwMGDKB+/frUqFGDEydOFORQspXXsabYtWvXTdO1a9fO8fZ9fHyoWLGiTeK/ceMGP/zwQ+4Gkkt5HXetWrVu6l9UVBR///23dbp27dqcOXOGM2fOWOcdOXKEv//+mzp16gAQFBTE3r17bdpOPy1SUBwlxym/OVB+K7CLvGIX1q1bZ9xxxx3G33//fdOykSNHWm9GzUjv3r2NgIAAY926dcbJkyeNb775xnrj8Ny5cw1vb29j48aNRlRUlDFu3Libbijt1atXodx7keJWxlqlShXD29vbeP31142oqChj/vz5hpOTk7Fx40ZrDBncg3LgwAGbdl577TWjdOnSxrp164yjR48azz//fIHfXHwr427fvr1Rv359Y9euXca+ffuM5s2b29xcnJSUZAQHBxv333+/8cMPPxi7d+/O8OZiNzc3Y9myZcbPP/9sTJkyxfD29jaCg4Pze6giN3GUHKf85jj5TYWdZOnhhx82OnTokOGy3bt3G4Dx448/Zrj8n3/+MYYMGWJUrFjRuOOOO4waNWoY//nPfwzDMIzr168bvXv3Nnx8fAxfX1+jf//+xujRo4u0sLuVsVapUsWYPHmy0a1bN8Pd3d2oUKGCMXfuXJuYnCS+hIQEY9CgQYa3t7fh6+trDB061AgNDS3Qn8OtjPuPP/4wOnbsaLi4uBh33nmnsWLFCqNKlSrWxGcYhnH69Gmjc+fOhoeHh+Hl5WV069bNOHfunE07r7zyilG2bFnD09PTePbZZ42XXnrJaNq0ab6NUSQzjpLjlN9uZq/5zWIYhfQSGRGRHGrbti0VKlTgvffeK+quiIjkq4LOb3p4QkSK1LVr13jrrbdo164dTk5OfPDBB2zdupUtW7YUdddERG5JUeQ3nbETkSL1zz//0KlTJw4cOMD169cJCgpi3LhxPProo0XdNRGRW1IU+U2FnYiIiIid0OtOREREROyECjsRERERO6HCTkRERMROqLATERERsRMq7KTAxcXFMWnSJOLi4oq6K4XGEccMjjtucWyO+HuvMd++9FSsFLjY2Fh8fHyIiYnB29u7qLtTKBxxzOC44xbH5oi/9xrz7TtmnbETERERsRMq7ERERETshP6kmINISkri7NmzeHl5YbFYCnXbsbGxNv91BI44ZijacRuGweXLl/H396dECR2zOhrluMKlMReu3OQ33WPnIH777TcCAgKKuhsiBe7MmTNUrly5qLshhUw5ThxBTvKbztg5CC8vr+R/nQFu35s+JZ/ETC/qHhS+2DgIiEjzuy6ORDnOwThajstFflNh5yBSL014o6TnALxdiroHRaawL8PJ7UE5zsE4aI7LSX7TjSgiIiIidkKFnYiIiIidUGEnIiIiYidU2ImIiIjYCRV2IiIiInZChZ2IiIiInVBhJyIiImInVNiJiIiI2AkVdiIiIiJ2QoWdiIiIiJ1QYSciIiJiJ1TYiYiIiNgJFXYiIiIidkKFnYiIiIidUGEnIiIiYidU2ImIiIjYCRV2IiIiInZChZ2IiIiInVBhJyIiImInVNiJiIiI2AkVdiIiIiJ2QoWdiIiIiJ3It8KuVatWDB48OL+ay9KkSZMIDg4ulG0VhN69e9O1a9ei7kYuLAACAVegCbAnm/gPgVrJ8XWBL9ItN4AJQEXADWgD/JIu5v+AnoA34AuEAVfy2P+8crBxf3saOn0A/rPAMhnWH8t+nW2/wr1vg8tUqPEmLDt4c8yCPRA4B1ynQpMlsOd32+XXb8CLG6DMDPB8FR5bC+cL+7vOmvJbzhW//AYOt68DDjdmB8pvxfKM3fDhw/nqq6+Kuht5NnfuXJYtW1bU3cihNcBQYCKwH6gHtAMuZBK/E3gSc4c9AHRN/hxKEzMDeBN4C9gNeCS3eT1NTE/gMLAF+Bz4Fuh768PJMQcc99V4qOcHCzrkLP7UJei4Ch4IhIP9YHBTeO5T2HQ8NWbNIRi6GSa2hP39zPbbvQ8XrqbGDNkIn/0MH3aD7b3h7GV4dG1+jqxYUX4rbA64rzvimB0ov1kMwzDyo6FWrVoRHBzMnDlz8qM5yWexsbH4+PgAMZhHSznVBGgEzE+eTgICgIHA6AziewBXMXfaFE2BYMwd3gD8gWHA8OTlMYAfsAx4AjgK1AH2Ag2TYzYCHYDfktcvaMV83MaknMdmxDIZ1vWArrUyjxm1BTb8AocGpM574iP4+zpsfNqcbrIEGvnD/ORkmmRAQAQMbAyjm0PMdSg3E1Y9Bo/XMWOO/Qm1F0BkGDStnPM+x8aBz2vExMTg7Z2b3/HsKb/d/pTjcsMOxnwrOc7O81uezthdvXqV0NBQPD09qVixIrNmzbJZfunSJUJDQylVqhTu7u489NBD/PKL7SnZxYsXExAQgLu7O4888gizZ8/G19c3R9tPf6ki5dT/q6++ip+fH76+vrzyyivcuHGDESNGULp0aSpXrszSpUtt2hk1ahR33XUX7u7uVKtWjfHjx5OQkGATM3XqVMqXL4+XlxfPPfcco0ePvukyyZIlS6hduzaurq7UqlWLhQsXZtn/9JcqWrVqRXh4OOHh4fj4+FC2bFnGjx9P2pr7jz/+oGPHjri5uVG1alVWrVpFYGBgAf+PJh74AfOUeooSydORmawTmS4ezKO2lPhTwLl0MT6YiSYlJhLzNH3DNDFtkre9OzcDyCNHHXcuRf4GbarZzmtX3ZwPEJ8IP5y1jSlhMadTYn74AxKSbGNqlYU7fSDyTMH2PxPKb8E2Mfab38Ax93VHHHMeFOP8lqfCbsSIEWzfvp1PPvmEzZs3s23bNvbv329d3rt3b/bt28enn35KZGQkhmHQoUMHa1LZsWMHL7zwAoMGDeLgwYO0bduWadOm3dJAvv76a86ePcu3337L7NmzmThxIg8//DClSpVi9+7dvPDCC/Tr14/ffvvNuo6XlxfLli3jyJEjzJ07l8WLFxMREWFdvnLlSqZNm8brr7/ODz/8wJ133smiRYtstrty5UomTJjAtGnTOHr0KK+++irjx49n+fLluer/8uXLcXZ2Zs+ePcydO5fZs2ezZMkS6/LQ0FDOnj3Ltm3b+Pjjj3nnnXe4cCGz0+b55U8gEfOoKy0/zJ04I+eyiT+XZl5WMeXTLXcGSmex3fzkqOPOpXNXwM/Ddp6fp3lk+U8C/HkNEo0MYjzMdVPauMMJfF0zjylkym/YxNhvfgPH3Ncdccx5UIzzm3NuV7hy5Qrvvvsu77//Pq1btwbMnbZyZfOU4i+//MKnn37Kjh07uO+++wAzOQQEBLB+/Xq6devGvHnzeOihhxg+3Dxle9ddd7Fz504+//zzjDeaA6VLl+bNN9+kRIkSBAUFMWPGDK5du8bLL78MwJgxY3jttdf4/vvveeKJJwAYN26cdf3AwECGDx/O6tWrGTlyJADz5s0jLCyMPn36ADBhwgQ2b97MlSupX8jEiROZNWsWjz76KABVq1blyJEjvP322/Tq1SvH/Q8ICCAiIgKLxUJQUBA//fQTERERPP/88xw7doytW7eyd+9eGjY0j3aWLFlCzZo1M20vLi6OuLg463RsbGyO+yLiqJTfikd+A+U4kczk+ozdiRMniI+Pp0mTJtZ5pUuXJigoCICjR4/i7Oxss7xMmTIEBQVx9OhRAKKiomjcuLFNu+mnc+vuu++mRInU4fj5+VG3bl3rtJOTE2XKlLE5ClyzZg3NmjWjQoUKeHp6Mm7cOKKjo63Ls+vn1atXOXHiBGFhYXh6elo/U6dO5cSJE7nqf9OmTbFYLNbpkJAQfvnlFxITE4mKisLZ2Zl7773XurxGjRqUKlUq0/amT5+Oj4+P9RMQEJCr/pjKAk7A+XTzzwMVMlmnQjbxFdLMyyom/dH6DcwnqjLbbn5y1HHnUgVPOH/Vdt75K+DtAm4loaw7OFkyiLlqrpvSRnyied9KZjGFSPmteOQ3UI7LO0cccx4U4/xWLJ+KzUjJkiVtpi0WS4bzkpKSAIiMjKRnz5506NCBzz//nAMHDjB27Fji4+NzvM2UI9vFixdz8OBB6+fQoUPs2rXrFkd0a8aMGUNMTIz1c+ZMXq7n3wE0ANI+oZeUPB2SyToh6eLBfAIqJb4q5k6cNiYW8x6LlJgQ4G/M+0BSfJ287SYUPEcddy6FVIavTtnO23LSnA/mJYgG/vDVydTlSYY5nRLToCKULGEbE/UnRMdASF7+R22flN9uphyXV4445jwoxvkt14Vd9erVKVmyJLt3p97seOnSJX7++WcAateuzY0bN2yW//XXX0RFRVGnjvlUSFBQEHv37rVpN/10Qdu5cydVqlRh7NixNGzYkJo1a3L69GmbmOz66efnh7+/PydPnqRGjRo2n6pVq+aqP2l/XgC7du2iZs2aODk5ERQUxI0bNzhw4IB1+fHjx7l06VKm7bm4uODt7W3zyZuhwGJgOeZTTf0xn47qk7w8FBiTJn4Q5pNOs4BjwCRgHxCevNwCDAamAp8CPyW34Y/5+DxAbaA98Dzmu5V2JK//BIXzRCw45LivxMPBc+YHzMf9D54zkxDAmK0Qui41/oWGcPISjNxiPum1cC+sPQxDmqbGDG0Ki/fD8oNw9CL0/xyuJkCfYHO5jyuE1TdfGfDNKfNm5D6fmIkxN0+M5RPlt+KR30A57tY44JgdKL/l+h47T09PwsLCGDFiBGXKlKF8+fKMHTvWepmgZs2adOnSheeff563334bLy8vRo8eTaVKlejSpQsAAwcOpEWLFsyePZtOnTrx9ddf8+WXX9qcqi9oNWvWJDo6mtWrV9OoUSM2bNjAunXrbGIGDhzI888/T8OGDbnvvvtYs2YN//vf/6hWLfUJl8mTJ/PSSy/h4+ND+/btiYuLY9++fVy6dImhQ4cC0Lp1ax555BHCw8PJTHR0NEOHDqVfv37s37+fefPmWZ/Gq1WrFm3atKFv374sWrSIkiVLMmzYMNzc3ArhZ9YDuIj54slzmI+3byT1JtlobI8P7gNWAeOAl4GawHrgX2liRmImkb6YR3DNk9tMe4PpSsydvnVy+49hviOpsDjguPedhQfS3BQ/dLP53171YFlX+ONKahIEqFoKNjwFQzbB3N1Q2RuWdIZ2NVJjevwLLl6DCdvMm4WDK8DGnuZNyCki2kOJTeaLO+MSzSfPFnYsyJFmSvnN0fIbOOS+7ohjdqD8luvCDmDmzJlcuXKFTp064eXlxbBhw4iJSf2BLF26lEGDBvHwww8THx9PixYt+OKLL6yXDpo1a8Zbb73F5MmTGTduHO3atWPIkCHMnz8/s03mu86dOzNkyBDCw8OJi4ujY8eOjB8/nkmTJlljevbsycmTJxk+fDjXr1+ne/fu9O7dmz17Ut/Q/dxzz+Hu7s7MmTMZMWIEHh4e1K1b1+Yt9SdOnODPP//Msj+hoaH8888/NG7cGCcnJwYNGkTfvqkvblyxYgVhYWG0aNGCChUqMH36dA4fPoyrq2sWreaXcFKPzNLblsG8bsmfzFiAV5I/mSmNmUiKkoONu1UgGBMzX76sa8brHOiXdbvhjc1PZlydYUFH83MbUH5ztPwGDrevAw43ZgfKb/n2guJblfJ01HfffVfUXclS27ZtqVChAu+9916e23jyySdxcnLi/fffB/L28tPffvuNgIAAtm7dan16Lyt5f3mnFEu3+oLi4qgAX1B8q5TfCja/gXKcw3G0HJeL/JanM3b54Y033qBt27Z4eHjw5Zdfsnz58mxffFnYrl27xltvvUW7du1wcnLigw8+YOvWrWzZsiVP7d24cYOff/6ZyMhI+vXL5iggna+//porV65Qt25d/vjjD0aOHElgYCAtWrTIU19EpOAovym/iRSVInsqds+ePbRt25a6devy1ltv8eabb/Lcc88B5qP9aR+vT/tZuXJlofXRYrHwxRdf0KJFCxo0aMBnn33Gxx9/TJs26d/AnTOHDh2iYcOG3H333bzwwgu5WjchIYGXX36Zu+++m0ceeYRy5cqxbdu2m56ME5Gip/ym/CZSVG6bS7FpnT59+qY/fZPCz88PLy+vQu5R8afLFA7G0S5TwG19KTYt5beCoRznYBwtxxWHS7FZqVKlSlF3QUSkQCi/iUhBspsXFIuIiIg4OhV2IiIiInZChZ2IiIiInVBhJyIiImInVNiJiIiI2AkVdiIiIiJ2QoWdiIiIiJ1QYSciIiJiJ1TYiYiIiNgJFXYiIiIidkKFnYiIiIidUGEnIiIiYidU2ImIiIjYCRV2IiIiInZChZ2IiIiInVBhJyIiImInVNiJiIiI2AkVdiIiIiJ2QoWdiIiIiJ1QYSciIiJiJ5yLugMiBc6YVNQ9KHyWSUXdgyIQC7xW1J0QKXzKcQ4g5/lNZ+xERERE7IQKOxERERE7ocJORERExE6osBMRERGxEyrsREREROyECjsRERERO6HCTkRERMROqLATERERsRMq7ERERETshAo7ERERETuhwk5ERETETqiwExEREbETKuxERERE7IQKOxERERE7ocJORERExE6osBMRERGxEyrsREREROyECjsRERERO6HCTkRERMROqLATERERsRMq7ERERETshAo7ERERETuhwk5ERETETqiwExEREbETKuwkBxYAgYAr0ATYk038h0Ct5Pi6wBfplhvABKAi4Aa0AX5JF/N/QE/AG/AFwoAreex/Ln17Gjp9AP6zwDIZ1h/Lfp1tv8K9b4PLVKjxJiw7eHPMgj0QOAdcp0KTJbDnd9vl12/AixugzAzwfBUeWwvnC2nMqZ3Eob5rEcDhfu8dNsc5xvdcLAu7Vq1aMXjw4KLuRp4FBgYyZ86cou5GDq0BhgITgf1APaAdcCGT+J3Ak5i/vAeArsmfQ2liZgBvAm8BuwGP5Davp4npCRwGtgCfA98CfW99ODlxNR7q+cGCDjmLP3UJOq6CBwLhYD8Y3BSe+xQ2HU+NWXMIhm6GiS1hfz+z/Xbvw4WrqTFDNsJnP8OH3WB7bzh7GR5dm58jy4YDfte3IeW3wuaAv/cOmeMc53u2GIZhFOgWCsD//d//UbJkSby8vIq6K3ly8eJFPDw8cHd3L7RtxsbG4uPjA8RgHjnkVBOgETA/eToJCAAGAqMziO8BXMX8BU7RFAjG/OU3AH9gGDA8eXkM4AcsA54AjgJ1gL1Aw+SYjUAH4Lfk9XPBmJS7+LQsk2FdD+haK/OYUVtgwy9waEDqvCc+gr+vw8anzekmS6CRP8xPTqRJBgREwMDGMLo5xFyHcjNh1WPweB0z5tifUHsBRIZB08q57Pek3MWbnaR4f9exgA8xMTF4e+fmd/z2ovyWN8pxeeQwOa64f885z2+33Rm7+Pj4bGNKly5dbJMeQLly5Qo96eVNPPAD5unlFCWSpyMzWScyXTyYRzAp8aeAc+lifDB3upSYSMxT1g3TxLRJ3vbu3AygcET+Bm2q2c5rV92cDxCfCD+ctY0pYTGnU2J++AMSkmxjapWFO30g8kzB9t/sJPquC57y2+1Gv/c5UuxznGN9z/lS2CUlJTFjxgxq1KiBi4sLd955J9OmTQNg1KhR3HXXXbi7u1OtWjXGjx9PQkKCdd1JkyYRHBzMkiVLqFq1Kq6urtluL/2lisDAQKZOnUpoaCienp5UqVKFTz/9lIsXL9KlSxc8PT2555572Ldvn3Wdv/76iyeffJJKlSrh7u5O3bp1+eCDD2y2c/nyZXr27ImHhwcVK1YkIiLipm3HxcUxfPhwKlWqhIeHB02aNGHbtm1Z9j/9pQqLxcKiRYt46KGHcHNzo1q1anz00Uc26+zcuZPg4GBcXV1p2LAh69evx2KxcPDgwWx/Xnn3J5CIeQSSlh/mL3RGzmUTfy7NvKxiyqdb7gyUzmK7RejcFfDzsJ3n5wmxcfBPAvx5DRKNDGI8zHVT2rjDCXxdM48pUPquM6P8Zq/5DfR7n0PFPsc51vecL4XdmDFjeO211xg/fjxHjhxh1apV+PmZg/Xy8mLZsmUcOXKEuXPnsnjxYiIiImzWP378OB9//DH//e9/87wjR0RE0KxZMw4cOEDHjh155plnCA0N5emnn2b//v1Ur16d0NBQUq48X79+nQYNGrBhwwYOHTpE3759eeaZZ9izJ/VmyqFDh7Jjxw4+/fRTtmzZwnfffcf+/fttthseHk5kZCSrV6/mf//7H926daN9+/b88kv6GyizNn78eB577DF+/PFHevbsyRNPPMHRo0cB8xJDp06dqFu3Lvv372fKlCmMGjUqy/bi4uKIjY21+YhI7im/3X75DZTjRDLjfKsNXL58mblz5zJ//nx69eoFQPXq1WnevDkA48aNs8YGBgYyfPhwVq9ezciRI63z4+PjWbFiBeXKlctzPzp06EC/fv0AmDBhAosWLaJRo0Z069YNMI+sQ0JCOH/+PBUqVKBSpUoMHz7cuv7AgQPZtGkTa9eupXHjxly+fJnly5ezatUqWrduDcDSpUvx90+9Jh4dHc3SpUuJjo62zh8+fDgbN25k6dKlvPrqqznuf7du3XjuuecAmDJlClu2bGHevHksXLiQVatWYbFYWLx4Ma6urtSpU4fff/+d559/PtP2pk+fzuTJk3O8/YyVBZyA8+nmnwcqZLJOhWziK6SZVzFdTHCamPQ3tN7AfLoos+0WoQqecP6q7bzzV8DbBdxKglMJcLJkEHPVXDeljfhE856VtEe0aWMKlL7rjCi/3Z75DZTjClWxz3GO9T3f8hm7o0ePEhcXZ00O6a1Zs4ZmzZpRoUIFPD09GTduHNHR0TYxVapUuaWkB3DPPfdY/51yNF23bt2b5l24YP6QExMTmTJlCnXr1qV06dJ4enqyadMma99OnjxJQkICjRs3trbh4+NDUFCQdfqnn34iMTGRu+66C09PT+tn+/btnDhxIlf9DwkJuWk65Yg2KiqKe+65x+YyTtp+ZWTMmDHExMRYP2fO5OUehjuABsBXaeYlJU+HZLiGOf+rdPO2pImvivkLnTYmFvN+g5SYEOBvzHsiUnydvO0muRlA4QipDF+dsp235aQ5H8zLDw384auTqcuTDHM6JaZBRShZwjYm6k+IjoGQgILtv9lJ9F3fTPnt9sxvoBxXqIp9jnOs7/mWz9i5ublluiwyMpKePXsyefJk2rVrh4+PD6tXr2bWrFk2cR4eHpm0kHMlS5a0/ttisWQ6LykpCYCZM2cyd+5c5syZQ926dfHw8GDw4ME5urk5xZUrV3BycuKHH37AycnJZpmnZ2GcZcmci4sLLi4u+dDSUKAX5s2fjYE5mE8K9UleHgpUAqYnTw8CWgKzgI7AamAf8E7ycgswGJgK1MTcOcZjPh3UNTmmNtAeeB7z6aMEIBzzKaNcPi2WF1fi4fj/pU6fugQHz0FpN/NG3zFb4ffLsOIRc/kLDWH+Xhi5BZ6tD1+fgrWHYcNTqW0MbQq91kNDf2hcCebsgqsJ0CfYXO7jCmH1zdcFlHYzj4QHfmkmxdw+LZZnDvhdZ0P57fbMb6Acd0scMsc5zvd8y4VdzZo1cXNz46uvvrKeak+xc+dOqlSpwtixY63zTp8+faubzBc7duygS5cuPP20+ah2UlISP//8M3XqmI9hV6tWjZIlS7J3717uvPNOAGJiYvj5559p0aIFAPXr1ycxMZELFy5w//3331J/du3aRWhoqM10/fr1AQgKCuL9998nLi7Omsj27t17S9vLuR7ARcyXMJ7DPMW8kdQbRqOxPfF7H7AKGAe8jPkLvx74V5qYkZg7VF/Mo5nmyW2mval2JeYO0Dq5/ccw3xdUCPadhQeWp04P3Wz+t1c9WNYV/rhiHmWmqFrKTHBDNsHc3VDZG5Z0hnY1UmN6/AsuXoMJ28wbhYMrwMae5g3IKSLaQ4lN5ks74xLNp84WdizIkabjgN91NpTf7D2/gUP+3jtkjnOc7/mWCztXV1dGjRrFyJEjueOOO2jWrBkXL17k8OHD1KxZk+joaFavXk2jRo3YsGED69aty49+37KaNWvy0UcfsXPnTkqVKsXs2bM5f/68NfF5eXnRq1cvRowYQenSpSlfvjwTJ06kRIkS1qPju+66i549exIaGsqsWbOoX78+Fy9e5KuvvuKee+6hY8eO/P7777Ru3ZoVK1ZkeXnhww8/pGHDhjRv3pyVK1eyZ88e3n33XQCeeuopxo4dS9++fRk9ejTR0dG88cYbQOqResEKT/5kZFsG87olfzJjAV5J/mSmNOZOVQRaBYIxMfPly7pmvM6Bflm3G97Y/GTG1RkWdDQ/RcbBvutsKL85Qn4Dh/u9d9gc5xjfc748FTt+/HiGDRvGhAkTqF27Nj169ODChQt07tyZIUOGEB4eTnBwMDt37mT8+PH5sclbNm7cOO69917atWtHq1atqFChAl27drWJmT17NiEhITz88MO0adOGZs2aUbt2bZt7QZYuXUpoaCjDhg0jKCiIrl272hwFJyQkEBUVxbVr17Lsz+TJk1m9ejX33HMPK1as4IMPPrAmYW9vbz777DMOHjxIcHAwY8eOZcKECQA5en2CiOSd8pvym0hxUiz/8kRRuXr1KpUqVWLWrFmEhYXluZ2KFSsyZcoU66Udi8XCunXrbkq8WVm5ciV9+vQhJiYmy/uAUuT9rex24Fbeyl5c5ekvTxR39vGXJ4pKcc5voBzncBwux+U8v93ypVh7duDAAY4dO0bjxo2JiYnhlVfM061dunTJU3vXrl1jx44dnD9/nrvvvjtX665YsYJq1apRqVIlfvzxR0aNGkX37t1znPRERNJSfhOxT7ddYRcdHW09RZ+RI0eOWC8DFIY33niDqKgo7rjjDho0aMB3331H2bJl89TWO++8w5QpUxg8ePBNj/9n59y5c0yYMIFz585RsWJFunXrZn37vYgUD8pvGVN+E8k/t92l2Bs3bvDrr79mujwwMBBn59uuHr3t6TKFg3G4yxRQHC7FKr8VHOU4B+NwOa4YX4p1dnamRo0a2QeKiBQzym8iUtDy5alYERERESl6KuxERERE7IQKOxERERE7ocJORERExE6osBMRERGxEyrsREREROyECjsRERERO6HCTkRERMROqLATERERsRMq7ERERETshAo7ERERETuhwk5ERETETqiwExEREbETKuxERERE7IQKOxERERE7ocJORERExE6osBMRERGxEyrsREREROyECjsRERERO+Fc1B2QQhYzHbxdiroXhcsyqah7ICKFRTlOHJzO2ImIiIjYCRV2IiIiInZChZ2IiIiInVBhJyIiImInVNiJiIiI2AkVdiIiIiJ2QoWdiIiIiJ1QYSciIiJiJ1TYiYiIiNgJFXYiIiIidkKFnYiIiIidUGEnIiIiYidU2ImIiIjYCRV2IiIiInZChZ2IiIiInVBhJyIiImInVNiJiIiI2AkVdiIiIiJ2QoWdiIiIiJ1QYSciIiJiJ1TYiYiIiNgJFXYiIiIidkKFnYiIiIidUGEnIiIiYidU2ImIiIjYiVwVdq1atWLw4MGZLg8MDGTOnDm57sSkSZMIDg7OUWzv3r3p2rVrrrdxu8juZ3jb+fY0dPoA/GeBZTKsP5b9Ott+hXvfBpepUONNWHbw5pgFeyBwDrhOhSZLYM/vtsuv34AXN0CZGeD5Kjy2Fs5fyYcB5cYCIBBwBZoAe7KJ/xColRxfF/gi3XIDmABUBNyANsAv6WL+D+gJeAO+QBhQmON2xDGnUo67dcUqxym/4Vj7umOMOV/P2O3du5e+ffvmZ5M3mTt3LsuWLSvQbRSk//73v0yZMqWou5FzV+Ohnh8s6JCz+FOXoOMqeCAQDvaDwU3huU9h0/HUmDWHYOhmmNgS9vcz22/3Ply4mhozZCN89jN82A2294azl+HRtfk5smysAYYCE4H9QD2gHXAhk/idwJOYO+0BoGvy51CamBnAm8BbwG7AI7nN62liegKHgS3A58C3QMHuU6kcccy5oxyXvWKV45TfcJx93XHGnK+FXbly5XB3d890eUJCwi1vw8fHB19f31tup6iULl0aLy+vou5Gzj1UE6b+Gx6pnbP4t/ZBVV+Y1Q5ql4PwxvB4HYjYlRozexc8fy/0qQ91ysFbD4N7SfjPAXN5zHV49wDMbgf/rgoN/GFpF9h5Bnb9lu9DzNhs4HmgD1AHc8d1B/6TSfxcoD0wAqgNTAHuBeYnLzeAOcA4oAtwD7ACOAusT445CmwElmAeTTYH5gGrk+MKmiOOOXeU47JXrHKc8huOs687zphzXdjduHGD8PBwfHx8KFu2LOPHj8cwDODmyxQWi4VFixbRuXNnPDw8mDZtGgCvvfYafn5+eHl5ERYWxvXr1zPaVIbSX6Zo1aoVAwcOZPDgwZQqVQo/Pz8WL17M1atX6dOnD15eXtSoUYMvv/zSuk5iYiJhYWFUrVoVNzc3goKCmDt37k3jfOmll/D19aVMmTKMGjWKXr162Ww7KSmJ6dOnW9upV68eH330UZb9T3+ZIjAwkClTpvDkk0/i4eFBpUqVWLBggc06x44do3nz5ri6ulKnTh22bt2KxWJh/fr1Of65FZrI36BNNdt57aqb8wHiE+GHs7YxJSzmdErMD39AQpJtTK2ycKcPRJ4p2P6bnQR+wDytbu1k8nRkJutEposH88gtJf4UcC5djA/mzp4SE4l5qr5hmpg2ydvenZsB5IEjjjljynGp21aOS0f5LY3itK871phzXdgtX74cZ2dn9uzZw9y5c5k9ezZLlizJNH7SpEk88sgj/PTTTzz77LOsXbuWSZMm8eqrr7Jv3z4qVqzIwoULb2kQy5cvp2zZsuzZs4eBAwfSv39/unXrxn333cf+/ft58MEHeeaZZ7h27RpgJqvKlSvz4YcfcuTIESZMmMDLL7/M2rWpp8Jff/11Vq5cydKlS9mxYwexsbE3JZnp06ezYsUK3nrrLQ4fPsyQIUN4+umn2b59e676P3PmTOrVq8eBAwcYPXo0gwYNYsuWLYCZoLt27Yq7uzu7d+/mnXfeYezYsdm2GRcXR2xsrM2nUJy7An4etvP8PCE2Dv5JgD+vQaKRQYyHuW5KG3c4ga9r5jEF6k8gEfBLN98Pc0fOyLls4s+lmZdVTPl0y52B0llsN7844pgzphyXSjkuHeW3TOJv933dscbsnNsVAgICiIiIwGKxEBQUxE8//URERATPP/98hvFPPfUUffr0sU4/8cQThIWFERYWBsDUqVPZunVrro5o06tXrx7jxo0DYMyYMbz22muULVvW2qcJEyawaNEi/ve//9G0aVNKlizJ5MmTretXrVqVyMhI1q5dS/fu3QGYN28eY8aM4ZFHHgFg/vz5fPFF6o2TcXFxvPrqq2zdupWQkBAAqlWrxvfff8/bb79Ny5Ytc9z/Zs2aMXr0aADuuusuduzYQUREBG3btmXLli2cOHGCbdu2UaFCBQCmTZtG27Zts2xz+vTpNmMUkZxRjjMpx4kUT7k+Y9e0aVMsFot1OiQkhF9++YXExMQM4xs2bGgzffToUZo0aWIzLyVp5NU999xj/beTkxNlypShbt261nl+fmZFfeFC6k2SCxYsoEGDBpQrVw5PT0/eeecdoqOjAYiJieH8+fM0btzYpt0GDRpYp48fP861a9do27Ytnp6e1s+KFSs4ceJErvqffvwhISEcPXoUgKioKAICAqwJD7DpV2bGjBlDTEyM9XPmTGGc4gcqeML5q7bzzl8BbxdwKwll3cHJkkHMVXPdlDbiE+Hv65nHFKiygBNwPt3880CFm8MheX5W8RXSzMsqJv2NvDcwn6rKbLv5xRHHnDHlOJNyXAaU3zKJv933dccac4G/x87DwyP7oFtUsmRJm2mLxWIzLyVJJyUlAbB69WqGDx9OWFgYmzdv5uDBg/Tp04f4+Pgcb/PKFfOU+YYNGzh48KD1c+TIkWzvQSkMLi4ueHt723wKRUhl+OqU7bwtJ835YF6CaOAPX51MXZ5kmNMpMQ0qQskStjFRf0J0DIQEFGz/zU4CDYCv0sxLSp7O7H/QIeniwXwKKiW+KuaOnDYmFvM+i5SYEOBvzHtBUnydvG3bQiH/OeKY84dyXNEokhyn/JZGcdrXHWvMub4Uu3u37Q1/u3btombNmjg5OeVo/dq1a7N7925CQ0Nt2ihMO3bs4L777mPAgAHWeWmPQH18fPDz82Pv3r20aNECMO8D2b9/v/VdVHXq1MHFxYXo6OhcXZLISPrx79q1i9q1zae0goKCOHPmDOfPn7cele/du/eWtpcrV+Lh+P+lTp+6BAfPQWk382bfMVvh98uwwrycwwsNYf5eGLkFnq0PX5+CtYdhw1OpbQxtCr3WQ0N/aFwJ5uyCqwnQJ9hc7uMKYfXNVwaUdjOPhgd+aSbGppULaeBDgV6YN702xnz66SrmE1UAoUAlYHry9CCgJTAL6Ij51NM+4J3k5RZgMDAVqImZFMYD/piP0IP55FV7zCe33gISgHDgieS4guaIY76Zclww4CA5TvkNx9nXHWfMuS7soqOjGTp0KP369WP//v3MmzePWbNm5Xj9QYMG0bt3bxo2bEizZs1YuXIlhw8fplq1atmvnE9q1qzJihUr2LRpE1WrVuW9995j7969VK1a1RozcOBApk+fTo0aNahVqxbz5s3j0qVL1iNjLy8vhg8fzpAhQ0hKSqJ58+bExMSwY8cOvL296dWrFwC1atVi+vTp1vtYMrJjxw5mzJhB165d2bJlCx9++CEbNmwAoG3btlSvXp1evXoxY8YMLl++bL3XJu3logKz7yw8sDx1euhm87+96sGyrvDHFfNIM0XVUmaSG7IJ5u6Gyt6wpDO0q5Ea0+NfcPEaTNhm3iwcXAE29jRvQk4R0R5KbDJf3BmXaD55trBjQY40nR7ARcyXT54DgjEfW0+5UTYa2xPe9wGrMB99fxlzR18P/CtNzEjMRNIX8yiueXKbaW+iXom547dObv8xzPckFQZHHPPNlOMcKMcpv+E4+7rjjDnXhV1oaCj//PMPjRs3xsnJiUGDBuXqhZ09evTgxIkTjBw5kuvXr/PYY4/Rv39/Nm3alNuu5Fm/fv04cOAAPXr0wGKx8OSTTzJgwACb1wWMGjWKc+fOERoaipOTE3379qVdu3Y2R+1TpkyhXLlyTJ8+nZMnT+Lr68u9997Lyy+/bI2JiooiJiaGrAwbNox9+/YxefJkvL29mT17Nu3atQPM+17Wr1/Pc889R6NGjahWrRozZ86kU6dOuLq6ZtluvmgVCMbEzJcv65rxOgf6Zd1ueGPzkxlXZ1jQ0fwUmfDkT0a2ZTCvW/InMxbgleRPZkpjJpOi4ohjtqUc50A5Tvktk2XbMphnD/u6Y4zZYqS8oEmylJSURO3atenevfstvVU9JCSE1q1bM3XqVMB8x9PgwYNz9Sd4duzYQfPmzTl+/DjVq1fP0TqxsbH4+PhAzGjz1L8jsUwq6h5IoYgFfIiJiSm8e0rtiHJcMaYc5wBynt9yfcbOUZw+fZrNmzfTsmVL4uLimD9/PqdOneKpp57KfuUMxMXF8dNPP3H48GFeeumlXK27bt06PD09qVmzJsePH2fQoEE0a9YsxwlPRCQ95TgR+1TgT8XmVtrH6tN/vvvuu0LrR4kSJVi2bBmNGjWiWbNm/PTTT2zdutV6w29uffnll/z73/+mc+fOPP7447la9/Lly7z44ovUqlWL3r1706hRIz755JM89UNEipZy3M2U40Tyz213Kfb48eOZLqtUqRJubm6F2Bv7ocsUYv+Kx6VY5biCoRwn9q0YX4qtUaNG9kEiIsWUcpyIFKTb7lKsiIiIiOSNCjsRERERO6HCTkRERMROqLATERERsRMq7ERERETshAo7ERERETuhwk5ERETETqiwExEREbETKuxERERE7IQKOxERERE7ocJORERExE6osBMRERGxEyrsREREROyECjsRERERO6HCTkRERMROqLATERERsRMq7ERERETshAo7ERERETuhwk5ERETETjgXdQekcBiGYf4jNq5oO1IkYou6A1IozO/Z+rsuDkU5TuxbzvObxVAWdAi//fYbAQEBRd0NkQJ35swZKleuXNTdkEKmHCeOICf5TYWdg0hKSuLs2bN4eXlhsVgKdduxsbEEBARw5swZvL29C3XbRcURxwxFO27DMLh8+TL+/v6UKKG7TByNclzh0phv3/ymS7EOokSJEkV+FsPb29thEkAKRxwzFN24fXx8Cn2bcntQjisaGnPhyWl+02GtiIiIiJ1QYSciIiJiJ1TYSYFzcXFh4sSJuLi4FHVXCo0jjhkcd9zi2Bzx915jvn3p4QkRERERO6EzdiIiIiJ2QoWdiIiIiJ1QYSciIiJiJ1TYiYiIiNgJFXYiIiIidkKFnYiIiIidUGEnIiIiYidU2ImIiIjYif8HPfsaqu6vL5wAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fp_text_audio_scores = softmax(embeddings[ModalityType.AUDIO] @ embeddings[ModalityType.TEXT].T, axis=-1)\n",
    "int_text_audio_scores = softmax(quantized_embeddings[ModalityType.AUDIO] @ quantized_embeddings[ModalityType.TEXT].T, axis=-1)\n",
    "\n",
    "visualize_prob_matrices(fp_text_audio_scores, int_text_audio_scores, text_list, image_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "d11e9584",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnYAAAD/CAYAAAB8dJvAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABVBUlEQVR4nO3dd3gUVRfA4d9m07MpkIQAISTUUASCoYOAItKbCopIQBEQCL1LFxAE6dJViiCgIqgfSlNBpQYBlRaaEHonFVLn+2OSTTZ1E7JZ2Jz3eeYhM3vnzr3Z3cOZuXcmGkVRFIQQQgghxDPPytwNEEIIIYQQ+UMSOyGEEEIICyGJnRBCCCGEhZDETgghhBDCQkhiJ4QQQghhISSxE0IIIYSwEJLYCSGEEEJYCEnshBBCCCEshCR2QgghhBAWQhI7ISzE6tWr0Wg0HDlyxNxNEUKIfKXRaAgODjZ3M54JktiJZ0JK0pLZMmbMGH05Pz8/g9eKFSvGCy+8wJYtWwzqO3z4MP379ycwMBAbGxs0Gk22x7916xZ9+/bF29sbe3t7/Pz86NWrl0n6KoQonDI7OZs8eTIajQYvLy9iYmIy7OPn50fbtm0B6NmzZ5ZxMu3Ss2dP/f5ff/019erVw83NDXd3d5o0acK2bdtM3ldhOtbmboAQufHhhx9SpkwZg23PPfecwXpAQADDhw8H4Pr16yxfvpxXX32VpUuX8v777wPw008/8dlnn1G9enXKli3L2bNnszzmlStXaNiwIQDvv/8+3t7eXL9+ncOHD+dn14QQIku3b99m6dKl+tiWmb59+/Lyyy/r1//77z8mTpxInz59eOGFF/Tby5UrB8CiRYsYNGgQbdq0YebMmTx+/JjVq1fTtm1bNm/ezKuvvmq6DgmTkcROPFNatWpFrVq1si3j7e3N22+/rV8PCgqifPnyzJs3T5/Y9evXj9GjR+Pg4EBwcHC2iV3fvn2xtrYmJCQEd3f3/OmIEZKSkoiLi8Pe3r7AjimEeDoFBAQwe/Zs+vfvj4ODQ6Zl6tevT/369fXrR44cYeLEidSvX98gJqZYtGgRtWvX5scff9SPWrz77rt4e3uzZs0akyZ20dHRODk5maz+wkyGYoXFK168OJUrV+a///7Tb/Py8soyOKZ15swZfv75Z0aOHIm7uzuPHz8mPj7e6GOnDKOcOXOGLl264OLigru7O4MHD+bx48cGZVPmkKxfv56qVatiZ2fH9u3bATh27BitWrXCxcUFnU5Hs2bNOHjwYKbHjImJoW/fvri7u+Pi4kJQUBAPHjwwKHPkyBFatGiBh4cHDg4OlClThnfffdfofgkhCtbEiRO5desWS5cuzbc6IyIiKFasmMFUlJQYk1N8vHTpEhqNhk8++YR58+bh6+uLg4MDTZo04cSJEwZle/bsiU6n48KFC7Ru3RpnZ2e6desGqAne8OHD8fHxwc7ODn9/fz755BMURcn0uOvXr8ff3x97e3sCAwP5/fffDV6PjIxkyJAh+Pn5YWdnR7FixWjevDlHjx7Ny6/omSRX7MQzJTw8nLt37xps8/DwyHaf+Ph4rly5kqerbbt37wbURLBZs2b8+uuvaLVamjdvztKlS/Hz8zOqni5duuDn58eMGTM4ePAgCxcu5MGDB6xdu9ag3K+//srXX39NcHAwHh4e+Pn5cfLkSV544QVcXFwYNWoUNjY2LF++nKZNm7J3717q1q1rUEdwcDBubm5MnjyZ0NBQli5dyuXLl9mzZw8ajYbbt2/zyiuv4OnpyZgxY3Bzc+PSpUt89913uf79CCEKxgsvvMBLL73ErFmz6Nevn1Enpjlp2rQp3377LYsWLaJdu3Y8fvyYRYsWER4ezuDBg42qY+3atURGRjJgwAAeP37MggULeOmll/j333/x8vLSl0tISKBFixY0atSITz75BEdHRxRFoX379vz222/06tWLgIAAduzYwciRI7l27Rrz5s0zONbevXvZtGkTgwYNws7OjiVLltCyZUsOHz6sn5Lz/vvv8+233xIcHEyVKlW4d+8ef/75J6dPn+b5559/4t/ZM0ER4hmwatUqBch0ScvX11d55ZVXlDt37ih37txR/v77b+XNN99UAGXgwIGZ1j1gwIAM9aQYNGiQAiju7u5Ky5YtlU2bNimzZ89WdDqdUq5cOSU6Ojrbdk+aNEkBlPbt2xts79+/vwIof//9t34boFhZWSknT540KNuxY0fF1tZWuXDhgn7b9evXFWdnZ6Vx48YZfkeBgYFKXFycfvusWbMUQPn+++8VRVGULVu2KIASEhKSbduFEAUr5Tuc9ruZEkPu3Lmj7N27VwGUuXPn6l/39fVV2rRpk2l9ISEhCqCsWrUq09dv3bqlNGvWzCCeenh4KPv378+xrf/9958CKA4ODsrVq1f12w8dOqQAytChQ/XbevTooQDKmDFjDOrYunWrAijTpk0z2P76668rGo1GOX/+vH5bSvuOHDmi33b58mXF3t5e6dSpk36bq6urMmDAgBzbb8lkKFY8UxYvXsyuXbsMlvR27tyJp6cnnp6e1KhRg2+++Ybu3bvz8ccf5/p4UVFRgDqcu23bNrp06cKIESNYuXIlFy5c4KuvvjKqngEDBhisDxw4EFBv4kirSZMmVKlSRb+emJjIzp076dixI2XLltVvL1GiBG+99RZ//vknERERBnX06dMHGxsb/Xq/fv2wtrbWH8vNzQ2A//3vf7kaVhZCmFfjxo158cUXmTVrFo8ePXri+hwdHfH396dHjx588803fPHFF5QoUYJXX32V8+fPG1VHx44d8fb21q/XqVOHunXrZohtoMaitH766Se0Wi2DBg0y2D58+HAUReHnn3822F6/fn0CAwP166VLl6ZDhw7s2LGDxMREQI1vhw4d4vr160a13xJJYieeKXXq1OHll182WNKrW7cuu3btYvfu3ezfv5+7d++ydu3aPA1dpOzTpUsXrKxSvy6dO3fG2tqa/fv3G1VPhQoVDNbLlSuHlZUVly5dMtie/o7fO3fuEBMTg7+/f4Y6K1euTFJSEleuXMn2WDqdjhIlSuiP1aRJE1577TWmTJmCh4cHHTp0YNWqVcTGxhrVFyGE+UyePJmbN2+ybNmyJ66rc+fOhIWFsXr1al5//XXeeecd9uzZQ1xcHOPGjTOqjvTxBqBixYoZYpu1tTWlSpUy2Hb58mVKliyJs7OzwfbKlSvrXzfmWDExMdy5cweAWbNmceLECXx8fKhTpw6TJ0/m4sWLRvXFUkhiJyyOh4cHL7/8Ms2aNaN+/fr6K1R5UbJkSQCDuSIAWq0Wd3f3DDclGCur5+blx7wZY4797bffcuDAAYKDg7l27RrvvvsugYGB+iuUQoinU+PGjWnatOkTX7W7ePEi27dvp3379gbbixYtSqNGjdi3b9+TNtWAnZ2dwcmxqXTp0oWLFy+yaNEiSpYsyezZs6latWqGq3+WTBI7IbKRctn/2rVrBtvj4uK4e/cunp6eRtVz7tw5g/Xz58+TlJSU480Xnp6eODo6EhoamuG1M2fOYGVlhY+PT7bHioqK4saNGxmOVa9ePaZPn86RI0dYv349J0+eZOPGjUb1RwhhPilX7ZYvX57nOm7dugWgH8JMKz4+noSEBKPqSR9vAM6ePWvUjWW+vr5cv36dyMhIg+1nzpzRv27MsRwdHQ1icYkSJejfvz9bt27lv//+w93dnenTpxvTHYsgiZ0Q2WjatCnFihVj/fr1Bo8nWb16NYmJiTRv3tyoehYvXmywvmjRIkB9Ll92tFotr7zyCt9//73B0MatW7f46quvaNSoES4uLgb7rFixwmDu3NKlS0lISNAf68GDBxkeJRAQEAAgw7FCPAOaNGlC06ZN+fjjjzM8NslY5cuXx8rKik2bNhnEg6tXr/LHH39Qs2ZNo+rZunWrwYnv4cOHOXToUI6xDaB169YkJiby6aefGmyfN28eGo0mQx0HDhwweGzJlStX+P7773nllVfQarUkJiYSHh5usE+xYsUoWbJkoYpt8rgTUShdvnyZL7/8EkD/53umTZsGqGeJ3bt3B9Thg9mzZ9OjRw8aN25M9+7dCQsLY8GCBbzwwgtGP8Dzv//+o3379rRs2ZIDBw6wbt063nrrLWrUqJHjvtOmTWPXrl00atSI/v37Y21tzfLly4mNjWXWrFkZysfFxdGsWTO6dOlCaGgoS5YsoVGjRvohlzVr1rBkyRI6depEuXLliIyMZOXKlbi4uNC6dWuj+iOEMK9Jkybx4osv5nl/T09P3n33XT777DOaNWvGq6++SmRkJEuWLOHRo0eMHTvWqHrKly9Po0aN6NevH7GxscyfPx93d3dGjRqV477t2rXjxRdfZNy4cVy6dIkaNWqwc+dOvv/+e4YMGaL/CxkpnnvuOVq0aGHwuBOAKVOmAOoz7EqVKsXrr79OjRo10Ol07N69m5CQEObMmZPL39AzzMx35QphlMweA5CZ7G79T+u3337L8vEpTZo0yVB+w4YNSo0aNRQ7OzvFy8tLCQ4OViIiInI8TsqjCk6dOqW8/vrrirOzs1KkSBElODhYefTokUFZIMvb9I8ePaq0aNFC0el0iqOjo/Liiy9meCRByu9o7969Sp8+fZQiRYooOp1O6datm3Lv3j2Durp27aqULl1asbOzU4oVK6a0bdvW4DECQoiCl9PjTtJr0qSJAuT5cSfx8fHKokWLlICAAEWn0yk6nU558cUXlV9//TXHtqY87mT27NnKnDlzFB8fH8XOzk554YUXDB7jpCjq406cnJwyrScyMlIZOnSoUrJkScXGxkapUKGCMnv2bCUpKcmgXEp8XLdunVKhQgXFzs5OqVmzpvLbb7/py8TGxiojR45UatSooTg7OytOTk5KjRo1lCVLluTYH0uiUZQsHu8shHhikydPZsqUKdy5cyfHBykLIcSz4tKlS5QpU4bZs2czYsQIczdHpCFz7IQQQgghLIQkdkIIIYQQFkISOyGEEEIICyFz7IQQQgghLIRcsRNCCCGEsBCS2AkhhBBCWAhJ7IQQQgghLIQkdkLkAz8/P+bPn59tGY1Gw9atW3Ndd8+ePenYsWOe2mUOxvwuhBDPDolvqZ6F+CaJnTDagQMH0Gq1tGnTxtxNMTlT9PXGjRtG/f1EcypM77EQ6RWWz7/EN8t+fyWxE0b7/PPPGThwIL///jvXr18v8OPHxcUV2LFM0dfixYtjZ2eX5evx8fH5cpwnYe73WAhzMvfnv6BinMQ3y45vktgJo0RFRbFp0yb69etHmzZtWL16dY77xMbGMnr0aHx8fLCzs6N8+fJ8/vnnACQmJtKrVy/KlCmDg4MD/v7+LFiwwGD/lEv006dPp2TJkvj7+5uiaxnkpa+g/gHqrl274uTkhLe3N4sXLzZ4Pe1QxaVLl9BoNGzatIkmTZpgb2/P+vXrSUxMZNiwYbi5uen/kHZBPZEoL/2+ffs27dq1w8HBgTJlyrB+/foMZcLCwujQoQM6nQ4XFxe6dOnCrVu3DMpMmzaNYsWK4ezszHvvvceYMWMICAjIp54JkbPCEuMkvhWC+GbOP1Qrnh2ff/65UqtWLUVRFOXHH39UypUrl+GPNKfXpUsXxcfHR/nuu++UCxcuKLt371Y2btyoKIqixMXFKRMnTlRCQkKUixcvKuvWrVMcHR2VTZs26ffv0aOHotPplO7duysnTpxQTpw4YboOppGXvvr6+irOzs7KjBkzlNDQUGXhwoWKVqtVdu7cqS8DKFu2bFEUJfUPaPv5+SmbN29WLl68qFy/fl35+OOPlSJFiiibN29WTp06pfTq1UtxdnZWOnToYKru6uWl361atVJq1KihHDhwQDly5IjSoEEDxcHBQZk3b56iKIqSmJioBAQEKI0aNVKOHDmiHDx4UAkMDFSaNGmir2PdunWKvb298sUXXyihoaHKlClTFBcXF6VGjRom6qkQGRWWGCfxzfLjmyR2wigNGjRQ5s+fryiKosTHxyseHh7Kb7/9lmX50NBQBVB27dpl9DEGDBigvPbaa/r1Hj16KF5eXkpsbGye250Xue2roqiBr2XLlgbb3njjDaVVq1b69cwCX8pxUpQoUUKZNWuWfj0+Pl4pVapUgQS+vL7Hhw8f1m87ffq0AugD386dOxWtVquEhYXpy5w8edJgv7p16yoDBgwwqLthw4aS2IkCVVhinMQ3y49vMhQrchQaGsrhw4fp2rUrANbW1rzxxhv6IYfMHD9+HK1WS5MmTbIss3jxYgIDA/H09ESn07FixQrCwsIMylSrVg1bW9v86YgR8tLXFPXr18+wfvr06Wz3qVWrlv7n8PBwbty4Qd26dfXbrK2tDcqYSl76ffr0aaytrQkMDNRvq1SpEm5ubgZlfHx88PHx0W+rUqUKbm5u+t9NaGgoderUMag7/boQplRYYpzEt8IR36xNWruwCJ9//jkJCQmULFlSv01RFOzs7Pj0009xdXXNsI+Dg0O2dW7cuJERI0YwZ84c6tevj7OzM7Nnz+bQoUMG5ZycnPKnE0bKS1+fREH3LysF3W8hniaFJcZJfCsc8U2u2IlsJSQksHbtWubMmcPx48f1y99//03JkiXZsGFDpvtVq1aNpKQk9u7dm+nr+/bto0GDBvTv35+aNWtSvnx5Lly4YMqu5CivfU1x8ODBDOuVK1c2+viurq6UKFHCIPAnJCTw119/5a4juZTXfleqVClD+0JDQ3n48KF+vXLlyly5coUrV67ot506dYqHDx9SpUoVAPz9/QkJCTGoO/26EKZSWGKcxLdCFN9MNsgrLMKWLVsUW1tb5eHDhxleGzVqlH4yamZ69uyp+Pj4KFu2bFEuXryo/Pbbb/qJwwsWLFBcXFyU7du3K6Ghocr48eMzTCjt0aNHgcy9SPEkffX19VVcXFyUjz/+WAkNDVU+/fRTRavVKtu3b9eXIZM5KMeOHTOoZ+bMmUrRokWVLVu2KKdPn1Z69+5t8snFT9Lvli1bKjVr1lQOHjyoHDlyRGnUqJHB5OKkpCQlICBAeeGFF5S//vpLOXToUKaTix0cHJTVq1crZ8+eVaZOnaq4uLgoAQEB+d1VITIoLDFO4lvhiW+S2IlstW3bVmndunWmrx06dEgBlL///jvT1x89eqQMHTpUKVGihGJra6uUL19e+eKLLxRFUZTHjx8rPXv2VFxdXRU3NzelX79+ypgxY8ya2D1JX319fZUpU6YonTt3VhwdHZXixYsrCxYsMChjTOCLj49XBg8erLi4uChubm7KsGHDlKCgIJP+Hp6k3zdu3FDatGmj2NnZKaVLl1bWrl2r+Pr66gOfoijK5cuXlfbt2ytOTk6Ks7Oz0rlzZ+XmzZsG9Xz44YeKh4eHotPplHfffVcZNGiQUq9evXzroxBZKSwxTuJbRpYa3zSKUkAPkRFCCCM1b96c4sWL8+WXX5q7KUIIka9MHd/k5gkhhFnFxMSwbNkyWrRogVarZcOGDezevZtdu3aZu2lCCPFEzBHf5IqdEMKsHj16RLt27Th27BiPHz/G39+f8ePH8+qrr5q7aUII8UTMEd8ksRNCCCGEsBDyuBMhhBBCCAshiZ0QQgghhIWQxE4IIYQQwkJIYieEEEIIYSEksRMmFxsby+TJk4mNjTV3UwpMYewzFN5+i8KtMH7upc9PL7krVphcREQErq6uhIeH4+LiYu7mFIjC2GcovP0WhVth/NxLn5/ePssVOyGEEEIICyGJnRBCCCGEhZA/KVZIJCUlcf36dZydndFoNAV67IiICIN/C4PC2Gcwb78VRSEyMpKSJUtiZSXnrIWNxLiCJX0uWLmJbzLHrpC4evUqPj4+5m6GECZ35coVSpUqZe5miAImMU4UBsbEN7liV0g4OzurP2z/CJzszduYgtbxTXO3oODZJpq7BQUvKRJuVUn9rItCRf++/zSj8MW4Lp3N3YKCpxTsVVmzS4qEu5WMim+S2BUS+qEJJ3vQOZi3MQXN6um9e8lkrAphYpesoIfhxNNBYlwhU9gSu2TGxDeZiCKEEEIIYSEksRNCCCGEsBCS2AkhhBBCWAhJ7IQQQgghLIQkdkIIIYQQFkISOyGEEEIICyGJnRBCCCGEhZDETgghhBDCQkhiJ4QQQghhISSxE0IIIYSwEJLYCSGEEEJYCEnshBBCCCEshCR2QgghhBAWQhI7IYQQQggLIYmdEEIIIYSFkMROCCGEEMJCSGInhBBCCGEhJLETQgghhLAQktgJIYQQQlgISeyEEEIIISyEJHZCCCGEEBZCEjshhBBCCAuRb4ld06ZNGTJkSH5Vl63JkycTEBBQIMcyhZ49e9KxY0dzN8N4m/ZA63FQdyB0/xhOXMq+/PpfoOMkqDcIWn4An3wDsfHG13n9HtTsl/my66/87Vt2Hi2He1Xgjjs8aArxR7IvH/sd3K+plr9fB2J3GL6uREHkMLhXEe54wP1AePRZ5nUpCjzsBHd0EPtjvnTHKFEr4WY1uFYMbr8Ecdn8vuNPw723k8u7QtSSvNWZcBHudYMbZeF6KbjfAxJv51+f8oHEN+M9c/EN4Os90PYDqB8MQTPhxH/Zl//qF3h1EjQYCK3HwpyvM8a4nOqMjYeZG+Cl4dBoMIxcDvci8rNX2YteAbefgxuecPdFiMsmvsWfhgdvJ5d3gejFeaszZhXcaw03vdV6kh7mW3eMEr0CbleFmx5wL4c+AzzeAneeV8vfrZsxpifehod94XYFuFkM7neChPOGZRIuwoOucMsPbpWEh0Emj2/P5BW7ESNG8Msvv5i7GXm2YMECVq9ebe5mGGfHEZizGfq2ga8+gIqloP9CuJ9FAPr5MCzcqpb/bhJMeht2/AWLvje+Tq8isGum4fJ+W3C0g4ZVTd5lAB5/C1FjwWksFPkTrJ+D8I6QlMUXMv4gRLwD9j2gyD6wawsRb0LCydQyUWMgbjc4fwZF/wKHARA1HGK3Zazv0WLQaEzStSzFbIbwD8B5NBT7HWyeg7udIPFO5uWVGLD2A5dJYOWVtzqTotV1AI8fwXMHKPFw7w1QkvK9i88CiW8FbOcRmPst9GkL65PjUfCi7GPcoi3Quw18OwkmdIedf8Hirbmrc8438Ps/MLM3rBwGdx7CyGWm7GmqR5sh4gPQjQGPP8C6Gtx/NfvvutYPnCdn/V03pk4lBuxeBt3w/O5Rzh5thsixye1LjukPsolvcQfh4TvgGKSWt2urJmjxp9TXFQUevgmJl6DIRrWM1gfut1fjGqj/PugIaKDoNii6C5Q4eNjFpPHtmUzsdDod7u7u5m5Gnrm6uuLm5mbuZhhn3S/wakPo0ADKlYBxXcHeFrYeyLz83xchoBy0qgMl3aF+FWhZC05eMr5OrRV4uBouvx2H5oHgaG/qHqsefQr2PcG+O1hXBt1C0DjA4y+zKL8EbJuD4xCwrgROE8E6QL3qlyL+ENi/BbaNQesLDu+qwS8h3Vljwj/waCE4LzVR57IQtRiceoDT22BTCdzmg8YRYrLos20guE4Dx9dBY5e3OuMOQmIYFFkKNlXVpchSiD8GsXtN0cunnsS3ArZuN3RqCO0bQNmS8MFbYG8D3+/PvPw/F6BGSozzUGNci9qGow451Rn5CL7fB8NehzqVoLIvTOqhxs9/L5q8y0R/Co49wDH5e+k6X41vj7L5rrtMA4dsvuvG1Ok0AHTDwKZ2fvcoZzGfgmNPcOyuxmiXBcntW5tF+aVqEuo0RC3vPAFsakBMckxPPA/xIeAyH2wCwbqi+jOP4PE3apn4g5B4GVyXpcY31+UQfxTiTBff8pTYRUdHExQUhE6no0SJEsyZM8fg9QcPHhAUFESRIkVwdHSkVatWnDt3zqDMypUr8fHxwdHRkU6dOjF37lyjg0H6oYqUS/8fffQRXl5euLm58eGHH5KQkMDIkSMpWrQopUqVYtWqVQb1jB49mooVK+Lo6EjZsmWZMGEC8fGGl9OnTZtGsWLFcHZ25r333mPMmDEZhkk+++wzKleujL29PZUqVWLJkiyGpNK1N0XTpk0JDg4mODgYV1dXPDw8mDBhAoqi6MvcuHGDNm3a4ODgQJkyZfjqq6/w8/Nj/vz5Rv3O8iQ+AU6HQd1KqdusrNT1f7IIPjXKwqmw1CB39Q7sOwmNqua9zlOXIfQqdGzwxF0yihIHCcfA9sXUbRorsHkR4g9nvk/8YfX1tGybGZa3qQtxP0HidfVsL26vGhxsmqU5dox65U83N+szY1NQ4iD+ONg1Td2msVLX40JMV6cSB2gM/7PQ2ANWatJnBhLfAgzKWGx8AzUenQmDOpVTt1lZqetZJVjVy6kxLGVo9eod2HcCGj1nfJ2nL0NCItRNU6ZMcSheNOs4mF/038t08c2uKcRlEd/MUWd+UuLUk0XbpqnbNFbqelYxPe6w4f8BALYvp5ZX4pLrSRu7rAA7iEu+SKHEknV8y+LiSD7IU2I3cuRI9u7dy/fff8/OnTvZs2cPR48e1b/es2dPjhw5wg8//MCBAwdQFIXWrVvrg8q+fft4//33GTx4MMePH6d58+ZMnz79iTry66+/cv36dX7//Xfmzp3LpEmTaNu2LUWKFOHQoUO8//779O3bl6tXr+r3cXZ2ZvXq1Zw6dYoFCxawcuVK5s2bp399/fr1TJ8+nY8//pi//vqL0qVLs3Sp4VWU9evXM3HiRKZPn87p06f56KOPmDBhAmvWrMlV+9esWYO1tTWHDx9mwYIFzJ07l88+S51/FRQUxPXr19mzZw+bN29mxYoV3L5t4nlID6IgMQmKuhhud3fJei5IqzrQry288wnUHgDtJkJgBejVKu91bt2vBr2Ack/WH2Ml3QMSwaqY4XarYpB0K4t9boGVZ/bldXNAWwnuV4S7RSC8k5rA2TZKLRM1GmzqqZf9C1JWfdZ6QmIWfc6POm1rg8YJwidBUow6dBE+Xt0v8WbejvuEJL5hUMZi4xvAw+R45J4+HjnD3Wxi3PvtoNcnUKc/dJgAgRXh3VbG13kvAmyswdkxYxlTz7PTfy9ziFfmrjM/ZRmLimU9vSbpVhblk/tjXRGsfCBqMiQ9UBO9qLmQdC21TEp8i5yonrQnRUPkOLUtSaaLb9a53SEqKorPP/+cdevW0ayZeqVhzZo1lCpVCoBz587xww8/sG/fPho0UK+wrF+/Hh8fH7Zu3Urnzp1ZtGgRrVq1YsSIEQBUrFiR/fv387///S/PHSlatCgLFy7EysoKf39/Zs2aRUxMDB988AEAY8eOZebMmfz555+8+eabAIwfP16/v5+fHyNGjGDjxo2MGjUKgEWLFtGrVy/eeecdACZOnMjOnTuJiorS7zdp0iTmzJnDq6++CkCZMmU4deoUy5cvp0ePHka338fHh3nz5qHRaPD39+fff/9l3rx59O7dmzNnzrB7925CQkKoVasWoJ5FV6hQIcv6YmNjiY2N1a9HRBTQpNwjZ+GLHTD2TahWBq7cgdlfw4qfoE/r3Nf3OA5+DoHeedj3afNoGSSEgMvXoC0N8X9C1DCwKqGeGcZug/jf1Tl6hYXWA4quhofDIHoZYKUO99jUSD77LVgS356N+AbmjHGhsGo7jOkKz5WBK7fhk69h5TZ13p0oHDQ2UGQ9hA+A26UBrRrHbV8Bkq9GW3mC21qIGKoO7WIF9p3VaTomnAmX65ovXLhAXFwcdevW1W8rWrQo/v7+AJw+fRpra2uD193d3fH39+f06dMAhIaGUqdOHYN606/nVtWqVbGySu2Ol5cX1apV069rtVrc3d0NzgI3bdpEw4YNKV68ODqdjvHjxxMWFqZ/Pad2RkdHc+HCBXr16oVOp9Mv06ZN48KFC7lqf7169dCkmSxfv359zp07R2JiIqGhoVhbW/P888/rXy9fvjxFihTJsr4ZM2bg6uqqX3x8fHLVHgCK6NT5buknEd+LyHg2mmLJD9CmDrzaCCp4w0sBENxBDYRJSbmvc/cxNblrWzfja6Zi5Q5oM57JJd3OenjUyguS7mRdXnkE0ZPBaQbYtVYn7jq8D3avQcwCtUz8Xki8CHe94Y6rugBEdIOHLfOrd1m0P4s+J94BbR6HhI2t074ZFP8bil+AEheh6ApIvKFO1i5gEt+ejfgG+RTj3JLjUfqrZPciwSOLGLf0R2hdFzqlxLiaENwxNcYZU6e7izpkGxmTsUxWsTW/6L+X2cSrp6HO/JRlLLqd8aqcfh+vLMqn6Y9NTfDYD8WuQrFzUHQLKPfVm8pS2DUDz3+g2EUodgncVkLSdcMy+eyZvHkiMzY2NgbrGo0m021JSeqdKAcOHKBbt260bt2a//3vfxw7doxx48YRFxdn9DFTzmxXrlzJ8ePH9cuJEyc4eNA884NSjB07lvDwcP1y5cqV3FdiYw2VS8Oh0NRtSUlwOBSql818n8dx6nyStFLWlTzUuXUfNKkORZ1z3/680tiCdU2I25O6TUmC+D1gk8V/0DZ11NfTivsttbwSD8ST8StnBSTfHeU4HIochCL7UxcAp5ngbOK75TS2YBNgeMOCkqSu2+ZxonNu69S6g5Wb+nrSHbC3gKu0+UTiW0b5FuMqlYaQM6nbkpLU9WrZxbh0d6ynj3E51VnZF6y1cDhNmUs34eb9rGNrftF/L/ekbtN/L/N4AmKKOvOTxlZNwuLSxaK4vVnHdNs6hv8HAMT9mnl5K1f16lzCefXGCLtMrtxaeRjGNzvTxbdcD8WWK1cOGxsbDh06ROnSpQF1MvHZs2dp0qQJlStXJiEhgUOHDumHKu7du0doaChVqlQBwN/fn5AQwwnZ6ddNbf/+/fj6+jJu3Dj9tsuXLxuUSWlnUFCQflvadnp5eVGyZEkuXrxIt27dnqg9hw4dMlg/ePAgFSpUQKvV4u/vT0JCAseOHSMwMBCA8+fP8+DBgyzrs7Ozw84ui7uXcuPtZjBxDVQpDc/5wVe/wqNY6FBffX38aijmBoM6quuNq6t3vfqXSh2KXfqjul1rZVydKcJuw9HzsGjAk/cjtxyCIbIv2DwP1oHq40eUGLB/W309ojdYlQTdlOTy/dWrajELwbYFxH4LCUfBeaH6upUL2DSC6HHq5NmUodjHG0A3I7mMV+Znt1qfgrl6pRsAD/qpAdA2UH0unRKt3uUGcL8vaEuA62R1XYmD+DOpPydeh7h/wMoJrMsZVydA9Dqw8VfPquNC4OFodT+b7IfiTEHi27MR3yA/Y9zLMGm1mmzp41GcekcrwMRV4OkGA5Mfy9O4mvqsTn+f1KHYpT+ki3E51OnsAB0aqo9EcXECnT3M2qQmdVkllPnJKRgevq9+L21qQcwSNb45JH8vH/ZR45vLZHVdiYOEtN/1GxD/jzp/LOW7nlOdoM6tTbqljkwAJJwCjQ60pcCqqGn77BgM4X2T2xcI0Snt657aZ20JcE6O6Y794H4riF4Idi3Ux6XEHwOXRal1Pt4CGg+1/QknIWK0Oj/aLs0NcTFfgrW/mtjFH4aIUeA4QJ2jZyK5Tux0Oh29evVi5MiRuLu7U6xYMcaNG6cfJqhQoQIdOnSgd+/eLF++HGdnZ8aMGYO3tzcdOnQAYODAgTRu3Ji5c+fSrl07fv31V37++WeDS/WmVqFCBcLCwti4cSO1a9dm27ZtbNmyxaDMwIED6d27N7Vq1aJBgwZs2rSJf/75h7JlU794U6ZMYdCgQbi6utKyZUtiY2M5cuQIDx48YNiwYQA0a9aMTp06ERwcnGV7wsLCGDZsGH379uXo0aMsWrRIfzdepUqVePnll+nTpw9Lly7FxsaG4cOH4+DgYPrfWYta6g0PS/+nDi34l4LFA1OHC27eNzx7fa8VaIAlP8Lth+rQa+Nq6nCssXWm+H4/eLlB/coUOPvXQbkL0dPUQGRdHVy3pCZeSVcwuPpmUw9cvoDoqeqQq7YcuGwE6zTP3XNZA1GTILKXOtlW6wNOk8D+vYLsWdYcX1MnGUd+pAZgm2rg8Z06YRgg8arhvLfEG3DnhdT1qEXqYtsIPLcZVydAwjmImJL8OykNziPUxM4MJL4VsvgG8EoteBAJy35U41HFUrAoXYxL245erdX1JT+oz55z06lJ3YAOxtcJMLyzGjtHLYe4BPWxKWO6mr6/AA6vQdJdiErzvSy62fC7Trrv+t00N3lFL1QX20bg/pNxdQLEfA5RM1PX7yVPMXFdCo5PdvKQo5T2RU5XY7pNdSiSNr5dQf3PK5ltPXD7AiI/hMgpagJbZAPYVEktk3gToscmDzkXB4euoBtteNzEc6k3WGhLg26kmmSaUK4TO4DZs2cTFRVFu3btcHZ2Zvjw4YSHh+tfX7VqFYMHD6Zt27bExcXRuHFjfvrpJ/3QQcOGDVm2bBlTpkxh/PjxtGjRgqFDh/Lpp5/mT6+M0L59e4YOHUpwcDCxsbG0adOGCRMmMHnyZH2Zbt26cfHiRUaMGMHjx4/p0qULPXv25PDh1Nuj33vvPRwdHZk9ezYjR47EycmJatWqGTyl/sKFC9y9ezfb9gQFBfHo0SPq1KmDVqtl8ODB9OnTR//62rVr6dWrF40bN6Z48eLMmDGDkydPYm9fAM91e7OpumTms2GG69Za6NtWXfJaZ4qBHdXFXBzeV5fMuG3PuM3uVXXJipUXuORySNUzKucy+UnXR10ybUu6Bylb+4J3eOZlja0TwHWKujwlJL4VsvgG8MaL6pKZFekepmutVR883CeHGJddnQB2NmoiV1DJXHpOfdUlMynJWgprXyhhxM0p2dUJ4PyBuphLtn3+OeM2+07qkmV9/dQlO84fqksB0ihpHyZkRil3R/3xxx/mbkq2mjdvTvHixfnyyywe5GiErl27otVqWbduHaA+5ykgICBXz2y6evUqPj4+7N69W3/3XnYiIiJwdXWFP+aCziGvTX82vdLd3C0oeLaJ5m5BwUuKgBs+hIeH4+Ji4gnouSTxzbTxDdLEuL3zCl+Ma/uWuVtQ8JQC/ss85pYUAbe9jYpvebpilx8++eQTmjdvjpOTEz///DNr1qzJ8cGXBS0mJoZly5bRokULtFotGzZsYPfu3ezatStP9SUkJHD27FkOHDhA377ZnNVk4tdffyUqKopq1apx48YNRo0ahZ+fH40bN85TW4QQpiPxTeKbEOZitrtiDx8+TPPmzalWrRrLli1j4cKFvPeeOteoatWqBrfXp13Wr19fYG3UaDT89NNPNG7cmMDAQH788Uc2b97Myy+/nKf6Tpw4Qa1atahatSrvv5/FEF8W4uPj+eCDD6hatSqdOnXC09OTPXv2ZLgzTghhfhLfJL4JYS5PzVBsWpcvX87wp29SeHl54excgI++sBAyFFvIyFCsuVuTJYlvpiFDsYWMDMVmyWxDsdnx9fU1dxOEEMIkJL4JIUzJYh5QLIQQQghR2EliJ4QQQghhISSxE0IIIYSwEJLYCSGEEEJYCEnshBBCCCEshCR2QgghhBAWQhI7IYQQQggLIYmdEEIIIYSFkMROCCGEEMJCSGInhBBCCGEhJLETQgghhLAQktgJIYQQQlgISeyEEEIIISyEJHZCCCGEEBZCEjshhBBCCAshiZ0QQgghhIWQxE4IIYQQwkJIYieEEEIIYSEksRNCCCGEsBCS2AkhhBBCWAhrczdAFLDOXcDKxdytKFi2ieZuQcHbssncLSh4UY/gJXM3Qphdn/agdTZ3KwqWxtwNMIM1W8zdgoIV/Qg6GVdUrtgJIYQQQlgISeyEEEIIISyEJHZCCCGEEBZCEjshhBBCCAshiZ0QQgghhIWQxE4IIYQQwkJIYieEEEIIYSEksRNCCCGEsBCS2AkhhBBCWAhJ7IQQQgghLIQkdkIIIYQQFkISOyGEEEIICyGJnRBCCCGEhZDETgghhBDCQkhiJ4QQQghhISSxE0IIIYSwEJLYCSGEEEJYCEnshBBCCCEshCR2QgghhBAWQhI7IYQQQggLIYmdEEIIIYSFkMROCCGEEMJCSGInhBBCCGEhJLETQgghhLAQ1uZugHgGRK+AqIWQeAtsngPX2WBbK/Oy8achcjrEH4fEMHCZAboBea9TUeD+axC7G4p8BQ5t87VrWYpaadg+t9lgG5h52fjTEDEd4v9W++w6A3T9c19n4i0InwCxv4ESBdblwXkEOHQwTR8z881vsG4n3AuHCqVgRFeoWibzsu9/AkfPZtze8DmYN0j9OeYxLP4O9h6H8Ggo6QFdXoLXmqSWvxsOi76FQ6fV8r5e8E5reCmL37cQ+e3+53BvCSTcBruqUOIjcHg+87KPz8Cdj+HxPxB/BbymgnvfjOXib8DtDyHqV0h6BLZloOQCcAhQX0+4DbemQvQeSIwAx3pQfAbYlTVVLw2ZK67HHYKIqRB/BNCCTTVw3wIah/zuYUY/7IZvfoL74VDWBwZ0h0rlMi874iP450zG7XVqwLThkJAAqzfD4b/hxm1wcoTnq0KvLuBexHCfQ8dh3Vb47wrY2kC1SjBlSD53LtUzecWuadOmDBkyxNzNyDM/Pz/mz59v7mYY59FmCP8AnMeA5x/ql/Deq5B4J/PySgxo/cBlMlh5PXmd0YsBTT51xkgxKe0bDcV+VwPU3U7Z99naD1wmZd1nY+p80BcSzoH7Rii2H+zbw/2eEPd3fvcwc7tCYP438F5bWDseKvjAoAVwPyLz8h/3g59mpy4bJoPWCpqlCeTzv4EDJ2FKL9g0Bd5sBp9sgN+Pp5aZ8gVcvgVzBsCGSdD0efhgBYSGmbK3Ty2JbwUsfCvcmgSeI6DsbrCvCpffgISsvu+PwNYXio0H62KZl0l8CJfaAjZQegOU+wO8JoPWNbkOBa70gPjL4LMWyv4CNqUg7HVIis7/PqZnrrgedwjuvQZ2L4HHb+C5B5z6UCCpyJ6DsPwreLsjLPkQypaGD2bDgyzi28RBsHFh6rLiI7CygsZ11Ndj4+DcJejWAZZMhUmD4MoNmDjPsJ4/QmDWcmjRGJZNg3kT4KX6puzps3nF7rvvvsPGxsbczcizkJAQnJyczN0M40R9Co49wPFtdd11PjzeATFfgvOwjOVtA1OvQkVMfrI64/9Ry3ruhVsV8qc/xohaDE49wCm5fW7z4fFOI/s8Je91xh0Gt7mpdbmMVPeLPw62NfKpc9n4ahd0bATtGqrrY7rBvn/hx33Qo1XG8q7pPsO7QsDeFpqludL2zwVoUx8C/dX1To1hy+9w8hI0DkgucxFGv5V6ZbBXG9iwG05fBv/S+dnDZ4LEtwJ2bxm4vQ1uXdX1ErMhahc83AAegzKWd6ipLgC3p2Ve591FYF0SvBembrP1Tf057iI8+gvK/g72lVKPe/Y5CN8CRd5+8n5lx1xxPXwsOPU1PIZ1AcX2zduhVVM1wQIY3FO92rZjL7zZLmN5F53h+p6Danx7ITmxc3KEj0cblgkOgoGT4fZdKOYBiYmwdB289ya0SjNK4eudT53K3FN3xS4uLi7HMkWLFsXZ2bkAWmManp6eODo6mrsZOVPi1KTC7sXUbRorsGsK8YdNW2dSDDzoBa5zQJvFGaIp6NvXNGP74kJMW6dtHYj5DpLug5IEMd8CsWDXKG/HzY34BDgTBrUrp26zslLX/71oXB0//AnNa4ODXeq26uXg97/h9gP1KsWRMxB2C+pWSVOmLOw6og7VJiXBzsMQF5+aDFoQiW9PGSUOHv8NTo1Tt2ms1PWYI3mvN3KHOuR6pReEVoGLL8GDL9McN1b91yrNd0VjBRpbiDmU9+Maw1xxPfGOOvxq5Ql3Xoab5eBuK4g9kLdj5kZ8gnp1rWbV1G1WVlCzCpw+b1wd23+HJvUM41t60TGg0UDKic25S3D3AVhpoN94eHMgfPAJ/Hc1rz0xSr4kdklJScyaNYvy5ctjZ2dH6dKlmT59OgCjR4+mYsWKODo6UrZsWSZMmEB8fLx+38mTJxMQEMBnn31GmTJlsLe3z/F46Ycq/Pz8mDZtGkFBQeh0Onx9ffnhhx+4c+cOHTp0QKfTUb16dY4cSf2i3rt3j65du+Lt7Y2joyPVqlVjw4YNBseJjIykW7duODk5UaJECebNm5fh2LGxsYwYMQJvb2+cnJyoW7cue/bsybb96YcqNBoNS5cupVWrVjg4OFC2bFm+/fZbg332799PQEAA9vb21KpVi61bt6LRaDh+/HiOv688S7oHJILW03C7VTF1DoUp64wYC7Z1waFN3o6TVynts0o3xKL1fPI+51Rn0dVAPNwoA9c94eFQKLoOrLOYA5KfHkZBYhIUdTHcXtRZnW+Xk5P/wYXr0CFdEjriTShTEtqOhgb9YPBCGPkWPF8xtcxHfSEhEZoPhYb9YcY6mNUPfLIY5ipgEt8sNL4BJNwHEsE6XTyy9lTnwOVV/GV4sBpsy0LpTVCkJ9wcBw83qq/bVVCHXm9PV4dtlTi4uxASrkNCHuOMscwV1xP/U/+NnAFOPcH9O7CpAffaQYKRyVVeRUSqJ41F0sW3Iq7qfLucnLkAl64aXnVLLy4OPvsamtYDp+T5gjeSh6G/3AJvdYAPh4GzI4z8CCKi8tYXI+RLYjd27FhmzpzJhAkTOHXqFF999RVeXupVFmdnZ1avXs2pU6dYsGABK1euZN48wzHo8+fPs3nzZr777rs8f5HnzZtHw4YNOXbsGG3atKF79+4EBQXx9ttvc/ToUcqVK0dQUBCKogDw+PFjAgMD2bZtGydOnKBPnz50796dw4dTz1iGDRvGvn37+OGHH9i1axd//PEHR48eNThucHAwBw4cYOPGjfzzzz907tyZli1bcu7cuVy1f8KECbz22mv8/fffdOvWjTfffJPTp08DEBERQbt27ahWrRpHjx5l6tSpjB49Otv6YmNjiYiIMFieGY9/gti94DLT3C0pWBHTISkc3L+HYnvUycn334H4k+ZuWc5++BPKe2e80eLr3+DERXX+3NrxMPh1mP0VHD6VWmbZ9xAVA58OhTXj4K3m6hy786Y9qzWWxLenL77BUx7jlCSwrwZe48ChGhQJUodXH6xRX9fYQKlVEHsBQivCaV+I3ge6ZjyFA2n5RP1s4vSuOlxrUwNcZ6pDsTHrzNu0nGz/Hcr4ZH2jRUICTFsMKDCoZ+p2JUn9t2t7eKE2VCwDw3urV/V+z+PVUSM88Ry7yMhIFixYwKeffkqPHj0AKFeuHI0aqWfu48eP15f18/NjxIgRbNy4kVGjRum3x8XFsXbtWjw902X7udC6dWv69lXvTJo4cSJLly6ldu3adO7cGVDPrOvXr8+tW7coXrw43t7ejBgxQr//wIED2bFjB19//TV16tQhMjKSNWvW8NVXX9GsWTMAVq1aRcmSJfX7hIWFsWrVKsLCwvTbR4wYwfbt21m1ahUfffSR0e3v3Lkz7733HgBTp05l165dLFq0iCVLlvDVV1+h0WhYuXIl9vb2VKlShWvXrtG7d+8s65sxYwZTpmQx38tYVu6ANuOE2qTbeR8eNabO2L3q2d1NH8MyD96G6Abg8VPejp2b9iWlO1tPvPPkfc6uzoSL6h1lxQ6CTfJwqE01iNuv3k1bZH7ejm0sN51640P6GyXuR4K7a/b7PoqFnSHQN93du4/jYMkW9epbo+rqtgql4OxVWLcL6lSBq7fVO3E3TIZyyd+tij5w/Bx8swfGmniuUQ4kvj2d8Q3yKcZZFwW0GW+USLiT9Y0RxrDxArt0UwlsK0DE/1LXHWpAud/UO2KVOLD2gIst1e2mZK64nnLThXW634u1PyReydtxjeXirA69pr9R4kE4FDUivu05CD1ezfz1lKTu9l2YNSb1ah1AUTf1X9/U7xW2NlDcE+7cy3U3jPXEpwanT58mNjZWHxzS27RpEw0bNqR48eLodDrGjx9PWJjh3W6+vr5PFPQAqlevrv855Wy6WrVqGbbdvq3+55qYmMjUqVOpVq0aRYsWRafTsWPHDn3bLl68SHx8PHXq1NHX4erqir9/6ofy33//JTExkYoVK6LT6fTL3r17uXDhQq7aX79+/QzrKWe0oaGhVK9e3WAYJ227MjN27FjCw8P1y5UrefjiaGzBJgDi9qRuU5LUxMsm++M/UZ26YeB5ADz3pS6g3mLvtiRvx81t+2L3ZmyfbW3T1ak8Sn4h/VdSCyTl7bi5YWMNlUpDSJrb+5OS4MhpqJbD4xd++Uudw9KyruH2hER1sUp3V7NWk3om+zh5zln6MlZW6pw8M5P49nTGN8jHGGdfA6L/SN2mJKnrjlk8+sMYDnUgNt3wYtxFdfg1Pa2LmtTFXoTHx8G5Zd6PawxzxXWtL1iVUO/8TyvhPGhNfJOUjTVU8IPjaUY/kpLg+CmoXD77ff84rMa3Zg0yvpaS1F27CTNHqwlkWhXKgI0NXLlpuM+t5JsrTOSJr9g5OGT97JkDBw7QrVs3pkyZQosWLXB1dWXjxo3MmTPHoFx+3EGV9i4yjUaT5bakJPU/lNmzZ7NgwQLmz59PtWrVcHJyYsiQIUZNbk4RFRWFVqvlr7/+QqvVGrym0+my2Ktg2NnZYWeXzSRPY+mC4cH7YFMTbGpB9BL11veUO58e9AFtSfU2eFDPPBPOpP6ceEO9u1XjlDpXLKc6tV6ZnzlqfdTHipiabgA86Ke2zzYQopaAEp3avvt9QVsCXCer60ocxKft83WI+wes0vY5hzqtK4K2LDwcAq7TwKoIPNqmPtPO/WvT9xnUIdApq6CyrzqkunE3PIqDtsl3yU76Aoq5wYB0Z67f/wlNAtSrfmnpHNS5dAs3g50tFHeHY2fhp4MwWL3ShF9xdS7djHXqMK2rk/rMu8OnYW6wiTucM4lvT2d8g3yMce7vw/WB6pUyh+fh3nL15i23N9XXrw0A6xLglXx1VomD2NDUnxNuwON/1e+7bfJJkHtf+K8N3JkPru3h0TH15omSn6QeN+IH0LqDjTfEnoab48G5FejS3IBgKuaI6xoN6Aapc+xsqqlLzFeQcBYc15q+z6+1hNkr1WSrUln4bic8jk29S3bWcvX5c726GO63fS80eD5j0paQAFMXwbnLMHWYmijef6i+5qxTk0knB2j7Inz5HXgWBS8P9Tl6kPrYFBN44sSuQoUKODg48Msvv+gvtafYv38/vr6+jBs3Tr/t8uXLT3rIfLFv3z46dOjA22+rH7qkpCTOnj1LlSrq3Xply5bFxsaGkJAQSpdWzybCw8M5e/YsjRurH4SaNWuSmJjI7du3eeGFF56oPQcPHiQoKMhgvWZN9ZZ6f39/1q1bR2xsrD6QhYTk8Q7N3HJ4DZLuQuRHyQ+drAbum0GbPEyReBWDq0yJN+BOmgn00QvVxbZR6hBqTnWam+Nr6mTgtO3z+M6wz5r0fU7z/kctUhfbRuC5zbg6NTbg8S2ET4J7b6hJn7YsFFkG9q8UTL+b14YHkbDiB7gXARVLwYJB4J484fjW/YxX1i7fhL/Pw6Ihmdc5rbc6HDvxc4iIhuJF4f2OqQ8otraGeQPVhxgP/xRiYqFUMZjUExpWy7zOAiTxzcLjG4BrR0i8B3dmJT+g+DkovTF1KDb+GgYxLv4mXExzBffeEnVxbAB+W9VtDjXBZ7V6c8TdOWBTGopPBdfX09RzC25OVId9bbzAtQt4ZvKoEVMwV1zXDVDvCA4fC8oDsH5OnVNsXQAPZW5aD8IjYe136hBs2dIwfaR6AwXA7Xtq8pnWlRtw4izMGJWxvrsP4MAx9ed+4w1fmz0WaiRPqen9Jmi1auIYFwf+5dQhW2fTPRLoiRM7e3t7Ro8ezahRo7C1taVhw4bcuXOHkydPUqFCBcLCwti4cSO1a9dm27ZtbNmyJT/a/cQqVKjAt99+y/79+ylSpAhz587l1q1b+sDn7OxMjx49GDlyJEWLFqVYsWJMmjQJKysr/dlxxYoV6datG0FBQcyZM4eaNWty584dfvnlF6pXr06bNm24du0azZo1Y+3atdkOL3zzzTfUqlWLRo0asX79eg4fPsznn38OwFtvvcW4cePo06cPY8aMISwsjE8+Uc/8NOk/iKbg1FddMpN+vpu1L5Q0YhJzdnVmxpg685Ouj7pkJiVZS2HtC95G3FmVXZ2gnvm6m3kScZeX1CUzy0Zk3OZbHA6vyLo+D1eY2DP7Y5b2Uh92/BSS+FYI4htA0V7qkpmUZC2FbWmoYsQds86vqEtW3Huri7mYK647D8v8WXkFoUNzdcnMJx9k3OZTAnZmcTWxuGfWr6VlbQ19uqpLAcmX228mTJjA8OHDmThxIpUrV+aNN97g9u3btG/fnqFDhxIcHExAQAD79+9nwoQJ+XHIJzZ+/Hief/55WrRoQdOmTSlevDgdO3Y0KDN37lzq169P27Ztefnll2nYsCGVK1c2mAuyatUqgoKCGD58OP7+/nTs2NHgLDg+Pp7Q0FBiYmKybc+UKVPYuHEj1atXZ+3atWzYsEEfhF1cXPjxxx85fvw4AQEBjBs3jokTJwIY9fgEIUTeSXyT+CbEs0SjKE/BDOVnRHR0NN7e3syZM4devbI4uzNCiRIlmDp1qn5oR6PRsGXLlgyBNzvr16/nnXfeITw8PNt5QCkiIiJwdXWF4lfByiXH8hZFUwg/4ls2mbsFBS/qEbw0mPDwcFxcCtlnPB88y/EN0sQ4/wugfXYf8JwnDwthArx6q7lbULCiH0GnvkbFt2fyT4oVlGPHjnHmzBnq1KlDeHg4H374IQAdOuTtj7LHxMSwb98+bt26RdWqVXPeIY21a9dStmxZvL29+fvvvxk9ejRdunQxOugJIURaEt+EsExPXWIXFhamv0SfmVOnTumHAQrCJ598QmhoKLa2tgQGBvLHH3/g4ZG325RXrFjB1KlTGTJkSIbb/3Ny8+ZNJk6cyM2bNylRogSdO3fWP/1eCPFskPiWOYlvQuSfp24oNiEhgUuXLmX5up+fH9bWT10++tSTodhCRoZizd2aTEl8Mx0Zii1kZCg2S09dBLG2tqZ8+RweGCiEEM8giW9CCFOz1D9KJ4QQQghR6EhiJ4QQQghhISSxE0IIIYSwEJLYCSGEEEJYCEnshBBCCCEshCR2QgghhBAWQhI7IYQQQggLIYmdEEIIIYSFkMROCCGEEMJCSGInhBBCCGEhJLETQgghhLAQktgJIYQQQlgISeyEEEIIISyEJHZCCCGEEBZCEjshhBBCCAshiZ0QQgghhIWQxE4IIYQQwkJIYieEEEIIYSEksRNCCCGEsBDW5m6AKGA3nQFnc7eiYCmTzd2CgqeZbO4WmEEEMNjcjRDmFuoBuJi7FQVLYlwhEAH0NaqkXLETQgghhLAQktgJIYQQQlgISeyEEEIIISyEJHZCCCGEEBZCEjshhBBCCAshiZ0QQgghhIWQxE4IIYQQwkJIYieEEEIIYSEksRNCCCGEsBCS2AkhhBBCWAhJ7IQQQgghLIQkdkIIIYQQFkISOyGEEEIICyGJnRBCCCGEhZDETgghhBDCQkhiJ4QQQghhISSxE0IIIYSwEJLYCSGEEEJYCEnshBBCCCEshCR2QgghhBAWQhI7IYQQQggLIYmdEEIIIYSFkMROCCGEEMJCSGInhBBCCGEhJLETQgghhLAQuUrsmjZtypAhQ7J83c/Pj/nz5+e6EZMnTyYgIMCosj179qRjx465PsbTIqff4dNpMeAH2AN1gcM5lP8GqJRcvhrwU7rXFWAiUAJwAF4GzqV5/RLQCyiT/Ho5YBIQl/cu5Nbiw+A3H+ynQd3P4PC1rMvGJ8KHe6HcQrV8jWWw/bxhmchYGLIdfOeDw3Ro8DmEZFLn6TvQfgO4zgSnj6D2SggLz8+e5aAQvtdpSIx7chLjIOfPPUB7oHRyHSWA7sD1vHYg98wV41K8/z/QTIH5B/OjN0YqHO+zdX5WFhISgpOTU35WmcGCBQtQFMWkxzCl7777DhsbG3M3Ixc2AcOAZahfhPlACyAUKJZJ+f1AV2AG0Bb4CugIHAWeSy4zC1gIrEH9D31Ccp2nUD/8Z4AkYDlQHjgB9AaigU/ytXeZ2nQChu2EZW2gbik18LRYB6HBUCyTz/f4X2Hdv7CyHVTygB3nodMm2P8u1CyhlnnvRzhxG77sBCWdYd0/8PKXcKo/eLuoZS7ch0aroFdNmNIUXOzg5B2wz9evaXYdp9C917kkMS5nEuMg5889wIvAB6j/2V8DRgCvJ9dvYuaKcSm2nIaDV9VyBabwvM/5OhTr6emJo6Njlq/Hx8c/8TFcXV1xc3N74nrMpWjRojg7F+SH+UnNRf2P9h2gCuqXwhH4IovyC4CWwEigMjAVeB74NPl1BfULNR7oAFQH1qKewWxNLtMSWAW8ApRFPeMZAXyXX53K3tyD0Pt5eKcmVPGEZW3B0Qa+OJZ5+S//gQ8aQesKULYI9Kut/jzngPr6o3jYfApmvQyNfaF8UZjcVP136ZHUesb9qu43q7kaLMsVhfb+mQdakyiE73UuSYzLmcQ4Yz73AEOBeoAv0AAYAxwEnvwzlCNzxTiAaxEw8GdY/yrYFORssMLzPuf6t5qQkEBwcDCurq54eHgwYcIE/dll+mEKjUbD0qVLad++PU5OTkyfPh2AmTNn4uXlhbOzM7169eLx48dGHz/9MEXTpk0ZOHAgQ4YMoUiRInh5ebFy5Uqio6N55513cHZ2pnz58vz888/6fRITE+nVqxdlypTBwcEBf39/FixYkKGfgwYNws3NDXd3d0aPHk2PHj0Mjp2UlMSMGTP09dSoUYNvv/022/anH6bw8/Nj6tSpdO3aFScnJ7y9vVm8eLHBPmfOnKFRo0bY29tTpUoVdu/ejUajYevWrUb/3vImDvgL9fJyCqvk9QNZ7HMgXXlQz2BSyv8H3ExXxhX1DCqrOgHCgaJGtfqJxCXCX9fh5bKp26w06vqBq5nvE5uY8aqagzX8Gab+nJAEiUr2ZZIU2HYOKhZVz5yLzVaHR7aeyZ9+5agQvtdZkBiXemyJcZkxxef+PrAe9T9+E1/tNFeMAzXOdd8CIxtA1cyukplK4Xqfc53YrVmzBmtraw4fPsyCBQuYO3cun332WZblJ0+eTKdOnfj333959913+frrr5k8eTIfffQRR44coUSJEixZsuSJOrFmzRo8PDw4fPgwAwcOpF+/fnTu3JkGDRpw9OhRXnnlFbp3705MTAygBqtSpUrxzTffcOrUKSZOnMgHH3zA119/ra/z448/Zv369axatYp9+/YRERGRIcjMmDGDtWvXsmzZMk6ePMnQoUN5++232bt3b67aP3v2bGrUqMGxY8cYM2YMgwcPZteuXYAaoDt27IijoyOHDh1ixYoVjBs3Lsc6Y2NjiYiIMFhy7y6QCHil2+6F+oHOzM0cyt9Ms83YOs8Di4C+ObQ3H9yNUQOUV7qrZF5OcDMq831alFPPgM/dUwPXrgvw3Wm4kVze2Q7ql4Kpv8P1SEhMUocpDlxNLXM7GqLiYOY+aFkOdnaHTpXg1U2w95LJupuqEL7XWZAYl0piXGby83M/GnAC3IEw4HujWv1EzBXjAD7+E6ytYFBd0/QtS4Xrfc715B0fHx/mzZuHRqPB39+ff//9l3nz5tG7d+9My7/11lu88847+vU333yTXr160atXLwCmTZvG7t27c3VGm16NGjUYP348AGPHjmXmzJl4eHjo2zRx4kSWLl3KP//8Q7169bCxsWHKlCn6/cuUKcOBAwf4+uuv6dKlCwCLFi1i7NixdOrUCYBPP/2Un35KnTgZGxvLRx99xO7du6lfvz4AZcuW5c8//2T58uU0adLE6PY3bNiQMWPGAFCxYkX27dvHvHnzaN68Obt27eLChQvs2bOH4sWLAzB9+nSaN2+ebZ0zZsww6OOz6xrq5fDOqJfRn0ILWkLvH6HSYtCgDqG+EwBfHE8t82UnePcH8J4LWg08XwK6Pgd/3VBfT0qeU9XBH4aqnycCisP+K7DsL2jiV3D9MZun472WGKeSGFcQRqLePHQZmAIEAf9DDSRPkfyIcX9dhwWH4Ghf0Dxl/TO5gn2fc33Frl69emjSvCn169fn3LlzJCYmZlq+Vq1aBuunT5+mbl3DbD0laORV9erV9T9rtVrc3d2pVq2afpuXl5pR3759W79t8eLFBAYG4unpiU6nY8WKFYSFqZeMw8PDuXXrFnXq1DGoNzAwUL9+/vx5YmJiaN68OTqdTr+sXbuWCxcu5Kr96ftfv359Tp8+DUBoaCg+Pj76gAcYtCsrY8eOJTw8XL9cuXIlV21SeQBa4Fa67beA4hmLQ/L27MoXT7Mtpzqvo048bQCsMK7JT8rDUQ1Kt6INt9+KhuK6zPfxdIKtb0L0B3B5CJwZADpbdS5KinJFYW9PiBoLV4bC4d4Qn5RaxsNRPZOt4mlYd2WPArorthC+11mQGKeSGFcQn3sPoCLQHNiIetelie8SNVeM+yNMHZkoPQ+sP1SXy+EwfKd6d65JFa732eS325n6DjIgwx1YGo3GYFtKkE5KSgJg48aNjBgxgjlz5lC/fn2cnZ2ZPXs2hw4dMvqYUVHq5eVt27bh7e1t8JqdnV2e+pGf7Ozs8qEdtkAg8Avq3UCg3sH4CxCcxT71k18fkmbbruTtoN45VDy5TEDytgjgENAvzT7XUP+jD0SdXF9Ak2xttRBYEn65CB0rqduSFHU9OIf/bOyt1bu/4hNh82noUjVjGSdbdXnwSL2zbFbz1OPWLgmh9wzLn70Pvq5P3q8cFcL3Op9IjDOPZz/GpZeU/G+skW3PI3PFuO7VDef1gTqfuHt19eqfSRWu9znXiV36wHDw4EEqVKiAVqs1av/KlStz6NAhgoKCDOooSPv27aNBgwb0799fvy3tGairqyteXl6EhITQuHFjQJ0HcvToUf2zqKpUqYKdnR1hYWG5GpLITPr+Hzx4kMqVKwPg7+/PlStXuHXrlv6sPCQk5ImOlzvDgB5ALaAO6l1A0ah3FoF6Sdkb9ZZwgMFAE2AO0Ab17OQIqVdhNKhflGlABVJvES9J6hfuGtAU9S6iT4A7adqT1dlVPhpWD3pshVoloY63+iiA6PjU4BO0BbydYUbypNlDV+FapDp0ei0CJu9VA+Wohql17jiv3kTl7w7n78PIXepjA9IGtJEN4I1voXFpeLGM+pyoH0NhT0/T9xkolO91JiTGBQAS40z7uT8EhACNgCLAheQy5UhNHEzIHDHO3VFd0rKxUq8S+nuYtr9AYXqfc53YhYWFMWzYMPr27cvRo0dZtGgRc+bMMXr/wYMH07NnT2rVqkXDhg1Zv349J0+epGzZsjnvnE8qVKjA2rVr2bFjB2XKlOHLL78kJCSEMmXK6MsMHDiQGTNmUL58eSpVqsSiRYt48OCB/szY2dmZESNGMHToUJKSkmjUqBHh4eHs27cPFxcXevToAUClSpWYMWOGfh5LZvbt28esWbPo2LEju3bt4ptvvmHbtm0ANG/enHLlytGjRw9mzZpFZGSkfq6NpkDmKbyB+p/tRNQJoQHAdlInjIZheIWlAerzfsajPrunAuqt38+lKTMK9QvVB3iI+qHfTupzf3ahTqI/D5RK154CeL7XG8/BnRiYuEedTBxQHLZ3A6/kYYqwcPUushSPE9TnPF18oA5PtK6gzjdxs08tEx4LY3+BqxFQ1AFeqwzTXwKbNMlCp8rqYwdm/AmDtqsBcnMXaFTa9H0GCuV7nQmJcRLjTP+5d0R9pM+k5HIlUOeXjgcK4GqouWKcWRWe9znXiV1QUBCPHj2iTp06aLVaBg8eTJ8+fYze/4033uDChQuMGjWKx48f89prr9GvXz927NiR26bkWd++fTl27BhvvPEGGo2Grl270r9/f4PHBYwePZqbN28SFBSEVqulT58+tGjRwuCsferUqXh6ejJjxgwuXryIm5sbzz//PB988IG+TGhoKOHh2c+RGj58OEeOHGHKlCm4uLgwd+5cWrRoAajzXrZu3cp7771H7dq1KVu2LLNnz6Zdu3bY29tnW2/+CSbry9V7MtnWOXnJigb4MHnJTM/kxYyC62Q9LJH+CloTPzg1IPv6ulTNfNgivXdrqovZFML3Oh2JcRLjDO3JZNuTfu6rAb8a2zjTMFeMS+vSkNyVf2KF433WKM/yI84LUFJSEpUrV6ZLly5MnTo1z/XUr1+fZs2aMW3aNEB9xtOQIUNy9Sd49u3bR6NGjTh//jzlypUzap+IiAhcXV1RnxHmklNxy6JMNncLCp5msrlbYAYRgCvh4eG4uBSyz3g+kBj3DJMYVwgYH98K6m8VPXMuX77Mzp07adKkCbGxsXz66af8999/vPXWW3mqLzY2ln///ZeTJ08yaNCgXO27ZcsWdDodFSpU4Pz58wwePJiGDRsaHfCEECI9iXFCWKan7vaztLfVp1/++OOPAmuHlZUVq1evpnbt2jRs2JB///2X3bt36yf85tbPP//MSy+9RPv27Xn99ddztW9kZCQDBgygUqVK9OzZk9q1a/P99wXwIEshRL6TGJeRxDgh8s9TNxR7/vz5LF/z9vbGwcGhAFtjOWSYopApdMMU8KwMxUqMMw2JcYVMoYtxz/BQbPny5c3dBCGEMBmJcUIIU3rqhmKFEEIIIUTeSGInhBBCCGEhJLETQgghhLAQktgJIYQQQlgISeyEEEIIISyEJHZCCCGEEBZCEjshhBBCCAshiZ0QQgghhIWQxE4IIYQQwkJIYieEEEIIYSEksRNCCCGEsBCS2AkhhBBCWAhJ7IQQQgghLIQkdkIIIYQQFkISOyGEEEIICyGJnRBCCCGEhZDETgghhBDCQkhiJ4QQQghhISSxE0IIIYSwENbmboAoGIqiJP8UYdZ2mEVErLlbYAaF8H1O7nPqZ10UJhLjCpvC9j4bH98ksSskIiMjk3/yMWs7zMLV3A0wh5nmboDZREZG4upaKN/0Qk1iXGFTOGOcMfFNo8jpbaGQlJTE9evXcXZ2RqPRFOixIyIi8PHx4cqVK7i4uBTosc2lMPYZzNtvRVGIjIykZMmSWFnJLJPCRmJcwZI+P73xTa7YFRJWVlaUKlXKrG1wcXEpNAEgRWHsM5iv33KlrvCSGGce0ueCY2x8k9NaIYQQQggLIYmdEEIIIYSFkMROmJydnR2TJk3Czs7O3E0pMIWxz1B4+y0Kt8L4uZc+P73k5gkhhBBCCAshV+yEEEIIISyEJHZCCCGEEBZCEjshhBBCCAshiZ0QQgghhIWQxE4IIYQQwkJIYieEEEIIYSEksRNCCCGEsBCS2AkhhBBCWIj/A3Qqo4phUxncAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fp_audio_vision_scores = softmax(embeddings[ModalityType.VISION] @ embeddings[ModalityType.AUDIO].T, axis=-1)\n",
    "int_audio_vision_scores = softmax(quantized_embeddings[ModalityType.VISION] @ quantized_embeddings[ModalityType.AUDIO].T, axis=-1)\n",
    "\n",
    "visualize_prob_matrices(fp_audio_vision_scores, int_audio_vision_scores, text_list, image_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5173300",
   "metadata": {},
   "source": [
    "#### Compare File Size\n",
    "[back to top ⬆️](#Table-of-contents:)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "d87de6af",
   "metadata": {},
   "outputs": [],
   "source": [
    "def calculate_compression_rate(modality):\n",
    "    fp16_ir_model_size = Path(fp_model_paths[modality]).with_suffix(\".bin\").stat().st_size / 1024\n",
    "    quantized_model_size = Path(int8_model_paths[modality]).with_suffix(\".bin\").stat().st_size / 1024\n",
    "    print(f'Modality: {modality}')\n",
    "    print(f\"    * FP16 IR model size: {fp16_ir_model_size:.2f} KB\")\n",
    "    print(f\"    * INT8 model size: {quantized_model_size:.2f} KB\")\n",
    "    print(f\"    * Model compression rate: {fp16_ir_model_size / quantized_model_size:.3f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "3d673a2c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Modality: text\n",
      "    * FP16 IR model size: 691481.77 KB\n",
      "    * INT8 model size: 347007.35 KB\n",
      "    * Model compression rate: 1.993\n",
      "Modality: vision\n",
      "    * FP16 IR model size: 1235995.26 KB\n",
      "    * INT8 model size: 620133.72 KB\n",
      "    * Model compression rate: 1.993\n",
      "Modality: audio\n",
      "    * FP16 IR model size: 168429.22 KB\n",
      "    * INT8 model size: 84818.78 KB\n",
      "    * Model compression rate: 1.986\n"
     ]
    }
   ],
   "source": [
    "for modality in modalities:\n",
    "    calculate_compression_rate(modality)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c5bd231",
   "metadata": {},
   "source": [
    "#### Compare inference time of the FP16 IR and quantized models\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "To measure the inference performance of the `FP16` and `INT8` models, we use median inference time on calibration dataset.\n",
    "So we can approximately estimate the speed up of the dynamic quantized models.\n",
    "\n",
    "\n",
    "> **NOTE**: For the most accurate performance estimation, it is recommended to run `benchmark_app` in a terminal/command prompt after closing other applications with static shapes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "92ce5218",
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "\n",
    "def calculate_inference_time(model_path, calibration_data):\n",
    "    model = core.compile_model(model_path)\n",
    "    output_layer = model.output(0)\n",
    "    inference_time = []\n",
    "    for batch in calibration_data:\n",
    "        start = time.perf_counter()\n",
    "        _ = model(batch)[output_layer]\n",
    "        end = time.perf_counter()\n",
    "        delta = end - start\n",
    "        inference_time.append(delta)\n",
    "    return np.median(inference_time)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "108e7533",
   "metadata": {},
   "source": [
    "Vision model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "8c59a370",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Performance speed up: 2.040\n"
     ]
    }
   ],
   "source": [
    "fp16_latency = calculate_inference_time(fp_model_paths[ModalityType.VISION], vision_data)\n",
    "int8_latency = calculate_inference_time(int8_model_paths[ModalityType.VISION], vision_data)\n",
    "print(f\"Performance speed up: {fp16_latency / int8_latency:.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4775b72a",
   "metadata": {},
   "source": [
    "Text model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "8ef63bf6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Performance speed up: 1.404\n"
     ]
    }
   ],
   "source": [
    "fp16_latency = calculate_inference_time(fp_model_paths[ModalityType.TEXT], text_data)\n",
    "int8_latency = calculate_inference_time(int8_model_paths[ModalityType.TEXT], text_data)\n",
    "print(f\"Performance speed up: {fp16_latency / int8_latency:.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1d075b5",
   "metadata": {},
   "source": [
    "Audio model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "88ed82c6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Performance speed up: 5.713\n"
     ]
    }
   ],
   "source": [
    "fp16_latency = calculate_inference_time(fp_model_paths[ModalityType.AUDIO], audio_calibration_data)\n",
    "int8_latency = calculate_inference_time(int8_model_paths[ModalityType.AUDIO], audio_calibration_data)\n",
    "print(f\"Performance speed up: {fp16_latency / int8_latency:.3f}\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {
     "033cb02215784f52b83bcd3ac7a9d112": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "FloatProgressModel",
      "state": {
       "bar_style": "danger",
       "layout": "IPY_MODEL_515071a03e134a409b39b24dfe00e29a",
       "max": 1,
       "style": "IPY_MODEL_8e66621fb88446f3bb36596a732a0132",
       "value": 1
      }
     },
     "0642738422a8408f9b23b3f811605520": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "0984cd875f0848288f2de73fd4cb20b9": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "0f65e2986e2a4c73ac0828c503cff91a": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "DropdownModel",
      "state": {
       "_options_labels": [
        "CPU",
        "GPU",
        "AUTO"
       ],
       "description": "Device:",
       "index": 2,
       "layout": "IPY_MODEL_ece349879b134acdbb09b19ead7d0004",
       "style": "IPY_MODEL_74e8984d95b44d0f9c65744c0b48eed5"
      }
     },
     "189dd7e80684460aa4430a96d95c112c": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "1945ed5eadf042ee9a48160a1278ce31": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "242df870a8da438a9a3b488455ea73f4": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HBoxModel",
      "state": {
       "children": [
        "IPY_MODEL_c63c2b954fa742b5b7f71895bc3380c0",
        "IPY_MODEL_033cb02215784f52b83bcd3ac7a9d112",
        "IPY_MODEL_89ec94b2f6bf4705806a2ba2e5ced0ab"
       ],
       "layout": "IPY_MODEL_0642738422a8408f9b23b3f811605520"
      }
     },
     "2ea11108cabe47c1bdd4eb2f8e3a4552": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "465031ca7a00440a8a356d6c56187da3": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "515071a03e134a409b39b24dfe00e29a": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {
       "width": "20px"
      }
     },
     "52886784d30d461299a8a9eaa40c7ad8": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_0984cd875f0848288f2de73fd4cb20b9",
       "style": "IPY_MODEL_85beef0141df48fc86991d89396b189a"
      }
     },
     "74e8984d95b44d0f9c65744c0b48eed5": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "description_width": ""
      }
     },
     "8455091be5fa4de887d1f8cdd79218b9": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "FloatProgressModel",
      "state": {
       "bar_style": "success",
       "layout": "IPY_MODEL_ea8c340979dd4b9499a4c24d634ec41a",
       "max": 1,
       "style": "IPY_MODEL_dc4fba684a1f44e88683812e6473004a",
       "value": 1
      }
     },
     "84e4df4dc5d6445c91e20fab53c00bf1": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "85beef0141df48fc86991d89396b189a": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "89ec94b2f6bf4705806a2ba2e5ced0ab": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_b5dc390390a54178ac97c4a33aa789f9",
       "style": "IPY_MODEL_84e4df4dc5d6445c91e20fab53c00bf1",
       "value": " 352/? [10:04&lt;00:00,  1.84s/it]"
      }
     },
     "8e66621fb88446f3bb36596a732a0132": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "description_width": ""
      }
     },
     "91de671970d5430fa5051869b1a09464": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HBoxModel",
      "state": {
       "children": [
        "IPY_MODEL_52886784d30d461299a8a9eaa40c7ad8",
        "IPY_MODEL_8455091be5fa4de887d1f8cdd79218b9",
        "IPY_MODEL_a06fb91d544a4d4080c7eb675a1f0aad"
       ],
       "layout": "IPY_MODEL_1945ed5eadf042ee9a48160a1278ce31"
      }
     },
     "a06fb91d544a4d4080c7eb675a1f0aad": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_465031ca7a00440a8a356d6c56187da3",
       "style": "IPY_MODEL_2ea11108cabe47c1bdd4eb2f8e3a4552",
       "value": " 300/? [00:49&lt;00:00, 29.39it/s]"
      }
     },
     "b5dc390390a54178ac97c4a33aa789f9": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "c63c2b954fa742b5b7f71895bc3380c0": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_189dd7e80684460aa4430a96d95c112c",
       "style": "IPY_MODEL_f43c5d5ad9ed4e109ef587756722680b"
      }
     },
     "dc4fba684a1f44e88683812e6473004a": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "description_width": ""
      }
     },
     "ea8c340979dd4b9499a4c24d634ec41a": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {
       "width": "20px"
      }
     },
     "ece349879b134acdbb09b19ead7d0004": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "f43c5d5ad9ed4e109ef587756722680b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     }
    },
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}