notebooks/244-named-entity-recognition/244-named-entity-recognition.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Named entity recognition with OpenVINO™\n",
    "\n",
    "\n",
    "The Named Entity Recognition(NER) is a natural language processing method that involves the detecting of key information in the unstructured text and categorizing it into pre-defined categories. These categories or named entities refer to the key subjects of text, such as names, locations, companies and etc.\n",
    "\n",
    "NER is a good method for the situations when a high-level overview of a large amount of text is needed. NER can be helpful with such task as analyzing key information in unstructured text or automates the information extraction of large amounts of data.\n",
    "\n",
    "\n",
    "This tutorial shows how to perform named entity recognition using OpenVINO. We will use the pre-trained model [`elastic/distilbert-base-cased-finetuned-conll03-english`](https://huggingface.co/elastic/distilbert-base-cased-finetuned-conll03-english). It is DistilBERT based model, trained on [`conll03 english dataset`](https://huggingface.co/datasets/conll2003). The model can recognize four named entities in text: persons, locations, organizations and names of miscellaneous entities that do not belong to the previous three groups. The model is sensitive to capital letters.\n",
    "\n",
    "To simplify the user experience, the [Hugging Face Optimum](https://huggingface.co/docs/optimum) library is used to convert the model to OpenVINO™ IR format and quantize it.\n",
    "\n",
    "#### Table of contents:\n",
    "- [Prerequisites](#Prerequisites)\n",
    "- [Download the NER model](#Download-the-NER-model)\n",
    "- [Quantize the model, using Hugging Face Optimum API](#Quantize-the-model,-using-Hugging-Face-Optimum-API)\n",
    "- [Compare the Original and Quantized Models](#Compare-the-Original-and-Quantized-Models)\n",
    "    - [Compare performance](#Compare-performance)\n",
    "    - [Compare size of the models](#Compare-size-of-the-models)\n",
    "- [Prepare demo for Named Entity Recognition OpenVINO Runtime](#Prepare-demo-for-Named-Entity-Recognition-OpenVINO-Runtime)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prerequisites\n",
    "[back to top ⬆️](#Table-of-contents:)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install -q \"diffusers>=0.17.1\" \"openvino>=2023.1.0\" \"nncf>=2.5.0\" \"gradio\" \"onnx>=1.11.0\" \"transformers>=4.33.0\" --extra-index-url https://download.pytorch.org/whl/cpu\n",
    "%pip install -q \"git+https://github.com/huggingface/optimum-intel.git\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download the NER model\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "We load the [`distilbert-base-cased-finetuned-conll03-english`](https://huggingface.co/elastic/distilbert-base-cased-finetuned-conll03-english) model from the [Hugging Face Hub](https://huggingface.co/models) with [Hugging Face Transformers library](https://huggingface.co/docs/transformers/index).\n",
    "\n",
    "Model class initialization starts with calling `from_pretrained` method. To easily save the model, you can use the `save_pretrained()` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import AutoTokenizer, AutoModelForTokenClassification\n",
    "\n",
    "model_id = \"elastic/distilbert-base-cased-finetuned-conll03-english\"\n",
    "model = AutoModelForTokenClassification.from_pretrained(model_id)\n",
    "\n",
    "original_ner_model_dir = 'original_ner_model'\n",
    "model.save_pretrained(original_ner_model_dir)\n",
    "\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Quantize the model, using Hugging Face Optimum API\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "Post-training static quantization introduces an additional calibration step where data is fed through the network in order to compute the activations quantization parameters. For quantization it will be used [Hugging Face Optimum Intel API](https://huggingface.co/docs/optimum/intel/index).\n",
    "\n",
    "To handle the NNCF quantization process we use class [`OVQuantizer`](https://huggingface.co/docs/optimum/intel/reference_ov#optimum.intel.OVQuantizer). The quantization with Hugging Face Optimum Intel API contains the next steps:\n",
    "* Model class initialization starts with calling `from_pretrained()` method.\n",
    "* Next we create calibration dataset with `get_calibration_dataset()` to use for the post-training static quantization calibration step. \n",
    "* After we quantize a model and save the resulting model in the OpenVINO IR format to save_directory with `quantize()` method. \n",
    "* Then we load the quantized model. The Optimum Inference models are API compatible with Hugging Face Transformers models and we can just replace `AutoModelForXxx` class with the corresponding `OVModelForXxx` class. So we use `OVModelForTokenClassification` to load the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:nncf:NNCF initialized successfully. Supported frameworks detected: torch, onnx, openvino\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'\n",
      "/home/ea/work/openvino_notebooks/test_env/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0.\n",
      "You can remove this warning by passing 'token=False' instead.\n",
      "  warnings.warn(\n",
      "No configuration describing the quantization process was provided, a default OVConfig will be generated.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:nncf:Not adding activation input quantizer for operation: 3 DistilBertForTokenClassification/DistilBertModel[distilbert]/Embeddings[embeddings]/NNCFEmbedding[position_embeddings]/embedding_0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 2 DistilBertForTokenClassification/DistilBertModel[distilbert]/Embeddings[embeddings]/NNCFEmbedding[word_embeddings]/embedding_0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 4 DistilBertForTokenClassification/DistilBertModel[distilbert]/Embeddings[embeddings]/__add___0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 5 DistilBertForTokenClassification/DistilBertModel[distilbert]/Embeddings[embeddings]/NNCFLayerNorm[LayerNorm]/layer_norm_0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 6 DistilBertForTokenClassification/DistilBertModel[distilbert]/Embeddings[embeddings]/Dropout[dropout]/dropout_0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 16 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[0]/MultiHeadSelfAttention[attention]/__truediv___0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 25 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[0]/MultiHeadSelfAttention[attention]/matmul_1\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 30 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[0]/__add___0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 31 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[0]/NNCFLayerNorm[sa_layer_norm]/layer_norm_0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 35 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[0]/__add___1\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 36 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[0]/NNCFLayerNorm[output_layer_norm]/layer_norm_0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 46 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[1]/MultiHeadSelfAttention[attention]/__truediv___0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 55 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[1]/MultiHeadSelfAttention[attention]/matmul_1\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 60 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[1]/__add___0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 61 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[1]/NNCFLayerNorm[sa_layer_norm]/layer_norm_0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 65 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[1]/__add___1\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 66 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[1]/NNCFLayerNorm[output_layer_norm]/layer_norm_0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 76 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[2]/MultiHeadSelfAttention[attention]/__truediv___0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 85 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[2]/MultiHeadSelfAttention[attention]/matmul_1\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 90 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[2]/__add___0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 91 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[2]/NNCFLayerNorm[sa_layer_norm]/layer_norm_0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 95 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[2]/__add___1\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 96 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[2]/NNCFLayerNorm[output_layer_norm]/layer_norm_0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 106 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[3]/MultiHeadSelfAttention[attention]/__truediv___0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 115 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[3]/MultiHeadSelfAttention[attention]/matmul_1\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 120 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[3]/__add___0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 121 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[3]/NNCFLayerNorm[sa_layer_norm]/layer_norm_0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 125 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[3]/__add___1\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 126 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[3]/NNCFLayerNorm[output_layer_norm]/layer_norm_0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 136 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[4]/MultiHeadSelfAttention[attention]/__truediv___0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 145 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[4]/MultiHeadSelfAttention[attention]/matmul_1\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 150 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[4]/__add___0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 151 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[4]/NNCFLayerNorm[sa_layer_norm]/layer_norm_0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 155 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[4]/__add___1\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 156 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[4]/NNCFLayerNorm[output_layer_norm]/layer_norm_0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 166 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[5]/MultiHeadSelfAttention[attention]/__truediv___0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 175 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[5]/MultiHeadSelfAttention[attention]/matmul_1\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 180 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[5]/__add___0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 181 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[5]/NNCFLayerNorm[sa_layer_norm]/layer_norm_0\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 185 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[5]/__add___1\n",
      "INFO:nncf:Not adding activation input quantizer for operation: 186 DistilBertForTokenClassification/DistilBertModel[distilbert]/Transformer[transformer]/ModuleList[layer]/TransformerBlock[5]/NNCFLayerNorm[output_layer_norm]/layer_norm_0\n",
      "INFO:nncf:Collecting tensor statistics |█               | 33 / 300\n",
      "INFO:nncf:Collecting tensor statistics |███             | 66 / 300\n",
      "INFO:nncf:Collecting tensor statistics |█████           | 99 / 300\n",
      "INFO:nncf:Compiling and loading torch extension: quantized_functions_cpu...\n",
      "INFO:nncf:Finished loading torch extension: quantized_functions_cpu\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using framework PyTorch: 2.1.0+cpu\n",
      "/home/ea/work/openvino_notebooks/test_env/lib/python3.8/site-packages/nncf/torch/dynamic_graph/wrappers.py:82: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.\n",
      "  result = operator(*args, **kwargs)\n",
      "Configuration saved in quantized_ner_model/openvino_config.json\n",
      "Compiling the model to CPU ...\n",
      "Setting OpenVINO CACHE_DIR to quantized_ner_model/model_cache\n"
     ]
    }
   ],
   "source": [
    "from functools import partial\n",
    "from optimum.intel import OVQuantizer\n",
    "\n",
    "from optimum.intel import OVModelForTokenClassification\n",
    "\n",
    "def preprocess_fn(data, tokenizer):\n",
    "    examples = []\n",
    "    for data_chunk in data[\"tokens\"]:\n",
    "        examples.append(' '.join(data_chunk))\n",
    "\n",
    "    return tokenizer(\n",
    "        examples, padding=True, truncation=True, max_length=128\n",
    "    )\n",
    "\n",
    "quantizer = OVQuantizer.from_pretrained(model)\n",
    "calibration_dataset = quantizer.get_calibration_dataset(\n",
    "    \"conll2003\",\n",
    "    preprocess_function=partial(preprocess_fn, tokenizer=tokenizer),\n",
    "    num_samples=100,\n",
    "    dataset_split=\"train\",\n",
    "    preprocess_batch=True,\n",
    ")\n",
    "\n",
    "# The directory where the quantized model will be saved\n",
    "quantized_ner_model_dir = \"quantized_ner_model\"\n",
    "\n",
    "# Apply static quantization and save the resulting model in the OpenVINO IR format\n",
    "quantizer.quantize(calibration_dataset=calibration_dataset, save_directory=quantized_ner_model_dir)\n",
    "\n",
    "# Load the quantized model\n",
    "optimized_model = OVModelForTokenClassification.from_pretrained(quantized_ner_model_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Compare the Original and Quantized Models\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "",
    "\n",
    "Compare the original [`distilbert-base-cased-finetuned-conll03-english`](https://huggingface.co/elastic/distilbert-base-cased-finetuned-conll03-english) model with quantized and converted to OpenVINO IR format models to see the difference."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Compare performance\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "",
    "\n",
    "As the Optimum Inference models are API compatible with Hugging Face Transformers models, we can just use `pipleine()` from [Hugging Face Transformers API](https://huggingface.co/docs/transformers/index) for inference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import pipeline\n",
    "\n",
    "ner_pipeline_optimized = pipeline(\"token-classification\", model=optimized_model, tokenizer=tokenizer)\n",
    "\n",
    "ner_pipeline_original = pipeline(\"token-classification\", model=model, tokenizer=tokenizer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Median inference time of quantized model: 0.008135671014315449 \n",
      "Median inference time of original model: 0.108725632991991 \n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "import numpy as np\n",
    "\n",
    "def calc_perf(ner_pipeline):\n",
    "    inference_times = []\n",
    "\n",
    "    for data in calibration_dataset:\n",
    "        text = ' '.join(data['tokens'])\n",
    "        start = time.perf_counter()\n",
    "        ner_pipeline(text)\n",
    "        end = time.perf_counter()\n",
    "        inference_times.append(end - start)\n",
    "\n",
    "    return np.median(inference_times)\n",
    "\n",
    "\n",
    "print(\n",
    "    f\"Median inference time of quantized model: {calc_perf(ner_pipeline_optimized)} \"\n",
    ")\n",
    "\n",
    "print(\n",
    "    f\"Median inference time of original model: {calc_perf(ner_pipeline_original)} \"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Compare size of the models\n",
    "[back to top ⬆️](#Table-of-contents:)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Size of original model in Bytes is 260803668\n",
      "Size of quantized model in Bytes is 133539000\n"
     ]
    }
   ],
   "source": [
    "from pathlib import Path\n",
    "\n",
    "pytorch_model_file = Path(original_ner_model_dir) / \"pytorch_model.bin\" \n",
    "if not pytorch_model_file.exists():\n",
    "    pytorch_model_file = pytorch_model_file.parent / \"model.safetensors\"\n",
    "print(f'Size of original model in Bytes is {pytorch_model_file.stat().st_size}')\n",
    "print(f'Size of quantized model in Bytes is {Path(quantized_ner_model_dir, \"openvino_model.bin\").stat().st_size}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prepare demo for Named Entity Recognition OpenVINO Runtime\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "",
    "\n",
    "Now, you can try NER model on own text. Put your sentence to input text box, click Submit button, the model label the recognized entities in the text."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [],
    "test_replace": {
     "demo.launch(debug=True)": "demo.launch()",
     "demo.launch(share=True, debug=True)": "demo.launch(share=True)"
    }
   },
   "outputs": [],
   "source": [
    "import gradio as gr\n",
    "\n",
    "examples = [\n",
    "    \"My name is Wolfgang and I live in Berlin.\",\n",
    "]\n",
    "\n",
    "def run_ner(text):\n",
    "    output = ner_pipeline_optimized(text)\n",
    "    return {\"text\": text, \"entities\": output} \n",
    "\n",
    "demo = gr.Interface(run_ner,\n",
    "                    gr.Textbox(placeholder=\"Enter sentence here...\", label=\"Input Text\"), \n",
    "                    gr.HighlightedText(label=\"Output Text\"),\n",
    "                    examples=examples,\n",
    "                    allow_flagging=\"never\")\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    try:\n",
    "        demo.launch(debug=True)\n",
    "    except Exception:\n",
    "        demo.launch(share=True, debug=True)\n",
    "# if you are launching remotely, specify server_name and server_port\n",
    "# demo.launch(server_name='your server name', server_port='server port in int')\n",
    "# Read more in the docs: https://gradio.app/docs/"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  },
  "vscode": {
   "interpreter": {
    "hash": "1c707170576399eaaed0c4f2e01a2d1b61ba791ba1842c47e5b3e4f6f79b82ab"
   }
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}