notebooks/mixtral-8x7b-qlora-finetune.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "6d32925a-874a-4ebb-9ad7-5fb5fe6c5030",
   "metadata": {},
   "source": [
    "# Fine-tune Mixtral 8x7B (MoE) model with QLoRA"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9233c0f4-f3ea-472c-a2e5-02bfee498a43",
   "metadata": {
    "tags": []
   },
   "source": [
    "----\n",
    "This notebook contains an example on how to fine-tune the Mixtral 8x7B model using Hugging Face's PEFT [LoRA](https://huggingface.co/docs/peft/main/en/conceptual_guides/lora), and bitsandbytes.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4bbd416b-4f94-498b-bc72-ed46c062e229",
   "metadata": {},
   "source": [
    "----\n",
    "This notebook was tested with the following configurations:\n",
    "1. Instance Type: `ml.g5.48xlarge` (192 vCPUs, 768 GB memory, 8 A10Gs, 24GB GPU memory) -- You may choose a smaller GPU instance, but make sure that the model fits (the model itself takes up around 100 GB of GPU memory)\n",
    "2. SageMaker Notebook with a Python3 kernel"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4e1022f-0e1e-4bc0-999c-cb7068c46b2e",
   "metadata": {},
   "source": [
    "## Step 0: Prerequisites"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4c5ae4a-24ee-4e44-ae00-6487980dadbb",
   "metadata": {},
   "source": [
    "To get started, install the required libraries -- this includes the relevant Hugging Face libraries. The commands below also check if Python 3 and the NVIDIA CUDA is installed. If you run into errors here, make sure you use a similar configuration as above and come back to this step.\n",
    "\n",
    "In this step, we will also be loading our dataset. The dataset we use here is the [GEM/Viggo](https://huggingface.co/datasets/GEM/viggo) Dataset available on Hugging Face. Feel free to swap out this dataset with one of your own: You may also have to change the System Prompt, as described in the function `generate_and_tokenize_promp()` below.\n",
    "\n",
    "To be able to download the dataset and the model from Hugging Face, you would need to follow these steps.\n",
    "1. On Hugging Face, visit the [Mixtral Model Page](https://huggingface.co/mistralai/Mixtral-8x7B-v0.1). On there, you will see that it is a gated repo. Please click \"Request Access\". If prompted to log-in or create-account, please do so.\n",
    "2. Once you have access to the model, you also need a [Hugging Face API Key/Security Token](https://huggingface.co/docs/hub/en/security-tokens). Please follow the steps to create the token. Make sure your token has at least READ permissions.\n",
    "3. On your SageMaker Notebook, open up a terminal (`Launcher/+ --> Other --> Terminal`)\n",
    "4. On the terminal, install the Hugging Face CLI: `pip3 install -U huggingface_hub[cli]`\n",
    "5. Lastly, run `huggingface-cli login` to login to your Hugging Face account and enter your Hugging Face username and password."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0464e462-3dd7-4a08-8dac-fed468fa4849",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python3 --version"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "652b5b0d-4aa7-478a-abef-10d3b4fd7185",
   "metadata": {},
   "source": [
    "Note: These commands below doesn't need to be run, but is very helpful for:\n",
    "1. Checking NVIDIA GPU Memory Usage\n",
    "2. Checking storage on your instance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7e42abb5-e4c4-49c4-81f6-d896370b96cf",
   "metadata": {},
   "outputs": [],
   "source": [
    "!nvidia-smi\n",
    "!df -h"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6b9a44d0-3d9b-4bdb-b928-8659d80dfe08",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip3 install -qU torch bitsandbytes transformers peft datasets\n",
    "!pip3 install -qU tensorboardX\n",
    "# Accelerate is required only if you want to use Flash Attention & FSDP. This notebook does not use this package.\n",
    "!pip3 install -qU accelerate\n",
    "# Matplotlib is used for plotting input lengths. This is optional, please commend out if not required.\n",
    "!pip3 install -qU matplotlib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f532d48b-1981-468e-9756-402eebcfe745",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add installed cuda runtime to path for bitsandbytes \n",
    "import nvidia\n",
    "import os\n",
    "\n",
    "cuda_install_dir = '/'.join(nvidia.__file__.split('/')[:-1]) + '/cuda_runtime/lib/'\n",
    "os.environ['LD_LIBRARY_PATH'] =  cuda_install_dir"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d428c595-2abd-4de3-bab5-591055dd8421",
   "metadata": {},
   "source": [
    "If you'd like to use the accelerate package for Flash Attention and/or FSDP, check out [Accelerate](https://huggingface.co/docs/accelerate/en/usage_guides/fsdp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4d4bc5e3-17b6-41a6-84b0-bcce58a8f935",
   "metadata": {},
   "outputs": [],
   "source": [
    "from datasets import load_dataset\n",
    "import torch\n",
    "\n",
    "train_set = load_dataset(\"gem/viggo\", split=\"train\")\n",
    "validation_set = load_dataset(\"gem/viggo\", split=\"validation\")\n",
    "test_set = load_dataset(\"gem/viggo\", split=\"test\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f518d59e-f7a8-4c46-8365-0a84906c45bb",
   "metadata": {},
   "source": [
    "## Step 1: Loading the Quantized Base Model\n",
    "\n",
    "In this step, we will load the Quantized base model (Mixtral 8x7B) from Hugging Face.\n",
    "\n",
    "### What is the Mixtral model?\n",
    "Architectural Details: The Mixtral 8x7B model is a decoder-only transformer model.\n",
    "\n",
    "Mixtral is a Mixture of Experts (MoE) model with 8 experts per MLP, with a total of 45 billion parameters. Despite the model having 45 billion parameters, the compute required for a single forward pass is the same as that of a 14 billion parameter model. \n",
    "\n",
    "To learn more about Mixture-of-Experts, please refer to the blog post. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bec3593a-5f5f-442a-8228-78f5a6aebe24",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Using 4 bit quantization\n",
    "# FOR SAGEMAKER NOTEBOOKS: Change cache_dir to /home/ec2-user/SageMaker (since EBS is mounted there)\n",
    "# FOR STUDIO NOTEBOOKS (DEFAULT): Change cache_dir to /mnt/sagemaker-nvme (since nvme is mounted there)\n",
    "\n",
    "from transformers import AutoTokenizer, AutoModelForCausalLM, DataCollatorForLanguageModeling, BitsAndBytesConfig\n",
    "\n",
    "model_id = \"mistralai/Mixtral-8x7B-v0.1\"\n",
    "bnb_config = BitsAndBytesConfig(\n",
    "    load_in_4bit=True,\n",
    "    bnb_4bit_use_double_quant=True,\n",
    "    bnb_4bit_compute_dtype=torch.bfloat16\n",
    ")\n",
    "\n",
    "model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config, device_map=\"auto\", cache_dir=\"/mnt/sagemaker-nvme\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08f32f8d-9c84-4cbc-a5b7-5d9ddee9548b",
   "metadata": {},
   "source": [
    "## Step 2: Tokenization\n",
    "\n",
    "To train our model, we need to convert our input data to tokens. We do this using the Hugging Face Transformers Tokenizer. To learn more about the Hugging Face \"Auto\" classes, check out [Auto Classes](https://huggingface.co/docs/transformers/en/model_doc/auto). To learn more about the Tokenizer, along with the EOS, BOS tokens, check out [Tokenizer](https://huggingface.co/docs/transformers/en/main_classes/tokenizer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fc5b782d-3456-4faf-bc5c-4eda98b602c1",
   "metadata": {},
   "outputs": [],
   "source": [
    "tokenizer = AutoTokenizer.from_pretrained(\n",
    "    model_id, \n",
    "    add_eos_token=True,    # End Of Sentence Token\n",
    "    add_bos_token=True     # Beginning Of Sentence Token\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24792146-e02e-4285-93b9-badac64aa1a1",
   "metadata": {},
   "source": [
    "We now need to tokenize our entire training and validation dataset. Before we do that, we also need to format all our data points so that:\n",
    "1. The labels that we pass in during our fine-tuning job are defined. In this case, we set `labels == input_ids`, which as you will see later will just be the entire un-tokenized prompt itself.\n",
    "2. A system prompt is passed in to the LLM while fine-tuning (or even for simple inference).\n",
    "\n",
    "In this example, along with the prompt, we pass in the \"Target Sentence\" from the GEM/viggo dataset, and expect the LLM to generate the \"Meaning Representation\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "53035c2b-8972-4382-9322-51c9b2bc7d88",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Formatting Step 1\n",
    "# Method to tokenize using the loaded tokenizer from above\n",
    "def tokenize(prompt):\n",
    "    result = tokenizer(prompt)\n",
    "    result[\"labels\"] = result[\"input_ids\"].copy()   # Setting the labels and input_ids to be the same\n",
    "    return result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "deccb6f5-f364-4583-b0ad-66454a16fe27",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Formatting Step 2\n",
    "def generate_and_tokenize_prompt(data_point):\n",
    "    full_prompt =f\"\"\"Given a target sentence construct the underlying meaning representation of the input sentence as a single function with attributes and attribute values.\n",
    "This function should describe the target string accurately and the function must be one of the following ['inform', 'request', 'give_opinion', 'confirm', 'verify_attribute', 'suggest', 'request_explanation', 'recommend', 'request_attribute'].\n",
    "The attributes must be one of the following: ['name', 'exp_release_date', 'release_year', 'developer', 'esrb', 'rating', 'genres', 'player_perspective', 'has_multiplayer', 'platforms', 'available_on_steam', 'has_linux_release', 'has_mac_release', 'specifier']\n",
    "\n",
    "### Target sentence:\n",
    "{data_point[\"target\"]}\n",
    "\n",
    "### Meaning representation:\n",
    "{data_point[\"meaning_representation\"]}\n",
    "\"\"\"\n",
    "    return tokenize(full_prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6d749887-3b40-4243-b358-04acc3472c86",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Tokenize the training and validation dataset\n",
    "tokenized_train_dataset = train_set.map(generate_and_tokenize_prompt)\n",
    "tokenized_validation_dataset = validation_set.map(generate_and_tokenize_prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4fc5a981-54de-4da9-a063-98fc728bfdd6",
   "metadata": {},
   "source": [
    "## Step 3: Padding & Re-tokenizing\n",
    "While passing in a test dataset to the LLM for fine-tuning, it's important to ensure that the inputs are all of a uniform length. To achieve this, we first visualize the distribution of the input token lengths (or alternatively, firectly find the max length). Based on these results, we identify the maximum input token length, and utilize \"padding\" to ensure all the inputs are of the same length."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1ede8d7-bfd4-43ba-958b-e1433b8643ce",
   "metadata": {},
   "source": [
    "Option 1: Using Matplotlib, we visualize the distribution of the input token lengths of the entire dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3e653643-2af6-4214-a554-65e090935c45",
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "def plot_data_lengths(tokenized_train_dataset, tokenized_validation_dataset):\n",
    "    lengths1 = [len(x[\"input_ids\"]) for x in tokenized_train_dataset]\n",
    "    lengths2 = [len(x[\"input_ids\"]) for x in tokenized_validation_dataset]\n",
    "    lengths = lengths1 + lengths2\n",
    "    \n",
    "    plt.figure(figsize=(10,6))\n",
    "    plt.hist(lengths, bins=20, alpha=0.7, color=\"blue\")\n",
    "    plt.xlabel(\"input_ids lengths\")\n",
    "    plt.ylabel(\"Frequency\")\n",
    "    plt.title(\"Distribution of lengths of input_ids\")\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a6c0e55-fb82-4f3f-b062-84ad2536812a",
   "metadata": {},
   "source": [
    "Option 2: You may also choose to use the Python `max` function instead of plotting it to directly find the maximum input length like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7ac85cf8-6f68-421c-9185-a6687cad82db",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(max([len(x[\"input_ids\"]) for x in tokenized_train_dataset]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0e7b8fa3-2520-484f-8c37-dd65aba86fcf",
   "metadata": {},
   "outputs": [],
   "source": [
    "# In our example, max length comes out to 344, so that's what we will be padding all the input data to.\n",
    "max_length = 344\n",
    "\n",
    "# Redefine tokenize function to make sure all tensors are same size through padding\n",
    "tokenizer = AutoTokenizer.from_pretrained(\n",
    "    model_id,\n",
    "    padding_side=\"left\",\n",
    "    add_eos_token=True,\n",
    "    add_bos_token=True\n",
    ")\n",
    "\n",
    "tokenizer.pad_token = tokenizer.eos_token    # Arbitrarily using the EOS Token as the Pad token"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d56f44fc-0103-427f-b7d8-06e82274dfe2",
   "metadata": {},
   "outputs": [],
   "source": [
    "def tokenize(prompt):\n",
    "    result = tokenizer(\n",
    "        prompt,\n",
    "        truncation=True,\n",
    "        max_length=max_length,\n",
    "        padding=\"max_length\"\n",
    "    )\n",
    "\n",
    "    result[\"labels\"] = result[\"input_ids\"].copy()\n",
    "    return result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0bfa0507-bfc3-4a32-9018-e4c185d3532d",
   "metadata": {},
   "outputs": [],
   "source": [
    "tokenized_train_dataset = train_set.map(generate_and_tokenize_prompt)\n",
    "tokenized_validation_dataset = validation_set.map(generate_and_tokenize_prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dd3d3903-cfa6-4a6c-a179-8cdde3d875ec",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run the plot function to confirm that your input tokens are all of the same size\n",
    "plot_data_lengths(tokenized_train_dataset, tokenized_validation_dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9af60b4-89c9-4f53-84f0-503da44baee6",
   "metadata": {},
   "source": [
    "## Step 4: Testing the Pre-finetuned Model\n",
    "\n",
    "To be able to compare your out-of-the-box Mixtral quantized model with the Fine-tuned model, you would need to check how the base Mixtral model performs. We can do this by feeding it some test input (i.e., \"Target Sentence\") and compare it with the Ground Truth from our GEM/viggo dataset, to test how far the \"Meaning Representation\" is.\n",
    "\n",
    "After fine-tuning, we will run the same test to verify if the model performs better."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8a5cebcb-03bf-4cba-8dfa-a96934466e70",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Arbitrarily picking from the test dataset\n",
    "print(\"TARGET: \" + test_set[1][\"target\"])\n",
    "print(\"MEANING REPRESENTATION: \" + test_set[35][\"meaning_representation\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "280d83e9-b030-41a5-b7af-c9a95a98a42e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Defining an evaluation tokenizer (similar definition as above) to tokenize the input data\n",
    "eval_tokenizer = AutoTokenizer.from_pretrained(\n",
    "    model_id,\n",
    "    add_bos_token=True\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e28461e0-3de5-4f48-a048-c79f67c49485",
   "metadata": {},
   "outputs": [],
   "source": [
    "eval_prompt = \"\"\"Given a target sentence construct the underlying meaning representation of the input sentence as a single function with attributes and attribute values.\n",
    "This function should describe the target string accurately and the function must be one of the following ['inform', 'request', 'give_opinion', 'confirm', 'verify_attribute', 'suggest', 'request_explanation', 'recommend', 'request_attribute'].\n",
    "The attributes must be one of the following: ['name', 'exp_release_date', 'release_year', 'developer', 'esrb', 'rating', 'genres', 'player_perspective', 'has_multiplayer', 'platforms', 'available_on_steam', 'has_linux_release', 'has_mac_release', 'specifier']\n",
    "\n",
    "### Target sentence:\n",
    "Earlier, you stated that you didn't have strong feelings about PlayStation's Little Big Adventure. Is your opinion true for all games which don't have multiplayer?\n",
    "\n",
    "### Meaning representation:\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9790d702-7c51-4421-8856-f1c20e67d481",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Using your GPUs to perform inference on the above prompt\n",
    "device = \"cuda\"\n",
    "model_input = eval_tokenizer(eval_prompt, return_tensors=\"pt\").to(device)\n",
    "\n",
    "model.eval()\n",
    "with torch.no_grad():\n",
    "    print(eval_tokenizer.decode(model.generate(**model_input, max_new_tokens=128)[0], skip_special_tokens=True))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ccde7a0-61cd-412d-a64b-b325bcb40f54",
   "metadata": {},
   "source": [
    "Here are the results from this run:\n",
    "\n",
    "- **Actual meaning representation** (Ground Truth): verify_attribute(name[Little Big Adventure], rating[average], has_multiplayer[no], platforms[PlayStation]) <br>\n",
    "- **Mixtral's (current) output for meaning representation** (Model Inference): inform(name(Little Big Adventure), has_multiplayer(Little Big Adventure))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cefb3733-1d06-4f5e-9756-51e1355b9b1e",
   "metadata": {},
   "source": [
    "## Step 5: Setting up QLoRA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cf61f9bd-2766-467a-b3c9-c9220b5d4ccc",
   "metadata": {},
   "outputs": [],
   "source": [
    "from peft import prepare_model_for_kbit_training\n",
    "\n",
    "model.gradient_checkpointing_enable()\n",
    "model = prepare_model_for_kbit_training(model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2974f68f-9404-4e3d-ba1f-9591be79e69c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Printing the model here to check the layers. This step is important so you can see what layers have been added after the QLoRA step.\n",
    "print(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93392aa1-f921-4d96-a16f-532ec1357726",
   "metadata": {},
   "source": [
    "In the model, we have the linear layers (`q_proj`, `k_proj`, `v_proj`, `o_proj`, `w1`, `w2`, `w3`, `lm_head`). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9dd1bfd7-70f1-436c-af48-ef5a6c574472",
   "metadata": {},
   "outputs": [],
   "source": [
    "def print_trainable_parameters(model):\n",
    "    \"\"\"\n",
    "    Prints the number of trainable parameters in the model.\n",
    "    \"\"\"\n",
    "    trainable_params = 0\n",
    "    all_param = 0\n",
    "    for _, param in model.named_parameters():\n",
    "        all_param += param.numel()\n",
    "        if param.requires_grad:\n",
    "            trainable_params += param.numel()\n",
    "    print(\n",
    "        f\"trainable params: {trainable_params} || all params: {all_param} || trainable%: {100 * trainable_params / all_param}\"\n",
    "    )    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "047dc723-bde0-4eb2-b02a-b03ad2a56363",
   "metadata": {},
   "outputs": [],
   "source": [
    "print_trainable_parameters(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9c01370-27ad-4869-8d69-665d620b715a",
   "metadata": {},
   "source": [
    "As expected, the base quantized model shows 0 trainable parameters. With QLoRA, we will add adapters to the existing model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "51cbb3bb-3561-48bd-8703-2df51f2c1875",
   "metadata": {},
   "outputs": [],
   "source": [
    "from peft import LoraConfig, PeftModel, get_peft_model\n",
    "\n",
    "target_modules = [\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\", \"w1\", \"w2\", \"w3\", \"lm_head\"]\n",
    "config = LoraConfig(\n",
    "    r=8, \n",
    "    lora_alpha=32, \n",
    "    target_modules=target_modules, \n",
    "    lora_dropout=0.05, \n",
    "    bias=\"none\", \n",
    "    task_type=\"CAUSAL_LM\"\n",
    ")\n",
    "\n",
    "model = get_peft_model(model, config)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b7191739-f48f-4267-83e3-1f42a4c1df7d",
   "metadata": {},
   "outputs": [],
   "source": [
    "print_trainable_parameters(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "77a5af00-ed53-49e5-a9d4-ac88842df034",
   "metadata": {},
   "source": [
    "The increase in the number of trainable parameters indicates the addition of the QLoRA Adapters! You may also choose to print the model to visualize the additional adapters added."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dad6e3f2-65f0-4560-8563-bb4429868476",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c8b5aae-412d-4411-b2b1-9642c1706869",
   "metadata": {},
   "source": [
    "## Step 6: Now we are ready to Fine-Tune!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "92653b69-b208-4968-bbcf-92b3fff6475a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# If you have more than 1 GPU available to use, you can parallelize the fine-tuning process!\n",
    "dev_count = torch.cuda.device_count()\n",
    "if dev_count > 1:\n",
    "    model.is_parallelizable = True\n",
    "    model.model_parallel = True"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de2f0a67-4129-4df8-bb34-d01ac0ec4586",
   "metadata": {},
   "source": [
    "For the training job below, you may choose to use the estimator of your choice! "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "29a3ba4a-08f0-4c3a-b8f4-831893b43469",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Creating an S3 Bucket to log our training metrics to TensorBoard.\n",
    "bucket = \"mixtral-qlora-finetune-results\"\n",
    "log_bucket = f\"s3://{bucket}/qlora-finetuning\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9daa6765-4147-4b35-94df-364129715fab",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this command and check if a bucket with the name specified above exists in your account\n",
    "!aws s3 ls | grep $bucket"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b7aa3c1b-363e-4bf7-8f6a-0dd448cc95d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# pip install tensorboard\n",
    "# import tensorflowsdf\n",
    "# To learn more about the training arguments used, check out https://huggingface.co/docs/transformers/en/main_classes/trainer#transformers.TrainingArguments ",
    "\n",
    "\n",
    "import transformers\n",
    "\n",
    "tokenizer.pad_token = tokenizer.eos_token\n",
    "\n",
    "training_args = transformers.TrainingArguments(\n",
    "    per_device_train_batch_size=8,\n",
    "    per_device_eval_batch_size=8,\n",
    "    max_steps=1000,\n",
    "    output_dir=\"mixtral-outputs\",\n",
    "    logging_dir=log_bucket,\n",
    "    logging_steps=2,\n",
    "    learning_rate=2e-4,\n",
    "    fp16=True,\n",
    "    save_strategy=\"steps\",\n",
    "    save_steps=50,\n",
    "    eval_strategy=\"steps\",\n",
    "    eval_steps=50,\n",
    "    do_eval=True,\n",
    "    warmup_steps=5,\n",
    "    gradient_checkpointing=True,\n",
    "    gradient_accumulation_steps=4,\n",
    "    optim=\"paged_adamw_8bit\",\n",
    "    report_to=\"tensorboard\",\n",
    ")\n",
    "\n",
    "trainer = transformers.Trainer(\n",
    "    model=model,\n",
    "    args=training_args,\n",
    "    train_dataset=tokenized_train_dataset,\n",
    "    eval_dataset=tokenized_validation_dataset,\n",
    "    data_collator=transformers.DataCollatorForLanguageModeling(tokenizer, mlm=False),\n",
    ")\n",
    "\n",
    "model.config.use_cache = False    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "054dcfa1-7cda-4050-93db-ce91bef4fd63",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Start training\n",
    "trainer.train()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3e6e2b74-022c-4c4f-958a-397df6474db5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Returns the metrics from training\n",
    "trainer.evaluate()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2aeee342-9e4d-4a8d-9c3f-f1d1c65cd1f1",
   "metadata": {},
   "source": [
    "## Step 7: Compare!\n",
    "\n",
    "Let's compare the fine-tuned model to how the quantized out-of-the-box model performed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ca6239b5-0c8e-4781-a6d6-cce82c06141a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from peft import PeftModel\n",
    "\n",
    "ft_model = PeftModel.from_pretrained(base_model, \"mixtral-8x7b-finetuning-job/checkpoint-500\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d7bfa64f-8a91-4971-915b-575ea9de5a21",
   "metadata": {},
   "outputs": [],
   "source": [
    "eval_prompt = \"\"\"Given a target sentence construct the underlying meaning representation of the input sentence as a single function with attributes and attribute values.\n",
    "This function should describe the target string accurately and the function must be one of the following ['inform', 'request', 'give_opinion', 'confirm', 'verify_attribute', 'suggest', 'request_explanation', 'recommend', 'request_attribute'].\n",
    "The attributes must be one of the following: ['name', 'exp_release_date', 'release_year', 'developer', 'esrb', 'rating', 'genres', 'player_perspective', 'has_multiplayer', 'platforms', 'available_on_steam', 'has_linux_release', 'has_mac_release', 'specifier']\n",
    "\n",
    "### Target sentence:\n",
    "Earlier, you stated that you didn't have strong feelings about PlayStation's Little Big Adventure. Is your opinion true for all games which don't have multiplayer?\n",
    "\n",
    "### Meaning representation:\n",
    "\"\"\"\n",
    "\n",
    "device = \"cuda\"\n",
    "model_input = eval_tokenizer(eval_prompt, return_tensors=\"pt\").to(device)\n",
    "\n",
    "ft_model.eval()\n",
    "with torch.no_grad():\n",
    "    print(eval_tokenizer.decode(ft_model.generate(**model_input, max_new_tokens=100)[0], skip_special_tokens=True))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "747620b3-8796-4d27-a267-9ddd9c7907c0",
   "metadata": {},
   "source": [
    "This is what we had before fine-tuning:\n",
    "- **Actual meaning representation** (Ground Truth): verify_attribute(name[Little Big Adventure], rating[average], has_multiplayer[no], platforms[PlayStation]) <br>\n",
    "- **Mixtral's (old) output for meaning representation** (Model Inference): inform(name(Little Big Adventure), has_multiplayer(Little Big Adventure))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "745da113-f4ae-4c31-b428-32406dc85766",
   "metadata": {},
   "source": [
    "This is what we have after fine-tuning:\n",
    "\n",
    "- **Actual meaning representation** (Ground Truth): verify_attribute(name[Little Big Adventure], rating[average], has_multiplayer[no], platforms[PlayStation])\n",
    "- **Mixtral's (new) output for meaning representation** (Model Inference): verify_attribute(name[Little Big Adventure], rating[average], has_multiplayer[no], platforms[PlayStation])"
   ]
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 57,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.trn1.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 58,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1.32xlarge",
    "vcpuNum": 128
   },
   {
    "_defaultOrder": 59,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1n.32xlarge",
    "vcpuNum": 128
   }
  ],
  "instance_type": "ml.g5.48xlarge",
  "kernelspec": {
   "display_name": "Python 3 (Data Science 3.0)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/sagemaker-data-science-310-v1"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}