Llama_2_13b_chat_and_llama_cpp_python.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/amitsangani/llama/blob/main/Llama_2_13b_chat_and_llama_cpp_python.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ihaZyg2aYQpH"
      },
      "source": [
        "# llama-cpp-python\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "* `llama.cpp`'s objective is to run the LLaMA model with 4-bit integer quantization on MacBook.\n",
        "* llama-cpp-python package provides simple Python bindings for llama.cpp library.\n",
        "* `GGML` is a C library for machine learning, facilitates the distribution of large language models (LLMs). It utilizes quantization to enable efficient LLM execution on consumer hardware."
      ],
      "metadata": {
        "id": "_AE_cDXT3l37"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6rkl4zg7nLhJ"
      },
      "source": [
        "If you want to use only the CPU, you can replace the content of the cell below with the following lines. The library works the same with a CPU, but the inference can take about three times longer compared to using it on a GPU.\n",
        "```\n",
        "# CPU llama-cpp-python\n",
        "!pip install llama-cpp-python\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rkBmY3vQvRSw"
      },
      "outputs": [],
      "source": [
        "# GPU llama-cpp-python\n",
        "!CMAKE_ARGS=\"-DLLAMA_CUBLAS=on\" FORCE_CMAKE=1 pip install llama-cpp-python --force-reinstall --upgrade --no-cache-dir --verbose"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "b_nBHTYSoIWV"
      },
      "outputs": [],
      "source": [
        "# For download the models\n",
        "!pip install huggingface_hub"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "R11KqY7lW0yv"
      },
      "source": [
        "# Select the \"Llama 2 13B-chat\" model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CzJkCoRRbICP"
      },
      "source": [
        "\n",
        "In Colab with T4 GPU, we can run models of up to 20B of parameters with all optimizations, but this may degrade the quality of the model's inference. In this case, we will use a [Llama 2 13B-chat](https://huggingface.co/meta-llama/Llama-2-13b-chat)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PdmNwEqjRcPY"
      },
      "source": [
        "# Model quantization"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tx-5Fgx80UXj"
      },
      "source": [
        "We can quantize the model using this library, but for practical purposes, it is\n",
        "better to use pre-quantized models. The resulting model is only compatible with libraries that support GGML.\n",
        "\n",
        "\n",
        "```\n",
        "# obtain the original LLaMA model weights and place them in ./models\n",
        "ls ./models\n",
        "65B 30B 13B 7B tokenizer_checklist.chk tokenizer.model\n",
        "\n",
        "# install Python dependencies\n",
        "python3 -m pip install -r requirements.txt\n",
        "\n",
        "# convert the 7B model to ggml FP16 format\n",
        "python3 convert.py models/7B/\n",
        "\n",
        "# quantize the model to 4-bits (using q4_0 method)\n",
        "./quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_0.bin q4_0\n",
        "\n",
        "# run the inference\n",
        "./main -m ./models/7B/ggml-model-q4_0.bin -n 128\n",
        "```\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GSeedwAFSay9"
      },
      "source": [
        "#  Quantized Models from the Hugging Face Community"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4QwfjGFYRM4g"
      },
      "source": [
        "The Hugging Face community provides quantized models, which allow us to efficiently and effectively utilize the model on the T4 GPU. It is important to consult reliable sources before using any model.\n",
        "\n",
        "There are several variations available, but the ones that interest us are based on the GGLM library.\n",
        "\n",
        "We can see the different variations that Llama-2-13B-GGML has [here](https://huggingface.co/models?search=llama%202%20ggml).\n",
        "\n",
        "\n",
        "\n",
        "In this case, we will use the model called [Llama-2-13B-chat-GGML](https://huggingface.co/TheBloke/Llama-2-13B-chat-GGML)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "oI-kXwg5bHF-"
      },
      "outputs": [],
      "source": [
        "model_name_or_path = \"TheBloke/Llama-2-13B-chat-GGML\"\n",
        "model_basename = \"llama-2-13b-chat.ggmlv3.q5_1.bin\" # the model is in bin format"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8Td05XSuiWdI"
      },
      "source": [
        "First, we download the model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 49,
          "referenced_widgets": [
            "d79e2e6cc61c4ee7a4e92042cd466997",
            "5f74de32c170425a973e90e7ea4c02e5",
            "40f647d365154ae0ab6a72046123f727",
            "d2eda363293c4314b7081de73bb68830",
            "ecd6f520640a482987726db1627a97bb",
            "9c2a44a202d94c09822f6879aedaa6c0",
            "506779972af0411f9958513190892e08",
            "eab2927024794068ab328221757eeb4e",
            "10a1c6c7fb0c4362bec148084e193dce",
            "65dd6de239cc497c8626f8205233a4f4",
            "5cd8b149457e4afdad511349e77ce37c"
          ]
        },
        "id": "cBEJr-G-2ht4",
        "outputId": "ec8e1ca5-85a7-41f8-9ba2-b970ad21c398"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)chat.ggmlv3.q5_1.bin:   0%|          | 0.00/9.76G [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "d79e2e6cc61c4ee7a4e92042cd466997"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "from huggingface_hub import hf_hub_download\n",
        "\n",
        "model_path = hf_hub_download(repo_id=model_name_or_path, filename=model_basename)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "max1jwxvCSbm"
      },
      "source": [
        "# Inference with llama-cpp-python"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8TOfnZpj394g"
      },
      "source": [
        "Loading the model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "oY1bItJu4Zfv"
      },
      "outputs": [],
      "source": [
        "# GPU\n",
        "from llama_cpp import Llama\n",
        "lcpp_llm = None\n",
        "lcpp_llm = Llama(\n",
        "    model_path=model_path,\n",
        "    n_threads=2, # CPU cores\n",
        "    n_batch=512, # Should be between 1 and n_ctx, consider the amount of VRAM in your GPU.\n",
        "    n_gpu_layers=32 # Change this value based on your model and your GPU VRAM pool.\n",
        "    )"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "For run in CPU\n",
        "```\n",
        "# CPU\n",
        "from llama_cpp import Llama\n",
        "\n",
        "lcpp_llm = Llama(\n",
        "    model_path=model_path,\n",
        "    n_threads=2, # CPU cores\n",
        "    )\n",
        "```\n",
        "\n"
      ],
      "metadata": {
        "id": "UdZnPtB8-Bhx"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BeH6eWiKuaxW"
      },
      "outputs": [],
      "source": [
        "# See the number of layers in GPU\n",
        "lcpp_llm.params.n_gpu_layers"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qLEEOufGVlID"
      },
      "source": [
        "We will use this prompt."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "-NzVIlMCVoVD"
      },
      "outputs": [],
      "source": [
        "prompt = \"What is the meaning of life?\"\n",
        "prompt_template=f'''SYSTEM: You are a helpful, respectful and honest assistant. Always answer as helpfully.\n",
        "\n",
        "USER: {prompt}\n",
        "\n",
        "ASSISTANT:\n",
        "'''"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qaFQjPhcFgfN"
      },
      "source": [
        "Generating response\n",
        "\n",
        "If you only use CPU, the response can take a long time. You can reduce the max_tokens to get a faster response."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "R76uxL293jTc",
        "outputId": "49bcfbdc-89a2-4325-fc5e-7b19a9cd315f"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "SYSTEM: You are a helpful, respectful and honest assistant. Always answer as helpfully.\n",
            "\n",
            "USER: What is the meaning of life?\n",
            "\n",
            "ASSISTANT:\n",
            "\n",
            "The meaning of life is a question that has puzzled philosophers and theologians for centuries, and there is no one definitive answer. However, here are some possible perspectives on what might give our lives meaning:\n",
            "\n",
            "1. Personal fulfillment: Some people believe that the purpose of life is to find personal fulfillment and happiness. This could involve pursuing our passions and interests, cultivating strong relationships with others, and taking care of ourselves physically and emotionally.\n",
            "2. Religious or spiritual beliefs: Many religious and spiritual traditions offer a sense of meaning and purpose that is derived from a higher power or divine being. For example, some believe that our lives have a specific purpose or mission that is ordained by God or the universe.\n",
            "3. Contribution to society: Some people find meaning in contributing to the greater good through their work, volunteering, or other forms of service. This could involve making a positive impact on others, improving our communities, or working towards social justice.\n",
            "4. Legacy: Leaving behind a lasting legacy that benefits future generations can give some people a sense of meaning and purpose in life. This could involve creating art, literature, music,\n"
          ]
        }
      ],
      "source": [
        "response = lcpp_llm(\n",
        "    prompt=prompt_template,\n",
        "    max_tokens=256,\n",
        "    temperature=0.5,\n",
        "    top_p=0.95,\n",
        "    repeat_penalty=1.2,\n",
        "    top_k=50,\n",
        "    echo=True\n",
        "    )\n",
        "\n",
        "print(response[\"choices\"][0][\"text\"])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "no531n827gI9"
      },
      "source": [
        "# Inference with langchain\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zG2uFQFl7jrq"
      },
      "outputs": [],
      "source": [
        "!pip -q install langchain"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-Kl8UYvXHYIt"
      },
      "source": [
        "We can use the model that we loaded earlier. However, for illustrative purposes, we will load one from the 'langchain' library. Due to vRAM limitations, before running these cells, you need to delete the previous model from memory."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "NWaQuYH0AT1z"
      },
      "outputs": [],
      "source": [
        "lcpp_llm.reset()\n",
        "lcpp_llm.set_cache(None)\n",
        "lcpp_llm = None\n",
        "del lcpp_llm"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "Prompt for the model."
      ],
      "metadata": {
        "id": "scri08pVDp8U"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "N5ijTbW6_dI_"
      },
      "outputs": [],
      "source": [
        "from langchain.llms import LlamaCpp\n",
        "from langchain import PromptTemplate, LLMChain\n",
        "from langchain.callbacks.manager import CallbackManager\n",
        "from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler\n",
        "from huggingface_hub import hf_hub_download\n",
        "\n",
        "template = \"\"\"USER: {question}\n",
        "ASSISTANT: Let's work this out in a step by step way to be sure we have the right answer.\"\"\"\n",
        "prompt = PromptTemplate(template=template, input_variables=[\"question\"])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hx06Ea1DH8ky"
      },
      "source": [
        "Stream tokens"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "iVIXxNoIHOe_"
      },
      "outputs": [],
      "source": [
        "# Callbacks support token-wise streaming\n",
        "callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])\n",
        "# Verbose is required to pass to the callback manager"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G2DHWIr-IEDE"
      },
      "source": [
        "Load the model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "iipsXBGhHNjf"
      },
      "outputs": [],
      "source": [
        "n_gpu_layers = 40  # Change this value based on your model and your GPU VRAM pool.\n",
        "n_batch = 512  # Should be between 1 and n_ctx, consider the amount of VRAM in your GPU.\n",
        "\n",
        "# Loading model,\n",
        "llm = LlamaCpp(\n",
        "    model_path=model_path,\n",
        "    max_tokens=1024,\n",
        "    n_gpu_layers=n_gpu_layers,\n",
        "    n_batch=n_batch,\n",
        "    callback_manager=callback_manager,\n",
        "    verbose=True,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JZ8kbRz2F9vi"
      },
      "source": [
        "Generating response"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Lw4MaS4WBlR-"
      },
      "outputs": [],
      "source": [
        "llm_chain = LLMChain(prompt=prompt, llm=llm)\n",
        "\n",
        "question = \"What is the meaning of life?\"\n",
        "\n",
        "llm_chain.run(question)"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true,
      "gpuType": "T4",
      "collapsed_sections": [
        "R11KqY7lW0yv",
        "PdmNwEqjRcPY",
        "GSeedwAFSay9"
      ],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "d79e2e6cc61c4ee7a4e92042cd466997": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_5f74de32c170425a973e90e7ea4c02e5",
              "IPY_MODEL_40f647d365154ae0ab6a72046123f727",
              "IPY_MODEL_d2eda363293c4314b7081de73bb68830"
            ],
            "layout": "IPY_MODEL_ecd6f520640a482987726db1627a97bb"
          }
        },
        "5f74de32c170425a973e90e7ea4c02e5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9c2a44a202d94c09822f6879aedaa6c0",
            "placeholder": "​",
            "style": "IPY_MODEL_506779972af0411f9958513190892e08",
            "value": "Downloading (…)chat.ggmlv3.q5_1.bin: 100%"
          }
        },
        "40f647d365154ae0ab6a72046123f727": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_eab2927024794068ab328221757eeb4e",
            "max": 9763701888,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_10a1c6c7fb0c4362bec148084e193dce",
            "value": 9763701888
          }
        },
        "d2eda363293c4314b7081de73bb68830": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_65dd6de239cc497c8626f8205233a4f4",
            "placeholder": "​",
            "style": "IPY_MODEL_5cd8b149457e4afdad511349e77ce37c",
            "value": " 9.76G/9.76G [01:46&lt;00:00, 125MB/s]"
          }
        },
        "ecd6f520640a482987726db1627a97bb": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9c2a44a202d94c09822f6879aedaa6c0": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "506779972af0411f9958513190892e08": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "eab2927024794068ab328221757eeb4e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "10a1c6c7fb0c4362bec148084e193dce": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "65dd6de239cc497c8626f8205233a4f4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5cd8b149457e4afdad511349e77ce37c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    },
    "accelerator": "GPU"
  },
  "nbformat": 4,
  "nbformat_minor": 0
}