t81_559_class_08_4_kaggle_llm.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "83GJJF9fAgyP"
   },
   "source": [
    "[![Open In Kaggle](https://img.shields.io/badge/Open%20In-Kaggle-blue?logo=kaggle)](https://www.kaggle.com/code/jeffheaton/t81-559-class-08-4-kaggle-llm-2)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HL640ydsAgyQ"
   },
   "source": [
    "# T81-559: Applications of Generative Artificial Intelligence\n",
    "**Module 8: Kaggle**\n",
    "* Instructor: [Jeff Heaton](https://sites.wustl.edu/jeffheaton/), McKelvey School of Engineering, [Washington University in St. Louis](https://engineering.wustl.edu/Programs/Pages/default.aspx)\n",
    "* For more information visit the [class website](https://sites.wustl.edu/jeffheaton/t81-558/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "a4ih9V7vAgyR"
   },
   "source": [
    "# Module 8 Material\n",
    "\n",
    "* Part 8.1: Introduction to Kaggle [[Video]](https://www.youtube.com/watch?v=t0iz2zZ-jXU&ab_channel=JeffHeaton) [[Notebook]](t81_559_class_08_1_kaggle_intro.ipynb)\n",
    "* Part 8.2: Kaggle Notebooks [[Video]](https://www.youtube.com/watch?v=5Bv8rFm_cas&ab_channel=JeffHeaton) [[Notebook]](t81_559_class_08_2_kaggle_notebooks.ipynb)\n",
    "* Part 8.3: Small Large Language Models [[Video]](https://www.youtube.com/watch?v=1Hm337_vVCM&ab_channel=JeffHeaton) [[Notebook]](t81_559_class_08_3_small_llm.ipynb)\n",
    "* **Part 8.4: Accessing Small LLM from Kaggle** [[Video]](https://www.youtube.com/watch?v=o5PriYNQrqo&ab_channel=JeffHeaton) [[Notebook]](t81_559_class_08_4_kaggle_llm.ipynb)\n",
    "* Part 8.5: Current Semester's Kaggle [[Video]]() [[Notebook]](t81_559_class_08_5_kaggle_project.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "uU7OTe1DAgyR"
   },
   "source": [
    "# Kaggle Notebook Instructions\n",
    "\n",
    "This cell sets up this notebook for Kaggle."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "execution": {
     "iopub.execute_input": "2024-09-07T11:28:49.487298Z",
     "iopub.status.busy": "2024-09-07T11:28:49.486319Z",
     "iopub.status.idle": "2024-09-07T11:29:24.724231Z",
     "shell.execute_reply": "2024-09-07T11:29:24.722703Z",
     "shell.execute_reply.started": "2024-09-07T11:28:49.487239Z"
    },
    "id": "NOdFRzaXAgyS",
    "outputId": "2475bc8b-19b2-487a-916a-3667060e76cf"
   },
   "outputs": [],
   "source": [
    "# Install Keras 3 last. See https://keras.io/getting_started/ for more details.\n",
    "!pip install -q -U keras-nlp\n",
    "!pip install -q -U keras>=3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LFMTMsOWAgyS"
   },
   "source": [
    "# 8.4: Accessing Small LLM from Kaggle\n",
    "\n",
    "You can make use of LLMs that run entirely within Kaggle. As we previously saw, these models must run inside your notebook's VM, so you are limited to small-large language models. The models supported inside of Kaggle are well-defined, and Kaggle designed their notebooks to facilitate using these models. Many of these models will work with or without a GPU; however, CPU-only configuration will be very slow.\n",
    "\n",
    "Some of the models automatically supported by Kaggle include:\n",
    "\n",
    "Efficient Net\n",
    "FLUX 3.1\n",
    "Gemma\n",
    "Lamma 3.1\n",
    "YOLO-v8\n",
    "\n",
    "## The Gemma LLM\n",
    "\n",
    "The Gemma model family, developed by Google DeepMind, offers cutting-edge performance in two sizes: 2B and 7B parameters. These models integrate seamlessly with popular machine learning platforms, including Kaggle, allowing easy deployment and experimentation. Gemma's lightweight architecture ensures it can deliver top-tier results, even compared to larger models, making it ideal for developers looking to balance performance and efficiency. The pre-trained and instruction-tuned versions of Gemma provide flexibility for both general-purpose and fine-tuned applications while maintaining Google's high standards for safety and responsible AI usage.\n",
    "\n",
    "Gemma's compatibility with Kaggle extends to ready-to-use notebooks and integration with common tools like Hugging Face, TensorFlow, and PyTorch. This capability makes it a great option for users familiar with the Kaggle ecosystem who wish to experiment or fine-tune the models for specific tasks. The ability to deploy Gemma models on local devices and in cloud environments like Google Cloud ensures versatility for various applications, from research projects to commercial use. In our upcoming projects, we will focus on the 2B and 7B versions of the Gemma models to explore their capabilities and optimize them for our use cases.\n",
    "\n",
    "\n",
    "The following code makes use of the Gemma model. \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-09-07T11:29:32.412090Z",
     "iopub.status.busy": "2024-09-07T11:29:32.411601Z",
     "iopub.status.idle": "2024-09-07T11:31:37.633972Z",
     "shell.execute_reply": "2024-09-07T11:31:37.632745Z",
     "shell.execute_reply.started": "2024-09-07T11:29:32.412044Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "normalizer.cc(51) LOG(INFO) precompiled_charsmap is empty. use identity normalization.\n"
     ]
    }
   ],
   "source": [
    "import keras\n",
    "import keras_nlp\n",
    "import os\n",
    "\n",
    "os.environ[\"KERAS_BACKEND\"] = \"jax\"  # Or \"tensorflow\" or \"torch\".\n",
    "\n",
    "gemma_lm = keras_nlp.models.GemmaCausalLM.from_preset(\"gemma_2b_en\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have the model defined we can give it prompts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-09-07T11:51:58.010604Z",
     "iopub.status.busy": "2024-09-07T11:51:58.009769Z",
     "iopub.status.idle": "2024-09-07T11:52:45.597430Z",
     "shell.execute_reply": "2024-09-07T11:52:45.596280Z",
     "shell.execute_reply.started": "2024-09-07T11:51:58.010536Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: All log messages before absl::InitializeLog() is called are written to STDERR\n",
      "I0000 00:00:1725709931.817510     100 service.cc:145] XLA service 0x7bb0ce11d790 initialized for platform Host (this does not guarantee that XLA will be used). Devices:\n",
      "I0000 00:00:1725709931.818006     100 service.cc:153]   StreamExecutor device (0): Host, Default Version\n",
      "I0000 00:00:1725709932.181999     100 device_compiler.h:188] Compiled cluster using XLA!  This line is logged at most once for the lifetime of the process.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'What is the meaning of life?\\n\\nThe question is one of the most important questions in the world.\\n\\nIt’s the question that has been asked by philosophers, theologians, and scientists for centuries.\\n\\nAnd it’s the question that has been asked by people who are looking for answers to their own lives'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gemma_lm.generate(\"What is the meaning of life?\", max_length=64)"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "colab": {
   "collapsed_sections": [],
   "name": "Copy of t81_558_class_08_5_kaggle_project.ipynb",
   "provenance": []
  },
  "kaggle": {
   "accelerator": "none",
   "dataSources": [
    {
     "databundleVersionId": 7771674,
     "modelInstanceId": 5171,
     "sourceId": 11371,
     "sourceType": "modelInstanceVersion"
    }
   ],
   "dockerImageVersionId": 30761,
   "isGpuEnabled": false,
   "isInternetEnabled": true,
   "language": "python",
   "sourceType": "notebook"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}