01_Using_APIs.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/wandb/edu/blob/main/llm-apps-course/notebooks/01.%20Using_APIs.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
    "<!--- @wandbcode{llmapps-intro} -->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Understanding LLM APIs\n",
    "\n",
    "We will explore OpenAI models API to generate text.\n",
    "\n",
    "<!--- @wandbcode{llmapps-intro} -->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install google-generativeai weave -qqU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from getpass import getpass\n",
    "import wandb\n",
    "import google.generativeai as genai\n",
    "import weave"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "Paste your Google API key: \n",
      " ········\n"
     ]
    }
   ],
   "source": [
    "os.environ[\"GOOGLE_API_KEY\"] = getpass(\"Paste your Google API key: \\n\")\n",
    "GOOGLE_API_KEY = os.getenv(\"GOOGLE_API_KEY\", \"\")\n",
    "genai.configure(api_key=GOOGLE_API_KEY)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Weights & Biases (W&B) is a powerful platform for **experiment tracking, model visualization, and collaboration in machine learning**. It simplifies the process of:\n",
      "\n",
      "* **Tracking experiments:**  W&B lets you automatically log metrics, parameters, code, and other artifacts associated with your machine learning experiments. This provides a clear and organized history of your work, making it easy to compare different approaches and identify trends.\n",
      "\n",
      "* **Visualizing models:**  W&B offers interactive visualizations for model performance, loss curves, feature importance, and other insights. This allows you to gain a deeper understanding of your models and identify potential areas for improvement.\n",
      "\n",
      "* **Collaborating with teams:**  W&B facilitates collaboration by providing a shared workspace where team members can track experiments, view visualizations, and discuss results. This promotes transparency and helps teams iterate faster.\n",
      "\n",
      "**Key Features of W&B:**\n",
      "\n",
      "* **Experiment Tracking:**  Logs metrics, parameters, code, and artifacts.\n",
      "* **Model Visualization:**  Provides interactive plots for model performance, loss curves, and feature importance.\n",
      "* **Collaboration Tools:**  Shared workspace for team communication and discussion.\n",
      "* **Integrations:**  Seamless integration with popular ML frameworks (TensorFlow, PyTorch, Keras, etc.) and cloud platforms.\n",
      "* **API and CLI:**  Allows programmatic access and automation of experiment tracking and visualization.\n",
      "\n",
      "**Benefits of using W&B:**\n",
      "\n",
      "* **Improved efficiency:** Streamlines experiment tracking and analysis.\n",
      "* **Enhanced understanding:** Provides visualizations and insights into model performance.\n",
      "* **Better collaboration:** Enables seamless team communication and sharing of results.\n",
      "* **Faster iteration:** Reduces the time it takes to identify and fix issues.\n",
      "* **Increased reproducibility:**  Provides a clear and organized record of experiments.\n",
      "\n",
      "W&B is a valuable tool for researchers, data scientists, and machine learning engineers looking to improve the efficiency and effectiveness of their work. \n",
      "\n"
     ]
    }
   ],
   "source": [
    "model = genai.GenerativeModel(\"gemini-1.5-flash\")\n",
    "response = model.generate_content(\"Say something about Weights & Biases\")\n",
    "print(response.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
      "weave 0.51.8 requires openai>=1.0.0, but you have openai 0.28.0 which is incompatible.\u001b[0m\u001b[31m\n",
      "\u001b[0mNote: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install --upgrade openai==0.28 tiktoken wandb -qq"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import openai\n",
    "import tiktoken\n",
    "import wandb\n",
    "from pprint import pprint\n",
    "from getpass import getpass\n",
    "from wandb.integration.openai import autolog"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You will need an OpenAI API key to run this notebook. You can get one [here](https://platform.openai.com/account/api-keys)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "Paste your OpenAI key from: https://platform.openai.com/account/api-keys\n",
      " ········\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OpenAI API key configured\n"
     ]
    }
   ],
   "source": [
    "if os.getenv(\"OPENAI_API_KEY\") is None:\n",
    "  if any(['VSCODE' in x for x in os.environ.keys()]):\n",
    "    print('Please enter password in the VS Code prompt at the top of your VS Code window!')\n",
    "  os.environ[\"OPENAI_API_KEY\"] = getpass(\"Paste your OpenAI key from: https://platform.openai.com/account/api-keys\\n\")\n",
    "  openai.api_key = os.getenv(\"OPENAI_API_KEY\", \"\")\n",
    "\n",
    "assert os.getenv(\"OPENAI_API_KEY\", \"\").startswith(\"sk-\"), \"This doesn't look like a valid OpenAI API key\"\n",
    "print(\"OpenAI API key configured\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's enable W&B autologging to track our experiments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[34m\u001b[1mwandb\u001b[0m: Using wandb-core as the SDK backend. Please refer to https://wandb.me/wandb-core for more information.\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Currently logged in as: \u001b[33msupriyagdptl\u001b[0m. Use \u001b[1m`wandb login --relogin`\u001b[0m to force relogin\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "Tracking run with wandb version 0.18.1"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Run data is saved locally in <code>/Users/supriya/MY_HOME/Work/Weights and biases/Builiding_LLM_App/wandb/run-20241001_144757-4823hjlh</code>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Syncing run <strong><a href='https://wandb.ai/supriyagdptl/llmapps/runs/4823hjlh' target=\"_blank\">light-bird-9</a></strong> to <a href='https://wandb.ai/supriyagdptl/llmapps' target=\"_blank\">Weights & Biases</a> (<a href='https://wandb.me/run' target=\"_blank\">docs</a>)<br/>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       " View project at <a href='https://wandb.ai/supriyagdptl/llmapps' target=\"_blank\">https://wandb.ai/supriyagdptl/llmapps</a>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       " View run at <a href='https://wandb.ai/supriyagdptl/llmapps/runs/4823hjlh' target=\"_blank\">https://wandb.ai/supriyagdptl/llmapps/runs/4823hjlh</a>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "ename": "TypeError",
     "evalue": "__name__ must be set to a string object",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[3], line 2\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[38;5;66;03m# start logging to W&B\u001b[39;00m\n\u001b[0;32m----> 2\u001b[0m \u001b[43mautolog\u001b[49m\u001b[43m(\u001b[49m\u001b[43m{\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mproject\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mllmapps\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mjob_type\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mintroduction\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m}\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m~/opt/anaconda3/envs/arxiv3d/lib/python3.10/site-packages/wandb/sdk/integration_utils/auto_logging.py:183\u001b[0m, in \u001b[0;36mAutologAPI.__call__\u001b[0;34m(self, init)\u001b[0m\n\u001b[1;32m    181\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m__call__\u001b[39m(\u001b[38;5;28mself\u001b[39m, init: AutologInitArgs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m    182\u001b[0m \u001b[38;5;250m    \u001b[39m\u001b[38;5;124;03m\"\"\"Enable autologging.\"\"\"\u001b[39;00m\n\u001b[0;32m--> 183\u001b[0m     \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43menable\u001b[49m\u001b[43m(\u001b[49m\u001b[43minit\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43minit\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m~/opt/anaconda3/envs/arxiv3d/lib/python3.10/site-packages/wandb/sdk/integration_utils/auto_logging.py:220\u001b[0m, in \u001b[0;36mAutologAPI.enable\u001b[0;34m(self, init)\u001b[0m\n\u001b[1;32m    217\u001b[0m logger\u001b[38;5;241m.\u001b[39minfo(\u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mEnabling \u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_name\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m autologging.\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[1;32m    218\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_run_init(init\u001b[38;5;241m=\u001b[39minit)\n\u001b[0;32m--> 220\u001b[0m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_patch_api\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mpatch\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_run\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    222\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_telemetry_feature:\n\u001b[1;32m    223\u001b[0m     \u001b[38;5;28;01mwith\u001b[39;00m wb_telemetry\u001b[38;5;241m.\u001b[39mcontext(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_run) \u001b[38;5;28;01mas\u001b[39;00m tel:\n",
      "File \u001b[0;32m~/opt/anaconda3/envs/arxiv3d/lib/python3.10/site-packages/wandb/sdk/integration_utils/auto_logging.py:138\u001b[0m, in \u001b[0;36mPatchAPI.patch\u001b[0;34m(self, run)\u001b[0m\n\u001b[1;32m    133\u001b[0m     \u001b[38;5;28msetattr\u001b[39m(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mset_api, symbol_parts[\u001b[38;5;241m0\u001b[39m], method_factory(original))\n\u001b[1;32m    134\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m    135\u001b[0m     \u001b[38;5;28msetattr\u001b[39m(\n\u001b[1;32m    136\u001b[0m         functools\u001b[38;5;241m.\u001b[39mreduce(\u001b[38;5;28mgetattr\u001b[39m, symbol_parts[:\u001b[38;5;241m-\u001b[39m\u001b[38;5;241m1\u001b[39m], \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mset_api),\n\u001b[1;32m    137\u001b[0m         symbol_parts[\u001b[38;5;241m-\u001b[39m\u001b[38;5;241m1\u001b[39m],\n\u001b[0;32m--> 138\u001b[0m         \u001b[43mmethod_factory\u001b[49m\u001b[43m(\u001b[49m\u001b[43moriginal\u001b[49m\u001b[43m)\u001b[49m,\n\u001b[1;32m    139\u001b[0m     )\n",
      "File \u001b[0;32m~/opt/anaconda3/envs/arxiv3d/lib/python3.10/site-packages/wandb/sdk/integration_utils/auto_logging.py:127\u001b[0m, in \u001b[0;36mPatchAPI.patch.<locals>.method_factory\u001b[0;34m(original_method)\u001b[0m\n\u001b[1;32m    125\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m functools\u001b[38;5;241m.\u001b[39mwraps(original_method)(async_method)\n\u001b[1;32m    126\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m--> 127\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfunctools\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mwraps\u001b[49m\u001b[43m(\u001b[49m\u001b[43moriginal_method\u001b[49m\u001b[43m)\u001b[49m\u001b[43m(\u001b[49m\u001b[43msync_method\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m~/opt/anaconda3/envs/arxiv3d/lib/python3.10/functools.py:56\u001b[0m, in \u001b[0;36mupdate_wrapper\u001b[0;34m(wrapper, wrapped, assigned, updated)\u001b[0m\n\u001b[1;32m     54\u001b[0m         \u001b[38;5;28;01mpass\u001b[39;00m\n\u001b[1;32m     55\u001b[0m     \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m---> 56\u001b[0m         \u001b[38;5;28;43msetattr\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mwrapper\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mattr\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mvalue\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m     57\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m attr \u001b[38;5;129;01min\u001b[39;00m updated:\n\u001b[1;32m     58\u001b[0m     \u001b[38;5;28mgetattr\u001b[39m(wrapper, attr)\u001b[38;5;241m.\u001b[39mupdate(\u001b[38;5;28mgetattr\u001b[39m(wrapped, attr, {}))\n",
      "\u001b[0;31mTypeError\u001b[0m: __name__ must be set to a string object"
     ]
    }
   ],
   "source": [
    "# start logging to W&B\n",
    "autolog({\"project\":\"llmapps\", \"job_type\": \"introduction\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tokenization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[56730, 612, 12371, 2315, 374, 12738, 0]\n",
      "Weights & Biases is awesome!\n"
     ]
    }
   ],
   "source": [
    "encoding = tiktoken.encoding_for_model(\"davinci-002\")\n",
    "enc = encoding.encode(\"Weights & Biases is awesome!\")\n",
    "print(enc)\n",
    "print(encoding.decode(enc))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "we can decode the tokens one by one"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "56730\tWeights\n",
      "612\t &\n",
      "12371\t Bi\n",
      "2315\tases\n",
      "374\t is\n",
      "12738\t awesome\n",
      "0\t!\n"
     ]
    }
   ],
   "source": [
    "for token_id in enc:\n",
    "    print(f\"{token_id}\\t{encoding.decode([token_id])}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Note how the leading tokens contain spacing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sampling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's sample some text from the model. For this, let's create a wrapper function around the temperature parameters.\n",
    "Higher temperature will result in more random samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "ename": "APIRemovedInV1",
     "evalue": "\n\nYou tried to access openai.Completion, but this is no longer supported in openai>=1.0.0 - see the README at https://github.com/openai/openai-python for the API.\n\nYou can run `openai migrate` to automatically upgrade your codebase to use the 1.0.0 interface. \n\nAlternatively, you can pin your installation to the old version, e.g. `pip install openai==0.28`\n\nA detailed migration guide is available here: https://github.com/openai/openai-python/discussions/742\n",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mAPIRemovedInV1\u001b[0m                            Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[4], line 12\u001b[0m\n\u001b[1;32m      9\u001b[0m   \u001b[38;5;28;01mreturn\u001b[39;00m response\u001b[38;5;241m.\u001b[39mchoices[\u001b[38;5;241m0\u001b[39m]\u001b[38;5;241m.\u001b[39mtext\n\u001b[1;32m     11\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m temp \u001b[38;5;129;01min\u001b[39;00m [\u001b[38;5;241m0\u001b[39m]:\u001b[38;5;66;03m#, 0.5, 1, 1.5, 2]:\u001b[39;00m\n\u001b[0;32m---> 12\u001b[0m   pprint(\u001b[38;5;124mf\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mTEMP: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mtemp\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m, GENERATION: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[43mgenerate_with_temperature\u001b[49m\u001b[43m(\u001b[49m\u001b[43mtemp\u001b[49m\u001b[43m)\u001b[49m\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m'\u001b[39m)\n",
      "Cell \u001b[0;32mIn[4], line 3\u001b[0m, in \u001b[0;36mgenerate_with_temperature\u001b[0;34m(temp)\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mgenerate_with_temperature\u001b[39m(temp):\n\u001b[1;32m      2\u001b[0m   \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mGenerate text with a given temperature, higher temperature means more randomness\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m----> 3\u001b[0m   response \u001b[38;5;241m=\u001b[39m \u001b[43mopenai\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mCompletion\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcreate\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m      4\u001b[0m \u001b[43m    \u001b[49m\u001b[43mmodel\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mdavinci-002\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[1;32m      5\u001b[0m \u001b[43m    \u001b[49m\u001b[43mprompt\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mSay something about Weights & Biases\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[1;32m      6\u001b[0m \u001b[43m    \u001b[49m\u001b[43mmax_tokens\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;241;43m50\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[1;32m      7\u001b[0m \u001b[43m    \u001b[49m\u001b[43mtemperature\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtemp\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m      8\u001b[0m \u001b[43m  \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m      9\u001b[0m   \u001b[38;5;28;01mreturn\u001b[39;00m response\u001b[38;5;241m.\u001b[39mchoices[\u001b[38;5;241m0\u001b[39m]\u001b[38;5;241m.\u001b[39mtext\n",
      "File \u001b[0;32m~/opt/anaconda3/envs/arxiv3d/lib/python3.10/site-packages/openai/lib/_old_api.py:39\u001b[0m, in \u001b[0;36mAPIRemovedInV1Proxy.__call__\u001b[0;34m(self, *_args, **_kwargs)\u001b[0m\n\u001b[1;32m     38\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m__call__\u001b[39m(\u001b[38;5;28mself\u001b[39m, \u001b[38;5;241m*\u001b[39m_args: Any, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39m_kwargs: Any) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m Any:\n\u001b[0;32m---> 39\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m APIRemovedInV1(symbol\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_symbol)\n",
      "\u001b[0;31mAPIRemovedInV1\u001b[0m: \n\nYou tried to access openai.Completion, but this is no longer supported in openai>=1.0.0 - see the README at https://github.com/openai/openai-python for the API.\n\nYou can run `openai migrate` to automatically upgrade your codebase to use the 1.0.0 interface. \n\nAlternatively, you can pin your installation to the old version, e.g. `pip install openai==0.28`\n\nA detailed migration guide is available here: https://github.com/openai/openai-python/discussions/742\n"
     ]
    }
   ],
   "source": [
    "def generate_with_temperature(temp):\n",
    "  \"Generate text with a given temperature, higher temperature means more randomness\"\n",
    "  response = openai.Completion.create(\n",
    "    model=\"davinci-002\",\n",
    "    prompt=\"Say something about Weights & Biases\",\n",
    "    max_tokens=50,\n",
    "    temperature=temp,\n",
    "  )\n",
    "  return response.choices[0].text\n",
    "\n",
    "for temp in [0]:#, 0.5, 1, 1.5, 2]:\n",
    "  pprint(f'TEMP: {temp}, GENERATION: {generate_with_temperature(temp)}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also use the [`top_p` parameter](https://platform.openai.com/docs/api-reference/completions/create#completions/create-top_p) to control the diversity of the generated text. This parameter controls the cumulative probability of the next token. For example, if `top_p=0.9`, the model will pick the next token from the top 90% most likely tokens. The higher the `top_p` the more likely the model will pick a token that it hasn't seen before. You should only use one of `temperature` or `top_p` at a given time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "invalid syntax (2463506640.py, line 1)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;36m  Cell \u001b[0;32mIn[19], line 1\u001b[0;36m\u001b[0m\n\u001b[0;31m    openai migrate\u001b[0m\n\u001b[0m           ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
     ]
    }
   ],
   "source": [
    "openai migrate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_with_topp(topp):\n",
    "  \"Generate text with a given top-p, higher top-p means more randomness\"\n",
    "  response = openai.Completion.create(\n",
    "    model=\"text-davinci-003\",\n",
    "    prompt=\"Say something about Weights & Biases\",\n",
    "    max_tokens=50,\n",
    "    top_p=topp,\n",
    "    )\n",
    "  return response.choices[0].text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for topp in [0.01, 0.1, 0.5, 1]:\n",
    "  pprint(f'TOP_P: {topp}, GENERATION: {generate_with_topp(topp)}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Chat API"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's switch to chat mode and see how the model responds to our messages. We have some control over the model's response by passing a `system-role`, here we can steer to model to adhere to a certain behaviour.\n",
    "\n",
    "> We are using `gpt-3.5-turbo`, this model is faster and cheaper than `davinci-003`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "ename": "APIRemovedInV1",
     "evalue": "\n\nYou tried to access openai.ChatCompletion, but this is no longer supported in openai>=1.0.0 - see the README at https://github.com/openai/openai-python for the API.\n\nYou can run `openai migrate` to automatically upgrade your codebase to use the 1.0.0 interface. \n\nAlternatively, you can pin your installation to the old version, e.g. `pip install openai==0.28`\n\nA detailed migration guide is available here: https://github.com/openai/openai-python/discussions/742\n",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mAPIRemovedInV1\u001b[0m                            Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[5], line 2\u001b[0m\n\u001b[1;32m      1\u001b[0m MODEL \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mgpt-3.5-turbo\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m----> 2\u001b[0m response \u001b[38;5;241m=\u001b[39m \u001b[43mopenai\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mChatCompletion\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcreate\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m      3\u001b[0m \u001b[43m    \u001b[49m\u001b[43mmodel\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mMODEL\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m      4\u001b[0m \u001b[43m    \u001b[49m\u001b[43mmessages\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43m[\u001b[49m\n\u001b[1;32m      5\u001b[0m \u001b[43m        \u001b[49m\u001b[43m{\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mrole\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43msystem\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mcontent\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mYou are a helpful assistant.\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m}\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m      6\u001b[0m \u001b[43m        \u001b[49m\u001b[43m{\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mrole\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43muser\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mcontent\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mSay something about Weights & Biases\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m}\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m      7\u001b[0m \u001b[43m    \u001b[49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m      8\u001b[0m \u001b[43m    \u001b[49m\u001b[43mtemperature\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;241;43m0\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[1;32m      9\u001b[0m \u001b[43m)\u001b[49m\n\u001b[1;32m     11\u001b[0m response\n",
      "File \u001b[0;32m~/opt/anaconda3/envs/arxiv3d/lib/python3.10/site-packages/openai/lib/_old_api.py:39\u001b[0m, in \u001b[0;36mAPIRemovedInV1Proxy.__call__\u001b[0;34m(self, *_args, **_kwargs)\u001b[0m\n\u001b[1;32m     38\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m__call__\u001b[39m(\u001b[38;5;28mself\u001b[39m, \u001b[38;5;241m*\u001b[39m_args: Any, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39m_kwargs: Any) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m Any:\n\u001b[0;32m---> 39\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m APIRemovedInV1(symbol\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_symbol)\n",
      "\u001b[0;31mAPIRemovedInV1\u001b[0m: \n\nYou tried to access openai.ChatCompletion, but this is no longer supported in openai>=1.0.0 - see the README at https://github.com/openai/openai-python for the API.\n\nYou can run `openai migrate` to automatically upgrade your codebase to use the 1.0.0 interface. \n\nAlternatively, you can pin your installation to the old version, e.g. `pip install openai==0.28`\n\nA detailed migration guide is available here: https://github.com/openai/openai-python/discussions/742\n"
     ]
    }
   ],
   "source": [
    "MODEL = \"gpt-3.5-turbo\"\n",
    "response = openai.ChatCompletion.create(\n",
    "    model=MODEL,\n",
    "    messages=[\n",
    "        {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
    "        {\"role\": \"user\", \"content\": \"Say something about Weights & Biases\"},\n",
    "    ],\n",
    "    temperature=0,\n",
    ")\n",
    "\n",
    "response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see above, the response is a JSON object with relevant information about the request."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pprint(response.choices[0].message.content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "wandb.finish()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "include_colab_link": true,
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}