t81_559_class_10_4_chat.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kxu1Gfhx1pHg"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/jeffheaton/app_generative_ai/blob/main/t81_559_class_10_4_chat.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZbNbAV281pHh"
   },
   "source": [
    "# T81-559: Applications of Generative Artificial Intelligence\n",
    "**Module 10: StreamLit**\n",
    "* Instructor: [Jeff Heaton](https://sites.wustl.edu/jeffheaton/), McKelvey School of Engineering, [Washington University in St. Louis](https://engineering.wustl.edu/Programs/Pages/default.aspx)\n",
    "* For more information visit the [class website](https://sites.wustl.edu/jeffheaton/t81-558/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HwnvSYEQ1pHi"
   },
   "source": [
    "# Module 10 Material\n",
    "\n",
    "Module 10: StreamLit\n",
    "\n",
    "* Part 10.1: Running StreamLit in Google Colab [[Video]]() [[Notebook]](t81_559_class_10_1_streamlit.ipynb)\n",
    "* Part 10.2: StreamLit Introduction [[Video]]() [[Notebook]](t81_559_class_10_2_streamlit_intro.ipynb)\n",
    "* Part 10.3: Understanding Streamlit State [[Video]]() [[Notebook]](t81_559_class_10_3_streamlit_state.ipynb)\n",
    "* **Part 10.4: Creating a Chat Application** [[Video]]() [[Notebook]](t81_559_class_10_4_chat.ipynb)\n",
    "* Part 10.5: MultiModal Chat Application [[Video]]() [[Notebook]](t81_559_class_10_5_chat_multimodal.ipynb)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XLFov09h18yC"
   },
   "source": [
    "# Google CoLab Instructions\n",
    "\n",
    "The following code ensures that Google CoLab is running and maps Google Drive if needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "AWGARRT92DrA",
    "outputId": "0c8c3445-de95-41b1-af0b-72859b164a09"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: using Google CoLab\n",
      "Collecting langchain\n",
      "  Downloading langchain-0.3.0-py3-none-any.whl.metadata (7.1 kB)\n",
      "Collecting langchain_openai\n",
      "  Downloading langchain_openai-0.2.0-py3-none-any.whl.metadata (2.6 kB)\n",
      "Collecting openai\n",
      "  Downloading openai-1.45.0-py3-none-any.whl.metadata (22 kB)\n",
      "Collecting streamlit\n",
      "  Downloading streamlit-1.38.0-py2.py3-none-any.whl.metadata (8.5 kB)\n",
      "Requirement already satisfied: PyYAML>=5.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (6.0.2)\n",
      "Requirement already satisfied: SQLAlchemy<3,>=1.4 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.0.34)\n",
      "Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (3.10.5)\n",
      "Requirement already satisfied: async-timeout<5.0.0,>=4.0.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (4.0.3)\n",
      "Collecting langchain-core<0.4.0,>=0.3.0 (from langchain)\n",
      "  Downloading langchain_core-0.3.0-py3-none-any.whl.metadata (6.2 kB)\n",
      "Collecting langchain-text-splitters<0.4.0,>=0.3.0 (from langchain)\n",
      "  Downloading langchain_text_splitters-0.3.0-py3-none-any.whl.metadata (2.3 kB)\n",
      "Collecting langsmith<0.2.0,>=0.1.17 (from langchain)\n",
      "  Downloading langsmith-0.1.120-py3-none-any.whl.metadata (13 kB)\n",
      "Requirement already satisfied: numpy<2,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (1.26.4)\n",
      "Requirement already satisfied: pydantic<3.0.0,>=2.7.4 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.9.1)\n",
      "Requirement already satisfied: requests<3,>=2 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.32.3)\n",
      "Collecting tenacity!=8.4.0,<9.0.0,>=8.1.0 (from langchain)\n",
      "  Downloading tenacity-8.5.0-py3-none-any.whl.metadata (1.2 kB)\n",
      "Collecting tiktoken<1,>=0.7 (from langchain_openai)\n",
      "  Downloading tiktoken-0.7.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)\n",
      "Requirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from openai) (3.7.1)\n",
      "Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai) (1.7.0)\n",
      "Collecting httpx<1,>=0.23.0 (from openai)\n",
      "  Downloading httpx-0.27.2-py3-none-any.whl.metadata (7.1 kB)\n",
      "Collecting jiter<1,>=0.4.0 (from openai)\n",
      "  Downloading jiter-0.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.6 kB)\n",
      "Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from openai) (1.3.1)\n",
      "Requirement already satisfied: tqdm>4 in /usr/local/lib/python3.10/dist-packages (from openai) (4.66.5)\n",
      "Requirement already satisfied: typing-extensions<5,>=4.11 in /usr/local/lib/python3.10/dist-packages (from openai) (4.12.2)\n",
      "Requirement already satisfied: altair<6,>=4.0 in /usr/local/lib/python3.10/dist-packages (from streamlit) (4.2.2)\n",
      "Requirement already satisfied: blinker<2,>=1.0.0 in /usr/lib/python3/dist-packages (from streamlit) (1.4)\n",
      "Requirement already satisfied: cachetools<6,>=4.0 in /usr/local/lib/python3.10/dist-packages (from streamlit) (5.5.0)\n",
      "Requirement already satisfied: click<9,>=7.0 in /usr/local/lib/python3.10/dist-packages (from streamlit) (8.1.7)\n",
      "Requirement already satisfied: packaging<25,>=20 in /usr/local/lib/python3.10/dist-packages (from streamlit) (24.1)\n",
      "Requirement already satisfied: pandas<3,>=1.3.0 in /usr/local/lib/python3.10/dist-packages (from streamlit) (2.1.4)\n",
      "Requirement already satisfied: pillow<11,>=7.1.0 in /usr/local/lib/python3.10/dist-packages (from streamlit) (9.4.0)\n",
      "Requirement already satisfied: protobuf<6,>=3.20 in /usr/local/lib/python3.10/dist-packages (from streamlit) (3.20.3)\n",
      "Requirement already satisfied: pyarrow>=7.0 in /usr/local/lib/python3.10/dist-packages (from streamlit) (14.0.2)\n",
      "Requirement already satisfied: rich<14,>=10.14.0 in /usr/local/lib/python3.10/dist-packages (from streamlit) (13.8.1)\n",
      "Requirement already satisfied: toml<2,>=0.10.1 in /usr/local/lib/python3.10/dist-packages (from streamlit) (0.10.2)\n",
      "Collecting gitpython!=3.1.19,<4,>=3.0.7 (from streamlit)\n",
      "  Downloading GitPython-3.1.43-py3-none-any.whl.metadata (13 kB)\n",
      "Collecting pydeck<1,>=0.8.0b4 (from streamlit)\n",
      "  Downloading pydeck-0.9.1-py2.py3-none-any.whl.metadata (4.1 kB)\n",
      "Requirement already satisfied: tornado<7,>=6.0.3 in /usr/local/lib/python3.10/dist-packages (from streamlit) (6.3.3)\n",
      "Collecting watchdog<5,>=2.1.5 (from streamlit)\n",
      "  Downloading watchdog-4.0.2-py3-none-manylinux2014_x86_64.whl.metadata (38 kB)\n",
      "Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (2.4.0)\n",
      "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.3.1)\n",
      "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (24.2.0)\n",
      "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.4.1)\n",
      "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (6.1.0)\n",
      "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.11.1)\n",
      "Requirement already satisfied: entrypoints in /usr/local/lib/python3.10/dist-packages (from altair<6,>=4.0->streamlit) (0.4)\n",
      "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from altair<6,>=4.0->streamlit) (3.1.4)\n",
      "Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.10/dist-packages (from altair<6,>=4.0->streamlit) (4.23.0)\n",
      "Requirement already satisfied: toolz in /usr/local/lib/python3.10/dist-packages (from altair<6,>=4.0->streamlit) (0.12.1)\n",
      "Requirement already satisfied: idna>=2.8 in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->openai) (3.8)\n",
      "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->openai) (1.2.2)\n",
      "Collecting gitdb<5,>=4.0.1 (from gitpython!=3.1.19,<4,>=3.0.7->streamlit)\n",
      "  Downloading gitdb-4.0.11-py3-none-any.whl.metadata (1.2 kB)\n",
      "Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.23.0->openai) (2024.8.30)\n",
      "Collecting httpcore==1.* (from httpx<1,>=0.23.0->openai)\n",
      "  Downloading httpcore-1.0.5-py3-none-any.whl.metadata (20 kB)\n",
      "Collecting h11<0.15,>=0.13 (from httpcore==1.*->httpx<1,>=0.23.0->openai)\n",
      "  Downloading h11-0.14.0-py3-none-any.whl.metadata (8.2 kB)\n",
      "Collecting jsonpatch<2.0,>=1.33 (from langchain-core<0.4.0,>=0.3.0->langchain)\n",
      "  Downloading jsonpatch-1.33-py2.py3-none-any.whl.metadata (3.0 kB)\n",
      "Collecting orjson<4.0.0,>=3.9.14 (from langsmith<0.2.0,>=0.1.17->langchain)\n",
      "  Downloading orjson-3.10.7-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (50 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.4/50.4 kB\u001b[0m \u001b[31m947.7 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas<3,>=1.3.0->streamlit) (2.8.2)\n",
      "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas<3,>=1.3.0->streamlit) (2024.2)\n",
      "Requirement already satisfied: tzdata>=2022.1 in /usr/local/lib/python3.10/dist-packages (from pandas<3,>=1.3.0->streamlit) (2024.1)\n",
      "Requirement already satisfied: annotated-types>=0.6.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3.0.0,>=2.7.4->langchain) (0.7.0)\n",
      "Requirement already satisfied: pydantic-core==2.23.3 in /usr/local/lib/python3.10/dist-packages (from pydantic<3.0.0,>=2.7.4->langchain) (2.23.3)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.3.2)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2.0.7)\n",
      "Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from rich<14,>=10.14.0->streamlit) (3.0.0)\n",
      "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.10/dist-packages (from rich<14,>=10.14.0->streamlit) (2.16.1)\n",
      "Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy<3,>=1.4->langchain) (3.1.0)\n",
      "Requirement already satisfied: regex>=2022.1.18 in /usr/local/lib/python3.10/dist-packages (from tiktoken<1,>=0.7->langchain_openai) (2024.5.15)\n",
      "Collecting smmap<6,>=3.0.1 (from gitdb<5,>=4.0.1->gitpython!=3.1.19,<4,>=3.0.7->streamlit)\n",
      "  Downloading smmap-5.0.1-py3-none-any.whl.metadata (4.3 kB)\n",
      "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->altair<6,>=4.0->streamlit) (2.1.5)\n",
      "Collecting jsonpointer>=1.9 (from jsonpatch<2.0,>=1.33->langchain-core<0.4.0,>=0.3.0->langchain)\n",
      "  Downloading jsonpointer-3.0.0-py2.py3-none-any.whl.metadata (2.3 kB)\n",
      "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6,>=4.0->streamlit) (2023.12.1)\n",
      "Requirement already satisfied: referencing>=0.28.4 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6,>=4.0->streamlit) (0.35.1)\n",
      "Requirement already satisfied: rpds-py>=0.7.1 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6,>=4.0->streamlit) (0.20.0)\n",
      "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py>=2.2.0->rich<14,>=10.14.0->streamlit) (0.1.2)\n",
      "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->pandas<3,>=1.3.0->streamlit) (1.16.0)\n",
      "Downloading langchain-0.3.0-py3-none-any.whl (1.0 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.0/1.0 MB\u001b[0m \u001b[31m16.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading langchain_openai-0.2.0-py3-none-any.whl (51 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m51.5/51.5 kB\u001b[0m \u001b[31m2.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading openai-1.45.0-py3-none-any.whl (374 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m374.1/374.1 kB\u001b[0m \u001b[31m19.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading streamlit-1.38.0-py2.py3-none-any.whl (8.7 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m8.7/8.7 MB\u001b[0m \u001b[31m65.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading GitPython-3.1.43-py3-none-any.whl (207 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m207.3/207.3 kB\u001b[0m \u001b[31m11.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading httpx-0.27.2-py3-none-any.whl (76 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m76.4/76.4 kB\u001b[0m \u001b[31m4.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading httpcore-1.0.5-py3-none-any.whl (77 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.9/77.9 kB\u001b[0m \u001b[31m5.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading jiter-0.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (318 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m318.9/318.9 kB\u001b[0m \u001b[31m16.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading langchain_core-0.3.0-py3-none-any.whl (405 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m405.1/405.1 kB\u001b[0m \u001b[31m23.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading langchain_text_splitters-0.3.0-py3-none-any.whl (25 kB)\n",
      "Downloading langsmith-0.1.120-py3-none-any.whl (289 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m289.8/289.8 kB\u001b[0m \u001b[31m16.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading pydeck-0.9.1-py2.py3-none-any.whl (6.9 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.9/6.9 MB\u001b[0m \u001b[31m65.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading tenacity-8.5.0-py3-none-any.whl (28 kB)\n",
      "Downloading tiktoken-0.7.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.1/1.1 MB\u001b[0m \u001b[31m35.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading watchdog-4.0.2-py3-none-manylinux2014_x86_64.whl (82 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m82.9/82.9 kB\u001b[0m \u001b[31m4.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading gitdb-4.0.11-py3-none-any.whl (62 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m62.7/62.7 kB\u001b[0m \u001b[31m3.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading jsonpatch-1.33-py2.py3-none-any.whl (12 kB)\n",
      "Downloading orjson-3.10.7-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (141 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m141.9/141.9 kB\u001b[0m \u001b[31m9.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading h11-0.14.0-py3-none-any.whl (58 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m3.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading jsonpointer-3.0.0-py2.py3-none-any.whl (7.6 kB)\n",
      "Downloading smmap-5.0.1-py3-none-any.whl (24 kB)\n",
      "Installing collected packages: watchdog, tenacity, smmap, orjson, jsonpointer, jiter, h11, tiktoken, pydeck, jsonpatch, httpcore, gitdb, httpx, gitpython, openai, langsmith, streamlit, langchain-core, langchain-text-splitters, langchain_openai, langchain\n",
      "  Attempting uninstall: tenacity\n",
      "    Found existing installation: tenacity 9.0.0\n",
      "    Uninstalling tenacity-9.0.0:\n",
      "      Successfully uninstalled tenacity-9.0.0\n",
      "Successfully installed gitdb-4.0.11 gitpython-3.1.43 h11-0.14.0 httpcore-1.0.5 httpx-0.27.2 jiter-0.5.0 jsonpatch-1.33 jsonpointer-3.0.0 langchain-0.3.0 langchain-core-0.3.0 langchain-text-splitters-0.3.0 langchain_openai-0.2.0 langsmith-0.1.120 openai-1.45.0 orjson-3.10.7 pydeck-0.9.1 smmap-5.0.1 streamlit-1.38.0 tenacity-8.5.0 tiktoken-0.7.0 watchdog-4.0.2\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "try:\n",
    "    from google.colab import drive, userdata\n",
    "    COLAB = True\n",
    "    print(\"Note: using Google CoLab\")\n",
    "except:\n",
    "    print(\"Note: not using Google CoLab\")\n",
    "    COLAB = False\n",
    "\n",
    "# OpenAI Secrets\n",
    "if COLAB:\n",
    "    os.environ[\"OPENAI_API_KEY\"] = userdata.get('OPENAI_API_KEY')\n",
    "\n",
    "# Install needed libraries in CoLab\n",
    "if COLAB:\n",
    "    !pip install langchain langchain_openai openai streamlit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "v2MPPX0c1pHi"
   },
   "source": [
    "# Part 10.4: Creating a Chat Application\n",
    "\n",
    "In this module, we will guide you through the process of creating a StreamLit-based LLM chat application. To keep things accessible and straightforward, we'll run our app using Google Colab. Additionally, we'll introduce you to llm_util.py, a utility script designed to make it easier to work with LangChain-compatible large language models (LLMs). For this example, we'll be using OpenAI's LLM to power our chat application. By the end of this module, you'll have a functional, interactive chat app and a solid understanding of how to integrate LLMs into your projects using StreamLit.\n",
    "\n",
    "We will now create three files:\n",
    "\n",
    "* **app.py** - The main StreamLit chat application.\n",
    "* **llm_util.py** - The LLM utility that allows us to utilize any LangChain LLM for our chat application.\n",
    "* **llms.yaml** - A config file to define which LLM's we will use; for this example it will be OpenAI.\n",
    "\n",
    "## Chat Application\n",
    "\n",
    "This following code sets up a simple chatbot application using the OpenAI API and Streamlit, a Python library that allows for easy creation of web apps. The script, named app.py, begins by importing various modules necessary for its functionality. It uses ```openai.OpenAI``` to interact with the OpenAI language model, streamlit to create the user interface, and sys for handling command-line arguments. Additionally, it imports custom utilities from ```llm_util```, along with specific components from the LangChain library, including ```ConversationSummaryMemory```, ```PromptTemplate```, and ```ConversationChain```. These tools are crucial for managing the conversation's context, creating input templates, and facilitating conversational interactions.\n",
    "\n",
    "The script starts by checking if the required command-line arguments are provided. It expects a language model profile as an argument; if this is not specified, it halts execution and prompts the user to provide the necessary input. This mechanism ensures that the appropriate language model is selected before launching the chatbot.\n",
    "\n",
    "The heart of the chatbot setup lies in the create_chatbot function. This function establishes the chatbot using the specified language model and incorporates a memory component, ```ConversationSummaryMemory```, which keeps track of the ongoing conversation's context. By utilizing this memory, the chatbot can generate more coherent and contextually relevant responses over time. Additionally, the function defines a simple template using LangChain's PromptTemplate, formatting how the conversation history and user input are combined. It then returns a ConversationChain object that processes these conversations, ensuring that responses are influenced by the entire chat history.\n",
    "\n",
    "To create an interactive interface, the script uses Streamlit. It begins by setting a title, \"Chat,\" for the application. To manage the chatbot’s state and keep track of the conversation, the script employs Streamlit's session state mechanism. It checks if a chatbot instance (st.session_state.chat) already exists; if not, it initializes a new one with the LLM profile provided via the command line. Similarly, it sets up a list to store the conversation messages (st.session_state.messages) if it does not already exist.\n",
    "\n",
    "The conversation history is then iterated through and displayed using Streamlit's chat-specific functions. Each message, whether from the user or the assistant, is rendered in the chat interface. This provides a running log of the conversation, giving the user an ongoing view of their interaction with the chatbot.\n",
    "\n",
    "User interaction occurs through a chat input box facilitated by st.chat_input. When the user provides input, the script appends the message to the conversation log and displays it in the interface. The chatbot then processes this input using the predict method of ConversationChain, generating a response based on the conversation's context. This response is displayed in the chat window and added to the session state, ensuring the entire dialogue is stored and utilized for subsequent interactions.\n",
    "\n",
    "In essence, this script leverages LangChain's memory management and templating capabilities to create a dynamic chatbot, while Streamlit provides a user-friendly interface for real-time interaction. The combination of these tools allows the chatbot to maintain context throughout the conversation, resulting in more meaningful and coherent exchanges. Furthermore, by requiring a language model profile as a command-line argument, the script is flexible and adaptable to various LLM configurations, allowing the user to customize the chatbot's behavior depending on their specific needs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "nEQ9c5Akqj_R",
    "outputId": "8c5eeabd-f406-4c8f-f330-52ed4e285985"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing app.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile app.py\n",
    "from openai import OpenAI\n",
    "import streamlit as st\n",
    "from llm_util import *\n",
    "from langchain.memory import ConversationSummaryMemory\n",
    "from langchain.prompts.prompt import PromptTemplate\n",
    "from langchain.chains import ConversationChain\n",
    "import sys\n",
    "\n",
    "# This retrieves all command line arguments as a list\n",
    "arguments = sys.argv\n",
    "if len(sys.argv) != 2:\n",
    "    print(\"Please specify the llm to use as the first argument\")\n",
    "    st.stop()\n",
    "else:\n",
    "    profile = sys.argv[1]\n",
    "\n",
    "\n",
    "def create_chatbot(llm):\n",
    "    memory = ConversationSummaryMemory(llm=llm)\n",
    "\n",
    "    template = \"\"\"{history}\\n{input}\\n\\n\n",
    "    \"\"\"\n",
    "    PROMPT = PromptTemplate(input_variables=[\"history\", \"input\"], template=template)\n",
    "    return ConversationChain(llm=llm, prompt=PROMPT, memory=memory, verbose=False)\n",
    "\n",
    "\n",
    "st.title(\"Chat\")\n",
    "\n",
    "if \"chat\" not in st.session_state:\n",
    "    client = open_llm(profile)\n",
    "    st.session_state.chat = create_chatbot(client)\n",
    "\n",
    "if \"messages\" not in st.session_state:\n",
    "    st.session_state.messages = []\n",
    "\n",
    "for message in st.session_state.messages:\n",
    "    with st.chat_message(message[\"role\"]):\n",
    "        st.markdown(\n",
    "            message[\"content\"],\n",
    "        )\n",
    "\n",
    "if prompt := st.chat_input(\"What is up?\"):\n",
    "    st.session_state.messages.append({\"role\": \"user\", \"content\": prompt})\n",
    "    with st.chat_message(\"user\"):\n",
    "        st.markdown(\n",
    "            prompt,\n",
    "        )\n",
    "\n",
    "    with st.chat_message(\"assistant\"):\n",
    "        response = st.session_state.chat.predict(input=prompt)\n",
    "        st.markdown(\n",
    "            response,\n",
    "        )\n",
    "    st.session_state.messages.append({\"role\": \"assistant\", \"content\": response})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HSKyHUHqhCcx"
   },
   "source": [
    "## LLM Utility\n",
    "\n",
    "\n",
    "This following code enables the chat application to use various language models supported by LangChain based on a configuration file. The llm_util.py script serves as a utility that dynamically loads and initializes different language models using configurations specified in a YAML file (llms.yaml). This approach provides a flexible way to change the language model without modifying the main application code, allowing for easy experimentation and customization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "bXml9nrGqj_R",
    "outputId": "5400a880-69fd-4b87-b11a-32e8132e06ef"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing llm_util.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile llm_util.py\n",
    "import yaml\n",
    "\n",
    "# Load the YAML file\n",
    "def load_yaml(file_path):\n",
    "    with open(file_path, \"r\") as file:\n",
    "        return yaml.safe_load(file)\n",
    "\n",
    "\n",
    "# Function to dynamically import a class based on a string path (e.g., \"langchain_community.chat_models.ChatOllama\")\n",
    "def get_class(class_path):\n",
    "    module_path, class_name = class_path.rsplit(\".\", 1)\n",
    "    module = __import__(module_path, fromlist=[class_name])\n",
    "    return getattr(module, class_name)\n",
    "\n",
    "\n",
    "# Open Language Model Server function\n",
    "def open_llm(server_name):\n",
    "    config = load_yaml(\"llms.yaml\")\n",
    "    for server in config[\"servers\"]:\n",
    "        if server[\"name\"] == server_name:\n",
    "            class_path = server[\"class\"]\n",
    "            clazz = get_class(class_path)\n",
    "            # Remove 'class' and 'name' from the parameters as they're not needed for initialization\n",
    "            params = {k: v for k, v in server.items() if k not in [\"class\", \"name\"]}\n",
    "\n",
    "            return clazz(**params)\n",
    "    raise ValueError(f\"Server '{server_name}' not found\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "wk6T0aJEqj_S",
    "outputId": "207c08e7-ee31-4752-e632-087d9cdccc40"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing llms.yaml\n"
     ]
    }
   ],
   "source": [
    "%%writefile llms.yaml\n",
    "\n",
    "servers:\n",
    "  - name: server1\n",
    "    class: langchain_openai.ChatOpenAI\n",
    "    model: gpt-4o-mini\n",
    "    temperature: 0\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "eMqVgvmxhySA"
   },
   "source": [
    "Next, we obtain the password for our StreamLit server we are about to launch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "0iJmAPR8c-JE",
    "outputId": "bfb03037-7bf6-45ef-bfb1-8315823f15c9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "34.138.208.107"
     ]
    }
   ],
   "source": [
    "!curl https://loca.lt/mytunnelpassword"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "H3lKrMcvh3_F"
   },
   "source": [
    "We launch the StreamLit server and obtain its URL. You will need the above password when you access the URL it gives you."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "YSLPrByztQOe",
    "outputId": "1eebbf3a-c3bf-4205-abf0-87c53f5397aa"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[K\u001b[?25hyour url is: https://lucky-colts-reply.loca.lt\n",
      "^C\n"
     ]
    }
   ],
   "source": [
    "!streamlit run app.py server1 &>/content/logs.txt &\n",
    "!npx --yes localtunnel --port 8501"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OXvoDfPzkgwc"
   },
   "source": [
    "The following config file shows how to utilize other LLM servers:\n",
    "\n",
    "* server1 - Ollama\n",
    "* server2 - OpenAI\n",
    "* server3 - Bedrock (AWS)\n",
    "* server4 - LMStudio"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LLPuHtYQqj_S"
   },
   "source": [
    "```\n",
    "servers:\n",
    "  - name: server1\n",
    "    class: langchain_community.chat_models.ChatOllama\n",
    "    base_url: http://localhost:11434\n",
    "    model: llama2\n",
    "    temperature: 0\n",
    "  - name: server2\n",
    "    class: langchain_openai.ChatOpenAI\n",
    "    model: gpt-4o-mini\n",
    "    temperature: 0\n",
    "  - name: server3\n",
    "    class: langchain_aws.ChatBedrock\n",
    "    model_id: amazon.titan-text-express-v1\n",
    "    model_kwargs:\n",
    "      temperature: 0.1\n",
    "  - name: server4\n",
    "    class: langchain_openai.ChatOpenAI\n",
    "    base_url: http://localhost:1234/v1/\n",
    "    model: TheBloke/Mistral-7B-Instruct-v0.1-GGUF\n",
    "    openai_api_key: None\n",
    "    temperature: 0\n",
    "  ```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "rUH52Q_wknk0"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}