t81_559_class_04_1_langchain_chat.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "whjsJasuhstV"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/jeffheaton/app_generative_ai/blob/main/t81_559_class_04_1_langchain_chat.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "euOZxlIMhstX"
   },
   "source": [
    "# T81-559: Applications of Generative Artificial Intelligence\n",
    "**Module 4: LangChain: Chat and Memory**\n",
    "* Instructor: [Jeff Heaton](https://sites.wustl.edu/jeffheaton/), McKelvey School of Engineering, [Washington University in St. Louis](https://engineering.wustl.edu/Programs/Pages/default.aspx)\n",
    "* For more information visit the [class website](https://sites.wustl.edu/jeffheaton/t81-558/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "d4Yov72PhstY"
   },
   "source": [
    "# Module 4 Material\n",
    "\n",
    "* **Part 4.1: LangChain Conversations** [[Video]](https://www.youtube.com/watch?v=Effbhxq07Ag) [[Notebook]](t81_559_class_04_1_langchain_chat.ipynb)\n",
    "* Part 4.2: Conversation Buffer Window Memory [[Video]](https://www.youtube.com/watch?v=14RgiFVGfAA) [[Notebook]](t81_559_class_04_2_memory_buffer.ipynb)\n",
    "* Part 4.3: Conversation Token Buffer Memory [[Video]](https://www.youtube.com/watch?v=QTe5g2c3bSM) [[Notebook]](t81_559_class_04_3_memory_token.ipynb)\n",
    "* Part 4.4: Conversation Summary Memory [[Video]](https://www.youtube.com/watch?v=asZQ8Ktqmt8) [[Notebook]](t81_559_class_04_4_memory_summary.ipynb)\n",
    "* Part 4.5: Persisting Langchain Memory [[Video]](https://www.youtube.com/watch?v=sjCyqqOQcPA) [[Notebook]](t81_559_class_04_5_memory_persist.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AcAUP0c3hstY"
   },
   "source": [
    "# Google CoLab Instructions\n",
    "\n",
    "The following code ensures that Google CoLab is running and maps Google Drive if needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "xsI496h5hstZ",
    "outputId": "e9c4abee-62fe-4204-f6b0-5c29bfcaaaed"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: using Google CoLab\n",
      "Collecting langchain\n",
      "  Downloading langchain-0.1.17-py3-none-any.whl (867 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m867.6/867.6 kB\u001b[0m \u001b[31m3.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting langchain_openai\n",
      "  Downloading langchain_openai-0.1.4-py3-none-any.whl (33 kB)\n",
      "Requirement already satisfied: PyYAML>=5.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (6.0.1)\n",
      "Requirement already satisfied: SQLAlchemy<3,>=1.4 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.0.29)\n",
      "Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (3.9.5)\n",
      "Requirement already satisfied: async-timeout<5.0.0,>=4.0.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (4.0.3)\n",
      "Collecting dataclasses-json<0.7,>=0.5.7 (from langchain)\n",
      "  Downloading dataclasses_json-0.6.5-py3-none-any.whl (28 kB)\n",
      "Collecting jsonpatch<2.0,>=1.33 (from langchain)\n",
      "  Downloading jsonpatch-1.33-py2.py3-none-any.whl (12 kB)\n",
      "Collecting langchain-community<0.1,>=0.0.36 (from langchain)\n",
      "  Downloading langchain_community-0.0.36-py3-none-any.whl (2.0 MB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.0/2.0 MB\u001b[0m \u001b[31m38.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting langchain-core<0.2.0,>=0.1.48 (from langchain)\n",
      "  Downloading langchain_core-0.1.48-py3-none-any.whl (302 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m302.9/302.9 kB\u001b[0m \u001b[31m16.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting langchain-text-splitters<0.1,>=0.0.1 (from langchain)\n",
      "  Downloading langchain_text_splitters-0.0.1-py3-none-any.whl (21 kB)\n",
      "Collecting langsmith<0.2.0,>=0.1.17 (from langchain)\n",
      "  Downloading langsmith-0.1.52-py3-none-any.whl (116 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m116.4/116.4 kB\u001b[0m \u001b[31m5.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: numpy<2,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (1.25.2)\n",
      "Requirement already satisfied: pydantic<3,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.7.1)\n",
      "Requirement already satisfied: requests<3,>=2 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.31.0)\n",
      "Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (8.2.3)\n",
      "Collecting openai<2.0.0,>=1.10.0 (from langchain_openai)\n",
      "  Downloading openai-1.25.0-py3-none-any.whl (312 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m312.9/312.9 kB\u001b[0m \u001b[31m10.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting tiktoken<1,>=0.5.2 (from langchain_openai)\n",
      "  Downloading tiktoken-0.6.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.8 MB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.8/1.8 MB\u001b[0m \u001b[31m19.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.3.1)\n",
      "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (23.2.0)\n",
      "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.4.1)\n",
      "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (6.0.5)\n",
      "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.9.4)\n",
      "Collecting marshmallow<4.0.0,>=3.18.0 (from dataclasses-json<0.7,>=0.5.7->langchain)\n",
      "  Downloading marshmallow-3.21.1-py3-none-any.whl (49 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.4/49.4 kB\u001b[0m \u001b[31m1.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting typing-inspect<1,>=0.4.0 (from dataclasses-json<0.7,>=0.5.7->langchain)\n",
      "  Downloading typing_inspect-0.9.0-py3-none-any.whl (8.8 kB)\n",
      "Collecting jsonpointer>=1.9 (from jsonpatch<2.0,>=1.33->langchain)\n",
      "  Downloading jsonpointer-2.4-py2.py3-none-any.whl (7.8 kB)\n",
      "Collecting packaging<24.0,>=23.2 (from langchain-core<0.2.0,>=0.1.48->langchain)\n",
      "  Downloading packaging-23.2-py3-none-any.whl (53 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m53.0/53.0 kB\u001b[0m \u001b[31m4.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting orjson<4.0.0,>=3.9.14 (from langsmith<0.2.0,>=0.1.17->langchain)\n",
      "  Downloading orjson-3.10.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (142 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m142.7/142.7 kB\u001b[0m \u001b[31m3.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.10.0->langchain_openai) (3.7.1)\n",
      "Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai<2.0.0,>=1.10.0->langchain_openai) (1.7.0)\n",
      "Collecting httpx<1,>=0.23.0 (from openai<2.0.0,>=1.10.0->langchain_openai)\n",
      "  Downloading httpx-0.27.0-py3-none-any.whl (75 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.6/75.6 kB\u001b[0m \u001b[31m3.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.10.0->langchain_openai) (1.3.1)\n",
      "Requirement already satisfied: tqdm>4 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.10.0->langchain_openai) (4.66.2)\n",
      "Requirement already satisfied: typing-extensions<5,>=4.7 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.10.0->langchain_openai) (4.11.0)\n",
      "Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1->langchain) (0.6.0)\n",
      "Requirement already satisfied: pydantic-core==2.18.2 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1->langchain) (2.18.2)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.3.2)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.7)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2.0.7)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2024.2.2)\n",
      "Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy<3,>=1.4->langchain) (3.0.3)\n",
      "Requirement already satisfied: regex>=2022.1.18 in /usr/local/lib/python3.10/dist-packages (from tiktoken<1,>=0.5.2->langchain_openai) (2023.12.25)\n",
      "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->openai<2.0.0,>=1.10.0->langchain_openai) (1.2.1)\n",
      "Collecting httpcore==1.* (from httpx<1,>=0.23.0->openai<2.0.0,>=1.10.0->langchain_openai)\n",
      "  Downloading httpcore-1.0.5-py3-none-any.whl (77 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.9/77.9 kB\u001b[0m \u001b[31m7.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting h11<0.15,>=0.13 (from httpcore==1.*->httpx<1,>=0.23.0->openai<2.0.0,>=1.10.0->langchain_openai)\n",
      "  Downloading h11-0.14.0-py3-none-any.whl (58 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m4.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting mypy-extensions>=0.3.0 (from typing-inspect<1,>=0.4.0->dataclasses-json<0.7,>=0.5.7->langchain)\n",
      "  Downloading mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)\n",
      "Installing collected packages: packaging, orjson, mypy-extensions, jsonpointer, h11, typing-inspect, tiktoken, marshmallow, jsonpatch, httpcore, langsmith, httpx, dataclasses-json, openai, langchain-core, langchain-text-splitters, langchain_openai, langchain-community, langchain\n",
      "  Attempting uninstall: packaging\n",
      "    Found existing installation: packaging 24.0\n",
      "    Uninstalling packaging-24.0:\n",
      "      Successfully uninstalled packaging-24.0\n",
      "Successfully installed dataclasses-json-0.6.5 h11-0.14.0 httpcore-1.0.5 httpx-0.27.0 jsonpatch-1.33 jsonpointer-2.4 langchain-0.1.17 langchain-community-0.0.36 langchain-core-0.1.48 langchain-text-splitters-0.0.1 langchain_openai-0.1.4 langsmith-0.1.52 marshmallow-3.21.1 mypy-extensions-1.0.0 openai-1.25.0 orjson-3.10.2 packaging-23.2 tiktoken-0.6.0 typing-inspect-0.9.0\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "try:\n",
    "    from google.colab import drive, userdata\n",
    "    COLAB = True\n",
    "    print(\"Note: using Google CoLab\")\n",
    "except:\n",
    "    print(\"Note: not using Google CoLab\")\n",
    "    COLAB = False\n",
    "\n",
    "# OpenAI Secrets\n",
    "if COLAB:\n",
    "    os.environ[\"OPENAI_API_KEY\"] = userdata.get('OPENAI_API_KEY')\n",
    "\n",
    "# Install needed libraries in CoLab\n",
    "if COLAB:\n",
    "    !pip install langchain langchain_openai"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pC9A-LaYhsta"
   },
   "source": [
    "# 4.1: LangChain Conversations\n",
    "\n",
    "Large language models (LLMs) facilitate interaction like human conversations. They are capable of referencing information shared earlier in the dialogue. In this module, we will explore managing an LLM's memory capabilities. Notably, LLMs need an inherent memory system beyond their immediate context buffer. Consequently, it falls upon external systems like LangChain to embed all previous conversational memory into each new prompt. To effectively remember past interactions, LangChain maintains a comprehensive transcript that accumulates over time. This transcript is reintroduced to the LLM with each exchange, which incorporates both user inputs and LLM responses. With each new prompt, the LLM is tasked to generate a suitable next response, thereby perpetuating the interactive process.\n",
    "\n",
    "## Creating a Chat Conversation\n",
    "\n",
    "The code snippet provides two functions designed to create a basic conversation utility for a Large Language Model (LLM). This is part of an effort to build foundational utilities before introducing more complex memory capabilities using LangChain classes in later sections.\n",
    "\n",
    "First, the necessary modules and classes are imported. HumanMessage and SystemMessage from langchain_core.messages are used to represent messages from a human user and system responses, respectively. The ChatPromptTemplate, HumanMessagePromptTemplate, and SystemMessagePromptTemplate from langchain_core.prompts.chat help format prompts for the chat. Additionally, ChatOpenAI from langchain_openai is used to interact with OpenAI's language models, and display_markdown from IPython.display allows for markdown rendering within IPython environments.\n",
    "\n",
    "The first function, begin_conversation, initializes a conversation with a system prompt, which is a predefined message declaring that the system's role is to assist (specified by the DEFAULT_SYSTEM variable). It creates an initial SystemMessage containing this prompt and returns a list containing this message, setting the stage for a conversation.\n",
    "\n",
    "The second function, converse, facilitates the interaction between the human user and the LLM. It takes an LLM instance, the ongoing conversation (a list of messages), and the user's prompt as inputs. It begins by appending the user's message (encapsulated as a HumanMessage) to the conversation. It then invokes the LLM to respond based on the entire conversation history up to that point. The LLM's response is captured and added back to the conversation. Finally, the content of the LLM's response is returned, providing the output for the user to see. This function enables a dynamic and continuous exchange between the user and the system, leveraging the LLM's capabilities to generate relevant responses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "TMF-rtxgRAea"
   },
   "outputs": [],
   "source": [
    "from langchain_core.messages import HumanMessage, SystemMessage\n",
    "from langchain_core.prompts.chat import (\n",
    "    ChatPromptTemplate,\n",
    "    HumanMessagePromptTemplate,\n",
    "    SystemMessagePromptTemplate,\n",
    ")\n",
    "from langchain_openai import ChatOpenAI\n",
    "from IPython.display import display_markdown\n",
    "\n",
    "DEFAULT_SYSTEM = \"You are a helpful assistant. Format answers with markdown.\"\n",
    "\n",
    "def begin_conversation(sys_prompt):\n",
    "  messages = [\n",
    "      SystemMessage(content=sys_prompt)\n",
    "  ]\n",
    "  return messages\n",
    "\n",
    "def converse(llm, conversation, prompt):\n",
    "  conversation.append(HumanMessage(content=prompt))\n",
    "  output = llm.invoke(conversation)\n",
    "  conversation.append(output)\n",
    "  return output.content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ClPhLkGldhPt"
   },
   "source": [
    "The provided code builds upon the initial snippet to create a simple conversation where the Large Language Model (LLM) recalls the user's name through a series of exchanges.\n",
    "\n",
    "The conversation starts by initializing an instance of ChatOpenAI, a class from the LangChain library, with a specific model ('gpt-4o-mini'). Several parameters are set for the LLM, including the temperature, which controls the randomness of the response, and n, which specifies the number of responses to generate (in this case, a single response).\n",
    "\n",
    "The function begin_conversation is called with the constant DEFAULT_SYSTEM as its argument. This function initializes the conversation with a system message stating, \"You are a helpful assistant.\" The conversation list, containing this initial system message, is created and will be used to track the entire conversation history.\n",
    "\n",
    "Next, the converse function is used to facilitate the dialogue between the user and the LLM. The user begins by sending a prompt: \"Hello, what is my name?\" This user's message is appended to the conversation history, and the LLM is invoked to generate a response based on the conversation so far. The LLM's response is added to the conversation and displayed using display_markdown to format the output appropriately.\n",
    "\n",
    "In response to realizing the LLM does not yet know the user's name, the user then provides their name with the statement, \"Oh sorry, my name is Jeff.\" This message is similarly processed: added to the conversation, and the LLM generates a new response acknowledging the name or continuing the conversation. Again, the response is displayed.\n",
    "\n",
    "Finally, the user asks again, \"What is my name?\" to test if the LLM recalls the name from earlier in the conversation. The same process ensues, with the LLM generating a response based on the updated conversation history that now includes the user's name.\n",
    "\n",
    "This sequence effectively demonstrates a simple use case where the conversation history maintained in the list enables the LLM to recall and utilize context from earlier exchanges, such as remembering the user's name. The ability to recall details like this is crucial for creating more engaging and personalized user interactions with LLMs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 93
    },
    "id": "ydaqwgRiH4D6",
    "outputId": "50bb7463-7b86-43e5-f04d-3fbe90f7e382"
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "I'm sorry, but I don't have access to your personal information such as your name. How can I assist you today?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "Nice to meet you, Jeff! How can I help you today?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "Your name is Jeff. How can I assist you further, Jeff?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "MODEL = 'gpt-4o-mini'\n",
    "\n",
    "# Initialize the OpenAI LLM with your API key\n",
    "llm = ChatOpenAI(\n",
    "  model=MODEL,\n",
    "  temperature= 0.3,\n",
    "  n= 1)\n",
    "\n",
    "conversation = begin_conversation(DEFAULT_SYSTEM)\n",
    "output = converse(llm, conversation, \"Hello, what is my name?\")\n",
    "display_markdown(output,raw=True)\n",
    "output = converse(llm, conversation, \"Oh sorry, my name is Jeff.\")\n",
    "display_markdown(output,raw=True)\n",
    "output = converse(llm, conversation, \"What is my name?\")\n",
    "display_markdown(output,raw=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ulYUrfVcnlgF"
   },
   "source": [
    "## Conversing with the LLM in Markdown\n",
    "\n",
    "The function chat serves as a facilitator for conversation between a human user and a Large Language Model (LLM) and enhances the display of the chat responses using Markdown formatting. It is built upon the earlier provided code snippets that set up a basic conversational framework with an LLM.\n",
    "\n",
    "Markdown is a lightweight markup language with plain-text formatting syntax. It is designed to be converted into HTML or other formats while remaining easy to read and write in its raw form. This feature makes it popular for writing on the web, as users can create formatted text (like headers, lists, italics, and bold text) using simple and readable symbols.\n",
    "\n",
    "In the chat function, the process starts by printing the user's prompt prefixed with \"Human: \". This mimics a real chat interface, clearly delineating the messages that are input by the user. Then, the function calls converse, which was previously described, to process the prompt within the ongoing conversation with the LLM. This function appends the human message to the conversation, invokes the LLM to generate a response based on the conversation's context, and then appends this system-generated message back into the conversation history.\n",
    "\n",
    "After obtaining the LLM's response through converse, the function display_markdown is used to render this response. The raw=True parameter tells the IPython display function to treat the string as raw Markdown. By using Markdown formatting, responses can include enhanced textual features such as italics, bolding, and lists, which can make the output more readable and engaging.\n",
    "\n",
    "By formatting LLM responses in Markdown, we reinforce the natural, text-based communication style of the LLM. Given that LLMs, such as those provided by OpenAI, often format their responses in Markdown to leverage its text-enhancement capabilities, this approach ensures that the conversation utility outputs responses that utilize the full range of expressive possibilities offered by Markdown, enhancing the user interaction experience. This design choice aligns well with the Markdown capabilities inherently supported by many LLMs, ensuring that the responses are both visually appealing and functionally informative."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "MHL6Ik8IM2PA"
   },
   "outputs": [],
   "source": [
    "def chat(llm, conversation, prompt):\n",
    "  print(f\"Human: {prompt}\")\n",
    "  output = converse(llm, conversation, prompt)\n",
    "  display_markdown(output,raw=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FvzsYl3Qplac"
   },
   "source": [
    "The provided code sequence demonstrates a conversation between a human user and a Large Language Model (LLM), making use of the chat function to interactively manage the conversation and display responses in Markdown format. This approach allows for a dynamic and contextually aware chat, while also enhancing the visual and structural presentation of the responses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 366
    },
    "id": "PtLDak7TM_FU",
    "outputId": "3b8831d1-4b26-4d54-d21a-3a6a8e07f101"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: What is my name?\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "I'm sorry, but I don't have access to your personal information such as your name. How can I assist you today?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: Okay, then let me introduce myself, my name is Jeff\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "Nice to meet you, Jeff! How can I assist you today?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: What is my name?\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "Your name is Jeff. How can I assist you today, Jeff?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: Give me a table of the 5 most populus cities with population and country.\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "| City          | Population     | Country |\n",
       "|---------------|----------------|---------|\n",
       "| Tokyo         | 37,833,000     | Japan   |\n",
       "| Delhi         | 30,291,000     | India   |\n",
       "| Shanghai      | 27,058,000     | China   |\n",
       "| Sao Paulo     | 22,043,000     | Brazil  |\n",
       "| Mumbai        | 21,042,000     | India   |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "conversation = begin_conversation(DEFAULT_SYSTEM)\n",
    "chat(llm, conversation, \"What is my name?\")\n",
    "chat(llm, conversation, \"Okay, then let me introduce myself, my name is Jeff\")\n",
    "chat(llm, conversation, \"What is my name?\")\n",
    "chat(llm, conversation, \"Give me a table of the 5 most populus cities with population and country.\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GeY-9YSOno_t"
   },
   "source": [
    "## Constraining the Conversation with a System Prompt\n",
    "\n",
    "You can use the system prompt to constrain the conversation to a specific topic. Here, we provide a simple agent that will only discuss life insurance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 451
    },
    "id": "NyGpLJmWjGNq",
    "outputId": "9d1b0cb5-ef71-4b3a-b8c7-c03c02d35858"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: What is my name?\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "I'm here to help answer questions about life insurance. How can I assist you today with life insurance-related queries?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: Okay, then let me introduce myself, my name is Jeff\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "I'm here to help answer questions about life insurance. How can I assist you today with life insurance-related queries?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: What is my name?\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "I'm here to help answer questions about life insurance. How can I assist you today with life insurance-related queries?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: What is your favorite programming language?\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "I'm here to help answer questions about life insurance. How can I assist you today with life insurance-related queries?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: What is the difference between a term and whole life policy?\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "- **Term Life Insurance:**\n",
       "  - Provides coverage for a specific period, such as 10, 20, or 30 years.\n",
       "  - Typically more affordable than whole life insurance.\n",
       "  - Does not build cash value.\n",
       "  - Coverage ends when the term expires unless renewed or converted.\n",
       "  \n",
       "- **Whole Life Insurance:**\n",
       "  - Provides coverage for your entire life.\n",
       "  - Premiums remain level throughout the policy's lifetime.\n",
       "  - Builds cash value over time that you can borrow against or withdraw.\n",
       "  - Generally more expensive than term life insurance.\n",
       "  \n",
       "It's important to consider your financial goals and needs when deciding between term and whole life insurance."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "conversation = begin_conversation(\"\"\"\n",
    "You are a helpful agent to answer questions about life insurance. Do not talk\n",
    "about anything else with users. . Format answers with markdown.\"\"\")\n",
    "chat(llm, conversation, \"What is my name?\")\n",
    "chat(llm, conversation, \"Okay, then let me introduce myself, my name is Jeff\")\n",
    "chat(llm, conversation, \"What is my name?\")\n",
    "chat(llm, conversation, \"What is your favorite programming language?\")\n",
    "chat(llm, conversation, \"What is the difference between a term and whole life policy?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3TjZs_TRht1n"
   },
   "source": [
    "# Module 4 Assignment\n",
    "\n",
    "You can find the first assignment here: [assignment 4](https://github.com/jeffheaton/app_generative_ai/blob/main/assignments/assignment_yourname_class4.ipynb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3.11 (genai)",
   "language": "python",
   "name": "genai"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}