notebooks/GenAI/notebooks/Pubmed_RAG_chatbot.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "2edc6187-82ae-44e2-852f-2ad2712c93aa",
   "metadata": {},
   "source": [
    "# Creating a PubMed Chatbot using Azure"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8acf2f72",
   "metadata": {},
   "source": [
    "## Overview\n",
    "[PubMed](https://pubmed.ncbi.nlm.nih.gov/about/) supports the search and retrieval of biomedical and life sciences literature with the aim of improving health both globally and personally. Here we create a chatbot that is grounded on PubMed data. Most Azure command line tools are already installed and it is recommended to use the **AzureML** kernel in your Jupyter notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58cb56d0",
   "metadata": {},
   "source": [
    "## Prerequisites\n",
    "We assume you have access to both Azure AI Studio and Azure AI Search, and have already deployed an LLM."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ecea2ad-7c65-4367-87e1-b021167c3a1d",
   "metadata": {},
   "source": [
    "## Learning objectives\n",
    "\n",
    "This tutorial will cover the following topics:\n",
    "+ Introduce Langchain\n",
    "+ Explain the differences between zero-shot, one-shot, and few-shot prompting\n",
    "+ Practice using different document retrievers"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f645b8cf",
   "metadata": {},
   "source": [
    "## Get started"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d01e74b-b5b4-4be9-b16e-ec55419318ef",
   "metadata": {},
   "source": [
    "### Optional: Deploy a model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9dbd13e7-afc9-416b-94dc-418a93e14587",
   "metadata": {},
   "source": [
    "In this tutorial we will be using Azure OpenAI which (if you havent already) you can learn how to deploy [here](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/create-resource?pivots=cli). This tutorial utilizes the model **gpt-35-turbo** version 0301 and the embeddings model **text-embedding-ada-002** version 2."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f3e3ab1-5f7e-4028-a66f-9619926a2afd",
   "metadata": {},
   "source": [
    "### PubMed API vs Azure AI Search"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a820eea-1538-4f40-86c4-eb14fe09e127",
   "metadata": {},
   "source": [
    "Our chatbot will rely on documents to answer our questions to do so we are supplying it a **vector index**. A vector index or index is a data structure that enables fast and accurate search and retrieval of vector embeddings from a large dataset of objects. We will be working with two options for our index: PubMed API vs Azure AI Search."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7314b115-9433-460d-b275-78aa50f0a858",
   "metadata": {},
   "source": [
    "**What is the difference?**\n",
    "\n",
    "The **PubMed API** is provided free by LangChain to connect your model to more than **35 million citations** for biomedical literature from MEDLINE, life science journals, and online books. \n",
    "\n",
    "**Azure AI Search** (formally known as Azure Cognitive Search) is a vector store from Azure that allows the user more **security and control** on which documents you wish to supply to your model. AI Search is a vector store or database that stores the **embeddings** of your documents and the metadata. It can also act as a retriever by using the LangChain tool **AzureCognitiveSearchRetriever** which will be implementing  **Retrieval-augmented generation** (RAG). RAG is a method or technique that **indexes documents** by first loading them in, splitting them into chucks (making it easier for our model to search for relevant splits), embedding the splits, then storing them in a vector store. The next steps in RAG are based on the question you ask your chatbot. If we were to ask it \"What is a cell?\" the vector store will be searched by a retriever to find relevant splits that have to do with our question, thus **retrieving relevant documents**. And finally our chatbot will **generate an answer** that makes sense of what a cell is, and point out which source documents it used to create the answer.\n",
    "\n",
    "We will be exploring both methods!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bcf1690d-e93d-4cd3-89c6-8d06b5a071a8",
   "metadata": {},
   "source": [
    "### Setting up Azure AI Search"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6330ddf-7972-4451-9fcb-98cf83f5d118",
   "metadata": {},
   "source": [
    "If you choose to use Azure AI Search to supply documents to your model follow the instructions below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c1b23ad-8809-4954-a4df-2ff3b8d9ee58",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install 'langchain' 'langchain-openai' 'langchain-community' 'unstructured' 'tiktoken'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "60b6cd79-0ffb-4af7-ae92-aaca231a8260",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install azure-search-documents --pre --upgrade"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d8c1803b-829a-4256-a88c-1f4b57372ba2",
   "metadata": {},
   "outputs": [],
   "source": [
    "#uncomment if update is needed\n",
    "#! pip install -U \"azure-storage-blob\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "abddde62-f269-454e-bea6-538bd4267277",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Authenticate to use azure cli\n",
    "! az login"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9428d5bc-76e3-4ee1-891b-0bc190c0ae2f",
   "metadata": {},
   "source": [
    "### Setting up our storage container"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05b93a90-ff0b-430d-a5f4-4640bfb77b38",
   "metadata": {},
   "source": [
    "The first step will be to create a container that we will later use as our data source for our index. Set your storage account name, location, and container name variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "43cdc419-25e5-4ba8-b836-a13b2ad77a26",
   "metadata": {},
   "outputs": [],
   "source": [
    "location = '<Enter Location>' #(e.g.,eastus2)\n",
    "container_name = '<Enter Container Name>'\n",
    "\n",
    "#this should be the same as the one you used to set up your workspace\n",
    "resource_group = '<Your Resource Group>'\n",
    "\n",
    "# storage_account_name can be found by going to Azure Machine Learning Workspace > Storage \n",
    "#or you can uncomment and run the command below to list the storage accounts names within your resource group\n",
    "storage_account_name = '<Your Storage Account Name>'\n",
    "\n",
    "#! az storage account list --resource-group \"${resource_group}\" --query \"[].{name:name}\" --output tsv"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a568fcc3-24a7-4f5d-9798-9016468a30ee",
   "metadata": {},
   "source": [
    "Create your container within your storage account running the command below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cdf6c373-a0ee-40c6-a5c6-6841c58cc3db",
   "metadata": {},
   "outputs": [],
   "source": [
    "! az storage container create -n {container_name} --account-name {storage_account_name}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7adafdba-4c4d-4b96-b9bb-33143a72eafc",
   "metadata": {},
   "source": [
    "Run the command below to list the key values of your storage account. The key values will be saved to a json file for protection. We will need one of these keys to create a SAS token that gives us temporary access and permissions to add objects to our container."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6414395e-80b1-4736-b116-70d82675b73b",
   "metadata": {},
   "outputs": [],
   "source": [
    "!az storage account keys list -g {resource_group} -n {storage_account_name} > keys.json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eb2a94da-40de-4b69-8de9-e001b4ea98c7",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "with open('keys.json', mode='r') as f:\n",
    "    data = json.load(f)\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "92cebae7-ddbb-4763-8d50-dd2c9f512696",
   "metadata": {},
   "outputs": [],
   "source": [
    "key=data[0]['value']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3ec2e81-f5c1-43f1-8db0-769069acf9f7",
   "metadata": {},
   "source": [
    "Now we can create our SAS token that will last for 2 hours. Here we are giving our token the ability to read, write, list, add, and create objects (blobs) within our container."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4b311d9b-1713-4699-ab62-b228d1decc2d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# create your SAS token\n",
    "from datetime import datetime, timedelta\n",
    "from azure.storage.blob import BlobServiceClient, generate_account_sas, ResourceTypes, AccountSasPermissions\n",
    "start_time = datetime.utcnow()\n",
    "expiry_time = start_time + timedelta(hours=2)\n",
    "sas_token = generate_account_sas(\n",
    "    account_name=storage_account_name,\n",
    "    container_name=container_name,\n",
    "    account_key=key,\n",
    "    resource_types=ResourceTypes(object=True),\n",
    "    permission=AccountSasPermissions(read=True, write=True, delete=True, list=True, add=True, create=True),\n",
    "    expiry=expiry_time,\n",
    "    start=start_time\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "900efd36-371b-4400-9a9f-fffd1bc14cce",
   "metadata": {},
   "source": [
    "### Gathering documents for the vector store"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d1c9de7-4a06-4f85-b9ff-c8c9e51f8c70",
   "metadata": {},
   "source": [
    "AWS marketplace has PubMed database named **PubMed Central® (PMC)** that contains free full-text archive of biomedical and life sciences journal article at the U.S. National Institutes of Health's National Library of Medicine (NIH/NLM). We will be subsetting this database to add documents to our AI Search Index. Ensure that you have the correct permissions to allow your environment to connect to containers and AI Search."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6ad30ba-cee8-47f9-bc1e-ece8961ac66a",
   "metadata": {},
   "source": [
    "Here we are downloading the metadata file from the PMC index directory, this will list all of the articles within the PMC bucket and their paths. We will use this to subset the database into our own blob storage. Here we are using curl to connect to the public AWS s3 bucket where the metadata and documents are originally stored."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7b395e34-062d-4f77-afee-3601d471954a",
   "metadata": {},
   "outputs": [],
   "source": [
    "#download the metadata file\n",
    "!curl -O http://pmc-oa-opendata.s3.amazonaws.com/oa_comm/txt/metadata/csv/oa_comm.filelist.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93a8595a-767f-4cad-9273-62d8e2cf60d1",
   "metadata": {},
   "source": [
    "We only want the metadata of the first 50 files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c26b0f29-2b07-43a6-800d-4aa5e957fe52",
   "metadata": {},
   "outputs": [],
   "source": [
    "#import the file as a dataframe\n",
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv('oa_comm.filelist.csv')\n",
    "#first 50 files\n",
    "first_50=df[0:50]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "abd1ae93-450e-4c79-83cc-ea46a1b507c1",
   "metadata": {},
   "source": [
    "Lets look at our metadata! We can see that the s3 bucket path to the files are under the **Key** column this is what we will use to loop through the PMC bucket and copy the first 50 files to our bucket."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ff77b2aa-ed1b-4d27-8163-fdaa7a304582",
   "metadata": {},
   "outputs": [],
   "source": [
    "first_50.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84e5f36a-239c-4c15-80ab-f896d45849d3",
   "metadata": {},
   "source": [
    "The following commands uses `azcopy`, a tool that allows you to copy objects from AWS s3 buckets. The for loop we created will gather the location of each document with in AWS s3 bucket and save the documents to our container in the form of a text file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7d63a7e2-dbf1-49ec-bc84-b8c2c8bde62d",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from io import BytesIO\n",
    "#gather path to files in bucket\n",
    "for i in first_50['Key']:\n",
    "    doc_name=i.split(r'/')[-1]\n",
    "    os.system(f'azcopy copy \"https://s3.amazonaws.com/pmc-oa-opendata/{i}\" \"https://{storage_account_name}.blob.core.windows.net/{container_name}/{doc_name}?{sas_token}\"')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "928de2ca-010a-4087-82a7-e548f84f3d95",
   "metadata": {},
   "source": [
    "If you run into any errors make sure you have the `Storage Blob Data Contributor` role assigned to your storage account."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5adf90e-e88b-4631-b860-81c2ea347786",
   "metadata": {},
   "source": [
    "The command below sees if our files have any metadata already associated with them. If your data does not have metadata you can add it to your blob following the section **Adding Metadata to Our Data**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3d2831bf-babd-45cf-9641-a34e1b9d7c37",
   "metadata": {},
   "outputs": [],
   "source": [
    "! az storage blob metadata show --container-name {container_name} --account-name {storage_account_name} --account-key {key} --name 'PMC5000000.txt'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "613cef7d-d0aa-42a8-a46e-7fd1f5c48c3b",
   "metadata": {},
   "source": [
    "### Optional: Adding metadata to our dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "acd6b7cf-decf-4e1d-8a36-86031cc64faf",
   "metadata": {},
   "source": [
    "If you are using a different dataset and it doesnt have metadata already saved to it run the code below. To add metadata, our keys can't have spaces and need to be strings. Here we are making a new dataframe with wanted columns for our metadata, these columns are from the `first_50` variable we created earlier you can replace them with your own column names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "22b9579b-bde9-4e57-bad6-700c7ee73645",
   "metadata": {},
   "outputs": [],
   "source": [
    "metadata_table = first_50[['Article Citation', 'AccessionID', 'PMID']].copy()\n",
    "#make sure that all keys and values are strings the blob metadata with not beable to parse through our metadata if it is a integer\n",
    "metadata_table['PMID'] = metadata_table['PMID'].apply(str)\n",
    "metadata_table.rename(columns={'Article Citation': 'Article_Citation'}, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c02c78e0-18d3-4162-9e58-c790ad85f76f",
   "metadata": {},
   "source": [
    "Transform our table into a dictionary to add to our blob metadata."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6ae31175-f664-413d-8d2f-ecaef67038dd",
   "metadata": {},
   "outputs": [],
   "source": [
    "metadata_dict = metadata_table.to_dict('records')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90dd8101-c635-43a0-9645-1115b32eb037",
   "metadata": {},
   "source": [
    "Let's look at our metadata!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5a9de234-c83a-4d94-87b8-3bd51fb0c531",
   "metadata": {},
   "outputs": [],
   "source": [
    "metadata_dict[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd877ef2-155f-4fe2-b0c1-8e45293196e2",
   "metadata": {},
   "source": [
    "Now that we have our metadata variables set we can connect to our container and the blobs within it by using a **BlobServiceClient**. This client service uses our storage account endpoint and our SAS token. Then we will construct a for loop that loops through the 'first_50' dataframe to gather our document name (which is also the blob name).\n",
    "\n",
    "Next it will do the following:\n",
    "- Gather the metadata (if any exists) of the blob\n",
    "- Update the metadata as the new metadata record we created 'metadata_dict'\n",
    "- Set the metadata on the blob. Although we have updated the metadata it will not save on your blob unless you set it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "46767760-8794-4860-9468-6b2d6b72022b",
   "metadata": {},
   "outputs": [],
   "source": [
    "from azure.storage.blob import BlobServiceClient\n",
    "blob_service = BlobServiceClient(account_url=f'https://{storage_account_name}.blob.core.windows.net', credential=sas_token)\n",
    "\n",
    "for i in range(len(first_50['Key'])):\n",
    "    document_name = first_50['Key'][i].split(\"/\")[-1]  \n",
    "    blob_client = blob_service.get_blob_client(container=container_name, blob=document_name)\n",
    "    # gather metadata properties for that blob\n",
    "    blob_metadata = blob_client.get_blob_properties().metadata\n",
    "    # Update blob metadata\n",
    "    more_blob_metadata = metadata_dict[i]\n",
    "    blob_metadata.update(more_blob_metadata)\n",
    "\n",
    "    # Set metadata on the blob\n",
    "    blob_client.set_blob_metadata(metadata=blob_metadata)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2eaf1733-a4f6-4d71-80d2-83089d6dd3f6",
   "metadata": {},
   "source": [
    "Lets check the metadata of one of our blobs!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2dc7caf8-f021-4406-93bb-e4834a17cc56",
   "metadata": {},
   "outputs": [],
   "source": [
    "! az storage blob metadata show --container-name {container_name} --account-name {storage_account_name} --account-key {key} --name 'PMC5000000.txt'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1b396c8-baa9-44d6-948c-2326dc514839",
   "metadata": {},
   "source": [
    "### Creating an Azure AI Search service"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb6fa941-bf59-4cae-9aa8-2f2741f3a1b1",
   "metadata": {},
   "source": [
    "To create our AI Search index, we will first need to create a search service, and request to create the free SKU to hold all our documents in our vector store. The **free** tier allows you to hold 50MB of data and 3 indexes, and indexers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "63226024-d03e-4fa0-9557-2f18fec07bd5",
   "metadata": {},
   "outputs": [],
   "source": [
    "#the name needs to be unique\n",
    "service_name = '<Enter_Service_Name>'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ea9458fa-3c0c-4249-a8bd-fd86f9bee8c7",
   "metadata": {},
   "outputs": [],
   "source": [
    "! az search service create --name {service_name} --sku free --location {location} --resource-group {resource_group} --partition-count 1 --replica-count 1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4eea51b2-6511-4ae4-ba9d-963a861376cd",
   "metadata": {},
   "source": [
    "Below will list the admin keys, select one of them to use for adding objects to our index."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "95382baa-abb9-4ad1-b1db-11cb9a606b7c",
   "metadata": {},
   "outputs": [],
   "source": [
    "! az search admin-key show --resource-group {resource_group} --service-name {service_name} > keys.json"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee7c2455-9e61-4568-b2f1-03546c1f9878",
   "metadata": {},
   "source": [
    "Save one of the keys."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1dde166b-d342-400a-ba1d-23436e1938ce",
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('keys.json', mode='r') as f:\n",
    "    data = json.load(f)\n",
    "search_key = data[\"primaryKey\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51fbde69-5a23-45a8-a000-9952824d973a",
   "metadata": {},
   "source": [
    "Now we can create our index using a SearchClient which will allow us to also define our fields within our index. Depending on the size of your documents you may need to split your document in chucks so that it fits within the token size of our model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59c5304e-8e14-485c-b452-e5af2da95e01",
   "metadata": {},
   "source": [
    "### Creating an index and loading small documents"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c80ae3c4-ddec-473d-98e9-034e58542968",
   "metadata": {},
   "source": [
    "Here we can create an index that connects our blobs in our container using an **Indexer** and a **Data Container**.\n",
    "\n",
    "**Warning:** This dataset contains large documents, while the below steps are only meant to show you how the process would go with smaller documents"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c0540581-7b16-49af-8c67-a3bbd8a247d2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from azure.search.documents import SearchClient\n",
    "from azure.core.credentials import AzureKeyCredential\n",
    "from azure.search.documents.indexes import SearchIndexClient, SearchIndexerClient\n",
    "from azure.search.documents.indexes.models import (\n",
    "    SearchIndexerDataContainer,\n",
    "    SearchIndexerDataSourceConnection,\n",
    "    SearchIndex,\n",
    "    SearchIndexer,\n",
    "    SearchableField,\n",
    "    SearchFieldDataType,\n",
    "    SimpleField,\n",
    ")\n",
    "\n",
    "endpoint = \"https://{}.search.windows.net/\".format(service_name)\n",
    "index_client = SearchIndexClient(endpoint, AzureKeyCredential(search_key))\n",
    "indexers_client = SearchIndexerClient(endpoint, AzureKeyCredential(search_key))\n",
    "connection_string = f\"DefaultEndpointsProtocol=https;AccountName={storage_account_name};AccountKey={key}\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0584fad3-a07f-4263-97c3-475d774e87a1",
   "metadata": {},
   "source": [
    "Here we are stating our schema or fields before we create our index, these fields are from when we ran the `az storage blob metadata show` command after loading our blobs to our container.\n",
    "\n",
    "- **SimpleField:** A field that you can retrieve values but not search them this is ideal for keys which is a unique ID for each blob. Here we are setting the Md5 value as our key.\n",
    "- **SearchableField:** A field that allows you to retrieve and search values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2bddb00f-fa9e-4677-9d02-484b2eb5b02d",
   "metadata": {},
   "outputs": [],
   "source": [
    "s_index_name = \"pubmed-index-smalldocs\"\n",
    "\n",
    "fields = [\n",
    "    SimpleField(\n",
    "        name=\"Md5\",\n",
    "        type=SearchFieldDataType.String,\n",
    "        key=True,\n",
    "    ),\n",
    "    SearchableField(\n",
    "        name=\"content\",\n",
    "        type=SearchFieldDataType.String,\n",
    "        searchable=True,\n",
    "    ),\n",
    "    SearchableField(\n",
    "        name=\"metadata_storage_path\",\n",
    "        type=SearchFieldDataType.String,\n",
    "        searchable=True,\n",
    "    ),\n",
    "    SearchableField(\n",
    "        name=\"metadata_storage_name\",\n",
    "        type=SearchFieldDataType.String,\n",
    "        searchable=True,\n",
    "    ),\n",
    "    SearchableField(\n",
    "        name=\"Citation\",\n",
    "        type=SearchFieldDataType.String,\n",
    "        searchable=True,\n",
    "    ),\n",
    "    SearchableField(\n",
    "        name=\"Accession_id\",\n",
    "        type=SearchFieldDataType.String,\n",
    "        searchable=True,\n",
    "    ),\n",
    "    SearchableField(\n",
    "        name=\"Pmid\",\n",
    "        type=SearchFieldDataType.String,\n",
    "        searchable=True,\n",
    "    )\n",
    "]\n",
    "#set our index values\n",
    "index = SearchIndex(name=s_index_name, fields=fields)\n",
    "#create our index\n",
    "index_client.create_index(index)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2db17d58-1cc3-4a9c-8872-aeca6da86638",
   "metadata": {},
   "source": [
    "Now that our index is created we can create our **Data Container** which is the storage container that holds our documents. Once this is created we then create a **Indexer** that will link our data container and our index together, it also has the option to update our index if you were to add new blobs to your storage container."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6ae3eddb-af30-40cc-95b9-bbba29109af2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a datasource\n",
    "container = SearchIndexerDataContainer(name=container_name)\n",
    "data_source_connection = SearchIndexerDataSourceConnection(\n",
    "    name=\"pubmed-datasource\", type=\"azureblob\", connection_string=connection_string, container=container\n",
    ")\n",
    "data_source = indexers_client.create_data_source_connection(data_source_connection)\n",
    "\n",
    "# create an indexer\n",
    "indexer = SearchIndexer(\n",
    "    name=\"pubmed-indexer\", data_source_name=\"pubmed-datasource\", target_index_name=s_index_name\n",
    ")\n",
    "result = indexers_client.create_indexer(indexer)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "39913a9f-5026-4b50-91f9-2acd49d2999f",
   "metadata": {},
   "source": [
    "Wait about 5 mins for the index and indexer to sync."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab4cbeb1-5ba2-4f56-bf8e-7b5875c29538",
   "metadata": {},
   "source": [
    "### Creating an index and loading large documents"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67ffaf25-4b54-498a-9b60-4fca4607e9e9",
   "metadata": {},
   "source": [
    "For our model to retrieve information from larger documents we need to split the text in our documents into smaller chucks.  This will make it easier for our model to sift through our docs to retrieve information without going over the model's token limit. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e7acee9-fa89-4d45-b577-5f374103792f",
   "metadata": {},
   "source": [
    "If you remember before, we mentioned **RAG**, the process below follows this technique using LangChain. First, we will add metadata to our docs, split our docs into chunks, and embed them. Then much like for smaller documents we will create an index, the fields in our index will be different compared to the small document index. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1727147-3cf9-4f9f-a21e-6b33a2f5640d",
   "metadata": {},
   "source": [
    "#### Adding metadata to loaded documents"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fa34e7b-99c7-4a2e-b73b-146636a98285",
   "metadata": {},
   "source": [
    "After we have our documents stored in our container we can start to load our files back. This step is necessary though redundant because we will need to embed our docs for our vector store and we need to attach metadata for each document. Although our blobs already have metadata attached to them, LangChain document loader tools only retrieves the path of our files so we need to add them back. In this case we will be using **AzureBlobStorageContainerLoader** to load in the container that holds all of our documents.\n",
    "\n",
    "If your data is in a directory within your container add the `prefix` variable to the loader definition.\n",
    "\n",
    "When we load in our documents they will be set as a tuple that is named **Documents**. This tuple will contain two items:\n",
    "- **page content:** The text or content within our document\n",
    "- **metadata:** The associated metadata which for now will only hold the source (path) to our documents"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4a11c98f-463b-48fd-84f7-f2b99f87d992",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders import AzureBlobStorageContainerLoader\n",
    "connection_string = f\"DefaultEndpointsProtocol=https;AccountName={storage_account_name};AccountKey={key}\"\n",
    "print(f\"Processing documents from {container_name}\")\n",
    "\n",
    "loader = AzureBlobStorageContainerLoader(\n",
    "    conn_str=connection_string, container=container_name\n",
    ")\n",
    "\n",
    "documents = loader.load()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b6ab068-2919-4d93-8711-15dd7eb19ada",
   "metadata": {},
   "source": [
    "Next we use our blob service client to retrieve our metadata from our blobs to add our metadata back to our loaded docs via a for loop. The metadata will consist of the source, title, and the original metadata fields from our blob."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d0a805af-98b1-4367-9aa9-de519e38bdea",
   "metadata": {},
   "outputs": [],
   "source": [
    "from azure.storage.blob import BlobServiceClient\n",
    "\n",
    "blob_service = BlobServiceClient(account_url=f'https://{storage_account_name}.blob.core.windows.net', credential=sas_token)\n",
    "\n",
    "for i in range(len(documents)):\n",
    "    #set metadata to variable\n",
    "    doc_md = documents[i].metadata\n",
    "    #gather document name from metadata to correct source formatting\n",
    "    document_name = doc_md[\"source\"].split(\"/\")[-1]\n",
    "    source = f'{container_name}/{document_name}'\n",
    "    #set the first two fields of our metadata\n",
    "    documents[i].metadata = {\"source\": source, \"title\": document_name}\n",
    "    #connect to our blob to gather the metadata\n",
    "    blob_client = blob_service.get_blob_client(container=container_name, blob=document_name)\n",
    "    other_metadata = blob_client.get_blob_properties().metadata\n",
    "    #add the blob metadata to our loaded documents\n",
    "    documents[i].metadata.update(other_metadata)\n",
    "print(f\"# of documents loaded (pre-chunking) = {len(documents)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4abb10cd-4eb9-4678-aa18-b4f168f1d927",
   "metadata": {},
   "source": [
    "Lets look at our metadata!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "db06d582-b84d-4b9e-9ff1-1695e37bb50e",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(documents[0].metadata)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57e21813-35fa-485a-ac2e-41d38676d87e",
   "metadata": {},
   "source": [
    "#### Splitting our documents"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9dfb34dc-4b8d-4c92-9e64-f94926bd8793",
   "metadata": {},
   "source": [
    "Splitting our data into chucks will help our vector store parse through our data faster and efficiently.\n",
    "\n",
    "For this step we will be using langchains **RecursiveCharacterTextSplitter**. This text splitter allows us to set the size of each chunk, if the chunks should have any text overlap (this is to help the model bridge some the chunks to make sense of them), and where best to separate texts. Each chunk will have the same metadata as the original document they came from."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3ae5e1eb-b2df-465c-a37d-3ddbad526602",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "\n",
    "text_splitter = RecursiveCharacterTextSplitter(\n",
    "    # Set a small chunk size.\n",
    "    chunk_size = 2000,\n",
    "    chunk_overlap  = 20,\n",
    "    length_function = len,\n",
    "    separators=[\"\\n\\n\", \"\\n\", \".\", \"!\", \"?\", \",\", \" \", \"\"]\n",
    ")\n",
    "chunk = text_splitter.split_documents(documents)\n",
    "\n",
    "print(f\"# of documents loaded (pre-chunking) = {len(chunks)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b70848ce-02ee-4c2c-9824-d231a4d9037a",
   "metadata": {},
   "source": [
    "lets look at one of our chunks!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a2ee4576-6d4b-4f70-8cf4-1f52abcb8208",
   "metadata": {},
   "outputs": [],
   "source": [
    "chunks[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c8ae84a-3e21-41b0-85d0-7093c563bb90",
   "metadata": {},
   "source": [
    "#### Create an Index with a Vector Field"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10628e98-5486-4222-ad36-52ae4ad3a5c0",
   "metadata": {},
   "source": [
    "For our index we will be adding in a **content_vector** field which represents each chuck embedded. **Embedding** means that we are converting our text into a **numerical vectors** that will help our model find similar objects like documents that hold similar texts or find similar photos based on the numbers assigned to the object, basically capturing texts meaning and relationship through numbers. Depending on the model you choose you have to find an embedder that is compatible to our model. Since we are using a OpenAI model the compatible embedding model will be **text-embedding-ada-002**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "11fbd018-4197-4641-bbc3-9feff8c4b4e9",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from langchain_openai import AzureOpenAIEmbeddings\n",
    "from azure.search.documents.indexes.models import SearchIndex\n",
    "from azure.search.documents.indexes import SearchIndexClient\n",
    "from azure.core.credentials import AzureKeyCredential\n",
    "from azure.search.documents.indexes.models import (\n",
    "    SearchableField,\n",
    "    SearchField,\n",
    "    SearchFieldDataType,\n",
    "    SimpleField,\n",
    "    TextWeights,\n",
    "    VectorSearch,\n",
    "    VectorSearchProfile,\n",
    "    HnswAlgorithmConfiguration,\n",
    "    ComplexField\n",
    ")\n",
    "\n",
    "endpoint = \"https://{}.search.windows.net/\".format(service_name)\n",
    "index_client = SearchIndexClient(endpoint, AzureKeyCredential(search_key))\n",
    "\n",
    "#Setup embeddings model\n",
    "os.environ[\"AZURE_OPENAI_API_KEY\"] = \"<Your Azure OpenAI API Key>\"\n",
    "os.environ[\"AZURE_OPENAI_ENDPOINT\"] = \"<Your Azure OpenAI Endpoint>\"\n",
    "\n",
    "embeddings = AzureOpenAIEmbeddings(\n",
    "    azure_deployment=\"text-embedding-ada-002\",\n",
    "    chunk_size=10, #processing our chunks in batches of 10\n",
    ")\n",
    "embedding_function = embeddings.embed_query"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8643cfd-861c-4d6b-92cf-f21f4e15ccfb",
   "metadata": {},
   "source": [
    "Now we can create our fields. You will notice that they are different from the small documents fields. Because we are using LangChain to add our chunks to our index all our metadata will be held in a field called metadata, the page_content will be held in content, and langchan will create ids for each chunk.\n",
    "\n",
    "Another field you might have noticed is the **content_vector** field this field will hold the content that has been embedded. To create this field we have to set a vector profile which dictates what algorithm we will have our vector store use to find text that are similar to each other (find the nearest neighbors) for this profile we will be using the **Hierarchical Navigable Small World (HNSW) algorithm**.\n",
    "\n",
    "- **SimpleField:** A field that you can retrieve values but not search them this is ideal for keys which is a unique ID for each blob. Here we are setting the id value as our key.\n",
    "- **SearchableField:** A field that allows you to retrieve and search values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8b542ff8-221a-4e7a-8fca-d5ce09e5976d",
   "metadata": {},
   "outputs": [],
   "source": [
    "fields = [\n",
    "    SimpleField(\n",
    "        name=\"id\",\n",
    "        type=SearchFieldDataType.String,\n",
    "        key=True\n",
    "    ),\n",
    "    SearchableField(\n",
    "        name=\"content\",\n",
    "        type=SearchFieldDataType.String,\n",
    "        searchable=True\n",
    "    ),\n",
    "    SearchField(\n",
    "        name=\"content_vector\",\n",
    "        type=SearchFieldDataType.Collection(SearchFieldDataType.Single),\n",
    "        searchable=True,\n",
    "        vector_search_dimensions=len(embedding_function(\"text\")),\n",
    "        vector_search_profile_name=\"my-vector-config\"\n",
    "    ),\n",
    "    SearchableField(name=\"metadata\", type=SearchFieldDataType.String, searchable=True),\n",
    "]\n",
    "\n",
    "vector_search = VectorSearch(\n",
    "    profiles=[VectorSearchProfile(name=\"my-vector-config\", algorithm_configuration_name=\"my-algorithms-config\")],\n",
    "    algorithms=[HnswAlgorithmConfiguration(name=\"my-algorithms-config\")],\n",
    ")\n",
    "    \n",
    "l_index_name = \"pubmed-index-largedocs\"\n",
    "index = SearchIndex(name=l_index_name, fields=fields, vector_search=vector_search)\n",
    "index_client.create_index(index)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e91998d-376f-4050-8080-50ee3c473ea6",
   "metadata": {},
   "source": [
    "Define your vector store for langchain."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1547a74-1727-4c14-8856-842b161fe201",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.vectorstores.azuresearch import AzureSearch\n",
    "\n",
    "vector_store = AzureSearch(\n",
    "    azure_search_endpoint=endpoint,\n",
    "    azure_search_key=search_key,\n",
    "    index_name=l_index_name,\n",
    "    embedding_function=embedding_function\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe658444-b194-4495-a53c-c39f98498178",
   "metadata": {},
   "source": [
    "#### Embedding and Adding Data to Vector Store"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e3bfb5b-a3a6-4156-bca3-394774a94565",
   "metadata": {},
   "source": [
    "For our chunks to be read by our embedding model we need split the tuple within each chunk, remember that the chunks consists of tuple called **Document** that contains **page content** and **metadata**. The code below loops through the chunks and splits the page_content and metadata saving them as separate variable lists."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1ba20bef-5d38-4a99-9374-7642563d8716",
   "metadata": {},
   "outputs": [],
   "source": [
    "texts = [doc.page_content for doc in chunks]\n",
    "metadatas = [doc.metadata for doc in chunks]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62095449-f2bd-4038-ac6a-e1569887680e",
   "metadata": {},
   "source": [
    "Finally we can upload our split content and metadata to our vector store! This may take 10 to 20 mins depending on how large your dataset is."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "27eff5e6-ff27-4ea8-9e6a-c0c5c05a245a",
   "metadata": {},
   "outputs": [],
   "source": [
    "vector_store.add_texts(texts=texts, metadatas=metadatas)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07b3bc6b-8c43-476f-a662-abda830dc2da",
   "metadata": {},
   "source": [
    "### Creating an inference script "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ba2291e-109e-4120-ad10-5dbfd341a07b",
   "metadata": {},
   "source": [
    "In order for us to fluidly send input and receive outputs from our chatbot we need to create an **inference script** that will format inputs in a way that the chatbot can understand and format outputs in a way we can understand. We will also be supplying instructions to the chatbot through the script.\n",
    "\n",
    "Our script will utilize **LangChain** tools and packages to enable our model to:\n",
    "- **Connect to sources of context** (e.g. providing our model with tasks and examples)\n",
    "- **Rely on reason** (e.g. instruct our model how to answer based on provided context)\n",
    "\n",
    "The following tools must be installed via your terminal `pip install \"langchain\" \"langchain-openai\" \"langchain-community\" \"xmltodict\" \"openai\"` and the general inference script must be run on the terminal via the command `python YOUR_SCRIPT.py`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad374085-c4b1-4083-85a5-90cba35846d6",
   "metadata": {},
   "source": [
    "The first section below will list all the tools that are required. \n",
    "-  **PubMedRetriever:** Utilizes the langchain retriever tool to specifically retrieve PubMed documents from the PubMed API.\n",
    "- **AzureCognitiveSearchRetriever:** Connects to Azure AI Search to be used as a langchain retriever tool by specifically retrieving embedded documents stored in your vector store.\n",
    "- **AzureChatOpenAI:** Connects to your deployed OpenAI model. \n",
    "- **ConversationalRetrievalChain:** Allows the user to construct a conversation with the model and retrieves the outputs while sending inputs to the model.\n",
    "- **PromptTemplate:** Allows the user to prompt the model to provide instructions, best method for zero and few shot prompting"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f0ad48d-c6c8-421a-a48b-88e979d15b57",
   "metadata": {},
   "source": [
    "```python\n",
    "from langchain_community.retrievers import PubMedRetriever\n",
    "from langchain_community.retrievers import AzureCognitiveSearchRetriever\n",
    "from langchain_openai import AzureChatOpenAI\n",
    "from langchain.chains import ConversationalRetrievalChain\n",
    "from langchain.prompts import PromptTemplate\n",
    "import sys\n",
    "import json\n",
    "import os\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "900f4c31-71cd-4f39-8bfc-de098bdbaafc",
   "metadata": {},
   "source": [
    "Second will build a class that will hold the functions we need to send inputs and retrieve outputs from our model. For the beginning of our class we will establish some colors to our text conversation with our chatbot which we will utilize later."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "decbb901-f811-4b8e-a956-4c8c7f914ae2",
   "metadata": {},
   "source": [
    "```python\n",
    "class bcolors:\n",
    "    HEADER = '\\033[95m'\n",
    "    OKBLUE = '\\033[94m'\n",
    "    OKCYAN = '\\033[96m'\n",
    "    OKGREEN = '\\033[92m'\n",
    "    WARNING = '\\033[93m'\n",
    "    FAIL = '\\033[91m'\n",
    "    ENDC = '\\033[0m'\n",
    "    BOLD = '\\033[1m'\n",
    "    UNDERLINE = '\\033[4m'\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba36d057-5189-4075-a243-18996c6fc932",
   "metadata": {},
   "source": [
    "We need to extract environmental variables to connect to our Open AI model. They will be :\n",
    "- OpenAI Key\n",
    "- OpenAI Endpoint (url)\n",
    "- Open AI Deployment Name\n",
    "\n",
    "If you are using Azure AI Search instead of the PubMed API we need to create a function that will gather the necessary information to connect to our vector store, which will be the:\n",
    "- Azure AI Search Service Name\n",
    "- Azure AI Search Index Name\n",
    "- Azure AI Search API Key"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f7a244a-7e71-40d3-ae78-8e166dd3c7ee",
   "metadata": {},
   "source": [
    "```python\n",
    "def build_chain():\n",
    "    os.getenv(\"AZURE_OPENAI_API_KEY\")\n",
    "    os.getenv(\"AZURE_OPENAI_ENDPOINT\")\n",
    "    os.getenv(\"AZURE_COGNITIVE_SEARCH_SERVICE_NAME\")\n",
    "    os.getenv(\"AZURE_COGNITIVE_SEARCH_INDEX_NAME\")\n",
    "    os.getenv(\"AZURE_COGNITIVE_SEARCH_API_KEY\")\n",
    "    AZURE_OPENAI_DEPLOYMENT_NAME = os.environ[\"AZURE_OPENAI_DEPLOYMENT_NAME\"]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dab1012f-ed20-47b9-9162-924e03e836d5",
   "metadata": {},
   "source": [
    "Now we can define our OpenAI model that has been predeployed. If you want to modify parameters, you can control them via:\n",
    "- Temperature: Controls randomness, higher values increase diversity meaning a more unique response make the model to think harder. Must be a number from 0 to 1, 0 being less unique.\n",
    "- Max Output Tokens: Limit of tokens outputted by the model.(optional: can assign if you like)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8cadb1af-2c46-4ab1-92f9-6e0861f83324",
   "metadata": {},
   "source": [
    "```python\n",
    "llm = AzureChatOpenAI(\n",
    "    openai_api_version=\"2023-05-15\",\n",
    "    azure_deployment=AZURE_OPENAI_DEPLOYMENT_NAME,\n",
    "    temperature = 0.5\n",
    "    #max_tokens = 3000\n",
    ")\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c44b4f91-0c64-459b-a6e9-8a955c0797c7",
   "metadata": {},
   "source": [
    "Make sure you use either the PubMed retreiver from LangChain or the Azure AI Search Index, but not both.\n",
    "\n",
    "If using Azure AI Search we need to specify what we are retrieving for our model to review, in this case it is the **content** part of our scheme we set within our index. We also set **'top_k'** to 2 meaning that our retriever will retrieve 2 documents that are the most similar to our query."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "21c61724-23d3-4b49-8c72-cbd208bdb5df",
   "metadata": {},
   "source": [
    "```python\n",
    "retriever= PubMedRetriever()\n",
    "\n",
    "#only if using Azure AI Search as a retriever\n",
    "\n",
    "retriever = AzureCognitiveSearchRetriever(content_key=\"content\", top_k=2)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec8e464a-0931-444a-aa58-09ee0c4c9884",
   "metadata": {},
   "source": [
    "Here we are constructing our **prompt_template**, this is where we can try zero-shot or few-shot prompting. Only add one method per script."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4431051e-0e84-408e-9821-f50a9b88c9c1",
   "metadata": {},
   "source": [
    "#### Zero-shot prompting\n",
    "\n",
    "Zero-shot prompting does not require any additional training, but rather it asks a pre-trained language model to respond directly to a prompt, similar to if you were to ask Chat GPT a quick question without context. The model relies on its general language understanding and the patterns it has learned during its training to produce relevant output. In our script we grounded our model via a **retriever** to make sure it gathers information from our input data (PubMed API or Azure AI Search). \n",
    "\n",
    "See below that the task is more like instructions notifying our model they will be asked questions which it will answer based on the info of the scientific documents provided from the index provided (this can be the PubMed API or Vector Search index). All of this information is established as a **prompt template** for our model to receive."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0316dc5-6274-4a5e-92e4-3d266ed6a4df",
   "metadata": {},
   "source": [
    "```python\n",
    "prompt_template = \"\"\"\n",
    "  Ignore everything before.\n",
    "  \n",
    "  Instructions:\n",
    "  I will provide you with research papers on a specific topic in English, and you will create a cumulative summary. \n",
    "  The summary should be concise and should accurately and objectively communicate the takeaway of the papers related to the topic. \n",
    "  You should not include any personal opinions or interpretations in your summary, but rather focus on objectively presenting the information from the papers. \n",
    "  Your summary should be written in your own words and ensure that your summary is clear, concise, and accurately reflects the content of the original papers.\n",
    "  \n",
    "  {question} Answer \"don't know\" if not present in the document. \n",
    "  {context}\n",
    "  Solution:\"\"\"\n",
    "  PROMPT = PromptTemplate(\n",
    "      template=prompt_template, input_variables=[\"context\", \"question\"],\n",
    "  )\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "edbe7032-8507-4d07-baab-1b3bf0e92074",
   "metadata": {},
   "source": [
    "#### One-shot and Few-shot Prompting"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5614ea04-e1f8-4941-ae16-4359f718f98f",
   "metadata": {},
   "source": [
    "One and few-shot prompting are similar to one-shot prompting, in addition to giving our model a task just like before we have also supplied an example of how we want the model to respond. See below for an example. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ffb9669-5b77-4d9b-9f4e-a0d3a18b0fae",
   "metadata": {},
   "source": [
    "```python\n",
    "prompt_template = \"\"\"\n",
    "  Instructions:\n",
    "  I will provide you with research papers on a specific topic in English, and you will create a cumulative summary. \n",
    "  The summary should be concise and should accurately and objectively communicate the takeaway of the papers related to the topic. \n",
    "  You should not include any personal opinions or interpretations in your summary, but rather focus on objectively presenting the information from the papers. \n",
    "  Your summary should be written in your own words and ensure that your summary is clear, concise, and accurately reflects the content of the original papers.\n",
    "  Examples:\n",
    "  Question: What is a cell?\n",
    "  Answer: '''\n",
    "  Cell, in biology, the basic membrane-bound unit that contains the fundamental molecules of life and of which all living things are composed. \n",
    "  Sources: \n",
    "  Chow, Christopher , Laskey, Ronald A. , Cooper, John A. , Alberts, Bruce M. , Staehelin, L. Andrew , \n",
    "  Stein, Wilfred D. , Bernfield, Merton R. , Lodish, Harvey F. , Cuffe, Michael and Slack, Jonathan M.W.. \n",
    "  \"cell\". Encyclopedia Britannica, 26 Sep. 2023, https://www.britannica.com/science/cell-biology. Accessed 9 November 2023.\n",
    "  '''\n",
    "  \n",
    "  {question} Answer \"don't know\" if not present in the document. \n",
    "  {context}\n",
    "  \n",
    "\n",
    "  \n",
    "  Solution:\"\"\"\n",
    "  PROMPT = PromptTemplate(\n",
    "      template=prompt_template, input_variables=[\"context\", \"question\"],\n",
    "  )\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82c66d53-97b2-46dc-a466-70a3d3bee4a7",
   "metadata": {},
   "source": [
    "The following set of commands control the chat history essentially telling the model to expect another question after it finishes answering the previous one. Follow up questions can contain references to past chat history so the **ConversationalRetrievalChain** combines the chat history and the followup question into a standalone question, then looks up relevant documents from the retriever, and finally passes those documents and the question to a question-answering chain to return a response.\n",
    "\n",
    "All of these pieces such as our conversational chain, prompt, and chat history are passed through a function called **run_chain** so that our model can return a response. We have also set the length of our chat history to one, meaning that our model can only refer to the pervious conversation as a reference."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fda4d33b-60f2-4462-a8e6-bbce7f8a7b07",
   "metadata": {},
   "source": [
    "```python\n",
    "condense_qa_template = \"\"\"\n",
    "  Chat History:\n",
    "  {chat_history}\n",
    "  Here is a new question for you: {question}\n",
    "  Standalone question:\"\"\"\n",
    "  standalone_question_prompt = PromptTemplate.from_template(condense_qa_template)\n",
    " \n",
    "    qa = ConversationalRetrievalChain.from_llm(\n",
    "        llm=llm, \n",
    "        retriever=retriever, \n",
    "        condense_question_prompt=standalone_question_prompt, \n",
    "        return_source_documents=True, \n",
    "        combine_docs_chain_kwargs={\"prompt\":PROMPT},\n",
    "        )\n",
    "      return qa\n",
    "\n",
    "def run_chain(chain, prompt: str, history=[]):\n",
    "    print(prompt)\n",
    "    return chain({\"question\": prompt, \"chat_history\": history})\n",
    "\n",
    "MAX_HISTORY_LENGTH = 1 #increase to refer to more pervious chats\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8f1ef8d-66fe-4f84-933b-af2d730bd114",
   "metadata": {},
   "source": [
    "The final part of our script utilizes our class and incorporates colors to add a bit of flare to our conversation with our model. The model when first initialized should greet the user asking **\"Hello! How can I help you?\"** then instructs the user to ask a question or exit the session **\"Ask a question, start a New search: or CTRL-D to exit.\"**. With every question submitted to the model it is labeled as a **new search** we then run the run_chain function to get the models response or answer and add the response to the **chat history**. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1aa6ef65-ced4-445e-875c-7fee3483b81d",
   "metadata": {},
   "source": [
    "```python\n",
    "if __name__ == \"__main__\":\n",
    "  chat_history = []\n",
    "  qa = build_chain()\n",
    "  print(bcolors.OKBLUE + \"Hello! How can I help you?\" + bcolors.ENDC)\n",
    "  print(bcolors.OKCYAN + \"Ask a question, start a New search: or CTRL-D to exit.\" + bcolors.ENDC)\n",
    "  print(\">\", end=\" \", flush=True)\n",
    "  for query in sys.stdin:\n",
    "    if (query.strip().lower().startswith(\"new search:\")):\n",
    "      query = query.strip().lower().replace(\"new search:\",\"\")\n",
    "      chat_history = []\n",
    "    elif (len(chat_history) == MAX_HISTORY_LENGTH):\n",
    "      chat_history.pop(0)\n",
    "    result = run_chain(qa, query, chat_history)\n",
    "    chat_history.append((query, result[\"answer\"]))\n",
    "    print(bcolors.OKGREEN + result['answer'] + bcolors.ENDC)  \n",
    "     if 'source_documents' in result:\n",
    "      print(bcolors.OKGREEN + 'Sources:')\n",
    "      for d in result['source_documents']:\n",
    "            ###Use this for Azure Search AI\n",
    "            dict_meta=json.loads(d.metadata['metadata'])\n",
    "            print(dict_meta['source'])\n",
    "            ###\n",
    "            #Use this for PubMed retriever:\n",
    "            #print(\"PubMed UID: \"+d.metadata[\"uid\"])\n",
    "    print(bcolors.ENDC)\n",
    "    print(bcolors.OKCYAN + \"Ask a question, start a New search: or CTRL-D to exit.\" + bcolors.ENDC)\n",
    "    print(\">\", end=\" \", flush=True)\n",
    "  print(bcolors.OKBLUE + \"Bye\" + bcolors.ENDC)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1abcbd48-bb84-4310-b8eb-ad87850a8649",
   "metadata": {},
   "source": [
    "Running our script in the terminal will require us to export the following global variables before using the command `python NAME_OF_SCRIPT.py`. Example scripts are also ready to use within our 'example_scripts' folder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ba97df23-6893-438d-8a67-cb7dbf83e407",
   "metadata": {},
   "outputs": [],
   "source": [
    "#retreive info to allow langchain to connect to Azure Search AI\n",
    "print(service_name)\n",
    "print(l_index_name)\n",
    "print(s_index_name)\n",
    "print(search_key)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7eab00a3-54ff-4873-8d25-eaf8bd18a2e6",
   "metadata": {},
   "outputs": [],
   "source": [
    "#enter the global variables in your terminal\n",
    "export AZURE_OPENAI_API_KEY='<AZURE_OPENAI_API_KEY>' \\\n",
    "export AZURE_OPENAI_ENDPOINT='<AZURE_OPENAI_ENDPOINT>' \\\n",
    "export AZURE_OPENAI_DEPLOYMENT_NAME='<AZURE_OPENAI_DEPLOYMENT_NAME>' \\\n",
    "export AZURE_COGNITIVE_SEARCH_SERVICE_NAME='<AZURE_COGNITIVE_SEARCH_SERVICE_NAME>' \\\n",
    "export AZURE_COGNITIVE_SEARCH_INDEX_NAME='<AZURE_COGNITIVE_SEARCH_INDEX_NAME>' \\\n",
    "export AZURE_COGNITIVE_SEARCH_API_KEY='<AZURE_COGNITIVE_SEARCH_API_KEY>' "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbe127e6-c0b1-4e07-ad56-38c30a9bf858",
   "metadata": {},
   "source": [
    "You should see similar results on the terminal. In this example we ask the chatbot to summarize one of our documents!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80c8fb4b-e74f-4e8d-892b-0f913eff747d",
   "metadata": {},
   "source": [
    "![PubMed Chatbot Results](../../../docs/images/azure_chatbot.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67776cc7",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "Here we built a chat bot using LangChain and Azure Open AI. Key skills you learned were to : \n",
    "+ Create embeddings and a vector store using Azure AI Search\n",
    "+ Use the PubMed API via LangChain\n",
    "+ Send prompts to the LLM and capture chat history\n",
    "+ Experiment with zero-shot and one/few-shot prompting"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a178c1c6-368a-48c5-8beb-278443b685a2",
   "metadata": {},
   "source": [
    "## Clean up"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ec06a34-dc47-453f-b519-424804fa2748",
   "metadata": {},
   "source": [
    "**Warning:** Dont forget to delete the resources we just made to avoid accruing additional costs!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c307bb17-757a-4579-a0d8-698eb1bb3f2e",
   "metadata": {},
   "outputs": [],
   "source": [
    "#delete search service this will also delete any indexes, datastore, and indexers\n",
    "! az search service delete --name {service_name} --resource-group {resource_group} -y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "280cea0a-a8fc-494e-8ce4-afb65847a222",
   "metadata": {},
   "outputs": [],
   "source": [
    "#delete container\n",
    "! az storage container delete -n {container_name} --account-name {storage_account_name}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6928d95d-d7ec-43f6-9135-79fcfc9520d9",
   "metadata": {},
   "source": [
    "Dont forget to also delete or undeploy your LLM and embeddings model within Azure AI Studio."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d7350f02-aaf2-444d-b32a-c414d7d857ee",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {},
 "nbformat": 4,
 "nbformat_minor": 5
}