t81_559_class_10_2_streamlit_intro.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kxu1Gfhx1pHg"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/jeffheaton/app_generative_ai/blob/main/t81_559_class_10_2_streamlit_intro.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZbNbAV281pHh"
      },
      "source": [
        "# T81-559: Applications of Generative Artificial Intelligence\n",
        "**Module 10: StreamLit**\n",
        "* Instructor: [Jeff Heaton](https://sites.wustl.edu/jeffheaton/), McKelvey School of Engineering, [Washington University in St. Louis](https://engineering.wustl.edu/Programs/Pages/default.aspx)\n",
        "* For more information visit the [class website](https://sites.wustl.edu/jeffheaton/t81-558/)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HwnvSYEQ1pHi"
      },
      "source": [
        "# Module 10 Material\n",
        "\n",
        "Module 10: StreamLit\n",
        "\n",
        "* Part 10.1: Running StreamLit in Google Colab [[Video]]() [[Notebook]](t81_559_class_10_1_streamlit.ipynb)\n",
        "* **Part 10.2: StreamLit Introduction** [[Video]]() [[Notebook]](t81_559_class_10_2_streamlit_intro.ipynb)\n",
        "* Part 10.3: Understanding Streamlit State [[Video]]() [[Notebook]](t81_559_class_10_3_streamlit_state.ipynb)\n",
        "* Part 10.4: Creating a Chat Application [[Video]]() [[Notebook]](t81_559_class_10_4_chat.ipynb)\n",
        "* Part 10.5: MultiModal Chat Application [[Video]]() [[Notebook]](t81_559_class_10_5_chat_multimodal.ipynb)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XLFov09h18yC"
      },
      "source": [
        "# Google CoLab Instructions\n",
        "\n",
        "The following code ensures that Google CoLab is running and maps Google Drive if needed."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "AWGARRT92DrA"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "\n",
        "try:\n",
        "    from google.colab import drive, userdata\n",
        "    COLAB = True\n",
        "    print(\"Note: using Google CoLab\")\n",
        "except:\n",
        "    print(\"Note: not using Google CoLab\")\n",
        "    COLAB = False\n",
        "\n",
        "# OpenAI Secrets\n",
        "if COLAB:\n",
        "    os.environ[\"OPENAI_API_KEY\"] = userdata.get('OPENAI_API_KEY')\n",
        "\n",
        "# Install needed libraries in CoLab\n",
        "if COLAB:\n",
        "    !pip install langchain langchain_openai openai streamlit"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v2MPPX0c1pHi"
      },
      "source": [
        "# Part 10.2: StreamLit Introduction\n",
        "\n",
        "\n",
        "In this module, we will explore Streamlit, a powerful tool for creating interactive web applications with minimal code. We will go through two hands-on examples to demonstrate its capabilities:\n",
        "\n",
        "1. Loan Amortization Table Generator: In the first example, we will use Streamlit's user interface elements to build a simple loan amortization table generator. You will see how easy it is to gather user input, process it, and display the results in an interactive format.\n",
        "\n",
        "2. Multimodal Image Transformation: The second example showcases the power of multimodal models. We'll create an application that allows you to upload images and transforms them into cartoon versions using AI. This will give you an understanding of how Streamlit can handle image processing and display results effectively.\n",
        "\n",
        "By the end of this module, you'll have a solid understanding of how to use Streamlit to build both data-driven and AI-enhanced applications.\n",
        "\n",
        "## Loan Calculation Example\n",
        "\n",
        "\n",
        "This code creates a loan amortization calculator using Streamlit, a library for building interactive web applications in Python. The application consists of a function to calculate the amortization schedule and a simple user interface for inputting loan details and displaying results.\n",
        "\n",
        "The function ```calculate_amortization``` takes three parameters: ```loan_amount```, ```annual_rate```, and ```years```. It first calculates the monthly interest rate and the total number of payments over the loan's term. Using these, it computes the monthly payment using the standard loan amortization formula. It then iteratively builds an amortization schedule, calculating the interest and principal portions of each payment and updating the remaining balance. The results are stored in a list, which is then converted into a pandas ```DataFrame`` with columns for each month, payment amount, principal paid, interest paid, and remaining balance.\n",
        "\n",
        "The Streamlit app's interface starts with a title: 'Loan Amortization Calculator'. It provides input fields for the user to specify the loan amount, annual interest rate, and loan term in years. These inputs use Streamlit's number_input method, which allows users to adjust values easily.\n",
        "\n",
        "Upon clicking the \"Calculate Amortization Table\" button, the application calls the calculate_amortization function with the input values and displays the resulting DataFrame. It also displays the monthly payment amount. Additionally, the application generates a downloadable CSV file of the amortization table, allowing users to save the results locally. This functionality is provided by the st.download_button method, which takes the DataFrame's CSV data and offers it as a downloadable file."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%writefile app.py\n",
        "\n",
        "import streamlit as st\n",
        "import pandas as pd\n",
        "\n",
        "# Function to calculate loan amortization\n",
        "def calculate_amortization(loan_amount, annual_rate, years):\n",
        "    monthly_rate = annual_rate / 12 / 100\n",
        "    num_payments = years * 12\n",
        "    monthly_payment = loan_amount * (monthly_rate * (1 + monthly_rate) ** num_payments) / ((1 + monthly_rate) ** num_payments - 1)\n",
        "\n",
        "    # Create amortization schedule\n",
        "    amortization_data = []\n",
        "    balance = loan_amount\n",
        "    for i in range(1, num_payments + 1):\n",
        "        interest_payment = balance * monthly_rate\n",
        "        principal_payment = monthly_payment - interest_payment\n",
        "        balance -= principal_payment\n",
        "        amortization_data.append([i, monthly_payment, principal_payment, interest_payment, max(balance, 0)])\n",
        "\n",
        "    # Create DataFrame\n",
        "    df = pd.DataFrame(amortization_data, columns=['Month', 'Payment', 'Principal', 'Interest', 'Balance'])\n",
        "    return df\n",
        "\n",
        "# Streamlit App\n",
        "st.title('Loan Amortization Calculator')\n",
        "\n",
        "# Input Fields\n",
        "loan_amount = st.number_input('Loan Amount', value=300000.0, min_value=0.0, step=1000.0)\n",
        "annual_rate = st.number_input('Annual Interest Rate (%)', value=7.5, min_value=0.0, step=0.1)\n",
        "years = st.number_input('Loan Term (years)', value=30, min_value=1, step=1)\n",
        "\n",
        "# Calculate amortization table\n",
        "if st.button('Calculate Amortization Table'):\n",
        "    amortization_df = calculate_amortization(loan_amount, annual_rate, years)\n",
        "    st.write(f'Monthly Payment: ${amortization_df[\"Payment\"][0]:,.2f}')\n",
        "    st.dataframe(amortization_df)\n",
        "\n",
        "    # Downloadable CSV\n",
        "    csv = amortization_df.to_csv(index=False)\n",
        "    st.download_button(label=\"Download Amortization Table as CSV\", data=csv, file_name='amortization_schedule.csv', mime='text/csv')\n"
      ],
      "metadata": {
        "id": "_d1jBuL-wiyh"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Next, we obtain the password for our StreamLit server we are about to launch."
      ],
      "metadata": {
        "id": "OWnPCS8v_iYS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!curl https://loca.lt/mytunnelpassword"
      ],
      "metadata": {
        "id": "Uisp3omywt5V"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We launch the StreamLit server and obtain its URL. You will need the above password when you access the URL it gives you."
      ],
      "metadata": {
        "id": "AZHCdk7j_mUQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!streamlit run app.py server1 &>/content/logs.txt &\n",
        "!npx --yes localtunnel --port 8501"
      ],
      "metadata": {
        "id": "fap8dxKTw1xF"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Cartoon Image\n",
        "\n",
        "This code creates a Streamlit application that transforms uploaded images into cartoon-like versions using a multimodal Large Language Model (LLM) and image generation tools. The application allows users to upload an image, processes it with an AI model, and generates a cartoonified version, which can then be downloaded.\n",
        "\n",
        "The core functionality is encapsulated in the modify_image function. This function starts by initializing the GPT model, specifically \"gpt-4o-mini.\" When an image is uploaded, it's converted into a base64-encoded string to facilitate communication with the LLM. A message is created combining a text prompt and the base64-encoded image, which is then sent to the GPT model to generate a modified prompt. The response from the model provides a prompt intended to render the image in a cartoon style.\n",
        "\n",
        "After obtaining this cartoon prompt, the function uses OpenAI's DALL-E model (\"dall-e-3\") to generate a new image based on the prompt. The image generation process returns a URL, which is then fetched and converted into a PIL Image object for further processing.\n",
        "\n",
        "The Streamlit app interface begins with a title: \"Cartoonify Image with Multimodal LLM.\" It provides an image uploader using the st.file_uploader method, allowing users to upload images in JPEG or PNG format. Once an image is uploaded, it is displayed using Streamlit's st.image method.\n",
        "\n",
        "The modify_image function is then called to process the uploaded image and transform it into a cartoon style. While this process is happening, a message (\"Generating cartoon version...\") is shown to indicate ongoing activity. The cartoonified image is then displayed in the app.\n",
        "\n",
        "Lastly, the app offers the option to download the cartoonified image. This is achieved by saving the modified image into a buffer and using Streamlit's st.download_button to allow the user to download it in JPEG format. This provides a seamless way for users to not only view the transformed image but also save it locally."
      ],
      "metadata": {
        "id": "gJAC0eCn3Qaf"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%%writefile app.py\n",
        "\n",
        "import streamlit as st\n",
        "from langchain_core.messages import HumanMessage\n",
        "from langchain_openai import ChatOpenAI\n",
        "import base64\n",
        "import httpx\n",
        "from openai import OpenAI\n",
        "from PIL import Image\n",
        "from io import BytesIO\n",
        "import matplotlib.pyplot as plt\n",
        "import requests\n",
        "\n",
        "def modify_image(image, prompt):\n",
        "    # Initialize the GPT model\n",
        "    model = ChatOpenAI(model=\"gpt-4o-mini\")\n",
        "\n",
        "    # Convert the uploaded image to base64\n",
        "    buffered = BytesIO()\n",
        "    image.save(buffered, format=\"JPEG\")\n",
        "    image_data = base64.b64encode(buffered.getvalue()).decode(\"utf-8\")\n",
        "\n",
        "    # Create a message with both text and the image\n",
        "    message = HumanMessage(\n",
        "        content=[\n",
        "            {\"type\": \"text\", \"text\": prompt},\n",
        "            {\"type\": \"image_url\", \"image_url\": {\"url\": f\"data:image/jpeg;base64,{image_data}\"}},\n",
        "        ],\n",
        "    )\n",
        "\n",
        "    # Get response with a modified prompt from GPT\n",
        "    response = model.invoke([message])\n",
        "    cartoon_prompt = response.content\n",
        "\n",
        "    # Initialize the DALL-E model to generate the image\n",
        "    client = OpenAI()\n",
        "\n",
        "    # Generate the image based on the GPT-generated cartoon prompt\n",
        "    response = client.images.generate(\n",
        "        model=\"dall-e-3\",\n",
        "        prompt=cartoon_prompt,\n",
        "        size=\"1024x1024\",\n",
        "        quality=\"standard\",\n",
        "        n=1,\n",
        "    )\n",
        "\n",
        "    # Get the image URL from DALL-E\n",
        "    image_url = response.data[0].url\n",
        "\n",
        "    # Fetch the generated image\n",
        "    response2 = requests.get(image_url)\n",
        "    img = Image.open(BytesIO(response2.content))\n",
        "\n",
        "    return img\n",
        "\n",
        "# Streamlit app\n",
        "st.title(\"Cartoonify Image with Multimodal LLM\")\n",
        "\n",
        "# Image upload\n",
        "uploaded_image = st.file_uploader(\"Upload an image\", type=[\"jpg\", \"jpeg\", \"png\"])\n",
        "\n",
        "if uploaded_image is not None:\n",
        "    # Open the image using PIL\n",
        "    image = Image.open(uploaded_image)\n",
        "\n",
        "    # Display the original image\n",
        "    st.image(image, caption='Uploaded Image', use_column_width=True)\n",
        "\n",
        "    # Modify the image to look like a cartoon\n",
        "    st.write(\"Generating cartoon version...\")\n",
        "    cartoon_img = modify_image(image, \"Output a prompt that would render this image as a cartoon.\")\n",
        "\n",
        "    # Display the cartoonified image\n",
        "    st.image(cartoon_img, caption='Cartoonified Image', use_column_width=True)\n",
        "\n",
        "    # Provide an option to download the cartoonified image\n",
        "    buffered = BytesIO()\n",
        "    cartoon_img.save(buffered, format=\"JPEG\")\n",
        "    st.download_button(\n",
        "        label=\"Download Cartoon Image\",\n",
        "        data=buffered.getvalue(),\n",
        "        file_name=\"cartoon_image.jpg\",\n",
        "        mime=\"image/jpeg\"\n",
        "    )\n"
      ],
      "metadata": {
        "id": "QGOuOGgD3Syp"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Next, we obtain the password for our StreamLit server we are about to launch."
      ],
      "metadata": {
        "id": "bkmzsav3AFS5"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!curl https://loca.lt/mytunnelpassword"
      ],
      "metadata": {
        "id": "AU4hiwY-3o2m"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We launch the StreamLit server and obtain its URL. You will need the above password when you access the URL it gives you."
      ],
      "metadata": {
        "id": "o_Aah1tqAJ42"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!streamlit run app.py server1 &>/content/logs.txt &\n",
        "!npx --yes localtunnel --port 8501"
      ],
      "metadata": {
        "id": "waRVCr4y3rJj"
      },
      "execution_count": null,
      "outputs": []
    }
  ],
  "metadata": {
    "anaconda-cloud": {},
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.12.4"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}