T81 559:Applications of Generative Artificial Intelligence

Washington University in St. Louis

Instructor: Jeff Heaton

• Section 1. Fall 2024, Tuesday, 6:00 PM, Location: Lopata Hall / 202

Course Description

This course covers the dynamic world of Generative Artificial Intelligence providing hands-on practical applications of Large Language Models (LLMs) and advanced text-to-image networks. Using Python as the primary tool, students will interact with OpenAI's models for both text and images. The course begins with a solid foundation in generative AI principles, moving swiftly into the utilization of LangChain for model-agnostic access and the management of prompts, indexes, chains, and agents. A significant focus is placed on the integration of the Retrieval-Augmented Generation (RAG) model with graph databases, unlocking new possibilities in AI applications.

As the course progresses, students will delve into sophisticated image generation and augmentation techniques, including LORA (LOw-Rank Adaptation), and learn the art of fine-tuning generative neural networks for specific needs. The final part of the course is dedicated to mastering prompt engineering, a critical skill for optimizing the efficiency and creativity of AI outputs. Ideal for students, researchers, and professionals in computer science or related fields, this course offers a transformative learning experience where technology meets creativity, paving the way for innovative applications in the realm of Generative AI.

Note: This course will require the purchase of up to $100 in OpenAI API credits to complete the course.

Objectives

1. Learn how Generative AI fits into the landscape of deep learning and predictive AI.
2. Be able to create ChatBots, Agents, and other LLM-based automation assistants.
3. Understand how to make use of image generative AI programatically.

Syllabus

This syllabus presents the expected class schedule, due dates, and reading assignments. Download current syllabus.

Module	Content
Module 1 Meet on 08/27/2024	Module 1: Introduction to Generative AI 1.1: Course Overview 1.2: Generative AI Overview 1.3: Introduction to OpenAI 1.4: Introduction to LangChain 1.5: Prompt Engineering We will meet on campus this week! (first meeting)
Module 2 Week of 09/03/2024	Module 2: Prompt Based Development 2.1: Prompting for Code Generation 2.2: Handling Revision Prompts 2.3: Using a LLM to Help Debug 2.4: Tracking Prompts in Software Development 2.5: Limits of LLM Code Generation Module 1 Program due: 09/04/2024 Icebreaker due: 09/04/2024
Module 3 Week of 09/10/2024	Module 3: Introduction to Large Language Models 3.1: Foundation Models 3.2: Text Generation 3.3: Text Summarization 3.4: Text Classification 3.5 LLM Writes a Book Module 2 Program due: 09/11/2024
Module 4 Week of 09/17/2024	Module 4: LangChain: Chat and Memory 4.1: LangChain Conversations 4.2: Conversation Buffer Window Memory 4.3: Conversation Token Buffer Memory 4.4: Conversation Summary Memory 4.5: Persisting Langchain Memory Module 3: Program due: 09/18/2024
Module 5 Week of 09/24/2024	Module 5: LangChain: Data Extraction 5.1: Structured Output Parser 5.2: Other Parsers (CSV, JSON, Pandas, Datetime) 5.3: Pydantic parser 5.4: Custom Output Parser 5.5: Output-Fixing Parser Module 4 Program due: 09/25/2024
Module 6 Meet on 10/01/2024	Module 6: Retrieval-Augmented Generation (RAG) 6.1 Introduction to RAG 6.2 Introduction to ChromaDB 6.3 Understanding Embeddings 6.4 Q&A Over Documents 6.5 Embedding Databases Module 5 Program due: 10/02/2024 We will meet on campus this week! (second meeting)
Module 7 Week of 10/15/2024	Module 7: LangChain: Agents 7.1: Introduction to LangChain Agents 7.2: Understanding LangChain Agent Tools 7.3: LangChain Retrival and Search Tools 7.4: Constructing LangChain Agents 7.5: Custom Agents Module 6 Program due: 10/16/2024
Module 8 Meet on 10/22/2024	Module 8: Kaggle Assignment 8.1: Introduction to Kaggle 8.2: Kaggle Notebooks 8.3: Small Large Language Models 8.4: Accessing Small LLM from Kaggle 8.5: Current Semester's Kaggle Module 7 Program due: 10/23/2024 We will meet on campus this week! (third meeting)
Module 9 Week of 10/25/2024	Module 9: MultiModal and Text to Image 9.1: Introduction to MultiModal and Text to Image 9.2: Generating Images with DALL·E 9.3: Editing Existing Images with DALL·E 9.4: MultiModal Models 9.5: Illustrated Book Module 8 Program due: 10/30/2024
Module 10 Week of 11/5/2024	Module 10: Introduction to StreamLit 10.1: Running StreamLit in Google Colab 10.2: StreamLit Introduction 10.3: Understanding Streamlit State 10.4: Creating a Chat Application 10.5: More Advanced Chat Application Module 9 Program due: 11/6/2024
Module 11 Week of 11/12/2024	Module 11: Fine Tuning 11.1: When is fine tuning necessary 11.2: Preparing a dataset for fine tuning 11.3: OepnAI Fine Tuning 11.4: Application of Fine Tuning 11.5: Evaluating Fine Tuning and Optimization Module 10 Program due: 11/13/2024
Module 12 Week of 11/19/2024	Module 12: Prompt Engineering Kaggle Assignment due: 11/20/2024 (approx 4-6PM, due to Kaggle GMT timezone) 12.1 Towards the Perfect Prompt 12.2 Prompt Engineering for Images 12.3 Prompt Engineering for LLM 12.4 Advanced Tips for Prompt Engineering 12.5 Evaluating Bias in Prompts
Module 13 Meet on 11/26/2024	Module 13: Speech Processing 13.1: Audio Generation and Recognition 13.2: OpenAI Speech Generation 13.3: OpenAI Speech Recognition Part 13.4: A Voice-Based ChatBot 13.5: Future Directions in GenAI We will meet on campus this week! (fourth meeting) Final project due: 12/03/2024