Adding Replicate Llama2 Example

examples/notebooks/ReplicateLlama2.ipynb

@@ -0,0 +1,240 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "0a13ddc8",
+ "metadata": {},
+ "source": [
+ "# Replicate Llama 2 Example"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "623f0cfe",
+ "metadata": {},
+ "source": [
+ "## Installations"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "id": "885dabeb",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# !pip install --quiet --force-reinstall prompttools"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2eac35f8",
+ "metadata": {},
+ "source": [
+ "## Setup imports and API keys"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "5edba05a",
+ "metadata": {},
+ "source": [
+ "First, we'll need to set our API keys."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "id": "ed4e635e",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import os\n",
+ "\n",
+ "os.environ[\"REPLICATE_API_TOKEN\"] = \"\" # Set your API token here"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "842f1e47",
+ "metadata": {},
+ "source": [
+ "Then we'll import the relevant `prompttools` modules to setup our experiment."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "id": "beaa70a1",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from prompttools.experiment import ReplicateExperiment"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "622dea9a",
+ "metadata": {},
+ "source": [
+ "## Run an experiment"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3babfe5a",
+ "metadata": {},
+ "source": [
+ "Next, we create our test inputs. We can iterate over models, inputs, and configurations."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "id": "347590cf",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "sd1 = \"replicate/llama-2-70b-chat:2796ee9483c3fd7aa2e171d38f4ca12251a30609463dcfd4cd76703f22e96cdf\"\n",
+ "models = [sd1] # You can specify multiple models here\n",
+ "input_kwargs = {\"prompt\": [\"Tell me a story about a 19th century wombat gentleman\",\n",
+ " \"Tell me a story about a 22nd century wombat gentleman\"]}\n",
+ "model_specific_kwargs = {sd1: {}}`w\n",
+ "\n",
+ "experiment = ReplicateExperiment(models, input_kwargs, model_specific_kwargs)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e3bbccfa",
+ "metadata": {},
+ "source": [
+ "We can then run the experiment to get results."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "7f6eac1e",
+ "metadata": {},
+ "outputs": [],
+ "source": []
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "id": "ca01ff10",
+ "metadata": {
+ "scrolled": true
+ },
+ "outputs": [],
+ "source": [
+ "experiment.run()\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "id": "772d0006",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "<div>\n",
+ "<style scoped>\n",
+ " .dataframe tbody tr th:only-of-type {\n",
+ " vertical-align: middle;\n",
+ " }\n",
+ "\n",
+ " .dataframe tbody tr th {\n",
+ " vertical-align: top;\n",
+ " }\n",
+ "\n",
+ " .dataframe thead th {\n",
+ " text-align: right;\n",
+ " }\n",
+ "</style>\n",
+ "<table border=\"1\" class=\"dataframe\">\n",
+ " <thead>\n",
+ " <tr style=\"text-align: right;\">\n",
+ " <th></th>\n",
+ " <th>input</th>\n",
+ " <th>response</th>\n",
+ " <th>latency</th>\n",
+ " </tr>\n",
+ " </thead>\n",
+ " <tbody>\n",
+ " <tr>\n",
+ " <th>0</th>\n",
+ " <td>{'prompt': 'Tell me a story about a 19th century wombat gentleman'}</td>\n",
+ " <td>&lt;generator object Prediction.output_iterator at 0x1583c3f10&gt;</td>\n",
+ " <td>0.384344</td>\n",
+ " </tr>\n",
+ " <tr>\n",
+ " <th>1</th>\n",
+ " <td>{'prompt': 'Tell me a story about a 22nd century wombat gentleman'}</td>\n",
+ " <td>&lt;generator object Prediction.output_iterator at 0x158488220&gt;</td>\n",
+ " <td>0.295275</td>\n",
+ " </tr>\n",
+ " </tbody>\n",
+ "</table>\n",
+ "</div>"
+ ],
+ "text/plain": [
+ " input \\\n",
+ "0 {'prompt': 'Tell me a story about a 19th century wombat gentleman'} \n",
+ "1 {'prompt': 'Tell me a story about a 22nd century wombat gentleman'} \n",
+ "\n",
+ " response latency \n",
+ "0 <generator object Prediction.output_iterator at 0x1583c3f10> 0.384344 \n",
+ "1 <generator object Prediction.output_iterator at 0x158488220> 0.295275 "
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "experiment.visualize()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "266c13eb",
+ "metadata": {},
+ "source": [
+ "## Evaluate the model response"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "bebb8023",
+ "metadata": {},
+ "source": [
+ "This notebook is meant to showcase how you can use Replicate's API for Llama2. Please reference other notebook for evaluating the model's response."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.11.4"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}