-
Notifications
You must be signed in to change notification settings - Fork 230
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'main' into musicgen_experiment
- Loading branch information
Showing
32 changed files
with
2,202 additions
and
246 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,173 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# Google Gemini Chat Experiment Example" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Installations" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "# !pip install --quiet --force-reinstall prompttools" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Setup imports and API keys" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "In order for the Google GenAI API to work, you must set up your Google AI Studio credentials (one example in the following cell) or execute this experiment on Google Colab.\n", | ||
| "\n", | ||
| "Executing on Google Colab may require the least amount of set-up." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 5, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "import google.generativeai as genai\n", | ||
| "\n", | ||
| "from google.colab import userdata\n", | ||
| "\n", | ||
| "GOOGLE_API_KEY = \"\" # You can manually set your key\n", | ||
| "# GOOGLE_API_KEY = userdata.get('GOOGLE_API_KEY') # Or, you can read it from your account\n", | ||
| "\n", | ||
| "genai.configure(api_key=GOOGLE_API_KEY)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Once you succeed in setting up your credential, you should be able to execute the following cell without error and see the list of models you have access to." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "for m in genai.list_models():\n", | ||
| " if 'generateContent' in m.supported_generation_methods:\n", | ||
| " print(m.name)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Then we'll import the relevant `prompttools` modules to setup our experiment." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": { | ||
| "ExecuteTime": { | ||
| "end_time": "2023-07-28T21:15:15.360723Z", | ||
| "start_time": "2023-07-28T21:15:15.230441Z" | ||
| }, | ||
| "collapsed": true, | ||
| "jupyter": { | ||
| "outputs_hidden": true | ||
| } | ||
| }, | ||
| "source": [ | ||
| "## Run an experiment" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Next, we create our test inputs. We can iterate over models (`\"gemini-pro\"` in this case, you can also use the ultra model if you have access to it), contents (equivalent of prompt). You can also experiment with configurations like temperature using `generation_config` or `safety_settings`.\n" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 31, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "from prompttools.experiment import GoogleVertexChatCompletionExperiment\n", | ||
| "\n", | ||
| "model = ['gemini-pro']\n", | ||
| "contents = [\"What is the meaning of life?\", \"Who was the first president?\"]\n", | ||
| "\n", | ||
| "experiment = GoogleVertexChatCompletionExperiment(model=model, contents=contents)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "experiment.run()\n", | ||
| "experiment.visualize()" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Evaluate the model response" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Please reference other notebooks (such as Google PaLM 2, Anthropic) for detailed evaluation of the model's response." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "kernelspec": { | ||
| "display_name": "Python 3 (ipykernel)", | ||
| "language": "python", | ||
| "name": "python3" | ||
| }, | ||
| "language_info": { | ||
| "codemirror_mode": { | ||
| "name": "ipython", | ||
| "version": 3 | ||
| }, | ||
| "file_extension": ".py", | ||
| "mimetype": "text/x-python", | ||
| "name": "python", | ||
| "nbconvert_exporter": "python", | ||
| "pygments_lexer": "ipython3", | ||
| "version": "3.11.7" | ||
| } | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 4 | ||
| } |
Oops, something went wrong.