bootcamp/03_classify.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# When using a colab notebook:\n",
    "#!git clone https://github.com/Aleph-Alpha/examples.git\n",
    "#!pip install -r examples/requirements.txt\n",
    "#!cp examples/bootcamp/data.md data.md"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from aleph_alpha_client import Client, SemanticEmbeddingRequest, SemanticEmbeddingResponse, SemanticRepresentation, Prompt, TextControl\n",
    "from scipy import spatial\n",
    "import numpy as np\n",
    "import os\n",
    "from dotenv import load_dotenv\n",
    "\n",
    "from langchain.llms import AlephAlpha\n",
    "from langchain.embeddings import AlephAlphaSymmetricSemanticEmbedding, AlephAlphaAsymmetricSemanticEmbedding\n",
    "\n",
    "import plotly.express as px\n",
    "import plotly.graph_objects as go\n",
    "\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.decomposition import PCA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "load_dotenv()\n",
    "\n",
    "client = Client(token=os.getenv(\"AA_TOKEN\"))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's use luminous embeddings as a classifier\n",
    "\n",
    "\n",
    "### Step 1:\n",
    "Define three classes that we want to classify.\n",
    "If you don't know any, you can use the following:\n",
    "Three classes of incoming emails:\n",
    "- class1 = IT support\n",
    "- class2 = HR\n",
    "- class3 = Sales\n",
    "\n",
    "Each class should have at least 3 examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Here we define three classes that we want to classify\n",
    "# add at least 3 elements to each class\n",
    "class_1 = []\n",
    "\n",
    "class_2 = []\n",
    "\n",
    "class_3 = []"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Next we generate embeddings for each class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: Use the AlephAlpha client to embed the sentences in the two classes\n",
    "embeddings_class_1 = # TODO create embeddings for class 1\n",
    "embeddings_class_2 = # TODO create embeddings for class 2\n",
    "embeddings_class_3 = # TODO create embeddings for class 3\n",
    "\n",
    "\n",
    "new_sentence = \"Hey, my stupid Internet isn't working. Can you help me?\"\n",
    "sentence_embedding = # TODO create embedding for new sentence"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great, now that we have the classes and embeddings, let's see how we can use them to classify a new sentence.\n",
    "If everything went well, the new sentence should have the highest cosine similarity with the class it belongs to."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: get the average similarity of the new sentence to the two classes\n",
    "\n",
    "similarities_class_1 = # TODO get the similarities of the new sentence to the embeddings of class 1\n",
    "similarities_class_2 = # TODO get the similarities of the new sentence to the embeddings of class 2\n",
    "similarities_class_3 = # TODO get the similarities of the new sentence to the embeddings of class 3\n",
    "\n",
    "# TODO get the average similarity of the new sentence to the two classes\n",
    "avg_similarity_class_1 = np.mean(similarities_class_1)\n",
    "avg_similarity_class_2 = np.mean(similarities_class_2)\n",
    "avg_similarity_class_3 = np.mean(similarities_class_3)\n",
    "\n",
    "print(\"Similarity to class 1: \", avg_similarity_class_1)\n",
    "print(\"Similarity to class 2: \", avg_similarity_class_2)\n",
    "print(\"Similarity to class 3: \", avg_similarity_class_3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualize the data\n",
    "\n",
    "To better understand what Luminous is doing, let's visualize the data. \n",
    "You don't need to understand the code below, but you can see that the data is displayed as a scatter plot.\n",
    "\n",
    "As you can see, the features `Luminous` extracts can be used to separate the data into several classes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's use PCA to reduce the dimensionality of the embeddings to 2D\n",
    "\n",
    "pca = PCA(n_components=2)\n",
    "pca.fit(embeddings_class_1 + embeddings_class_2 + embeddings_class_3)\n",
    "pca_embeddings_class_1 = pca.transform(embeddings_class_1)\n",
    "pca_embeddings_class_2 = pca.transform(embeddings_class_2)\n",
    "pca_embeddings_class_3 = pca.transform(embeddings_class_3)\n",
    "pca_embeddings_new_sentence = pca.transform([sentence_embedding])\n",
    "\n",
    "# Now let's plot the embeddings from all three classes\n",
    "\n",
    "fig = go.Figure()\n",
    "\n",
    "for i, embeddings in enumerate([pca_embeddings_class_1, pca_embeddings_class_2, pca_embeddings_class_3, pca_embeddings_new_sentence]):\n",
    "\n",
    "    fig.add_trace(go.Scatter(\n",
    "        x=embeddings[:,0],\n",
    "        y=embeddings[:,1],\n",
    "        mode=\"markers\",\n",
    "        name=f\"Class {i+1}\",\n",
    "        marker=dict(\n",
    "            size=12,\n",
    "            color=[\"red\", \"green\", \"blue\", \"yellow\"][i],\n",
    "        ),\n",
    "        text=class_1 + class_2 + class_3 + [new_sentence],\n",
    "        hovertemplate=\n",
    "        \"<b>%{text}</b><br><br>\" +\n",
    "\n",
    "        \"<i>Similarity to new sentence:</i><br>\" +\n",
    "        \"%{marker.color:.2f}<br>\" +\n",
    "        \"<extra></extra>\"\n",
    "    ))\n",
    "\n",
    "    \n",
    "\n",
    "fig.update_traces(textposition='top center')\n",
    "\n",
    "fig.show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Let's actually train a classifier on these embeddings\n",
    "\n",
    "You don't have to only rely on cosine similarity. You can train a classifier on these embeddings and use that to predict the class of a new sentence."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define a classifier\n",
    "clf = # TODO create a Nearest Neighbors classifier with 3 neighbors (link to documentation: https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html)\n",
    "# use the embeddings and the class labels to train a classifier\n",
    "X = # TODO create a list of embeddings\n",
    "y = # TODO create a list of class labels\n",
    "\n",
    "# fit the classifier\n",
    "clf.fit(X, y)\n",
    "\n",
    "print(\"Predicted class: \", clf.predict([sentence_embedding]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's try a different classifier\n",
    "svm = # TODO create a Support Vector Machine classifier (link to documentation: https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html)\n",
    "svm.fit(X, y)\n",
    "\n",
    "print(\"Predicted class: \", svm.predict([sentence_embedding]))\n",
    "\n",
    "# get the probabilities for each class\n",
    "print(\"Probabilities: \", svm.predict_proba([sentence_embedding]))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "playground",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}