1_classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "# Introduction to data science: classification\n",
    "\n",
    "This is where the **science** comes in!\n",
    "\n",
    "Data science depends hugely on data, but the 'science' part requires us to select the best (i.e. optimal) model -- and this means having an objective measure of 'best' and a way to prove that you have found it.\n",
    "\n",
    "First we'll import some data. I'm using an extract from the Rock Property Catalog, https://subsurfwiki.org/wiki/Rock_Property_Catalog"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Interlude: what is NumPy?\n",
    "\n",
    "- Motivation\n",
    "- Making arrays\n",
    "- Methods on arrays\n",
    "- Accessing rows and columns\n",
    "- Broadcasting\n",
    "- Modules\n",
    "\n",
    "If you're writing a loop, you're doing it wrong!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Back to the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.scatter(*X.T)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "LITHS = ['limestone', 'dolomite', 'shale']\n",
    "\n",
    "def lith_index(y):\n",
    "    return [LITHS.index(lith) for lith in y]\n",
    "\n",
    "plt.scatter(*X.T, c=lith_index(y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## A linear model: SVM\n",
    "\n",
    "\n",
    "The **support vector machine** is a reliable classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.svm import SVC\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "plt.scatter(*X.T, c=lith_index(y), s=80, cmap='coolwarm')\n",
    "plt.scatter(*X.T, c=lith_index(y_pred), cmap='bwr')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "accuracy_score(y, y_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "**❓ What do we think? Are we satisfied?**\n",
    "\n",
    "<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## Scoring\n",
    "\n",
    "Scores matter in all machine learning tasks. It is very common to see people reporting only **accuracy** for classification tasks, or only **R<sup>2</sup>** for regression tasks. It is almost never enough to only look at (or report) the 'obvious' score -- especially for multiclass problems like this one, and especially when there is class imbalance.\n",
    "\n",
    "**This is one of the most important review papers in all of machine learning: [Raschka 2018](https://arxiv.org/abs/1811.12808). Read it and share it.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import classification_report\n",
    "\n",
    "print(classification_report(y, y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## Validation\n",
    "\n",
    "We should not train the model then check its accuracy only on that same training dataset. It's cheating.\n",
    "\n",
    "Let's hold out some validation data, or 'blind' data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "train_test_split?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y)\n",
    "\n",
    "X_train.shape, y_train.shape, X_test.shape, y_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "### EXERCISE\n",
    "\n",
    "Now let's train a model _on only the training data_ and validate it properly _on only the test data_.\n",
    "\n",
    "**❓ Do we think the score will be better or worse than before?**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "We need to think about:\n",
    "\n",
    "- Independence: can you shuffle the data wthout losing information?\n",
    "- Identical distributions: are all the data from the same distribution?\n",
    "- Reproducibility: what can we do to make this reproducible?\n",
    "- Stratification: How can we deal with class imbalance?\n",
    "\n",
    "Can you demonstrate that the test sample is fair and reproducible?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## A non-linear SVM model\n",
    "\n",
    "If we employ the **kernel trick** we can fit a nonlinear model. Scikit-learn's `SVC` actually uses this by default:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "svc = SVC()  # Default is kernel='rbf'\n",
    "\n",
    "svc.fit(X_train, y_train)\n",
    "\n",
    "y_pred = svc.predict(X_test)\n",
    "\n",
    "plt.scatter(*X_test.T, c=lith_index(y_test), s=80, cmap='coolwarm')\n",
    "plt.scatter(*X_test.T, c=lith_index(y_pred), cmap='bwr')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "The score is muuuuuuuuuuuch worse, especially for the minority class:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(classification_report(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "**❓ What could the problem be?**\n",
    "\n",
    "<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "### Scale matters\n",
    "\n",
    "We are not looking at the data in its 'true' numerically isotropic space:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "plt.scatter(*X_train.T)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## Standardize the data\n",
    "\n",
    "It's essential to train SVMs on scaled data, usually the Z-scores of your data, i.e. zero mean, unit variance. This ensures that the different scales of the features is not causing a problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "scaler = StandardScaler()\n",
    "\n",
    "scaler.fit(X_train)\n",
    "\n",
    "X_train_sc = scaler.transform(X_train)\n",
    "X_test_sc = scaler.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "This doesn't change how the data are **relatively** distributed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "fig, axs = plt.subplots(ncols=2, figsize=(15, 5))\n",
    "\n",
    "axs[0].scatter(*X_train.T)\n",
    "axs[1].scatter(*X_train_sc.T)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But it dramatically changes how they are disitributed in absolute terms:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import seaborn as sns\n",
    "\n",
    "fig, axs = plt.subplots(ncols=2, figsize=(15, 5))\n",
    "sns.kdeplot(X_train, ax=axs[0])\n",
    "sns.kdeplot(X_train_sc, ax=axs[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "### EXERCISE\n",
    "\n",
    "Re-fit the **linear** model and look at the scores."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "Solving one problem gives us a new one. Now we have a new pitfall: it is essential to scale the data now before inference -- although the model will happily make (terrible) predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "This is known as an \"out of distribution\" or OOD error, and it's a classic pitfall in machine learning."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## Put everything in a pipeline\n",
    "\n",
    "This is the most flexible way to compose data pipelines in `sklearn`. It is usually better than implementing everything individually in a stepwise manner.\n",
    "\n",
    "For now, it won't change anything."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.pipeline import make_pipeline\n",
    "\n",
    "pipe = "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "And now the nonlinear SVM is basically identical to the linear one:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## Hyperparameter tuning\n",
    "\n",
    "Most algorithms have **hyperparameters** which control how the algorithm learns -- and which often implement some kind of **regularization**. This means different things for different algorithms, but in general it involves simplifying or smoothing the model in some way.\n",
    "\n",
    "For the [support vector machine](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html), there is a parameter `C` to control the model complexity (high `C` means high complexity and low regularization)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "exercise"
    ]
   },
   "source": [
    "### EXERCISE\n",
    "\n",
    "Write a loop to step over values of the regularization parameter `C`. Start with:\n",
    "\n",
    "    C = np.logspace(-3, 5, 17)\n",
    "    train, test = [], []\n",
    "    for Ci in C:\n",
    "        # Instantiate the pipeline, making you use `SVC(C=Ci)`.\n",
    "        # Get the f1_score for both train and test sets.\n",
    "        # Gather these scores in two lists.\n",
    "\n",
    "When you have 2 lists of mean scores, plot them and compare how they vary with C. Put the x-axis on a log scale with `plt.xscale('log')`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "hide"
    ]
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import f1_score\n",
    "\n",
    "C = np.logspace(-3, 5, 17)\n",
    "\n",
    "train, test = [], []\n",
    "for Ci in C:\n",
    "\n",
    "    # Instantiate the pipeline, making you use `SVC(C=Ci)`.\n",
    "    # Get the f1_score for both train and test sets.\n",
    "    # Gather these scores in two lists.\n",
    "\n",
    "plt.plot(C, test)\n",
    "plt.plot(C, train, c='r')\n",
    "plt.xscale('log')\n",
    "plt.ylim(1.0, 0.65)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "**❓ Do you see a potential issue here?**\n",
    "\n",
    "<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;<br />&nbsp;\n",
    "\n",
    "Sometimes you will get a lucky test split, sometimes not. This feels a bit aribtrary. And we might overfit this model selection process to the particular test set that we have."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## Cross validation\n",
    "\n",
    "Instead of choosing a single holdout set for evaluation, we can choose many. This is useful for tuning hyperparameters, for example.\n",
    "\n",
    "But then we have more ways to leak information into our evaluation, so it is smart to continue to hold out our `test` set. **In general, you should always have at least 3 datasets: `train`, `val` (for tuning) and `test` (for the final test).\n",
    "\n",
    "Fair evaluation is hard!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import cross_validate, StratifiedKFold\n",
    "from sklearn.metrics import make_scorer\n",
    "\n",
    "pipe = make_pipeline(StandardScaler(), SVC(C=0.1))\n",
    "scoring = make_scorer(f1_score, average='weighted')\n",
    "skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)\n",
    "\n",
    "cross_validate(pipe,\n",
    "               X_train, y_train,  # Will be split into train and val sets.\n",
    "               scoring=scoring,\n",
    "               cv=skf,\n",
    "               return_train_score=True,\n",
    "               # Also useful: groups\n",
    "              )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "Notice that we no longer use `test` during the model fitting process."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "### EXERCISE\n",
    "\n",
    "Use cross-validation in the loop over values of `C`. Start with:\n",
    "\n",
    "    C = np.logspace(-3, 5, 17)\n",
    "    mean_val, mean_train = [], []\n",
    "    for Ci in C:\n",
    "        # Instantiate the pipeline, making you set `C=Ci`.\n",
    "        # Do the cross validation model fits with `cross_validate()`.\n",
    "        # Gather the means of the test and train scores.\n",
    "\n",
    "Then make the plot, as before."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "mean_val, mean_train = [], []\n",
    "\n",
    "for Ci in C:\n",
    "\n",
    "    # Instantiate the pipeline, making you set `C=Ci`.\n",
    "    # Do the cross validation model fits with `cross_validate()`.\n",
    "    # Gather the means of the test and train scores.\n",
    "    \n",
    "plt.plot(C, mean_val)\n",
    "plt.plot(C, mean_train, c='r')\n",
    "plt.xscale('log')\n",
    "plt.ylim(1, 0.75)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## Explore the model zoo!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "### EXERCISE\n",
    "\n",
    "- Add `Vs` (shear velocity) to the features and see if it improves the prediction quality.\n",
    "- Choose another algorithm to try a prediction with, and implement it in a pipeline.\n",
    "- Choose a hyperparameter of the new algorithm and tune it. (If you have done this kind of thing before, try tuning 2 or 3 hyperparameters with grid or random search.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## Test\n",
    "\n",
    "When you have tuned the predictor and are satisfied that it is as good as it can be, you can test your prediction power."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "C_opt = C[np.argmax(mean_val)]\n",
    "C_opt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "pipe = make_pipeline(StandardScaler(), SVC(C=C_opt))\n",
    "\n",
    "pipe.fit(X_train, y_train)\n",
    "                    \n",
    "y_pred = pipe.predict(X_test)\n",
    "\n",
    "print(classification_report(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "If you are satisfied (think hard about what this means... you really have to decide before you start the model fitting process) then you are ready to fit the final model. If not, you must start all over again."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## Using this model\n",
    "\n",
    "We do not want to use this model &mdash; if we like its performance then we should now retrain it on all the data. Presumably, this new model will be at least as good as the one trained on the training set, we just don't have a way to check it now 😬"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "scaler = StandardScaler().fit(X)\n",
    "X_ = scaler.transform(X)\n",
    "svc = SVC(C=C_opt).fit(X_, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "There is no way for us to test this model, but we should monitor it in production."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "&copy; 2024 Matt Hall / Equinor CC BY"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "py311",
   "language": "python",
   "name": "py311"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}