03a-fashion-mnist-multiclass.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The Fashion MNIST - multiclass\n",
    "Last week we worked on the Fashion MNIST problem using only two classes. This week we will work on the whole dataset and try to classify all 10 classes. This is a slightly harder problem.\n",
    "\n",
    "You will preprocess the data to get it into the right shape to process with Keras. This includes scaling the images so that the input range is appropriate and mapping the labels to the **one-hot encoding representation**. Since you have formatted the labels to one-hot encodings we will need to use the **categorical cross entropy loss function**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's load the data set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "library(tensorflow)\n",
    "library(keras)\n",
    "source(\"02-helpers.R\")\n",
    "\n",
    "use_multi_cpu()\n",
    "\n",
    "data <- dataset_fashion_mnist()\n",
    "data_train <- data$train\n",
    "data_test <- data$test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 1\n",
    "Last week we only used two classes from the dataset, but since we are now using all of them, our dataset is larger.\n",
    "How many images are in the training dataset? Inspect the dimensions of the training and test data. What do the different dimensions represent for the `x` and `y` variables inside each set?\n",
    "\n",
    "**Hint 1:**\n",
    "First inspect the contents of data_train and data_test using the [`names`](https://www.rdocumentation.org/packages/base/versions/3.5.1/topics/names) function and `dim` function.\n",
    "\n",
    "**Hint 2:**\n",
    "For our input `data_train$x`, the first axis is the number of images, the second is the height and the last axis is the width. So if we specify `data_train$x[1,1:28,1:28]` we select the first image, all the height pixels and all the width pixels, the whole image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "<FILL IN>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us see how our images look like, run the cell below to plot image `6001`. Feel free to change the index to plot different images from the training set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "library(ggplot2)\n",
    "library(reshape2)\n",
    "\n",
    "index <- 6001\n",
    "options(repr.plot.width = 3, repr.plot.height = 3)\n",
    "ggplot(melt(t(apply(data_train$x[index,,], 2, rev)), varnames=c('x', 'y')), aes(x=x, y=y, fill=value)) +\n",
    "    geom_raster() +\n",
    "    scale_x_continuous(expand = c(0, 0)) +\n",
    "    scale_y_continuous(expand = c(0, 0)) +\n",
    "    scale_fill_gradient(low=\"#000000\", high=\"#FFFFFF\") +\n",
    "    theme_void() +\n",
    "    theme(legend.position = \"none\") +\n",
    "    ggtitle(paste('Label:', data_train$y[index]))\n",
    "options(repr.plot.width = 6, repr.plot.height = 5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following exercises are very similar to the exercises last week. First try to solve them without looking at the solutions from last week. If you are unable to, feel free to work from those solutions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2.1\n",
    "Like last week our images are pixels which are not in the range 0 to 1. Verify this be slicing out a single image from the training set and printing out the exact pixel values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "<FILL IN>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2.2\n",
    "Now that you have seen the pixel values, rescale the **training and test datasets** so that they are in the range 0 to 1. Like last week.\n",
    "\n",
    "**Hint**: Like last week, use R's `min` and `max` functions to find out the minimum and maximum."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "<FILL IN>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2.3\n",
    "Verify that you have correctly rescaled a single image by slicing out the same image as in exercise 2.1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "<FILL IN>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3.1\n",
    "Now we will also need to preprocess the labels of the images. First start by displaying the first ten labels in the training set (`data_train$y`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "<FILL IN>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3.2\n",
    "As you see, the labels range from 0 to 9 (you can verify this if you want to by using the `max` and `min` functions on the labels). We need to convert these labels to one-hot encodings. To do this we use the Keras helper function [`to_categorical()`](https://www.rdocumentation.org/packages/keras/versions/2.2.4/topics/to_categorical). Notice that we have made [`to_categorical()`](https://www.rdocumentation.org/packages/keras/versions/2.2.4/topics/to_categorical) a clickable link which will direct you to the documentation of that function. Use the documentation to guide you in using the function.\n",
    "\n",
    "Apply this function to the **training and test dataset labels** and store the output in `data_train$y` and `data_test$y`, respectively (overwriting the old values).\n",
    "\n",
    "If you want additional quick reading about categorical variables and one-hot encodings see [here](https://machinelearningmastery.com/why-one-hot-encode-data-in-machine-learning/). There are also many good "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "<FILL IN>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3.3\n",
    "Verify that you have done this correctly by\n",
    "- Displaying the `dim` of the new labels. We have changed each label to a vector, so we should have more dimensions.\n",
    "- Displaying the first ten rows of the labels in the training set (same as in exercise 3.1 but now we have more dimensions). See if they make sense."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "<FILL IN>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 4.1\n",
    "Divide the original training data 80%-20% between the new training and validation set. Make sure that you slice both the images and the labels. If you are copying from the previous week, be careful that now our `y`s have an extra dimension."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "<FILL IN>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 4.2\n",
    "Verify the dimensions of the two sets using `dim`. They should be\n",
    "- 12000 28 28 for the validation inputs\n",
    "- 12000 10 for the validation one-hot encodings\n",
    "- 48000 28 28 for the training inputs\n",
    "- 48000 10 for the training one-hot encodings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "<FILL IN>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 5\n",
    "Create an initial model by filling out the skeleton below. Apply the following in this order.\n",
    "- Flatten the input, convert the two-dimensional 28 x 28 image into a one-dimensional vector of length 784.\n",
    "- Add a dense layer with 256 units.\n",
    "- Add a dense layer with 10 units and softmax activation. The output.\n",
    "\n",
    "The functions you will need to use.\n",
    "- `layer_flatten()`, accepts parameters `input_shape`\n",
    "- `layer_dense()`, accepts parameters `units`, `activation`\n",
    "\n",
    "Values for parameters.\n",
    "- `activation`, accepts `\"relu\"`, `\"sigmoid\"`, `\"softmax\"`\n",
    "\n",
    "Print a summary of the model and verify the output shapes of each layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model <- keras_model_sequential() %>%\n",
    "    <FILL IN>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 6\n",
    "Compile the model by filling out the skeleton below according to the following requirements:\n",
    "1. The optimizer should be `optimizer_adam` with learning rate 0.001\n",
    "1. The loss should be `categorical_crossentropy`\n",
    "1. The metric should be accuracy (warning: Keras expects a character vector)\n",
    "\n",
    "**Hint**: it may help to keep the previous notebook open for reference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model %>% compile(\n",
    "    optimizer = <FILL IN>,\n",
    "    loss = <FILL IN>,\n",
    "    metrics = <FILL IN>\n",
    ")\n",
    "weights <- get_weights(model)  # We will use the initial weights later"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 7\n",
    "Fit the model and plot the training history by filling out the skeleton below. We supply the validation set as well so we can see if we start overfitting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model %>% set_weights(weights)  # this will make sure we reset the model to its initial weights\n",
    "history <- model %>% fit(\n",
    "    x = <FILL IN>,\n",
    "    y = <FILL IN>,\n",
    "    epochs = 80,\n",
    "    validation_data = list(<FILL IN THE VALIDATION IMAGES, FILL IN THE VALIDATION LABELS>),\n",
    "    batch_size = 8096,\n",
    "    callbacks=list(Progress$new())\n",
    ")\n",
    "plot(history)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 8\n",
    "We can see that we seem to be able to fit the data alright using this network, our training loss continues to go down, but slowly. At the same time we can see the validation and training accuracy starting to diverge. This is an indication that we might be beginning to overfit. That is to say, we have stopped training this network at the correct moment.\n",
    "\n",
    "We can still imporove this network by adjusting the layers in hopes to reduce the number of parameters. Try reducing the units in the first layer by half (and possibly more) and add smaller additional layers, with ever decreasing number of units. This will speed up the training time as we are using less parameters and might perform equally well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model <- keras_model_sequential() %>%\n",
    "    layer_flatten(input_shape = c(28, 28)) %>%\n",
    "    <FILL IN>\n",
    "\n",
    "cat(summary(model))\n",
    "\n",
    "model %>% compile(\n",
    "    optimizer = optimizer_adam(lr = 0.001),\n",
    "    loss = \"categorical_crossentropy\",\n",
    "    metrics = c(\"accuracy\")\n",
    ")\n",
    "\n",
    "history <- model %>% fit(\n",
    "    x = x_train,\n",
    "    y = y_train,\n",
    "    validation_data = list(<FILL IN THE VALIDATION IMAGES, FILL IN THE VALIDATION LABELS>),\n",
    "    epochs = 80,\n",
    "    batch_size = 8096,\n",
    "    callbacks=list(Progress$new())\n",
    ")\n",
    "plot(history)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When you are ok with the performance of your model [`evaluate`](https://tensorflow.rstudio.com/keras/reference/evaluate.html) it on the test set to obtain a final estimate of our model's performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_test <- data_test$x\n",
    "y_test <- data_test$y\n",
    "model %>% evaluate(x_test, y_test)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "R",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "codemirror_mode": "r",
   "file_extension": ".r",
   "mimetype": "text/x-r-source",
   "name": "R",
   "pygments_lexer": "r",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}