DS102-Statistical-Programming-in-R/DS102-L6-Data-Frames.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "50e6ddbb-52d3-4a02-9b69-100a4181e3b0",
   "metadata": {},
   "source": [
    "# DS102 Statistical Programming in R : Lesson - Data Frames"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1bd4bfc-9b15-4b80-87b4-b8b60eb56371",
   "metadata": {},
   "source": [
    "### Table of Contents <a class=\"anchor\" id=\"DS102L6_toc\"></a>\n",
    "\n",
    "* [Table of Contents](#DS102L6_toc)\n",
    "    * [Page 1 - Introduction](#DS102L6_page_1)\n",
    "    * [Page 2 - Creating and Viewing Data Frames](#DS102L6_page_2)\n",
    "    * [Page 3 - Built-in Data Frames](#DS102L6_page_3)\n",
    "    * [Page 4 - Importing CSV Data](#DS102L6_page_4)\n",
    "    * [Page 5 - Importing MS Excel Files](#DS102L6_page_5)\n",
    "    * [Page 6 - Manipulating Data](#DS102L6_page_6)\n",
    "    * [Page 7 - Filtering Data](#DS102L6_page_7)\n",
    "    * [Page 8 - Ordering Data](#DS102L6_page_8)\n",
    "    * [Page 9 - Selecting and Manipulating Data](#DS102L6_page_9)\n",
    "    * [Page 10 - Grouping and Summarizing Data](#DS102L6_page_10)\n",
    "    * [Page 11 - Graphing Data Grouped by Factors](#DS102L6_page_11)\n",
    "    * [Page 12 - Key Terms](#DS102L6_page_12)\n",
    "    * [Page 13 - Hands-On](#DS102L6_page_13)\n",
    "    * [Page 14 - Hands-On Solution](#DS102L6_page_14)\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef5a8c65-ed48-4354-a257-e01b0fd733d4",
   "metadata": {},
   "source": [
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n",
    "\n",
    "# Page 1 - Introduction<a class=\"anchor\" id=\"DS102L6_page_1\"></a>\n",
    "\n",
    "[Back to Top](#DS102L6_toc)\n",
    "\n",
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "620547f8-794b-47e8-8646-c2413eb6e761",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"720\"\n",
       "            height=\"480\"\n",
       "            src=\"https://player.vimeo.com/video/247057946\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "            \n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.VimeoVideo at 0x103f4e130>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import VimeoVideo\n",
    "# Tutorial Video Name: Introduction to R\n",
    "VimeoVideo('247057946', width=720, height=480)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b6ad526-95fa-43c4-bc43-1fe807708ed3",
   "metadata": {},
   "source": [
    "A data frame is a very powerful way of representing a set of data in R; it is the fundamental type of object upon which many of R's analysis tools work. You saw previously that the ```ggplot2``` package uses data frames. In this lesson, you will use data frames to manipulate data and select observations.\n",
    "\n",
    "Conceptually, a data frame can be thought of as a table of data. Typically, each column in the data frame represent a variable. Each row represents an observation that has values for some or all of the columns.\n",
    "\n",
    "Note that the term *variable* gets used in two ways: \n",
    "\n",
    "1. A column name in a data frame. \n",
    "2. A storage location. \n",
    "\n",
    "Typically, context will help you deduce with type of variable is meant.  Nine times out of ten, from here on out, the first one will be used!\n",
    "\n",
    "In this lesson, you will learn how to: \n",
    "\n",
    "* Create a data frame\n",
    "* Access built-in data frames\n",
    "* Import data frames from ```.csv``` and ```.xlsx``` files\n",
    "* Manipulate (wrangle) data using the ```dplyr``` package \n",
    "\n",
    "For the hands on, you will access one of R's built in data frames to summarize and graph data about cars. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7990ac7a-5a2f-4744-af14-1b7e385ea484",
   "metadata": {},
   "source": [
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n",
    "\n",
    "# Page 2 - Creating a Viewing Data Frame<a class=\"anchor\" id=\"DS102L6_page_2\"></a>\n",
    "\n",
    "[Back to Top](#DS102L6_toc)\n",
    "\n",
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "f32358be-aedc-45a9-b82f-b7c6937c6efa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"720\"\n",
       "            height=\"480\"\n",
       "            src=\"https://player.vimeo.com/video/327990142\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "            \n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.VimeoVideo at 0x103f4e430>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import VimeoVideo\n",
    "# Tutorial Video Name: Introduction to R\n",
    "VimeoVideo('327990142', width=720, height=480)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "612429df-e4ef-4d85-8460-5bc8914079c3",
   "metadata": {},
   "source": [
    "The transcript for the above topic tutorial video **[is located here](https://repo.exeterlms.com/documents/V2/DataScience/Video-Transcripts/DSO102-L06-pg2tutorial.zip)**.\n",
    "\n",
    "# Creating and Viewing Data Frame\n",
    "\n",
    "To begin creating a data frame, suppose you collect information on four of your friends and organize it into the following table:\n",
    "\n",
    "<table class=\"table table-striped\">\n",
    "    <tr>\n",
    "        <th>Name</th>\n",
    "        <th>Age</th>\n",
    "        <th>Dominant Hand</th>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>Bob</td>\n",
    "        <td>36</td>\n",
    "        <td>Right</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>Nancy</td>\n",
    "        <td>31</td>\n",
    "        <td>Right</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>Cyrus</td>\n",
    "        <td>26</td>\n",
    "        <td>Left</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>Jackie</td>\n",
    "        <td>34</td>\n",
    "        <td>Right</td>\n",
    "    </tr>\n",
    "</table>\n",
    "\n",
    "In this table, the top row is comprised of a title for each column. This is sometimes called a *header row*. The header row indicates what variables will be in the data frame. Each subsequent row represents three pieces of information about a particular friend: their name, their age, and whether they are right or left handed. Each row represents an *observation*.\n",
    "\n",
    "You can create a data frame that represents this table with the following commands. You first make lists of the contents of the columns; you store each list in a variable with the column name. You then use these lists as arguments to the ```data.frame()``` function.\n",
    "\n",
    "```{r}\n",
    "Name <- c(\"Bob\", \"Nancy\", \"Cyrus\", \"Jackie\")\n",
    "Age <- c(36, 31, 26, 34)\n",
    "Dominant_Hand <- c(\"Right\", \"Right\", \"Left\", \"Right\")\n",
    "friends <- data.frame(Name, Age, Dominant_Hand)\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "## Viewing your Data Frame\n",
    "\n",
    "There are a few different ways to view the data frame you have just created.  \n",
    "\n",
    "---\n",
    "\n",
    "### 1. In the Console, in its Entirety\n",
    "\n",
    "After creating the data frame, you can type the name of the data frame, ```friends```, to see the contents of the data frame you have created:\n",
    "\n",
    "```{r}\n",
    "friends\n",
    "```\n",
    "\n",
    "Name Age Dominant_Hand\n",
    "\n",
    "1 Bob 36 Right\n",
    "\n",
    "2 Nancy 31 Right\n",
    "\n",
    "3 Cyrus 26 Left\n",
    "\n",
    "4 Jackie 34 Right\n",
    "\n",
    "---\n",
    "\n",
    "### 2. In the Console, by Variable\n",
    "\n",
    "You can see the data in each column by using the ```$``` syntax. For example,\n",
    "\n",
    "```{r}\n",
    "friends$Age\n",
    "```\n",
    "\n",
    "[1] 36 31 26 34\n",
    "\n",
    "```{r}\n",
    "friends$Name\n",
    "```\n",
    "\n",
    "[1] Bob Nancy Cyrus Jackie\n",
    "\n",
    "Levels: Bob Cyrus Jackie Nancy\n",
    "\n",
    "```{r}\n",
    "friends$Dominant_Hand\n",
    "```\n",
    "\n",
    "[1] Right Right Left Right\n",
    "\n",
    "Levels: Left Right\n",
    "\n",
    "---\n",
    "\n",
    "### 3. Accessing Individual Elements of the Data Frame\n",
    "\n",
    "Elements of the data frame can be accessed in many ways. You can use array index notation to access an element. For example, if you want to access the element in the third row and the second column, you could type:\n",
    "\n",
    "```{r}\n",
    "friends[3,2]\n",
    "```\n",
    "\n",
    "[1] 26\n",
    "---\n",
    "\n",
    "### 4. In the View Pane\n",
    "\n",
    "Once you have created the data frame, you can also click on it in the Environment pane, located on the right hand side of your RStudio.  Clicking once yields a beautifully laid out data frame that is easy to read and will show all of your data points and columns.  Even if they don't fit in the view pane, there will be scroll buttons.  Here's what the view pane data frame looks like: \n",
    "\n",
    "![The data frame in R Studio showing data points and columns. The title on the tab reads friends. Column headings are name, age, dominant hand. Row one, Bob, thirty six, right. Row two, Nancy, thirty one, right. Row three, Cyrus, twenty six, left. Row four, Jackie, thirty four, right.](Media/dataFrames1.png)\n",
    "\n",
    "You can also have the view pane automatically pop up, without needing to click on it in the Environment pane, with the ```View()``` function. For example:\n",
    "\n",
    "```{r}\n",
    "View(friends)\n",
    "```\n",
    "\n",
    "will also display the above. \n",
    "\n",
    "---\n",
    "\n",
    "## Levels and Factors\n",
    "\n",
    "What is a *factor*? In short, factor is R's word for a categorical variable.  *Levels* are the different categorical options for the data that can be contained within the factor. \n",
    "\n",
    "So the ```friends$Name``` factor can have four levels, namely the four names of your friends. The ```friends$Dominant_Hand``` factor can have two levels: Right or Left.\n",
    "\n",
    "---\n",
    "\n",
    "## Adding Columns to a Data Frame\n",
    "\n",
    "You can add a new column to a data frame. Suppose that you want to add ```shoe size``` to the information you have compiled in the ```friends``` data frame. You could do this with this command:\n",
    "\n",
    "```{r}\n",
    "friends$Shoe_Size <- c(10,8,14,9)\n",
    "```\n",
    "\n",
    "This code states that you are creating a vector full of the values 10, 8, 14, and 9 using the ```c()``` function and that it will be appended into the ```friends``` data frame under the ```Shoe_Size``` column.\n",
    "\n",
    "After executing this command, view the new data frame:\n",
    "\n",
    "![The data frame in R Studio showing data points and columns. The title on the tab reads friends. Column headings are name, age, dominant hand, shoe size. Row one, Bob, thirty six, right, ten. Row two, Nancy, thirty one, right, eight. Row three, Cyrus, twenty six, left, fourteen. Row four, Jackie, thirty four, right, nine.](Media/dataFrames2.png)\n",
    "\n",
    "The ```Shoe_Size``` column has been added to the right of the data frame and populated with values from the vector.\n",
    "\n",
    "---\n",
    "\n",
    "## Review\n",
    "Below is a quiz to review the recently covered material. Quizzes are _not_ graded."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "0225e89a-2e17-46d9-9c23-5b40ee50cb77",
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    from DS_Students import MultipleChoice\n",
    "    from ipynb.fs.full.DS102Questions import *\n",
    "except:\n",
    "    !pip install DS_Students\n",
    "    from DS_Students import MultipleChoice\n",
    "    from ipynb.fs.full.DS102Questions import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "eaf53bf8-0bcd-4451-9743-54f80bbf605c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "782f98a6d8f14255be72e85d1dc6fd26",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. Working, Herding, and …"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "bc200bbc5ed14544a54e0364151b80a4",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. True', 0), ('B. False'…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8c11be683def462db94eb414d93b05d0",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. c()', 0), ('B. []', 1)…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "try:\n",
    "    display(L6P2Q1, L6P2Q2, L6P2Q3)\n",
    "except:\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "832358fd-55fa-420a-8aa2-46fbf2097986",
   "metadata": {},
   "source": [
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n",
    "\n",
    "# Page 3 - Built-in Data Frames<a class=\"anchor\" id=\"DS102L6_page_3\"></a>\n",
    "\n",
    "[Back to Top](#DS102L6_toc)\n",
    "\n",
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "3805fe13-1a6d-48a8-bdeb-16d2e1a98fdc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"720\"\n",
       "            height=\"480\"\n",
       "            src=\"https://player.vimeo.com/video/327990125\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "            \n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.VimeoVideo at 0x1167a8a90>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import VimeoVideo\n",
    "# Tutorial Video Name: Introduction to R\n",
    "VimeoVideo('327990125', width=720, height=480)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f564e3e7-3a7f-4936-b3a3-2e0a9ab78512",
   "metadata": {},
   "source": [
    "The transcript for the above topic tutorial video **[is located here](https://repo.exeterlms.com/documents/V2/DataScience/Video-Transcripts/DSO102-L06-pg3tutorial.zip)**.\n",
    "\n",
    "# Built-in Data Frames\n",
    "\n",
    "Creating a data frame as you did at the beginning of this lesson is very cumbersome for all but the simplest of data. Generally, you will not create data frames this way. Instead, you will import them into R from an MS Excel or ```.csv``` file. But you may also work with built-in R data frames, especially for practice.\n",
    "\n",
    "Several of the built-in datasets in R are data frames. This includes the ```mtcars``` dataset. This is a dataset taken from the Motor Trend magazine in 1974. You can see this data set by typing ```mtcars``` into the console. R will print the dataset for you. You can print just the first six rows of the data frame using the ```head()``` function.\n",
    "\n",
    "```{r}\n",
    "head(mtcars)\n",
    "```\n",
    "\n",
    "```text\n",
    "mpg cyl disp hp drat wt qsec vs am gear carb\n",
    "\n",
    "Mazda RX4 21.0 6 160 110 3.90 2.620 16.46 0 1 4 4\n",
    "\n",
    "Mazda RX4 Wag 21.0 6 160 110 3.90 2.875 17.02 0 1 4 4\n",
    "\n",
    "Datsun 710 22.8 4 108 93 3.85 2.320 18.61 1 1 4 1\n",
    "\n",
    "Hornet 4 Drive 21.4 6 258 110 3.08 3.215 19.44 1 0 3 1\n",
    "\n",
    "Hornet Sportabout 18.7 8 360 175 3.15 3.440 17.02 0 0 3 2\n",
    "\n",
    "Valiant 18.1 6 225 105 2.76 3.460 20.22 1 0 3 1\n",
    "```\n",
    "\n",
    "You can also print the last six rows of the data with the ```tail()``` function.\n",
    "\n",
    "```{r}\n",
    "tail(mtcars)\n",
    "```\n",
    "\n",
    "Each row of this data set represents an observation; each observation is labeled by the name of the car from which the data comes. You can get more information about what each of the columns in the data set represents with the ```help()``` function: \n",
    "\n",
    "```{r}\n",
    "help(mtcars)\n",
    "```\n",
    "\n",
    "<div class=\"panel panel-success\">\n",
    "    <div class=\"panel-heading\">\n",
    "        <h3 class=\"panel-title\">Want to see all the built-in datasets in R?</h3>\n",
    "    </div>\n",
    "    <div class=\"panel-body\">\n",
    "        <p>They are available with the R Datasets Package and their names are shown <a href=\"https://stat.ethz.ch/R-manual/R-devel/library/datasets/html/00Index.html\">here.</a></p>\n",
    "    </div>\n",
    "</div>\n",
    "\n",
    "---\n",
    "\n",
    "## Review\n",
    "Below is a quiz to review the recently covered material. Quizzes are _not_ graded."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "b91d782e-79fb-4759-8a9d-6a6443671299",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b2800d74b9b14880b47a1dae19938ab4",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. mpg', 0), ('B. Valiant…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "07d390e0fbd54f3f9c350990d1d3b0bb",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. View(mtcars)', 0), ('B…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "try:\n",
    "    display(L6P3Q1, L6P3Q2)\n",
    "except:\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb1aade2-59af-4cbc-b5ed-da31e39f266d",
   "metadata": {},
   "source": [
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n",
    "\n",
    "# Page 4 - Importing CSV Data<a class=\"anchor\" id=\"DS102L6_page_4\"></a>\n",
    "\n",
    "[Back to Top](#DS102L6_toc)\n",
    "\n",
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "9d017638-2d38-49bc-9d66-36137824fa20",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"720\"\n",
       "            height=\"480\"\n",
       "            src=\"https://player.vimeo.com/video/247057946\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "            \n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.VimeoVideo at 0x1167c64f0>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import VimeoVideo\n",
    "# Tutorial Video Name: Introduction to R\n",
    "VimeoVideo('247057946', width=720, height=480)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eba2ed6e-98dd-43f7-8761-6ecc9a13af08",
   "metadata": {},
   "source": [
    "The transcript for the above topic tutorial video **[is located here](https://repo.exeterlms.com/documents/V2/DataScience/Video-Transcripts/DSO102-L06-pg4tutorial.zip)**.\n",
    "\n",
    "# Importing CSV Data\n",
    "\n",
    "Building a data frame by creating its component vectors as you've done previously is fine when the data set is small. But for data sets of any appreciable size, it is very difficult. Fortunately, R has facilities to import data sets in several different formats. You will cover how to import a ```.csv``` file.\n",
    "\n",
    "Generally, the data is collected and cleaned up as necessary using a tool other than R: a spreadsheet, for example. Once this is done, the data is saved in a file and imported into R.\n",
    "\n",
    "---\n",
    "\n",
    "## Importing a CSV File\n",
    "\n",
    "CSV is an acronym for Comma Separated Values. A ```.csv``` file is a text file with rows of data values, separated by commas.\n",
    "\n",
    "---\n",
    "\n",
    "### Importing Using read.csv()\n",
    "\n",
    "The R command to import a ```.csv``` file is ```read.csv()```. A file ```Pets.csv``` is available at **[this link.](https://repo.exeterlms.com/documents/V2/DataScience/Stat-Prog-R/PetsCSV.zip)**\n",
    "\n",
    "You can import this file into a data frame in the variable ```my_pets``` using the ```read.csv``` function:\n",
    "\n",
    "```{r}\n",
    "my_pets <- read.csv(\"PetsCSV.csv\")\n",
    "```\n",
    "\n",
    "If you type this command into R, you will get an error message if the ```Pets.csv``` file is not in your current working directory. (A directory is another name for a folder.) R will try to import data files and run R programs from your working directory.\n",
    "\n",
    "When R starts up, it sets your home directory as your working directory. You will probably need to change this to the directory where you are doing your work. You can do this in RStudio by going to the ```Session``` menu, then the ```Set Working Directory``` submenu, and the ```Choose Directory``` command on this menu. Choose the directory of the file you wish to import.\n",
    "\n",
    "![R Studio dropdown menu. Session is selected, which presents a dropdown menu in which set working directory is selected, which presents another dropdown menu in which choose directory is selected.](Media/dataFrames3.png)\n",
    "\n",
    "---\n",
    "\n",
    "### Importing Using the Environment Pane\n",
    "\n",
    "In RStudio, you can also import data at the click of a few buttons using a wizard! This is a nice feature because it allows you to preview your data and set headers.  In the Environment pane, you will see a button for ```Import Dataset```.  Clicking on the down arrow yields this menu: \n",
    "\n",
    "![The environment panel in R Studio. The button import dataset has been selected and has produced a dropdown menu with the options from text open parentheses base close parentheses, from text open parentheses reader closer parentheses, from excel, from S P S S, from S A S, and from Stata. From text open parentheses base close parentheses is selected.](Media/dataFrames4.png)\n",
    "\n",
    "You want the ```From Text(base)``` option that is highlighted.\n",
    "\n",
    "Once you click on it, you can navigate to where your file is stored and select the file.  Then, you will see this window: \n",
    "\n",
    "![The import dataset window that appears once from text open parentheses base close parentheses is selected from the environment pane in R Studio and once you navigate to the file you want to open. The window shows the name of the file you have chosen and options for encoding, heading, row names, separator, decimal, quote, comment, and N A dot strings. The window allows the user to choose strings as factors or not. In the upper right of the window is the content of the input file. In the bottom right of the window is the data frame.](Media/dataFrames5.png)\n",
    "\n",
    "The original file is shown on the top right hand side, and a preview of the data frame R will create is on the bottom right. You'll want to make sure that the radio button ```Yes``` is clicked for ```Heading``` if you have one.  The other important feature to note here is the last checkbox at the bottom, for ```Strings as factors```. Most of the time, it is a good idea to leave this checked, as it will turn your character data into a categorical variable that can be used for most analyses.  However, a few commands need to be done with strings only, so if you come up with errors, you may need to re-import without this check mark.\n",
    "\n",
    "Once you've changed all the options you need, hit the ```Import``` button and away you go!\n",
    "\n",
    "---\n",
    "\n",
    "## Review\n",
    "Below is a quiz to review the recently covered material. Quizzes are _not_ graded."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "d3b1d8f3-4ec9-4409-877a-c2071fcdf6e5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c30063f8f1ac4f228c246687b2d2b0a5",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. Space', 0), ('B. Pipe'…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "64924c2b8d15479885b0b61dee5e6cf8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. True', 0), ('B. False'…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e856cb48160246d79e2c9c1696f9e09e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. Your RStudio interface…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "try:\n",
    "    display(L6P4Q1, L6P4Q2, L6P4Q3)\n",
    "except:\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ead4ad3c-2fe6-457e-8924-1410e951bebd",
   "metadata": {},
   "source": [
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n",
    "\n",
    "# Page 5 - Importing MS Excel Files<a class=\"anchor\" id=\"DS102L6_page_5\"></a>\n",
    "\n",
    "[Back to Top](#DS102L6_toc)\n",
    "\n",
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "bcc91723-634b-4553-89f6-8ee07d68fa64",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"720\"\n",
       "            height=\"480\"\n",
       "            src=\"https://player.vimeo.com/video/328060080\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "            \n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.VimeoVideo at 0x1167c6880>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import VimeoVideo\n",
    "# Tutorial Video Name: Introduction to R\n",
    "VimeoVideo('328060080', width=720, height=480)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbbc06a2-e702-47a1-942f-cbcbe31221a7",
   "metadata": {},
   "source": [
    "The transcript for the above topic tutorial video **[is located here](https://repo.exeterlms.com/documents/V2/DataScience/Video-Transcripts/DSO102-L06-pg5tutorial.zip)**.\n",
    "\n",
    "# Importing an MS Excel File\n",
    "\n",
    "MS Excel is very common in businesses and you may receive data in MS Excel in a ```.xlsx``` format.  Although you could convert it into a ```.csv``` file by using the ```Save as``` button in MS Excel, R will also support importing files directly from MS Excel in the ```.xlsx``` format.  \n",
    "\n",
    "As with ```.csv``` files, there are two ways to import MS Excel data.  \n",
    "\n",
    "---\n",
    "\n",
    "### Importing Using read_excel()\n",
    "\n",
    "The R command to import a ```.xlsx``` file is ```read_excel()```. The ```Pets``` file is **[now available as an MS Excel document](https://repo.exeterlms.com/documents/V2/DataScience/Stat-Prog-R/Pets.zip)** for you to try.\n",
    "\n",
    "You can import this file into a data frame in the variable ```my_petsExcel``` using the ```read_excel``` function:\n",
    "\n",
    "```{r}\n",
    "library(readxl)\n",
    "my_petsExcel <- read_excel(\"Pets.xlsx\")\n",
    "```\n",
    "\n",
    "Just like with CSV files, you may end up with an error message if you have not placed the new MS Excel file in R's working directory.  If you type this command into R, you will get an error message if the ```Pets.csv``` file is not in your current working directory. \n",
    "\n",
    "---\n",
    "\n",
    "### Importing Using the Environment Pane\n",
    "\n",
    "There is also an option to import MS Excel files in the Environment pane.  Once you click the button for ```Import Dataset```, you will want to choose the ```From Excel``` option: \n",
    "\n",
    "![The environment panel in R Studio. The button import dataset has been selected and has produced a dropdown menu with the options from text open parentheses base close parentheses, from text open parentheses reader closer parentheses, from excel, from S P S S, from S A S, and from Stata. From excel is selected.](Media/dataFrames6.png)\n",
    "\n",
    "Once you click on it, you will use the ```Browse``` button in the upper right hand corner to navigate to where your file is stored and select the file.  Then, you will see this window: \n",
    "\n",
    "![The import excel data window that appears once from excel is selected from the environment pane in R Studio and once you navigate to the file you want to open. The window shows the path and name of the file, a data preview, and import options for name, sheet, range, max rows, and skips. It allows the user to choose first row as names or not, and open data viewer or not. In the bottom right is a code preview.](Media/dataFrames7.png)\n",
    "\n",
    "The data preview shows below, and it even tells you what data types each variable will come in as by default and allows you to change it.  R also gives you an option to either Include that variable or skip it, so you can easily choose what data you want to go into your data frame.\n",
    "\n",
    "![The import excel data window that appears once from excel is selected from the environment pane in R Studio and once you navigate to the file you want to open. The window shows the path and name of the file, a data preview, and import options for name, sheet, range, max rows, and skips. It allows the user to choose first row as names or not, and open data viewer or not. In the bottom right is a code preview.](Media/dataFrames8.png)\n",
    "\n",
    "There is also a header option for MS Excel files as well.  The ```First Row as Names``` checkbox will provide automatic headings for you as long as it is checked. \n",
    "\n",
    "Once you've changed all the options you need, hit the ```Import``` button and away you go!\n",
    "\n",
    "---\n",
    "\n",
    "## Review\n",
    "Below is a quiz to review the recently covered material. Quizzes are _not_ graded."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "851daf84-0364-4ae2-a1db-12ec87120eb2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d43078bb9c784800a34d9bcc43db9d4b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. Exports CSV files into…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "try:\n",
    "    display(L6P5Q1)\n",
    "except:\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd9e3b8a-9224-447f-8cf7-5035cae86421",
   "metadata": {},
   "source": [
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n",
    "\n",
    "# Page 6 - Manipulating Data<a class=\"anchor\" id=\"DS102L6_page_6\"></a>\n",
    "\n",
    "[Back to Top](#DS102L6_toc)\n",
    "\n",
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "e13e98bf-06ad-4e17-a5fa-faf38cf5618f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"720\"\n",
       "            height=\"480\"\n",
       "            src=\"https://player.vimeo.com/video/327990281\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "            \n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.VimeoVideo at 0x1167cf4f0>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import VimeoVideo\n",
    "# Tutorial Video Name: Introduction to R\n",
    "VimeoVideo('327990281', width=720, height=480)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a81e2098-28ef-4ff5-a155-afe824de4323",
   "metadata": {},
   "source": [
    "The transcript for the above topic tutorial video **[is located here](https://repo.exeterlms.com/documents/V2/DataScience/Video-Transcripts/DSO102-L06-pg6tutorial.zip)**.\n",
    "\n",
    "# Manipulating Data\n",
    "\n",
    "One of R's strengths is the ability to manipulate data to facilitate statistical analysis and to extract meaning and information from the data. Consider the following types of manipulation:\n",
    "\n",
    "* **Filtering:** Selecting a subset of observations that meet specific criteria from a data frame.\n",
    "* **Ordering:** Arranging the observations in a data frame into a particular order according to given criteria.\n",
    "* **Selection:** Choosing certain variables from a data frame.\n",
    "* **Mutation:** Computing new variables from variables that already exist in the data frame.\n",
    "* **Grouping:** Creating groups of observations that have given common characteristics.\n",
    "* **Summarizing:** Describing observations using sample statistics.\n",
    "\n",
    "The base R language has sophisticated functions that will perform each of these manipulations. Unfortunately, like many things in the base R language, learning to use these functions effectively can be frustrating, difficult, and time consuming.\n",
    "\n",
    "In this lesson, instead of using the base R language functions, you will be introduced and use the ```dplyr``` package to manipulate data. The ```dplyr``` package is part of the ```tidyverse``` collection of R packages; you were introduced the ```tidyverse``` earlier. You can find more information about the ```tidyverse``` **[here](https://www.tidyverse.org/)**. \n",
    "\n",
    "First, install ```dplyr``` with the command:\n",
    "\n",
    "```{r}\n",
    "install.packages(\"dplyr\")\n",
    "```\n",
    "\n",
    "You will only need to install the ```dplyr``` package once; at the start of each R session, you will then need to load the ```dplyr``` library with the command:\n",
    "\n",
    "```{r}\n",
    "library(\"dplyr\")\n",
    "```\n",
    "\n",
    "This makes the functions in ```dplyr``` available for your use.\n",
    "\n",
    "<div class=\"panel panel-success\">\n",
    "    <div class=\"panel-heading\">\n",
    "        <h3 class=\"panel-title\">Additional Info!</h3>\n",
    "    </div>\n",
    "    <div class=\"panel-body\">\n",
    "        <p>You may want to watch <a href=\"https://vimeo.com/418281958\">this recorded live workshop </a> on the anatomy of dplyr functions, which is meant to go along with the remainder of this lesson.</p>\n",
    "    </div>\n",
    "</div>\n",
    "\n",
    "---\n",
    "\n",
    "## Review\n",
    "Below is a quiz to review the recently covered material. Quizzes are _not_ graded."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "86958e8d-fab3-4545-9bc6-1b15096cb59e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "73d4a03da444456d8b182967bfbbeaf3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. Filtering', 0), ('B. S…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ada720735b8a41118c1d38e51eaf19a4",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. Ordering', 0), ('B. Se…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b74127c927a84a01a834f16017d5834d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. Mutation', 0), ('B. Or…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1c4c254c90bf4be8a1f010bb6ec9acfa",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. Ordering', 0), ('B. Fi…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3a30a4858eaa4b4684ffe0ff6b556978",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. Selection', 0), ('B. S…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "22389398fb074d13a91018cfac5cbee3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. Filtering', 0), ('B. M…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "try:\n",
    "    display(L6P6Q1, L6P6Q2, L6P6Q3, L6P6Q4, L6P6Q5, L6P6Q6)\n",
    "except:\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42cb7d8a-aced-42e5-ac24-32acf8c95473",
   "metadata": {},
   "source": [
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n",
    "\n",
    "# Page 7 - Filtering Data<a class=\"anchor\" id=\"DS102L6_page_7\"></a>\n",
    "\n",
    "[Back to Top](#DS102L6_toc)\n",
    "\n",
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "ca834619-01b6-4d75-b148-692eca8b29a9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"720\"\n",
       "            height=\"480\"\n",
       "            src=\"https://player.vimeo.com/video/327990180\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "            \n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.VimeoVideo at 0x1167cfa30>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import VimeoVideo\n",
    "# Tutorial Video Name: Introduction to R\n",
    "VimeoVideo('327990180', width=720, height=480)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf0debaf-5fae-4035-8e7d-ae62e62456cb",
   "metadata": {},
   "source": [
    "The transcript for the above topic tutorial video **[is located here](https://repo.exeterlms.com/documents/V2/DataScience/Video-Transcripts/DSO102-L06-pg7tutorial.zip)**.\n",
    "\n",
    "# Filtering\n",
    "\n",
    "You will use your improved ```my_pets``` data frame to practice data manipulations. \n",
    "\n",
    "Suppose you wish to get all observations (rows) that contain goats. You can do this with the following ```filter()``` function:\n",
    "\n",
    "```{r}\n",
    "filter(my_pets, Animal == \"Goat\")\n",
    "```\n",
    "\n",
    "Which provides this data back:\n",
    "\n",
    "```text\n",
    "Name Animal Age Weight Skin\n",
    "\n",
    "1 Bruno Goat 4 35 hair\n",
    "\n",
    "2 Jacko Goat 1 45 hair\n",
    "```\n",
    "\n",
    "In this example, ```filter()``` has two arguments. The first is the data frame from which you will get the rows; in this case, it is ```my_pets```. The second is a condition that is used to select rows.  The condition is that the ```Animal``` variable must be equal to ```Goat```.\n",
    "\n",
    "You can add more conditions as arguments to ```filter()``` if you want to select on multiple variables. For example, to get all the rows containing goats whose age is two or less, you could use the following:\n",
    "\n",
    "```{r}\n",
    "filter(my_pets, Animal == \"Goat\", Age <= 2)\n",
    "```\n",
    "\n",
    "Which yields only one goat: \n",
    "\n",
    "```text\n",
    "Name Animal Age Weight Skin\n",
    "\n",
    "1 Jacko Goat 1 45 hair\n",
    "```\n",
    "\n",
    "You can use ```filter()``` to get all the observations of animals heavier than three pounds:\n",
    "\n",
    "```{r}\n",
    "filter(my_pets, Weight > 3)\n",
    "```\n",
    "\n",
    "```text\n",
    "Name Animal Age Weight Skin\n",
    "\n",
    "1 Bruno Goat 4 35.0 hair\n",
    "\n",
    "2 Jacko Goat 1 45.0 hair\n",
    "\n",
    "3 Sophie Cat 12 3.5 fur\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "### %in%\n",
    "\n",
    "Suppose you want all the observations of mammals in the data frame. You could do this as follows:\n",
    "\n",
    "```{r}\n",
    "filter(my_pets, Animal %in% c(\"Goat\", \"Cat\", \"Guinea Pig\"))\n",
    "```\n",
    "\n",
    "With the following data meeting those criteria:\n",
    "\n",
    "```text\n",
    "Name     Animal Age Weight Skin\n",
    "\n",
    "1 Bruno Goat 4 35.0 hair\n",
    "\n",
    "2 Jacko Goat 1 45.0 hair\n",
    "\n",
    "3 Sophie Cat 12 3.5 fur\n",
    "\n",
    "4 Patches Guinea Pig 2 2.0 fur\n",
    "```\n",
    "\n",
    "This works as follows. ```c(\"Goat\", \"Cat\", \"Guinea Pig\")``` creates a vector of the three animal types, and the ```%in%``` code looks for anything in the vector criteria specified. \n",
    "\n",
    "In this particular data set, you could also get all of the mammals by getting everything that is not a gold fish.  Put those logical operators to work!\n",
    "\n",
    "```{r}\n",
    "filter(my_pets, Animal != \"Gold Fish\")\n",
    "```\n",
    "\n",
    "```text\n",
    "Name     Animal Age Weight Skin\n",
    "\n",
    "1 Bruno Goat 4 35.0 hair\n",
    "\n",
    "2 Jacko Goat 1 45.0 hair\n",
    "\n",
    "3 Sophie Cat 12 3.5 fur\n",
    "\n",
    "4 Patches Guinea Pig 2 2.0 fur\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "### Filtering into a Data Frame\n",
    "\n",
    "Often you will want to save the data frame that is returned by ```filter()``` for further analysis. You can do this by assigning the returned value to a variable. For example:\n",
    "\n",
    "```{r}\n",
    "mammals <- filter(my_pets, Animal != \"Gold Fish\")\n",
    "mammals\n",
    "```\n",
    "\n",
    "```text\n",
    "Name     Animal Age Weight Skin\n",
    "\n",
    "1 Bruno Goat 4 35.0 hair\n",
    "\n",
    "2 Jacko Goat 1 45.0 hair\n",
    "\n",
    "3 Sophie Cat 12 3.5 fur\n",
    "\n",
    "4 Patches Guinea Pig 2 2.0 fur\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "## Review\n",
    "Below is a quiz to review the recently covered material. Quizzes are _not_ graded."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "3622c203-3862-402a-b558-b73143696fca",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1d78ba82bc1f40e08b7f9dc4ca5b13d5",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. arrange(mtcars, cyl)`'…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "13ec4ef3f4ae4854a028e626e0674cb3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. filter(mtcars, hp < 10…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "bfc7a1a1f580424199900d7db2c7435c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. summarize(mtcars, medi…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6755056a9a644a3687a30b47e125eefa",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. mtcars %>% group_by(cy…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "try:\n",
    "    display(L6P7Q1, L6P7Q2, L6P7Q3, L6P7Q4)\n",
    "except:\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7b3f800-0a14-4d13-8c32-390965c537ca",
   "metadata": {},
   "source": [
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n",
    "\n",
    "# Page 8 - Ordering Data<a class=\"anchor\" id=\"DS102L6_page_8\"></a>\n",
    "\n",
    "[Back to Top](#DS102L6_toc)\n",
    "\n",
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "eacc6b4c-e028-4a7e-b9d1-58e8b14471eb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"720\"\n",
       "            height=\"480\"\n",
       "            src=\"https://player.vimeo.com/video/327990291\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "            \n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.VimeoVideo at 0x1167d9640>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import VimeoVideo\n",
    "# Tutorial Video Name: Introduction to R\n",
    "VimeoVideo('327990291', width=720, height=480)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4597e4b4-ce22-448d-843e-eab5ba74a9c4",
   "metadata": {},
   "source": [
    "The transcript for the above topic tutorial video **[is located here](https://repo.exeterlms.com/documents/V2/DataScience/Video-Transcripts/DSO102-L06-pg8tutorial.zip)**.\n",
    "\n",
    "# Ordering Data\n",
    "\n",
    "You can use the ```arrange()``` function to order the observations of the data frame in a certain order. For example, if you wanted to sort the rows from youngest to oldest (ascending order), you could do this as follows:\n",
    "\n",
    "```{r}\n",
    "arrange(my_pets, Age)\n",
    "```\n",
    "\n",
    "This yields the following:\n",
    "\n",
    "```text\n",
    "Name     Animal Age Weight   Skin\n",
    "\n",
    "1 Jacko Goat 1 45.00 hair\n",
    "\n",
    "2 Patches Guinea Pig 2 2.00 fur\n",
    "\n",
    "3 Boozer Gold Fish 2 0.03 scales\n",
    "\n",
    "4 Boozer Gold Fish 3 0.04 scales\n",
    "\n",
    "5 Bruno Goat 4 35.00 hair\n",
    "\n",
    "6 Sophie Cat 12 3.50 fur\n",
    "```\n",
    "\n",
    "The first argument to ```arrange()``` is the data frame; the second argument is the variable upon which to sort. You can add other variables if you want to sort on them as well.\n",
    "\n",
    "---\n",
    "\n",
    "## Sorting in Descending Order\n",
    "\n",
    "You could sort from oldest to youngest as follows using the ```desc()``` function (which is short for \"descending\"):\n",
    "\n",
    "```{r}\n",
    "arrange(my_pets, desc(Age))\n",
    "```\n",
    "\n",
    "And now the data is shown in reverse order:\n",
    "\n",
    "```text\n",
    "Name     Animal Age Weight   Skin\n",
    "\n",
    "1 Sophie Cat 12 3.50 fur\n",
    "\n",
    "2 Bruno Goat 4 35.00 hair\n",
    "\n",
    "3 Boozer Gold Fish 3 0.04 scales\n",
    "\n",
    "4 Patches Guinea Pig 2 2.00 fur\n",
    "\n",
    "5 Boozer Gold Fish 2 0.03 scales\n",
    "\n",
    "6 Jacko Goat 1 45.00 hair\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "## The Pipe Operator\n",
    "\n",
    "Suppose you want to select only the mammals, and then sort them from youngest to oldest. You could do the following:\n",
    "\n",
    "```{r}\n",
    "mammals <- filter(my_pets, Animal != \"Gold Fish\")\n",
    "arrange(mammals, Age)\n",
    "```\n",
    "\n",
    "You know have all the mammals sorted in ascending order:\n",
    "\n",
    "```text\n",
    "Name     Animal Age Weight Skin\n",
    "\n",
    "1 Jacko Goat 1 45.0 hair\n",
    "\n",
    "2 Patches Guinea Pig 2 2.0 fur\n",
    "\n",
    "3 Bruno Goat 4 35.0 hair\n",
    "\n",
    "4 Sophie Cat 12 3.5 fur\n",
    "```\n",
    "\n",
    "This requires you to create a variable to store the filtered results. You can achieve the same results by using the *pipe operator*, which is ```%>%```.  The pipe operator basically orders things, so it says: \"take the data frame ```my_pets``` and then ```filter``` it and then ```arrange``` it.\" The \"and then\" parts of that sentence is the pipe operator at work!\n",
    "\n",
    "```{r}\n",
    "my_pets %>% filter(Animal != \"Gold Fish\") %>% arrange(Age)\n",
    "```\n",
    "\n",
    "When you are using the pipe operator to connect data frames and functions, you do not include the data frame as the first argument of the function; it instead goes first. Thus, the only argument to ```filter()``` is the condition that selects rows; the only argument to ```arrange()``` is the variable on which you want to sort. The pipe operator can make data analysis easier to understand and read; it feels more like a logical sentence.\n",
    "\n",
    "You'll notice that whether you do the two commands separately, as above, or with the pipe operator, as below, the results will be the same. \n",
    "\n",
    "```text\n",
    "Name     Animal Age Weight Skin\n",
    "\n",
    "1 Jacko Goat 1 45.0 hair\n",
    "\n",
    "2 Patches Guinea Pig 2 2.0 fur\n",
    "\n",
    "3 Bruno Goat 4 35.0 hair\n",
    "\n",
    "4 Sophie Cat 12 3.5 fur\n",
    "```\n",
    "\n",
    "<div class=\"panel panel-danger\">\n",
    "    <div class=\"panel-heading\">\n",
    "        <h3 class=\"panel-title\">Caution!</h3>\n",
    "    </div>\n",
    "    <div class=\"panel-body\">\n",
    "        <p>Don't confuse the pipe operator %>% with the pipe symbol | . Although they essentially mean the same thing, the | won't work in R.</p>\n",
    "    </div>\n",
    "</div>\n",
    "\n",
    "---\n",
    "\n",
    "## Review\n",
    "Below is a quiz to review the recently covered material. Quizzes are _not_ graded."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "96576a76-5d3f-49c5-a5bd-aa06748a14ed",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d1ff4964f201482d920200e844e9f0e1",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. Reduces the number of …"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8aac3413e1dd4303af3547c94f1dab01",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. select(mtcars, hp)', 0…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a4144ffe3c4c4a83bb98c9dbc3a70573",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. desc()', 0), ('B. asc(…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "try:\n",
    "    display(L6P8Q1, L6P8Q2, L6P8Q3)\n",
    "except:\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36943bcd-7e16-4af1-97d3-c09e81ae5e99",
   "metadata": {},
   "source": [
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n",
    "\n",
    "# Page 9 - Selecting and Mutating Data<a class=\"anchor\" id=\"DS102L6_page_9\"></a>\n",
    "\n",
    "[Back to Top](#DS102L6_toc)\n",
    "\n",
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "2d4bab3c-bd55-4060-87e1-cc1501259fe4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"720\"\n",
       "            height=\"480\"\n",
       "            src=\"https://player.vimeo.com/video/327990315\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "            \n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.VimeoVideo at 0x1167d9b50>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import VimeoVideo\n",
    "# Tutorial Video Name: Introduction to R\n",
    "VimeoVideo('327990315', width=720, height=480)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdf2cdde-9447-49b5-8936-17d80f46b45e",
   "metadata": {},
   "source": [
    "The transcript for the above topic tutorial video **[is located here](https://repo.exeterlms.com/documents/V2/DataScience/Video-Transcripts/DSO102-L06-pg9tutorial.zip)**.\n",
    "\n",
    "# Selection\n",
    "\n",
    "In many cases, you may end up working with data sets that have hundreds of variables. In these cases, only some of the variables will be of interest at any given time. You can select to use only certain variables using the ```select()``` function. As an example, suppose you need only the name and types of the pets; you can select these columns as follows:\n",
    "\n",
    "```{r}\n",
    "select(my_pets, Name, Animal)\n",
    "```\n",
    "\n",
    "Which provides data with just those two columns, as expected:\n",
    "\n",
    "```text\n",
    "Name     Animal\n",
    "\n",
    "1 Bruno Goat\n",
    "\n",
    "2 Jacko Goat\n",
    "\n",
    "3 Sophie Cat\n",
    "\n",
    "4 Patches Guinea Pig\n",
    "\n",
    "5 Boozer Gold Fish\n",
    "\n",
    "6 Boozer Gold Fish\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "# Mutation\n",
    "\n",
    "Mutation is the process of computing new columns in the data frame from existing columns. As an example, suppose you want the weights of the pets in kilograms. You can compute kilograms from pounds by dividing a weight in pounds by 2.20462. To create a new column with weights in kilograms, you use the ```mutate()``` function:\n",
    "\n",
    "```{r}\n",
    "mutate(my_pets, Weight_kg = Weight/2.20462)\n",
    "```\n",
    "\n",
    "The first argument is the data frame name ```my_pets```, the second argument is ```Weight_kg = Weight/2.20462```.  The variable on the left of the equals sign becomes the new column name, and the formula on the right of the equals sign shows how to compute this new column from the existing columns. The formula can include multiple columns if need be.\n",
    "\n",
    "```mutate()``` adds the new column to the end of the data frame:\n",
    "\n",
    "```text\n",
    "Name     Animal Age Weight   Skin   Weight_kg\n",
    "\n",
    "1 Bruno Goat 4 35.00 hair 15.87575183\n",
    "\n",
    "2 Jacko Goat 1 45.00 hair 20.41168092\n",
    "\n",
    "3 Sophie Cat 12 3.50 fur 1.58757518\n",
    "\n",
    "4 Patches Guinea Pig 2 2.00 fur 0.90718582\n",
    "\n",
    "5 Boozer Gold Fish 3 0.04 scales 0.01814372\n",
    "\n",
    "6 Boozer Gold Fish 2 0.03 scales 0.01360779\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "## Adding a Column with the Same Value\n",
    "\n",
    "Sometimes it can be helpful to add a column in which has the same value for every row. So, for example, you could add a ```Status``` column that indicates that all of the pets are alive:\n",
    "\n",
    "```{r}\n",
    "mutate(my_pets, Status = \"Alive\")\n",
    "```\n",
    "\n",
    "Again, the new column is added to the end of the data frame, and you will see that every column is filled with ```Alive``` because no logical operators were used to conditionally fill the column. \n",
    "\n",
    "```text\n",
    "Name     Animal Age Weight   Skin Status\n",
    "\n",
    "1 Bruno Goat 4 35.00 hair Alive\n",
    "\n",
    "2 Jacko Goat 1 45.00 hair Alive\n",
    "\n",
    "3 Sophie Cat 12 3.50 fur Alive\n",
    "\n",
    "4 Patches Guinea Pig 2 2.00 fur Alive\n",
    "\n",
    "5 Boozer Gold Fish 3 0.04 scales Alive\n",
    "\n",
    "6 Boozer Gold Fish 2 0.03 scales Alive\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "## Review\n",
    "Below is a quiz to review the recently covered material. Quizzes are _not_ graded."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "73a8d856-26dc-4af6-9e39-bc43cd56d62d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "08c6ea46a9df45fe869e9631a2d06154",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. select(mtcars, hp, cyl…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "17af7360689d4dd5a8968830d74cf7bc",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. What happens to superh…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "try:\n",
    "    display(L6P9Q1, L6P9Q2)\n",
    "except:\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a0d02d5-04fa-4220-9637-d02c3b32dce5",
   "metadata": {},
   "source": [
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n",
    "\n",
    "# Page 10 - Grouping and Summarizing Data<a class=\"anchor\" id=\"DS102L6_page_10\"></a>\n",
    "\n",
    "[Back to Top](#DS102L6_toc)\n",
    "\n",
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "fb8e7c84-1ca4-4096-bade-4326cee5fe71",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"720\"\n",
       "            height=\"480\"\n",
       "            src=\"https://player.vimeo.com/video/328466905\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "            \n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.VimeoVideo at 0x1167e34f0>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import VimeoVideo\n",
    "# Tutorial Video Name: Introduction to R\n",
    "VimeoVideo('328466905', width=720, height=480)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f82e1a7-4ece-40f1-94e3-db74e5d78f59",
   "metadata": {},
   "source": [
    "The transcript for the above topic tutorial video **[is located here](https://repo.exeterlms.com/documents/V2/DataScience/Video-Transcripts/DSO102-L06-pg10tutorial.zip)**.\n",
    "\n",
    "# Grouping\n",
    "\n",
    "Grouping is organizing the data into groups of interest that have common characteristics. Grouping is not useful by itself. It actually does nothing to your data when done by itself, but is powerful when used for computing summary statistics or creating plots.\n",
    "\n",
    "---\n",
    "\n",
    "# Summarizing\n",
    "\n",
    "You can get summaries of your data using the ```summarize()``` function. For example, to get the mean age, you can use the command:\n",
    "\n",
    "```{r}\n",
    "summarize(my_pets, ave_age = mean(Age))\n",
    "```\n",
    "\n",
    "And this is the output you will receive:\n",
    "\n",
    "```text\n",
    "ave_age\n",
    "\n",
    "1 4\n",
    "```\n",
    "\n",
    "The first argument is the data frame; the second argument is ```ave_age = mean(Age)```. As with the ```mutate()``` function, the term on the left side of the equals sign gives the name of the column to be created; you are creating something called ```ave_age```. The term on the right of the equals sign is the function to be applied.  In this case, you are applying the mean.\n",
    "\n",
    "Note that the output of ```summarize()``` is only one row, because it computed the mean of all the rows in the data frame.  It is only one column, too: the value computed by the summary.\n",
    "\n",
    "---\n",
    "\n",
    "# Combining Group By and Summarize\n",
    "\n",
    "Suppose you want to know the average age of each type of animal. To find this, you would combine ```summarize()``` and ```group_by()``` as follows; you will use a pipe ```(%>%)``` to combine them:\n",
    "\n",
    "```{r}\n",
    "my_pets %>% group_by(Animal) %>% summarize(ave.age = mean(Age))\n",
    "```\n",
    "\n",
    "```text\n",
    "# A tibble: 4 x 2\n",
    "\n",
    "Animal ave_age\n",
    "\n",
    "<fctr>   <dbl>\n",
    "\n",
    "1 Cat 12.0\n",
    "\n",
    "2 Goat 2.5\n",
    "\n",
    "3 Gold Fish 2.5\n",
    "\n",
    "4 Guinea Pig 2.0\n",
    "```\n",
    "\n",
    "The first thing you will notice with this output is that you have created a *tibble*.  What is a tibble, you ask? Well, it is essentially a data frame with some additional information.  You can think of it as a modern alternative to a data frame; the ```tidyverse``` packages all work with tibbles. \n",
    "\n",
    "You also get a note in the output telling you what the tibble has been grouped by, ```Animal```, and there are four groups. \n",
    "\n",
    "You can see that the results of this summary is a tibble that has four rows and two columns, ```Animal``` and ```ave.age```. Each row shows one type of animal and the mean Age value for that animal type.\n",
    "\n",
    "---\n",
    "\n",
    "## Functions for summarize()\n",
    "\n",
    "There are many functions that can be used with ```summarize()```. Some of these include:\n",
    "\n",
    "* **mean() :** Computes the mean of the values.\n",
    "\n",
    "* **median() :** Computes the median of the values.\n",
    "\n",
    "* **sd() :** Computes the standard deviation of the values.\n",
    "\n",
    "* **min() and max()** : Compute the minimum and maximum values.\n",
    "\n",
    "* **n()** : Compute the number of values.\n",
    "\n",
    "---\n",
    "\n",
    "### n()\n",
    "\n",
    "```n()``` is very useful in determining the number of rows in a data frame that meet a given condition. For example, suppose you want to determine how many animals of each type are in the data frame. You can do it as follows:\n",
    "\n",
    "```{r}\n",
    "my_pets %>% group_by(Animal) %>% summarize(count = n())\n",
    "```\n",
    "\n",
    "That will provide these results: \n",
    "\n",
    "```text\n",
    "# A tibble: 4 x 2\n",
    "\n",
    "Animal count\n",
    "\n",
    "<fctr> <int>\n",
    "\n",
    "1 Cat 1\n",
    "\n",
    "2 Goat 2\n",
    "\n",
    "3 Gold Fish 2\n",
    "\n",
    "4 Guinea Pig 1\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "## Grouping by Multiple Variables\n",
    "\n",
    "Finally, you can group by more than one factor. So, for example, you can group by ```Animal``` and ```Name```, and count the number of pets that have a distinct pair of values:\n",
    "\n",
    "```{r}\n",
    "my_pets %>% group_by(Name, Animal) %>% summarize(count = n())\n",
    "```\n",
    "\n",
    "That yields this tibble:\n",
    "\n",
    "```text\n",
    "# A Tibble: 5 x 3\n",
    "\n",
    "# Groups: Name [?]\n",
    "\n",
    "Name     Animal count\n",
    "\n",
    "<fctr> <fctr> <int>\n",
    "\n",
    "1 Boozer Gold Fish 2\n",
    "\n",
    "2 Bruno Goat 1\n",
    "\n",
    "3 Jacko Goat 1\n",
    "\n",
    "4 Patches Guinea Pig 1\n",
    "\n",
    "5 Sophie Cat 1\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "## Review\n",
    "Below is a quiz to review the recently covered material. Quizzes are _not_ graded."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "c8150e8a-9bd4-4c92-a9bf-00cdc0cd97e7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ca8a25a99a1e4fba942cc7805cbd6365",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. A function', 0), ('B. …"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "dcf28c769c4e44269fc3b9ac9d707d6f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. summarize()', 0), ('B.…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "900c160f2d9c479bb681314c74771569",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. True', 0), ('B. False'…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "try:\n",
    "    display(L6P10Q1, L6P10Q2, L6P10Q3)\n",
    "except:\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7bd842c2-d518-4aaa-b20c-bbb2f677c93a",
   "metadata": {},
   "source": [
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n",
    "\n",
    "# Page 11 - Graphing Data Grouped by Factors<a class=\"anchor\" id=\"DS102L6_page_11\"></a>\n",
    "\n",
    "[Back to Top](#DS102L6_toc)\n",
    "\n",
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "6325c8b6-63aa-4843-8c44-2c688925fe4f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"720\"\n",
       "            height=\"480\"\n",
       "            src=\"https://player.vimeo.com/video/327990198\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "            \n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.VimeoVideo at 0x1167e39a0>"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import VimeoVideo\n",
    "# Tutorial Video Name: Introduction to R\n",
    "VimeoVideo('327990198', width=720, height=480)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97a5071f-7488-49d6-b420-231fdebff81b",
   "metadata": {},
   "source": [
    "The transcript for the above topic tutorial video **[is located here](https://repo.exeterlms.com/documents/V2/DataScience/Video-Transcripts/DSO102-L06-pg11tutorial.zip)**.\n",
    "\n",
    "# Grouping Data Grouped by Factors\n",
    "\n",
    "A structure like a data frame allows you to group data by factors to determine what effect the factors have on the data. To illustrate this, you will use the ```morley``` data set that is built into R. The data in this data set are the values of the speed of light measured by Michelson in 1879.\n",
    "\n",
    "The data were collected in five experiments; in each experiment, the speed of light was measured 20 times. Each measurement is called a ```Run```. The measured value was the speed of light in km/sec; in the morley data set, 299000 is subtracted from the measured value.\n",
    "\n",
    "You can see the first several rows in this data set with the command ```head(morley)```. This gives:\n",
    "\n",
    "```\n",
    "Expt Run Speed  \n",
    "\n",
    "001 1 1 850  \n",
    "\n",
    "002 1 2 740  \n",
    "\n",
    "003 1 3 900  \n",
    "\n",
    "004 1 4 1070  \n",
    "\n",
    "005 1 5 930  \n",
    "\n",
    "006 1 6 850\n",
    "```\n",
    "You can also see view it in your Data Pane. ```Expt``` is the number of the experiment, ```Run``` is the number of the specific measurement, and ```Speed``` is the measured value.\n",
    "\n",
    "You can create a box plot of this data, grouped by ```Expt```, using the following command:\n",
    "```\n",
    "ggplot(morley, aes(x = Expt, y = Speed)) + geom_boxplot(aes(group=Expt))\n",
    "```\n",
    "```x``` will be the variable you are grouping by, and ```y``` will be the continuous variable to look at. Then you'll specify your grouping variable again in ```(aes(group=)```. These commands give the following box plot: \n",
    "\n",
    "![A boxplot for morley dataframe, grouped by Expt. The x-axis on the bottom reads Expt. The y-axis on the left reads Speed. Experiment 1 and 3 show outliers.](Media/L06-SpeedBoxPlot.png)\n",
    "\n",
    "The argument ```aes(group=Expt)``` tells R to group the data by experiment. All the speed values from Experiment 1 (in other words, each of the speed values from a row in which Expt is equal to 1) are used to create the leftmost box plot. Similarly, all the speed values from Experiment 2 are used to create the next box plot, and so on.\n",
    "\n",
    "You can find the mean speed for each experiment using the ```group_by()``` and ```summarize()``` functions as follows:\n",
    "```\n",
    "morley %>% group_by(Expt) %>% summarize(m.speed = mean(Speed))\n",
    "```\n",
    "Which will yield this tibble:\n",
    "```\n",
    "# A tibble: 5 x 2\n",
    "\n",
    "Expt m.speed\n",
    "\n",
    "<int> <dbl>\n",
    "\n",
    "1 1 909.0\n",
    "\n",
    "2 2 856.0\n",
    "\n",
    "3 3 845.0\n",
    "\n",
    "4 4 820.5\n",
    "\n",
    "5 5 831.5\n",
    "```\n",
    "From this, you can see that the mean speed from Experiment 1 is 909.0; the mean speed from Experiment 2 is 856.0; etc.\n",
    "\n",
    "---\n",
    "\n",
    "# Summary\n",
    "\n",
    "From here on out, most of the data you will work with in R will be stored in data frames or tibbles, as those frameworks are easily fed into statistical analyses. In data frames, the variables are the columns, and the observations are known as the rows. However, to analyze the exact data you want, you may have to manipulate it some, which is where the ```dplyr``` package comes in. Using ```dplyr```, you can filter down to only the data or variables you're interested in, easily summarize and group data, and even create new variables using ```mutate()```.\n",
    "\n",
    "---\n",
    "\n",
    "# Review\n",
    "Below is a quiz to review the recently covered material. Quizzes are *not* graded.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "c1a85f64-1784-40c5-86df-e7905e262b76",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "386db14f942346d6afce3092bc080da7",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Output(), RadioButtons(layout=Layout(width='max-content'), options=(('A. A tibble summarized by…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "try:\n",
    "    display(L6P11Q1)\n",
    "except:\n",
    "    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e891d2d8-fcfc-48d2-b708-d330b2b95824",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}