fall20/625_Data_Transformation_R.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "625-Data-Transformation-R.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyN2DEeH7JB4VBqNr+TqBRzb",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "ir",
      "display_name": "R"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/cs625-datavis-fall20/assignments/blob/master/625_Data_Transformation_R.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QzFjNkPdOLIN"
      },
      "source": [
        "# Data Transformation / CS 625, Fall 2020\n",
        "\n",
        "Exercises: https://github.com/cs625-datavis-fall20/assignments/blob/master/Data-Transformations.md\n",
        "\n",
        "Based on [Chapter 5 (Data Transformations)](https://r4ds.had.co.nz/transform.html) from *R for Data Science*."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yNZyjycigCzZ",
        "outputId": "3f8c5f78-b812-4205-b03d-a31b98905f0e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 81
        }
      },
      "source": [
        "library(tidyverse)\n",
        "install.packages(\"nycflights13\")\n",
        "library(nycflights13)"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Installing package into ‘/usr/local/lib/R/site-library’\n",
            "(as ‘lib’ is unspecified)\n",
            "\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lV4BJ7nqf9XM"
      },
      "source": [
        "## filter()\n",
        "\n",
        "[Exercises (5.2.4)](https://r4ds.had.co.nz/transform.html#exercises-8) from [Chapter 5.2](https://r4ds.had.co.nz/transform.html#filter-rows-with-filter)\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wsAJauGLhOiJ"
      },
      "source": [
        "### Q1\n",
        "\n",
        "Find all flights that\n",
        "\n",
        "* Had an arrival delay of two or more hours"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pFkhHtXJgxnn"
      },
      "source": [
        "* Flew to Houston (IAH or HOU)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L7CiQ63bg9oy"
      },
      "source": [
        "* Were operated by United, American, or Delta"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GM75M-y6g_EA"
      },
      "source": [
        "* Departed in summer (July, August, and September)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v-wVM6UUhAOj"
      },
      "source": [
        "* Arrived more than two hours late, but didn’t leave late"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gIMhdaelhC-L"
      },
      "source": [
        "* Were delayed by at least an hour, but made up over 30 minutes in flight"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ofhbmt8hhE3L"
      },
      "source": [
        "* Departed between midnight and 6am (inclusive)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uOhPjDmbhF-p"
      },
      "source": [
        "### Q2\n",
        "\n",
        "What does `between()` do? Can you use it to simplify the code needed to answer the previous challenges?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ah0fNTBshRTB"
      },
      "source": [
        "## arrange()\n",
        "\n",
        "[Exercises (5.3.1)](https://r4ds.had.co.nz/transform.html#exercises-9) from [Chapter 5.3](https://r4ds.had.co.nz/transform.html#arrange-rows-with-arrange)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yQON4S9qhT14"
      },
      "source": [
        "### Q3\n",
        "How could you use `arrange()` to sort all missing values to the start? \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Po96XN-whXRx"
      },
      "source": [
        "### Q4\n",
        "Sort `flights` to find the most delayed flights. Find the flights that left earliest.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QVB_CTMThZaf"
      },
      "source": [
        "### Q5\n",
        "Sort `flights` to find the fastest flights."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "38B_GlslhcI9"
      },
      "source": [
        "### Q6\n",
        "Which flights travelled the longest? Which travelled the shortest?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZV23JRTEhfez"
      },
      "source": [
        "## select()\n",
        "\n",
        "[Chapter 5.4](https://r4ds.had.co.nz/transform.html#select)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vu3IgmTIh3Jj"
      },
      "source": [
        "### Q7\n",
        "Use ```select()``` in conjunction with ```arrange()``` to sort by the shortest flight (as above) and move the distance column to the first position. This can be in two commands -- we'll look at using pipes later."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wWBY_nRAiLWg"
      },
      "source": [
        "## mutate()\n",
        "\n",
        "[Exercises (5.5.2)](https://r4ds.had.co.nz/transform.html#exercises-11) from [Chapter 5.5](https://r4ds.had.co.nz/transform.html#add-new-variables-with-mutate)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7iJXeo9QiNHX"
      },
      "source": [
        "### Q8\n",
        "\n",
        "Currently `dep_time` and `sched_dep_time` are convenient to look at, but hard to compute with because they’re not really continuous numbers. Convert them to a more convenient representation of number of minutes since midnight.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UvVyvVkhiTxw"
      },
      "source": [
        "### Q9\n",
        "Compare `air_time` with `arr_time - dep_time`. What do you expect to see? What do you see? What do you need to do to fix it?"
      ]
    }
  ]
}