03-viz.Rmd

# (PART) Data Exploration {-} 

# Data Visualization via ggplot2 {#viz}

```{r setup-viz, include=FALSE, purl=FALSE}
chap <- 3
lc <- 0
rq <- 0
# **`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`**
# **`r paste0("(RQ", chap, ".", (rq <- rq + 1), ")")`**
knitr::opts_chunk$set(
  tidy = FALSE, 
  out.width = '\\textwidth', 
  fig.height = 4,
  warning = FALSE
  )
# This bit of code is a bug fix on asis blocks, which we use to show/not show LC solutions, which are written like markdown text. In theory, it shouldn't be necessary for knitr versions <=1.11.6, but I've found I still need to for everything to knit properly in asis blocks. More info here:
# https://stackoverflow.com/questions/32944715/conditionally-display-block-of-markdown-text-using-knitr
library(knitr)
knit_engines$set(asis = function(options) {
  if (options$echo && options$eval) knit_child(text = options$code)
})

# This controls which LC solutions to show. Options for solutions_shown: "ALL" (to show all solutions), or subsets of c('4-3', '4-4', '4-5','4-6','4-7','4-8'), including the null vector c('') to show no solutions.
solutions_shown <- ""
show_solutions <- function(section){return(solutions_shown == "ALL" | section %in% solutions_shown)}
```

We begin the development of your data science toolbox with data visualization. By visualizing our data, we will be able to gain valuable insights from our data that we couldn't initially see from just looking at the raw data in spreadsheet form.  We will use the `ggplot2` package as it provices an easy way to customize your plots and is rooted in the data visualization theory known as _The Grammar of Graphics_ [@wilkinson2005].

At the most basic level, graphics/plots/charts (we use these terms interchangeably in this book) provide a nice way for us to get a sense for how quantitative variables compare in terms of their center (where the values tend to be located) and their spread (how they vary around the center).  The most important thing to know about graphics is that they should be created to make it obvious for your audience to understand the findings and insight you want to get across.  This does however require a balancing act.  On the one hand, you want to highlight as many meaningful relationships and interesting findings as possible, but on the other you don't want to include so many as to overwhelm your audience.  

As we will see, plots/graphics also help us to identify patterns and outliers in our data.  We will see that a common extension of these ideas is to compare the *distribution* of one quantitative variable (i.e., what the spread of a variable looks like or how the variable is *distributed* in terms of its values) as we go across the levels of a different categorical variable.

### Needed packages {-}

Let's load all the packages needed for this chapter (this assumes you've already installed them). Read Chapter \@ref(packages) for information on how to install and load R packages.

```{r message=FALSE}
library(nycflights13)
library(ggplot2)
library(dplyr)
library(knitr)
```

```{r message=FALSE, warning=FALSE, echo=FALSE}
# Packages needed internally, but not in text.
library(gapminder)
library(knitr)
```





<!--Subsection on Grammar of Graphics -->

## The Grammar of Graphics {#grammarofgraphics}

We begin with a discussion of a theoretical framework for data visualization known as the "The Grammar of Graphics," which serves as the basis for the `ggplot2` package.  Much like how we construct sentences in any language by using a linguistic grammar (nouns, verbs, subjects, objects, etc.), the theoretical framework given by Leland Wilkinson [@wilkinson2005] allows us to specify the components of a statistical graphic. 

### Components of Grammar

In short, the grammar tells us that:

> **A statistical graphic is a mapping of `data` variables to `aes`thetic attributes of `geom`etric objects.**

Specifically, we can break a graphic into the following three essential components:

1. `data`: the dataset comprised of variables that we map.
1. `geom`: the geometric object in question. This refers to our type of objects we can observe in our plot. For example, points, lines, bars, etc.
1. `aes`: aesthetic attributes of the geometric object that we can perceive on a graphic. For example, x/y position, color, shape, and size.  Each assigned aesthetic attribute can be mapped to a variable in our dataset.

Let's break down the grammar with an example.

### Gapminder

```{r, echo=FALSE}
data("gapminder")
gapminder <- gapminder %>% 
  filter(year == 2007) %>% 
  select(-year) %>% 
  rename(
    Country = country,
    Continent = continent,
    `Life Expectancy` = lifeExp,
    `Population` = pop,
    `GDP per Capita` = gdpPercap
  )
```

In February 2006, a statistician named Hans Rosling gave a TED talk titled ["The best stats you've ever seen"](https://www.ted.com/talks/hans_rosling_shows_the_best_stats_you_ve_ever_seen) where he presented global economic, health, and development data from the website [gapminder.org](http://www.gapminder.org/tools/#_locale_id=en;&chart-type=bubbles). For example, from the `r nrow(gapminder)` countries included from 2007, consider only the first 6 countries when listed alphabetically:

```{r, echo=FALSE}
gapminder %>% 
  head() %>% 
  kable(
    digits=2,
    caption = "Gapminder 2007 Data", 
    booktabs = TRUE
  )
```

Each row in this table corresponds to a country in 2007. For each row, we have 5 columns:

1. Country: Name of country.
1. Continent: Which of the 5 continents the country is part of.
1. Life Exp: Life expectancy in years.
1. Population: Population.
1. GDP per Capita: Gross domestic product (in US dollars).

Now consider Figure \@ref(fig:gapminder), which plots this data for all `r nrow(gapminder)` countries. Note that in the legend for "Population", 1.25e+09 = $1.25 \times 10^{9}$ = 1,250,000,000 = 1.25 billion. 

```{r gapminder, echo=FALSE, fig.cap="Life Expectancy over GDP per Capita in 2007"}
ggplot(data=gapminder, aes(x=`GDP per Capita`, y=`Life Expectancy`, size=Population, col=Continent)) +
  geom_point()
```

Let's view this plot through the grammar of graphics:

1. The `data` variable "GDP per Capita" gets mapped to the `x`-position `aes`thetic of the points.
1. The `data` variable "Life Expectancy" gets mapped to the `y`-position `aes`thetic of the points.
1. The `data` variable "Population" gets mapped to the `size` `aes`thetic of the points.
1. The `data` variable "Continent" gets mapped to the `color` `aes`thetic of the points.

While in this example we are considering one type of `geom`etric object (of type `point`), graphics are not limited to just points. Some plots involve lines while others involve bars. Let's summarize the three essential components of the grammar in a table:

```{r, echo=FALSE}
map <- data_frame(
  data = c("GDP per Capita", "Life Expectancy", "Population", "Continent"),
  aes = c("x", "y", "size", "color"),
  geom = c("point", "point", "point", "point")
)

map %>% 
  kable(
    caption = "Summary of Grammar of Graphics", 
    booktabs = TRUE
    )
```

### Other Components of the Grammar

There are other components of the Grammar of Graphics we can control.  As you start to delve deeper into the Grammar of Graphics, you'll start to encounter these topics more and more often. In this book however we'll only work with the first two; the other components are left to a more advanced text.

- `facet`ting breaks up a plot into subsets (Chapter \@ref(facets))
- `position` adjustments for barplots (Chapter \@ref(geombar))
- `scales` that both
    + convert *data units* to *physical units* the computer can display. For example, apply a log-transformation on one of the axes to focus on multiplicative rather than additive changes.
    + draw a legend and/or axes, which provide an inverse mapping to make it possible to read the original data values from the graph.
- `coord`inate system for x/y values: typically `cartesian`, but can also be `map` or `polar`.
- `stat`istical transformations: this includes smoothing, binning values into a histogram, or no transformation at all (known as the `"identity"` transformation).

In general, the Grammar of Graphics allows for a high degree of customization and also a consistent framework for easy updating/modification of plots.

### The ggplot2 Package

In this book, we will be using the `ggplot2` package for data visualization, which is an implementation of the Grammar of Graphics for R [@R-ggplot2]. You may have noticed that a lot of the previous text in this chapter is written in computer font. This is because the various components of the Grammar of Graphics are specified in the `ggplot` function, which expects at a bare minimal as arguments:

* The data frame where the variables exist: the `data` argument.
* The mapping of the variables to aesthetic attributes: the `mapping` argument, which specifies the `aes`thetic attributes involved.

After we've specified these components, we then add *layers* to the plot. The most essential layer to add to a plot is the specification of which type of `geom`etric object we want the plot to involve e.g. points, lines, bars. Other layers we can add include the specfication of the plot title, axes labels, and visual themes for the plot.

Let's now put the theory of the Grammar of Graphics into practice.

<!--
The plot given above is not a histogram, but the output does show us a bit of what is going on with `ggplot(data = weather, mapping = aes(x = temp))`.  It is producing a backdrop onto which we will "paint" elements.  We next proceed by adding a layer---hence, the use of the `+` symbol---to the plot to produce a histogram.  (Note also here that we don't have to specify the `data = ` and `mapping = ` text in our function calls.  This is covered in more detail in Chapter 5 of the "Getting Used to R, RStudio, and R Markdown" book [@usedtor2016]).
-->

<!--
```{block viz_review, type='review'}
**_Review questions_**
```
**`paste0("(RQ", chap, ".", (rq <- rq + 1), ")")`**
- Have a variety of bad plots with data for the readers and have readers create better plots with `ggplot2`
- Have sample datasets to work with from problem statements
    + Identify the appropriate plot to address the questions of interest
- Why is it important for barplots to start at zero?
-->





<!--Subsection on 5NG -->

## Five Named Graphs - The 5NG {#5NG}

For our purposes, we will be limiting consideration to five different types of graphs.  We term these five named graphs the **5NG**:

1. scatter-plots
1. line-graphs
1. boxplots
1. histograms
1. barplots

We will discuss some variations of these plots, but with this basic repertoire in your toolbox you can visualize a wide array of different data variable types. Note that certain plots are only appropriate for categorical/logical variables and others only for quantitative variables. You'll want to quiz yourself often as we go along on which plot makes sense a given a particular problem or dataset.





<!--Subsection on scatter plots-->

## 5NG#1: Scatter-plots {#scatterplots}

The simplest of the 5NG are *scatter-plots* (also called bivariate plots); they allow you to investigate the relationship between two continuous variables. While you may already be familiar with this type of plot, let's view it through the lens of the Grammar of Graphics. Specifically, we will graphically investigate the relationship between the following two continuous variables in the `flights` data frame:

1. `dep_delay`: departure delay on the horizontal "x" axis and
1. `arr_delay`: arrival delay on the vertical "y" axis

for Alaska Airlines flights leaving NYC in 2013. This requires paring down the `flights` data frame to a smaller data frame `all_alaska_flights` consisting of only Alaska Airlines (carrier code "AS") flights. 

```{r}
data(flights)
all_alaska_flights <- flights %>% 
  filter(carrier == "AS")
```

This code snippet makes use of functions in the `dplyr` package for data manipulation to achieve our goal: it takes the `flights` data frame and `filter`s it to only return the rows which meet the condition `carrier == "AS"`. Recall from Chapter \@ref(code) that testing for equality is specified with `==` and not `=`. You will see many more examples of this in Chapter \@ref(wrangling).        

***

```{block lc-all_alaska_flights, type='learncheck', purl=FALSE}
**_Learning check_**
```

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`**  Take a look at both the `flights` and `all_alaska_flights` data frames by running `View(flights)` and `View(all_alaska_flights)` in the console. In what respect do these data frames differ?

```{asis lc-all_alaska_flights-solutions, include=show_solutions('4-3')}
Fill in LC solution here:
**Learning Check Solutions**  
`flights` contains all flight data, while `all_alaska_flights` contains only data from Alaskan carrier "AS". We can see that flights has `r nrow(flights)` rows while `all_alaska_flights` has only `r nrow(all_alaska_flights)`
```
***

### Scatter-plots via geom_point {#geompoint}

We proceed to create the scatter-plot using the `ggplot()` function:

```{r noalpha, fig.cap="Arrival Delays vs Departure Delays for Alaska Airlines flights from NYC in 2013"}
ggplot(data = all_alaska_flights, aes(x = dep_delay, y = arr_delay)) + 
  geom_point()
```

In Figure \@ref(fig:noalpha) we see that a positive relationship exists between `dep_delay` and `arr_delay`: as departure delays increase, arrival delays tend to also increase.  We also note that the majority of points fall near the point (0, 0).  There is a large mass of points clustered there. Let's break down this keeping in mind our discussion in Chapter \@ref(grammarofgraphics):

* Within the `ggplot()` function call, we specify two of the components of the grammar:
    1. The `data` frame to be `all_alaska_flights` by setting `data = all_alaska_flights`
    1. The `aes`thetic mapping by setting `aes(x = dep_delay, y = arr_delay)`. Specifically
        * `dep_delay` maps to the `x` position
        * `arr_delay` maps to the `y` position
* We add a layer to the `ggplot()` function call using the `+` sign. The layer in question specifies the third component of the grammar:  the `geom`etric object. In this case the geometric object are `point`s, set by specifying `geom_point()`.

Some notes on layers:

* Note that the `+` sign comes at the end of lines, and not at the beginning.
* When adding layers to a plot, you are encouraged to hit *Return* on your keyboard after entering the `+` so that the code for each layer is on a new line.  As we add more and more layers to plots, you'll see this will greatly improve the legibility of your code.
* To stress the importance of adding layers, in particular the layer specifying the `geom`etric object, consider Figure \@ref(fig:nolayers) where no layers are added. A not very useful plot!

```{r nolayers, fig.cap="Plot with No Layers"}
ggplot(data = all_alaska_flights, aes(x = dep_delay, y = arr_delay))
```


***

```{block lc-scatter-plots, type='learncheck', purl=FALSE}
**_Learning check_**
```

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`**  What are some practical reasons why `dep_delay` and `arr_delay` have a positive relationship?

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`**  What variables (not necessarily in the `flights` data frame) would you expect to have a negative correlation (i.e. a negative relationship) with `dep_delay`? Why? Remember that we are focusing on continuous variables here.

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** Why do you believe there is a cluster of points near (0, 0)? What does (0, 0) correspond to in terms of the Alaskan flights?

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** What are some other features of the plot that stand out to you?

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** Create a new scatter-plot using different variables in the `all_alaska_flights` data frame by modifying the example above.

```{asis lc-scatter-plots-solutions, include=show_solutions('4-3')}
**Learning Check Solutions**  
**4.2**: What are some practical reasons why `dep_delay` and `arr_delay` have a positive relationship? *The later a plane departs, typically the later it will arrive.*  
**4.3** An example in the `weather` dataset is `visibility`, which measure visibility in miles. As visibility increases, we would expect departure delays to decrease. 

**4.4**: What does (0, 0) correspond to from the point of view of a passenger on an Alaskan flight?  Why do you believe there is a cluster of points near (0, 0)? *The point (0,0) means no delay in departure and arrival. From the passenger's point of view, this means the flight was on time. It seems most flights are at least close to being on time.*  

**4.6**: Create a similar plot, but one showing the relationship between departure time and departure delay. What hypotheses do you have about the patterns you see? *We now put `dep_time` as the `x`-aesthetic and `dep_delay` as the `y`-aesthetic*  
```

``` {r, include=show_solutions('4-3'), echo=show_solutions('4-3')}
ggplot(data=all_alaska_flights, aes(x = dep_time, y = dep_delay)) +
  geom_point()
```

***


### Over-Plotting

The large mass of points near (0, 0) can cause some confusion.  This is the result of a phenomenon called *over-plotting*.  As one may guess, this corresponds to values being plotted on top of each other _over_ and _over_ again.  It is often difficult to know just how many values are plotted in this way when looking at a basic scatter-plot as we have here. There are two ways to address this issue:

1. By adjusting the transparency of the points via the `alpha` argument
1. By jittering the points via `geom_jitter()`

The first way of relieving over-plotting is by changing the `alpha` argument in `geom_point()` which controls the transparency of the points.  By default, this value is set to `1`.  We can change this to any value between `0` and `1` where `0` sets the points to be 100% transparent and `1` sets the points to be 100% opaque. Note how the following function call is identical to the one in Chapter \@ref(scatterplots), but with `alpha = 0.2` added to the `geom_point()`.

```{r alpha, fig.cap="Delay scatterplot with alpha=0.2"}
ggplot(data = all_alaska_flights, aes(x = dep_delay, y = arr_delay)) + 
  geom_point(alpha = 0.2)
```

The key feature to note in Figure \@ref(fig:alpha) is that the transparency of the points is cummulative: areas with a high-degree of over-plotting are darker, whereas areas with a lower degree are less dark. 

The second way of relieving over-plotting is to *jitter* the points a bit.  In other words, we are going to add just a bit of random noise to the points to better see them and remove some of the over-plotting.  You can think of "jittering" as shaking the points around a bit on the plot. Instead of using `geom_point`, we use `geom_jitter` to perform this shaking. To specify how much jitter to add, we adjust the `width` and `height` arguments.  This corresponds to how hard you'd like to shake the plot in units corresponding to those for both the horizontal and vertical variables (in this case minutes).

```{r jitter, fig.cap="Jittered delay scatterplot"}
ggplot(data = all_alaska_flights, aes(x = dep_delay, y = arr_delay)) + 
  geom_jitter(width = 30, height = 30)
```

Note how this function call is identical to the one in Chapter \@ref(geompoint), but with `geom_point()` replaced with `geom_jitter()`. The plot in \@ref(fig:jitter) helps us a little bit in getting a sense for the over-plotting, but with a relatively large dataset like this one (`r nrow(all_alaska_flights)` flights), it can be argued that changing the transparency of the points by setting `alpha` proved more effective.

***

```{block lc-overplotting, type='learncheck', purl=FALSE}
**_Learning check_**
```

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`**  Why is setting the `alpha` argument value useful with scatter-plots? What further information does it give you that a regular scatter-plot cannot?

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** After viewing the Figure \@ref(fig:alpha) above, give an approximate range of arrival times and departure times that occur the most frequently.  How has that region changed compared to when you observed the same plot without the `alpha = 0.2` set in Figure \@ref(fig:noalpha)?

```{asis lc-overplotting-solutions, include=show_solutions('4-3')}
**Learning Check Solutions**  
**4.7**: Why is setting the `alpha` argument value useful with scatter-plots? What further information does it give you that a regular scatter-plot cannot? *It thins out the points so we address over-plotting. But more importantly it hints at the (statistical) **density** and **distribution** of the points: where are the points concentrated, where do they occur. We will see more about densities and distributions in Chapter 6 when we switch gears to statistical topics.*
  
**4.8**: After viewing the Figure \@ref(fig:alpha) above, give a range of arrival times and departure times that occur most frequently?  How has that region changed compared to when you observed the same plot without the `alpha = 0.2` set in Figure \@ref(fig:noalpha)? *The lower plot suggests that most Alaska flights from NYC depart between 12 minutes early and on time and arrive between 50 minutes early and on time.*  

```
***

<!--
Maybe include a shading of the points by another variable example here for multivariate thinking?
-->

### Summary

Scatter-plots display the relationship between two continuous variables and is among the most commonly used plot because they can provide an immediate way to see the trend in one variable versus another.  However, if you try to create a scatter-plot where either one of the two variables is not quantitative, you will get strange results.  Be careful!

With medium to large datasets, you may need to play with either `geom_jitter` or the `alpha` argument in order to get a good feel for relationships in your data.  This tweaking is often a fun part of data visualization since you'll have the chance to see different relationships come about as you make subtle changes to your plots.





<!--Subsection on line graphs-->

## 5NG#2: Line-graphs {#linegraphs}

The next of the 5NG is a line-graph.  They are most frequently used when the x-axis represents time and the y-axis represents some other numerical variable; such plots are known as *time series*.  Time represents a variable that is connected together by each day following the previous day.  In other words, time has a natural ordering.  Line-graphs should be avoided when there is not a clear sequential ordering to the explanatory variable, i.e. the x-variable or the *predictor* variable.

Our focus now turns to the `temp` variable in this `weather` dataset. By

* Looking over the `weather` dataset by typing `View(weather)` in the console. 
* Running `?weather` to bring up the help file.

We can see that the `temp` variable corresponds to hourly temperature (in Fahrenheit) recordings at weather stations near airports in New York City. Instead of considering all hours in 2013 for all three airports in NYC, let's focus on the hourly temperature at Newark airport (`origin` code "EWR") for the first 15 days in January 2013. The `weather` data frame in the `nycflights13` package contains this data, but we first need to filter it to only include those rows that correspond to Newark in the first 15 days of January.

```{r}
data(weather)
early_january_weather <- weather %>% 
  filter(origin == "EWR" & month == 1 & day <= 15)
```

This is similar to the previous use of the `filter` command in Chapter \@ref(scatterplots), however we now use the `&` operator. The above selects only those rows in `weather` where `origin == "EWR"` **and** `month = 1` **and** `day <= 15`.
     
***

```{block lc-early_january_weather, type='learncheck', purl=FALSE}
**_Learning check_**
```

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`**  Take a look at both the `weather` and `early_january_weather` data frames by running `View(weather)` and `View(early_january_weather)` in the console. In what respect do these data frames differ?

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`**  The weather data is recorded hourly. Why does the `time_hour` variable correctly identify the hour of the measurement whereas the `hour` variable does not?

```{asis lc-early_january_weather-solutions, include=show_solutions('4-4')}
**Learning Check Solutions**  
**LC4.9**: Take a look at both the `weather` and `early_january_weather` data frames by running `View(weather)` and `View(early_january_weather)` in the console. In what respect do these data frames differ? *The rows of `early_january_weather` are a subset of `weather`.*  
**LC4.10**: The `weather` data is recorded hourly. Why does the `time_hour` variable correctly identify the hour of the measurement whereas the `hour` variable does not? *Because to uniquely identify an hour, we need the `year`/`month`/`day`/`hour` sequence, whereas there are only 24 possible `hour`'s. Note that in the case of `weather`, there is a timezone bug: the `time_hour` variable is off by 5 hours from the `year`/`month`/`day`/`hour` sequence, since the Eastern Time Zone is 5 hours off UTC.*  
```
***

### Line-graphs via geom_line {#geomline}

We plot a line-graph of hourly temperature using `geom_line()`:

```{r hourlytemp, fig.cap="Hourly Temperature in Newark for Jan 1-15 2013"}
ggplot(data = early_january_weather, aes(x = time_hour, y = temp)) +
  geom_line()
```

Much as with the `ggplot()` call in Chapter \@ref(geompoint), we specify the components of the Grammar of Graphics:

* Within the `ggplot()` function call, we specify two of the components of the grammar:
    1. The `data` frame to be `early_january_weather` by setting `data = early_january_weather`
    1. The `aes`thetic mapping by setting `aes(x = time_hour, y = temp)`. Specifically
        * `time_hour` (i.e. the time variable) maps to the `x` position
        * `temp` maps to the `y` position
* We add a layer to the `ggplot()` function call using the `+` sign
* The layer in question specifies the third component of the grammar:  the `geom`etric object in question. In this case the geometric object is a `line`, set by specifying `geom_line()`. 

***

```{block lc-line-graph, type='learncheck', purl=FALSE}
**_Learning check_**
```

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`**  Why should line-graphs be avoided when there is not a clear ordering of the horizontal axis?

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** Why are line-graphs frequently used when time is the explanatory variable?

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** Plot a time series of a variable other than `temp` for Newark Airport in the first 15 days of January 2013.

```{asis lc-line-graph-solutions, include=show_solutions('4-4')}
**Learning Check Solutions**  
**LC4.11**: Why should line-graphs be avoided when there is not a clear ordering of the horizontal axis? *Because lines suggest connectedness and ordering.*  
**LC4.12**: Why are line-graphs frequently used when time is the explanatory variable? *Because time is sequential: subsequent observations are closely related to each other.*  
**LC4.13**: Plot a time series of a variable other than `temp` for Newark Airport in the first 15 days of January 2013. *Humidity is a good one to look at, since this very closely related to the cycles of a day.*  
```

```{r, include=show_solutions('4-4'), echo=show_solutions('4-4')}
ggplot(data = early_january_weather, aes(x = time_hour, y = humid)) +
  geom_line()
```

***

### Summary

Line-graphs, just like scatter-plots, display the relationship between two continuous variables. However, the variable on the x-axis (i.e. the explanatory variable) should have a natural ordering, like some notion of time.  We can mislead our audience if that isn't the case.





<!--Subsection on histograms -->

## 5NG#3: Histograms {#histograms}

Let's consider the `temp` variable in the `weather` data frame once again, but now unlike with the line-graphs in Chapter \@ref(linegraphs), let's say we don't care about the relationship of temperature to time, but rather we care about the *(statistical) distribution* of temperatures. We could just produce points where each of the different values appear on something similar to a number line:

```{r echo=FALSE, fig.height=0.8, fig.cap="Plot of Hourly Temperature Recordings from NYC in 2013"}
ggplot(data = weather, mapping = aes(x = temp, y = factor("A"))) +
  geom_point() +
  theme(axis.ticks.y = element_blank(), 
        axis.title.y = element_blank(),
        axis.text.y = element_blank())
hist_title <- "Histogram of Hourly Temperature Recordings from NYC in 2013"
```

This gives us a general idea of how the values of `temp` differ.  We see that temperatures vary from around `r round(min(weather$temp, na.rm = TRUE), 0)` up to `r round(max(weather$temp, na.rm = TRUE), 0)` degrees Fahrenheit.  The area between 40 and 60 degrees appears to have more points plotted than outside that range.


### Histograms via geom_histogram {#geomhistogram}

What is commonly produced instead of the above plot is a plot known as a *histogram*.  The histogram shows how many elements of a single numerical variable fall in specified *bins*.  In this case, these bins may correspond to between 0-10&deg;F, 10-20&deg;F, etc. We produce a histogram of the hour temperatures at all three NYC airports in 2013:

```{r, warning=TRUE, fig.cap=hist_title}
ggplot(data = weather, mapping = aes(x = temp)) +
  geom_histogram()
```

Note here:

* There is only one variable being mapped in `aes()`: the single continuous variable `temp`. You don't need to compute the y-aesthetic: it gets computed automatically.
* We set the `geom`etric object to be `geom_histogram()`
* We got a warning message of `1 rows containing non-finite values` being removed. This is due to one of the values of temperature being missing. R is alerting us that this happened.


### Adjusting the Bins {#adjustbins}

We can adjust characteristics of the bins in one of *two* ways:

1. By adjusting the number of bins via the `bins` argument
1. By adjusting the width of the bins via the `binwidth` argument

First, we have the power to specify how many bins we would like to put the data into as an argument in the `geom_histogram` function.  By default, this is chosen to be 30 somewhat arbitrarily; we have received a warning above our plot that this was done.

```{r fig.cap=paste(hist_title, "- 60 Bins")}
ggplot(data = weather, mapping = aes(x = temp)) +
  geom_histogram(bins = 60, color = "white")
```

Note the addition of the `color` argument.  If you'd like to be able to more easily differentiate each of the bins, you can specify the color of the outline as done above.

Second, instead of specifying the number of bins, we can also specify the width of the bins by using the `binwidth` argument in the `geom_histogram` function. 

```{r fig.cap=paste(hist_title, "- Binwidth = 10"), fig.height=5}
ggplot(data = weather, mapping = aes(x = temp)) +
  geom_histogram(binwidth = 10, color = "white")
```

***

```{block lc-histogram, type='learncheck', purl=FALSE}
**_Learning check_**
```

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** What does changing the number of bins from 30 to 60 tell us about the distribution of temperatures?

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** Would you classify the distribution of temperatures as symmetric or skewed?

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** What would you guess is the "center" value in this distribution?  Why did you make that choice?

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** Is this data spread out greatly from the center or is it close?  Why?


```{asis lc-histogram-solutions, include=show_solutions('4-5')}
**Learning Check Solutions**  
**LC 4.14: What does changing the number of bins from 30 to 60 tell us about the distribution of temperatures?** The distribution doesn't change much. But by refining the bid width, we see that the temperature data has a high degree of accuracy. What do I mean by accuracy? Looking at the `temp` variabile by `View(weather)`, we see that the precision of each temperature recording is 2 decimal places.

**LC 4.15: Would you classify the distribution of temperatures as symmetric or skewed?** It is rather symmetric, i.e. there are no __long tails__ on only one side of the distribution

**LC 4.16: What would you guess is the "center" value in this distribution?  Whydid you make that choice?** The center is around 55&deg;F. By running the `summary()` command, we see that the mean and median are very similar. In fact, when the distribution is symmetric the mean equals the median.

**4.17: Is this data spread out greatly from the center or is it close? Why?**

This can only be answered relatively speaking! Let's pick things to be relative to Seattle, WA temperatures:

![alt text](images/temp.png)

While, it appears that Seattle weather has a similar center of 55&deg;F, its
temperatures are almost entirely between 35&deg;F and 75&deg;F for a range of
about 40&deg;F. Seattle temperatures are much less spread out than New York i.e.
much more consistent over the year. New York on the other hand has much colder
days in the winter and much hotter days in the summer. Expressed differently,
the middle 50% of values, as delineated by the interquartile range is 30&deg;F:
```

```{r, echo=show_solutions('4-5'), include=show_solutions('4-5'), message=FALSE, warning=FALSE}
IQR(weather$temp, na.rm=TRUE)
```

```{r, echo=show_solutions('4-5'), include=show_solutions('4-5'), message=FALSE, warning=FALSE}
summary(weather$temp)
```

***


### Summary

Histograms, unlike scatter-plots and line-graphs, present information on only a single continuous variable. In particular they are visualizations of the (statistical) distribution of values. 





## Facets {#facets}

Before continuing the 5NG, we briefly introduce a new concept called *faceting*.  Faceting is used when we'd like to create small multiples of the same plot over a different categorical variable.  By default, all of the small multiples will have the same vertical axis. 

For example, suppose we were interested in looking at how the temperature histograms we saw in Chapter \@ref(histograms) varied by month.  This is what is meant by "the distribution of a variable over another variable": `temp` is one variable and `month` is the other variable. In order to look at histograms of `temp` for each month, we add a layer `facet_wrap(~month)`.  You can also specify how many rows you'd like the small multiple plots to be in using `nrow` or how many columns using `ncol` inside of `facet_wrap`.

```{r facethistogram, fig.cap="Faceted histogram"}
ggplot(data = weather, aes(x = temp)) +
  geom_histogram(binwidth = 5, color = "white") +
  facet_wrap(~ month, nrow = 4)
```

As we might expect, the temperature tends to increase as summer approaches and then decrease as winter approaches.

***

```{block lc-facet, type='learncheck', purl=FALSE}
**_Learning check_**
```

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** What other things do you notice about the faceted plot above?  How does a faceted plot help us see relationships between two variables?

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** What do the numbers 1-12 correspond to in the plot above?  What about 25, 50, 75, 100?

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** For which types of datasets would these types of faceted plots not work well in comparing relationships between variables? Give an example describing the nature of these variables and other important characteristics.

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** Does the `temp` variable in the `weather` dataset have a lot of variability?  Why do you say that?

```{asis lc-facet-solutions, include=show_solutions('4-6')}
**Learning Check Solutions**  
**`r paste0("(LC", chap, ".", (lc - 3), ")")`: What other things do you notice about the faceted plot above? How does afaceted plot help us see relationships between two variables?** 
* Certain months have much more consistent weather (August in particular), while others have crazy variability like January and October, representing changes in the seasons.
* The two variables we are see the relationship of are `temp` and `month`.
    
**LC `r paste0("(LC", chap, ".", (lc - 2), ")")`: What do the numbers 1-12 correspond to in the plot above? What about 25, 50, 75, 100?**
* While month is technically a number between 1-12, we're viewing it as a categorical variable here. Specifically an **ordinal categorical** variable since there is a ordering to the categories
* 25, 50, 75, 100 are temperatures

**`r paste0("(LC", chap, ".", (lc - 1), ")")`: For which types of datasets would these types of faceted plots not work well in comparing relationships between variables? Give an example describing the variability of the variables and other important characteristics?**
Having histograms split by day would not be great:
* We'd have 365 facets to look at. Way to many.
* We don't really care about day-to-day fluctuation in weather so much, but maybe more week-to-week variation. We'd like to focus on seasonal trends.

**`r paste0("(LC", chap, ".", (lc), ")")`: Does the `temp` variable in the `weather` dataset have a lot of variability? Why do you say that?**
Again, like in LC 4.17, this is a relative question. I would say yes, because in New York City, you have 4 clear seasons with different weather. Whereas in Seattle WA and Portland OR, you have two seasons: summer and rain!

```
***





<!--Subsection on boxplots -->

## 5NG#4: Boxplots {#boxplots}

While using faceted histograms can provide a way to compare distributions of a continuous variable split by groups of a categorical variable as in Chapter \@ref(facets), an alternative plot called a *boxplot* (also called a *side-by-side boxplot*) achieves the same task and is frequently preferred.  The *boxplot* uses the information provided in the *five-number summary* referred to in Appendix \@ref(appendixA).  It gives a way to compare this summary information across the different levels of a categorical variable.  


### Boxplots via geom_boxplot {#geomboxplot}

Let's create a boxplot to compare the monthly temperatures as we did above with the faceted histograms.

```{r badbox, fig.cap="Invalid boxplot specification", fig.height=3.5}
ggplot(data = weather, aes(x = month, y = temp)) +
  geom_boxplot()
```

```
Warning messages:
1: Continuous x aesthetic -- did you forget aes(group=...)? 
2: Removed 1 rows containing non-finite values (stat_boxplot). 
```

Note the set of warnings that is given here. The second warning corresponds to missing values in the data frame and it is turned off on subsequent plots. Let's focus on the first warning. 

Observe that this plot does not look like what we were expecting.  We were expecting to see the distribution of temperatures for each month (so 12 different boxplots).  The first warning is letting us know that we are plotting a continuous, and not categorical variable, on the x-axis. This gives us the overall boxplot without any other groupings.  We can get around this by introducing a new function for our `x` variable:

```{r monthtempbox, fig.cap="Month by temp boxplot", fig.height=3.7}
ggplot(data = weather, mapping = aes(x = factor(month), y = temp)) +
  geom_boxplot()
```

We have introduced a new function called `factor()` here.  One of the things this function does is to convert a discrete value like `month` (1, 2, ..., 12) into a categorical variable.  The "box" part of this plot represents the 25^th^ percentile, the median (50^th^ percentile), and the 75^th^ percentile.  The dots correspond to *outliers*.  (The specific formulation for these outliers is discussed in Appendix \@ref(appendixA).)  The lines show how the data varies that is not in the center 50% defined by the first and third quantiles.  Longer lines correspond to more variability and shorter lines correspond to less variability.

***

```{block lc-boxplot, type='learncheck', purl=FALSE}
**_Learning check_**
```

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** What does the dot at the bottom of the plot for May correspond to?  Explain what might have occurred in May to produce this point.

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** Which months have the highest variability in temperature?  What reasons do you think this is?

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** We looked at the distribution of a continuous variable over a categorical variable here with this boxplot.  Why can't we look at the distribution of one continuous variable over the distribution of another continuous variable?  Say, temperature across pressure, for example?

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** Boxplots provide a simple way to identify outliers.  Why may outliers be easier to identify when looking at a boxplot instead of a faceted histogram?

```{asis lc-boxplot-solutions, include=show_solutions('4-7')}
**Learning Check Solutions**  
**`r paste0("(LC", chap, ".", (lc - 3), ")")`: What does the dot at the bottom of the plot for May correspond to? Explain what might have occurred in May to produce this point.**
It appears to be an outlier. Let's revisit the use of the `filter` command to hone in on it. We want all data points where the `month` is 5 and `temp<25`

```

```{r, echo=show_solutions('4-7'), eval=FALSE}
weather %>% 
  filter(month==5 & temp < 25)
```

```{r, include=show_solutions('4-7'), echo=FALSE}
weather %>% 
  filter(month==5 & temp < 25) %>% 
  kable()
```

```{asis, include=show_solutions('4-7')}
There appears to be only one hour and only at JFK that recorded 13.1 F (-10.5 C) in the month of May. This is probably a data entry mistake!
Why wasn't the weather at least similar at EWR (Newark) and LGA (La Guardia)?


**`r paste0("(LC", chap, ".", (lc - 2), ")")`: Which months have the highest variability in temperature? What reasons do you think this is?**
We are now interested in the **spread** of the data. One measure some of you may have seen previously is the standard deviation. But in this plot we can read off the Interquartile Range (IQR):

* The distance from the 1st to the 3rd quartiles i.e. the length of the boxes
* You can also think of this as the spread of the **middle 50%** of the data

Just from eyeballing it, it seems

* November has the biggest IQR, i.e. the widest box, so has the most variation in temperature
* August has the smallest IQR, i.e. the narrowest box, so is the most consistent temperature-wise

Here's how we compute the exact IQR values for each month (we'll see this more in depth Chapter 5 of the text):

1. `group` the observations by `month` then
1. for each `group`, i.e. `month`, `summarise` it by applying the summary statistic function `IQR()`, while making sure to skip over missing data via `na.rm=TRUE` then
1. `arrange` the table in `desc`ending order of `IQR`
```

```{r, echo=show_solutions('4-7'), eval=FALSE}
weather %>% 
  group_by(month) %>% 
  summarise(IQR = IQR(temp, na.rm=TRUE)) %>% 
  arrange(desc(IQR))
```

```{r, echo=FALSE, include=show_solutions('4-7')}
weather %>% 
  group_by(month) %>% 
  summarise(IQR = IQR(temp, na.rm=TRUE)) %>% 
  arrange(desc(IQR)) %>% 
  kable()
```

```{asis, include=show_solutions('4-7')}
**`r paste0("(LC", chap, ".", (lc - 1), ")")`: We looked at the distribution of a continuous variable over a categorical variable here with this boxplot. Why can't we look at the distribution of one continuous variable over the distribution of another continuous variable? Say, temperature across pressure, for example?**
Because we need a way to group many continuous observations together, say by grouping by month. For pressure, we have near unique values for pressure, i.e. no groups, so we can't make boxplots.


**`r paste0("(LC", chap, ".", (lc), ")")`: Boxplots provide a simple way to identify outliers. Why may outliers be easier to identify when looking at a boxplot instead of a faceted histogram?**
In a histogram, the bin corresponding to where an outlier lies may not by high enough for us to see. In a boxplot, they are explicitly labelled separately. 
```
***

### Summary

Boxplots provide a way to compare and contrast the distribution of one quantitative variable across multiple levels of one categorical variable. One can see where the median falls across the different groups by looking at the center line in the box. To see how spread out the variable is across the different groups, look at both the width of the box and also how far the lines stretch vertically from the box. (If the lines stretch far from the box but the box has a small width, the variability of the values closer to the center is much smaller than the variability of the outer ends of the variable.) Outliers are even more easily identified when looking at a boxplot than when looking at a histogram.





<!--Subsection on barplots -->

## 5NG#5: Barplots {#geombar}

Both histograms and boxplots represent ways to visualize the variability of continuous variables. Another common task is to present the distribution of a categorical variable. This is a simpler task, focused on how many elements from the data fall into different categories of the categorical variable. Often the best way to visualize these different counts (also known as *frequencies*) is via a barplot, also known as a barchart.

One complication, however, is how your counts are represented in your data. For example, run the following code in your Console. This code manually creates two data frames representing counts of fruit.

```{r}
fruits <- data_frame(
  fruit = c("apple", "apple", "apple", "orange", "orange")
)
fruits_counted <- data_frame(
  fruit = c("apple", "orange"),
  number = c(3, 2)
)
```

We see both the `fruits` and `fruits_counted` data frames represent the same collection of fruit: three apples and two bananas. However, whereas `fruits` just lists the fruit:

```{r, echo=FALSE}
kable(
    fruits,
    digits=2,
    caption = "Fruits", 
    booktabs = TRUE
  )
```

`fruits_counted` has a variable `count`, where the counts are pre-tabulated. 

```{r, echo=FALSE}
kable(
    fruits_counted,
    digits=2,
    caption = "Fruits (Pre-Counted)", 
    booktabs = TRUE
  )
```

Compare the barcharts in Figures \@ref(fig:geombar) and \@ref(fig:geomcol), which are identical, but are based on two different data frames:

```{r geombar, fig.cap="Barplot when counts are not pre-tabulated", fig.height=2.5}
ggplot(data = fruits, mapping = aes(x=fruit)) +
  geom_bar()
```

```{r, geomcol, fig.cap="Barplot when counts are pre-tabulated", fig.height=2.5}
ggplot(data = fruits_counted, mapping = aes(x=fruit, y=number)) +
  geom_col()
```

Observe that:

* The code that generates Figure \@ref(fig:geombar) based on `fruits` does not have an explicit `y` `aes`thetic and uses `geom_bar()`
* The code that generates Figure \@ref(fig:geomcol) based on `fruits_counted` has an explicit `y` `aes`thetic (to the variable `number`) and uses `geom_col()`

This one aspect of creating barplots using `ggplot2` that causes the most initial confusion: when the categorical variable you want to plot is not pre-tabulated in your data frame you need to use `geom_bar`, but if the categorical variable is pre-tabulted and stored in a variable, you need to use `geom_col` and explicitly map this variable to the `y` aesthetic.



### Barplots via geom_bar/geom_col

Consider the distribution of airlines that flew out of New York City in 2013.  Here we explore the number of flights from each airline/`carrier`.  This can be plotted by invoking the `geom_bar` function in `ggplot2`:

```{r flightsbar, fig.cap="Number of flights departing NYC in 2013 by airline using geom_bar", fig.height=2.5}
ggplot(data = flights, mapping = aes(x = carrier)) +
  geom_bar()
```

To get an understanding of what the names of these airlines are corresponding to these `carrier` codes, we can look at the `airlines` data frame in the `nycflights13` package.  Note the use of the `kable` function here in the `knitr` package, which produces a nicely-formatted table of the values in the `airlines` data frame. 

```{r}
data(airlines)
kable(airlines)
```

Going back to our barplot, we see that United Air Lines, JetBlue Airways, and ExpressJet Airlines had the most flights depart New York City in 2013.  To get the actual number of flights by each airline we can use the `group_by()`, `summarize()`, and `n()` functions in the `dplyr` package on the `carrier` variable in `flights`, which we will introduce formally in Chapter \@ref(wrangling).

```{r message=FALSE}
flights_table <- flights %>% 
  group_by(carrier) %>% 
  summarize(n=n())
kable(flights_table)
```

In this table, the counts of the carriers are pre-tabulated. To create a barchart using the data frame `flights_table`, we use `geom_col` and map the `y` aesthetic to the variable `n`. Compare this barplot using `geom_col` in Figure \@ref(fig:flightscol) with the earlier barplot using `geom_bar` in Figure \@ref(fig:flightsbar). They are identical.

```{r flightscol, fig.cap="Number of flights departing NYC in 2013 by airline using geom_col", fig.height=2.5}
ggplot(data = flights_table, mapping = aes(x = carrier, y = n)) +
  geom_col()
```

<!--
**Technical note**: Refer to the use of `::` in both lines of code above.  This is another way of ensuring the correct function is called.  A `count` exists in a couple different packages and sometimes you'll receive strange errors when a different instance of a function is used.  This is a great way of telling R that "I want this one!".  You specify the name of the package directly before the `::` and then the name of the function immediately after `::`.
-->

***

```{block lc-barplot, type='learncheck', purl=FALSE}
**_Learning check_**
```

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** Why are histograms inappropriate for visualizing categorical variables?

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** What is the difference between histograms and barplots?

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** How many Envoy Air flights departed NYC in 2013?

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** What was the seventh highest airline in terms of departed flights from NYC in 2013? How could we better present the table to get this answer quickly.


```{asis lc-barplot-solutions, include=show_solutions('4-8')}
**Learning Check Solutions**
* **`r paste0("(LC", chap, ".", (lc - 3), ")")`: Why are histograms inappropriate for visualizing categorical variables?** Histograms are for continuous variables i.e. the horizontal part of each histogram bar represents an interval, whereas for a categorical variable each bar represents only one level of the categorical variable.
* **`r paste0("(LC", chap, ".", (lc - 2), ")")`: What is the difference between histograms and barplots?** See above.
* **`r paste0("(LC", chap, ".", (lc - 1), ")")`: How many Envoy Air flights departed NYC in 2013?** Envoy Air is carrier code `MQ` and thus 26397 flights departed NYC in 2013.
* **`r paste0("(LC", chap, ".", (lc), ")")`: What was the seventh highest airline in terms of departed flights from NYC in 2013? How could we better present the table to get this answer quickly.** What a pain! We'll see in Chapter 5 on Data Wrangling that applying `arrange(desc(n))` will sort this table in descending order of `n`!
```
***

### Must avoid pie charts!

Unfortunately, one of the most common plots seen today for categorical data is the pie chart.  While they may see harmless enough, they actually present a problem in that humans are unable to judge angles well.  As Naomi Robbins describes in her book "Creating More Effective Graphs" [@robbins2013], we overestimate angles greater than 90 degrees and we underestimate angles less than 90 degrees.  In other words, it is difficult for us to determine relative size of one piece of the pie compared to another.

Let's examine our previous barplot example on the number of flights departing NYC by airline.  This time we will use a pie chart.  As you review this chart, try to identify 

- how much larger the portion of the pie is for ExpressJet Airlines (`EV`)  compared to US Airways (`US`), 
- what the third largest carrier is in terms of departing flights, and
- how many carriers have fewer flights than United Airlines (`UA`)?


```{r carrierpie, echo=FALSE, fig.cap="The dreaded pie chart", fig.height=5}
ggplot(flights, aes(x = factor(1), fill = carrier)) +
  geom_bar(width = 1) +
  coord_polar(theta = "y") +
  theme(axis.title.x = element_blank(), 
    axis.title.y = element_blank(),
    axis.ticks = element_blank(),
    axis.text.y = element_blank(),
    axis.text.x = element_blank(),
    panel.grid.major = element_blank(),
    panel.grid.minor = element_blank()) +
  guides(fill = guide_legend(keywidth = 0.8, keyheight = 0.8))
```

While it is quite easy to look back at the barplot to get the answer to these questions, it's quite difficult to get the answers correct when looking at the pie graph.  Barplots can always present the information in a way that is easier for the eye to determine relative position.  There may be one exception from Nathan Yau at [FlowingData.com][fd] but we will leave this for the reader to decide:

[fd]: https://flowingdata.com/2008/09/19/pie-i-have-eaten-and-pie-i-have-not-eaten/  "Pie I Have Eaten and Pie I Have Not Eaten"

```{r echo=FALSE, fig.align='center', fig.cap="The only good pie chart", out.height=if(knitr:::is_latex_output()) '2.5in', purl=FALSE}
knitr::include_graphics("images/Pie-I-have-Eaten.jpg")
```

***

```{block lc-pie-charts, type='learncheck', purl=FALSE}
**_Learning check_**
```

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** Why should pie charts be avoided and replaced by barplots?

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** What is your opinion as to why pie charts continue to be used?

```{asis lc-pie-charts-solutions, include=show_solutions('4-8')}
**Learning Check Solutions**
* **`r paste0("(LC", chap, ".", (lc - 1), ")")`: Why should pie charts be avoided and replaced by barplots?** In my **opinion**, comparisons using horizontal lines are easier than comparing angles and areas of circles.
* **`r paste0("(LC", chap, ".", (lc), ")")`: What is your opinion as to why pie charts continue to be used?** Legacy?
```

***

### Using barplots to compare two variables

Barplots are the go-to way to visualize the frequency of different categories of a categorical variable. They make it easy to order the counts and to compare the frequencies of one group to another. Another use of barplots (unfortunately, sometimes inappropriately and confusingly) is to compare two categorical variables together.  Let's examine the distribution of outgoing flights from NYC by `carrier` and `airport`.

We begin by getting the names of the airports in NYC that were included in the `flights` dataset. Here, we preview the `inner_join` function from Chapter \@ref(wrangling). This function will join the data frame `flights` with the data frame `airports` by match rows that have the same airport code. However, in `flights` the airport code is included in the `origin` variable whereas in `airports` the airport code is included in the `faa` variable. We will revisit such examples in Chapter \@ref(joins) on joining datasets.

```{r message=FALSE}
flights_namedports <- flights %>% 
  inner_join(airports, by = c("origin" = "faa"))
```

After running `View(flights_namedports)`, we see that `name` now corresponds to the name of the airport as referenced by the `origin` variable.  We will now plot `carrier` as the horizontal variable.  When we specify `geom_bar`, it will specify `count` as being the vertical variable.  A new addition here is `fill = name`.  Look over what was produced from the plot to get an idea of what this argument gives.

```{r, fig.cap="Stacked barplot comparing the number of flights by carrier and airport", fig.height=3.5}
ggplot(data = flights_namedports, mapping = aes(x = carrier, fill = name)) +
  geom_bar()
```

Note that `fill` is an `aes`thetic just like `x` is an `aes`thetic, and thus must be included within the parentheses of the `aes()` mapping. The following code, where the `fill` `aes`thetic is specified on the outside will yield an error. This is a fairly common error that new ggplot users make:

```
ggplot(data = flights_namedports, mapping = aes(x = carrier), fill = name) +
  geom_bar()
```

This plot is what is known as a *stacked barplot*.  While simple to make, it often leads to many problems. For example in this plot, it is difficult to compare the heights of the different colors (corresponding to the number of flights from each airport) between the bars (corresponding to the different carriers).

***

```{block lc-barplot-two-var, type='learncheck', purl=FALSE}
**_Learning check_**
```

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** What kinds of questions are not easily answered by looking at the above figure?

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** What can you say, if anything, about the relationship between airline and airport in NYC in 2013 in regards to the number of departing flights?

```{asis lc-barplot-two-var-solutions, include=show_solutions('4-8')}
**Learning Check Solutions**
* **`r paste0("(LC", chap, ".", (lc - 1), ")")` What kinds of questions are not easily answered by looking at the above figure?** Because the red, green, and blue bars don't all start at 0 (only red does), it makes comparing counts hard.
* **`r paste0("(LC", chap, ".", (lc), ")")` What can you say, if anything, about the relationship between airline and airport in NYC in 2013 in regards to the number of departing flights?** The different airlines prefer different airports. For example, United is mostly a Newark carrier and JetBlue is a JFK carrier. If airlines didn't prefer airports, each color would be roughly one third of each bar.}
```
***

Another variation on the stacked barplot is the *side-by-side barplot*.

```{r, fig.cap="Side-by-side barplot comparing the number of flights by carrier and airport", fig.height=5}
ggplot(data = flights_namedports, mapping = aes(x = carrier, fill = name)) +
  geom_bar(position = "dodge")
```

***

```{block lc-barplot-stacked, type='learncheck', purl=FALSE}
**_Learning check_**
```

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** Why might the side-by-side barplot be preferable to a stacked barplot in this case?

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** What are the disadvantages of using a side-by-side barplot, in general?
```{asis lc-barplot-stacked-solutions, include=show_solutions('4-9')}
**Learning Check Solutions**
* **`r paste0("(LC", chap, ".", (lc - 1), ")")` Why might the side-by-side barplot be preferable to a stacked barplot in this case?** We can easily compare the different aiports for a given carrier using a single comparison line i.e. things are lined up
* **`r paste0("(LC", chap, ".", (lc - 0), ")")` What are the disadvantages of using a side-by-side barplot, in general?** It is hard to get totals for each airline.

```
***

Lastly, an often preferred type of barplot is the *faceted barplot*.  We already saw this concept of faceting and small multiples in Chapter \@ref(facets).  This gives us a nicer way to compare the distributions across both `carrier` and airport/`name`.

```{r, fig.cap="Faceted barplot comparing the number of flights by carrier and airport", fig.height=7.5}
ggplot(data = flights_namedports, mapping = aes(x = carrier, fill = name)) +
  geom_bar() +
  facet_grid(name ~ .)
```

Note how the `facet_grid` function arguments are written here.  We are wanting the names of the airports vertically and the `carrier` listed horizontally.  As you may have guessed, this argument and other _formulas_ of this sort in R are in `y ~ x` order.  We will see more examples of this in Chapter \@ref(regress). <!--Nina says look at this.-->



***

```{block lc-barplot-facet, type='learncheck', purl=FALSE}
**_Learning check_**
```

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** Why is the faceted barplot preferred to the side-by-side and stacked barplots in this case?

**`r paste0("(LC", chap, ".", (lc <- lc + 1), ")")`** What information about the different carriers at different airports is more easily seen in the faceted barplot?
```{asis lc-barplot-facet-solutions, include=show_solutions('4-8')}
**Learning Check Solutions**  
* **`r paste0("(LC", chap, ".", (lc - 1), ")")` Why is the faceted barplot preferred to the side-by-side and stacked barplots in this case?** Not that different than using side-by-side; depends on how you want to organize your presentation.
* **`r paste0("(LC", chap, ".", (lc), ")")`What information about the different carriers at different airports is more easily seen in the faceted barplot?** Now we can also compare the different carriers **within** a particular airport easily too. For example, we can read off who the top carrier for each airport is easily using a single horizontal line.
```
***

### Summary

Barplots are the preferred way of displaying categorical variables.  They are easy-to-understand and make it easy compare across groups of a categorical variable.  When dealing with more than one categorical variable, faceted barplots are frequently preferred over side-by-side or stacked barplots.  Stacked barplots are sometimes nice to look at, but it is quite difficult to compare across the levels since the sizes of the bars are all of different sizes.  Side-by-side barplots can provide an improvement on this, but the issue about comparing across groups still must be dealt with.





<!--Subsection on Conclusion -->

## Conclusion

### Review Questions

Review questions have been designed using the `fivethirtyeight` R package [@R-fivethirtyeight] with links to the corresponding FiveThirtyEight.com articles in our free DataCamp course **Effective Data Storytelling using the `tidyverse`**.  The material in this chapter is covered in the chapters of the DataCamp course available below:

- [Scatter-plots & Line-graphs](https://campus.datacamp.com/courses/effective-data-storytelling-using-the-tidyverse/scatter-plots-line-graphs)
- [Histograms & Boxplots](https://campus.datacamp.com/courses/effective-data-storytelling-using-the-tidyverse/histograms-boxplots)
- [Barplots](https://campus.datacamp.com/courses/effective-data-storytelling-using-the-tidyverse/barplots)
- [ggplot2 Review](https://campus.datacamp.com/courses/effective-data-storytelling-using-the-tidyverse/ggplot2-review)



### What's to come?

In Chapter \@ref(wrangling), we'll further explore data by grouping our data, creating summaries based on those groupings, filtering our data to match conditions, and other manipulations with our data including defining new columns/variables.  These data manipulation procedures will go hand-in-hand with the data visualizations you've produced here. 


### Resources

An excellent resource as you begin to create plots using the `ggplot2` package is a cheatsheet that RStudio has put together entitled "Data Visualization with ggplot2" available 

* by clicking [here](https://www.rstudio.com/wp-content/uploads/2016/11/ggplot2-cheatsheet-2.1.pdf) or
* by clicking the RStudio Menu Bar -> Help -> Cheatsheets -> "Data Visualization with `ggplot2`"

This covers more than what we've discussed in this chapter but provides nice visual descriptions of what each function produces.

In addition, we've created a mind map to help you remember which types of plots are most appropriate in a given situation by identifying the types of variables involved in the problem.  It is available [here](https://coggle.it/diagram/V_G2gzukTDoQ-aZt-) and below.

```{r viz-map, echo=FALSE, fig.cap="Mind map for Data Visualization", out.width="200%"}
#library(knitr)
#if(knitr:::is_html_output()){
#  include_url("https://coggle.it/diagram/V_G2gzukTDoQ-aZt-", 
#              height = "1000px")
#} else {
  include_graphics("images/coggleviz.png", dpi = 300)
#}
```

### Script of R code

```{r include=FALSE, eval=FALSE, purl=FALSE}
knitr::purl("03-viz.Rmd", "docs/scripts/03-viz.R")
```

An R script file of all R code used in this chapter is available [here](http://ismayc.github.io/moderndiver-book/scripts/03-viz.R).