RR.Rmd

% A Shot at Reproducible Data Analysis
% Toni Verbeiren
% 13/3/2015

# Introduction

In this talk/document/presentation I showcase some of the possibilities that a combination of _tools_ provides:

* [Markdown](http://daringfireball.net/projects/markdown/)
* [RMarkdown](http://rmarkdown.rstudio.com/)
* [Knitr](http://yihui.name/knitr/)
* [Pandoc](http://johnmacfarlane.net/pandoc/)
* [Reveal.js](http://lab.hakim.se/reveal-js/#/)
* [Latex](http://www.latex-project.org/)

- - -

In order to make sure things look good from the first start, you might check out some additional projects and files:

* Bootstrap template for Pandoc: 
  <https://github.com/tonyblundell/pandoc-bootstrap-template>
* Alternative LaTeX templates: 
  <https://github.com/kjhealy/latex-custom-kjh>
* Alternative Pandoc template: 
  <https://github.com/kjhealy/pandoc-templates>
* Non-official KU Leuven templates:
  <https://github.com/exporl/kuleuven-templates>

- - -

# Idea

- - -

## Workflow

1. Write data generation, data manipulation and discussion in **one text file**.
    * Syntax for text is Markdown.
    * Code lines start with `tab` or delimited by `` ``` ``
    * Call this file `file.Rmd`, even if it includes more than `R` code.

2. Call `knitr` on the `.Rmd` file in order to **execute** the code blocks and **include** the output of the code in one file. The output is a `.md` file.

3. Call `Pandoc` on the file, given suitable options (see below). `Pandoc` is responsible for translating the `.md` file to **any format** you want. 

- - -

## RMarkdown format

The `.Rmd` source of this report looks like this (50 lines):

```{r, results="markup", comment=""}
text <- readLines("RR.Rmd",encoding="UTF-8")
tail(head(text, 70),50)
```

- - -

## Markdown format

The `.md` source of this report looks like this (50 lines):

```{r, results="markup", comment=""}
text <- readLines("RR.md",encoding="UTF-8")
tail(head(text, 70),50)
```

Conversion is done using `knitr`.

- - -

## Pandoc

A simple and a more involved example of running `Pandoc`:

```shell
pandoc file.md -o file.docx

pandoc file.md -o file.html \
      -t html5 \
			--template template.html \
			--css template.css \
			--highlight-style=tango --mathjax \
			--toc --toc-depth 2
```

Dust off your `Makefile` skills!

- - - 

# Some Examples

- - -

## Simple example

The first example is in `R`. Let's say I want to plot a function

$$ f(x) = \frac{\log(x^2+x+1)}{2x} $$

We first define $x$ and the function value $y$ (in doing so we have used some inline equations as well):

```{r}
x <- seq(from=-5,to=10,by=.01)
y <- (log(x*x + x + 1))/(2*x)
```

- - -

Then we can plot the function. We use the `ggplot2` package.

```{r chunk, fig.cap="Plot of the very special function defined above.",echo=TRUE,message=FALSE}
library(ggplot2)
qplot(x,y,geom="line")
```

See the figure for the result.

- - -

## Working with data

Let us take a look at a dataset that comes with `R`, `mtcars`:

```{r}
summary(mtcars)
```

- - -

Now the fun starts. Let's fit a model relates how many Miles/Gallon are consumed, given a weight.

```{r}
model <- lm(mpg ~ wt, data=mtcars)
summary(model)
```

- - -

This is verbatim output, we can use some `R` package magic to get proper tables as output as well using the `pander` package:

```{r, results="asis", message=FALSE}
library(pander)
pander(model)
```

- - -

We can also plot this information using the code below.

```{r, fig.cap="A scatterplot of the fuel consumption versus the weight of the car, along with the results of a linear regression. See the text for more information."}
qplot(x=wt, y=mpg, data=mtcars, xlab="Weight (lb/1000)", ylab="Miles per Gallon",
      geom=c("point","smooth"), method="lm")
```

- - -

## Scraping the web

This script parses the Wikipedia page with Belgian Beers in order to get the data out.
It then does some cleaning up and converts the data to different formats.
The result can be stored  in a file, but just display the first 10 rows.

- - -

```{r, message=FALSE}
library(XML)
rawBeers <- readHTMLTable(doc="http://nl.wikipedia.org/wiki/Lijst_van_Belgische_bieren")
beers <- NULL

# The first table is not relevant, the rest is:
for (i in seq(2,28)) {
  beers <- rbind(beers,rawBeers[[i]])
}

# Remove the percentage sign and convert to numbers:
beers$Percentagealcohol <- gsub("%","",beers$Percentagealcohol)
beers$Percentagealcohol <- gsub(",",".",beers$Percentagealcohol)
beers$Percentagealcohol <-as.numeric(beers$Percentagealcohol)

# A few entries do not have a percentage entry
nas <- length(beers[is.na(beers$Percentagealcohol),])
```

- - -

The number of entries without percentage entry is: `r nas`.

We use `pander` again for displaying the top-10 of beers with the highest amount of alcohol:

- - -

```{r, results="asis"}
pander(
  head(
    beers[order(beers$Percentagealcohol,decreasing=TRUE),
          c("Merk","Percentagealcohol")],
    10)
  )
```

- - -

# Different languages

- - -

## Python

```{r , cache=TRUE, engine="python"}
import pprint
pprint.pprint(zip(('Byte', 'KByte', 'MByte', 'GByte', 'TByte'), 
              (1 << 10*i for i in xrange(5))))
```

- - -

## Scala

```{r , cache=TRUE, engine="scala"}
val collection = for {i <- 1 to 10} yield {i}
val mapped = collection map (x => x*x)
val reduced = mapped reduce (_ + _)
println(reduced)
```

- - -

# Sweave

- - -

`knitr` can handle sweave documents as well.

```R
library(knitr)
Sweave2knitr('dummy.Rnw')
knit('dummy-knitr.Rnw')
```

- - -

Or, just write in RMarkdown:

```script
Rscript -e 'library(knitr); knit("rmarkdown-version.Rmd")
pandoc rmarkdown-version.md -o rmarkdown-version.pdf --toc
```

Text (and code) can be translated using `Pandoc`

- - -

![Side-by-side view of the same text/code in RMarkdown and Sweave](figure/SweaveVsRmd.png)


- - -

# What to use it for?

I use it for:

* Creating presentations (`reveal.js`)
* Writing reports (including code)
* Writing papers (just text)
* Making coffee

- - -

# How to use it?

- - -

## RStudio

![Screenshot of (part of) RStudio](figure/RStudio-Screenshot.png)

- - -

## _your favourite editor here_

![Screenshot of Sublime Editor with Markdown mode](figure/Sublime-Screenshot.png)

- - -

# Additional pointers

* Markdown to `Reveal.js`: 
  <http://tverbeiren.github.io/BigDataBe-Spark/#/>
* Markdown and `Pandoc` for writing a paper: 
  <http://homes.esat.kuleuven.be/~bioiuser/blog/?p=243>
* Markdown and `Pandoc` for lecture notes: 
  <https://bitbucket.org/tverbeiren/i0u19a>
* You can find everything I showed here at: 
  <http://github.io/tverbeiren/ReproducibleDataAnalysis/>