01-bodwin/jsm_2020.html

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="" xml:lang="">
  <head>
    <title>Inside-Out Statistics</title>
    <meta charset="utf-8" />
    <meta name="author" content="Kelly Bodwin" />
    <script src="libs/kePrint/kePrint.js"></script>
    <link rel="stylesheet" href="xaringan-themer.css" type="text/css" />
  </head>
  <body>
    <textarea id="source">
class: center, middle, inverse, title-slide

# Inside-Out Statistics
## <i>Teaching Evidence-Based Reasoning</i>
### Kelly Bodwin
### These slides: <strong>kbod.win/talks/jsm2020</strong>

---


&lt;style type="text/css"&gt;
img {
  height: 400px;
}

.remark-code {
  font-size: 30px;
}

.remark-slide {
  background-size: cover;
}
&lt;/style&gt;
---
class: center, middle

# A quick grading exercise

---

# The Question:

In 1952, a group of scientists decided to study the effect of Vitamin C supplements
on tooth growth.  They gathered a set of 60 guinea pigs, and gave half the animals
pure vitamin C and half the animals Orange Juice.  They then measured the length
of the guinea pigs' incisor teeth.

The results of the study are summarized below:

&lt;table class="table" style="margin-left: auto; margin-right: auto;"&gt;
 &lt;thead&gt;
  &lt;tr&gt;
   &lt;th style="text-align:left;"&gt; Supplement Type &lt;/th&gt;
   &lt;th style="text-align:right;"&gt; Mean Tooth Length &lt;/th&gt;
   &lt;th style="text-align:right;"&gt; SD of Tooth Length &lt;/th&gt;
   &lt;th style="text-align:right;"&gt; Sample Size &lt;/th&gt;
  &lt;/tr&gt;
 &lt;/thead&gt;
&lt;tbody&gt;
  &lt;tr&gt;
   &lt;td style="text-align:left;"&gt; Orange Juice &lt;/td&gt;
   &lt;td style="text-align:right;"&gt; 20.66333 &lt;/td&gt;
   &lt;td style="text-align:right;"&gt; 6.605561 &lt;/td&gt;
   &lt;td style="text-align:right;"&gt; 30 &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
   &lt;td style="text-align:left;"&gt; Vitamin C &lt;/td&gt;
   &lt;td style="text-align:right;"&gt; 16.96333 &lt;/td&gt;
   &lt;td style="text-align:right;"&gt; 8.266029 &lt;/td&gt;
   &lt;td style="text-align:right;"&gt; 30 &lt;/td&gt;
  &lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;

Is there evidence at the 0.05 level that the orange juice is more beneficial to tooth
growth than Vitamin C supplements?

---
# The Correct Answer


&lt;p style="border:3px; border-style:solid; border-color:#6495ed; padding: 1em;"&gt;
With a t-score of &lt;b&gt;1.91&lt;/b&gt;, we get a p-value of &lt;b&gt;0.03&lt;/b&gt;.
&lt;br&gt;
There &lt;b&gt;is evidence&lt;/b&gt; that Orange Juice leads to more tooth
growth than Vitamin C.
&lt;/p&gt;

---
# Student Responses

&lt;p style="border:3px; border-style:solid; border-color:#6495ed; padding: 1em;"&gt;
With a t-score of &lt;b&gt;1.91&lt;/b&gt;, we get a p-value of &lt;b&gt;0.03&lt;/b&gt;.
&lt;br&gt;
There &lt;b&gt;is evidence&lt;/b&gt; that Orange Juice leads to more tooth
growth than Vitamin C.
&lt;/p&gt;


**Student A:**

&lt;p style="border:3px; border-style:solid; padding: 1em;"&gt;
With a t-score of &lt;b&gt;1.91&lt;/b&gt;, we get a p-value of &lt;b&gt;0.03&lt;/b&gt;.  
&lt;br&gt;
There &lt;font color = "red"&gt;&lt;b&gt; is not evidence&lt;/b&gt;&lt;/font&gt; that Orange 
Juice leads to more tooth
growth than Vitamin C.
&lt;/p&gt;


---
# Student Responses

&lt;p style="border:3px; border-style:solid; border-color:#6495ed; padding: 1em;"&gt;
With a t-score of &lt;b&gt;1.91&lt;/b&gt;, we get a p-value of &lt;b&gt;0.03&lt;/b&gt;.
&lt;br&gt;
There &lt;b&gt;is evidence&lt;/b&gt; that Orange Juice leads to more tooth
growth than Vitamin C.
&lt;/p&gt;

**Student B:**

&lt;p style="border:3px; border-style:solid; padding: 1em;"&gt;
With a t-score of &lt;font color = "red"&gt;&lt;b&gt;-1.91&lt;/b&gt;&lt;/font&gt;, we get a p-value of &lt;b&gt;0.03&lt;/b&gt;.  
&lt;br&gt;
There &lt;b&gt;is evidence&lt;/b&gt; that &lt;font color = "red"&gt;Vitamin C&lt;/font&gt; leads to more tooth growth than &lt;font color = "red"&gt;Orange Juice&lt;/font&gt;.
&lt;/p&gt;

---
# Student Responses

&lt;p style="border:3px; border-style:solid; border-color:#6495ed; padding: 1em;"&gt;
With a t-score of &lt;b&gt;1.91&lt;/b&gt;, we get a p-value of &lt;b&gt;0.03&lt;/b&gt;.
&lt;br&gt;
There &lt;b&gt;is evidence&lt;/b&gt; that Orange Juice leads to more tooth
growth than Vitamin C.
&lt;/p&gt;

**Student C:**


&lt;p style="border:3px; border-style:solid; padding: 1em;"&gt;
&lt;font color = "red"&gt;The standard deviation of the mean for the OJ group is 1.2.
The mean of the OJ group is about 4mm above the mean of the VC group.  This
difference is a lot bigger than the standard deviation. &lt;/font&gt;
&lt;br&gt;
Thus, I think there &lt;b&gt;is evidence&lt;/b&gt; that Orange Juice leads to more tooth
growth than Vitamin C.
&lt;/p&gt;


---
class: center, middle

# How did you score these students?

---

# Who got what wrong?

**Student A** got
* the **correct** t-score
* the **correct** p-value
* the **wrong** conclusion

**Student B** got
* the **correct** (ish) t-score
* the **correct** p-value
* the **wrong** conclusion

**Student C** got
* **no** t-score
* **no** p-value
* the **correct** conclusion


---
class: center, middle

# Which student would you want to hire?

---

&lt;table class="table" style="margin-left: auto; margin-right: auto;"&gt;
 &lt;thead&gt;
  &lt;tr&gt;
   &lt;th style="text-align:left;"&gt; Supplement Type &lt;/th&gt;
   &lt;th style="text-align:right;"&gt; Mean Tooth Length &lt;/th&gt;
   &lt;th style="text-align:right;"&gt; SD of Tooth Length &lt;/th&gt;
   &lt;th style="text-align:right;"&gt; Sample Size &lt;/th&gt;
  &lt;/tr&gt;
 &lt;/thead&gt;
&lt;tbody&gt;
  &lt;tr&gt;
   &lt;td style="text-align:left;"&gt; Orange Juice &lt;/td&gt;
   &lt;td style="text-align:right;"&gt; 20.66333 &lt;/td&gt;
   &lt;td style="text-align:right;"&gt; 6.605561 &lt;/td&gt;
   &lt;td style="text-align:right;"&gt; 30 &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
   &lt;td style="text-align:left;"&gt; Vitamin C &lt;/td&gt;
   &lt;td style="text-align:right;"&gt; 16.96333 &lt;/td&gt;
   &lt;td style="text-align:right;"&gt; 8.266029 &lt;/td&gt;
   &lt;td style="text-align:right;"&gt; 30 &lt;/td&gt;
  &lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;


**Student A:**


&lt;p style="border:3px; border-style:solid; padding: 1em;"&gt;
There &lt;b&gt; is not evidence&lt;/b&gt; that Orange 
Juice leads to more tooth growth than Vitamin C.
&lt;/p&gt;

**Student B:**

&lt;p style="border:3px; border-style:solid; padding: 1em;"&gt;
There &lt;b&gt;is evidence&lt;/b&gt;  that Vitamin C leads to more tooth growth than Orange Juice.
&lt;/p&gt;

**Student C:**

&lt;p style="border:3px; border-style:solid; padding: 1em;"&gt;
I think there &lt;b&gt;is evidence&lt;/b&gt; that Orange Juice leads to more tooth
growth than Vitamin C.
&lt;/p&gt;

--

I argue that **Student C** deserves the highest grade.

---
class: center, middle, inverse

# Being able to draw 
# **reasonable conclusions**
# from data is 
# **way more important**
# than being able to 
# perform a hypothesis test.

---
class: middle
&gt; Many students leave their introductory course thinking of statistics only as a 
**disconnected collection** of methods and tools. 


&gt; - Guidelines for Assessment and Instruction in Statistics Education (GAISE), 2016

---
# The Train Model
.center[
![](metro_edited.jpg)
]


---
# The Mountain Model
.center[
![](mountain_edited.png)
]
---
class: middle

&gt; We recommend that throughout the entire introductory course, instructors illustrate
the **complete investigative cycle** with every example/exercise presented, starting with the
**motivating question** that led to the data collection and ending with the **scope of conclusions** and
directions for future work.  

&gt; (GAISE 2016)

---
# The Cycle Model

.center[
![](dietrich_cycle.png)
]

(Dietrich, 2013)

---
class: middle

&gt; It can be challenging to present material in a way that facilitates students’ development
of more than just a **surface level understanding of important concepts and ideas**. 

&gt; (GAISE 2016)

---
# An "Inside-Out" Statistics Class
.center[
![](earth_edited.jpg)
]


---
class: center, middle

# But how?

---
class: center, middle, inverse
# Technique 1:
# "Imaginary Results"


---

# Imaginary Results

**Before** you examine any data, ask students to imagine what they **might**
observe.

--

&lt;p style="border:3px; border-style:solid; border-color:#6495ed; padding: 1em;"&gt;
&lt;b&gt; ACTIVITY &lt;/b&gt; 
&lt;br&gt;
&lt;ul&gt;
&lt;li&gt; Choose a research question from the supplied list. &lt;/li&gt;
&lt;li&gt; Imagine a dataset that might address this research question.  Write down your
dataset, inventing possible numbers for the first few rows. &lt;/li&gt;
&lt;li&gt; Sketch a visualization that summarizes the data in your imaginary dataset.
What might it look like, and what would you conclude? &lt;/li&gt;
&lt;/p&gt;

---
# Example
.center[
![](activity.jpeg)
]

---
# "Imaginary Results"

* Trains students to **think** before they **assess**.

--

* Helps prevent **mismatch** between the *research question* and the *dataset*

--

* Fosters *good instincts*.

---
class: inverse, center, middle
# Technique 2: 
# "Expectations vs. Reality"

---
# Expectations vs. Reality

Ask students to write down:

--

* What did you **expect to see** if the answer to the research question is "no"?

--

* What did you **expect to see** if the answer to the research question is "yes"?

--

* What did you **actually** see?


---
# Expectations vs Reality

&lt;p style="border:3px; border-style:solid; border-color:#6495ed; padding: 1em;"&gt;
&lt;b&gt; Research Question: &lt;/b&gt; Do CSM students get more sleep than CLA students?
&lt;br&gt;&lt;br&gt;
&lt;b&gt; Expect if yes: &lt;/b&gt; The average hours of sleep among CSM students in this class
will be a bigger number than the average hours among CLA students.
&lt;br&gt;&lt;br&gt;
&lt;b&gt; Expect if no: &lt;/b&gt;  The average hours of sleep will be about the same.
&lt;br&gt;&lt;br&gt;
&lt;b&gt; Actually saw: &lt;/b&gt;  In this class 15 CLA students averaged 7.1 hours/night and 
17 CSM students averaged 7.6 hours/night.
&lt;/p&gt;

--

Good discussion Qs:

- Why "about the same", not "CLA gets *more* hours"?

- Why not "exactly the same"?

---
# Expectations vs Reality

Benefits of these activities:

--

* Primes students for the notion of a **null** and **alternate** hypothesis.

--

* Prompts discussion of **statistics** versus **parameters**.

--

* Gets students thinking about strength of **evidence**

---
class: inverse, center, middle
# Technique 3:
# "What would convince you?"
---
# What would convince you?

In-class activity:

&lt;p style="border:3px; border-style:solid; border-color:#6495ed; padding: 1em;"&gt;
Using my computer, I'm going to generate coin flips from a coin that is
unfair.
&lt;br&gt;&lt;br&gt;
Raise your hand when you think you know whether the coin is more likely to be heads
or to be tails
&lt;/p&gt;

--
H
--
T
--
T
--
T
--
H
--
T
--
H
--
T
--
T
--
H
--
T
--
H
--
H
--
T
--
T
--
T

---
# What would convince you?

Advantages:

--

* Gets students thinking about **strength of evidence** and **critical values**

--

* Primes students for **Type I** and **Type II** error

--

* Forces students to **commit** to a significance level **before** seeing the data

---
class: inverse, center, middle
# So, how did it go?!
---

# Midterm 1

(Week 4 of 10)

We **have** studied:

* types of variables

* data visualization

* *statistics* and *parameters*

* center and spread

* the 68-95-99 rule for bell curves

* the variance of the sample mean shrinks by `\(n\)`

* sample means usually follow a bell curve

---
# Midterm 1

(Week 4 of 10)

We **have not** studied:

* anything about distributions or densities

* any part of a formal hypothesis test of any kind

* anything about confidence intervals

---
# Midterm 1


&lt;p style="border:3px; border-style:solid; border-color:#6495ed; padding: 1em;"&gt;
Professor Snape is a teacher at Hogwarts School of Witchcraft and Wizardry.
In his classes, final grades out of 100 are always curved to a mean of 60. 
&lt;br&gt;&lt;br&gt;
At Hogwarts, students are assigned to one of four houses.  Headmaster Dumbledore suspects that Prof. Snape is unfairly favoring students in his favorite house, Slytherin.  
&lt;br&gt;&lt;br&gt;
He investigates, and finds that the average grade of the 70 students in Slytherin House was 65, with
a standard deviation of 8.
&lt;br&gt;&lt;br&gt;
Dumbledore has hired to you do an analysis of this situation.  What do you conclude, and why?
&lt;/p&gt;

--

**Almost every student "invented" p-values!!!**

---
# When I formally taught them hypothesis tests:
.center[
![](duh.jpg)
]

---
# When I finally taught them p-values
.center[
![](duh2.gif)
]

---
# Midterm 2


&lt;p style="border:3px; border-style:solid; border-color:#6495ed; padding: 1em;"&gt;
Choose any two of research questions from the list, and address them.
&lt;/p&gt;

--

(Plus a **data handout** and list of several research questions.)


---
class: inverse, center, middle
# What's the punchline?
---
# Takeaway messages

1. Basic data summaries and visualizations are **not** "building blocks" to a greater goal.
They are the core of every analysis.

--

2. A well-reasoned, informal analysis is **much better than** a formal statistical
analysis that lacks intuition.

--

3. Removing **all** concrete procedures from the first part of a class encourages **conceptual thought**.

--

4. Intuition and concepts **can be deliberately taught in a classroom environment**.

---
.center[
![](earth_edited.jpg)
]

---
class: inverse, center, middle

# Thanks for joining!

# Find me:

## Twitter: **@KellyBodwin**

## Slides: **kbod.win/talks/jsm_2020**
    </textarea>
<style data-target="print-only">@media screen {.remark-slide-container{display:block;}.remark-slide-scaler{box-shadow:none;}}</style>
<script src="https://remarkjs.com/downloads/remark-latest.min.js"></script>
<script>var slideshow = remark.create({
"countIncrementalSlides": false,
"highlightLines": true,
"highlightStyle": "github"
});
if (window.HTMLWidgets) slideshow.on('afterShowSlide', function (slide) {
  window.dispatchEvent(new Event('resize'));
});
(function(d) {
  var s = d.createElement("style"), r = d.querySelector(".remark-slide-scaler");
  if (!r) return;
  s.type = "text/css"; s.innerHTML = "@page {size: " + r.style.width + " " + r.style.height +"; }";
  d.head.appendChild(s);
})(document);

(function(d) {
  var el = d.getElementsByClassName("remark-slides-area");
  if (!el) return;
  var slide, slides = slideshow.getSlides(), els = el[0].children;
  for (var i = 1; i < slides.length; i++) {
    slide = slides[i];
    if (slide.properties.continued === "true" || slide.properties.count === "false") {
      els[i - 1].className += ' has-continuation';
    }
  }
  var s = d.createElement("style");
  s.type = "text/css"; s.innerHTML = "@media print { .has-continuation { display: none; } }";
  d.head.appendChild(s);
})(document);
// delete the temporary CSS (for displaying all slides initially) when the user
// starts to view slides
(function() {
  var deleted = false;
  slideshow.on('beforeShowSlide', function(slide) {
    if (deleted) return;
    var sheets = document.styleSheets, node;
    for (var i = 0; i < sheets.length; i++) {
      node = sheets[i].ownerNode;
      if (node.dataset["target"] !== "print-only") continue;
      node.parentNode.removeChild(node);
    }
    deleted = true;
  });
})();
// adds .remark-code-has-line-highlighted class to <pre> parent elements
// of code chunks containing highlighted lines with class .remark-code-line-highlighted
(function(d) {
  const hlines = d.querySelectorAll('.remark-code-line-highlighted');
  const preParents = [];
  const findPreParent = function(line, p = 0) {
    if (p > 1) return null; // traverse up no further than grandparent
    const el = line.parentElement;
    return el.tagName === "PRE" ? el : findPreParent(el, ++p);
  };

  for (let line of hlines) {
    let pre = findPreParent(line);
    if (pre && !preParents.includes(pre)) preParents.push(pre);
  }
  preParents.forEach(p => p.classList.add("remark-code-has-line-highlighted"));
})(document);</script>

<script>
(function() {
  var links = document.getElementsByTagName('a');
  for (var i = 0; i < links.length; i++) {
    if (/^(https?:)?\/\//.test(links[i].getAttribute('href'))) {
      links[i].target = '_blank';
    }
  }
})();
</script>

<script>
slideshow._releaseMath = function(el) {
  var i, text, code, codes = el.getElementsByTagName('code');
  for (i = 0; i < codes.length;) {
    code = codes[i];
    if (code.parentNode.tagName !== 'PRE' && code.childElementCount === 0) {
      text = code.textContent;
      if (/^\\\((.|\s)+\\\)$/.test(text) || /^\\\[(.|\s)+\\\]$/.test(text) ||
          /^\$\$(.|\s)+\$\$$/.test(text) ||
          /^\\begin\{([^}]+)\}(.|\s)+\\end\{[^}]+\}$/.test(text)) {
        code.outerHTML = code.innerHTML;  // remove <code></code>
        continue;
      }
    }
    i++;
  }
};
slideshow._releaseMath(document);
</script>
<!-- dynamically load mathjax for compatibility with self-contained -->
<script>
(function () {
  var script = document.createElement('script');
  script.type = 'text/javascript';
  script.src  = 'https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-MML-AM_CHTML';
  if (location.protocol !== 'file:' && /^https?:/.test(script.src))
    script.src  = script.src.replace(/^https?:/, '');
  document.getElementsByTagName('head')[0].appendChild(script);
})();
</script>
  </body>
</html>