---
title: "In-class exercise CLT and NHST"
author: "<Name>"
format: html
jupyter: ir
engine: knitr
# Set to true to evaluate the code cells when rendering the notebook
eval: false
# Embed notebook view, together with a download link of the qmd notebook
notebook-view:
  - notebook: clt_nhst.qmd
    title: "CLT and NHST: qmd notebook"
---

```{r set-up}
library(tidyverse)
```

## Part 1: CLT is more than an airport in NC -- the Central Limit Theorem

### Instructions

1. Set the random seed to 1.

(It is good practice to do so for reproducibility. The actual value does not and
should not matter. It is set to "1" because the exercise refers to specific
samples below.)

```{r set-seed}
set.seed(1)
```

2. Create a population:
  * 1M normally distributed values.
  * Set mean and standard deviation such that  about 96% of all values are between 80 and 120.

  **Test whether the created population meets the stated criteria (see Q1).
  It is sufficient to ballpark the output of a simple calculation, but the output
  should appear in your solution.**

```{r pop-create}
pop <-
```

3. From this population, draw 10 samples -- each with 10 observations
   (values from the population -- sampled without replacement).

4. Create a tidy tibble (long format) with two columns: (1) `Sample` and (2) `Value`.
  Make `Sample` a factor (that represents the sample id).

```{r samples-create}
# Number of samples
S <- 10
# Sample size for each sample
N <- 10
# Construct tidy tibble for all samples
samples <- 
```

5. Plot the population (black line) and all samples (color-coded) using a density
  plot.

```{r pop-samples-plot}
ggplot() +
    geom_density(data=samples, aes(x=Value, color=Sample)) +
    geom_density(data=pop, aes(x=Value), color="black") +
    theme_bw() + theme(legend.position="none")
```

6. Print the mean and standard deviation of the population and the means and
   standard deviations of each of the samples.
   Compare the values between the population and the samples (plot and printed
   values) -- note your observations (see Q3).

```{r pop-sample-stats}
# Mean and sd of population
mean()
sd()
# Mean and sd of each sample
```

7. Create a tibble with two columns: (1) `Sample` and (2) `Mean` that contains
   the mean of each sample (one row per sample). What is the name of this new
   distribution of sample means (see Q2)?

```{r samples-means}
samples.means <- 
```

8. Print the mean and standard deviation of the population and the mean and
   standard deviation of the distribution of sample means. Note your
   observations (see Q3).

```{r samples-means-stats}
samples.means <- 
```

9. Plot the population (black line) and all sample means (red line) using a
    density plot.

```{r pop-sample-means-plot}
ggplot() +
    geom_density(data=samples.means, aes(x=Mean), color="red") +
    geom_density(data=pop, aes(x=Value), color="black") +
    theme_bw() + theme(legend.position="none")
```

10. Increase `S` to 100 and `N` from 10 to 30, and then to 100. For each value
    of `N`, repeat steps 9 and 10 and note your observations (see Q4).
    (Keep `S=100` and `N=100` going forward.)

11. Print the mean and the standard error (sd/sqrt(N)) of the first sample and
    compare the outputs with those of step 9. Note your observations (see Q3).


### Questions
* Q1 Briefly explain your test in step 2.

* Q2 In your own words, describe the terms population, sample, and sampling distribution.

* Q3 What did you observe when comparing the mean(s) and standard deviation(s)
   between the three distributions (population, first sample, sampling
   distribution)? Briefly explain the relationship between these three
   distributions (w.r.t. mean, standard deviation, and standard error).

* Q4 What did you observe when increasing N, what are the implications?

* Q5 (Optional) Repeat the exercise by changing the distribution of the
     population.

* Q6 (Optional) Create an animated visualization that shows how the sample size
     `N` affects the sample distributions and the sampling distribution.

## Part 2: General properties of the U test

*Note: This part is independent of part 1.* In particular, *independent samples*
in questions Q7 and Q8 refer to samples that you can make up (encoded manually
or simulated using a common distribution) such that these samples satisfy the
stated criteria.

#### Code for questions Q7 and Q8

Supporting code for Q7
```{r u-test-q5}
# Create two samples A and B
```

Supporting code for Q8
```{r u-test-q6}
# Create two samples A and B
```

### Questions
* Q7 Consider the non-parametric U test: Create two independent samples A and B
  such that (1) each sample has five observations and (2) the p value is
  truly minimal when comparing A and B. State the null hypothesis for this U
  test and visualize the two samples with a point plot.

* Q8 Consider the non-parametric U test: Create two independent samples A and B
  such that the p value is significant (p<0.05) but the medians are the same.
  Describe your approach (with a justification) to creating the two samples and
  visualize the two samples. (Depending on the samples, a point plot, histogram,
  or density plot may be an appropriate choice).

* Q9 Under what assumption(s) can the U test of independent samples be
  interpreted as a significance test for the difference in medians?

## Part 3: Testing your understanding

### Questions

* Q10 Which of the following seven statements are wrong? Briefly explain why.
  - The CLT only applies to distributions with finite variance.
  
  - The CLT is only applicable if the sample size is at least 30.

  - For a large enough sample size, the sampling distribution of the sample means tends to be
    normally distributed.

  - For a large enough sample size, the sampling distribution of any sample statistic
    tends to be normally distributed.
  
  - For a large enough sample size, the distribution of a sample from a
    population with finite variance tends to be normally distributed.

  - The mean of the sample means converges to the mean of the population.

  - The standard deviation of the sample means is approximately equal to
    standard deviation of the population.



