Confidence Intervals

One way to find out what a statistic is telling us about the corresponding parameter is to find a confidence interval, that is a range of likely values for the parameter.

Case Study: Exam Scores

Here we will consider the variable Score, the score on two exams takes by students in a course What can we say about the average score?

attach(examscores)
head(examscores)
##     Exam Score
## 1 Exam 1   115
## 2 Exam 1    35
## 3 Exam 1    15
## 4 Exam 1    80
## 5 Exam 1    50
## 6 Exam 1    45
hplot(Score)

shows that the histogram of the scores is roughly bell-shaped, with the peak at around 50. More precisely we find

mean(Score)
## [1] 53.78

But this is based on exactly these 200 randomly selected scores, if we repeat the experiment we get different ones, and therefore a (somewhat) different mean. What can we say about the mean of all possible scores? (the population mean)

one.sample.t(Score) 
## A 95% confidence interval for the population mean is (50.1, 57.4)

So what does it mean to say a 95% confidence interval for the population mean score of (50.1, 57.4)? The idea here is that we now know that with a high likelihood the true population mean score is between 50.1 and 57.4.

Notice I used the word likelihood. It is very tempting (and many people do it in real live) to use the word probability here.

Unfortunately for technical reasons to difficult for us to discuss this would be wrong!

The correct interpretation of a confidence interval is this: suppose that over the next year statisticians (and other people using statistics) all over the world compute 100,000 95% confidence intervals, some for the mean, others maybe for medians or standard deviations or …, than about 95% or about 95,000 of those intervals will actually contain the parameter that is supposed to be estimated, the other 5,000 or so will not.

If the chance of getting a bad interval of 5% is to high, we can change that easily, say be finding a 99% confidence interval:

one.sample.t(Score, conf.level = 99) 
## A 99% confidence interval for the population mean is (49, 58.6)

So this interval has a likelihood of 99% so we will get a wrong one just 1 in 100 times. But there is a price:

58.6 - 49
## [1] 9.6
57.4 - 50.1
## [1] 7.3

so this interval is larger than the 95% one, which means there is a larger uncertainty on exactly what the value is. Finding confidence intervals always involves a trade-off:

size of interval vs confidence level

App: confint

run.app(confint)

This illustrates the idea of confidence intervals.

What to do:

As the app starts the page on the right is empty, there is no data yet. In the panel on the left you can choose the population parameters that you want.

Next move the slider to 1. Now on the Single Experiment tab you get on simulated dataset, the Summary Statistics and the confidence interval calculations. You can now run the movie and see a sequence of simulated datasets.

You can also play around and see the effects of a

  1. larger sample size n → smaller intervals

  2. change population mean \(\mu\) → changes location of interval but not its size

  3. increase population standard deviation \(\sigma\) → increases range of data, increases length of interval.

  4. increase confidence level \(\alpha\) → increases length of interval.

  1. on Many Experiment tab

no matter how n, \(\mu\) or \(\sigma\) are changed, the percentage of good intervals always matches the chosen confidence level