Hypothesis Test vs Confidence Interval

We already mentioned before that there is a close connection between hypothesis testing and finding a confidence interval. In fact, in principle any test can be turned into an interval and vice versa. One direction is pretty simple:

Example Over the last five years the average score in the final exam of a course was 73 points. This semester a class with 27 students used a new textbook, and the mean score in the final was 78.1 points with a standard deviation of 7.1.

Now a \(95\%\) confidence interval for the true population mean score is

one.sample.t(y=78.1, shat=7.1, n=27)

## A 95% confidence interval for the population mean is (75.3, 80.9)

Now notice that any value in the interval (75.3, 80.9) is larger than the one we would have in the null hypothesis of the test \(H_0: \mu=73\), so any likely value of \(\mu\) is higher and in fact we would reject the null hypothesis of this test!

In general if we have an interval that does NOT include the value from the null hypothesis, we would reject that null.

There is a nice way to visualize this, with a number line:

draw.ci(ci=c(75.3, 89.9), 
        xlims=c(65, 95), xnull=73)

so the small vertical line is NOT in the interval, and so we would reject the null.

This also allows us to include some additional information. Say we determined before we we ran the experiment that only an increase of at least two points would have been sufficient to be of any practical importance:

so we see that we also passed that, but barely!

Example A certain manufacturing process causes the air in a factory to be quite bad. Using the current filters the mean pollution is 2570 ppb (parts per billion). The environmental engineer is testing a new type of filter. She has determined that because of the cost of changing them the new filters would have to lower the ppb to no more than 2300 to be useful. She tests a number of the filters and finds a 95% confidence interval.

Here are a couple of possible results:

1. 

the likely ppb using the new filter is lower than that of the old filters and also lower than the pracically useful ppb. They should change!

2. 

the likely ppb using the new filter is lower than that of the old filters but not necessarily low enough to be practically useful. They should test a few more filters to shrink the interval.

3. 

the likely ppb using the new filter is a bit lower than that of the old filters, but it is not at all clear whether it is low enough to be practically useful. It would probably take a much larger study to determine that.

4. 

the likely ppb using the new filter is not any different from the old one, certainly not low enough to be practically useful.

so by doing a confidence interval we can still do the hypothesis test, but we can also include information such as the practical significance.