Interpreting hypothesis test results
1. Interpreting hypothesis test results
Now let's talk about how to interpret hypothesis test results!2. Life expectancy in Chicago vs. Bangkok
Suppose we want to test if there is a difference in life expectancy in Chicago and Bangkok. Our null hypothesis is that no difference exists, and our alternative hypothesis could be that Chicago residents have a longer life expectancy than Bangkok residents.3. Sampling distribution
We can collect data on the age of death from 100 residents each in Chicago and Bangkok. This histogram shows the life expectancy sample distributions for each city. The mean life expectancy of the Chicago sample is 79.3, and for Bangkok it's 73.9. But how do we know these are the actual mean values for each population?4. Different samples
We could collect age of death data from 100 more residents in each city. This time we get different results. So, are we sure that a difference in life expectancy really exists, or are the results due to chance? Put another way, do the samples truly represent these populations?5. Sampling distribution of mean life expectancy
We can't collect entire population data, so one approach is to perform sampling with replacement on our original data from each city and calculate the mean life expectancy for each sample. Repeating this 10000 times and visualizing the results, we can see normal distributions for mean life expectancy in Bangkok and Chicago, and Chicago has a larger expected value! So, can we now conclude that a difference in life expectancy truly exists?6. p-value
When drawing conclusions in hypothesis testing we use a metric called a p-value. This is the probability of achieving a result at least as extreme as the one we have observed, assuming the null hypothesis is true. Suppose we want to know the probability of a sample mean for Chicago life expectancy being more than or equal to 82, given a population mean of 79.3. We can visualize the sample means distribution and look at the total area from 82 onwards to determine the p-value of 0.037, meaning there is a 3.7 percent chance of observing a mean life expectancy of 82 or more.7. p-value
We can visualize the p-value for our two sample mean distributions as the total area that overlaps between them. So, how small an overlap is needed to be confident in our conclusion?8. Significance level ($\alpha$)
To reduce the risk of drawing a false conclusion, we set a probability threshold for falsely rejecting the null hypothesis. This probability threshold is known as alpha or the significance level. It is decided before collecting data to minimize bias, as a researcher may choose a different threshold after they've seen the data so that they can draw a conclusion that serves their interests. A typical value for this is 0.05, meaning there is a five percent chance of wrongly concluding that Chicago residents live longer than Bangkok residents. After data collection, we look at whether the p-value is less than or equal to alpha. If the p-value meet this criterion we can feel confident in rejecting the null hypothesis. If this occurs we describe the results as being statistically significant.9. Type I/II error
In hypothesis testing there are four potential conclusions we can make based on the null hypothesis. We can wrongly reject our null hypothesis when it was actually true. This is known as a type one error.10. Type I/II error
We can wrongly accept our null hypothesis when it's false. This is known as a type two error.11. Type I/II error
We can correctly accept the null hypothesis when it's true,12. Type I/II error
and we can correctly reject the null hypothesis when it's false.13. Drawing a conclusion
Having set alpha, we can now draw a conclusion based on our sample mean distributions. The overlap of distributions accounts for less than our threshold for alpha, 0.05, meaning the likelihood of the difference in mean life expectancy between the two cities occurring by chance is less than 5%. Therefore, we can reject the null hypothesis and reasonably conclude that the mean life expectancy in Chicago is higher than in Bangkok!14. Let's practice!
Now let's check our understanding of how to interpret the results of hypothesis tests!Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.