Non-parametric ANOVA and unpaired t-tests

1. Non-parametric ANOVA and unpaired t-tests

The simulation-based proportion test you performed using the infer pipeline avoided the assumption that the test statistic is normally distributed.

2. Non-parametric tests

Hypothesis tests that don't assume a probability distribution for the test statistic are called non-parametric tests. There are two types of non-parametric test. The infer pipeline performs simulation-based tests. Other tests use the rank of the data.

3. t_test()

Here's a t-test from Chapter 2. This tested differences in mean annual compensation for Stack Overflow users depending on whether they first learned to program as a child or as an adult. Rather than calculating it step-by step, I've used t_test from infer. The degrees of freedom are different here because they are calculated using an equation called the Welch approximation, instead of taking the number of observations minus two. In practice, the difference isn't important. The p-value is point-zero-zero-eight-one-four instead of point-zero-zero-eight-one-three. Let's explore some non-parametric alternatives to the t-test, starting with the simulation-based infer pipeline.

4. Calculating the null distribution

Just as you did with the simulation-based proportion test, you start by specifying the variables of interest. It's the same formula passed to t_test: the numeric compensation on the left and the age categories on the right. Next you declare the null hypothesis, that compensation is independent of the age category. Thirdly, you generate permutation replicates of the data. Finally, you calculate the non-standardized test statistic for each replicate. The stat argument is "diff in means". The order argument is the same as in the call to t_test.

5. Calculating the observed statistic

To calculate the difference in means observed in the Stack Overflow survey, we reuse the specify and calculate steps from the previous pipeline.

6. Get the p-value

Finally, we get the p-value from the null distribution and observed statistic. The direction argument to get_p_value is the same as the alternative argument in t_test. The p-value is point-zero-six-six rather than point-zero-eight-one.

7. Ranks of vectors

Consider this numeric vector, x. The first value of x, one, is the smallest. The second value, fifteen, is the fifth smallest. These orderings from smallest to largest are known as the ranks of the elements of x. You can access them with the rank function. You can avoid assumptions about normally distributed data by performing hypothesis tests on the ranks of a numeric input. The Wilcoxon-Mann-Whitney test is, very roughly speaking, a t-test on ranked data.

8. Wilcoxon-Mann-Whitney test

You can run a Wilcoxon-Mann-Whitney test using wilcox-dot-test from base-R. It accepts a formula and data argument, though these are swapped compared to the infer functions, so they are less pipe-friendly. alternative sets the type of alternative hypothesis, and correct determines whether or not to apply a continuity correction to the z-scores. Since ranks are integers and the normal distribution is continuous, this correction can improve the approximation. However, for large sample sizes its effect is trivial and not needed. Here, the p-value is shown as less than two times ten to the minus sixteen, less than the significance level.

9. Kruskal-Wallis test

In the same way that ANOVA extends t-tests to more than two groups, the Kruskal-Wallace test extends the Wilcoxon-Mann-Whitney to more than two groups. That is, the Kruskal-Wallace test is a non-parametric version of ANOVA. You pass a formula with the numerical variable on the left and the categorical variable on the right, and set data to the sample data frame. Again, the p-value here is very small.

10. Let's practice!

Almost there!

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.