1. Performing t-tests
In the previous chapter, we calculated the z-score, which was a test statistic for a single variable.
2. Two-sample problems
Here, we'll look at a related problem of comparing sample statistics across groups in a variable.
In the Stack Overflow dataset, converted_comp is a numerical variable of annual compensation.
age_first_code_cut is a categorical variable with two levels: child and adult, which describe when the user started programming.
We can ask questions about differences in compensation across the two age groups, such as, are users who first programmed as a child better compensated than those that started as adults?
3. Hypotheses
The null hypothesis is that the population mean for the two groups is the same, and the alternative hypothesis is that the population mean for users who started coding as children is greater than for users who started coding as adults.
We can write these hypotheses using equations.
Mu represents an unknown population mean, and we use subscripts to denote which group the population mean belongs to.
An alternate way of writing the equations is to compare the differences in population means to zero. Zero here corresponds to our hypothesized value for the difference in means.
4. Calculating groupwise summary statistics
To calculate summary statistics for each group, we start with the sample, group by the categorical variable, and then compute on the numeric variable.
A pandas way of doing this is shown, calculating the mean of the converted_comp column after grouping by age_first_code_cut.
Here, the child programmers have a mean compensation of 132,000 dollars compared to around 111,000 for adult programmers. Is that increase statistically significant or could it be explained by sampling variability?
5. Test statistics
Although we don't know the population mean, we estimate it using the sample mean.
x-bar is used to denote a sample mean.
Then we use subscripts to denote which group a sample mean corresponds to.
The difference between these two sample means is the test statistic for the hypothesis test.
The z-scores we saw in Chapter 1 are a type of standardized test statistic.
6. Standardizing the test statistic
z-scores are calculated by taking the sample statistic, subtracting the mean of this statistic as the population parameter of interest, then dividing by the standard error.
In the two sample case, the test statistic, denoted t, uses a similar equation. We take the difference between the sample statistics for the two groups, subtract the population difference between the two groups, then divide by the standard error.
7. Standard error
To calculate the standard error, needed for the denominator of the test statistic equation, bootstrapping tends to be a good option.
However, there is an easier way to approximate it. We calculate the standard deviation of the numeric variable for each group in the sample, and the number of observations in each group. Then enter those values into the equation and compute the result.
8. Assuming the null hypothesis is true
Here's the test statistic equation again. If we assume that the null hypothesis is true, there's a simplification we can make.
The null hypothesis assumes that the population means are equal, and their difference is zero, so the population term in the numerator disappears.
Inserting the approximation for the standard error, we now have a way of calculating the test statistic using only calculations on the sample dataset.
9. Calculations assuming the null hypothesis is true
We need the mean, standard deviation, and number of observations for each group to fill in the formula for t.
We again use groupby and method combinations with mean, std, and count.
10. Calculating the test statistic
Assigning the values to six different variables, the numerator is a subtraction of the sample means, and the denominator is like a weighted hypotenuse.
The t-statistic is around one-point-eight-seven. Just as with z-scores, we can't draw any conclusions yet; for that, we'll need to wait for the next video.
11. Let's practice!
In the mean time, let's calculate some test statistics.