1. Summary of gender discrimination
The observed gender discrimination data is not really consistent with the permuted null differences.
2. Significance
Only 30 of the 1000 permuted differences were larger than or equal to the observed statistic. That is, we would have observed data like ours only 3% of the time if men and women were equally likely to be promoted.
Pay special attention to how the p-value is computed here. First we identify permuted differences that are larger than or equal to the observed statistic and label those situations with a 1, all other permutations receiving a zero. By averaging the 0s and 1s, the mean gives the proportion of times the permuted difference is larger than or equal to the observed difference.
Because (point) 03 is less than (point) 05, we reject the null hypothesis and claim that men are promoted at a higher rate than women. That is, we conclude that it was not simply random variability which led to a higher proportion of men being promoted. A p-value of (point) 03 is reasonably close to (point) 05 which means we should be somewhat careful in making strong claims. We should take the results as a indication that more work should be done on the claims. Indeed, in 40 years since this research was published, many social scientists have been able to replicate research on gender discrimination in the workforce.
3. Causation
Because the study was randomized, that is, they randomly assigned the resumes to the managers, there is nothing systematically different about the two groups except the name on the resume. As an example, it wouldn't make sense to have given the female resumes to the first 24 managers who arrived at the training. Those early arriving individuals might be less inclined to promote anyone given their strict adherence to being on time.
The only difference in the two groups, both the participants as well as the resumes, was the name on the top of the resume.
Therefore, any difference in promotion rates is due to the gender of the applicant. That is, we can infer a causal connection between the gender of the applicant being male and a higher promotion rate.
4. Random sample
The 35 individuals in the sample were not randomly sampled from all possible American bank managers; they were at a management training session. In order to generalize the results of the study to a larger population, we would need more information about the study and careful thinking about who the study participants might represent.
5. Let's practice!
OK, great, let's move on to the next example.