1. Learn
    2. /
  2. Courses
    3. /
  3. A/B Testing in Python

Exercise

Corrected p-values

Imagine you are a Data Scientist working for a subscription company. The web design team is working on finding the perfect CTA (call-to-action) button to urge page visitors to sign up for their service. They presented you with 4 different designs besides the current version.

After running an experiment comparing each variant to the control, you generated a list of p-values loaded in the pvals variable. Comparing them directly to the significance threshold would result in an inflated Type I error rate. To avoid this, you can use the smt.multipletests() function from Python's statsmodels library to correct the p-values and test for statistical significance with a FWER = 5%.

Instructions 1/2

undefined XP
  • 1
    • Perform a Bonferroni correction to the list of p-values using alpha = 0.05.
  • 2
    • Perform a Sidak correction to the list of p-values using alpha = 0.05.