1. Learn
    2. /
  2. Courses
    3. /
  3. A/B Testing in Python

Exercise

T-test for difference in means

Imagine that you are a Data Scientist at an e-commerce company where you were tasked with running an experiment to help the payments team decide which checkout page design yields higher order value and fastest purchase decision time. Assuming you already estimated the required sample size per variant and successfully passed all the sanity checks, you will analyze the differences in average order_value and time_on_page metrics between the checkout_page variants, and make a decision on the best performing design.

checkout DataFrame is available for you and pingouin has been loaded along with pandas and numpy.

Instructions 1/3

undefined XP
  • 1
    • Calculate the mean order_value per variant, and run a t-test for difference in order_value between variants A and B.
  • 2
    • Calculate the mean time_on_page per variant, run a t-test for its difference between variants A and B, and note the p-value and confidence interval of the difference. Will you reject the Null hypothesis?
  • 3
    • Calculate the mean time_on_page per variant, run a t-test for its difference between variants A and C, and note the p-value and confidence interval of the difference. Will you reject the Null hypothesis?