Plotting power curves
Performing a power analysis prior to running an A/B test is one of the most important steps. It enables us to improve our chances of reaching conclusive results and better plan for resources and time allocated to each test. Moreover, plotting power curves makes it easier to visualize the impact of varying certain parameters on the required sample size to hit the desired levels of confidence. Although a larger sample size betters our chances of reaching a conclusive result, gathering additional units or spending more time is a waste of resources as the power curves saturate.
Let's see how it works in practice.
This exercise is part of the course
A/B Testing in Python
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
from statsmodels.stats.power import TTestIndPower
# Specify parameters for power analysis
sample_sizes = ____(range(____))
effect_sizes = ____([____])