Day-of-the-week effect
Although A/B testing enables us to control for external variables such weather and holidays, some treatments may have stronger or weaker effects than others that need to be included to fully capture the user behavior around fluctuating periods. Depending on the tracked metric, you may need to consider running your A/B tests for full weekly increments. Some metrics exhibit seasonality and are affected by holidays, weekends, etc. while others don't fluctuate as much. It is up to you as the data scientist to advise the team on whether they can stop the test upon reaching the required sample size, or extend by a few days to cover seasonal effects such as day-of-the-week.
Examine the novelty dataset that is loaded for you to select the metrics that need to run for weekly increments. pandas and matplotlib are also loaded for you.
Diese Übung ist Teil des Kurses
A/B Testing in Python
Interaktive Übung
Versuche dich an dieser Übung, indem du diesen Beispielcode vervollständigst.
# Create a line plot
novelty.____('____', '____')
plt.title('Purchase Rate Over Test Duration')
plt.ylabel('Purchase Rate [%]')
plt.ylim([0, 0.04])
plt.show()