Centrale limietstelling voor gemiddelden

Ongeacht de verdeling van de data stelt de centrale limietstelling (CLT) ons, naast andere voordelen, in staat om normaliteit aan te nemen voor de steekproefverdelingen van veelgebruikte A/B-teststatistieken zoals gemiddelden, sommen, proporties, standaardafwijkingen en percentielen. Toetsen voor statistische significantie die normaliteit veronderstellen, kunnen daarom eenvoudig worden toegepast in dit soort situaties om stevige conclusies over onze experimenten te trekken.

Het doel van deze oefening is te laten zien hoe de CLT geldt voor verschillende verdelingen en om de kracht ervan te waarderen.

Het volgende is alvast voor je geladen:

de checkout DataFrame
pandas als pd
numpy als np
matplotlib als plt
seaborn als sns

Deze oefening maakt deel uit van de cursus

A/B-testen in Python

Cursus bekijken

Praktische interactieve oefening

Zet theorie om in actie met een van onze interactieve oefeningen.

Begin met trainen

Deze oefening maakt deel uit van de cursus

A/B-testen in Python

SkillTag.level.intermediateSkillTag.label

4.8+

Begin de cursus gratis

In this chapter, you’ll learn the foundations of A/B testing. You’ll explore clear steps and use cases, learn the reasons and value of designing and running A/B tests, and discover the most commonly used metrics design and estimation frameworks.

Exercise 1: What is A/B testing?Exercise 2: When an A/B test is not best Exercise 3: A/B testing steps Exercise 4: Randomization effects Exercise 5: Why run experiments?Exercise 6: Correlation visualization Exercise 7: Correlation or causation?Exercise 8: Metrics design and estimation Exercise 9: Means and proportions Exercise 10: Ad impressions metrics

In Chapter 2, you’ll cover the experiment design process. Starting with learning how to formulate strong A/B testing hypotheses, you’ll also cover statistical concepts such as power, error rates, and minimum detectable effects. You’ll finish the chapter by learning to estimate the appropriate sample size needed to yield conclusive results and tackle scenarios with multiple comparisons.

Exercise 1: Hypothesevorming en verdelingen Exercise 2: Sterke hypotheseformulering Exercise 3: Distributies plotten Exercise 4: Centrale limietstelling voor gemiddelden

Huidige oefening

Exercise 5: Experimenteel ontwerp: testparameters instellen Exercise 6: P-waarden interpreteren Exercise 7: Foutpercentages in de praktijk Exercise 8: Experimenteel ontwerp: poweranalyse Exercise 9: Powercurves plotten Exercise 10: Steekproefgrootte voor gemiddelden Exercise 11: Steekproefgrootte voor proporties Exercise 12: Toetsen voor multiple comparisons Exercise 13: Is een correctie voor multiple comparisons nodig?Exercise 14: Gecorrigeerde p-waardes

Here, you’ll discover a concrete workflow for cleaning, preprocessing, and exploring AB testing data, as well as learn the necessary sanity checks we need to follow to ensure valid results. You’ll explore a detailed explanation and example of analyzing difference in proportions A/B tests.

Exercise 1: Data cleaning and exploratory analysis Exercise 2: Proportions EDA Exercise 3: A/B test data cleaning Exercise 4: Sanity checks: Internal validity Exercise 5: SRM Exercise 6: Distributions balance Exercise 7: Sanity checks: external validity Exercise 8: Novelty effects detection Exercise 9: Simpson's paradox in action Exercise 10: Analyzing difference in proportions A/B tests Exercise 11: Difference in proportions A/B test Exercise 12: Interpretation of confidence intervals Exercise 13: Confidence intervals for proportions

In the final chapter, you’ll develop frameworks for analyzing differences in means and leveraging non-parametric tests when several assumptions aren't met. You’ll also learn how to apply the Delta method when analyzing ratio metrics and discover the best practices and some advanced topics to continue the A/B testing mastery journey.

Exercise 1: Analyzing difference in means A/B tests Exercise 2: T-test for difference in means Exercise 3: Pairwise t-tests Exercise 4: Non-parametric statistical tests Exercise 5: Parametric or non-parametric?Exercise 6: Mann-Whitney U test Exercise 7: Chi-square test for independence Exercise 8: Ratio metrics and the delta method Exercise 9: Delta or not?Exercise 10: Delta method Exercise 11: A/B Testing best practices and advanced topics intro Exercise 12: Best practices Exercise 13: Day-of-the-week effect Exercise 14: Wrap-up: A/B testing in python