In this chapter, you’ll learn the foundations of A/B testing. You’ll explore clear steps and use cases, learn the reasons and value of designing and running A/B tests, and discover the most commonly used metrics design and estimation frameworks. 

What is A/B testing?

When an A/B test is not best

A/B testing steps

Randomization effects

Why run experiments?

Correlation visualization

Correlation or causation?

Metrics design and estimation

Means and proportions

Ad impressions metrics

Overview of A/B Testing

In Chapter 2, you’ll cover the experiment design process. Starting with learning how to formulate strong A/B testing hypotheses, you’ll also cover statistical concepts such as power, error rates, and minimum detectable effects. You’ll finish the chapter by learning to estimate the appropriate sample size needed to yield conclusive results and tackle scenarios with multiple comparisons.

Hypothesis formulation and distributions

Strong hypothesis formulation

Plotting distributions

Central limit theorem for means

Experimental design: setting up testing parameters

Interpreting p-values

Error rates in the wild

Experimental design: power analysis

Plotting power curves

Sample size for means

Sample size for proportions

Multiple comparisons tests

Is a multiple comparisons correction needed?

Corrected p-values

Experiment Design and Planning

Here, you’ll discover a concrete workflow for cleaning, preprocessing, and exploring AB testing data, as well as learn the necessary sanity checks we need to follow to ensure valid results. You’ll explore a detailed explanation and example of analyzing difference in proportions A/B tests.

Data cleaning and exploratory analysis

Proportions EDA

A/B  test data cleaning

Sanity checks: Internal validity

Distributions balance

Sanity checks: external validity

Novelty effects detection

Simpson's paradox in action

Analyzing difference in proportions A/B tests

Difference in proportions A/B test

Interpretation of confidence intervals

Confidence intervals for proportions

Data Processing, Sanity Checks, and Results Analysis

In the final chapter, you’ll develop frameworks for analyzing differences in means and leveraging non-parametric tests when several assumptions aren't met. You’ll also learn how to apply the Delta method when analyzing ratio metrics and discover the best practices and some advanced topics to continue the A/B testing mastery journey.

Analyzing difference in means A/B tests

T-test for difference in means

Pairwise t-tests

Non-parametric statistical tests

Options

Parametric

Non-parametric

Parametric or non-parametric?

Mann-Whitney U test

Chi-square test for independence

Ratio metrics and the delta method

Delta method

Delta not needed

Delta or not?

A/B Testing best practices and advanced topics intro

Best practices

Day-of-the-week effect

Wrap-up: A/B testing in python

Practical Considerations and Making Decisions

checkout

admissions

In this course, you will dive into A/B testing by learning to design, run, and analyze these A/B tests in Python. You’ll start by learning to define the right metrics before estimating the appropriate sample size and duration to yield conclusive results. Throughout this course, you’ll use a range of Python packages to help with A/B testing, including statsmodels, scipy, and pingouin. By the end, you will be able to run checks that guarantee accurate results, master the art of p-values, and analyze the results of A/B tests with ease and confidence to guide the most critical business decisions.

In this course, you will dive into the world of A/B testing, gain a deep understanding of the practical use cases, and learn to design, run, and analyze these A/B tests in Python.
<br><br>
<h2>Discover How A/B Tests Work</h2>
<br><br>
Did you know that you are almost guaranteed to participate in an A/B test every time you browse the internet? From search engines and e-commerce sites to social networks and marketing campaigns — all businesses hire the best data analysts, scientists, and engineers to leverage the power of AB testing. Testing different variants can help optimize the customer experience, maximize profits, inform the next best design, and much more.
<br><br>
<h2>Learn About A/B Testing in Python</h2>
<br><br>
You’ll start by learning how to define the right metrics before learning how to estimate the appropriate sample size and duration to yield conclusive results. Throughout this course, you’ll use a range of Python packages to help with A/B testing, including statsmodels, scipy, and pingouin.
<br><br>
By the end of the course, you will be able to run the necessary checks that guarantee accurate results, master the art of p-values, and analyze the results of A/B tests with ease and confidence to guide the most critical business decisions.

Hypothesis Testing in Python

Learn the practical uses of A/B testing in Python to run and analyze experiments. Master p-values, sanity checks, and analysis to guide business decisions.

A/B Testing in Python Course | Design, Run, and Analyze Your Experiments

A/B Testing in Python

Angewandte Statistik in Python

Principal Data Science Manager

A/B testing summary

Congratulations!

https://s3.amazonaws.com/assets.datacamp.com/production/course_30539/subtitles/course_30539_998867221b5d68558c7185f0b0a496b8.vtt

Practical Considerations and Making Decisions - Wrap-up: A/B testing in python

Wrap-up: A/B testing in python

Create Your Free Account