Reporting A/B test results
1. Reporting A/B test results
Creating visual representations is helpful to present and fully understand results.2. A/B visualizations
AB is a common business design and conclusions may be provided to decision-makers outside a data team. Regardless of audience, present data with clear visualizations to accurately convey findings.3. Bar plots
When the hypothesis compares group means of a normal distribution, such as the t-test, a bar plot can be used. The bar plot denotes the mean of each group and should include error bars of the standard deviation, standard error, or confidence interval.4. Bar data formatting
The mean and standard deviation must be calculated to include in the plot. When in long format, group the data by the column denoting the groups, topping, using group-underscore-by. Pipe into summarise to calculate the mean time, saved to AvgTime, and standard deviation saved to SdTime.5. Using bar plots
Create the plot by denoting the topping groups as x, mean time as y, and color groups using fill. Geom-underscore-bar creates the bars, which needs the stat as summary and fun, for function, as mean. Geom-underscore-error creates the error bars. ymin denotes the lower edge of the bar with the mean minus sd, and ymax uses mean plus sd for the upper bar. Presenting the comparison of group means with a bar plot makes the distinction, or lack thereof, easily accessible. This is helpful in businesses interested in AB group design results.6. Box plots
Box plots are particularly representative of tests comparing group medians, such as the Mann-Whitney U, chi-square, and Fisher's Exact tests, and can be used for t-tests. The dark line in the box indicates the median of each group. The bottom and top of the boxes represent the lower and upper quartile, or point where 25% of the data points fall below and 75% of data points fall below, respectively. The whiskers, or lines above and below the box, indicate the most extreme values, excluding outliers, which are indicated with a dot, as in the Cheese topping. Box plots are more informative, but bar plots are easier to understand.7. Using box plots
When raw data is in long format, use ggplot to call the x, y, and coloring with fill. Geom-underscore-boxplot creates the plot. Box plots are ideal when presenting results to an audience such as researchers or statisticians or assessing non-normal distributions.8. Scatter plots
When assessing the relationship between variables, such as correlations, scatter plots are ideal, displaying every subject data point for the variables. The trend of the data, or direction and rate of change, is visible and great for ease of understanding.9. Using scatter plots
Use geom-underscore-point to create the scatter plot. Since AB designs have two groups, specify the group points by color using fill.10. Using scatter plots
If assessing one group, subset the data. Call the group column, topping, percent-in-percent, then the group in quotes, Pepperoni.11. Presenting data
When AB test data are presented to individuals without a statistics background, p-values and test statistics are uninformative. Discuss real-world findings, presenting test conclusion and relevant test information including power analysis and error bars with our visualization.12. Presenting data example
For example, if presenting a significant t-test with a power of point-nine using this bar plot, we would say "A significant difference was found between the groups such that Pepperoni pizza was eaten quicker than Cheese pizza. The analysis indicating the difference was found to be reliable and confidence can be instilled in this conclusion given the sample size and data collected. The pizzas produced similar dispersion, indicating the toppings do not induce large variation in time".13. Let's practice!
Let's practice visual data representation.Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.