1. It's easy to compare and contrast!
We’ve now seen that Bayesian Data Analysis makes it easy to compare and contrast parameter estimates. And it’s generally the case that it’s easy to calculate any derivative quantities once we’ve fitted a Bayesian model and gotten out posteriors. You’ve already seen some examples of that in the first chapter. Given
2. A probability distribution over an interesting parameter
a probability distribution over some interesting parameter you calculated a point estimate: A single value summarizing the whole distribution. You did
3. A probability distribution over an interesting parameter
the posterior median, but could also have calculated, say, the posterior mean. It’s important to remember that the posterior median or mean are not the same as the median or mean of the data. The posterior median is a summary of a probability distribution over some parameter, while the median of the data is a summary of a data set. Two different things. You calculated the amount of probability favoring a positive value.
4. A probability distribution over an interesting parameter
And while you didn’t, you could also have calculated a credible interval, an interval that contains the underlying parameter with a certain probability, for example,
5. A probability distribution over an interesting parameter
here is an 80% credible interval for this probability distribution calculated using the quantile function. So these are some of the things commonly calculated after having fitted a Bayesian model, but we can go further, and a third reason
6. Next up on reasons to use Bayesian data analysis
Bayes is useful is because you can use the result of a Bayesian analysis to do Decision Analysis. So what’s decision analysis here? Simply put, it’s when you take the result of a statistical analysis and post-process it to make it more about what you really care about, in order to make informed decisions. Often, you don’t really care about the values of parameters. What you really care about is to save the most lives or to make the most money. In the case of your zombie website you definitely care about money, the reason you want more visitors to your site is that you want them to buy
7. A patented zombie repellent
your patented zombie repellent. You’ve tried both video and text ads and now you have to decide which to use for a larger campaign you’re planning.
8. Video vs text - posteriors
Looking at the posteriors for the underlying proportion of clicks, there is pretty strong evidence that video ads would give the most visitors per ad shown, but will they result in the most money? Let’s do
9. A small decision analysis
a small decision analysis to figure that out. The important numbers to know for this is that the social media site charges 25 cents for a video ad, 5 cents for a text ad, and based on historical data we make 2-point-53 dollars per visitor on average. Let’s again work directly with
10. A small decision analysis
the posterior samples. So we can calculate anything row-wise, and the resulting distribution will be correct. Let’s start by
11. A small decision analysis
how much we earn on average per shown video ad. This is then the proportion of people that will click times how much we’ll make on average per visitor minus the cost of showing the ad. Looking at the first row we see that if the underlying proportion of clicks actually was 0-point-08, you would lose 4 cents on average per ad, but it seems for most samples, you’ll make a profit.
12. A small decision analysis
We can calculate the text ad profit in the same way, but this time subtracting the cost of a text ad instead. And finally, we can calculate
13. A small decision analysis
profit_diff, the probability distribution over the profit difference between video and text ads. This distribution is now much closer to what we really care about: which type of ad will give us the most money, where a positive number is in favor of video ads and a negative number in favor of text ads. We could now use profit_diff directly to
14. Make a data informed decision!
make a data-informed decision, so what is that decision? You’ll calculate that yourself in the following exercises!