1. Decision analysis
Welcome back! We will now learn how to make informed decisions based on the results of a Bayesian analysis.
2. Decision analysis
Most often, decision-makers don't care about the parameters of statistical models. What they do care about is maximizing profit, reducing costs, or saving lives.
Consider the efficacy of the drug you have been estimating previously. Say the posterior mean of the efficacy rate is 99%. It's the most effective drug according to the model. However, it is also expensive and with limited budget the government cannot afford enough doses.
3. Decision analysis
Now, consider another drug, with slightly lower efficacy, but much cheaper. This other drug might allow the government to save many more lives!
This is the role of decision analysis: to translate the parameters of statistical models into relevant metrics in order to inform decision-making.
4. From posteriors to decisions
Businesses and government agencies alike constantly operate under uncertainty. To make strategic decisions, it helps to know the probabilities of different scenarios playing out. Bayesian methods allow us to translate posterior distributions of the parameters that capture this uncertainty into relevant metrics very easily.
Imagine you have just sampled from the posterior distribution of a model parameter, say: a click-through rate. Your company, however, is not very interested in the rate itself, but rather in the revenue the clicks generate.
5. From posteriors to decisions
You can multiply each draw from the posterior with the number of impressions, that is the number of people to whom the ad will be displayed. This way, you get a distribution of the number of clicks instead of the click-through rate.
6. From posteriors to decisions
You can then multiply the result with the expected revenue per click, a metric the company already knows.
7. From posteriors to decisions
As a result, you obtain a posterior distribution of the revenue generated by the clicks. Notice how capturing uncertainty via probability distributions has been translated from model parameters to a business-relevant metric! Based on the outcome, the decision-makers know what revenue to expect, as well as what's the worst-case scenario and how probable it is, and they can decide accordingly.
8. Posterior revenue
Let's see this in action. Say you have simulated posterior draws of the click rates for two ads: A and B.
You are going to display the ads to 1000 users. The expected revenue from one click, however, is different for each ad, as they advertise different products. For ad A, it's $3-point-6 and for ad B it's $3. To get the distribution of the number of clicks for both ads, you can multiply the posterior with the number of impressions. Then, multiply the outcome with the revenue per click to get the distribution of the total revenue.
9. Forest plot
So far, we have been plotting multiple density plots together using seaborn. This is not very convenient and gets messy with more plots. The solution is to use a forest plot instead. It's included in pymc3, the same package you have already used for calculating credible intervals, and you will get more familiar with it in the next chapter.
The only thing to do is to create a dictionary with the posterior draws. Then, we can call the pm-dot-forestplot function on this dictionary. Each density is represented as a line. The bolder line in the middle of each density denotes the interquartile range, while the thin whiskers show the HDI credible interval of 94% by default.
10. Forest plot
One can easily adjust it by passing the hdi_prob argument with the desired HDI value: here, 99%.
11. Let's analyze decisions!
Let's analyze decisions!