1. Bayes' theorem
Throughout this chapter, we've used simulation to estimate a conditional probability that a coin is fair or biased.
2. Probabilities
For example, we looked at a distribution of 90,000 fair coins and 10,000 biased coins, and saw how many coins from each pile resulted in 14 heads. We saw about three thousand resulting in 14 heads among the fair coins, and about fifteen hundred among the biased coins.
But what we're really working with in these simulations is probability densities. Consider this histogram in terms of the probability a coin out of the one hundred thousand we're flipping ends up with that outcome.
In this lesson, rather than simulating many coins to determine whether a coin is biased, we'll use probability density- specifically the dbinom function- to find an exact answer. In the process, we'll introduce one of the most important equations in probability: Bayes' Theorem.
3. Probability of fair coin with 14 heads
Recall that we used simulation to find the number of fair coins resulting in 14, out of the 90 thousand in the original pile. Now, rather than simulating it with rbinom, let's find the exact probability of 14 heads with the dbinom function. But notice we also multiply it with the prior probability that the coin is fair, .9. That's the equivalent of the "90,000" in the rbinom line: it calculates the probability the coin is both fair and resulted in 14 heads.
We also simulated 10,000 biased coins and found the number of times there were 14 heads out of 20 flips. We could replace that with dbinom using a .75 probability, and multiplying it by the .1 prior probability that a coin was biased. These get the exact probabilities shown in red in the histograms.
4. Conditional probability
Now that we have those probabilities, we can combine them the same way we did the coins.
To find the probability a coin is biased conditional on it resulting in 14 heads out of 20, we look at the probability it is both 14 heads and biased out of the total probability that it resulted in 14 heads. This is the equivalent of looking at "both of the red bars", and asking which red bar the coin is in.
As we did in the last slide, we can compute each of those probabilities by multiplying the probability density by the prior probability. In the numerator we're multiplying the probability a biased coin would give 14 heads, by the prior probability the coin is biased.
You're now able to calculate this in R, by using dbinom to calculate each of the densities, multiplying each by the prior probabilities, and then putting them into a fraction. You'll be able to try this in the exercises.
5. Bayes' Theorem
This exact solution is an application of Bayes' Theorem. Bayes' Theorem is usually written in terms of finding the probability of event A given event B when you knew the probability of event B given event A. In this problem we explored in this chapter, event A is that the coin is biased, and event B is that it resulted in 14 heads out of 20. This is why Bayes Theorem was useful: we knew the probability of getting 14 heads given that the coin is biased, but we needed to convert it to the probability that a coin is biased given that it resulted in 14 heads.
We could have just started the chapter with this equation, but our simulation showed what the numerator and denominator really represent, by imagining what fraction of all coins resulting in 14 heads were biased. This will help you understand and apply the theorem to calculate conditional probabilities in the future.
6. Let's practice!