1. Updating from evidence
Throughout this course we've been talking about biased coins: coins that have a chance of heads that isn't 50%. And that knowledge is going to come in handy, because I'm worried I could have a biased coin here.
My friend Nick and I made a bet on this coin- heads I pay him, tails he pays me. It's Nick's coin, though, and I'm not sure I trust him, because I think he might have given me a trick coin that so that it comes up heads 75% of the time. How can we tell whether this coin is fair, or biased?
2. 20 flips of a coin
Well, we could run an experiment. Suppose after flipping twenty times, we see 14 heads, and 6 tails. Now that we've seen that, do we believe that the coin is fair or biased, and could we give a probability?
We're describing the process of updating our beliefs after seeing evidence, which is at the heart of Bayesian statistics. This chapter will talk about Bayesian statistics in terms of determining whether a coin is fair or biased based on evidence.
3. Two piles of 50,000 coins
Let's imagine that before we ran the experiment, we think there is a fifty percent chance the coin is fair, and fifty percent chance it's biased. Picture that as two piles of coins: one of 50,000 fair coins, one of 50,000 biased coins.
Imagine that you took every coin from each of the two piles, flipped it 20 times, and recorded the results. You would get two histograms like this: one for the fair coins, one for the biased coins.
Here's the trick of Bayesian statistics: when we see 14 heads out of 20, and we know that it's either a fair or biased coin, we know that we're in one of those red bars in the histogram. All we need to know is which.
Let's run this as a simulation. First go to every coin in the 50,000 fair coins, flip it twenty times, and record the number of heads. Then we find out how many were 14s. There were nearly two thousand coins that resulted in 14 heads. So it is possible to get that from a fair coin.
We do the same to see how many biased coins resulted in 14 heads. Notice that this time we changed the probability to .75.
The original piles were of equal size, but a lot more of the biased coins ended up resulting in 14 heads. Notice that the red bar is a lot taller in the "biased" histogram than the fair histogram. And we can add them up to see that between the two piles, there were a total of 10 thousand two hundred and sixty coins that resulted in 14 heads- that is, the total of the two red bars.
Now we can get the conditional probability: the probability the coin is biased given the condition that we got 14 heads. Note the vertical line in the probability means "given" in this context. We take the number of biased coins that resulted in 14, and divide it by the total number of coins that resulted in 14. This gives us an 82% chance the coin is biased.
We originally thought there was a 50% probability the coin was biased (that is, the piles were equal size), but conditional on seeing 14 heads, the probability has been updated to 82%. If we'd seen, say, 9 heads out of 20, which is more likely to come from a fair coin than a biased coin, the probability would have been updated in the opposite direction.
In the exercises, you'll run more simulations like this to find conditional probabilities based on other outcomes.
4. Let's practice!