1. Who is Bayes? What is Bayes?
Welcome! My name is Micha? Oleszak and I will be your instructor for this course in Bayesian data analysis. But what exactly does "Bayesian" mean?
2. Who is Bayes?
Thomas Bayes was an eighteenth-century English statistician, whose name was given to an entire branch of statistics. What was his revolutionary idea?
3. Should you take your umbrella?
Imagine you are going out and are not sure whether to take an umbrella. On TV, you see a forecast predicting a 50% chance of rain. You start to consider taking the umbrella. Then, you see this, and are now sure the umbrella will come in handy. You have just performed a Bayesian analysis!
4. What is Bayes?
Bayesian inference means updating one's belief about something as the new information becomes available. Based on the forecast and the rain outside, you have just updated your belief about today's weather. Similarly, one might update their belief about the parameters of a statistical model based on incoming data.
5. What is Bayes?
This Bayesian approach to statistical inference is quite different from the more common frequentist or classical approach in two major ways.
6. What is Bayes?
First, in the meaning of the word "probability". For frequentists, probability is the proportion of outcomes. In 6000 dice rolls, approximately 1000 rolls would come up 6. That's why the probability of rolling a 6 is 1/6.
For a Bayesian, on the other hand, probability means a degree of belief. A valid Bayesian statement of probability might be "I am 90% sure this model's parameter should be larger than 1".
7. What is Bayes?
The second difference between the two approaches is in the nature of the parameters of statistical models. For frequentists, they are simply fixed values. For Bayesians, parameters are random variables. This means they can be described by probability distributions and can take many different values with different probabilities.
These two differences are what makes the Bayesian approach so appealing!
8. It pays to go Bayes!
First, since the parameters have distributions, the Bayesian approach can naturally handle uncertainty. If a distribution of a parameter describes many different values as equally likely, for instance, the uncertainty is large.
Second, since probability is defined as a degree of belief, it's easy to include expert opinion or domain knowledge in a Bayesian model in a statistically sound way.
There are many other reasons to go Bayes as well: Bayesian inference does not rely on fixed constants such as p-values, a common critique on the classical approach. Bayesian inference is statistically correct even with little data, and finally, it often coincides with the frequentist results, but offers more flexibility to build custom models.
9. Probability distributions
Given the important role they play in Bayesian data analysis, let's quickly review probability distributions.
A distribution of a random variable specifies what values this variable can take, and with what probabilities. If the set of possible values is finite, we talk about discrete distributions. When the variable can take infinitely many values, such distributions are called continuous. Continuous distributions can be visualized on a density plot.
Consider this density plot with heights in centimeters on the horizontal axis and their respective probabilities on the vertical axis. The values around 165 where the plot peaks are very likely, but 140 or 200 are possible, too, albeit rather uncommon.
10. Distributions in Python
In Python, a convenient way to represent a distribution is as a long list of numbers typically called draws, where the probability of each value appearing in the list reflects the probability that the random variable takes this value.
Here, we have a list of 10000 draws from some distribution.
We can use the kdeplot function from seaborn to produce the density plot from the previous slide. We set "shade" to "true" to make the plot clearer.
11. Let's go Bayes!
Let's go Bayes in the exercises!