Bayesian Linear Regression
1. Bayesian Linear Regression
Now that we've reviewed the characteristics of a frequentist regression, let's examine how to estimate that same regression using Bayesian methods.2. Why use Bayesian methods?
As we learned in the last lesson, if we want to make inferences about the actual values of parameters, p-values and frequentist regression fails us. Bayesian estimation is one solution to this problem. With Bayesian methods the likelihood that used for frequentist regression, and what's known as a prior, form a posterior distribution. The details of this process are beyond the scope of this course. The key point is that Bayesian methods sample from this posterior distribution, and we can then create summaries of the distributions to make parameter inferences. Using the summaries allows us to make inferences about what values parameters might take.3. The rstanarm package
To estimate Bayesian regression models in this course, we'll be using the rstanarm package. rstanarm is an interface to Stan, which is a programming language for Bayesian inference. The rstanarm package offers a high level interface with pre-written Stan scripts for common models, like linear regression.4. Using rstanarm
We can load rstanarm using the normal library command that we use to load all packages. We can then estimate a linear regression using the stan_glm function. Here, we estimate the same model that we estimated using the lm function. Normally, when using the stan_glm function, we would see a great deal of output that looks like this. This output mainly provides progress updates. However, the models we'll estimate in this course all estimate very quickly. Therefore, this output has been suppressed in the rest of the course.5. Examining an rstanarm model
Just like a regression estimated with lm, we can look at a summary of a regression estimated with stan_glm. Using the summary function, we are presented with some information about the estimated model, the parameter estimates, and some model diagnostics.6. rstanarm summary: Estimates
Notice the the parameter estimates no longer have test statistics and p-values as in the frequentist regression. This is because Bayesian estimation samples from the posterior distribution. This means that instead of a point estimate and a test statistic, we get a distribution of plausible values for the parameters, and the estimates section summarizes those distributions. Specifically we get the mean, standard deviation, and commonly used percentiles. We also see that there are parameters in the estimates section other than the (Intercept) and mom_iq coefficient that we entered into the model. Sigma represents the standard deviation of errors, mean_ppd is the mean of the posterior predictive distribution our our outcome variable, kid_iq. We'll learn more about predictive distributions in Chapter 3. Finally, log-posterior is analogous to the likelihood of a frequentist regression. This represents the log of the combined posterior distributions. This will be used for model comparisons, which we'll also explore in Chapter 3.7. rstanarm summary: Diagnostics
In the diagnostics section, the most important statistic to pay attention to is the R-hat. Unlike in frequentist regression where there is always a solution using ordinary least squares, in Bayesian models we have to check to make sure the model converged. If a model is converged, then the parameter estimates are stable. Otherwise, our results will be unreliable. In Bayesian estimation, posterior distributions are sampled in groups, known as chains. By comparing the variance within chains to the variance across chains, we can measure the stability of our estimates. This is the R-hat statistic. In general, we want all R-hat values to be less than 1.1 in order to conclude the model has converged, as in this example.8. Let's practice!
Now you try estimating a Bayesian regression model.Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.