1. Wrapping up linear regression
Now you have the basics of linear regression from a machine learning perspective. Before we move to other algorithms, let’s discuss advantages and limitations of linear regression.
2. Pros and Cons of Linear Regression
Linear regression models are easy to fit and to apply. They are concise, so they don’t need much storage. They are smooth and less prone to overfit than other types of models. This means that their prediction performance on new data is usually quite similar to their performance on the training data.
3. Pros and Cons of Linear Regression
They are also somewhat interpretable. As an example, here we show a model of systolic blood pressure as a function of age and weight. The coefficients of the model are both positive, telling us that blood pressure tends to increase as both age and weight increase.
4. Pros and Cons of Linear Regression
The primary disadvantage of linear regression is that it can’t express complex, non linear or non-additive relationships in the data. In data where the relationships are highly complex, linear regression will not predict as well as other models.
5. Collinearity
One last issue with linear regression is collinear variables.
Collinearity is when the input (or independent) variables are partially correlated. In the blood pressure example, weight tends to increase as people age, so weight and age could be partially correlated.
6. Collinearity
When variables are highly correlated, the signs of coefficients may not be what you expect: for example you might get a model where blood pressure appears to decrease with weight. This means you can’t interpret coefficients, but it does not necessarily affect the model’s prediction accuracy. So from a machine learning perspective, collinearity may not be too much of an issue.
7. Collinearity
However, unusually large coefficients or standard errors could indicate high collinearity, leading to an unstable model that gives unreliable predictions.
Upcoming chapters will present modeling approaches that are less sensitive to collinearity between variables.
8. Coming Next
In the next chapter, you will learn how to evaluate regression models visually and numerically. You will also learn the proper, safe procedure for training a regression model.
9. Let's practice!