1. GAM to learn non-linear transformations
In this lesson, you will learn about generalized additive models, to automatically learn input variable transformations.
2. Generalized Additive Models (GAMs)
With a generalized additive model (GAM), the outcome depends additively on unknown smooth functions of the input variables. A GAM learns the functions s_i and the intercept b0 that best fit the data.
3. Learning Non-linear Relationships
Remember the hassles data from a previous lesson? If we want to model the non-linear relationship between hassles and anxiety levels, but we aren't sure if the relationship is quadratic, cubic, or something else, we can try a GAM to learn the relationship.
4. gam() in the mgcv package
We will use the gam function from the mgcv package to fit GAM models. gam takes a formula, a dataset, and a family as arguments. For regular regression, use the gaussian family. For predicting probabilities, use binomial, and for counts, poisson or quasipoisson.
Because GAMs are more complex than linear models, they are more likely to overfit, and so are best used on larger datasets.
5. The s() function
To specify that an input has a non-linear relationship with the outcome, use the s function in the formula. Here, we specify that we want to model anxiety as a non-linear function of hassles.
It's not recommended to use s with a categorical variable; the s function tries to fit a spline between the input and the outcome. To do this, the input should take on about 10 or more unique values.
Let's work an example.
6. Revisit the hassles data
Let's revisit the hassles data from a previous lesson. Last time, we tried three variations
7. Revisit the hassles data
a linear,quadratic, and cubic model, and saw that cubic seemed to be the best.
8. GAM of the hassles data
Now let's try a GAM. We specify in the formula that we want a non-linear fit to the hassles variable, and use family equals gaussian. A summary of the model reports deviance explained, or pseudo-R2 of 0.64, almost as good as the cubic model.
9. Examining the Transformations
Calling plot on the model gives you plots of the variable transformations that the algorithm learned. To get the y-values of these plots, call predict on the model with the argument type = "terms".
10. Predicting with the Model
To make predictions with the model, call predict with the argument type = "response".
If we compare the GAM to our previous models on the training data, we see that the GAM learned a relationship that's close to the cubic model, except at the high ends.
11. Comparing out-of-sample performance
When we look at the GAM's out-of-sample RMSE, we see that the performance is better than the linear model, though not quite as good as the quadratic or cubic models. This may be because the dataset is small, and so the model is noisier.
The takeaway is that while knowing the appropriate transformations is best, using GAM to learn the transformations is useful when you don't have the domain knowledge to tell you the correct transform.
12. Let's practice!
Now let's practice fitting and predicting with generalized additive models.