Evaluate the folds
Now that you fit 10 models using all 10 of your folds and calculated the MAE and RMSE of each of these models, it's time to visualize how large the errors are. This way, you build an intuition of the out-of-sample error distribution, which is helpful in assessing your model quality.
You will plot all these errors as a histogram and display the summary statistics across all folds.
The result of the previous exercise, fits_cv
, is pre-loaded.
This exercise is part of the course
Machine Learning with Tree-Based Models in R
Exercise instructions
- Collect the out-of-sample errors of all models of
fits_cv
using a singleyardstick
function and save them asall_errors
. - Create a
ggplot2
histogram using the.estimate
as thex
aesthetic andfill
the bars by.metric
. - Use the same function as in the first instruction with
summarize = TRUE
to display summary statistics offits_cv
.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
library(ggplot2)
# Collect the errors
all_errors <- ___(___, summarize = ___)
# Plot an error histogram
ggplot(___, aes(___, ___)) +
___()
# Collect and print error statistics
___(fits_cv, ___)