In-sample ROC and AUC
How well do bagged trees capture the structure of your training set? Are they better than decision trees? Do they overfit? Using ROC and AUC is a great way of assessing this.
In this exercise, you are going to generate these in-sample predictions and calculate their ROC and AUC. Listen up, there will be surprises!
Pre-loaded is the result of the previous exercise, model_bagged
, and training data, customers_train
.
This exercise is part of the course
Machine Learning with Tree-Based Models in R
Exercise instructions
- Use
model_bagged
to generate probability predictions with your training set and add them to the training set tibble, saving the result aspredictions
. - Generate the ROC curve of the
predictions
tibble and plot the result. - Calculate the AUC of the
predictions
tibble.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Predict on training set and add to training set
predictions <- ___(___,
new_data = ___,
type = "___") %>%
bind_cols(___)
# Create and plot the ROC curve
roc_curve(___,
estimate = ___,
truth = ___) %>% autoplot()
# Calculate the AUC
___(predictions,
estimate = ___,
truth = ___)