Session Ready
Exercise

Compare all models based on AUC

In this final exercise, we will perform a model comparison across all types of models that we've learned about so far: Decision Trees, Bagged Trees, Random Forest and Gradient Boosting Machine (GBM). The models were all trained on the same training set, credit_train, and predictions were made for the credit_test dataset.

We have pre-loaded four sets of test set predictions, generated using the models we trained in previous chapters (one for each model type). The numbers stored in the prediction vectors are the raw predicted values themselves -- not the predicted class labels. Using the raw predicted values, we can calculate test set AUC for each model and compare the results.

Instructions
100 XP

Loaded in your workspace are four numeric vectors:

  • dt_preds
  • bag_preds
  • rf_preds
  • gbm_preds

These predictions were made on credit_test, which is also loaded into the workspace.

  • Apply the Metrics::auc() function to each of these vectors to calculate test set AUC. Recall that the higher the AUC, the better the model.
    Loading