Exercise

Evaluate test set AUC

Compute test set AUC of the GBM model for the two sets of predictions. We will notice that they are the same value. That's because AUC is a rank-based metric, so changing the actual values does not change the value of the AUC.

However, if we were to use a scale-aware metric like RMSE to evaluate performance, we would want to make sure we converted the predictions back to the original scale of the response.

Instructions

100 XP

The preds1 and preds2 prediction vectors from the previous exercise are pre-loaded into the workspace.

Compute AUC of the predictions.
Compute AUC of the predictions (scaled to response).
Notice that the AUC is the same!