1. Learn
    2. /
  2. Courses
    3. /
  3. Credit Risk Modeling in Python

Exercise

Comparing with ROCs

You should use ROC charts and AUC scores to compare the two models. Sometimes, visuals can really help you and potential business users understand the differences between the various models under consideration.

With the graph in mind, you will be more equipped to make a decision. The lift is how far the curve is from the random prediction. The AUC is the area between the curve and the random prediction. The model with more lift, and a higher AUC, is the one that's better at making predictions accurately.

The trained models clf_logistic and clf_gbt have been loaded into the workspace. The predictions for the probability of default clf_logistic_preds and clf_gbt_preds have been loaded as well.

Instructions 1/2

undefined XP
    1
    2
  • Calculate the fallout, sensitivity, and thresholds for the logistic regression and gradient boosted tree.
  • Plot the ROC chart for the lr then gbt using the fallout on the x-axis and sensitivity on the y-axis for each model.