Comparing with ROCs
You should use ROC charts and AUC scores to compare the two models. Sometimes, visuals can really help you and potential business users understand the differences between the various models under consideration.
With the graph in mind, you will be more equipped to make a decision. The lift is how far the curve is from the random prediction. The AUC is the area between the curve and the random prediction. The model with more lift, and a higher AUC, is the one that's better at making predictions accurately.
The trained models clf_logistic and clf_gbt have been loaded into the workspace. The predictions for the probability of default clf_logistic_preds and clf_gbt_preds have been loaded as well.
This exercise is part of the course
Credit Risk Modeling in Python
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# ROC chart components
fallout_lr, sensitivity_lr, thresholds_lr = ____(____, ____)
fallout_gbt, sensitivity_gbt, thresholds_gbt = ____(____, ____)
# ROC Chart with both
plt.plot(____, ____, color = 'blue', label='%s' % 'Logistic Regression')
plt.plot(____, ____, color = 'green', label='%s' % 'GBT')
plt.plot([0, 1], [0, 1], linestyle='--', label='%s' % 'Random Prediction')
plt.title("ROC Chart for LR and GBT on the Probability of Default")
plt.xlabel('Fall-out')
plt.ylabel('Sensitivity')
plt.legend()
plt.show()