Exercise

Column selection and model performance

Creating the training set from different combinations of columns affects the model and the importance values of the columns. Does a different selection of columns also affect the F-1 scores, the combination of the precision and recall, of the model? You can answer this question by training two different models on two different sets of columns, and checking the performance.

Inaccurately predicting defaults as non-default can result in unexpected losses if the probability of default for these loans was very low. You can use the F-1 score for defaults to see how the models will accurately predict the defaults.

The credit data, cr_loan_prep and the two training column sets X and X2 have been loaded in the workspace. The models gbt and gbt2 have already been trained.

Instructions 1/2

undefined XP
    1
    2
  • Use both gbt and gbt2 to predict loan_status and store the values in gbt_preds and gbt2_preds.
  • Print the classification_report() of the first model.
  • Print the classification_report() of the second model.