The forward stepwise variable selection procedure provides an order in which variables are optimally added to the predictor set. In order to decide where to cut off the variables, you can make the train and test AUC curves. These curves plot the train and test AUC using the first, first two, first three, … variables in the model.

In this exercise you will learn to plot these AUC curves. The method auc_train_test to calculate the AUC values has been implemented for you and can be used as follows:

auc_train, auc_test = auc_train_test(variables, target, train, test)

where variables is the set of variables used in the logistic regression model, target is a list with the target name, and train and test are the train and test basetable respectively.

The variables ordered according to the forward stepwise procedure are given in the list variables. You can explore it in the console. Additionally, three empty lists have been defined for you:

auc_values_train, which will contain the train AUC values of the model at each iteration
auc_values_test, which will contain the test AUC values of the model at each iteration
variables_evaluate, which will contain the variables evaluated at each iteration

Deciding the cut-off

Introduction to Predictive Analytics in Python

Praktische interactieve oefening