1. Preparing the predictor insight graph table
Now that you know how to discretize the variables, you are ready to construct the table that is needed to plot the predictor insight graphs.
2. The predictor insight graph table
The predictor insight graph table is a table that has all the necessary information to create the predictor insight graph. It has one row for each group in the variable that you want to plot, and three columns.
The first column contains the names of the groups. In case the original variable is continuous, these are the names of intervals it was discretized in. The second column shows the average target incidence of the group: what is the mean target in this group. The final third columns shows the size of each group, that is the number of observations that belongs to the particular group.
3. Calculating the predictor insight graph table
You can easily construct a function that creates the predictor insight graph for a given basetable, target and variable.
In this function, you first group the basetable by the variable you want to make the predictor insight graph for. In these groups, you only need the variable and target values, that is why you only select these in this step.
Next, you can use the aggregate function on these groups to create two columns. The first column is the target incidence, which is the mean of the target. The second column is the size, that is, the number of observations in each group.
With this function, you can easily calculate the predictor insight graph for any variable. For instance, if you want to calculate the predictor insight graph table for the variable country, you can simply call this method with the given basetable, target and country as arguments.
4. Calculating multiple predictor insight graph tables
In some cases, you want to calculate the predictor insight graphs for many variables. Instead of calculating them one by one, you could do this automatically and store the predictor insight graph tables in a dictionary.
Assume that there are 4 variables for which you want to calculate the predictor insight graph table, namely country, gender, disc_mean_gift and age. You first create an empty dictionary that will keep track of the predictor insight graph tables. Then, you loop over the variables one by one, and each time calculate the predictor insight graph tables. These tables are stored in a dictionary, with as key the names of the variables.
If you later want to plot a particular variable, you can easily look up the variable's predictor insight graph in the dictionary.
5. Let's practice!
Now it's your turn.