Confusion matrix
When the response variable has just two outcomes, like the case of churn, the measures of success for the model are "how many cases where the customer churned did the model correctly predict?" and "how many cases where the customer didn't churn did the model correctly predict?". These can be found by generating a confusion matrix and calculating summary metrics on it.
Recall the following definitions:
Accuracy is the proportion of predictions that are correct. $$ \text{accuracy} = \frac{TN + TP}{TN + FN + FP + TP} $$
Sensitivity is the proportion of true observations that are correctly predicted by the model as being true. $$ \text{sensitivity} = \frac{TP}{TP + FN} $$
Specificity is the proportion of false observations that are correctly predicted by the model as being false. $$ \text{specificity} = \frac{TN}{TN + FP} $$
churn and mdl_churn_vs_both_inter are available.
This exercise is part of the course
Intermediate Regression with statsmodels in Python
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Create conf_matrix
conf_matrix = ____
# Print it
print(conf_matrix)