Precision, ROI, and AUC

The return on investment (ROI) can be decomposed into the precision multiplied by a ratio of return to cost. As discussed, it is possible for the precision of a model to be low, even while AUC of the ROC curve is high. If the precision is low, then the ROI will also be low. In this exercise, you will use a MLP to compute a sample ROI assuming a fixed r, the return on a click per number of impressions, and cost, the cost per number of impressions, along with precision and AUC of ROC curve values to check how the three values vary.

X_train, y_train, X_test, y_test are available in your workspace, along with clf as a MLP classifier, probability scores stored in y_score and predicted targets in y_pred. pandas as pd and sklearn are also available in your workspace.

Calculate the precision prec of the MLP classifier.
Calculate the total ROI based on the precision prec.

Exercise

Precision, ROI, and AUC

Instructions