Predictive performance
1. Predictive performance
Another way of thinking about the logistic model is with respect to classification, as it predicts the probability of an outcome which can only result into a Purchase or no Purchase. The logistic response model itself does not perform classification but it can be used to classify the observations based on their predicted probabilities.2. Classification
To turn the logistic model into a classifier typically a cutoff value is chosen. The aim is to classify observations with a predicted probability greater than the cutoff as one and observations with a predicted probability smaller than the cutoff as zero. To force our extended model to predict whether we observe a Purchase or no-Purchase we can use the function ifelse() on the fitted.values(). The function ifelse() allows to classify all predicted purchase probabilities that are larger than the cutoff into one to indicate Purchases and zero otherwise. In our example, the cutoff value is chosen to be 0-point-5. Although other cutoffs can be used, the result is stored in an object named predicted. The classification results are summarized by using the function table() and divided by the number of observations. Our classification model classified 97 percent of the cases as purchase and 3 percent as no-purchase.3. Model confusion
The performance of our classification model can be summarized by simply cross-tabulating the observed vs. the classified purchases. This can, again, be done by using the function table() on the predicted purchases stored in predicted and the observed data values, which are stored in an object named observed. The obtained matrix shows the ways in which the classification of the predicted purchase probabilities is confused. This confusion matrix does not only inform you about the prediction accuracy of your classifier, more importantly, it gives you insights into the types of errors that are being made. In our example, the classifier predicts more of the purchase events falsely as no-Purchase than correctly as purchase. Therefore we need to evaluate somehow if our classifier predicts better than randomly guessing if there was a purchase or not.4. ROC curves
The Receiver Operating Characteristics plot, or short ROC curve, is a popular method for evaluating a classifier’s performance. Typically, the rate of correctly classified events, also known as sensitivity, is plotted against the rate of falsely classified events, also known as false alarms, at various cutoff values. We can create such an ROC curve by using the add-on package, pROC. The function roc() takes as inputs the fitted purchase probabilities of the extended model and the true observed purchases. Once we have created the Roc object, we can plot the ROC curve using the plot() function. The coordinates of the points of the resulting ROC curve are the rate of correctly classified Purchases and rate of falsely classified Purchases for each cutoff value. The 45-degree line is an ROC curve of random classification and serves as a baseline. The more "up and to the left" the ROC curve is of this line, the better the model. The ROC curve for an ideal classifier would go up to 100% quickly on the left side and then stay level at 100% across the top. In our example, the curve shows a moderate classification performance.5. Let's practice!
Great! Let’s classify some purchases.Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.