1. Learn
    2. /
  2. Courses
    3. /
  3. Machine Learning with Tree-Based Models in R

Exercise

Calculate specificity

Using different measures for model performance allows you to more accurately assess it. There are several metrics for different use cases. Specificity measures the proportion of true negative outcomes correctly identified:

$$\text{specificity} = \frac{TN}{TN + FP}$$

This formula implies that with specificity approaching 100%, the number of false positives (FP) approaches 0.

In this exercise, you are going to investigate the out-of-sample specificity of your model with cross-validation.

Pre-loaded is the training data of the credit card customers dataset, customers_train, and a decision tree specification, tree_spec, which was generated using the following code:

tree_spec <- decision_tree() %>% 
                set_engine("rpart") %>%
                set_mode("classification")

Instructions

100 XP
  • Create three CV folds of customers_train and save them as folds.
  • Calculate cross-validated specificity using the fit_resamples() function that takes your specification tree_spec, a model formula, the CV folds, and an appropriate metric set. Use all predictors to predict still_customer, saving the results to specificities.
  • Aggregate the results using a single function.