CommencerCommencer gratuitement

Calculate specificity

Using different measures for model performance allows you to more accurately assess it. There are several metrics for different use cases. Specificity measures the proportion of true negative outcomes correctly identified:

$$\text{specificity} = \frac{TN}{TN + FP}$$

This formula implies that with specificity approaching 100%, the number of false positives (FP) approaches 0.

In this exercise, you are going to investigate the out-of-sample specificity of your model with cross-validation.

Pre-loaded is the training data of the credit card customers dataset, customers_train, and a decision tree specification, tree_spec, which was generated using the following code:

tree_spec <- decision_tree() %>% 
                set_engine("rpart") %>%
                set_mode("classification")

Cet exercice fait partie du cours

Machine Learning with Tree-Based Models in R

Afficher le cours

Instructions

  • Create three CV folds of customers_train and save them as folds.
  • Calculate cross-validated specificity using the fit_resamples() function that takes your specification tree_spec, a model formula, the CV folds, and an appropriate metric set. Use all predictors to predict still_customer, saving the results to specificities.
  • Aggregate the results using a single function.

Exercice interactif pratique

Essayez cet exercice en complétant cet exemple de code.

# Create CV folds of the training data
folds <- ___(customers_train, v = ___)

# Calculate CV specificity
specificities <- ___(___, 
                     ___,
                     resamples = ___,
                     metrics = ___)

# Collect the metrics
___(specificities)
Modifier et exécuter le code