LoslegenKostenlos loslegen

Calculate specificity

Using different measures for model performance allows you to more accurately assess it. There are several metrics for different use cases. Specificity measures the proportion of true negative outcomes correctly identified:

$$\text{specificity} = \frac{TN}{TN + FP}$$

This formula implies that with specificity approaching 100%, the number of false positives (FP) approaches 0.

In this exercise, you are going to investigate the out-of-sample specificity of your model with cross-validation.

Pre-loaded is the training data of the credit card customers dataset, customers_train, and a decision tree specification, tree_spec, which was generated using the following code:

tree_spec <- decision_tree() %>% 
                set_engine("rpart") %>%
                set_mode("classification")

Diese Übung ist Teil des Kurses

Machine Learning with Tree-Based Models in R

Kurs anzeigen

Anleitung zur Übung

  • Create three CV folds of customers_train and save them as folds.
  • Calculate cross-validated specificity using the fit_resamples() function that takes your specification tree_spec, a model formula, the CV folds, and an appropriate metric set. Use all predictors to predict still_customer, saving the results to specificities.
  • Aggregate the results using a single function.

Interaktive Übung

Versuche dich an dieser Übung, indem du diesen Beispielcode vervollständigst.

# Create CV folds of the training data
folds <- ___(customers_train, v = ___)

# Calculate CV specificity
specificities <- ___(___, 
                     ___,
                     resamples = ___,
                     metrics = ___)

# Collect the metrics
___(specificities)
Code bearbeiten und ausführen