Traccia le curve ROC

Hai visto di nuovo che il boosted tree ottiene l’AUC più alta. I numeri vanno bene, ma i grafici sono ancora meglio! Perché non visualizzare questi risultati?

Illustrerai le prestazioni dei modelli tracciando tutte le curve ROC su un unico grafico. Poiché l’AUC è letteralmente l’area sotto queste curve ROC, il modello potenziato dovrebbe avere l’area più grande sotto la sua curva ROC ed essere quello nell’angolo in alto a sinistra del grafico.

Il tibble con le predizioni, preds_combined, è ancora caricato.

Questo esercizio fa parte del corso

Machine Learning con modelli ad albero in R

Visualizza il corso

Istruzioni dell'esercizio

Rimodella il tibble preds_combined in modo che tutte le colonne che iniziano con "preds_" diventino righe invece di colonne. Converti i nomi in una colonna "model" e i valori in una colonna chiamata "predictions".
Raggruppa i risultati per model.
Calcola i valori ROC per tutti i cutoff.
Produci un grafico delle curve.

Esercizio pratico interattivo

Prova a risolvere questo esercizio completando il codice di esempio.

# Reshape the predictions into long format
predictions_long <- tidyr::pivot_longer(___,
                                        cols = starts_with("___"),
                                        names_to = "___",
                                        values_to = "___")

predictions_long %>% 
  # Group by model
  ___(___) %>% 
  # Calculate values for every cutoff
  ___(truth = ___, 
      estimate = ___) %>%
  # Create a plot from the calculated data
  ___()

Modifica ed esegui il codice

Questo esercizio fa parte del corso

Machine Learning con modelli ad albero in R

InicianteNível de habilidade

4.9+

Inizia il corso gratis

Ready to build a real machine learning pipeline? Complete step-by-step exercises to learn how to create decision trees, split your data, and predict which patients are most likely to suffer from diabetes. Last but not least, you’ll build performance measures to assess your models and judge your predictions.

Exercise 1: Welcome to the course!Exercise 2: Why tree-based methods?Exercise 3: Specify that tree Exercise 4: Train that model Exercise 5: How to grow your tree Exercise 6: Train/test split Exercise 7: Avoiding class imbalances Exercise 8: From zero to hero Exercise 9: Predict and evaluate Exercise 10: Make predictions Exercise 11: Crack the matrix Exercise 12: Are you predicting correctly?

Ready for some candy? Use a chocolate rating dataset to build regression trees and assess their performance using suitable error measures. You’ll overcome statistical insecurities of single train/test splits by applying sweet techniques like cross-validation and then dive even deeper by mastering the bias-variance tradeoff.

Exercise 1: Continuous outcomes Exercise 2: Train a regression tree Exercise 3: Predict new values Exercise 4: Inspect model output Exercise 5: Performance metrics for regression trees Exercise 6: In-sample performance Exercise 7: Out-of-sample performance Exercise 8: Bigger mistakes, bigger penalty Exercise 9: Cross-validation Exercise 10: Create the folds Exercise 11: Fit the folds Exercise 12: Evaluate the folds Exercise 13: Bias-variance tradeoff Exercise 14: Call things by their names Exercise 15: Adjust model complexity Exercise 16: In-sample and out-of-sample performance

Time to get serious with tuning your hyperparameters and interpreting receiver operating characteristic (ROC) curves. In this chapter, you’ll leverage the wisdom of the crowd with ensemble models like bagging or random forests and build ensembles that forecast which credit card customers are most likely to churn.

Exercise 1: Tuning hyperparameters Exercise 2: Generate a tuning grid Exercise 3: Tune along the grid Exercise 4: Pick the winner Exercise 5: More model measures Exercise 6: Calculate specificity Exercise 7: Draw the ROC curve Exercise 8: Area under the ROC curve Exercise 9: Bagged trees Exercise 10: Create bagged trees Exercise 11: In-sample ROC and AUC Exercise 12: Check for overfitting Exercise 13: Random forest Exercise 14: Bagged trees vs. random forest Exercise 15: Variable importance

Ready for the high society of tree-based models? Apply gradient boosting to create powerful ensembles that perform better than anything that you have seen or built. Learn about their fine-tuning and how to compare different models to pick a winner for production.

Exercise 1: Introduzione al boosting Exercise 2: Bagging vs. boosting Exercise 3: Specifica un ensemble potenziato (boosted)Exercise 4: Gradient boosting Exercise 5: Allena un ensemble potenziato Exercise 6: Valuta l'insieme Exercise 7: Confronta con un singolo classificatore Exercise 8: Ottimizza l'ensemble potenziato Exercise 9: Preparazione al tuning Exercise 10: La vera fase di tuning Exercise 11: Finalizza il modello Exercise 12: Confronto tra modelli Exercise 13: Confronta l'AUC Exercise 14: Traccia le curve ROC

Esercizio in corso

Exercise 15: Riepilogo