Classification and Regression Trees (CART) are a set of supervised learning models used for problems involving classification and regression. In this chapter, you'll be introduced to the CART algorithm.

Decision tree for classification

Train your first classification tree

Evaluate the classification tree

Logistic regression vs classification tree

Classification tree Learning

Growing a classification tree

Using entropy as a criterion

Entropy vs Gini index

Decision tree for regression

Train your first regression tree

Evaluate the regression tree

Linear regression vs regression tree

Classification and Regression Trees

The bias-variance tradeoff is one of the fundamental concepts in supervised machine learning. In this chapter, you'll understand how to diagnose the problems of overfitting and underfitting. You'll also be introduced to the concept of ensembling where the predictions of several models are aggregated to produce predictions that are more robust.

Generalization Error

Complexity, bias and variance

Overfitting and underfitting

Diagnose bias and variance problems

Instantiate the model

Evaluate the 10-fold CV error

Evaluate the training error

High bias or high variance?

Ensemble Learning

Define the ensemble

Evaluate individual classifiers

Better performance with a Voting Classifier

The Bias-Variance Tradeoff

Bagging is an ensemble method involving training the same algorithm many times using different subsets sampled from the training data. In this chapter, you'll understand how bagging can be used to create a tree ensemble. You'll also learn how the random forests algorithm can lead to further ensemble diversity through randomization at the level of each split in the trees forming the ensemble.

Bagging

Define the bagging classifier

Evaluate Bagging performance

Out of Bag Evaluation

Prepare the ground

OOB Score vs Test Set Score

Random Forests (RF)

Train an RF regressor

Evaluate the RF regressor

Visualizing features importances

Bagging and Random Forests

Boosting refers to an ensemble method in which several models are trained sequentially with each model learning from the errors of its predecessors. In this chapter, you'll be introduced to the two boosting methods of AdaBoost and Gradient Boosting.

Adaboost

Define the AdaBoost classifier

Train the AdaBoost classifier

Evaluate the AdaBoost classifier

Gradient Boosting (GB)

Define the GB regressor

Train the GB regressor

Evaluate the GB regressor

Stochastic Gradient Boosting (SGB)

Regression with SGB

Train the SGB regressor

Evaluate the SGB regressor

Boosting

The hyperparameters of a machine learning model are parameters that are not learned from data. They should be set prior to fitting the model to the training set. In this chapter, you'll learn how to tune the hyperparameters of a tree-based model using grid search cross validation.

Tuning a CART's Hyperparameters

Tree hyperparameters

Set the tree's hyperparameter grid

Search for the optimal tree

Evaluate the optimal tree

Tuning a RF's Hyperparameters

Random forests hyperparameters

Set the hyperparameter grid of RF

Search for the optimal forest

Evaluate the optimal forest

Congratulations!

Model Tuning

Auto-mpg

Bike Sharing Demand

Wisconsin Breast Cancer

Indian Liver Patient

Decision trees are supervised learning models used for problems involving classification and regression. Tree models present a high flexibility that comes at a price: on one hand, trees are able to capture complex non-linear relationships; on the other hand, they are prone to memorizing the noise present in a dataset. By aggregating the predictions of trees that are trained differently, ensemble methods take advantage of the flexibility of trees while reducing their tendency to memorize noise. Ensemble methods are used across a variety of fields and have a proven track record of winning many machine learning competitions.
In this course, you'll learn how to use Python to train decision trees and tree-based models with the user-friendly scikit-learn machine learning library. You'll understand the advantages and shortcomings of trees and demonstrate how ensembling can alleviate these shortcomings, all while practicing on real-world datasets. Finally, you'll also understand how to tune the most influential hyperparameters in order to get the most out of your models.

Supervised Learning with scikit-learn

In this course, you'll learn how to use tree-based models and ensembles for regression and classification using scikit-learn. 

Instantiate the model

“Machine Learning with Tree-Based Models in Python”

Exercise instructions

Hands-on interactive exercise

Machine Learning with Tree-Based Models in Python

Chapter 1: Classification and Regression Trees

Chapter 2: The Bias-Variance Tradeoff

Chapter 3: Bagging and Random Forests

Chapter 4: Boosting

Chapter 5: Model Tuning

What is DataCamp?