Build your first stacked ensemble
1. Build your first stacked ensemble
In this lesson, you'll learn to build your first stacked ensemble using scikit-learn.2. Stacking models with scikit-learn
Let's mention some features of the stacking implementation from scikit-learn. Since version 0.22, there are available implementations for stacking estimators. These make use of the scikit-learn estimators which you already know, both for the first-level as well as the second-level estimators. To generalize and avoid over-fitting, this final estimator is internally trained using cross validation.3. General Steps
As a reminder, here's the diagram depicting a stacking ensemble. There are 5 steps to build it. Step one, prepare the dataset. Step two, build the first-layer estimators. Step three, append the predictions from the individual estimators to the original dataset. Step four, build the second-layer meta estimator. And step five, use the stacked ensemble model for the final predictions.4. Stacking classifier
To build a stacking classifier using scikit-learn, you need to import the class from its ensemble module. We have to prepare a list of string - estimator tuples with the first-layer classifiers and their appropriate labels, as we did in the first chapter with Voting and Averaging. You must also instantiate the second-layer classifier of your preference. When calling the StackingClassifier, you have to pass the parameter estimators, which is the list you prepared. Also you need to specify the final_estimator parameter, which is the meta classifier. The third one is cv, the value you want to use for cross-validation, which is 5 by default. Another useful parameter is stack_method, which specifies the method called for each base estimator. If you want to use probabilities instead of class labels, set it to 'predict_proba'. Otherwise, you can leave the default which is 'auto'. We also have an interesting parameter called passthrough. This allows to train the final classifier on both the individual predictions and the original input features. It is false by default. Once your stacking classifier is ready, you can use from it the fit and predict methods just like you would with any other scikit-learn estimator.5. Stacking regressor
In a similar fashion, you can build a stacking regressor with scikit-learn. In this case, you import StackingRegressor class from the scikit-learn dot ensemble module. Make sure to instantiate the 1st-layer regressors to be used and their labels in a list of tuples. And don't forget the meta regressor. Now you're all set to build the Stacking regressor. The first parameter is also the list of estimators. The second parameter, with no surprises, is the final_estimator. This corresponds to the reg_meta object instantiated before. There is no stack_method parameter, as this is a regressor. However, passthrough parameter is also available if you want to use the original input features as well as the predictions from the first-layer regressors. With an instantiated stacking regressor, you can fit it to the training set and use it to make predictions.6. It's your turn!
Now's your turn to build your first stacked ensemble!Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.