Ensembling models

One approach to improve predictions from machine learning models is ensembling. A basic approach is to average the predictions from multiple models. A more complex approach is to feed predictions of models into another model, which makes final predictions. Both approaches usually improve our overall performance (as long as our individual models are good). If you remember, random forests are also using ensembling of many decision trees.

To ensemble our neural net predictions, we'll make predictions with the 3 models we just created -- the basic model, the model with the custom loss function, and the model with dropout. Then we'll combine the predictions with numpy's .hstack() function, and average them across rows with np.mean(predictions, axis=1).

Create predictions on the scaled_train_features and scaled_test_features for the 3 models we fit (model_1, model_2, model_3) using the .predict() method.
Horizontally stack (np.hstack() the predictions into a matrix, and take the row-wise averages to get average predictions for the train and test sets.

Exercise

Ensembling models

Instructions