Boosting

In the last exercise, you started off with ensemble techniques by using bagging. In a machine learning interview, you might be prompted to try out or discuss more than one ensemble technique.

Here, you'll practice Boosting which uses all data to train each learner, but instances that were misclassified by the previous learners are given more weight so that subsequent learners give more focus to them during training. This results in a model with decreased bias.

All relevant packages have been imported for you: pandas as pd, train_test_split from sklearn.model_selection, accuracy_score from sklearn.linear_model, LogisticRegression from sklearn.linear_model, and BaggingClassifier and AdaBoostClassifier from sklearn.ensemble.

The loan_data DataFrame is already split into X_train, X_test, y_train and y_test.