Visualizing feature importances: What features are most important in my dataset
Another way to visualize your XGBoost models is to examine the importance of each feature column in the original dataset within the model.
One simple way of doing this involves counting the number of times each feature is split on across all boosting rounds (trees) in the model, and then visualizing the result as a bar graph, with the features ordered according to how many times they appear. XGBoost has a plot_importance()
function that allows you to do exactly this, and you'll get a chance to use it in this exercise!
This exercise is part of the course
Extreme Gradient Boosting with XGBoost
Exercise instructions
- Create your
DMatrix
fromX
andy
as before. - Create a parameter dictionary with appropriate
"objective"
("reg:squarederror"
) and a"max_depth"
of4
. - Train the model with
10
boosting rounds, exactly as you did in the previous exercise. - Use
xgb.plot_importance()
and pass in the trained model to generate the graph of feature importances.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Create the DMatrix: housing_dmatrix
housing_dmatrix = ____
# Create the parameter dictionary: params
params = ____
# Train the model: xg_reg
xg_reg = ____
# Plot the feature importances
____
plt.show()