Variable Importance

1. Variable Importance

Complex models can be accurate, but how do we interpret them?

2. Adding more predictors

Real-life models include many variables. Sometimes thousands or even millions. Some models can be very precise but relatively obscure. Often referred to as black boxes. However, it is possible to lend interpretability by estimating variable importance by quantifying the relationship between the predictor and the outcome by changing each predictor individually and assessing the loss of performance when the effect of the predictor is absent.

3. Which variables matter most?

We can plot features by importance with the aid of the vip() package. In our case, contract_value_band seems to be the most influent feature in our model, followed by sponsor_code and category_code. Understanding variable importance can help us select model features and eliciting potential theoretical features.

4. Variable importance and feature engineering

Variable importance is closely related to domain knowledge when building a high-performing machine learning model. Domain knowledge can help identify which variables are likely to be important for making accurate predictions, and this understanding can guide the feature selection and engineering process. Once a model is built, variable importance measures can be used to assess the relative importance of each variable in making predictions. This information can be used to further refine the model and focus on the most informative variables. For example, variables with low importance scores can potentially be removed to simplify the model and improve its interpretability, or new variables can be added to better capture important information that was not included in the original model. In short, domain knowledge can guide the feature selection and engineering process, which in turn can impact the variable importance scores in the final model. Once the model is built, variable importance measures can provide further insight into which variables are most relevant for accurate predictions and can guide further refinement of the model.

5. Let's practice!

Time to give it a try.

Feature Engineering in R

IntermediateSkill Level

4.9+

35 reviews

You’ll wrap up the course by learning about feature engineering and machine learning techniques. You’ll begin by focusing on the problems associated with using all available features in a model and the importance of identifying irrelevant and redundant features and learning to remove these features using embedded methods such as lasso and elastic-net. Next, you’ll explore shrinkage methods such as lasso, ridge, and elastic-net, which can be used to regularize feature weights or select features by setting coefficients to zero. Finally, you’ll finish by focusing on creating an end-to-end feature engineering workflow and reviewing and practicing the previously learned concepts and functions in a small project.

Exercise 1: Reducing the model's features Exercise 2: Sifting through variable importance Exercise 3: Assessing model performance using all available predictors Exercise 4: Building a reduced model Exercise 5: Shrinkage methods Exercise 6: Manual regularization with Lasso Exercise 7: Tuning the penalty Exercise 8: Finalizing the model Exercise 9: Putting it all together Exercise 10: Prep and split Exercise 11: Preprocess Exercise 12: Model Exercise 13: Assess Exercise 14: Congratulations!