Advanced transformations

1. Advanced transformations

We will now move on to learn and try out a few less common, yet powerful, feature transformation techniques.

2. The data

We will be working with a numeric cut of the loans dataset that contains Loan_Status and five features for 480 observations, such as ApplicantIncome, LoanAmount, and Credit_History. Our goal is to predict Loan_Status and explore the effect of a few new transformations.

3. Predict with plain workflow

We configure our plain recipe by defining a prediction formula and setting the data to "train", and bundling it with our model via a "workflow" to set our baseline. We then fit the workflow using the train data and assess its performance with the test set. Our plain workflow does a better than random job in predicting Loan_Status with an accuracy of 0-point-75. Lets see if can we improve it with a few feature engineering tricks.

4. The step_poly() function

step_poly() will seamlessly create new variables based on powers of the original ones. The default degree, which is what we are using here, is two which will add a squared feature to our data set so we can capture non-linear relationships. Of course, increasing the degree to gain more flexibility is always tempting, but this should be done carefully, as it can result in overfitting. This transformation maintained our accuracy metric but improved our ROC AUC significantly.

5. The step_percentile() function

step_percentile() determines the empirical distribution of a variable based on the training set and converts all values to percentiles. Sometimes replacing variables by their percentiles can improve our model. There is no clear-cut rule for this and it is one example of feature engineering being as much art as it is science. In this case, we are going to convert the numeric predictors to percentiles. Implementing this transformation improves our accuracy, but decreases ROC AUC.

6. Let's practice!

It's now time to get our hands dirty!

Feature Engineering in R

IntermediateSkill Level

4.9+

34 reviews

You’ll wrap up the course by learning about feature engineering and machine learning techniques. You’ll begin by focusing on the problems associated with using all available features in a model and the importance of identifying irrelevant and redundant features and learning to remove these features using embedded methods such as lasso and elastic-net. Next, you’ll explore shrinkage methods such as lasso, ridge, and elastic-net, which can be used to regularize feature weights or select features by setting coefficients to zero. Finally, you’ll finish by focusing on creating an end-to-end feature engineering workflow and reviewing and practicing the previously learned concepts and functions in a small project.

Exercise 1: Reducing the model's features Exercise 2: Sifting through variable importance Exercise 3: Assessing model performance using all available predictors Exercise 4: Building a reduced model Exercise 5: Shrinkage methods Exercise 6: Manual regularization with Lasso Exercise 7: Tuning the penalty Exercise 8: Finalizing the model Exercise 9: Putting it all together Exercise 10: Prep and split Exercise 11: Preprocess Exercise 12: Model Exercise 13: Assess Exercise 14: Congratulations!