Setting up your data for analysis

You will look at a version of the nycflights13 dataset, loaded as flights. It contains information on flights departing from New York City. You are interested in predicting whether or not they will arrive late to their destination, but first, you need to set up the data for analysis.

After discussing our model goals with a team of experts, you selected the following variables for your model: flight, sched_dep_time, dep_delay, sched_arr_time, carrier, origin, dest, distance, date, arrival.

You will also mutate() the date using as.Date() and convert character type variables to factors.

Lastly, you will split the data into train and test datasets.

This exercise is part of the course

Feature Engineering in R

View Course

Exercise instructions

Transform all character-type variables to factors.
Split the flights data into test and train sets.

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

flights <- flights %>%
  select(flight, sched_dep_time, dep_delay, sched_arr_time, carrier, origin, dest, distance, date, arrival) %>%

# Tranform all character-type variables to factors
  mutate(date = as.Date(date), ___(where(is.character), as.factor))

# Split the flights data into test and train sets
set.seed(246)
split <- flights %>% initial_split(prop = 3/4, strata = arrival)
test <- ___(split)
train <- ___(split)

test %>% select(arrival) %>% table() %>% prop.table()
train %>% select(arrival) %>% table() %>% prop.table()

Edit and Run Code

Feature Engineering in R

IntermediateSkill Level

4.9+

40 reviews

You’ll wrap up the course by learning about feature engineering and machine learning techniques. You’ll begin by focusing on the problems associated with using all available features in a model and the importance of identifying irrelevant and redundant features and learning to remove these features using embedded methods such as lasso and elastic-net. Next, you’ll explore shrinkage methods such as lasso, ridge, and elastic-net, which can be used to regularize feature weights or select features by setting coefficients to zero. Finally, you’ll finish by focusing on creating an end-to-end feature engineering workflow and reviewing and practicing the previously learned concepts and functions in a small project.

Exercise 1: Reducing the model's features Exercise 2: Sifting through variable importance Exercise 3: Assessing model performance using all available predictors Exercise 4: Building a reduced model Exercise 5: Shrinkage methods Exercise 6: Manual regularization with Lasso Exercise 7: Tuning the penalty Exercise 8: Finalizing the model Exercise 9: Putting it all together Exercise 10: Prep and split Exercise 11: Preprocess Exercise 12: Model Exercise 13: Assess Exercise 14: Congratulations!