1. Machine learning with mlr
In order to fully understand the concepts of hyperparameter tuning, we will look at implementations in different contexts and packages.
Here, I will introduce the package mlr.
2. The mlr package
mlr is another very popular framework for machine learning in R. It provides methods for a number of algorithms and machine learning tasks, including supervised classification and regression.
mlr also allows you to easily perform hyperparameter tuning.
However, mlr has a slightly different way of defining machine learning tasks. Therefore, we will use this lesson to go over the basics of machine learning with mlr before we dive into hyperparameter tuning.
The workflow for training models with mlr follows three steps:
First, you need to define the task,
then you define the learner
and only then can you fit the model. We will go over each step in detail.
You can follow this link to find out more about mlr and how it works.
3. New dataset: User Knowledge Data
The dataset we'll be working with in this chapter is a real-world dataset on students' knowledge status about the subject of Electrical DC Machines.
It consists of 150 observations and 6 variables. The five features are
- STG (The degree of study time for goal object materials)
- SCG (The degree of repetition for goal object materials)
- STR (The degree of study time for related objects with goal object)
- LPR (The exam performance for related objects with goal object)
- and PEG (The exam performance for goal objects)
The response variable UNS (The knowledge level of the student) can be one of three classes: Low, Middle or high.
4. Tasks in mlr for supervised learning
Tasks define the data and - in case of supervised learning - the response variable or target. mlr has a number of different tasks you can choose from:
regression
classification
multi-label classification
and cost-sensitive classification
Here, we'll focus on tasks for supervised learning.
To create a classification task, we are using the makeClassifTask function. If you want to know which task functions you can define, either check the mlr manual or - if you are using RStudio - start typing make in your console and autocomplete will suggest different functions.
5. Learners in mlr
Next, we'll need to define our learner. You can find out which learners you can choose from by calling the listlearners function.
This will return a table of available learners and the name you will need to use as class if you want to create a learner object for it.
By convention, all classification learners start with “classif.”, all regression learners with “regr.” and all multilabel classification learners start with “multilabel.”.
Here, we want to create a learner for a deep neural network from the h2o package.
We create a new object with the makelearner function and "classif.h2o.deeplearning". In this function, we could also define whether we wanted labels or predictions as output, set hyperparameters, and more. Check the help function for makelearner to find out more.
Sometimes, you might have classification models where your target column contains more or fewer factor levels than the target column in your test or validation data. This can lead to problems but if you set fix.factors.prediction to TRUE a factor level for missing data is added.
And you can also define whether to return predicted labels or probabilities.
6. Model fitting in mlr
Finally, we can take our task and our learner and use them with the train function of mlr to fit our model.
We are again using the tictoc package to calculate how long the runtimes are for our models:
Here, the training took about 4 seconds.
7. Let's practice!
Alright, now it's your turn to try out mlr!