Define aggregated measures
Now, you are going to define performance measures.
The knowledge_train_data
dataset has already been loaded for you, as have the packages mlr
and tidyverse
. And the following code has also been run:
task <- makeClassifTask(data = knowledge_train_data,
target = "UNS")
lrn <- makeLearner(cl = "classif.nnet", fix.factors.prediction = TRUE)
param_set <- makeParamSet(
makeIntegerParam("size", lower = 1, upper = 5),
makeIntegerParam("maxit", lower = 1, upper = 300),
makeNumericParam("decay", lower = 0.0001, upper = 1)
)
ctrl_random <- makeTuneControlRandom(maxit = 10)
This exercise is part of the course
Hyperparameter Tuning in R
Exercise instructions
- Use the
setAggregation
function, which aggregates the standard deviation of performance metrics. - Apply
setAggregation
to the mean misclassification error and accuracy after resampling. - Optimize your model by mean misclassification error. Remember that the first argument is used for optimization.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Create holdout sampling
holdout <- makeResampleDesc("Holdout", predict = "both")
# Perform tuning
lrn_tune <- tuneParams(learner = lrn,
task = task,
resampling = holdout,
control = ctrl_random,
par.set = param_set,
measures = list(___, ___(___, train.mean), ___, ___(___, train.mean)))