Try another threshold
In the previous exercises, you used a threshold of 0.50 to cut your predicted probabilities to make class predictions (rock vs mine). However, this classification threshold does not always align with the goals for a given modeling problem.
For example, pretend you want to identify the objects you are really certain are mines. In this case, you might want to use a probability threshold of 0.90 to get fewer predicted mines, but with greater confidence in each prediction.
The code pattern for cutting probabilities into predicted classes, then calculating a confusion matrix, was shown in Exercise 7 of this chapter.
This exercise is part of the course
Machine Learning with caret in R
Exercise instructions
- Use
ifelse()
to create a character vector,m_or_r
that is the positive class,"M"
, whenp
is greater than 0.9, and the negative class,"R"
, otherwise. - Convert
m_or_r
to be a factor,p_class
, with levels the same as those oftest[["Class"]]
. - Make a confusion matrix with
confusionMatrix()
, passingp_class
and the"Class"
column from thetest
dataset.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# If p exceeds threshold of 0.9, M else R: m_or_r
# Convert to factor: p_class
# Create confusion matrix