Novel levels

When a level of a categorical variable is rare, sometimes it will fail to show up in training data. If that rare level then appears in future data, downstream models may not know what to do with it. When such novel levels appear, using model.matrix or caret::dummyVars to one-hot-encode will not work correctly.

vtreat is a "safer" alternative to model.matrix for one-hot-encoding, because it can manage novel levels safely. vtreat also manages missing values in the data (both categorical and continuous).

In this exercise you will see how vtreat handles categorical values that did not appear in the training set. The treatment plan treatplan and the set of variables newvars from the previous exercise are still in your workspace. dframe and a new data frame testframe are also in your workspace.

Print dframe and testframe.
- Are there colors in testframe that didn't appear in dframe?
Call prepare() to create a one-hot-encoded version of testframe (without the outcome). Call it testframe.treat and print it.
- Use the varRestriction argument to restrict to only the variables in newvars.
- How are the yellow rows encoded?

Take Hint (-30 XP)

Exercise

Novel levels

Instructions