Create the evaluator

The first thing you need when doing cross validation for model selection is a way to compare different models. Luckily, the pyspark.ml.evaluation submodule has classes for evaluating different kinds of models. Your model is a binary classification model, so you'll be using the BinaryClassificationEvaluator from the pyspark.ml.evaluation module.

This evaluator calculates the area under the ROC. This is a metric that combines the two kinds of errors a binary classifier can make (false positives and false negatives) into a simple number. You'll learn more about this towards the end of the chapter!

Import the submodule pyspark.ml.evaluation as evals.
Create evaluator by calling evals.BinaryClassificationEvaluator() with the argument metricName="areaUnderROC".

Take Hint (-30 XP)

Exercise

Create the evaluator

Instructions