Make the validator
The submodule pyspark.ml.tuning also has a class called CrossValidator for performing cross validation. This Estimator takes the modeler you want to fit, the grid of hyperparameters you created, and the evaluator you want to use to compare your models.
The submodule pyspark.ml.tune has already been imported as tune. You'll create the CrossValidator by passing it the logistic regression Estimator lr, the parameter grid, and the evaluator you created in the previous exercises.
This exercise is part of the course
Foundations of PySpark
Exercise instructions
- Create a
CrossValidatorby callingtune.CrossValidator()with the arguments:estimator=lrestimatorParamMaps=gridevaluator=evaluator
- Name this object
cv.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Create the CrossValidator
cv = tune.____(estimator=____,
estimatorParamMaps=____,
evaluator=____
)