Build your cross validation pipeline
Now that we have our data, our train/test splits, our model, and our hyperparameter values, let's tell Spark how to cross validate our model so it can find the best combination of hyperparameters and return it to us.
This exercise is part of the course
Building Recommendation Engines with PySpark
Exercise instructions
- Create a
CrossValidator
calledcv
with ourals
model as the estimator, settingestimatorParamMaps
to theparam_grid
you just built. Tell Spark that theevaluator
to be used is the"evaluator"
we built previously. Set thenumFolds
to 5. - Confirm that our
cv
was built by printingcv
.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Build cross validation using CrossValidator
____ = CrossValidator(estimator=____, estimatorParamMaps=____, evaluator=____, numFolds=____)
# Confirm cv was built
print(____)