Build implicit models
Now that you have all of your hyperparameter values specified, let's have Spark build enough models to test each combination. To facilitate this, a for
loop is provided here. Follow the instructions below to automatically create these ALS
models. In subsequent exercises you will run these models on test datasets to see which one performs the best.
The ALS
algorithm is already imported for you. The lists you created in the last exercise (ranks
, maxIters
, regParams
, alphas
) have been created for you.
This exercise is part of the course
Building Recommendation Engines with PySpark
Exercise instructions
- An empty list called
model_list
is provided here. Thefor
loop will create a model for each combination of hyperparameters it is given and put it intomodel_list
. - Complete the
for
loop by referencing the contents of each list wherer
represents the elements of theranks
list,mi
represents the elements of themaxIters
list,rp
represents the elements of theregParams
list anda
represents the elements of thealphas
list. - Print
len(model_list)
as well asmodel_list
to ensure that each model was created. The length should be equal to the product of all the lengths of each hyperparameter list above. You can run the additional validation provided to be sure.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# For loop will automatically create and store ALS models
for r in ____:
for mi in ____:
for rp in ____:
for a in ____:
model_list.append(ALS(userCol= "userId", itemCol= "songId", ratingCol= "num_plays", rank = r, maxIter = mi, regParam = rp, alpha = a, coldStartStrategy="drop", nonnegative = True, implicitPrefs = True))
# Print the model list, and the length of model_list
print (model_list, "Length of model_list: ", len(____))
# Validate
len(model_list) == (len(ranks)*len(maxIters)*len(regParams)*len(alphas))