Exercise

# Model evaluation using MSE

After generating the predicted ratings from the test data using ALS model, in this final part of the exercise, you'll prepare the data for calculating Mean Square Error (MSE) of the model. The MSE is the average value of `(original rating – predicted rating)^2`

for all users and indicates the absolute fit of the model to the data. To do this, first, you'll organize both the ratings and prediction RDDs to make a tuple of ((user, product), rating)), then join the ratings RDD with prediction RDD and finally apply a squared difference function along with `mean()`

to get the MSE.

Remember, you have a SparkContext `sc`

available in your workspace. Also, `ratings_final`

and `predictions`

RDD are already available in your workspace.

Instructions

**100 XP**

- Organize
`ratings`

RDD to make`((user, product), rating)`

. - Organize
`predictions`

RDD to make`((user, product), rating)`

. - Join the prediction RDD with the ratings RDD.
- Evaluate the model using MSE between original rating and predicted rating and print it.