Measure performance

Now that we've fit our neural net, let's check performance to see how well our model is predicting new values. There's not a built-in .score() method like with sklearn models, so we'll use the r2_score() function from sklearn.metrics. This calculates the R\(^2\) score given arguments (y_true, y_predicted). We'll also plot our predictions versus actual values again. This will yield some interesting results soon (once we implement our own custom loss function).

This exercise is part of the course

Machine Learning for Finance in Python

View Course

Exercise instructions

Obtain predictions from model_1 on the scaled test set data (scaled_test_features and test_targets).
Print the R\(^2\) score on the test set (test_targets and test_preds).
Plot the test_preds versus test_targets in a scatter plot with plt.scatter().

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

from sklearn.metrics import r2_score

# Calculate R^2 score
train_preds = model_1.predict(scaled_train_features)
test_preds = model_1.predict(____)
print(r2_score(train_targets, train_preds))
print(r2_score(____, ____))

# Plot predictions vs actual
plt.scatter(train_preds, train_targets, label='train')
plt.scatter(____)
plt.legend()
plt.show()

Edit and Run Code