1. Single model for classification and regression
In this lesson, I will demonstrate how to build a simple model that performs both classification and regression.
2. Build a simple regressor/classifier
This is another example of a model with two outputs. In this case, however, rather than using two regression outputs, I have a regression output and a classification output.
As before, I define the regression part of this model with a single input and a Dense output layer with a single unit.
For the classification part of the model, I use the regression model prediction as input and then add another Dense output layer on top of it using the sigmoid activation, which will map the predicted score differences to probabilities that team 1 wins.
3. Make a regressor/classifier model
With two output models, each output needs its own loss function.
For this model, I've specified two different loss functions, one for the regression model and one for the classification model.
As with all the models in this course, I used adam for the optimizer.
4. Fit the combination classifier/regressor
To fit the combination classification/regression model, you must provide the y data as a list. Recall that this is similar to the way you built two input models in chapter 2.
Use seed_difference as the only input to this model.
For the regression output, I use score_difference, and for the classification output, I use whether or not team 1 won the game.
I split both of these variables out from the training set and pass them to the fit method as a list.
This model fits a bit more quickly than the last one, so I only use 100 epochs.
5. Look at the model's weights
This model's weight structure is a bit different from the last model, where both outputs were regression targets.
The first layer has a weight of 1.24 and a bias of almost zero. This means that a 1 unit change in the teams' seed difference yields about 1.24 additional points in their score difference.
So 2 teams with a seed difference of 1 would be expected to have team 1 win by 1.2 points. But 2 teams with a seed difference of 10 would be expected to have team 1 win by 12 points.
The next layer maps predicted score difference to predicted win/loss. Recall that the final layer in the model uses sigmoid activation.
6. Look at the model's weights
You can manually calculate the final layer in the model for some example data, to get an understanding of how the model has learned to relate score difference to win probabilities.
Scipy has a function called expit(), which is an efficient implementation of the sigmoid function.
Let's manually calculate the win probability for 2 teams that are predicted to have a score difference of 1.
First, multiply 1 by the weight for the final layer in the model: 0.14. Add the bias for the final layer: 0.0007. Since the bias is very close to zero, the result is still 0.14.
Finally, we apply the sigmoid function to 0.14, which yields a prediction of 0.54.
In other words, the model has learned that an expected score difference of 1 point is equal to an expected win probability of 54%.
7. Evaluate the model on new data
Finally, you can evaluate the model on new data.
First, split the evaluation dataset into a regression target and a classification target, and provide the same list of 2 targets to the evaluate() method.
This outputs 3 numbers now, instead of 1 as with the models we looked at in other chapters. The first number is the loss function used by the model, which is the sum of all the output losses.
The second number is the loss for the regression part of the model, and the third number is the loss for the classification part of the model.
So our model has a mean absolute error of 9.28 and a logloss of 0.58, which is pretty good, but I think you can do better with more data when you try for yourself.
8. Now you try!
Now you try fitting your own combination regression-classification model!