Stacking models

1. Stacking models

In this video, I will introduce you to the concept of "model stacking" or using the predictions from one model as an input to another model. Model stacking is a very advanced data science concept. It is the most sophisticated way of combining models, and when done right can yield some of the most accurate models in existence. Model stacking is often employed to win popular predictive modeling competitions.

2. Stacking models requires 2 datasets

In this course, you have been working with two datasets: the college basketball data from the regular season and the college basketball data from the post-season tournament. Both datasets contain the two teams playing, whether team 1 is home or away, and the score difference of the games. The tournament dataset additionally contains the difference in seeds of the two teams playing.

3. Enrich the tournament data

There's a lot more data on regular season games than there is on tournament games. The regular season dataset has over 300,000 rows, but the tournament dataset only has about 4,000 rows. 4,000 rows of data is pretty small. Recall that our embedding layer has about 11,000 inputs. 4,000 rows of data is not enough to learn all 11,000 parameters in our embedding layer. In the previous lesson, you built a three-input model on the regular season data. You can re-use this model to add predictions from the regular season model to the tournament dataset.

4. Enrich the tournament data

This diagram shows the process for stacking these 2 models. You start with the regular season dataset and fit a model to it. You then predict on the tournament dataset, using this model. This gives you predicted tournament outcomes, which you can now use to build a better model of the actual tournament outcomes. You additionally use the tournament seeds when modeling the tournament. These tournament seeds come from a committee, and are intended to, like your model, capture each team's "strength," without using an embedding layer. The tournament seeds can be thought of as a simplified version of your team strength model, determined by a human rather than a computer.

5. 3 input model with pure numeric data

The prediction from the regular season model captures the effects of team_1 and team_2, which means you now don't need to use those two variables in the tournament model, and can avoid the use of an embedding layer. You can focus your modeling efforts on the purely numeric data, which is a little easier to work with. With purely numeric inputs, you can pass all of them to a single input layer.

6. 3 input model with pure numeric data

In other words, an input layer with a shape of 3 is another way of defining a 3 input model. The only drawback of this approach is that all the inputs must be numeric.

7. 3 input model with pure numeric data

A huge advantage of this approach is simplicity. You can create a model with a single input tensor and an output tensor, and fit it using a single dataset. Similarly, evaluating the model requires a single dataset, rather than a list. As you can see, this stacked model is pretty accurate! It's off, on average, by about 9 points in a given game.

8. Let's practice!

To recap: stacking keras models means using the predictions from one model as an input to a second model. When stacking, it's important to use different datasets for each model. In this case, you use the regular season data for one model and the tournament dataset for the second model. Finally, if your input dataset is purely numeric, you can put multiple inputs in a single input layer. Lets practice using these concepts!

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.