1. Creating a Keras model
Congrats. You've learned the theory of back-propagation, which is core to understanding deep learning. Now you'll learn how to create and optimize these networks using the Keras interface to the TensorFlow deep learning library.
2. Model building steps
The Keras workflow has 4 steps. First, you specify the architecture, which is things like: how many layers do you want? how many nodes in each layer? What activation function do you want to use in each layer? Next, you compile the model. This specifies the loss function, and some details about how optimization works. Then you fit the model. Which is that cycle of back-propagation and optimization of model weights with your data. And finally you will want to use your model to make predictions. We'll go through these steps sequentially. The first step is creating or specifying your model.
3. Model specification
Here is the code to do that. This code has three blocks. First we import what we will need. Numpy is here only for reading some data. The other two imports are used for building our model. The second block of two lines reads the data. We read the data here so we can find the number of nodes in the input layer. That is stored as the variable n_cols. We always need to specify how many columns are in the input when building a Keras model, because that is the number of nodes in the input layer. We then start building the model. The first line of model specification is model equals Sequential. There are two ways to build up a model, and we will focus on sequential, which is the easier way to build a model. Sequential models require that each layer has weights or connections only to the one layer coming directly after it in the network diagram. There are more exotic models out there with complex patterns of connections, but Sequential will do the trick for everything we need here. We start adding layers using the add method of the model. he type of layer you have seen, that standard layer type, is called a Dense layer. It is called Dense because all of the nodes in the previous layer connect to all of the nodes in the current layer. As you advance in deep learning, you may start using layers that aren't Dense. In each layer, we specify the number of nodes as the first positional argument, and the activation function we want to use in that layer using the keyword argument activation. Keras supports every activation function you will want in practice. In the first layer, we need to specify input shapes as shown here. That says the input will have n_cols columns, and there is nothing after the comma, meaning it can have any number of rows, that is, any number of data points. You'll notice the last layer has 1 node. That is the output layer, and it matches those diagrams where we ended with only a single node as the output or prediction of the model. This model has 2 hidden layers, and an output layer. You may be struck that each hidden layers has 100 nodes. Keras and TensorFlow do the math for us, so don't feel afraid to use much bigger networks than we've seen before. It's quite common to use 100 or 1000s nodes in a layer. You'll learn more about choosing an appropriate number of nodes later.
4. Let's practice!
For now, you can focus on how models are specified. It's time to practice that.