1. Forward propagation
We’ll start by showing how neural networks use data to make predictions. This is called the forward propagation algorithm.
2. Bank transactions example
Let's revisit our example predicting how many transactions a user will make at our bank. For simplicity, we'll make predictions based on only the number of children and number of existing accounts.
3. Forward propagation
This graph shows a customer with
4. Forward propagation
two children and
5. Forward propagation
three accounts. The forward-propagation algorithm will pass this information through the network to make a prediction in the output layer.
6. Forward propagation
Lines connect the inputs to the hidden layer.
7. Forward propagation
Each line has a weight indicating how strongly that input effects the hidden node that the line ends at. These are the first set of weights. We have one weight from the
8. Forward propagation
top input into the top node of the layer,
9. Forward propagation
and one weight from the bottom input to the top node of the hidden layer. These weights are the parameters we train or change when we fit a neural network to data, so these weights will be a focus throughout this course. To make predictions for the top node of the hidden layer, we take the value of each node in the input layer, multiply it by the weight that ends at that node, and then sum up all the values. In this case, we get (2 times 1) plus (3 times 1), which is
10. Forward propagation
5.Now do the same to fill in the value of this node on the bottom. That is
11. Forward propagation
(two times (minus one))
12. Forward propagation
plus (three times one).
13. Forward propagation
That's one. Finally, repeat this process
14. Forward propagation
for the next layer, which is the output layer. That is
15. Forward propagation
(five times two) plus
16. Forward propagation
(one times -1). That gives an output
17. Forward propagation
of 9. We predicted nine transactions. That’s forward-propagation. We moved from
18. Forward propagation
the inputs on the left, to
19. Forward propagation
the hidden layer in the middle, and then from the hidden layers
20. Forward propagation
to the output on the right. We always
21. Forward propagation
use that same multiply then add process. If you're familiar with vector algebra or linear algebra, that operation is a dot product. If you don't know about dot products, that's fine too. That was forward propagation for a single data point. In general, we do forward propagation for one data point at a time. The value in that last layer is the model's prediction for that data point.
22. Forward propagation code
Let's see the code for this. We import Numpy for some of the mathematical operations. We've stored the input data as an array. We then have weights into each node in the hidden layer and to the output. We store the weights going into each node as an array, and we use a dictionary to store those arrays. Let’s start forward propagating. We fill in the top hidden node here, which is called node zero. We multiply the inputs by the weights for that node, and then sum both of those terms together. Notice that we had two weights for node_0. That matches the two items in the array it is multiplied by, which is the input_data. These get converted to a single number by the sum function at the end of the line. We then do the same thing for the bottom node of the hidden layer, which is called node 1.Now, both node zero and node one have numeric values.
23. Forward propagation code
To simplify multiplication, we put those in an array here. If we print out the array, we confirm that those are the values from the hidden layer you saw a moment ago. It can also be instructive to verify this by hand with pen and paper. To get the output, we multiply the values in the hidden layer by the weights for the output. Summing those together gives us 10 minus 1, which is 9.
24. Let's practice!
In the exercises, you'll practice performing forward propagation in small neural networks.