Get startedGet started for free

Forward propagation

1. Forward propagation

We’ll start by showing how neural networks use data to make predictions. This is called the forward propagation algorithm.

2. Bank transactions example

Let's revisit our example predicting how many transactions a user will make at our bank. For simplicity, we'll make predictions based on only the number of children and number of existing accounts.

3. Forward propagation

This graph shows a customer with

4. Forward propagation

two children and

5. Forward propagation

three accounts. The forward-propagation algorithm will pass this information through the network to make a prediction in the output layer.

6. Forward propagation

Lines connect the inputs to the hidden layer.

7. Forward propagation

Each line has a weight indicating how strongly that input effects the hidden node that the line ends at. These are the first set of weights. We have one weight from the

8. Forward propagation

top input into the top node of the layer,

9. Forward propagation

and one weight from the bottom input to the top node of the hidden layer. These weights are the parameters we train or change when we fit a neural network to data, so these weights will be a focus throughout this course. To make predictions for the top node of the hidden layer, we take the value of each node in the input layer, multiply it by the weight that ends at that node, and then sum up all the values. In this case, we get (2 times 1) plus (3 times 1), which is

10. Forward propagation

5.Now do the same to fill in the value of this node on the bottom. That is

11. Forward propagation

(two times (minus one))

12. Forward propagation

plus (three times one).

13. Forward propagation

That's one. Finally, repeat this process

14. Forward propagation

for the next layer, which is the output layer. That is

15. Forward propagation

(five times two) plus

16. Forward propagation

(one times -1). That gives an output

17. Forward propagation

of 9. We predicted nine transactions. That’s forward-propagation. We moved from

18. Forward propagation

the inputs on the left, to

19. Forward propagation

the hidden layer in the middle, and then from the hidden layers

20. Forward propagation

to the output on the right. We always

21. Forward propagation

use that same multiply then add process. If you're familiar with vector algebra or linear algebra, that operation is a dot product. If you don't know about dot products, that's fine too. That was forward propagation for a single data point. In general, we do forward propagation for one data point at a time. The value in that last layer is the model's prediction for that data point.

22. Forward propagation code

Let's see the code for this. We import Numpy for some of the mathematical operations. We've stored the input data as an array. We then have weights into each node in the hidden layer and to the output. We store the weights going into each node as an array, and we use a dictionary to store those arrays. Let’s start forward propagating. We fill in the top hidden node here, which is called node zero. We multiply the inputs by the weights for that node, and then sum both of those terms together. Notice that we had two weights for node_0. That matches the two items in the array it is multiplied by, which is the input_data. These get converted to a single number by the sum function at the end of the line. We then do the same thing for the bottom node of the hidden layer, which is called node 1.Now, both node zero and node one have numeric values.

23. Forward propagation code

To simplify multiplication, we put those in an array here. If we print out the array, we confirm that those are the values from the hidden layer you saw a moment ago. It can also be instructive to verify this by hand with pen and paper. To get the output, we multiply the values in the hidden layer by the weights for the output. Summing those together gives us 10 minus 1, which is 9.

24. Let's practice!

In the exercises, you'll practice performing forward propagation in small neural networks.