Running a forward pass
1. Running a forward pass
We've explored tensors, small networks, and activation functions. Now, let's dive into generating predictions. This process is called "running a forward pass" through a network.2. What is a forward pass?
When input data flows through a neural network in the forward direction to produce outputs or predictions, it passes through each network layer. Calculations transform the data into new representations at every layer, which are passed to the next layer until the final output is produced. The purpose of the forward pass is to pass input data through the network and produce predictions or outputs based on the model's learned parameters, also known as weights and biases. This process is essential for both training and making new predictions.3. What is a forward pass?
The final output can be binary classifications, multi-class classifications, or numerical predictions (regressions). We'll look at an example for each.4. Binary classification: forward pass
Say we have input_data of five animals, with six features, or neurons, per data point. We create a small network with two linear layers and one sigmoid activation function in sequence. The first layer takes six features as input, outputs four, and the second layer processes this into a final probability.5. Binary classification: forward pass
The output of our binary classification is a single probability between zero and one for each of our five animals. Recall that we commonly use a threshold of 0.5 to turn these probabilities into class labels, such as 1 (mammal) or 0 (not mammal). This output will not be meaningful until we use backpropagation to update layer weights and biases. More on that later.6. Multi-class classification: forward pass
The model would be similar if we wanted to run multi-class classification. Say we are predicting three classes: mammal, bird, or reptile. We specify our model has three classes, setting this value as the last linear layer's output dimension. We use softmax instead of sigmoid, with dim=-1 to indicate the five animals have the same last dimension as the last linear layer's output. Using the same input as before, the output shape is five by three.7. Multi-class classification: forward pass
When we print the output, each row represents probabilities for three classes, which sum to one. The predicted label for each row is assigned to the class with the highest probability. In our example, the first and second rows are mammals, the third is a reptile, and so on.8. Regression: forward pass
The last model we'll look at is regression: predicting continuous numerical values. We'll now use the same data to predict the animal weights based on their properties. This time, there is no activation function at the end, and the last linear layer's last dimension returns an output with one feature. Output dimensions are five by one: five continuous values, one for each row.9. Let's practice!
Fast forward to some practice!Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.