1. Neural networks and layers
Let's build our first neural network using PyTorch Tensors.
2. Neural network layers
A neural network consists of input, hidden, and output layers.
3. Neural network layers
The input layer contains the dataset features,
4. Neural network layers
the output layer contains the predictions,
5. Neural network layers
and hidden layers lie in between.
6. Our first neural network
While a network can have any number of hidden layers, we'll begin by building a network with no hidden layers where the output layer is a linear layer. Here, every input neuron connects to every output neuron, referred to as a "fully connected" network. This network is equivalent to a linear model and helps us understand the fundamentals before adding complexity.
7. Designing a neural network
We'll use the torch.nn module to build our networks.
It makes neural network code more concise and flexible and is conventionally imported as nn.
When designing a neural network, the input and output layer dimensions are predefined.
The number of neurons in the input layer is the number of features in our dataset, and the number of neurons in the output layer is the number of classes we want to predict.
Say we create an input_tensor of shape 1 by 3. We can think of this as one row with three "features" or "neurons".
8. Designing a neural network
Next, we pass this input_tensor to a linear layer, which applies a linear function to make predictions.
nn.Linear() takes two arguments: in_features is the number of features in our input (three), and out_features is the desired size of the output tensor (in this case, two). Correctly specifying in_features ensures our linear layer can receive the input_tensor.
9. Designing a neural network
Lastly, we pass input_tensor to linear_layer to generate an output.
10. Designing a neural network
Notice that the output has two features or neurons due to the out_features specified in our linear layer.
11. Weights and biases
When input_tensor is passed to linear_layer, a linear operation is performed to include weights and biases.
12. Weights and biases
Each linear layer has a set of associated weights and biases.
These are the key quantities that define a neuron. The weights reflect the importance of different features. The bias is an additional term that is independent of the weights, providing the neuron with a baseline output.
At first, the linear layer assigns random weights and biases; these are tuned later.
13. A fully connected network in action
Let's imagine our fully connected network in action.
14. A fully connected network in action
Say we have a weather dataset with three features: temperature, humidity, and wind.
15. A fully connected network in action
We want to predict whether it will rain or be cloudy. The humidity feature will have a more significant weight compared to the other features, as it's a strong predictor of rain and clouds. The weather data is for a tropical region with a high probability of rain, so a bias is added to account for this baseline information. With this information, our model makes a prediction.
16. Let's practice!
Now it's your turn to practice.