1. Learn
    2. /
  2. Courses
    3. /
  3. Intermediate Deep Learning with PyTorch

Exercise

Initialization and activation

The problems of unstable (vanishing or exploding) gradients are a challenge that often arises in training deep neural networks. In this and the following exercises, you will expand the model architecture that you built for the water potability classification task to make it more immune to those problems.

As a first step, you'll improve the weights initialization by using He (Kaiming) initialization strategy. To do so, you will need to call the proper initializer from the torch.nn.init module, which has been imported for you as init. Next, you will update the activations functions from the default ReLU to the often better ELU.

Instructions

100 XP
  • Call the He (Kaiming) initializer on the weight attribute of the second layer, fc2, similarly to how it's done for fc1.
  • Call the He (Kaiming) initializer on the weight attribute of the third layer, fc3, accounting for the different activation function used in the final layer.
  • Update the activation functions in the forward() method from relu to elu.