CommencerCommencer gratuitement

Adding the transformer head

Time to design a transformer head that could be used for classification tasks like sentiment analysis or categorization. You'll define a ClassifierHead class, create instances of the body and head, and pass a series of token IDs through them both to test that they work as expected.

Note: because this model has been trained yet, the outputs will be meaningless, but testing the code can process inputs and generate outputs in the form you expect is a good test.

Cet exercice fait partie du cours

Transformer Models with PyTorch

Afficher le cours

Exercice interactif pratique

Essayez cet exercice en complétant cet exemple de code.

# Complete the classification head
class ClassifierHead(nn.Module):
    def __init__(self, d_model, num_classes):
        super().__init__()
        self.fc = ____

    def forward(self, x):
        logits = self.fc(x)
        return F.____(logits, dim=-1)
Modifier et exécuter le code