LoslegenKostenlos loslegen

Adding the transformer head

Time to design a transformer head that could be used for classification tasks like sentiment analysis or categorization. You'll define a ClassifierHead class, create instances of the body and head, and pass a series of token IDs through them both to test that they work as expected.

Note: because this model has been trained yet, the outputs will be meaningless, but testing the code can process inputs and generate outputs in the form you expect is a good test.

Diese Übung ist Teil des Kurses

Transformer Models with PyTorch

Kurs anzeigen

Interaktive Übung

Versuche dich an dieser Übung, indem du diesen Beispielcode vervollständigst.

# Complete the classification head
class ClassifierHead(nn.Module):
    def __init__(self, d_model, num_classes):
        super().__init__()
        self.fc = ____

    def forward(self, x):
        logits = self.fc(x)
        return F.____(logits, dim=-1)
Code bearbeiten und ausführen