ComeçarComece de graça

Adding the transformer head

Time to design a transformer head that could be used for classification tasks like sentiment analysis or categorization. You'll define a ClassifierHead class, create instances of the body and head, and pass a series of token IDs through them both to test that they work as expected.

Note: because this model has been trained yet, the outputs will be meaningless, but testing the code can process inputs and generate outputs in the form you expect is a good test.

Este exercício faz parte do curso

Transformer Models with PyTorch

Ver curso

Exercício interativo prático

Experimente este exercício completando este código de exemplo.

# Complete the classification head
class ClassifierHead(nn.Module):
    def __init__(self, d_model, num_classes):
        super().__init__()
        self.fc = ____

    def forward(self, x):
        logits = self.fc(x)
        return F.____(logits, dim=-1)
Editar e executar o código