1. Learn
    2. /
  2. Courses
    3. /
  3. Transformer Models with PyTorch
Connected

Exercise

Adding the transformer head

Time to design a transformer head that could be used for classification tasks like sentiment analysis or categorization. You'll define a ClassifierHead class, create instances of the body and head, and pass a series of token IDs through them both to test that they work as expected.

Note: because this model has been trained yet, the outputs will be meaningless, but testing the code can process inputs and generate outputs in the form you expect is a good test.

Instructions 1/3

undefined XP
    1
    2
    3
  • Complete the transformer head to convert the encoder outputs into class predictions (the number of classes is represented by num_classes).