1. Learn
    2. /
  2. Courses
    3. /
  3. Transformer Models with PyTorch
Connected

Exercise

Completing the decoder transformer

Time to build the decoder transformer body! This will mean combining the InputEmbeddings, PositionalEncoding, and DecoderLayer classes you've created previously.

Instructions

100 XP
  • Define a list of num_layers decoder layers using a list comprehension and the DecoderLayer class.
  • Define a linear layer to project the hidden states into word likelihoods.
  • Complete the forward pass through the layers defined in __init__.
  • Instantiate a decoder transformer and apply it to input_tokens and tgt_mask.