1. Learn
    2. /
  2. Courses
    3. /
  3. Transformer Models with PyTorch
Connected

Exercise

Constructing the encoder-decoder transformer

Now that you've updated the DecoderLayer class, and the equivalent changes have been made to TransformerDecoder, you're ready to put everything together. Because you've built your classes in a modular and hierarchical way, you only need to call instantiate two of them to build the encoder-decoder transformer: TransformerDecoder and TransformerEncoder.

Instructions

100 XP
  • Complete the forward() pass to compute the encoder and decoder outputs.
  • Instantiate and call the transformer on input_tokens using src_mask, tgt_mask, and cross_mask provided.