1. Learn
    2. /
  2. Courses
    3. /
  3. Transformer Models with PyTorch
Connected

Exercise

The encoder transformer body

Your encoder-only transformer body is almost complete! It's time to combine the InputEmbeddings, PositionalEncoding, and EncoderLayer classes you've created previously into a TransformerEncoder class.

Instructions

100 XP
  • Define the token embedding, positional encoding, and encoder layers (use the list comprehension to create num_layers encoder layers).
  • Perform the forward pass through these layers.