1. Learn
    2. /
  2. Courses
    3. /
  3. Transformer Models with PyTorch
Connected

Exercise

PyTorch Transformers

Now you're familiar with the different components of the transformer architecture, it's time to define one! The torch.nn module, imported for you as nn, provides a really nice way to do this in just a few lines of code.

Instructions

100 XP
  • Define a transformer with 8 attention heads, 6 encoder and decoder layers, and for input sequence embeddings of length 1536.
  • Print the model object to view the model architecture.