1. Learn
    2. /
  2. Courses
    3. /
  3. Deep Learning for Text with PyTorch

Exercise

Training and testing the Transformer model

With the TransformerEncoder model in place, the next step at PyBooks is to train the model on sample reviews and evaluate its performance. Training on these sample reviews will help PyBooks understand the sentiment trends in their vast repository. By achieving a well-performing model, PyBooks can then automate sentiment analysis, ensuring readers get insightful recommendations and feedback.

The following packages have been imported for you: torch, nn, optim.

The model instance of the TransformerEncoder class, token_embeddings, and the train_sentences, train_labels ,test_sentences,test_labels are preloaded for you.

Instructions

100 XP
  • In the training loop, split the sentences into tokens and stack the embeddings.
  • Zero the gradients and perform a backward pass.
  • In the predict function, deactivate the gradient computations then get the sentiment prediction.