1. Learn
    2. /
  2. Courses
    3. /
  3. Deep Learning for Text with PyTorch
Connected

Exercise

Training and testing the RNN model with attention

At PyBooks, the team had previously built an RNN model for word prediction without the attention mechanism. This initial model, referred to as rnn_model, has already been trained and its instance is preloaded. Your task now is to train the new RNNWithAttentionModel and compare its predictions with that of the earlier rnn_model.

The following has been preloaded for you:

  • inputs: list of input sequences as tensors
  • targets: tensor containing target words for each input sequence
  • optimizer: Adam optimizer function
  • criterion: CrossEntropyLoss function
  • pad_sequences: function to pad input sequences for batching
  • attention_model: defined model class from the previous exercise
  • rnn_model:trained RNN model from the team at PyBooks

Instructions

100 XP
  • Set the RNN model to evaluation mode before testing it with the test data.
  • Get the RNN output by passing the appropriate input to the RNN model.
  • Extract the word with the highest prediction score from the RNN output.
  • Similarly, for the attention model, extract the word with the highest prediction score from the attention output.