Get startedGet started for free

Building a GRU model for text

At PyBooks, the team has been impressed with the performance of the two models you previously trained. However, in their pursuit of excellence, they want to ensure the selection of the absolute best model for the task at hand. Therefore, they have asked you to further expand the project by experimenting with the capabilities of GRU models, renowned for their efficiency and effectiveness in text classification tasks. Your new assignment is to apply the GRU model to classify articles from the Newsgroup dataset into the following categories:

rec.autos, sci.med, and comp.graphics.

The following packages have been loaded for you: torch, nn, optim.

This exercise is part of the course

Deep Learning for Text with PyTorch

View Course

Exercise instructions

  • Complete the GRU class with the required parameters.
  • Initialize the model with the same parameters.
  • Train the model: pass the parameters to the criterion function, and backpropagate the loss.

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

# Complete the GRU model
class GRUModel(nn.Module):
    def __init__(self, input_size, hidden_size, num_layers, num_classes):
        super(GRUModel, self).__init__()
        self.hidden_size = hidden_size
        self.num_layers = num_layers
        self.gru = ____
        self.fc = nn.Linear(hidden_size, num_classes)       
    def forward(self, x):
        h0 = torch.zeros(self.num_layers, x.size(0), self.hidden_size) 
        out, _ = self.gru(x, h0)
        out = out[:, -1, :] 
        out = self.fc(out)
        return out

# Initialize the model
gru_model = ____
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(gru_model.parameters(), lr=0.01)

# Train the model and backpropagate the loss after initialization
for epoch in range(15): 
    optimizer.zero_grad()
    outputs = ____
    loss = criterion(____, y_train_seq)
    ____
    optimizer.step()
    print(f'Epoch: {epoch+1}, Loss: {loss.item()}')
Edit and Run Code