Building a GRU model for text
At PyBooks, the team has been impressed with the performance of the two models you previously trained. However, in their pursuit of excellence, they want to ensure the selection of the absolute best model for the task at hand. Therefore, they have asked you to further expand the project by experimenting with the capabilities of GRU models, renowned for their efficiency and effectiveness in text classification tasks. Your new assignment is to apply the GRU model to classify articles from the Newsgroup dataset into the following categories:
rec.autos
, sci.med
, and comp.graphics
.
The following packages have been loaded for you: torch
, nn
, optim
.
This exercise is part of the course
Deep Learning for Text with PyTorch
Exercise instructions
- Complete the GRU class with the required parameters.
- Initialize the model with the same parameters.
- Train the model: pass the parameters to the criterion function, and backpropagate the loss.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Complete the GRU model
class GRUModel(nn.Module):
def __init__(self, input_size, hidden_size, num_layers, num_classes):
super(GRUModel, self).__init__()
self.hidden_size = hidden_size
self.num_layers = num_layers
self.gru = ____
self.fc = nn.Linear(hidden_size, num_classes)
def forward(self, x):
h0 = torch.zeros(self.num_layers, x.size(0), self.hidden_size)
out, _ = self.gru(x, h0)
out = out[:, -1, :]
out = self.fc(out)
return out
# Initialize the model
gru_model = ____
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(gru_model.parameters(), lr=0.01)
# Train the model and backpropagate the loss after initialization
for epoch in range(15):
optimizer.zero_grad()
outputs = ____
loss = criterion(____, y_train_seq)
____
optimizer.step()
print(f'Epoch: {epoch+1}, Loss: {loss.item()}')