Training the model

You will train the previously implemented model in this exercise. Do you know that the Google's encoder-decoder based machine translation model took 2-4 days to train?

For this exercise you will be using a small dataset of 1500 sentences (i.e. en_text and fr_text) to train the model. This amount will hardly be enough to see good performance, but the method will remain the same. It is a matter of training on more data for longer. You have also been provided with the model nmt, and sents2seqs() function that you implemented previously. You will be reversing the encoder text to get better performance in this exercise. Here, en_x refers to the encoder input, where de_x refers to the decoder input.

This exercise is part of the course

Machine Translation with Keras

View Course

Exercise instructions

Get a single batch of encoder inputs (English sentences from index i to i+bsize) using the sents2seqs() function. Inputs need to be reversed and onehot encoded.
Get a single batch of decoder outputs (French sentences from index i to i+bsize) using the sents2seqs() function. Inputs need to be onehot encoded.
Train the model on a single batch of data containing en_x and de_y.
Obtain the evaluation metrics for en_x and de_y by evaluating the model with a batch_size of bsize.

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

n_epochs, bsize = 3, 250

for ei in range(n_epochs):
  for i in range(0,data_size,bsize):
    # Get a single batch of encoder inputs
    en_x = ____('source', ____, onehot=____, reverse=____)
    # Get a single batch of decoder outputs
    de_y = sents2seqs('target', fr_text[____], onehot=____)
    
    # Train the model on a single batch of data
    nmt.____(____, ____)    
    # Obtain the eval metrics for the training data
    res = nmt.____(____, de_y, batch_size=____, verbose=0)
    print("{} => Train Loss:{}, Train Acc: {}".format(ei+1,res[0], res[1]*100.0))

Edit and Run Code