Get startedGet started for free

Training the word embedding based model

Here you will learn how to implement the training process for a machine translator model that uses word embeddings. A word is represented as a single number instead of a one-hot encoded vector as you did in previous exercises. You will train the model for multiple epochs while traversing through the full dataset in batches.

For this exercise you are provided with training data (tr_en and tr_fr) in the form of a list of sentences. You will only use a very small sample (1000 sentences) of the actual data as it can otherwise take very long to train. You also have the sents2seqs() function and the model, nmt_emb, that you implemented in the previous exercise. Remember that we use en_x to refer to encoder inputs and de_x to refer to decoder inputs.

This exercise is part of the course

Machine Translation with Keras

View Course

Exercise instructions

  • Get a single batch of French sentences without onehot encoding using the sents2seqs() function.
  • Get all the words except the last from de_xy.
  • Get all the words except the first from de_xy_oh (French words with onehot encoding).
  • Train the model using a single batch of data

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

for ei in range(3):
  for i in range(0, train_size, bsize):    
    en_x = sents2seqs('source', tr_en[i:i+bsize], onehot=False, reverse=True)
    # Get a single batch of French sentences with no onehot encoding
    de_xy = ____('target', ____[i:i+bsize], ____=____)
    # Get all words except the last word in that batch
    de_x = de_xy[:,____]
    de_xy_oh = sents2seqs('target', tr_fr[i:i+bsize], onehot=True)
    # Get all words except the first from de_xy_oh
    de_y = de_xy_oh[____,____,____]
    # Training the model on a single batch of data
    nmt_emb.train_on_batch([____,____], ____)    
    res = nmt_emb.evaluate([en_x, de_x], de_y, batch_size=bsize, verbose=0)
    print("{} => Loss:{}, Train Acc: {}".format(ei+1,res[0], res[1]*100.0))
Edit and Run Code