Part 1: Defining the full model
Here you will be implementing the last few layers of the encoder-decoder model. You will be using Dense
and TimeDistributed
layers to get the final predictions (i.e. predicted French word probabilities) of the encoder-decoder model.
You are provided with the encoder and decoder (without the top-part) you implemented so far. The decoder GRU
layer's output de_out
is provided. We use the prefix en
(e.g. en_gru
) to indicate anything encoder related and de
to indicate decoder related things (e.g. de_gru
).
This exercise is part of the course
Machine Translation with Keras
Exercise instructions
- Import
Dense
andTimeDistributed
layers from Keras. - Define a
Dense
layer withsoftmax
activation which hasfr_vocab
outputs. - Wrap the
Dense
layer in aTimeDistributed
layer. - Get the final prediction of the model by passing
de_out
to thede_dense_time
layer.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Import Dense and TimeDistributed layers
from tensorflow.keras.____ import ____, ____
# Define a softmax dense layer that has fr_vocab outputs
de_dense = ____(____, ____)
# Wrap the dense layer in a TimeDistributed layer
de_dense_time = ____(de_dense)
# Get the final prediction of the model
de_pred = ____(de_out)
print("Prediction shape: ", de_pred.shape)