Defining the decoder

In this exercise, you will implement the decoder and define an end-to-end model going from encoder inputs to the decoder GRU outputs. The decoder uses the same model as the encoder. However there are differences in the inputs and states fed to the decoder, compared to the encoder. For example, the decoder consumes the context vector produced by the encoder as inputs as well as the initial state to the decoder. Remember that we will use the prefix en (e.g. en_gru) to indicate anything encoder related and de to indicate decoder related things (e.g. de_gru).

To implement the decoder you will use RepeatVector and GRU layers.

For this exercise you have been provided with the encoder model and the various layers of the encoder that you have already implemented. For example, the encoder inputs are provided as en_inputs and the context vector as en_state. Also note that the GRU and Model objects have been already imported.

This exercise is part of the course

Machine Translation with Keras

View Course

Exercise instructions

Define an RepeatVector layer that takes en_state as an input and repeats it fr_len times.
Define a GRU layer, decoder_gru, which has hidden units equal to hsize and returns all the outputs produced.
Get the output of the decoder_gru layer by feeding in de_inputs as the input and en_state as the initial state of the decoder.
Define a model that takes en_inputs as the input and gru_outputs as the output.

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

from tensorflow.keras.layers import RepeatVector

hsize = 48
fr_len = 20
# Define a RepeatVector layer
de_inputs = ____(____)(____)
# Define a GRU model that returns all outputs
decoder_gru = ____(____, ____=____)
# Get the outputs of the decoder
gru_outputs = ____(____, initial_state=____)
# Define a model with the correct inputs and outputs
enc_dec = ____(inputs=____, outputs=____)
enc_dec.summary()

Edit and Run Code