Defining the decoder
In this exercise, you will implement the decoder and define an end-to-end model going from encoder inputs to the decoder GRU outputs. The decoder uses the same model as the encoder. However there are differences in the inputs and states fed to the decoder, compared to the encoder. For example, the decoder consumes the context vector produced by the encoder as inputs as well as the initial state to the decoder. Remember that we will use the prefix en
(e.g. en_gru
) to indicate anything encoder related and de
to indicate decoder related things (e.g. de_gru
).
To implement the decoder you will use RepeatVector
and GRU
layers.
For this exercise you have been provided with the encoder model and the various layers of the encoder that you have already implemented. For example, the encoder inputs are provided as en_inputs
and the context vector as en_state
. Also note that the GRU
and Model
objects have been already imported.
This exercise is part of the course
Machine Translation with Keras
Exercise instructions
- Define an
RepeatVector
layer that takesen_state
as an input and repeats itfr_len
times. - Define a GRU layer,
decoder_gru
, which has hidden units equal tohsize
and returns all the outputs produced. - Get the output of the
decoder_gru
layer by feeding inde_inputs
as the input anden_state
as the initial state of the decoder. - Define a model that takes
en_inputs
as the input andgru_outputs
as the output.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
from tensorflow.keras.layers import RepeatVector
hsize = 48
fr_len = 20
# Define a RepeatVector layer
de_inputs = ____(____)(____)
# Define a GRU model that returns all outputs
decoder_gru = ____(____, ____=____)
# Get the outputs of the decoder
gru_outputs = ____(____, initial_state=____)
# Define a model with the correct inputs and outputs
enc_dec = ____(inputs=____, outputs=____)
enc_dec.summary()