Defining the embedding model

You will be defining a Keras model that:

Uses Embedding layers
Will be trained with Teacher Forcing

This model will have two embedding layers; an encoder embedding layer and a decoder embedding layer. Furthermore, as the model is trained using Teacher Forcing, it will use a sequence length of fr_len-1 in the decoder Input layer.

For this exercise, you have all the required keras.layers and Model imported. Furthermore the variables, en_len (English sequence length), fr_len (French sequence length), en_vocab (English vocabulary size), fr_vocab (French vocabulary size) and hsize (hidden size) have been defined.

Latihan ini adalah bagian dari kursus

Machine Translation with Keras

Lihat Kursus

Petunjuk latihan

Define an Input layer which accepts a sequence of word IDs.
Define an Embedding layer that embeds en_vocab words, has length 96 and can accept a sequence of IDs (sequence length is specified using the input_length argument).
Define an Embedding layer that embeds fr_vocab words, has length 96 and can accept a sequence of fr_len-1 IDs.
Define a model that takes an input from the encoder and an input from the decoder (in that order) and outputs the word predictions.

Latihan interaktif praktis

Cobalah latihan ini dengan menyelesaikan kode contoh berikut.

# Define an input layer which accepts a sequence of word IDs
en_inputs = Input(____=(____,))
# Define an Embedding layer which accepts en_inputs
en_emb = ____(____, ____, input_length=____)(en_inputs)
en_out, en_state = GRU(hsize, return_state=True)(en_emb)

de_inputs = Input(shape=(fr_len-1,))
# Define an Embedding layer which accepts de_inputs
de_emb = Embedding(____, 96, input_length=____)(____)
de_out, _ = GRU(hsize, return_sequences=True, return_state=True)(de_emb, initial_state=en_state)
de_pred = TimeDistributed(Dense(fr_vocab, activation='softmax'))(de_out)

# Define the Model which accepts encoder/decoder inputs and outputs predictions 
nmt_emb = Model([____, ____], ____)
nmt_emb.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['acc'])

Edit dan Jalankan Kode