Build your LSTM model

You've already prepared your sequences of text, with each of the sequences consisting of four words. It's time to build your LSTM model!

Your model will be trained on the first three words of each sequence, predicting the 4th one. You are going to use an Embedding layer that will essentially learn to turn words into vectors. These vectors will then be passed to a simple LSTM layer. Our output is a Dense layer with as many neurons as words in the vocabulary and softmax activation. This is because we want to obtain the highest probable next word out of all possible words.

The size of the vocabulary of words (the unique number of words) is stored in vocab_size.

Import the Embedding, LSTM and Dense layer from Keras layers.
Add an Embedding() layer of the vocabulary size, that will encode words into 8 number vectors and receive sequences of length 3.
Add a 32 unit LSTM() layer.
Add a hidden Dense() layer of 32 units and an output layer of vocab_size with softmax.

Take Hint (-30 XP)

Exercise

Build your LSTM model

Instructions