Introduction to RNN inside Keras

1. Introduction to RNN inside Keras

In this lesson, we implement the RNN models using keras. Previously, you were introduced to the architecture of language models. Now we will use keras to create and train RNN models.

2. What is keras?

Keras is a high-level API built on top of the deep learning frameworks Tensorflow two. To install keras, we can simply install Tensorflow using the Python package manager pip. After installation, we can use its modules to execute fast experimentation and research. Next we will introduce the main modules of keras that will be useful for the language models

3. keras.models

keras models contain two classes of models. The Sequential class has a structure where each layer is implemented one after the other, meaning that the output of one layer is the input of the next one. The Model class is a generic definition of a model that is more flexible and allows multiple inputs and outputs.

4. keras.layers

keras layers contains the different types of layers including the LSTM and GRU cells. Other layers that we will use are Dense, Dropout, Embedding and Bidirectional.

5. keras.preprocessing

keras preprocessing contains useful functions for pre-processing the data such as the pad_sequences method that transforms text data into fixed-length vectors. In the example, we padded the texts to equal length of 3.

6. keras.datasets

The datasets module contains useful datasets. For example, the imdb movie reviews that is used for sentiment analysis. Also, the reuters newswire dataset used for topic classification with 46 classes. There are also other datasets that you can check on the keras website.

7. Creating a model

You can build a Sequential model in keras with just a few lines of code. Import the required classes as: from tensorflow import keras from tensorflow dot keras dot models import sequential from tensorflow dot keras dot layers import dense Then, instantiate the class in the variable called model with: model equals to sequential open and close parenthesis. Add desired layers with the method add as in: model dot add dense 64, activation equals to the string relu, input_dim equals to 100 The parameter input dim declares the shape of the input data, which is mandatory for the first layer in the model. Then add the output layer: model dot add dense 1, activation equal to the string sigmoid Finally, we compile the model by executing the compile method of the class. We pass the string adam to the optimizer parameter, the string mean squared error to loss, and a single-element list containing the string accuracy to the metrics parameter

8. Training the model

To train the model, we use the fit method on the training data. For example: model dot fit x train, y train, epochs equal to 10 batch_size equal to 32 epochs is the number of iterations over the entire dataset and defaults to one. batch size is the size of a subset of the data that will be used on each step. When the dataset cannot fit in the memory this is crucial. It defaults to 32.

9. Model evaluation and usage

To analyze the model's performance, we can use the method evaluate as model dot evaluate x-test comma y-test This method returns the loss and accuracy values. To use the model on new data, use the method predict as: model dot predict new_data

10. Full example: IMDB Sentiment Classification

To create a full example, let's instantiate the Sequential class, add three layers (don't bother with new layers for now, we will explain them in details on chapter 2) and compile. Next, we can use the training set to fit the model. And measure its accuracy on the test set.

11. Time to practice!

You learned the main modules present in keras, and how to create, evaluate and use the model. Now, it's time to practice!

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.