Keras callbacks
1. Keras callbacks
By now you've trained a lot of models. It's time to learn more about how to better control and supervise model training by using callbacks.2. What is a callback?
A callback is a function that is executed after some other function, event, or task has finished. For instance, when you touch your phone screen, a block of code that identifies the type of gesture will be triggered. Since this block of code has been called after the touching event occurred, it's a callback.3. Callbacks in Keras
In the same way, a keras callback is a block of code that gets executed after each epoch during training or after the training is finished. They are useful to store metrics as the model trains and to make decisions as the training goes by.4. A callback you've been missing
Every time you call the fit method on a keras model there's a callback object that gets returned after the model finishes training. This is the history object. Accessing the history attribute, which is a python dictionary, we can check the saved metrics of the model during training as an array of numbers.5. A callback you've been missing
To get the most out of the history object we should use the validation_data parameter in our fit method, passing X_test and y_test as a tuple. The validation_split parameter can be used instead, specifying a percentage of the training data that will be left out for testing purposes. That way we not only have the training metrics but also the validation metrics.6. History plots
You can compare training and validation metrics with a few matplotlib commands. We just need to define a figure. Plot the values of the history attribute for the training accuracy and the validation accuracy. We can then make our graph prettier by adding a title, axis labels and a legend.7. History plots
We can see our model accuracy increases for both training and test sets till it reaches epoch 25. Then accuracy flattens for the test set whilst the training keeps improving. Overfitting it's taking place since we see the training accuracy keeps improving whilst the test data decreases in accuracy. More on this in the next chapter.8. Early stopping
Early stopping a model can solve the overfitting problem. Since it stops its training when it no longer improves. This is extremely useful since deep neural models can take a long time to train and we don't know beforehand how many epochs will be needed. Early stopping, like other keras callbacks can be imported from tensorflow.keras.callbacks. We then need to instantiate it. The early stopping callback can monitor several metrics, like validation accuracy, validation loss, etc. These can be specified in the monitor parameter. It's also important to define a patience argument, that is the number of epochs to wait for the model to improve before stopping it's training. There's no rules to decide which patience number works best at all times,this depends on the implementation. It's good to avoid low values, that way your model has a chance to improve at a later epoch.The callback is passed as a list to the callbacks parameter in the model fit method.9. Model checkpoint
The model checkpoint callback can also be imported from keras callbacks. This callback allows us to save our model as it trains. We specify the model filename with a name and the .hdf5 extension. You can also decide what to monitor to determine which model is best with the monitor parameter, by default validation loss is monitored. Setting the save_best_only parameter to True guarantees that the latest best model according to the quantity monitored will not be overwritten.10. Let's practice!
Let's apply these callbacks to make our training smarter!Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.