The training loop

1. The training loop

While some other libraries give you one method that takes care of training a model, spaCy gives you full control over the training loop.

2. The steps of a training loop

The training loop is a series of steps that's performed to train or update a model. We usually need to perform it several times, for multiple iterations, so that the model can learn from it effectively. If we want to train for 10 iterations, we need to loop 10 times. To prevent the model from getting stuck in a suboptimal solution, we randomly shuffle the data for each iteration. This is a very common strategy when doing stochastic gradient descent. Next, we divide the training data into batches of several examples, also known as minibatching. This makes it easier to make a more accurate estimate of the gradient. Finally, we update the model for each batch, and start the loop again until we've reached the last iteration. We can then save the model to a directory and use it in spaCy.

3. Recap: How training works

To recap: The training data are the examples we want to update the model with. The text should be a sentence, paragraph or longer document. For the best results, it should be similar to what the model will see at runtime. The label is what we want the model to predict. This can be a text category, or an entity span and its type. The gradient is how we should change the model to reduce the current error. It's computed when we compare the predicted label to the true label.

4. Example loop

Here's an example. Let's imagine we have a list of training examples consisting of texts and entity annotations. We want to loop for 10 iterations, so we're iterating over a range of 10. Next, we use the random module to randomly shuffle the training data. We then use spaCy's minibatch utility function to divide the examples into batches. For each batch, we get the texts and annotations and call the nlp dot update method to update the model. Finally, we call the nlp dot to disk method to save the trained model to a directory.

5. Updating an existing model

spaCy lets you update an existing pre-trained model with more data – for example, to improve its predictions on different texts. This is especially useful if you want to improve categories the model already knows, like "person" or "organization". You can also update a model to add new categories. Just make sure to always update it with examples of the new category and examples of the other categories it previously predicted correctly. Otherwise improving the new category might hurt the other categories.

6. Setting up a new pipeline from scratch

In this example, we start off with a blank English model using the spacy dot blank method. The blank model doesn't have any pipeline components, only the language data and tokenization rules. We then create a blank entity recognizer and add it to the pipeline. Using the "add label" method, we can add new string labels to the model. We can now call nlp dot begin training to initialize the model with random weights. To get better accuracy, we want to loop over the examples more than once and randomly shuffle the data on each iteration. On each iteration, we divide the examples into batches using spaCy's minibatch utility function. Each example consists of a text and its annotations. Finally, we update the model with the texts and annotations and continue the loop.

7. Let's practice!

Time to practice! Now that you've seen the training loop, let's use the data created in the previous exercise to update a model.

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.