Natural Language Processing

1. Natural Language Processing

Earlier in this chapter, we mentioned that deep learning is especially well-suited for working with images and text. Now, we'll focus on text data and how it is leveraged in Natural Language Processing.

2. Natural Language Processing (NLP)

Natural Language Processing, or NLP, is the ability for computers to understand the meaning of human language. On the slide you can see how through NLP the computer is able to locate and classify named entities in the text into pre-defined categories such as the names of persons and locations.

3. Bag of words

Previously, the features for our machine learning problems have been numbers or categories. What do we do when our data is text? A simple option is to count the number of times important words appear in each piece of text. This technique is called bag of words.

4. Bag of words

Suppose we wanted to analyze the following two sentences: "U2 is a great band" and "Queen is a great band". We might end up with the word counts shown in the table.

5. Bag of words: n-grams

Let's now consider the following sentence. When counting the individual words, "great" gets added to the list even though the sentiment towards the book is actually the opposite. This can be solved by counting sequences of words, a technique called n-grams. Here we are counting two-word sequences, allowing us to capture more information. Bag of words is a useful technique that is commonly used in NLP. There are some limitations but for how simple it is, it yields some pretty impressive results.

6. Bag of words: limitations

A limitation of bag of words is that word counts don't help us consider synonyms. For example, there are many words that all mean "blue", such as "sky-blue", "aqua" and "cerulean". Ideally, we would like to group these as a single feature.

7. Word embeddings

One solution to these problems is Word Embeddings. It is a special way of creating features that group together similar words. Word embeddings would create similar features for various shades of blue. Word embeddings have another interesting property: they are mathematical representations of words that obey intuitive rules. For example, in word embeddings, if we take the features for "King", subtract the features for "man", and add the features for "woman", we get a set of features that are very close to those of "queen".

8. Language translation

After mapping words or sentences to numbers, with the bag of words technique or using word embeddings, we can pass them to a neural network whose job it is to translate the input sentence to a different language. Here you can see the dutch sentence, "met of zonder jou", being translated to "with or without you".

9. Applications

Natural language processing is the driving force behind the following common applications: language translation applications such as Google Translate, chatbots used by companies to message customers, personal assistant applications such as Siri and Alexa, sentiment analysis, which can be used to quantify how positive or negative the emotion expressed by a segment of text is, and many more.

10. Deep learning

We just covered two special fields of machine learning: computer vision and natural language processing. So why is deep learning preferred when working with image and text data? First of all, these problems are very complex and neural networks are much more efficient than traditional machine learning algorithms. Second, it is often unclear what the features of the model should be for text and image data. Deep learning doesn't require human intervention and can learn the features on its own, like which pixels make up a nose. Lastly, when working with these data types you have many data points. Even a single image can consist of millions of pixels and a body of text can contain millions of words. Traditional machine learning algorithms fail to improve a whole lot when increasing the amount of data. With deep learning, however, as the data increases the performance of the model also increases.

11. Let's practice!

Time for exercises!

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.