Text preprocessing: remove stop words
Stop words are unavoidable in writing. However, to determine how similar two pieces of text are to each other are or when trying to find themes within text, stop words can make things difficult. In the book Animal Farm, the first chapter contains only 2,636 words, while almost 200 of them are the word "the".
Usually, "the" will not help us in text analysis projects. In this exercise you will remove the stop words from the first chapter of Animal Farm.
This exercise is part of the course
Introduction to Natural Language Processing in R
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Tokenize animal farm's text_column column
tidy_animal_farm <- animal_farm %>%
___(word, ___)