1. Learn
    2. /
  2. Courses
    3. /
  3. Introduction to Natural Language Processing in R

Exercise

Text preprocessing: Stemming

The root of words are often more important than their endings, especially when it comes to text analysis. The book Animal Farm is obviously about animals. However, knowing that the book mentions animal's 248 times, and animal 107 times might not be helpful for your analysis.

tidy_animal_farm contains a tibble of the words from Animal Farm, tokenized and without stop words. The next step is to stem the words and explore the results.

Instructions

100 XP
  • Use dplyr and SnowballC to stem the words from tidy_animal_farm.
  • Print the old word frequencies from tidy_animal_farm.
  • Print the new word frequencies from stemmed_animal_farm.