Creating a tibble from a corpus
To further explore the corpus on crude oil data that you received from a coworker, you have decided to create a pipeline to clean the text contained in the documents. Instead of exploring how to do this with the tm
package, you have decided to transform the corpus into a tibble so you can use the functions unnest_tokens()
, count()
, and anti_join()
that you are already familiar with. The corpus crude
contains both the metadata and the text of each document.
This exercise is part of the course
Introduction to Natural Language Processing in R
Exercise instructions
- Convert the corpus into a tibble.
- Use
names
to print out the column names. - Tokenize (by word), count, and remove stop words from the
text
column ofcrude_tibble
.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Create a tibble & Review
crude_tibble <- ___(crude)
___(crude_tibble)
crude_counts <- crude_tibble %>%
# Tokenize by word
___(___, text) %>%
# Count by word
___(word, sort = TRUE) %>%
# Remove stop words
___(stop_words)