Tokenization with spaCy
In this exercise, you'll practice tokenizing text. You'll use the first review from the Amazon Fine Food Reviews dataset for this exercise. You can access this review by using the text object provided.
The en_core_web_sm model is already loaded for you. You can access it by calling nlp(). You can use list comprehension to compile output lists.
This exercise is part of the course
Natural Language Processing with spaCy
Exercise instructions
- Store Doc container for the pre-loaded review in a
documentobject. - Store and review texts of all the tokens of the
documentin the variablefirst_text_tokens.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Create a Doc container of the given text
document = ____(____)
# Store and review the token text values of tokens for the Doc container
first_text_tokens = [____ for ____ in ____]
print("First text tokens:\n", first_text_tokens, "\n")