Get startedGet started for free

Preprocessing text

Building a recommendation system, or any model, requires text to be preprocessed first.

A block of text from Sherlock Holmes is loaded here. Preprocess this text using the various techniques presented in the video to prepare it for further analysis.

The text variable is an excerpt from The Hound of the Baskervilles by Arthur Conan Doyle.

The following packages and functions have been loaded for you: nltk, torch, get_tokenizer, PorterStemmer, stopwords.

This exercise is part of the course

Deep Learning for Text with PyTorch

View Course

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

# Initialize and tokenize the text
tokenizer = ____("basic_english")
tokens = ____(____)
print(tokens)
Edit and Run Code