Exercise

Sentence and word tokenization

Tokenization is an important first step in NLP. It involves breaking text into smaller units called tokens, which is key to working with language data. Your task is to tokenize a snippet of a news article into both sentences and words.

Instructions 1/2

undefined XP

1

2

Import the nltk library.
Download the punkt_tab package.
Tokenize the text into sentences.

.css-6su6fj{-webkit-flex-shrink:0;-ms-flex-negative:0;flex-shrink:0;}Exercise

Instructions 1/2

Exercise