Exercise

Part 1: Exploring the dataset

You will now explore the dataset a little bit. You will first get a feel of what the data looks like. You will print some of the data and learn how to tokenize the sentences in the data to individual words. For the English language, tokenization appears to be a trivial task, however, there are languages such as Japanese, which are not as consistently delimited as English.

For this exercise, you have been provided with two datasets: en_text and fr_text. The en_text contains a list of English sentences, where the fr_text contains the corresponding list of French sentences.

Instructions

100 XP
  • Write a zip() function that iterates through the first 5 sentences of the English sentences (en_text) and French sentences (fr_text).
  • Get the first English sentence from en_text.
  • Tokenize the obtained sentence using the split() function and the space character and assign it to first_words.
  • Print the tokenized words.