Testing your NLP credentials with n-grams

You're well on your way to NLP superiority. Let's test your mastery of n-grams!

In the workspace, we have the loaded a python list, one_grams, which contains all 1-grams of the string petro-vend fuel and fluids, tokenized on punctuation. Specifically,

one_grams = ['petro', 'vend', 'fuel', 'and', 'fluids']

In this exercise, your job is to determine the sum of the sizes of 1-grams, 2-grams and 3-grams generated by the string petro-vend fuel and fluids, tokenized on punctuation.

Recall that the n-gram of a sequence consists of all ordered subsequences of length n.

Exercise

Testing your NLP credentials with n-grams

Instructions

Possible Answers