TfIdf on Twitter airline sentiment data

You will now build features using the TfIdf method. You will continue to work with the tweets dataset.

In this exercise, you will utilize what you have learned in previous lessons and remove stop words, use a token pattern and specify the n-grams.

The final output will be a DataFrame, of which the columns are created using the TfidfVectorizer(). Such a DataFrame can directly be passed to a supervised learning model, which is what we will tackle in the next chapter.

Import the required package to build a TfidfVectorizer and the default list of English stop words, ENGLISH_STOP_WORDS.
Build a TfIdf vectorizer from the text column of the tweets dataset, specifying uni- and bi-grams as a choice of n-grams,tokens which include only alphanumeric characters, and the stop words should correspond to the ENGLISH_STOP_WORDS.
Transform the vectorizer.
Specify the column names in the DataFrame() function.

Exercise

TfIdf on Twitter airline sentiment data

Instructions