Analyzing dimensionality and preprocessing

In this exercise, you have been provided with a lem_corpus which contains the pre-processed versions of the movie taglines from the previous exercise. In other words, the taglines have been lowercased and lemmatized, and stopwords have been removed.

Your job is to generate the bag of words representation bow_lem_matrix for these lemmatized taglines and compare its shape with that of bow_matrix obtained in the previous exercise. The first five lemmatized taglines in lem_corpus have been printed to the console for you to examine.

Este exercício faz parte do curso

Feature Engineering for NLP in Python

Ver curso

Instruções do exercício

Import the CountVectorizer class from sklearn.
Instantiate a CountVectorizer object. Name it vectorizer.
Using fit_transform(), generate bow_lem_matrix for lem_corpus.

Exercício interativo prático

Experimente este exercício completando este código de exemplo.

# Import CountVectorizer
from sklearn.feature_extraction.text import ____

# Create CountVectorizer object
____ = ____

# Generate matrix of word vectors
bow_lem_matrix = ____.____(lem_corpus)

# Print the shape of bow_lem_matrix
print(bow_lem_matrix.shape)

Editar e executar o código