Logistic regression and feature selection
In this exercise we'll perform feature selection on the movie review sentiment data set using L1 regularization. The features and targets are already loaded for you in X_train and y_train.
We'll search for the best value of C using scikit-learn's GridSearchCV(), which was covered in the prerequisite course.
This exercise is part of the course
Linear Classifiers in Python
Exercise instructions
- Instantiate a logistic regression object that uses L1 regularization.
- Find the value of
Cthat minimizes cross-validation error. - Print out the number of selected features for this value of
C.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Specify L1 regularization
lr = LogisticRegression(solver='liblinear', ____)
# Instantiate the GridSearchCV object and run the search
searcher = GridSearchCV(lr, {'C':[0.001, 0.01, 0.1, 1, 10]})
searcher.fit(X_train, y_train)
# Report the best parameters
print("Best CV params", searcher.best_params_)
# Find the number of nonzero coefficients (selected features)
best_lr = searcher.best_estimator_
coefs = best_lr.____
print("Total number of features:", coefs.size)
print("Number of selected features:", np.count_nonzero(coefs))