1. Learn
    2. /
  2. Courses
    3. /
  3. Feature Engineering for NLP in Python
  • 1

    Basic features and readability scores

    Free

    Learn to compute basic features such as number of words, number of characters, average word length and number of special characters (such as Twitter hashtags and mentions). You will also learn to compute readability scores and determine the amount of education required to comprehend a piece of text.

  • 2

    Text preprocessing, POS tagging and NER

    In this chapter, you will learn about tokenization and lemmatization. You will then learn how to perform text cleaning, part-of-speech tagging, and named entity recognition using the spaCy library. Upon mastering these concepts, you will proceed to make the Gettysburg address machine-friendly, analyze noun usage in fake news, and identify people mentioned in a TechCrunch article.

  • 3

    N-Gram models

    Learn about n-gram modeling and use it to perform sentiment analysis on movie reviews.

  • 4

    TF-IDF and similarity scores

    Learn how to compute tf-idf weights and the cosine similarity score between two vectors. You will use these concepts to build a movie and a TED Talk recommender. Finally, you will also learn about word embeddings and using word vector representations, you will compute similarities between various Pink Floyd songs.

Initializing

Exercise

Noun usage in fake news

In this exercise, you have been given a dataframe headlines that contains news headlines that are either fake or real. Your task is to generate two new features num_propn and num_noun that represent the number of proper nouns and other nouns contained in the title feature of headlines.

Next, we will compute the mean number of proper nouns and other nouns used in fake and real news headlines and compare the values. If there is a remarkable difference, then there is a good chance that using the num_propn and num_noun features in fake news detectors will improve its performance.

To accomplish this task, the functions proper_nouns and nouns that you had built in the previous exercise have already been made available to you.

Instructions 1/2

undefined XP
  • 1
    • Create a new feature num_propn by applying proper_nouns to headlines['title'].
    • Filter headlines to compute the mean number of proper nouns in fake news using the mean method.
  • 2
    • Repeat the process for other nous: create a feature 'num_noun' using nouns and compute the mean of other nouns