In this chapter, you will explore what feature engineering is and how to get started with applying it to real-world data. You will load, explore and visualize a survey response dataset, and in doing so you will learn about its underlying data types and why they have an influence on how you should engineer your features. Using the pandas package you will create new features from both categorical and continuous columns.

Why generate features?

Getting to know your data

Selecting specific data types

Dealing with categorical features

One-hot encoding and dummy variables

Dealing with uncommon categories

Numeric variables

Binarizing columns

Binning values

Creating Features

This chapter introduces you to the reality of messy and incomplete data. You will learn how to find where your data has missing values and explore multiple approaches on how to deal with them. You will also use string manipulation techniques to deal with unwanted characters in your dataset.

Perché esistono i valori mancanti?

Quanto è sparso il mio insieme di dati?

Individuare i valori mancanti

Gestire i valori mancanti (I)

Eliminazione per lista (listwise deletion)

Sostituire i valori mancanti con costanti

Gestire i valori mancanti (II)

Compilare i valori mancanti continui

Imputazione dei valori nei modelli predittivi

Gestire altri problemi nei dati

Gestire i caratteri indesiderati (I)

Gestire i caratteri fuori posto (II)

Chaining dei metodi

Dealing with Messy Data

In this chapter, you will focus on analyzing the underlying distribution of your data and whether it will impact your machine learning pipeline. You will learn how to deal with skewed data and situations where outliers may be negatively impacting your analysis.

Data distributions

What does your data look like? (I)

What does your data look like? (II)

When don't you have to transform your data?

Scaling and transformations

Normalization

Standardization

Log transformation

When can you use normalization?

Removing outliers

Percentage based outlier removal

Statistical outlier removal

Scaling and transforming new data

Train and testing transformations (I)

Train and testing transformations (II)

Conforming to Statistical Assumptions

Finally, in this chapter, you will work with unstructured text data, understanding ways in which you can engineer columnar features out of a text corpus. You will compare how different approaches may impact how much context is being extracted from a text, and how to balance the need for context, without too many features being created.

Encoding text

Cleaning up your text

High level text features

Word counts

Counting words (I)

Counting words (II)

Limiting your features

Text to DataFrame

Term frequency-inverse document frequency

Tf-idf

Inspecting Tf-idf values

Transforming unseen data

N-grams

Using longer n-grams

Finding the most common words

Wrap-up

Dealing with Text Data

Stack Overflow Survey Responses (Modified)

US Presidential Inauguration Addresses

Ogni giorno si leggono notizie sui sorprendenti progressi con cui le applicazioni più recenti di Machine Learning stanno cambiando il mondo. Spesso però si sorvola sul fatto che, prima di usare questi modelli avanzati, serve un grande lavoro di pulizia dei dati e di feature engineering. In questo corso imparerai proprio questo. Lavorerai con il sondaggio degli sviluppatori di Stack Overflow e con i discorsi storici di insediamento dei presidenti degli Stati Uniti per capire come preprocessare al meglio e progettare feature a partire da dati categorici, continui e non strutturati. Questo corso ti offrirà esperienza pratica su come preparare qualsiasi dato per i tuoi modelli di Machine Learning.

Supervised Learning with scikit-learn

Impara a preparare dati per modelli di machine learning esplorando la pre-elaborazione e l'ingegneria.

Feature Engineering per il Machine Learning in Python

Crea nuove funzionalità per migliorare le prestazioni dei tuoi modelli di Machine Learning. 

Perché esistono i valori mancanti?

Create Your Free Account