1. Apprendre
    2. /
  2. Cours
    3. /
  3. Feature Engineering with PySpark
  • 1

    Exploratory Data Analysis

    Gratuit

    Get to know a bit about your problem before you dive in! Then learn how to statistically and visually inspect your dataset!

  • 2

    Wrangling with Spark Functions

    Real data is rarely clean and ready for analysis. In this chapter learn to remove unneeded information, handle missing values and add additional data to your analysis.

  • 3

    Feature Engineering

    In this chapter learn how to create new features for your machine learning model to learn from. We'll look at generating them by combining fields, extracting values from messy columns or encoding them for better results.

  • 4

    Building a Model

    In this chapter we'll learn how to choose which type of model we want. Then we will learn how to apply our data to the model and evaluate it. Lastly, we'll learn how to interpret the results and save the model for later!

Initializing

Exercice

Scaling your scalers

In the previous exercise, we minmax scaled a single variable. Suppose you have a LOT of variables to scale, you don't want hundreds of lines to code for each. Let's expand on the previous exercise and make it a function.

Instructions

100 XP
  • Define a function called min_max_scaler that takes parameters df a dataframe and cols_to_scale the list of columns to scale.
  • Use a for loop to iterate through each column in the list and minmax scale them.
  • Return the dataframe df with the new columns added.
  • Apply the function min_max_scaler() on df and the list of columns cols_to_scale.