1. Learn
    2. /
  2. Courses
    3. /
  3. Feature Engineering for Machine Learning in Python

Exercise

Log transformation

In the previous exercises you scaled the data linearly, which will not affect the data's shape. This works great if your data is normally distributed (or closely normally distributed), an assumption that a lot of machine learning models make. Sometimes you will work with data that closely conforms to normality, e.g the height or weight of a population. On the other hand, many variables in the real world do not follow this pattern e.g, wages or age of a population. In this exercise you will use a log transform on the ConvertedSalary column in the so_numeric_df DataFrame as it has a large amount of its data centered around the lower values, but contains very high values also. These distributions are said to have a long right tail.

Instructions

100 XP
  • Import PowerTransformer from sklearn's preprocessing module.
  • Instantiate the PowerTransformer() as pow_trans.
  • Fit the PowerTransformer on the ConvertedSalary column of so_numeric_df.
  • Transform the same column with the scaler you just fit.