1. Learn
    2. /
  2. Courses
    3. /
  3. Feature Engineering with PySpark

Exercise

Using Corr()

The old adage 'Correlation does not imply Causation' is a cautionary tale. However, correlation does give us a good nudge to know where to start looking promising features to use in our models. Use this exercise to get a feel for searching through your data for the first time, trying to find patterns.

A list called columns containing column names has been created for you. In this exercise you will compute the correlation between those columns and 'SALESCLOSEPRICE', and find the maximum.

Instructions

100 XP
  • Use a for loop iterate through the columns.
  • In each loop cycle, compute the correlation between the current column and 'SALESCLOSEPRICE' using the corr() method.
  • Create logic to update the maximum observed correlation and with which column.
  • Print out the name of the column that has the maximum correlation with 'SALESCLOSEPRICE'.