Exercise

# Filtering out highly correlated features

You're going to automate the removal of highly correlated features in the numeric ANSUR dataset. You'll calculate the correlation matrix and filter out columns that have a correlation coefficient of more than 0.95 or less than -0.95.

Since each correlation coefficient occurs twice in the matrix (correlation of A to B equals correlation of B to A) you'll want to ignore half of the correlation matrix so that only one of the two correlated features is removed. Use a mask trick for this purpose.

Instructions

**100 XP**

- Calculate the correlation matrix of
`ansur_df`

and take the absolute value of this matrix. - Create a boolean mask with
`True`

values in the upper right triangle and apply it to the correlation matrix. - Set the correlation coefficient threshold to
`0.95`

. - Drop all the columns listed in
`to_drop`

from the dataframe.