Dropping a list of columns
Our data set is rich with a lot of features, but not all are valuable. We have many that are going to be hard to wrangle into anything useful. For now, let's remove any columns that aren't immediately useful by dropping them.
'STREETNUMBERNUMERIC': The postal address number on the home'FIREPLACES': Number of Fireplaces in the home'LOTSIZEDIMENSIONS': Free text describing the lot shape'LISTTYPE': Set list of values of sale type'ACRES': Numeric area of lot size
Este exercício faz parte do curso
Feature Engineering with PySpark
Instruções do exercício
- Read the list of column descriptions above and explore their top 30 values with
show(), the dataframe is already filtered to the listed columns asdf - Create a list of two columns to drop based on their lack of relevance to predicting house prices called
cols_to_drop. Recall that computers only interpret numbers explicitly and don't understand context. - Use the
drop()function to remove the columns in the listcols_to_dropfrom the dataframedf.
Exercício interativo prático
Experimente este exercício completando este código de exemplo.
# Show top 30 records
df.____(____)
# List of columns to remove from dataset
cols_to_drop = [____, ____]
# Drop columns in list
df = df.____(____)