Exercise

Infer and filter

Imagine you have a census dataset that you know has a header and a schema. Let's load that dataset and let PySpark infer the schema. What do you see if you filter on adults over 40?

Remember, there's already a SparkSession called spark in your workspace!

Instructions

100 XP

Load a JSON file adults.json.
Filter the data to include adults over the age of 40.
Show the results.

.css-6su6fj{-webkit-flex-shrink:0;-ms-flex-negative:0;flex-shrink:0;}Exercise

Instructions

Exercise