CommencerCommencer gratuitement

Infer and filter

Imagine you have a census dataset that you know has a header and a schema. Let's load that dataset and let PySpark infer the schema. What do you see if you filter on adults over 40?

Remember, there's already a SparkSession called spark in your workspace!

Cet exercice fait partie du cours

Introduction to PySpark

Afficher le cours

Instructions

  • Load a JSON file adults.json.
  • Filter the data to include adults over the age of 40.
  • Show the results.

Exercice interactif pratique

Essayez cet exercice en complétant cet exemple de code.

# Load the dataframe
census_df = spark.read.json("adults.json")

# Filter rows based on age condition
salary_filtered_census = census_df.____(census_df[____]____)

# Show the result
____
Modifier et exécuter le code