Get startedGet started for free

Inspecting data in PySpark DataFrame

Inspecting data is very crucial before performing analysis such as plotting, modeling, training etc. In this simple exercise, you'll inspect the data in the people_df DataFrame that you have created in the previous exercise using basic DataFrame operators.

Remember, you already have a SparkSession spark and a DataFrame people_df available in your workspace.

This exercise is part of the course

Big Data Fundamentals with PySpark

View Course

Exercise instructions

  • Print the first 10 observations in the people_df DataFrame.
  • Count the number of rows in the people_df DataFrame.
  • How many columns does people_df DataFrame have and what are their names?

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

# Print the first 10 observations 
people_df.____(10)

# Count the number of rows 
print("There are {} rows in the people_df DataFrame.".format(people_df.____()))

# Count the number of columns and print their names
print("There are {} columns in the people_df DataFrame and their names are {}".format(len(people_df.____), people_df.____))
Edit and Run Code