Part 2: SQL Queries on DataFrame
The fifa_df DataFrame that we created has additional information about datatypes and names of columns associated with it. This additional information allows PySpark SQL to run SQL queries on DataFrame. SQL queries are concise and easy to run compared to DataFrame operations. But in order to apply SQL queries on DataFrame first, you need to create a temporary view of DataFrame as a table and then apply SQL queries on the created table (Running SQL Queries Programmatically).
In the second part, you'll create a temporary table of fifa_df DataFrame and run SQL queries to extract the 'Age' column of players from Germany.
You already have a SparkContext spark and fifa_df available in your workspace.
Questo esercizio fa parte del corso
Big Data Fundamentals with PySpark
Istruzioni dell'esercizio
- Create temporary table
fifa_df_tablefromfifa_dfDataFrame. - Construct a "query" to extract the "Age" column from Germany players in
fifa_df_table. - Apply the SQL "query" and create a new DataFrame
fifa_df_germany_age. - Computes basic statistics of the created DataFrame.
Esercizio pratico interattivo
Prova a risolvere questo esercizio completando il codice di esempio.
# Create a temporary view of fifa_df
fifa_df.____('fifa_df_table')
# Construct the "query"
query = '''SELECT ____ FROM ____ WHERE Nationality == "Germany"'''
# Apply the SQL "query"
fifa_df_germany_age = spark.____(____)
# Generate basic statistics
fifa_df_germany_age.____().show()