Bringing it all together II
Create a DataFrame, apply transformations, cache it, and check if it’s cached. Then, uncache it to release memory.
For this exercise a spark session has been made for you! Look carefully at the outcome of the .explain() method to understand what the outcome is!
Bu egzersiz
Introduction to PySpark
kursunun bir parçasıdırEgzersiz talimatları
- Cache the
dfDataFrame. - Explain the processing of the
agg_resultDataFrame. - Unpersist the cached
dfDataFrame after processing.
Uygulamalı interaktif egzersiz
Bu örnek kodu tamamlayarak bu egzersizi bitirin.
# Cache the DataFrame
df.____
# Perform aggregation
agg_result = df.groupBy("Department").sum("Salary")
agg_result.show()
# Analyze the execution plan
agg_result.____
# Uncache the DataFrame
df.____