partitioned_df is available. It is used to register a temporary table called
text is then cached using
spark.catalog.cacheTable('text'). The Spark UI is available at
http://localhost:4040/storage/. It shows the following once the cache for
text is loaded:
This shows that a table called
text having seven partitions is cached in memory. Which of the following would immediately cause the above to appear in Spark UI?
Performing a transform on the underlying dataframe, for example
df = partitioned_df.distinct().
Counting the underlying dataframe, for example:
Querying the table using, say:
spark.sql("select count(*) from text")
Querying and showing the result, say:
spark.sql("select count(*) from text").show()