datacamp-logo

Inspecting cache in the Spark UI

A dataframe partitioned_df is available. It is used to register a temporary table called text. text is then cached using spark.catalog.cacheTable('text'). The Spark UI is available at http://localhost:4040/storage/. It shows the following once the cache for text is loaded:

Spark UI Storage

This shows that a table called text having seven partitions is cached in memory. Which of the following would immediately cause the above to appear in Spark UI?

  1. Performing a transform on the underlying dataframe, for example df = partitioned_df.distinct().

  2. Counting the underlying dataframe, for example: partitioned_df.count()

  3. Querying the table using, say: spark.sql("select count(*) from text")

  4. Querying and showing the result, say: spark.sql("select count(*) from text").show()

Answer the question
50 XP
Possible Answers
  • press
  • press
  • press
  • press
  • press
  • press
  • press