Unique 5-tuples in sorted order
A previous lesson taught an operation that eliminates duplicates, fetching unique records. In a previous exercise you obtained common 5-tuples. We will combine these two capabilities to find the unique 5-tuples, sorted alphabetically in descending order.
The table text contains the first four chapters of the Sherlock Holmes text. It has the following columns: word, id, and part.
This exercise is part of the course
Introduction to Spark SQL in Python
Exercise instructions
- Retrieve the last ten unique 5-tuples sorted alphabetically in descending order.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Unique 5-tuples sorted in descending order
query = """
SELECT ____ w1, w2, w3, w4, w5 FROM (
SELECT word AS w1,
____(word,____) OVER(PARTITION BY ____ ORDER BY ____ ) AS w2,
____(word,____) OVER(PARTITION BY ____ ORDER BY ____ ) AS w3,
____(word,____) OVER(PARTITION BY ____ ORDER BY ____ ) AS w4,
____(word,____) OVER(PARTITION BY ____ ORDER BY ____ ) AS w5
FROM text
)
ORDER BY w1 DESC, w2 DESC, ____ DESC, w4 ____, ____ ____
LIMIT 10
"""
df = spark.sql(query)
df.show()