Get startedGet started for free

File size optimization

Consider if you're given 2 large data files on a cluster with 10 nodes. Each file contains 10M rows of roughly the same size. While working with your data, the responsiveness is acceptable but the initial read from the files takes a considerable period of time. Note that you are the only one who will use the data and it changes for each run.

Which of the following is the best option to improve performance?

This exercise is part of the course

Cleaning Data with PySpark

View Course

Hands-on interactive exercise

Turn theory into action with one of our interactive exercises

Start Exercise