Data Filtering with PySpark

In this exercise, we'll upload a file into a lakehouse and use PySpark to filter data. The file has a list of video games and their publishers. We want to reduce the dataset so we deal only with video games from the publisher Atari.

Note: If you get an error about an API rate limit, this usually happens because a previous Fabric task has not completed. You can view active Fabric tasks from the Monitor page (located on the left vertical menu). Canceling old Fabric tasks in the Monitor page will usually resolve the rate limit error.