Module 2 Quiz: Design and transformations — Question 4

You are building a new pipeline using Serverless for Apache Spark. The source data is highly structured, with well-defined columns like userid and purchaseamount. The primary task involves complex filtering and calculations on these columns. According to modern Spark best practices, which core API should you use to represent and manipulate this data?