Get startedGet started for free

Flight duration model: Pipeline model

You're now ready to put those stages together in a pipeline.

You'll construct the pipeline and then train the pipeline on the training data. This will apply each of the individual stages in the pipeline to the training data in turn. None of the stages will be exposed to the testing data at all: there will be no leakage!

Once the entire pipeline has been trained it will then be used to make predictions on the testing data.

The data are available as flights, which has been randomly split into flights_train and flights_test.

This exercise is part of the course

Machine Learning with PySpark

View Course

Exercise instructions

  • Import the class for creating a pipeline.
  • Create a pipeline object and specify the indexer, onehot, assembler and regression stages, in this order.
  • Train the pipeline on the training data.
  • Make predictions on the testing data.

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

# Import class for creating a pipeline
from pyspark.____ import ____

# Construct a pipeline
pipeline = ____(____=[____])

# Train the pipeline on the training data
pipeline = pipeline.____(____)

# Make predictions on the testing data
predictions = ____.____(____)
Edit and Run Code