Get startedGet started for free

Comparing estimated and realized performance

Now that you have seen how performance calculation works, your task is to calculate the realized performance for our tip prediction model for the NYC green taxi dataset.

The reference and analysis set is already loaded and saved in the reference and analysis variables.

In addition, results from the DLE algorithm for tip prediction are stored in the estimated_results variable.

This exercise is part of the course

Monitoring Machine Learning in Python

View Course

Exercise instructions

  • Specify problem type as regression in calculator initialization.
  • Fit the calculator with reference data and calculate performance for the analysis set.
  • Show comparison plot between realized_results and estimated_results using compare() method.

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

# Intialize the calculator
calculator = nannyml.PerformanceCalculator(
    y_true='tip_amount',
    y_pred='y_pred',
    chunk_period='d',
  	metrics=['mae'],
    timestamp_column_name='lpep_pickup_datetime',
    problem_type=____)

# Fit the calculator
calculator.fit(____)
realized_results = calculator.____(____)

# Show comparison plot for realized and estimated performance
____.____(____).plot().show()
Edit and Run Code