1. Learn
    2. /
  2. Courses
    3. /
  3. Designing Forecasting Pipelines for Production
Connected

Exercise

Evaluating forecast performance

In this exercise, you'll evaluate and visualize the performance of the forecast model you built in the previous exercise.

The test dataset, ml_forecast results, and plot_series are preloaded, along with evaluation functions (mape, rmse, coverage, shown below) and pandas as pd.

def mape(y, yhat):
    mape = mean(abs(y - yhat)/ y) 
    return mape

def rmse(y, yhat):
    rmse = (mean((y - yhat) ** 2 )) ** 0.5
    return rmse

def coverage(y, lower, upper):
    coverage = sum((y <= upper) & (y >= lower)) / len(y)
    return coverage

Let's first assess the model's performance and then visualize the forecast.

Instructions

100 XP
  • Create fc by merging the ml_forecast and test datasets.
  • Calculate rmse using the rmse() custom function provided, passing fc["y"] and fc[model] to the two arguments in that order; storing as a variable called r within the for loop.
  • Complete the evaluation by sorting fc_performance by rmse in ascending order.