Forecast evaluation & experimentation

In this exercise, you'll evaluate the forecast model's performance to explore the use cases of experimentation.

The merged forecast (fc), combining predictions with actual test results, is preloaded. Evaluation functions (mape, rmse, coverage) and pandas (as pd) are also ready for use. Here's a quick reference for the functions:

def mape(y, yhat):
    mape = mean(abs(y - yhat) / y) 
    return mape

def rmse(y, yhat):
    rmse = (mean((y - yhat) ** 2)) ** 0.5
    return rmse

def coverage(y, lower, upper):
    coverage = sum((y <= upper) & (y >= lower)) / len(y)
    return coverage

First, compute performance metrics for the model. Then, answer a question about the goals of experimentation in forecasting.

本练习是课程的一部分

Designing Forecasting Pipelines for Production

查看课程

交互式实操练习

通过完成这段示例代码来试试这个练习。

performance_metrics = []

# Loop through models and calculate metrics
for model in ["LGBMRegressor", "XGBRegressor", "LinearRegression"]:
    performance_metrics.append({
        "model": model,
        "mape": ____(fc["y"], fc[model]),
        "rmse": ____(fc["y"], fc[____]),
        "coverage": ____(fc["y"], fc[f"{model}-lo-95"], fc[f"{model}-hi-95"])
    })

# Create DataFrame and sort by RMSE
fc_performance = pd.DataFrame(performance_metrics).sort_values("____")

print(fc_performance)

编辑并运行代码