Calculating a threshold and moving averages
You'll track forecast accuracy over time to detect model drift. By calculating rolling windows and defining a threshold level for triggering drift alerts, you can identify when the model becomes misaligned with reality and requires retraining.
You'll use the first 14 forecasts from fc_log_test to establish the threshold, then apply it to the remaining forecast logs. The forecast logs fc_log_test and fc_log containing model performance scores have been preloaded, along with pandas as pd.
이 연습은 강의의 일부입니다
Designing Forecasting Pipelines for Production
연습 안내
- Define the threshold level from
fc_log_testby adding three standard deviations to the RMSE mean, storing asrmse_threshold. - Calculate the RMSE moving average using 7-day and 14-day rolling windows for
fc_log.
실습형 인터랙티브 연습
이 예제를 이 샘플 코드를 완성하여 풀어보세요.
# Set threshold: mean + 3 standard deviations
rmse_threshold = fc_log_test["____"].mean() + 3 * fc_log_test["____"].____()
# Create rolling window averages for RMSE
fc_log["rmse_ma_7"] = fc_log["rmse"].rolling(window=____).____()
fc_log["rmse_ma_14"] = fc_log["rmse"].rolling(window=____).____()
print(f"RMSE threshold: {round(rmse_threshold, 2)}")
print()
print("Forecast log with rolling averages:")
print(fc_log[["forecast_start", "rmse", "rmse_ma_7", "rmse_ma_14"]].head(20))