Backtesting with MAE, MSE
In this exercise, you will practice how to evaluate model performance by conducting backtesting. The out-of-sample forecast accuracy is assessed by calculating MSE and MAE.
You can easily estimate prediction errors MSE and MAE with pre-defined functions in the sklearn.metrics package. The actual variance and predicted variance have been preloaded in actual_var and forecast_var respectively.
Diese Übung ist Teil des Kurses
GARCH Models in Python
Anleitung zur Übung
- In
evaluate(), perform the MAE calculation by calling the corresponding function fromsklean.metrics. - In
evaluate(), perform the MSE calculation by calling the corresponding function fromsklean.metrics. - Pass variables to
evaluate()in order to perform the backtest.
Interaktive Übung
Versuche dich an dieser Übung, indem du diesen Beispielcode vervollständigst.
def evaluate(observation, forecast):
# Call sklearn function to calculate MAE
mae = ____(observation, forecast)
print('Mean Absolute Error (MAE): {:.3g}'.format(mae))
# Call sklearn function to calculate MSE
mse = ____(observation, forecast)
print('Mean Squared Error (MSE): {:.3g}'.format(mse))
return mae, mse
# Backtest model with MAE, MSE
evaluate(____, ____)