Backtesting using ugarchroll

1. Use only the data that were available at the time of prediction

A realistic prediction uses only the data that were available at the time of prediction.

2. Volatility prediction by applying `sigma()` to `ugarchforecast` object

You respect this condition when using ugarchforecast to predict volatility.

3. Volatility estimation by applying `sigma()` to `ugarchfit` object (i)

But, when you apply the sigma method to a ugarchfit object, then you use all the past, present and future returns in the sample to obtain estimates of past volatility.

4. Volatility estimation by applying sigma() to ugarchfit object (ii)

Those volatility estimates have a look ahead bias, since they use information that you do not have at the time of prediction.

5. Solution to avoid look-ahead bias: Rolling estimation

You can avoid the look ahead bias by running a loop across the prediction times and use both ugarchfit and ugarchforecast to make the volatility forecast. A computationally more convenient approach is to use the ugarchroll function. The function lets you choose the length of the estimation sample and how often to re-estimate the model.

6. Expanding window estimation

You could take an expanding window for estimation. You then thus use all the returns available from inception till the time of prediction.

7. Moving window estimation

It is also possible to use only a fixed number of the most recent returns. For each new prediction, you then add the new returns to the sample and discard the most distant ones.

8. Properties of rolling window estimation

The rolling window approach has the advantage of that the parameter estimates automatically adapting to new values of the parameters. To reduce the computational cost of re-estimating the GARCH model at each prediction time, can specify to re-estimate the model at a lower frequency.

9. Rolling and re-estimation

It then follows that in between two model estimations, you use the previously estimated model to make the volatility forecast.

10. Function ugarchroll

The implementation in R requires to specify five arguments for the ugarchroll function, namely the GARCH model specification, the return data, the size of the estimation sample, whether a moving or expanding window is used and how often the model should be re-estimated. Here take an initial sample of 2500 observations and refit every 500 observations.

11. Example on the Jan 1999-Dec 2018 daily EUR/USD returns

For the daily EUR/USD returns used, this leads to a total of five estimation samples.

12. Output of changing parameters

The list with estimated parameters for each sample is obtained by applying the coef function to the ugarchroll output variable.

13. Changes between estimation windows

Here you see the estimates for the first and fifth estimation. Note how much the estimated parameters have changed over time. The alpha parameters was 0-point-028 and it is now 0-point-041. This means that the GARCH process has become more responsive to shocks.

14. What is the rolling predicted mean and volatility?

ugarchroll also computes the forecasted mean and volatility. In case of the EUR/USD returns with initial sample of 2500 observation, the first prediction is available for observation 2501, namely December 9, 2008.

15. Method as.data.frame()

Those predictions are available in a data frame. The column with the title Mu is the time series of predicted mean for each day. The column with title Sigma shows the predicted volatility.

16. Predicted volatilities

Here you see the time series plot with forecasted volatilities.

17. Accuracy of rolling predictions

A final question is how accurate the mean and volatility predictions are. For this, you need to compare the predicted values in the first two columns with the actual return in the last column.

18. Mean squared prediction error for the mean

If you take the difference between the actual return and the predicted mean, you get the prediction error for the mean. The accuracy can then be quantified through the mean squared prediction error.

19. Mean squared prediction error for the variance

For the variance, the prediction error is the difference between the squared error in the mean and the predicted variance. To interpret the mean squared prediction error you need to compare it with another model.

20. Compare two models

As an example, let's compare the forecasting accuracy of a standard GARCH model with student t distribution with a GJR GARCH model skewed student t distribution.

21. Comparison of prediction accuracny

You can see here that the GJR GARCH model with skewed student t distribution has a lower mean squared error and is therefore better.

22. The proof of the pudding is in the eating

The proof of the pudding is in the eating. You can use the ugarchroll function to backtest the actual forecasting accuracy of a GARCH model.

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.