Goodness of fit
1. Do the GARCH predictions fit will with the observed returns?
A reliable GARCH model makes predictions that fit well with the observed returns.2. Evaluation criterion
To evaluate the accuracy of the GARCH model, you need to choose which part of the prediction you want to evaluate: is it the mean, the variance, or the full distribution?3. 1) Goodness of fit for the mean prediction
Let's start with the mean prediction. It's the predicted return at time t given the information available at time t minus 1. The prediction error is the difference between the actual and predicted return. You achieve a good fit when the mean squared prediction error is small. In R, you can compute the prediction errors by applying the residuals method to the GARCH estimation output. By taking the mean of the square of these errors, you obtain the mean squared prediction error.4. 2) Goodness of fit for the variance prediction
The variance is the predicted squared deviation of the return from its mean. This is the same as the expected squared prediction error. The estimation error for the variance is the difference between the squared error in the returns and the predicted variance. A variance prediction is accurate if the mean squared prediction error is small. The calculation in R is again straightforward using the residuals method.5. Example for EUR/USD returns
Here you see the complete analysis in case of daily EUR/USD returns. First the model is specified and estimated. Then the mean squared prediction errors are computed for the mean prediction, and for the variance prediction.6. 3) Goodness of fit for the distribution
Third, we need to evaluate the quality of the GARCH prediction about the density of each return. The higher the density, the more likely it is to observe the return and thus the better the model fits with the data. The likelihood of the complete sample of returns uses the densities to measures how likely it is that the observed returns come from the estimated GARCH model. The higher this likelihood, the better the model fits with your data.7. Example for EUR/USD returns
You can compute it by applying the likelihood method to the GARCH estimation output. You should not interpret the value itself, but compare it with the likelihood obtained using other models, like a GJR model. This model has a slightly higher likelihood and thus seems better.8. Risk of overfitting
The caveat is that we use an in-sample evaluation approach where the estimation sample and evaluation sample coincides. This approach has the risk of overfitting and thus choosing a complex model that has a good fit in the estimation sample but is not accurate for predicting the returns outside of the sample.9. Solution: Balance goodness of fit with a penalty for the complexity
To avoid overfitting you need to balance the likelihood with a penalty for the complexity of the model. This leads to choosing a model that is parsimonious: it has a high likelihood and uses a low number of parameters.10. Information criteria
You can measure the parsimony of a model using information criteria which equal minus the likelihood plus the penalty for the number of parameters. Good models have low information criteria.11. Results information criteria
The method infocriteria computes four types of information criteria. They have different penalties. To interpret them you need to compare them with the information criteria of other models.12. Illustration on the EUR/USD returns
As an example we estimate a simple and complex GARCH model for the daily EUR/USD returns. The simple model has 5 parameters, the complex one has 8 parameters. The more complex one has only a slightly higher likelihood.13. Which model is most parsimonious for EUR/USD returns?
From the information criteria it is clear that the simple model is most parsimonious for the EUR/USD returns: it has the lowest value for all information criteria and is thus the winner.14. Result is case-specific: case of MSFT returns
However, if we do the same for the Microsoft returns, we find that the complex model wins.15. KISS: Keep it Sophisticatedly Simple
Let's kiss by keeping it sophisticatedly simple when choosing a parsimonious GARCH model.Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.