Comparing auto.arima() and ets() on non-seasonal data
The AICc statistic is useful for selecting between models in the same class. For example, you can use it to select an ETS model or to select an ARIMA model. However, you cannot use it to compare ETS and ARIMA models because they are in different model classes.
Instead, you can use time series cross-validation to compare an ARIMA model and an ETS model on the austa data. Because tsCV() requires functions that return forecast objects, you will set up some simple functions that fit the models and return the forecasts. The arguments of tsCV() are a time series, forecast function, and forecast horizon h. Examine this code snippet from the second chapter:
e <- matrix(NA_real_, nrow = 1000, ncol = 8)
for (h in 1:8)
e[, h] <- tsCV(goog, naive, h = h)
...
Furthermore, recall that pipe operators in R take the value of whatever is on the left and pass it as an argument to whatever is on the right, step by step, from left to right. Here's an example based on code you saw in an earlier chapter:
# Plot 20-year forecasts of the lynx series modeled by ets()
lynx %>% ets() %>% forecast(h = 20) %>% autoplot()
In this exercise, you will compare the MSE of two forecast functions applied to austa, and plot forecasts of the function that computes the best forecasts. Once again, austa has been loaded into your workspace.
This exercise is part of the course
Forecasting in R
Exercise instructions
- Fill in the
farima()function to forecast the results ofauto.arima(). Follow the structure of the pre-written code infets()that does the same forets(). - Compute cross-validated errors for ETS models on
austausingtsCV()with one-step errors, and save this toe1. - Compute cross-validated errors for ARIMA models on
austausingtsCV()with one-step errors, and save this toe2. - Compute the cross-validated MSE for each model class and remove missing values. Refer to the previous chapter if you cannot remember how to calculate MSE.
- Produce and plot 10-year forecasts of future values of
austausing the best model class.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Set up forecast functions for ETS and ARIMA models
fets <- function(x, h) {
forecast(ets(x), h = h)
}
farima <- function(x, h) {
forecast(___, ___)
}
# Compute CV errors for ETS on austa as e1
e1 <- tsCV(austa, ___, ___)
# Compute CV errors for ARIMA on austa as e2
e2 <- tsCV(___, ___, ___)
# Find MSE of each model class
mean(___, ___)
mean(___, ___)
# Plot 10-year forecasts using the best model class
austa %>% ___ %>% ____