ARIMA models: estimation and forecasting

Firmin Doko Tchatoka

[email protected]
https://www.adelaide.edu.au/directory/firmin.dokotchatoka
The University of Adelaide
August 6, 2021

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 1 / 23
 Estimation

Estimation steps for yt ∼ ARIMA (p,d,q)

I Identification:
- Identify the order of integration d: must run d + 1 unit root tests
- Filter yet = ∆d (yt ) ≡ I(0) transformed series

- Plot the ACF and PACF of the filtered series yet

1 use PACF to identify p: last statistically significant lag of the autoregression

2 use ACF to identify q: last statistically significant autocorrelation

3 where there is no clear choice, select all potential candidates p and q

I Estimation and validation:

- Estimate all your model candidates and retain those that passed the diagnostic tests

- Use model selection criteria– AIC/SBIC/HQIC to choose the ‘best’ model

I Proceed to forecasting
Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 2 / 23
 Estimation

command in STATA

I Syntax: arima depvar [indepvar] [if] [in] [weights] [,options]

Options include:
- arima(#p , #d #q ): specify ARIMA(p, d,q) model
- noconstant: suppress constant term

I Menu: Statistics → Time series → ARIMA and ARMAX models

I Application: log U.S. Wholesale Price Index (WPI)

- ADF tests indicate that ln−wpi ∼ I(1)

- First test is run with drift + trend, the second with only a drift
Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 3 / 23
 Estimation

Application: correlogram

(b) ACF

(c) PACF

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 4 / 23
 Estimation

Correlogram

- Oscillations of both ACFs and PCFs may be due to seasonal variations

- ACFs ⇒ pure AR(p) processes: couple (p, q) candidates lie in {2, 4} × {0},
i.e., (p, q) ∈ {(2, 0); (4, 0)}
- Presence of seasonal variations may impede on p and q’s identification!!!
Do-file:
use "your file direction", clear
generate tdate=tq(1960q1)+ −n-1
format tdate tsset tdate
tsline lnwpi
varsoc lnwpi
dfuller lnwpi, lags(2) trend regress
dfuller d.lnwpi, lags(2) regress /*First-Dif*/
ac d.lnwpi, lags(40)
pac d.lnwpi, lags(40)
Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 5 / 23
 Estimation

Estimation results

I Estimation:
- Estimates store in a table with their s.e’s

- Model selection statistics are also provided: AIC & BIC

- The AIC indicates that ARIMA (4,1,0) model fits the data better

- Whereas the BIC indicates that it is ARIMA (2,1,0)

- As is often the case, different model-selection criteria have led to conflicting conclusions

- Both criteria select a pure AR process: no MA component is selected!

I Use comparative forecasting performance: which model forecasts better?

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 6 / 23
 Estimation

Estimation code

Do-file:
arima lnwpi, arima(2,1,0)
estat ic
estimates store ARIMA210
estat aroots /* check stability of polynomial roots*/
arima lnwpi, arima(4,1,0)
estat ic
estimates store ARIMA410
/* store all models in a Table */
estimates table ARIMA210 ARIMA410, se stats(N)

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 7 / 23
 Forecasting with ARIMA

Forecasting techniques

I Static forecasts: produces forecast from past data; limited in out-of-sample forecasting

I Dynamic forecasts: uses past forecasts for the currents; unrestricted in out-of-sample case

arima lnwpi, arima(2,1,0)

predict AR2s, y
predict AR2d, dynamic(tq(1965q1)) y
tsline lnwpi AR2s AR2d, title("Forecasts-ARIMA (2,1,0)") ///
legend(label(1 "Observed") ///
label(2 "One-step ahead: Static") label(3 "One-step ahead: Dynamic"))
arima lnwpi, arima(4,1,0)
predict AR4s, y
predict AR4d, dynamic(tq(1965q1)) y
tsline lnwpi AR4s AR4d, title("Forecasts-ARIMA (4,1,0)") ///
legend(label(1 "Observed") ///
label(2 "One-step ahead: Static") label(3 "One-step ahead: Dynamic"))
Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 8 / 23
 Forecasting with ARIMA

Forecasts: 1 step-ahead

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 9 / 23
 Forecasting with ARIMA

Forecasts: 1 step-ahead

- Static > Dynamic: prior forecast errors accumulate over time with dynamic
forecasts ⇒ Static is better!

- However, there is a problem with static: cannot obtain out-of-sample

forecasts at T + 2, T + 3, . . .
(S) (S)
→ Static forecast ybT +1 can be generated using yT , but generating ybT +2 requires
observing yT +1, which we don’t

⇒ Dynamic forecasting is what we should focus on!!

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 10 / 23
 Forecasting with ARIMA

forecast performance: ARIMA (2,1,0) vs. ARIMA (4,1,0)

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 11 / 23
 Forecasting with ARIMA

ARIMA (2,1,0) vs. ARIMA (4,1,0)

- Static forecasts: ARIMA (2,1,0) & ARIMA (4,1,0) show equal performance:
both do well in mimicking the real data

- Dynamic forecasts: ARIMA (2,1,0) outperforms ARIMA (4,1,0)

- ARIMA (2,1,0) should be retained ≡ same choice of model as the BIC.

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 12 / 23
 Seasonal ARIMA models

Seasonal Adjustment techniques

I Seasonality in a time series:

- regular pattern of changes that repeats over S time periods, where S defines the number of
time periods until the pattern repeats again

- Monthly data for which high values tend always to occur in some particular months & low
values tend always to occur in other particular months ⇒ S = 12. If quarterly data ⇒ S = 4
I Seasonal ARIMA model:
- seasonal AR and MA terms predict the series using data values and errors at times with lags
that are multiples of S

- with monthly data (and S = 12), a seasonal first-order autoregressive model would use
yt−12 to predict yt . A seasonal second-order autoregressive model would use yt−12 and
yt−24 to predict yt

- a seasonal first-order MA(1) model (with S = 12) would use εt−12 as a predictor. A
seasonal second-order MA(2) model would use εt−12 and εt−24.

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 13 / 23
 Seasonal ARIMA models

Seasonal Adjustment techniques

I Seasonality usually causes non-stationarity:

- average values at some particular times within the seasonal span may be different than the
average values at other times

- Seasonal differencing renders the series stationary: With S = 12, (1 − L12)yt = yt − yt−12 is
purged of seasonal variations.

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 14 / 23
 Seasonal ARIMA models

Seasonal Adjustment techniques

I Non-seasonal differencing:
- If trend is present in the data, we may also need non-seasonal differencing

- Often (not always) a first-difference (nonseasonal) will “detrend" the data, i.e., we use
(1 − L)yt = yt − yt−1 in the presence of trend

I Differencing for Trend and Seasonality

- When both trend and seasonality → apply both a non-seasonal first-difference and a
seasonal difference ⇒ examine ACF and PACF of
(1 − L12)(1 − L)yt = (yt − yt−1) − (yt−12 − yt−13)

- Removing trend doesn’t mean that we have removed the dependency: We may have
removed the mean, µt , part of which may include a periodic component

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 15 / 23
 Seasonal ARIMA models

SARIMA Models

I SARIMA Models: incorporates both non-seasonal and seasonal factors in

two ways
1 multiplicative: shorthand notation is

ARIMA(p, d, q) × (P, D, Q)S , where

− p ≡ AR order, d≡ order of integration, q ≡ MA order

− P ≡ seasonal AR order, D ≡ seasonal differencing, Q ≡ seasonal MA order

− S ≡ time span of repeating seasonal pattern

− yt ∼ ARIMA(p, d, q) × (P, D, Q)S ⇔ φ(LS )ϕ(L)yt = Θ(LS )θ(L)εt

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 16 / 23
 Seasonal ARIMA models

SARIMA Models

• Non-seasonal: ϕ(L) = 1 − ϕ1L − . . . − ϕp Lp , θ(L) = 1 + θ1L + . . . + θq Lq ,

• Seasonal: Φ(LS ) = 1 − φ1LS − . . . − φP LSP , Θ(LS ) = 1 + θ1LS + . . . + θQ LSQ ,

• Examples: ARIMA(1, 0, 0) × (1, 0, 0)12, ARIMA(0, 0, 1) × (0, 0, 1)12

I additive:yt ∼ ARIMA(p, d, q) + (P, D, Q)S

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 17 / 23
 Seasonal ARIMA models

Application of SARIMA to Airline data

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 18 / 23
 Seasonal ARIMA models

Application of SARIMA to Airline data

• After first- and seasonally differencing the data:

− No presence of a trend component in the transformed data

- Use the “noconstant" option with ARIMA

− Stata command: arima lnair, arima(0,1,1) sarima(0,1,1,12) noconstant

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 19 / 23
 Seasonal ARIMA models

Application of SARIMA to Airline data

I We can write the outcome of the regression as:

(1 − L12)(1 − L)lnairt = −0.402εt−1−0.557εt−12+0.224εt−13 + εt

- Coefficient on εt−13 is the product of the coefficients on the εt−1 and εt−12 terms:

(−0.402) × (−0.557) = 0.224

- ARIMA labeled the dependent variable DS12.lnair to indicate that it has applied the
difference operator ∆ and the lag-12 seasonal difference operator ∆12 to “lnair "

- This model could have been fit by typing the command:

arima DS12.lnair, ma(1) mma(1, 12) noconstant

- For simple multiplicative models, using the sarima() option is easier, though this second
syntax allows us to incorporate more complicated seasonal terms

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 20 / 23
 Seasonal ARIMA models

Forecasting with SARIMA: Airline data

• SARIMA models have a good forecast performance:

- Static: close to the observed data

- Dynamic: not as good as static but shows an overall acceptable performance.

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 21 / 23
 X-12-ARIMA Seasonal Adjustment

X-12-ARIMA Seasonal Adjustment in STATA

I X-12-ARIMA was the U.S. Census Bureau’s software package for seasonal
adjustment:
H can be used together with many statistical packages:

- Gretl or EViews which provides a graphical user interface for X-12-ARIMA

- NumXL avails X-12-ARIMA functionality in Microsoft Excel

I Many agencies presently are using X-12-ARIMA for seasonal adjustment:

- Statistics Canada

- U.S. Bureau of Labor Statistics

- Brazilian Institute of Geography and Statistics

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 22 / 23
 X-12-ARIMA Seasonal Adjustment

X-12-ARIMA Seasonal Adjustment in STATA

I Menu-driven X-12-ARIMA seasonal adjustment in Stata by:

- Qunyong Wang, Institute of Statistics and Econometrics, Nankai University

- Na Wu, Economics School, Tianjin University of Finance and Economics

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV August 6, 2021 23 / 23