Applied Econometrics 3rd edition

Dimitrios Asteriou
and
Stephen G Hall
 Applied Econometrics 3rd edition

TIME SERIES MODELS

1. ARIMA models
2. Stationarity
3. Autoregressive time series models
4. Moving average models
5. ARMA models
6. Integrated processes and the ARIMA models
7. Box–Jenkins model selection
 Applied Econometrics 3rd edition

Learning Objectives
1. Understand the concept of ARIMA models.
2. Differentiate between univariate and multivariate
time series models.
3. Understand the Box–Jenkins approach for model
selection in the univariate time series framework.
4. Know how to estimate ARIMA(p, d, q) models
using econometric software.
 Applied Econometrics 3rd edition

ARIMA Models
Box and Jenkins (1976) first introduced ARIMA
models, the term deriving from:

• AR = autoregressive
• I = integrated
• MA = moving average.
 Applied Econometrics 3rd edition

Stationarity
A key concept underlying time series processes is that of
stationarity. A time series is covariance stationary
when it has the following three characteristics:
(a) exhibits mean reversion in that it fluctuates around a
constant long-run mean;
(b) has a finite variance that is time-invariant; and
(c) has a theoretical correlogram that diminishes as the lag
length increases.
 Applied Econometrics 3rd edition

Stationarity
In its simplest terms a time series Yt is said to be
stationary if:

(a) E(Yt ) = constant for all t;

(b) Var(Yt ) = constant for all t; and
(c) Cov(Yt,Yt+k) = constant for all t and all k = 0,

or if its mean, its variance and its covariances remain

constant over time.
 Applied Econometrics 3rd edition

Stationarity
Thus, these quantities would remain the same whether
observations for the time series were, for example, from
1975 to 1985 or from 1985 to 1995.

Stationarity is important because if the series is non-

stationary then all the typical results of the classical
regression analysis are not valid. Regressions with non-
stationary series may have no meaning and are therefore
called ‘spurious’.
 Applied Econometrics 3rd edition

Autoregressive models
The AR(1) model
The simplest, purely statistical time series model is the
autoregressive of order one model, or AR(1) model,
which is given below:
Yt = φYt−1 + ut

where, for simplicity, we do not include a constant and |

φ| < 1 and ut is a gaussian (white noise) error term.
The implication behind the AR(1) model is that the time
series
 Applied Econometrics 3rd edition

Autoregressive models
The implication behind the AR(1) model is that the
time series behaviour of Yt is largely determined by
its own value in the preceding period.

So, what will happen in t is largely dependent on

what happened in t −1, or alternatively what will
happen in t + 1 will be determined by the behaviour
of the series in the current time t.
 Applied Econometrics 3rd edition

Autoregressive models
• Condition for stationarity
• The AR(1) equation introduces the constraint
|φ| < 1, in order to guarantee stationarity as
defined in the previous section.
• If we have |φ| > 1, then Yt will tend to get
bigger and bigger each period and so we
would have an explosive series.
 Applied Econometrics 3rd edition

Autoregressive models
Open EViews and create a new workfile by choosing File/NewWorkfile. In the
workfile range choose undated or irregular and define the start observation as
1 and the end observation as 500. To create a stationary time series process type
the following commands in the EViews command line (the bracketed comments
provide a description of each command):

smpl 1 1 [sets the sample to be the first observation only]

genr yt=0 [generates a new variable yt with the value of 0]
smpl 2 500 [sets the sample to be from the 2nd to the 5
hundredth observation]
genr yt=0.4∗yt(−1)+nrnd [creates yt as an AR(1) model with φ =0.4]
smpl 1 500 [sets the sample back to the full sample]
plot yt [provides a plot of the yt series]
 Applied Econometrics 3rd edition

Autoregressive models
4

-1

-2

-3
100 200 300 400 500

YT
 Applied Econometrics 3rd edition

Autoregressive models
If we want to see the exploding series we change the phi coefficient to 1.4 and we
get

• smpl 1 1 [sets the sample to be the first observation only]

• genr yt=0 [generates a new variable yt with the value of 0]
• smpl 2 500 [sets the sample to be from the 2nd to the 5
• hundredth observation]
• genr yt=1.4∗yt(−1)+nrnd [creates yt as an AR(1) model with φ =0.4]
• smpl 1 500 [sets the sample back to the full sample]
• plot yt [provides a plot of the yt series]
 Applied Econometrics 3rd edition

Autoregressive models
2.4E+15

2.0E+15

1.6E+15

1.2E+15

8.0E+14

4.0E+14

0.0E+00

-4.0E+14
25 50 75 100 125 150 175 200

XT
 Applied Econometrics 3rd edition

Autoregressive models
The AR( p) model
• A generalization of the AR(1) model is the AR(p)
model; the number in parenthesis denotes the order
of the autoregressive process and therefore the
number of lagged dependent variables that the
model will have.
• For example, the AR(2) model will be an
autoregressive model of order two, and will have the
form:
Yt = φ1Yt−1 + φ2Yt−2 + ut
 Applied Econometrics 3rd edition

Autoregressive models
Similarly the AR(3) model will be an
autoregressive model of order three, and will
have the form:
Yt = φ1Yt−1 + φ2Yt−2 + φ3Yt−3 + ut

And in general the AR(p) model will be an

autoregressive model of order p, and will have p
lagged terms as in the following:
Yt = φ1Yt−1 + φ2Yt−2 +· · ·+φpYt−p + ut
 Applied Econometrics 3rd edition

Autoregressive models
Using the summation symbol:
Yt =ΣφiYt−i + ut

Or using the lag operator L (the lag operator L has

the property: LnYt = Yt−n) we can write the AR(p)
model as:
Yt (1 − φ1L − φ2L2 −· · ·−φpLp) = ut
Φ(L)Yt = ut
where Φ(L)Yt , is a polynomial function of Yt.
 Applied Econometrics 3rd edition

Moving Average Models

The MA(1) model
The simplest moving average model is that of order
one, or the MA(1) model, which has the form:
Yt = ut + θut−1

Thus, the implication behind the MA(1) model is that

Yt depends on the value of the immediate past error,
which is known at time t.
 Applied Econometrics 3rd edition

Moving Average Models

The MA(q) model
The general form of the MA model has the form:
Yt = ut + θ1ut−1 + θ2ut−2+…+ θqut−q
Using the summation symbol:
Yt = ut + Σ θjut−j
Or using the lag operator L :
Yt =(1 + θ1L + θ2L2 +· · ·+θqLq)ut
Yt = Θ(L)ut
where Θ(L)ut , is a polynomial function of ut.
 Applied Econometrics 3rd edition

Moving Average Models

Invertibility in MA models
A property often discussed in connection with the
moving average processes is that of invertibility. A
time series Yt is invertible if it can be represented
by a finite-order MA or convergent autoregressive
process.
Invertibility is important because the use of the ACF
and PACF for identification implicitly assumes that
the Yt sequence can be well-approximated by an
autoregressive model.
 Applied Econometrics 3rd edition

Moving Average Models

For an example consider the simple MA(1) model:
Yt = ut + θut−1
Using the lag operator this can be rewritten as:
Yt = (1 + θL)ut
ut =Yt/(1 + θL)
If |θ| < 1, then the left-hand side of the above equation
can be considered as the sum of an infinite geometric
progression:
ut = Yt (1 − θL + θ2L2 − θ3L3 +· · · )
 Applied Econometrics 3rd edition

Moving Average Models

To understand this consider the MA(1) process:
Yt = ut − θut−1
Lagging this relationship one period and solving
for ut we have:
ut−1 = Yt−1 − θut−2
Substituting this into the original expression we
have:
Yt = ut − θ(Yt−1 − θut−2) = ut − θYt−1 + θ2ut−2
 Applied Econometrics 3rd edition

Moving Average Models

Lagging the above expression one period and solving
for ut−2 and resubstituting we get:
Yt = ut − θYt−1 + θ2Yt−2 − θ3ut−3
and repeating this an infinite number of times we
finally get the expression we wanted to prove.
Thus, the MA(1) process has been inverted into an
infinite order AR process with geometrically
declining weights. Note that for the MA(1) process to
be invertible it is necessary that |θ| < 1.
In general the MA(q) processes are invertible if the
roots of the polynomial Θ(z) = 0 are greater than 1
 Applied Econometrics 3rd edition

ARMA Models
The general form of the ARMA model is an
ARMA(p, q) model of the form:

Yt = φ1Yt−1 + φ2Yt−2 +· · ·+φpYt−p + ut+

+θ1ut−1 + θ2ut−2 +· · ·+θqut−q

or
Yt = ΣφiYt−i + ut +Σ θjut−j
 Applied Econometrics 3rd edition

Integrated processes and

ARIMA Models
An integrated series
ARMA models can only be made on time series Yt that are stationary.
Most economic and financial time series show trends over time, and so
the mean of Yt during one year will be different from its mean in
another year.
Thus, the mean of most economic and financial time series is not
constant over time, which means that the series are non-stationary.
In order to avoid this problem, and in order to induce stationarity, we
need to detrend the raw data through a process called differencing.
 Applied Econometrics 3rd edition

Integrated processes and

ARIMA Models
The first differences of a series Yt are given by the equation:
ΔYt = Yt − Yt−1
If, after first differencing, a series is stationary then the series is also
called integrated to order one, and denoted I(1).
If the series, even after first differencing is not stationary, then we need
to take second differences by the equation:
ΔΔYt = Δ2Yt = Yt − Yt−1
If the series becomes stationary after second differences, then it is
integrated of order two and denoted by I(2).
And in general if it is stationary after d differences then it is called I(d).
Thus, we have ARIMA(p,d,q).
 Applied Econometrics 3rd edition

The Box-Jenkins Approach

A fundamental idea in the Box–Jenkins approach is
the principle of parsimony.
Parsimony (meaning sparseness or stinginess) should
come as second nature to economists and financial
analysts.
Incorporating additional coefficients will necessarily
increase the fit of the regression equation (i.e. the
value of the R2 will increase), but the cost will be a
reduction of the degrees of freedom. Box and
Jenkins argue that parsimonious models produce
better forecasts than overparametrized models.
 Applied Econometrics 3rd edition

The Box-Jenkins Approach

In general Box and Jenkins popularized a three-stage
method aimed at selecting an appropriate
(parsimonious) ARIMA model for the purpose of
estimating and forecasting a univariate time series.
The three stages are:
(a) identification,
(b) estimation, and
(c) diagnostic checking
 Applied Econometrics 3rd edition

The Box-Jenkins Approach

Step 1 Calculate the ACF and PACF of the raw data, and check
whether the series is stationary or not. If the series are
stationary go to step 3, if not go to step 2.
Step 2 Take the logarithm and the first differences of the raw data
and calculate the ACF and PACF for the first logarithmic
differenced series.
Step 3 Examine the graphs of the ACF and PACF and determine
which models would be good starting points.
Step 4 Estimate those models.
 Applied Econometrics 3rd edition

The Box-Jenkins Approach

Step 5 For each of these estimated models:
(a) check to see if the parameter of the longest lag is significant.
If not, then you probably have too many parameters, and
should decrease the order of p and/or q.
(b) check the ACF and PACF of the errors. If the model has at
least enough parameters, then all error ACFs and PACFs will
be insignificant.
(c) check the AIC and SBC together with the adj-R2 of the
estimated models to detect which model is the parsimonious
one (i.e. the one that minimizes AIC and SBC and has the
highest adj-R2).
Step 6 If changes in the original model are needed, go back to
step 4.