Econometrics [EM2008]

Lecture 7
Stationary univariate time series

Irene Mammi

[email protected]

Academic Year 2018/2019

1 / 29
outline

I stationary univariate time series

I AR, MA and ARMA processes
I autocorrelations and partial autocorrelations
I seasonality

I References:
I Johnston, J. and J. DiNardo (1997), Econometric Methods, 4th
Edition, McGraw-Hill, New York, Chapter 7

2 / 29
introduction
I a univariate time series models the outcome variable in terms of its
own past values and some disturbance:

xt = f (xt −1 , xt −2 , . . . , ut )

I need to specify three features: the functional form f (·), the number
of lags, and a structure for the error term
I autoregressive, AR(p), process:

xt = α1 xt −1 + α2 xt −2 + · · · + αp xt −p + ut

I moving average, MA(q), process

ut = e t − β 1 e t − 1 − · · · − β q e t − q

I autoregressive moving average, ARMA(p, q), process

xt = α1 xt −1 + · · · + αp xt −p + et − β 1 et −1 − · · · − β q et −q

3 / 29
introduction (cont.)
I it might seem reductive to consider a univariate series but, e.g., take
the equations

Ct = α0 + α1 Yt + α2 Ct −1 + ut
Yt ≡ Ct + It

where we generally consider C and Y to be endogenous variables and

I to be the exogenous variable
I some substitutions and simple algebra lead to
α2 α0 α1 1
Ct − Ct −1 = + It + ut
1 − α1 1 − α1 1 − α1 1 − α1
and
α2 α0 1 1
Yt − Yt −1 = + (It − α2 It −1 ) + ut
1 − α1 1 − α1 1 − α1 1 − α1
so that C and Y both have an AR(1) component with the same
coefficient on the lag term and in the RHS there is an error term
whose properties depend on the behavior of I
4 / 29
lag operator

I the lag operator is a linear operator which, for any value of xt , gives
the previous values of the series:

L ( xt ) = xt − 1
L2 (xt ) = L[L(xt )] = L(xt −1 ) = xt −2
Ls (xt ) = xt −s
(1 − L)xt = xt − xt −1 = ∆xt
L(1 − L)xt = xt −1 − xt −2 = ∆xt −1

where ∆ is the first difference operator

I let A(L) = 1 − αL denote a first-order polynomial in L and consider

(1 − αL)(1 + αL + α2 L2 + α3 L3 + · · · + αp Lp ) = 1 − αp +1 Lp +1

where, as p → ∞, αp +1 Lp +1 → 0 provided that |α| < 1

5 / 29
lag operator (cont.)
I we may write the reciprocal, or inverse, of A(L) as

1
A−1 (L ) = = 1 + αL + α2 L2 + α3 L3 + · · ·
(1 − αL)
I e.g., the economic model:

Ct = α0 + α1 Yt + α2 Ct −1 + ut
It = β 0 + β 1 (Yt −1 − Yt −2 ) + νt
Yt ≡ Ct + It + Gt

may be rewritten as
      
(1 − α2 L) 0 − α1 Ct α0 0   ut
D t
 0 1 − β 1 L(1 − L)   It = β 0
  0  + νt 

Gt
−1 −1 1 Yt 0 1 0

where Dt is a dummy variable

I in matrix form, the system becomes A(L)x t = Bz t + w t
6 / 29
stationarity

I a stochastic process is said to be strictly stationary if its properties

are unaffected by a change of time origin: in other words, the joint
probability distribution at t1 , t2 , . . . , tm must be the same as the joint
probability distribution at t1+k , t2+k , . . . , tm+k , where k is an
arbitrary shift in time
I a stochastic process is said to be weakly (or covariance) stationary if

1. E(yt ) = E (yt −s ) = µ
2. E[(yt − µ)2 ] = E[(yt −s − µ)2 ] or var(yt ) = var(yt −s ) = σ2
3. E[(yt − µ)(yt −s − µ)] = E[(yt −j − µ)(yt −j −s − µ)] =
γs or cov(yt , yt −s ) = cov(yt −j , yt −j −s ) = γs

7 / 29
stationarity (cont.)

stationary process:
I observations are regularly spread
about the mean: mean and
variance do not vary over time

non stationary process: non stationary process:

I considering different time I considering different time
intervals, the mean varies intervals, the variance varies
8 / 29
ARMA modeling

AR(1) process

I the model reads

yt = m + αyt −1 + et
where et is a white noise
I can rewrite the equation as

(1 − αL)yt = m + et

which gives

yt = (1 + αL + α2 L2 + · · · )(m + et )

I since m is a constant, can write

yt = (1 + α + α2 + · · · )m + (et + αet −1 + α2 et −2 + · · · )

9 / 29
ARMA modeling (cont.)
I provided |α| < 1 this gives
m
E ( yt ) = =µ
1−α
⇒ y has a constant unconditional mean, independent of time
I y also has constant unconditional variance

σe2
σy2 = E(yt − µ)2 =
1 − α2
I alternatively, write
xt = αxt −1 + et
where xt = yt − µ
I then square both sides and take expectations to find

E(xt2 ) = α2 E(xt2−1 ) + E(et2 ) + 2α E(xt −1 et )

where the last term on RHS vanishes

10 / 29
ARMA modeling (cont.)
I when |α| < 1
σy2 = E(xt2 ) = E(xt2−1 ) = · · ·
so that
σy2 = α2 σy2 + σe2
I multiplying both sides of xt = αxt −1 + et by xt −1 and taking
expectations gives

E(xt xt −1 ) = α E(xt2−1 ) + E(xt −1 et )

I defining autocovariance coefficients as γs = E(xt xt −s ) we have

γ1 = αγ0 ,

γ2 = αγ1
and, in general,

γk = αγk −1 = αk γ0 k = 1, 2, . . .
11 / 29
ARMA modeling (cont.)

I the autocorrelation coefficients for a stationary series are defined by

E(xt xt −k ) γ
ρk = p p = k
var(xt ) var(xt −k ) γ0

I nb: the autocovariances and autocorrelation coefficients are

symmetrical about lag zero
I the autocorrelation coefficients for the AR(1) process are

ρk = αρk −1 = αk k = 1, 2, . . .

I the formula giving the autocorrelation coefficients is the

autocorrelation function (acf): its graphical representation is the
correlogram

12 / 29
ARMA modeling (cont.)

Figure 1: correlograms for stationary AR(1) series

13 / 29
ARMA modeling (cont.)
AR(2) process

I the AR(2) process is defined as

yt = m + α 1 yt − 1 + α 2 yt − 2 + e t

I by assuming stationarity, the unconditional mean is

µ = m/(1 − α1 − α2 )
I defining xt = yt − µ, we may rewrite the previous equation as

xt = α1 xt −1 + α2 xt −2 + et

I multiplying by xt and taking expectations, we have

γ0 = α1 γ1 + α2 γ2 + E(xt et )

I since E(xt et ) = σe2 , then

γ0 = α1 γ1 + α2 γ2 + σe2
14 / 29
ARMA modeling (cont.)
I similarly

γ1 = α1 γ0 + α2 γ1
γ2 = α1 γ1 + α2 γ0

I after some substitutions and simple algebra we get

(1 − α2 )σe2
γ0 =
(1 + α2 )(1 − α1 − α2 )(1 + α1 − α2 )
I under stationarity, this variance must be a constant, positive number
I the stationarity conditions for the AR(2) process therefore are:

α2 + α1 < 1
α2 − α1 < 1
| α2 | < 1

15 / 29
ARMA modeling (cont.)

I we can also restate the autocorrelation coefficients in terms of the

Yule-Walker equations for AR(2) processes as:

ρ1 = α1 + α2 ρ1
ρ2 = α1 ρ1 + α2

I solving for the first two autocorrelation coefficients gives

α1 α21
ρ1 = ρ2 = + α2
1 − α2 1 − α2
I the acf for the AR(2) process is

ρ k = α 1 ρ k −1 + α 2 ρ k −2 k = 3, 4, . . .

16 / 29
ARMA modeling (cont.)
roots of the polynomial in the lag operator

I rewrite xt = α1 xt −1 + α2 xt −2 + et as

A ( L ) xt = e t

where
A ( L ) = 1 − α 1 L − α 2 L2
I express the quadratic form as the product of two factors

A(L) = 1 − α1 L − α2 L2 = (1 − λ1 L)(1 − λ2 L)

I the link between the α and λ parameters is

λ1 + λ2 = α1 and λ1 λ2 = − α2

I the λ’s may be seen as the roots of λ2 − α1 λ − α2 = 0, which is the

characteristic equation of the second-order process

17 / 29
ARMA modeling (cont.)
I its roots are q
α1 ± α21 + 4α2
λ1 , λ2 =
2
and need to satisfy λ1 + λ2 = α1 and λ1 λ2 = −α2
I we can write
1 c d
A−1 (L ) = = +
(1 − λ1 L)(1 − λ2 L) 1 − λ1 L 1 − λ2 L

where c = −λ1 /(λ2 − λ1 ) and d = λ2 /(λ2 − λ1 )

I then
c d
xt = A − 1 ( L ) e t = et + et
1 − λ1 L 1 − λ2 L
I from the results on the AR(1) process, stationarity of the AR(2)
process requires
|λ1 | < 1 and |λ2 | < 1

18 / 29
ARMA modeling (cont.)
I the AR(2) case allows for the possibility of complex unit roots, which
will occur if α21 + 4α2 < 0
I the roots may be written

λ1 , λ2 = h ± vi
q
where h and v are the real numbers h = α1 /2, v = 12 −(α21 + 4α2 ),
√
and i is the imaginary number i = −1, giving i 2 = −1.
I the absolute value or modulus of each complex root is
p
|λj | = h2 + v 2 = −α2 j = 1, 2

giving 0 < −α2 < 1 as the condition for the autocorrelations to

approach zero as time increases
I nb: for the stationarity condition to be met, the roots of the relevant
polynomial in the lag operator A(z ) = 1 − α1 z − α2 z 2 should lie
outside the unit circle
19 / 29
ARMA modeling (cont.)

I the roots of A(z ) are the values of z that solve the equation

A(z ) = 1 − α1 z − α2 z 2 = (1 − λ1 z )(1 − λ2 z ) = 0

I the roots are zj = 1/λj (j = 1, 2)

20 / 29
ARMA modeling (cont.)

Figure 2: correlograms for stationary AR(2) series

21 / 29
ARMA modeling (cont.)
partial autocorrelation function (pacf)

I in an AR(2) process the α2 parameter is the partial correlation

between xt and xt −2 with xt −1 held constant
I recall partial correlations in the three-variable case
r13 − r12 r23
r13.2 = q q
2
1 − r12 2
1 − r23

I for stationary time series, letting 1,2,3 denote x and its first and
second lags, we have

r12 = corr(xt , xt −1 ) = corr(xt −1 , xt −2 ) = r23 = ρ1

r13 = corr(xt , xt −2 ) = ρ2
I therefore
ρ2 − ρ21
r13.2 =
1 − ρ21
22 / 29
ARMA modeling (cont.)

I can form the partial correlations from autocorrelations by exploiting

the Yule-Walker equations to get

ρ2 − ρ21
α2 = = r13.2
1 − ρ21

I the pacf of an AR(2) process will cut after the second lag

23 / 29
ARMA modeling (cont.)
MA process

I the AR(1) process xt = αxt −1 + et may be inverted to give

xt = et + αet −1 + α2 et −2 + · · ·

which is a MA process of infinite order, MA(∞)

I the MA(1) process is

xt = e t − β 1 e t − 1

I the autocovariances are

γ0 = (1 + β21 )σe2
γ1 = − β 1 σe2
γ2 = γ3 = · · · = 0

24 / 29
ARMA modeling (cont.)
which gives the autocorrelations coefficients as
− β1
ρ1 =
1 + β21
ρ2 = ρ3 = · · · = 0
I the MA(1) process may be inverted to give et as an infinite series in
xt , xt −1 , . . ., i.e.

et = xt + β 1 xt −1 + β21 xt −2 + · · ·

that is
xt = − β 1 xt −1 − β21 xt −2 − · · · + et
I because this is an AR(∞) series, partial autocorrelations do not cut
off but the autocorrelations are zero after the first
I the properties of a pure MA are the converse of those of a pure AR
I the acf of an MA process cuts off after the order of the process, and
the pacf damps towards zero
I invertibility condition: | β 1 | < 1
25 / 29
ARMA modeling (cont.)
ARMA process

I the general ARMA(p, q) process is

A ( L ) xt = B ( L ) e t

where

A(L) = 1 − α1 L − α2 L2 − · · · − αp Lp
B (L) = 1 − β 1 L − β 2 L2 − · · · − β q Lq

I stationarity requires the roots of A(L) to lie outside the unit circle;
invertibility places the same condition on the roots of B (L)
I given these conditions the ARMA(p, q) may alternatively be
expressed as a pure AR process of infinite order or as a pure MA
process of infinite order, i.e.

B −1 (L)A(L)xt = et or xt = A − 1 ( L ) B ( L ) e t

26 / 29
ARMA modeling (cont.)

I the simplest mixed process is the ARMA(1,1):

xt = αxt −1 + et − βet −1

I squaring the previous equation and taking expectations, after some

algebra we get
1 − 2αβ + β2 2
σx2 = γ0 = σe
1 − α2
I multiplying the ARMA(1,1) by xt −1 then taking expectations leads to

(α − β)(1 − αβ) 2
γ1 = αγ0 − βσe2 = σe
1 − α2
I higher order covariances are given by

γk = αγk −1 k = 2, 3, . . .

27 / 29
ARMA modeling (cont.)

I the acf for the ARMA(1, 1) is thus

(α − β)(1 − αβ)
ρ1 =
1 − 2αβ + β2
ρk = αρk −1 k = 2, 3, . . .

Table 1: correlation patterns

process acf pacf

AR(p) infinite: damps out finite: cuts off after lag p
MA(q) finite: cuts off after lag q infinite: damps out
ARMA(p, q) infinite: damps out infinite: damps out

28 / 29