Time Series

Chapter 3 - ARMA Model

ARMA Model (Time Series) Chapter 3 1 / 39

...
Agenda

1 Introduction

2 Causality and Invertibility (AR and MA Representations)

3 Stationarity of ARMA models

4 Derive Autocovariance γ(k) via Yule-Walker equation

5 ARIMA model and Financial Time Series

6 Seasonal Model

7 Summary

ARMA Model (Time Series) Chapter 3 2 / 39

...
Introduction
ARMA model: the most common model for stationary time series
φ(B)Yt = θ(B)Zt
Yt − φ1 Yt−1 − · · · − φp Yt−p = Zt − θ1 Zt−1 − · · · − θq Zt−q
| {z } | {z }
Autoregressive (AR) Moving Average (MA)
Yt is observation,
Zt ∼ WN (0, σ 2 ) is white noise.
Characteristic polynomials:

φ(B) = 1 − φ1 B − φ2 B 2 − · · · − φp B p
θ(B) = 1 − θ1 B − θ2 B 2 − · · · − θq B q
ARIMA model: the most common for non-stationary time series
φ(B) (1 − B)d Yt = φ(B)∆d Yt = θ(B)Zt
| {z }
Integrated
If Yt follows an ARIMA model with integrated parameter d,
then 4d Yt = (1 − B)d Yt follows an ARMA model.
ARMA Model (Time Series) Chapter 3 3 / 39
...
Common Roots Can Be Canceled Out!
Consider the common root case (1 − aB)Yt = (1 − aB)Zt , i.e.,
Yt − aYt−1 = Zt − aZt−1 , Zt ∼ W N (0, σ 2 ) . (1)

Now, φ(B)
= (1 − aB)

= θ(B)
=⇒ φ a1 = θ a1 = 0
=⇒ a1 is the common root.

Apply (1) repeatedly, we have for all k ≥ 1 that

Yt − Zt = aYt−1 − aZt−1 = a(Yt−1 − Zt−1 ) = · · · = ak (Yt−k − Zt−k ) .

Claim: Equation (1) implies that Yt = Zt for all integer t if a 6= 1.

k→∞
a < 1: Yt − Zt = ak (Yt−k − Zt−k ) −→ 0
1 k→∞
a > 1: Yt − Zt = ak
(Yt+k − Zt+k ) −→ 0 (Replace t by t + k)

a = 1: Yt − Zt are equal for all t

ARMA Model (Time Series) Chapter 3 4 / 39

...
Identification of ARIMA models
Definition 1
Given Zt ∼ WN, φ(B)(1 − B)d Yt = θ(B)Zt is called an ARIMA(p, d, q) model if
φ(B) = 1 − φ1 B − φ2 B 2 − · · · − φp B p is of order p, and φ(1) 6= 0
θ(B) = 1 − θ1 B − θ2 B 2 − · · · − θq B q is of order q.

φ(B) and θ(B) have no common roots: no x s.t. φ(x) = θ(x) = 0.

Exercises: Identify the the following ARIMA models

a) Yt − 0.5Yt−1 + 0.06Yt−2 = Zt + 0.6Zt−1 − 0.16Zt−2

b) Yt − 0.8Yt−1 − 0.2Yt−2 = Zt + 0.2Zt−1

c) Yt − 0.8Yt−1 − 0.2Yt−2 = Zt − 0.2Zt−1

ARMA Model (Time Series) Chapter 3 5 / 39

...
Special Cases of ARIMA(p, d, q) Models
ARMA(p, q) = ARIMA(p, 0, q): φ(B)Yt = θ(B)Zt
ARIMA(p, d, q) model is always not stationary for d ≥ 1
Some but not all ARMA(p,q) models are stationary
AR(p) = ARMA(p, 0): φ(B)Yt = Zt
Explicit formula: Yt = φ1 Yt−1 + · · · + φp Yt−p + Zt
AR model is very popular because:
It closely resembles traditional regression model:

y = β0 + β1 x1 + β2 x2 + z

∴ traditional theoretical results can be used

It is intuitive: today’s value depends on previous days
MA(q) = ARMA(0, q): Yt = θ(B)Zt
Formula: Yt = Zt − θ1 Zt−1 − · · · − θq Zt−q
Easy to conduct calculations since {Zk } ∼WN are uncorrelated
ARMA Model (Time Series) Chapter 3 6 / 39
...
Identification of ARIMA models
ARMA models with mean E(Yt ) = µ:
φ(B)(Yt − µ) = θ(B)Zt .
Exercises:
1 If {Y } is stationary, show that φ(B)Y = θ(B)Z implies E(Y ) = 0.
t t t t

2 Express the AR(2) model (1 − φ1 B − φ2 B 2 )(Yt − µ) = Zt in the form

Yt = φ̃0 + φ̃1 Yt−1 + φ̃2 Yt−2 + Zt .

ARMA Model (Time Series) Chapter 3 7 / 39

...
Agenda

1 Introduction

2 Causality and Invertibility (AR and MA Representations)

3 Stationarity of ARMA models

4 Derive Autocovariance γ(k) via Yule-Walker equation

5 ARIMA model and Financial Time Series

6 Seasonal Model

7 Summary

ARMA Model (Time Series) Chapter 3 8 / 39

...
Motivating Example: Express MA(1) as AR(∞)
Example 2 (MA(1): Yt = Zt − θZt−1 = (1 − θB)Zt with |θ| < 1)
Using definition of MA(1) model repeatedly:
Zt = Yt + θZt−1 = Yt + θ(Yt−1 + θZt−2 )
= Yt + θYt−1 + θ2 Zt−2 = Yt + θYt−1 + θ2 (Yt−2 + θZt−3 )
= Yt + θYt−1 + θ2 Yt−2 + θ3 Zt−3
= Yt + θYt−1 + θ2 Yt−2 + θ3 Yt−3 + θ4 Yt−4 + · · · (only if |θ| < 1)

Using (1 − x)−1 = 1 + x + x2 + · · · + xk + · · · , we “invert” (1 − θB)

Zt = (1 − θB)−1 Yt
= (1 + θB + θ2 B 2 + θ3 B 3 + θ4 B 4 + · · · )Yt
= Yt + θYt−1 + θ2 Yt−2 + θ3 Yt−3 + θ4 Yt−4 + · · ·

P∞
Zt = (1 − θB)−1 Yt = Yt + θYt−1 + θ2 Yt−2 + · · · = k=0 θk Yt−k should be
understood as notation rather than numerical computation, since B is not a
real number.
ARMA Model (Time Series) Chapter 3 9 / 39
...
Causality and Invertibility
Definition 3 (Causality; MA/Causal Representation)
{Yt } is Causal if it can be expressed as a linear combination of the past white
noise Zt , Zt−1 ,...,i.e.,
X∞
Yt = ψk Zt−k ,
k=0
P∞
where k=0 |ψk | < ∞ (so that the summation is well defined).

Definition 4 (Invertibility; AR Representation)

A model is invertible if the white noise Zt can be expressed as a linear
combination of the past observations Yt , Yt−1 ,...,i.e.,
X∞
Zt = ψk Yt−k ,
k=0
P∞
where k=0 |ψk | < ∞ (so that the summation is well defined).
P∞ P∞ 1
• If ψk = θk with |θ| < 1, then k=0 |ψk | = k=0 |θk | = 1−|θ| <∞
ARMA Model (Time Series) Chapter 3 10 / 39
...
Advantage of Causal and Invertible Models
P∞ P∞
Causal: Exist MA representation: Yt = k=0
ψk Zt−k , k=0
|ψk | < ∞.

Physical meaning: Data is generated by randomness from the past.

Easy to conduct calculations since {Zk } ∼WN are uncorrelated
P∞ P∞
Invertible: Exist AR representation: Zt = k=0
ψk Yt−k , k=0
|ψk | < ∞.

Physical meaning: Unobserved noise can be computed by past data.

Important for prediction: (see Chapter 5)
Consider ARMA(1,1) model Yt = φYt−1 + Zt + θZt−1 , t = 1, . . . , n.
In predicting Yn+1 , we need to estimate Zn :

Ŷn+1 = En (Yn+1 ) = En (φYn + Zn+1 + θZn ) = Yn + θZn .

ARMA Model (Time Series) Chapter 3 11 / 39

...
Causality Theorem
Theorem 5 (Causality)
An ARMA(p, q) process φ(B)Yt = θ(B)Zt , Zt ∼ W N (0, σ 2 ) is causal if
the roots of the characteristic polynomial
φ(B) = 1 − φ1 B − · · · − φp B p
are outside the unit circle.

Proof.
Factorization: φ(B)Yt = (1 − ξ1 B)(1 − ξ2 B) · · · (1 − ξp B)Yt = θ(B)Zt
The roots of φ(x), x = ξ11 , . . . , ξ1p , are outside unit circle
 
⇔  ξ1k  > 1, all k = 1, . . . , p
 

⇔ |ξk | < 1, all k = 1, . . . , p

⇔ (1 − ξk B), all k = 1, . . . , p, can be “inverted” to the other side
⇔ Yt = (1 − ξ1 B)−1 · · · (1 − ξp B)−1 θ(B)Zt is represented by past noises.
⇔ {Yt } is causal.
ARMA Model (Time Series) Chapter 3 12 / 39
...
Exercises
Which of the following model(s) is(are) causal? If yes, give the MA/causal
representation up to the second term.
a) Yt + 0.6Yt−1 − 0.16Yt−2 = Zt
b) Yt − 0.6Yt−1 − 0.72Yt−2 = Zt
c) Yt − 0.5Yt−2 − 0.5Yt−4 = Zt

ARMA Model (Time Series) Chapter 3 13 / 39

...
Invertibility Theorem
Theorem 6 (Invertibility)
An ARMA(p, q) process φ(B)Yt = θ(B)Zt , Zt ∼ W N (0, σ 2 ) is invertible
if the roots of the characteristic polynomial
θ(B) = 1 − θ1 B − · · · − θq B q
are outside the unit circle.

Proof.
Factorization: φ(B)Yt = θ(B)Zt = (1 − ξ1 B)(1 − ξ2 B) · · · (1 − ξp B)Zt
The roots of θ(x), x = ξ11 , . . . , ξ1p , are outside unit circle
 
⇔  ξ1k  > 1, all k = 1, . . . , p
 

⇔ |ξk | < 1, all k = 1, . . . , p

⇔ (1 − ξk B), all k = 1, . . . , p, can be “inverted” to the other side
⇔ Zt = (1 − ξ1 B)−1 · · · (1 − ξp B)−1 φ(B)Yt is represented by past data.
⇔ {Yt } is invertible.
ARMA Model (Time Series) Chapter 3 14 / 39
...
Exercises
Which of the following model(s) is(are) invertible? If yes, give the AR
representation up to the second term.
a) Yt = Zt + 0.6Zt−1 − 0.16Zt−2
b) Yt = Zt − 0.6Zt−1 − 0.72Zt−2
c) Yt = Zt − 0.5Zt−2 − 0.5Zt−4

ARMA Model (Time Series) Chapter 3 15 / 39

...
Example of ARMA(p, q) Model
Example 7
Consider the ARMA(1,1) model
i.i.d.
Yt − 0.5Yt−1 = Zt − 0.3Zt−1 , Zt ∼ W N (0, 1) .
1 Is the model causal/invertible?
P
2 What is the MA representation Yt = j≥0 ψj Zt−j ?

1 As |0.5| and |0.3| are less than 1, the model is causal and invertible.
2 MA representation:
(1 − 0.5B)Yt = (1 − 0.3B)Zt
⇒ Yt = (1 − 0.5B)−1 (1 − 0.3B)Zt
= (1 + 0.5B + 0.52 B 2 + · · · )(1 − 0.3B)Zt
= (1 + 0.5B + 0.52 B 2 + 0.53 B 2 + · · · )Zt
−(0.3B + 0.5(0.3)B 2 + 0.52 (0.3)B 3 + · · · )Zt
= (1 + 0.2B + 0.2(0.5)B 2 + 0.2(0.5)2 B 3 + · · · )Zt

∴ ψ0 = 1 , ψj = 0.2(0.5)j−1 , j = 1, 2, . . .
ARMA Model (Time Series) Chapter 3 16 / 39
...
Agenda

1 Introduction

2 Causality and Invertibility (AR and MA Representations)

3 Stationarity of ARMA models

4 Derive Autocovariance γ(k) via Yule-Walker equation

5 ARIMA model and Financial Time Series

6 Seasonal Model

7 Summary

ARMA Model (Time Series) Chapter 3 17 / 39

...
Simulation of ARMA models
ARMA model: Yt − φ1 Yt−1 − · · · − φp Yt−p = Zt − θ1 Zt−1 − · · · − θq Zt−q

R codes for simulating ARMA models:

sim=function(n=100,ar=c(0.5),ma=c(0.2,0.3)){
Z=rnorm(n); Y=rep(0,n)
p=length(ar); q=length(ma); m=max(p,q)
for (t in (m+1):n){
Y[t]=ar%*% Y[(t-1):(t-p)]+c(1,-ma)%*% Z[t:(t-q)]
}
ts.plot(Y)}

Try: par(mfrow=c(2,3))
sim(100,ar=-0.6); sim(100,ar=0.9); sim(100,ar=1.1)
sim(100,ma=10); sim(100,ar=0.9,ma=10); sim(100,ar=1.1,ma=0.2)
Build-in function in R:
arima.sim(model, n, ...)

e.g. arima.sim(n=63, list(ar=c(0.9, -0.5), ma=c(-0.2)), sd=2)

When is an ARMA(p, q) model process stationary?
ARMA Model (Time Series) Chapter 3 18 / 39
...
Stationarity
Lemma 8 (MA(∞): Stationarity)
∞ ∞
ψi Zt−i , Zt ∼ W N (0, σ 2 ) and |ψi | < ∞, then {Yt } is
P P
If Yt =
i=−∞ i=−∞
∞
σ2
P
weakly stationary with ACVF γ(k) = ψi ψi+k .
i=−∞

Proof.
Using the properties of WN, i.e., E(Zt ) = 0, Cov(Zt , Zk ) = σ 2 1{t=k} , we have
P∞
1 E(Yt ) = i=−∞
ψi E(Zt−i ) = 0
P∞ P∞
2 γ(k) = Cov(Yt , Yt+k ) = E(Yt , Yt+k ) = i=−∞ j=−∞
ψi ψj E(Zt−i Zt+k−j )
P∞ 2
= ψ ψ E(Zt−i
i=−∞ i i+k
) (nonzero only when j = i + k)
2
P ∞
=σ i=−∞
ψi ψi+k , independent of t

Therefore, {Yt } is weakly stationary.

P∞
Note that i=−∞
|ψi | < ∞ guarantees that
X∞ X∞
γ(k) = σ 2 ψi ψi+k < max |ψi |σ 2 |ψi | < ∞ .
i=−∞ i i=−∞

ARMA Model (Time Series) Chapter 3 19 / 39

...
Moving Average Model
∞ ∞
ψi Zt−i , Zt ∼ W N (0, σ 2 ) and |ψi | < ∞,
P P
Lemma: If Yt =
i=−∞ i=−∞
∞
then {Yt } is weakly stationary with ACVF γ(k) = σ 2
P
ψi ψi+k .
i=−∞

Example 9
Find the ACF of the MA(2) model: Yt = Zt + 0.6Zt−1 − 0.16Zt−2 ,
Zt ∼ W.N.(0, 2).

ARMA Model (Time Series) Chapter 3 20 / 39

...
ARMA model: Stationarity

Theorem 10 (Stationarity of ARMA model)

An ARMA(p, q) process φ(B)Yt = θ(B)Zt , Zt ∼ W N (0, σ 2 ), is weakly
stationary if it is causal, i.e., the roots of φ(B) are outside the unit circle.

ARMA Model (Time Series) Chapter 3 21 / 39

...
ARMA model: Stationarity
P∞ P∞
Lemma: If Yt = i=−∞
ψi Zt−i , i=−∞
|ψi | < ∞, then {Yt } is stationary.

Theorem: An ARMA(p, q) process φ(B)Yt = θ(B)Zt , Zt ∼ W N (0, σ 2 ), is weakly

stationary if it is causal

Proof of Theorem (Optional):

P∞ P∞
First consider the case (1 − φB)Yt = θ Z
k=0 k t−k
, where k=0
|θk | < ∞ and |φ| < 1

∞ ∞ ∞ ∞ ∞
X X X X X
Yt = (1 − φB)−1 θk Zt−k = φj B j θk Zt−k = θk φj Zt−k−j
k=0 j=0 k=0 j=0 k=0
∞ X
∞ ∞ i
!
i=k+j
X X X
= θk φi−k Zt−i 1{i≥k} = θk φi−k Zt−i .
i=0 k=0 i=0 k=0

Yt is stationary by Lemma, since

P∞ Pi  ∞ ∞   ∞ ∞   ∞
 θk φi−k  ≤ P P θk φi−k  1{k≤i} = P |θk | P φi−k  = 1
1−|φ|
P
|θk | < ∞

i=0 k=0
 i=0 k=0 k=0 i=k k=0

For general causal process, φ(B)Yt = (1 − ξ1 B) · · · (1 − ξp B)Yt = θ(B)Zt , all |ξi | < 1,
Yt = (1 − ξ1 B)−1 · · · (1 − ξp B)−1 θ(B)Zt is stationary by repeating the argument p times.
ARMA Model (Time Series) Chapter 3 22 / 39
...
ARMA process can be Stationary even if Not Causal
Proposition 11 (Stationarity of AR(1) process)
The process Yt = φYt−1 + Zt , Zt ∼ W N (0, σ 2 ) is stationary if |φ| =
6 1.
Proof (Optional):
A) For |φ| < 1, inverting the AR polynomial gives P∞
(1 − φB)Yt = Zt =⇒ Yt = (1 − φB)−1 Zt = k=0 φk Zt−k .
B) For |φ| > 1, inverting a “modified” AR polynomial gives ∞
−1
1 1 1
   X
−φB 1 − Yt = Zt =⇒ (−φB)Yt = 1− Zt = Zt+k
φB φB φk
k=0
∞
X 1
=⇒ Yt = − Zt+1+k
φk+1
k=0
P∞ P∞

(A) and (B) are stationary by Lemma: Yt =
i=−∞
ψi Zt−i with
−∞
|ψi | < ∞

But (B) is not useful: It depends on future noises (not causal)

Theorem 12 (Stationarity of ARMA(p, q) process)

The process φ(B)Yt = θ(B)Zt , Zt ∼ W N (0, σ 2 ) is stationary if no root of φ(x) is on
the unit circle.
ARMA Model (Time Series) Chapter 3 23 / 39
...
Agenda

1 Introduction

2 Causality and Invertibility (AR and MA Representations)

3 Stationarity of ARMA models

4 Derive Autocovariance γ(k) via Yule-Walker equation

5 ARIMA model and Financial Time Series

6 Seasonal Model

7 Summary

ARMA Model (Time Series) Chapter 3 24 / 39

...
Yule-Walker Equations
Causal ARMA process φ(B)Yt = θ(B)Zt , Zt ∼WN(0, σ 2 ) is
stationary as it can be expressed as
∞
X
Yt = ψj Zt−j ,
j=0
P∞
where j=0 |ψj | < ∞.

ACVF:
∞
X
γ(k) = σ 2 ψj ψj+k ,
j=0

is not explicitly related to the ARMA parameters {φj } and {θj }.

(Except pure MA models!)

Yule-Walker equations provide a systematic way to find explicit

formula for γ(k).
ARMA Model (Time Series) Chapter 3 25 / 39
...
Yule-Walker equation for deriving γ(k) of AR(p) model

AR(p) model: Yt = φ1 Yt−1 + · · · + φp Yt−p + Zt (2)

Yule-Walker Equations:
Find covariance of both sides of (2) with Yt
⇒ γ(0) = φ1 γ(1) + · · · + φp γ(p) + σ 2
Find covariance of both sides of (2) with Yt−1
⇒ γ(1) = φ1 γ(0) + φ2 γ(1) + · · · + φp γ(p − 1)
......
Find covariance of both sides of (2) with Yt−k
⇒ γ(k) = φ1 γ(k − 1) + · · · + φp γ(k − p)
......
Find covariance of both sides of (2) with Yt−p
⇒ γ(p) = φ1 γ(p − 1) + φ2 γ(p − 2) + · · · + φp γ(0)

Solve p + 1 equations for p + 1 unknowns for (γ(0), γ(1), ..., γ(p))

ARMA Model (Time Series) Chapter 3 26 / 39
...
Yule-Walker equation for deriving γ(k) of AR(p) model
Yule-Walker Equations:
γ(0) = φ1 γ(1) + φ2 γ(2) + · · · + φp γ(p) + σ 2
γ(1) = φ1 γ(0) + φ2 γ(1) + · · · + φp γ(p − 1)
γ(2) = φ1 γ(1) + φ2 γ(0) + · · · + φp γ(p − 2)
··· = ···
γ(p) = φ1 γ(p − 1) + φ2 γ(p − 2) + · · · + φp γ(0) .

In matrix form, we have the Yule-Walker Equations

    
1 −φ1 −φ2 · · · −φp γ(0) σ2
−φ1 1 − φ2 −φ3 · · · −φp−1  γ(1)  0 
    
 .  =  . 
 . .. .. .. 
 . ..    
 . . . . .   ..   .. 
−φp −φp−1 −φp−2 · · · 1 γ(p) 0
We can solve for γ(0), γ(1), . . . , γ(p) since φj s are known
For {γ(k)}k>p , compute recursively
γ(k) = φ1 γ(k − 1) + · · · + φp γ(k − p)
ARMA Model (Time Series) Chapter 3 27 / 39
...
Exercises
Find the ACVF of the following AR models with Zt ∼ WN(0, 1).
a) Yt − 0.4Yt−1 = Zt
b) Yt − 0.6Yt−1 − 0.27Yt−2 = Zt
c) Yt − 0.6Yt−1 − 0.72Yt−2 = Zt (Is the answer reasonable? Why?)

ARMA Model (Time Series) Chapter 3 28 / 39

...
Yule-Walker equation for ARMA(p, q) model
Yule-Walker Equations:
[Take Cov between Yt−k and Yt = φ1 Yt−1 + φ2 Yt−2 + · · · + φp Yt−p + θ(B)Zt ]
γ(0) = φ1 γ(1) + φ2 γ(2) + · · · + φp γ(p) + Cov(Yt , θ(B)Zt )
γ(1) = φ1 γ(0) + φ2 γ(1) + · · · + φp γ(p − 1) + Cov(Yt−1 , θ(B)Zt )
γ(2) = φ1 γ(1) + φ2 γ(0) + · · · + φp γ(p − 2) + Cov(Yt−2 , θ(B)Zt )
··· = ···
γ(p) = φ1 γ(p − 1) + φ2 γ(p − 2) + · · · + φp γ(0) + Cov(Yt−p , θ(B)Zt ) .
In matrix form, we have the Yule-Walker Equations
1 −φ1 −φ2 ··· −φp γ(0) Cov(Yt , θ(B)Zt )
    
−φ1 1 − φ2 −φ3 ··· −φp−1  γ(1) Cov(Yt−1 , θ(B)Zt )
  ...  = 
 . .. .. .. ..     .. 
 .. . . . . .

−φp −φp−1 −φp−2 ··· 1 γ(p) Cov(Yt−p , θ(B)Zt )

We can solve for γ(0), γ(1), . . . , γ(p) after finding Cov(Yt−j , θ(B)Zt )s.
For {γ(k)}k>p , compute recursively
γ(k) = φ1 γ(k − 1) + · · · + φp γ(k − p) + Cov(Yt−k , θ(B)Zt )
ARMA Model (Time Series) Chapter 3 29 / 39
...
i.i.d.
ARMA(1,1): Yt − 0.5Yt−1 = Zt − 0.3Zt−1 , Zt ∼ N (0, 1)
Q: What is the ACVF γ(k) of the process?
P
A: Method 1: Using MA representation Yt = j≥0 ψj Zt−j ,
ψ0 = 1, ψj = 0.2(0.5)j−1 .
 
X X X∞
γ(k) = Cov  ψj Zt−j , ψj Zt+k−j  = ψj ψj+k
j=0
j≥0 j≥0
 P∞
ψ0 ψk + j=1 ψj ψj+k , k 6= 0
= ∞
ψ02 + j=1 ψj2 ,
P
k=0
 k−1 2
P∞ 2j+k−2
0.2(0.5) + 0.2 j=1 0.5 , k 6= 0
= ∞
1 + 0.22 j=1 0.52j−2 ,
P
k=0
(
0.5k
0.2(0.5)k−1 + 0.22 1−0.5 2 , k 6= 0
= 0.2 2
1 + 1−0.52 , k=0
 17 k−1
75 (0.5) , k 6= 0
= 79
75 , k=0

ARMA Model (Time Series) Chapter 3 30 / 39

...
i.i.d.
ARMA(1,1): Yt − 0.5Yt−1 = Zt − 0.3Zt−1 , Zt ∼ N (0, 1)
Q: What is the ACVF γ(k) of the process?

A: Method 2: Yule-Walker Equation

Find covariance of both sides with Yt and Yt−1 , respectively
γ(0) − 0.5γ(1) = 1 − 0.3(0.5) + 0.32 = 0.94
γ(1) − 0.5γ(0) = −0.3
Solving the two equations gives
79 17
γ(0) = and γ(1) = .
75 75
For k > 1, find covariance of both sides with Yt−k gives
γ(k) − 0.5γ(k − 1) = 0
⇒ γ(k) 0.5γ(k − 1) = 0.52 γ(k − 2) = · · ·
=
17
= 0.5k−1 γ(1) = (0.5)k−1 ,
75
which agrees with the answer in Method 1.
ARMA Model (Time Series) Chapter 3 31 / 39
...
Agenda

1 Introduction

2 Causality and Invertibility (AR and MA Representations)

3 Stationarity of ARMA models

4 Derive Autocovariance γ(k) via Yule-Walker equation

5 ARIMA model and Financial Time Series

6 Seasonal Model

7 Summary

ARMA Model (Time Series) Chapter 3 32 / 39

...
ARIMA Model for Non-Stationary Time Series

ARIMA model: φ(B)(1 − B)d Yt = θ(B)Zt , Zt ∼WN(0, σ 2 )

Examples:
1 Linear Drift: Yt = α + βt + Zt

(1 − B)Yt = β + Zt − Zt−1 = β + (1 − B)Zt

{Yt } ∼ ARIMA(0,1,1)

2 Random Walk: Yt = Yt−1 + Zt

(1 − B)Yt = Zt

{Yt } ∼ ARIMA(0,1,0)

ARMA Model (Time Series) Chapter 3 33 / 39

...
Stock Price and ARIMA model
Notations:
Pt = Price of stock at the end of day t
Yt = log Pt = log(stock price)
Differenced log-price (1 − B)Yt can model daily return
By Taylor’s series expansion of f (x) = log x around x = 1:
1 1
log x = (x − 1) − (x − 1)2 + (x − 1)3 + · · ·
2 3
Therefore,
Pt Pt
(1 − B)Yt = Yt − Yt−1 = log = − 1 + ignorable terms
Pt−1 Pt−1
Pt − Pt−1
≈ = daily return , Zt
Pt−1

Empirical evidences:
Zt = (1 − B)Yt ∼ ARMA(p, q) model
log Pt = Yt follows ARIMA(p, 1, q) model
ARMA Model (Time Series) Chapter 3 34 / 39
...
Agenda

1 Introduction

2 Causality and Invertibility (AR and MA Representations)

3 Stationarity of ARMA models

4 Derive Autocovariance γ(k) via Yule-Walker equation

5 ARIMA model and Financial Time Series

6 Seasonal Model

7 Summary

ARMA Model (Time Series) Chapter 3 35 / 39

...
Seasonal ARIMA
Definition 13
{Yt } follows an SARIMA(p, d, q) × (P, D, Q)s model if

φ(B)ΦP (B s )(1 − B)d (1 − B s )D Yt = θ(B)ΘQ (B s )Zt

where
φ(B) = 1 − φ1 B − · · · − φp B p
θ(B) = 1 − θ 1 B − · · · − θq B q
ΦP (B s ) = 1 − Φ1 B s − · · · − ΦP B sP
ΘQ (B s ) = 1 − Θ1 B s − · · · − ΘQ B sQ

Example:
Yt = φ1 Yt−4 + φ2 Yt−8 + Zt for modeling quarterly data
Remarks:
SARIMA can be expressed as high order ARIMA model
But we love the physical interpretation about seasonal effect
ARMA Model (Time Series) Chapter 3 36 / 39
...
Exercises
Identify the order of the following Seasonal-ARIMA models
a) Yt − 0.2Yt−1 − 0.3Yt−4 + 0.06Yt−5 = Zt + 0.6Zt−4 − 0.16Zt−8
b) Yt + 0.2Yt−1 − Yt−6 − 0.2Yt−7 = Zt + 0.2Zt−6
c) Yt − Yt−1 + 0.2Yt−6 − 0.2Yt−7 = Zt + 0.2Zt−6

ARMA Model (Time Series) Chapter 3 37 / 39

...
Agenda

1 Introduction

2 Causality and Invertibility (AR and MA Representations)

3 Stationarity of ARMA models

4 Derive Autocovariance γ(k) via Yule-Walker equation

5 ARIMA model and Financial Time Series

6 Seasonal Model

7 Summary

ARMA Model (Time Series) Chapter 3 38 / 39

...
Summary
ARIMA model: φ(B)(1 − B)d Yt = θ(B)Zt , Zt ∼ W N (0, σ 2 )

Stationarity, Causality and Invertibility -

Characteristic roots of φ(x) = 0, θ(x) = 0 lie outside unit circle
P
Find Causal/MA representation Yt = j≥0 ψj Zt−j

P
Find AR representation Zt = j≥0 ψj Yt−j

Find ACVF γ(k)

P
(M1) Use causal representation Yt = j≥0 ψj Zt−j
(M2) Yule-Walker equations:
1 Cov(Yt−k , Yt ) = Cov(Yt−k , φ1 Yt−1 + · · · + φp Yt−p + θ(B)Zt ) for
0≤k≤p
2 solve for γ(k)s.
SARIMA model:
φ(B)ΦP (B s )(1 − B)d (1 − B s )D Yt = θ(B)ΘQ (B s )Zt
ARMA Model (Time Series) Chapter 3 39 / 39