Statistics 3B (STA03B3)

STOCHASTIC PROCESSES

Lecture 19

Chapter 3 → ARIMA Models (Part 6)

Dr V. van Appel
Department of Statistics
Faculty of Science, University of Johannesburg

1 / 23
Outline

3.9 Multiplicative Seasonal ARIMA Models

Seasonal ARIMA Models
A multiplicative seasonal autoregressive moving average model

2 / 23
Seasonal ARIMA Models
Examples:
▶ In this section, we introduce several modifications made to the
ARIMA model to account for seasonal and nonstationary
behaviour.
▶ Often, the dependence on the past tends to occur most
strongly at multiples of some underlying seasonal lag S.
▶ For example, with monthly economic data, there is a strong
yearly component occurring at lags that are multiples of
S = 12, because of the strong connections of all activity to
the calendar year.
▶ Data taken quarterly will exhibit the yearly repetitive period at
S = 4 quarters. Natural phenomena such as temperature also
have strong components corresponding to seasons.
▶ Hence, the natural variability of many physical, biological, and
economic processes tends to match with seasonal fluctuations.
3 / 23
Pure Seasonal Models

Pure seasonal autoregressive moving average model

The resulting pure seasonal autoregressive moving average model,
say, ARMA(P, Q)S , then takes the form

ΦP (B S )xt = ΘQ (B S )wt (1)

where the operators

ΦP (B S ) = 1 − Φ1 B S − Φ2 B 2S − · · · − ΦP B P S (2)

and
ΘQ (B S ) = 1 + Θ1 B S + Θ2 B 2S + · · · + ΘQ B QS (3)
are the seasonal autoregressive operator and the seasonal moving
average operator of orders P and Q, respectively, with seasonal
period S.

4 / 23
Analogous to the properties of nonseasonal ARMA models,
the pure seasonal ARMA(P, Q)S is causal only when the roots
of ΦP (z S ) lie outside the unit circle, and it is invertible only
when the roots of ΘQ (z S ) lie outside the unit circle.

5 / 23
Example 3.46 (A Seasonal AR Series)
A first-order seasonal autoregressive series that might run over
months could be written as

1 − ΦB 12 xt = wt

or
xt = Φxt−12 + wt

▶ This model exhibits the series xt in terms of past lags at the

multiple of the yearly seasonal period S = 12 months.
▶ It is clear from the above form that estimation and forecasting
for such a process involves only straightforward modifications
of the unit lag case already treated. In particular, the causal
condition requires |Φ| < 1.

6 / 23
 We simulated 3 years of data from the model with Φ = 0.9,
and exhibit the theoretical ACF and PACF of the model. See
the following Figure.

▶ Generate a data from the pure seasonal model (S = 12)

AR(1) which is given by

xt = 0.9xt−12 + wt

▶ The true ACF and PACF of this model is given as follows:

7 / 23
> # Plot the ACF and PACF
> set.seed(666)
> phi=c(rep(0,11),0.9)
> sAR=arima.sim(list(order=c(12,0,0),ar=phi),n=37)
> sAR=ts(sAR,freq=12)
> layout(matrix(c(1,1,2,1,1,3), nc=2))
> par(mar=c(3,3,2,1), mgp=c(1.6,.6,0))
> plot(sAR, axes=FALSE, main=’seasonal AR(1)’,xlab="year",
+ type=’c’)
> Months=c("J","F","M","A","M","J","J","A","S","O","N","D")
> points(sAR, pch=Months, cex=1.25, font=4, col=1:4)
> axis(1,1:4); abline(v=1:4, lty=2, col=gray(.7))
> axis(2); box()
> ACF = ARMAacf(ar=phi, ma=0, 100)
> PACF = ARMAacf(ar=phi, ma=0, 100, pacf=TRUE)
> plot(ACF,type="h", xlab="LAG", ylim=c(-.1,1)); abline(h=0)
> plot(PACF, type="h", xlab="LAG", ylim=c(-.1,1)); abline(h=0)

8 / 23
 seasonal AR(1)
J J
J
J J
J
D D

2
D
A

A AS A
0 S A
A
sAR

N
F O F S
F O
M N M M N
−2

M M
M J O
J
−4

J
J
1 2 3 4
year
0.8

0.8
PACF
ACF
0.4

0.4
0.0

0.0

0 20 40 60 80 100 0 20 40 60 80 100
LAG LAG

9 / 23
For the first-order seasonal (S = 12) MA model,

xt = wt + Θwt−12 ,

▶ Show that γ(0) = (1 + Θ2 )σ 2

γ(0) = E x2t
 

= E [(wt + Θwt−12 )(wt + Θwt−12 )]

= E wt2 + Θ2 wt−12
2
 
2
= σw + Θ 2 σw
2

= (1 + Θ2 )σw
2

10 / 23
 2
▶ Show that γ(±12) = Θσw

γ(+12) = E [(wt+12 + Θwt )(wt + Θwt−12 )]

= E Θwt2
 
2
= Θσw
2 . Thus, γ(±12) = Θσ 2
Similarly, γ(−12) = Θσw w

11 / 23
▶ Show that ρ(±12) = Θ/(1 + Θ2 ).

γ(±12)
ρ(±12) =
γ(0)
Θσw2
=
(1 + Θ2 )σw
2

= Θ/(1 + Θ2 ).

The invertible condition requires Θ < 1.

12 / 23
 Table: Behavior of the ACF and PACF for Pure SARMA Models
AR(P )S MA(Q)S ARMA(P, Q)S
ACF* Tails off at lags Cuts off after Tails off at
ks, k = 1, 2, . . . lag Qs lags ks
PACF* Cuts off after Tails off at lags Tails off
lag P s ks, k = 1, 2, . . . at lags ks

*The values at nonseasonal lags h ̸= ks, for k = 1, 2, . . ., are zero.

13 / 23
A multiplicative seasonal autoregressive moving
average model

A multiplicative seasonal autoregressive moving average model

In general, we can combine the seasonal and nonseasonal operators
into a multiplicative seasonal autoregressive moving average
model, denoted by ARMA(p, q) × (P, Q)s , and write

ΦP (B s )ϕ(B)xt = ΘQ (B s )θ(B)wt (4)

as the overall model.

In fitting such models, focusing on the seasonal autoregressive and
moving average components first generally leads to more
satisfactory results.

14 / 23
Example 3.47 (A Mixed Seasonal Model)
Consider an ARMA(0, 1) × (1, 0)12 model

xt = Φxt−12 + wt + θwt−1 ,

where |Φ| < 1, and the invertible condition requires |θ| < 1.
Then, because xt−12 , wt and wt−1 are uncorrelated, and xt is
stationary,

γ(0) = var(xt )
= var (Φxt−12 + wt + θwt−1 )
= Φ2 var(xt−12 ) + var(wt ) + θ2 var(wt−1 )
= Φ2 γ(0) + σw
2
+ θ2 σw
2

1 + θ2 2
=⇒ γ(0) = σ
1 − Φ2 w

15 / 23
Example 3.47 (A Mixed Seasonal Model) Cont ...
In addition, multiplying the model by xt−h , h > 0, and taking
expectations, we have

E [xt xt−h ] = ΦE [xt−12 xt−h ] + E [wt xt−h ] + θE [wt−1 xt−h ]

For h = 1, we have

E [xt xt−1 ] = ΦE [xt−12 xt−1 ] + E [wt xt−1 ] + θE [wt−1 xt−1 ]

2
γ(1) = Φγ(11) + θσw

For h ≥ 2, we have

E [xt xt−h ] = ΦE [xt−12 xt−h ] + E [wt xt−h ] + θE [wt−1 xt−h ]

γ(h) = Φγ(h − 12)

16 / 23
Example 3.47 (A Mixed Seasonal Model) Cont ...
The ACF for this model is
γ(12h)
ρ(12h) = = Φh h = 1, 2, · · ·
γ(0)
γ(12h − 1) θ
ρ(12h + 1) = ρ(12h − 1) = = Φh h = 0, 1, 2, · · ·
γ(0) 1 + θ2
ρ(h) = 0, otherwise.

▶ The ACF and PACF for this model, with Φ = 0.8 and
θ = −0.5, are shown in the following Figure.
▶ These type of correlation relationships, although idealized
here, are typically seen with seasonal data. Use the following
commands to produce the plots in R.

17 / 23
ACF and PACF of ARIMA(0, 0, 1) × (1, 0, 0)S=12
The ACF and PACF for this mixed model:

xt = 0.8xt−12 + wt − 0.5wt−1

▶ Seasonal AR(1): Value this month is related to last year’s

value xt−12
▶ MA(1): This month’s value related to last month’s shock
wt−1

18 / 23
> ##### Plot the ACF and PACF
> phi = c(rep(0,11),.8)
> ACF = ARMAacf(ar=phi, ma=-.5, 50)[-1] #[-1] removes 0 lag
> PACF = ARMAacf(ar=phi, ma=-.5, 50, pacf=TRUE)
> par(mfrow=c(1,2))
> plot(ACF, type="h", xlab="LAG", ylim=c(-.4,.8));
+ abline(h=0)
> plot(PACF, type="h", xlab="LAG", ylim=c(-.4,.8));
+ abline(h=0)

19 / 23
 ACF

−0.4 −0.2 0.0 0.2 0.4 0.6 0.8

LAG
10 20 30 40 50
PACF

−0.4 −0.2 0.0 0.2 0.4 0.6 0.8

LAG
10 20 30 40 50

20 / 23
Homework Problems

Additional Problems
For the following seasonal models, derive the autocorrelation
functions:
1.
ARIMA(0, 0, 1) × (0, 0, 2)12 .
2.
ARIMA(0, 0, 1) × (0, 0, 1)4 .

21 / 23
References I

R H. Shumway and D S. Stoffer.

Time Series Analysis and Its Applications: With R Examples.
Fourth Edition, Springer, 2016.

22 / 23
Questions?

23 / 23