Statistics 5350/7110

Forecasting

Lecture 19
Seasonal ARIMA Models

Professor Robert Stine

 Admin Issues
• Questions

• Assignments

• Quick review
• Building ARIMA models in pr ctice
• Filling in some det ils: Lecture_18.Rmd
- Simil rity of the models for GDP
- Model drift
- De ling with n outlier
• IMA models s v ri tions on exponenti l smoothing (next slide)

2
a
a
a
a
a
a
a
a
a
 Review: IMA Model as Average
• Exponenti l smoothing
• Equiv lent to IMA(1,1) if tre ted s forec sting procedure (i.e., with time shift)
• Forec sts re weighted ver ge of p st
X̂n+1|n = wj = 1, wj = (1 − λ) λ j
∑ ∑
wj Xn−j where
j=0 j=0
• All ARIMA(0,1,q) models h ve this gener l form:
One-step he d prediction is weighted ver ge of prior v lues

• Deriv tion
• ARIMA(0,1,q) h s the form
(1 − B)Xt = wt + θ1wt−1 + ⋯ + θqwt−q or (1 − B) Xt = θ(B) wt
• As n in inite AR model
(1 − B)θ(B)−1 Xt = wt π(B) = 1 + π1B + π2B 2 + ⋯ so th t X̂n+1|n = −
∑
where πj Xn+1−j
j=1
π(B)

• Invertibility implies π(1) = 0 so th t π1 + π2 + ⋯ = − 1.

3
a
a
a
a
f
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
 Today’s Topics
Text, §5.3

• Se son l ARIMA models (SARIMA) Expl ins the n me of the

“s rim ” functions in R
• P rsimonious represent tion for e iciency
• S me s ARIMA model with cert in coe icients constr ined to be zero
• Not new model, just convenient w y to express constr ints

• Pure nd mixed types

• Pure: Non-zero correl tions t multiples of speci ic period
• Mixed: Combine two types of dependence

• Relev nce
• Just bout every m croeconomic time series
you’ll ind h s been se son lly djusted.
• SARIMA models llow you to incorpor te th t
djustment into YOUR model.
4
a
a
a
a
a
a
a
a
a
f
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
ff
a
a
ff
a
f
Seasonal ARIMA Models
 Purely Seasonal ARIMA Model
• Nonzero utocorrel tions t multiples of se son l period
• Qu rterly d t : utocorrel tions t 4 months
• Monthly d t : utocorrel tions t 12 months

• Ex mple
• Simul ted for cl rity (these re r re 😁) Ex mple 5.11
Figure 5.9
• Regul r p ttern from ye r to ye r:
Febru ry routinely higher
April routinely lower
• Nonzero utocorrel tions t se son l sp cing

• Model
• Not tion: monthly process (S=12) SARIMA(1,0,0)12
Xt = Φ Xt−12 + wt or (1 − ΦB 12)Xt = wt
• Equiv lent to AR(12) model with ll of the intervening
coe icients φ1, φ2, … φ11 = 0.
• St tion rity requires |Φ| < 1 s in usu l AR models 6
a
a
a
a
a
ff
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
 Purely Seasonal ARIMA Model
• Se son l AR(1) model
• Speci ic tion is Xt = Φ Xt−12 + wt
• P tterns in ACF nd PACF
• Why re the intervening correl tions zero?
Ex mple 5.11
• In inite MA represent tion Figure 5.9
Xt = Xt 12 + wt
<latexit sha1_base64="ef41Eq+bTTBlVZAtXQWgc9PXiJw=">AAADBniclZJPb9MwGMadbMAof9aNIxdrFVOnQZpkpVuRJk1w4VgkulVaushx3M5t4kS2w1ZFufM1+AKcNu26G1c48m1w/rSFCQ68UqTXz/s8P+mN7cUBFdI0f2r6yuq9+w/WHtYePX7ydL2+sXksooRj0sdREPGBhwQJKCN9SWVABjEnKPQCcuJN3+Xzk0+ECxqxj3IWk2GIxoyOKEZSSe6G9sbxyJiyFHGOZlnKcZDBGoQDV8Ltw23o9M6pOqTylWVncBdeKN3BXDnymjuaS5vdLm1lYuevkVzZLYLOy4W1Us7sZjUocHudBU6Rd/4LtIjNlb0l884OIgnddHJoZmcOZSM5K/2pZUMFnmRz9iSrOYT51c9y6w3T6JpW9/U+LJuDdtV0utAyzKIaoKqeW79y/AgnIWESB0iIU8uM5VDRJMUBUexEkBjhKRqTU9UyFBIxTItLzuALpfhwFHH1MQkL9fdEikIhZqGnnCGS5+LuLBeXM8riRKatvlBvo+VFXusi4tNWjOL8LCRiPuK+IcllVlNrzneB/26ObcPqGJ0P7cbR22rhNfAcbIEmsMA+OALvQQ/0Ada+aN+079oP/bP+Vb/Wb0qrrlWZZ+CP0m9/AXdG38A=</latexit>

= ( Xt 24 + wt 12 ) + wt
= wt + wt 12 + 2 ( Xt 36 + wt 24 )
= w
Pt 1+ wt 12 + 2 wt 24 + 3 Xt 36
12 j
= j=0 wt 12j
• Nonzero ACF t sp cing given by se son l period
• PACF: zero once we know v lue 12 months previous
• Expl ins st tion rity condition

7
a
a
f
a
a
a
f
a
a
a
a
a
a
a
a
a
a
a
a
 Mixed Seasonal ARIMA Model Ex mple 5.12

• Autocorrel tions t m ny l gs
• Never so “pure” s in prior ex mples, with nothing but few spikes
• We observe estim tes r ther th n true utocorrel tions

• Ex mple
• Simul ted for cl rity (so we know the process ACF)
• MA(1) combined with SAR(1)
• Nonzero utocorrel tions cluster ne r se son l period

• Not tion
• Monthly process (S=12) Not tion uses “c pit lized”
letters for the se son l
Xt = Φ Xt−12 + wt + θwt−1 p r meters

• B ckshift polynomi l form

(1 − ΦB 12)Xt = θ(B) wt
• Equiv lent to n ARMA(12, 1) model with ll of the intervening coe icients φ1, φ2, … φ11 = 0.

8
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
 Mixed Seasonal ARIMA Model Ex mple 5.12

• Process
• Monthly process (S=12) Xt = Φ Xt−12 + wt + θwt−1
• Autocorrel tions
• V ri nce is e sy to ind since the v ri bles on the right side re uncorrel ted
2 2 1 + θ2 2
Var(Xt) = γ(0) = Φ γ(0) + (1 + θ )σw2 ⇒ γ(0) = σw
1−Φ 2
• Compute utocov ri nces s in Yule-W lker equ tions
γ(1) = Φγ(11) + θ σw2 γ(12) = Φγ(0) γ(11) = γ(13) = Φγ(1)
Hence
θ θ
γ(1) = Φ2γ(1) + θσw2 ⇒ γ(1) = σw2 and γ(11) = γ(13) = Φ σw2
1−Φ2 1−Φ 2
• Autocorrel tions
θ θ
ρ(1) = γ(1)/γ(0) = and ρ(11) = ρ(13) = γ(11)/γ(0) = Φ
1 + θ2 1+θ 2
In gener l
θ
ρ(12h) = Φh, ρ(12h ± 1) = Φh
, h = 1,2,…
1 + θ2 9
a
a
a
a
a
a
a
a
a
f
a
a
a
a
a
a
 Mixed Seasonal ARIMA Model
• Underst nding p tterns in ACF
• Model is
Xt = Φ Xt−12 + wt + θwt−1
• P tterns in ACF
• In inite MA represent tion sheds more light Ex mple 5.12
Figure 5.10
Xt = Xt 12 + wt + ✓wt 1
<latexit sha1_base64="XH5QpuQ7n5M0fboSN3Ns+QICMHI=">AAAC1XicjZFNbxMxEIa9Wz7a8JWWI5cREVVRymY3hJQckCq4cAwSaSLVIfJ63cTKrteyHUq08g1x5cYVfgR/hn+DN1kIqUBiJEsz78z7WKOJZcq1CcMfnr9z7fqNm7t7tVu379y9V98/ONP5QlE2oHmaq1FMNEu5YAPDTcpGUjGSxSkbxvNXZX/4ninNc/HWLCUbZ2Qq+AWnxDhpsu/t4ZhNuSiIUmRpC0VTCzWA0cTA4YtDwP0Zd0VhnkRtC024dHoTsJkxQ1xR6hYwVc5SRmn560hzRTra4NqdNe43ecvw1D6G/8duYVyNj6/AKv1dG/Dx5vuSj5lIqt0n9UYY9MKo9+wE1snzTpV0exAF4SoaqIr+pP4dJzldZEwYmhKtz6NQmrGjGU5TZmt4oZkkdE6m7NylgmRMj4vVzSw8ckoCF7lyTxhYqX86CpJpvcxiN5kRM9NXe6W46XEhF6ZoDbQ7dSvO49ZlruYtSWRZa0NEQlQSGPbB1tyav3aBfydn7SDqBt03ncbpy2rhXfQAPURHKEIn6BS9Rn00QNST3hfvq/fNH/rW/+h/Wo/6XuW5j7bC//wTsdnSsA==</latexit>

= wt + ✓wt 1 + ( Xt 24 + wt 12+ ✓wt 13 )

2
= wt + ✓wt 1 + wt 12 + ✓wt 13 + Xt 24
• Nonzero weights t multiples of se son l period nd
ne r the se son l period
• Nonzero utocorrel tions symmetric round se son l

• PACF
• Condition l utocorrel tions less obvious
• Intuition from non-se son l models
10
a
a
f
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
 Seasonal ARIMA, SARIMA
• Not tion
• C pit l letters for the se son l terms, with se son l period S
• Write model using b ckshift not tion
• SARIMA(p,0,q)(P,0,Q)S
(1 − Φ1B S − ⋯ − ΦPB PS)(1 − ϕ1B − ⋯ − ϕpB p)Xt = (1 + Θ1B S + ⋯ + ΘQB QS)(1 + θ1B + ⋯ + θqB p)wt
or in more comp ct form
Φ(B S) ϕ(B) Xt = Θ(B S) θ(B) wt
• Constr ined multiplic tive structure
• SARIMA(1,0,1)(1,0,1)4 (se son l period S=4)
(1 − Φ1B 4)(1 − ϕ1B)Xt = (1 + Θ1B 4)(1 + θ1B)wt
• Equiv lent to ARIMA(5,5) with constr ined estim tes (estim te 4 p r meters, not 10)
(1 − ϕ1B − Φ1B 4 + ϕ1Φ1B 5) Xt = (1 + θ1B + Θ1B 4 + θ1Θ1B 5) wt
• Di erencing De inition 5.13

• SARIMA(p,d,q)(P,D,Q)S llows di erencing Φ(B S)ϕ(B) (1 − B S)D(1 − B)d Xt = Θ(B S)θ(B) wt

11
a
f
ff
a
a
a
a
a
a
a
a
a
a
a
a
ff
a
a
a
a
 Example: Births
Ex mple 5.12
a
 Example: Mixed Seasonal Model
• Birth series
• Live births, monthly in the US from 1948 - 1979

Post-w r b by boom

How would you h ve

modeled this process
before this cl ss?

13
a
a
a
a
 Example: Mixed Seasonal
• Birth series
• Persistent utocorrel tions suggest better to model di erences (non-st tion ry)

14
a
a
 Example: Mixed Seasonal
• Birth series
• Month-to-month di erences in the live births, monthly in the US from 1949 - 1979

15
ff
 Example: Mixed Seasonal
• P r meters re signi ic nt, but subst nti l utocorrel tion rem ins
• Autocorrel tion in residu ls t l g 12 Interpreting the process l bel in the igure

(1 − ΦB 12) ∇Xt = (1 − θB)wt

16
a
a
a
a
a
f
a
f
a
a
a
 Example: Mixed Seasonal
• Exp nd model
• ARMA(1,1) plus se son l ARMA(1,1) (1 − ϕB)(1 − ΦB 12) ∇Xt = (1 − θB)(1 − ΘB 12)wt
• Se son l AR coe icient ne r bound ry of st tion rity

17
a
a
a
a
ff
a
a
a
a
 Example: Mixed Seasonal
• Se son l di erence
(1 − ϕB)(1 − B 12) ∇Xt = (1 − θB)(1 − ΘB 12)wt
• Di erence t se son l sp cing r ther th n
it the AR term t se son l sp cing (1 − ϕB) ∇12 ∇Xt = (1 − θB)(1 − ΘB 12)wt

`s rim ` does not provide const nt/me n when

both se son l nd regul r di erencing

18
f
a
ff
a
a
a
a
a
a
a
a
ff
a
a
a
a
ff
a
a
a
a
a
a
a
 Example: Forecasts of Live Births
• Non-trivi l extr pol tion
• Contr st to simple forec st evolution with st tion ry nd ARIMA models

19
a
a
a
a
a
a
Example: CO2 Levels
 CO2 Levels Ex mple 5.15

• CO2 me sured t M un Lo Observ tory

• Monthly, from M rch 1958 through M rch 2023
• De inition
Dry mole fr ction de ined s the number of
molecules of c rbon dioxide divided by the
number of molecules of dry ir multiplied by
one million (ppm)

• Discussion
• Regul r oscill tion
• F ster growth in l ter d t
• Cert inly non st tion ry

21
a
a
f
a
a
a
a
a
a
a
a
a
f
a
a
a
a
a
a
a
a
a
a
 Persistent Seasonality
• Regul r p ttern
• Periodic plot
• Center v lues for 12 months
• Plot versus month.

• Colors
• Dintinguish curves over time
• Viridis p lette
d rk purple -> yellow

• Interpret tion
• Deeper Sep-Oct dip in recent ye rs

22
a
a
a
a
a
a
a
 Modeling: Seasonal Differences
• Se son l di erences
• Plot shows (1-B12) Xt
• Evident upw rd trend consistent with
growth r te in initi l sequence plot.

23
a
a
a
a
ff
a
 Modeling: Seasonal Differences
• Di erence the se son l di erences
• Plot shows (1-B)(1-B12) Xt
• This second round of di erencing t kes
c re of the ch nging slope noticed in
the origin l sequence plot

• Loess smooth
• Visu lly con irm bsence of trend
• Import nt since `s rim ` will not it
const nt term in presence of two l yers
of di erencing.

24
a
ff
a
ff
a
a
a
f
a
a
a
a
a
ff
a
a
f
a
ff
a
a
 Modeling: ACF and PACF
• Model selection
• Plot shows ACF nd PACF of (1-B)(1-B12) Xt
• Could use “se son l version” of the
it_ rm _models function to help choosing
p nd q.

• Comments
• ACF h s l rge terms t l gs 1 nd 12
• PACF shows gr du l dec y t se son l
sp cing nd perh ps t the origin.

• Initi l choice of model?

• SARIMA(0,1,1)(0,1,1)12
(1 − B 12)(1 − B)Xt = (1 − θB)(1 − ΘB 12)wt

25
f
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
 Modeling: Estimated Model
• Fitted model
• Text dds nother p r meter, but doesn’t dd much.

26
a
a
a
a
a
 Predictions
• Comp re to ctu l v lues
• Use d t from 2020-2023 to ev lu te model
• Note the n rrow width of the prediction interv ls
Prediction interv ls for 2020

27
a
a
a
a
a
a
a
a
a
a
a
 What’s next?
• More ex mples

• Comp rison to regression se son l models

• Rigid vs luctu ting se son l p tterns
• Incorpor ting c lend r e ects

28
f
a
a
a
a
a
a
a
ff
a
a
a
a