Stationary Time Series

Firmin Doko Tchatoka

[email protected]
https://www.adelaide.edu.au/directory/firmin.dokotchatoka
The University of Adelaide

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 1 / 26

 Objectives of this lecture

Objectives of the chapter

H Study the properties of a class of parametric univariate time series models:

ARIMA = AR/I/MA
- AR ≡ Autoregressive

- I ≡ Integrated

- MA ≡ Moving Average

I General characterization of stationarity

I Stationarity in ARIMA framework: unit root testing

I Properties of a stationary time series: autocorrelation & partial

autocorrelation functions

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 2 / 26

 Stationarity

Strict stationarity

- Let {Yt } be a stochastic process

Definition
Let FY (yt1+τ , . . . , ytn +τ ) represent the cumulative distribution function of the
unconditional joint distribution of {Yt } at times t1 + τ, . . . , tn + τ . Then, {Yt } is said to
be strictly stationary (or strongly stationary/strict-sense stationary) if

FY (yt1+τ , . . . , ytn +τ ) = FY (yt1 , . . . , ytn ) for all τ, t1, . . . , tn ∈ R and for all n ∈ N (1)

- Example 1. Let X be any scalar random variable, and define a time-series {Yt } :

Yt = X for all t. (2)

Then {Yt } is a strictly stationary time series.

- Example 2. Let X have a uniform distribution on (0, 2π] and define

Yt = cos(t + X ) for t ∈ R. (3)

Then {Yt } is strictly stationary
Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 3 / 26
 Stationarity

Weak stationarity

- Let {Yt } be a stochastic process and let mY (t) = E(Yt ) and

KYY (t1, t2) = E[(Yt1 − mY (t1))(Yt2 − mY (t2))] be its mean and autocovariance
functions respectively.
Definition
{Yt } is said to be weakly stationary if
1 mY (t) = mY (t + τ ) for all τ ∈ R

2 KYY (t1, t2) = KYY (t1 − t2, 0) for all t1, t2 ∈ R

3 E[Yt2] < ∞ for all t ∈ R.

- Remarks:
∗ First property ⇒ mY (t) must be constant

∗ Second property ⇒ KYY (t1, t2) depends only on the difference between t1 and t2 ≡ only
need to be indexed by one variable (i.e., τ = t1 − t2) rather than two variables.

∗ Third property says that the second moments must be finite for any time t.
Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 4 / 26
 ARIMA Processes AR models

AR processes

I A time series yt ∼ AR(p) ≡ linear regression of yt on a constant and first p lags

of yt :

yt = φ0 + φ1yt−1 + φ2yt−2 + . . . + φp yt−p + εt

Q(L)yt = φ0 + εt , Q(L) = 1 − φ1L − . . . − φp Lp , (4)

− Q(L) ≡ lag polynomial

− εt ≡ error term ≡ i.i.d. with E(εt ) = 0 and E(ε2t ) = σ 2 (≡ White Noise)

− Parameters φ’s and σ 2 are unknown

− Lag length p is also unknown: we only have data y1, y2, . . . , yT

− AR(1): yt = φ0 + φ1yt−1 + εt , AR(2): yt = φ0 + φ1yt−1 + φ2yt−2 + εt

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 5 / 26

 ARIMA Processes AR models

AR processes (cont’d)

I Stationarity of an AR(p) → characterized by looking at the roots of polynomial:

Q(z) = 1 − φ1z − φ2z 2 . . . − φp z p , z ∈ C (5)

- First, solve Q(z) = 0 to get the p roots z1, z2 . . . , zp (solution always exists in C)

• yt is weakly stationary (or S2) if all the p roots z1 , z2 . . . , zp of Q(z) = 0 lie outside the Unit Circle
≡ {z ∈ C : |z| = 1}

• Otherwise, yt is not weakly stationary

• yt has (contains) a unit root if at least one of p roots z1 , z2 . . . , zp lies on the Unit Circle. yt is integrated of
order d if it has d unit roots ≡ yt ∼ I(d)

√
- Note: z ∈ C ⇒ z = a + ib, where i = −1. C ≡ complex set, a = <(z) ≡ real part of z,
√
b = =(z) ≡ imaginary part of z. Modulus of z is: |z| = a2 + b2. We can always write z as:
√
z = Re where R ≡ |z| = a2 + b2 (circle radius) eiθ = cos(θ) + i sin(θ), cos(θ) = Ra and
iθ

sin(θ) = Rb .
Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 6 / 26
 ARIMA Processes Moving Average (MA) processes

MA processes

I A time series yt ∼ MA(q) if

yt = θ0 + εt + θ1εt−1 + θ2εt−2 + . . . + θq εt−q = θ0 + Θ(L)εt (6)

− εt ∼ WN(0, σ 2), Θ(L) = 1 + θ1L + . . . + θq Lq ≡ lag polynomial

− parameters θ0, θ1, θ2, . . . , θq , σ 2 and current + lagged errors: unknown

− Lag length q is also unknown ⇒ must be estimated

− MA(1): yt = θ0 + εt + θ1εt−1; MA(2): yt = θ0 + εt + θ1εt−1 + θ2εt−2

− MA(q)≡ linear regression of yt on a constant and first q lags of error term:

cannot run OLS because independent (explanatory) variables are unobserved

− Any MA process with εt ∼ WN(0, σ 2) is weakly stationary

− yt is invertible if all the q roots z1, z2 . . . , zq of Θ(z) = 0 lie outside the Unit
Circle: any invertible MA (q) can be written as AR(∞).
Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 7 / 26
 ARIMA Processes ARIMA processes

ARIMA(p,d,q)

I A time series yt ∼ ARIMA(p, d, q) if its dth difference ∆d yt ∼ ARMA(p, q) :

− yt contains d unit roots≡ yt ∼ I(d)

− ∆d yt ∼ I(0) (second-order stationary):

Φ(L)yt = µ + Θ(L)εt , (7)

− µ ≡ drift of the process, Φ(L)yt ≡ AR part, Θ(L)εt ≡ MA part

− Can include deterministic trend in (7)

− Stationarity of yt is the property of the AR part (all p roots of Φ(z) = 0 lie
outside the Unit Circle), while invertibility is the property of the MA part (all
q roots of Θ(z) = 0 lie outside the Unit Circle).

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 8 / 26

 Unit root processes

Unit roots

I A time series yt has a unit root if |φ1| = 1 in the regression

yt = φ0 + φ1yt−1 + εt

- If yt contains exactly two unit roots, then the first-difference ∆yt = yt − yt−1 contains one unit
root

- A time series yt is integrated of order d if it contains exactly d unit roots: yt ∼ I(d)

- A time series yt is weakly stationary if it does not contain a unit root, i.e., yt ∼ I(0)

- Unit root is usually referred to as stochastic trend

- yt is a random walk if it contains a stochastic trend:

yt = |{z}
β0 + β1t
|{z} +yt−1 + εt
drift deterministic trend

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 9 / 26

 Unit root processes Testing for unit roots

Testing for unit roots

I Dickey-Fuller (DF) original test for unit roots involves fitting the AR(1) model:

yt = α + δt + φyt−1 + εt (8)

− Null hypothesis of unit root is: H0 : φ = 1

− Regression (8) is likely to be plagued by serial correlation

− To control for that, the augmented Dickey-Fuller (ADF) test instead fits a
model of the form

∆yt = α + δt + ρyt−1 + ζ1∆yt−1 + . . . + ζk ∆yt−k + ut (9)

where ∆yt−h = yt−h − yt−h−1 for any h ∈ N, ρ = φ − 1 → H0 : ρ = 0 v.s. H1 : ρ < 0

− H1 : ρ < 0 is chosen as the case ρ > 0 is unlikely

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 10 / 26

 Unit root processes Testing for unit roots

Testing for unit roots (cont’d)

- We must consider one of the four cases for H0:

Case Process Restriction DF options

1 Random walk without drift α = 0, δ = 0 noconstant
2 Random walk without drift δ=0 default
3 Random walk with drift δ=0 drift
4 Random walk with drift & trend none trend

- Require choosing the optimal k in (9) → Command: ‘varsoc varname’ → use

SBIC/HQIC

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 11 / 26

 Unit root processes Testing for unit roots

ADF test: Application to airline data

I ADF test on airline passengers data

- Plot of data indicates both a drift and a deterministic trend

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 12 / 26

 Unit root processes Testing for unit roots

ADF test: Application to airline data

- Result indicates no evidence of a unit root

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 13 / 26

 Unit root processes Testing for unit roots

ADF test: Application to German log of consumption

- Plot of data indicates both a drift and a deterministic trend

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 14 / 26

 Unit root processes Testing for unit roots

ADF test: Application to German log of consumption

- Result shows strong evidence for the presence of unit root

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 15 / 26

 Unit root processes Testing for unit roots

Other unit root tests in STATA

I Phillips-Perron (PP) test:

- adjusts the ADF test statistic to account for the potential serial correlation/heteroskedasticity
in the errors

- command: pperron varname, lag(#) trend → lag(#)≡ lag length of Newey-West HAC
estimator
I GLS detrended ADF test:
- similar to the ADF test but prior to fitting the model in (9), one first transforms the actual
series via a generalized least-squares (GLS) regression

- More powerful than the ADF test

- command: dfgls varname, maxlag(#) trend

- maxlag(#) sets the value of k, the highest lag order for the first-differenced, detrended
T +1 41
 
variable in the DF regression: by default, kmax = floor 12{ 100 } → Schwert, G. W (1989,
JBES)
Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 16 / 26
 Unit root processes Random Walk

Generating a random walk

clear all
set seed 2016
local T = 200
set obs ‘T’
gen time =− n
label var time "Time"
tsset time
gen eps =rnormal(0,5)
/*Random walk*/
gen yrw = eps in 1 /*keep first value of eps and replace other by dots */
replace yrw = l.yrw + eps in 2/l
/*Random walk with drift*/
gen yrwd1 = 0.1 + eps in 1
replace yrwd1 = 0.1 + l.yrwd1 + eps in 2/l

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 17 / 26

 Unit root processes Random Walk

Generating a random walk (cont’d)

/*Random walk with drift*/

gen yrwd2 = 1 + eps in 1
replace yrwd2 = 1 + l.yrwd2 + eps in 2/l
/*Stationary around a time trend model*/
gen yt = 0.5 + 0.1*time + eps in 1
replace yt = 0.5 + 0.1*time +1*l.yt+ eps in 2/l
drop in 1/50 /*drop obs 1 to 50*/
tsline yrw yrwd1, title("Stochastic trend") ///
legend(label(1 "Random walk") ///
label(2 "Random walk with drift"))
tsline yt yrwd2, ///
legend(label(1 "Deterministic time trend") ///
label(2 "Random walk with drift")) ///
title("Stochastic and deterministic trend")

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 18 / 26

 Unit root processes Random Walk

Properties of a Random Walk

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 19 / 26

 Autocorrelation and Partial Autocorrelation functions Autocorrelation function of stationary time series

Autocorrelation function (ACF)

I Autocovariance function:

γk = cov (yt , yt−k ) : k = 0, 1, 2, . . . (10)

I Autocorrelation function:
cov (yt , yt−k )
: k = 0, 1, 2, . . .
ρk = (11)
γ0
Both γk and ρk are symmetric function of k , i.e., γ−k = γk and ρ−k = ρk . Note that
ρ0 = 1 and −1 ≤ ρk ≤ 1.

- Stationary AR(1): yt = φ0 + φ1yt−1 + εt

φ0 σ2
µ =: E[yt ] = , var (yt ) = ,
1 − φ1 1 − φ21
2
k σ k
γk = φ1 , ρ k = φ 1 , k = 0, 1, 2, . . .
1 − φ21

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 20 / 26

 Autocorrelation and Partial Autocorrelation functions Autocorrelation function of stationary time series

ACF (cont’d)

- MA(1): yt = εt + θ0 + θ1εt−1

µ =: E[yt ] = θ0, var (yt ) = σ 2(1 + θ12),

2 θ1
γ1 = θ1σ , ρ1 = 2
and ρk = 0 ∀k > 1.
1 + θ1

I Stationarity is a property of the AR part of the process ⇒ MA processes are

always stationary and ρk = 0 for all k > q for an MA(q).

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 21 / 26

 Autocorrelation and Partial Autocorrelation functions Partial autocorrelation function of stationary time series

Partial autocorrelation function (PACF)

I Consider the AR(k) regression:

yt = β0 + β1yt−1 + . . . + βk yt−k + ut , k = 1, 2, . . . (12)

I k th-order PAC of yt for any k = 1, 2, 3, . . . is:

PACk = βk (13)

H For an AR(p) process, ρk is not zero after lag p but PACk = 0 for k > p ⇒ PACk is used to
identify p

H For an MA(q) process, PACk is not zero after lag q but ρk = 0 for k > q ⇒ ρk is used to
identify q

I Estimating ARMA(p,q) models requires identifying both p and q → properties

discussed above are key ingredients to achieve this goal

I In STATA, the command corrgram plots the estimated ACs and PACs.
Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 22 / 26
 Autocorrelation and Partial Autocorrelation functions Partial autocorrelation function of stationary time series

Corrgram in STATA

H Stata syntax: ‘corrgram’ tabulates autocorrelations, partial autocorrelations,

and portmanteau (Q) statistics

- Menu: Statistics → Time series → Graphs → Autocorrelations & partial autocorrelations

Command: corrgram varname [if] [in] [, corrgram−options]

- We can use ‘ac’ to produce a graph of the autocorrelations → Command: ac varname [if]

[in] [, ac−options]

- We can use ‘pac’ to produce a graph of the partial autocorrelations → Command: pac

varname [if] [in] [, pac−options]

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 23 / 26

 Autocorrelation and Partial Autocorrelation functions Application to international airline passengers

Application

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 24 / 26

 Autocorrelation and Partial Autocorrelation functions Application to international airline passengers

Application (cont’d)

(c) Autocorrelogram

(d) Partial Autocorrelogram

Firmin Doko Tchatoka (UoA) ECON 7223-Time Series Metrics IV 25 / 26

 Autocorrelation and Partial Autocorrelation functions Application to international airline passengers

Application (cont’d)

I From the PCF:

- Data probably have a trend component as well as a seasonal component

- First-differencing will mitigate the effects of the trend

- Seasonal differencing will help control for seasonality

I Can account for both together with time-series operators: command→ pac DS12.air,
lags(20) srv
- Here we graph the partial autocorrelations after controlling for trends and seasonality

- Use ‘srv ’ to include the standardized residual variances

Firmin Doko Tchatoka (UoA) ECON(e)

7223-Time
PartialSeries Metrics IV
Autocorrelogram 26 / 26