Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

Unit 3: Stationarity

1 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

Readings for Unit 3

Textbook chapter 1.5, 1.6.

2 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

Last Unit

1 White noise.
2 Random walk model.
3 Autoregressive model.
4 Moving average model.
5 Mean function.
6 Measures of Dependence.

3 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

This Unit

1 Stationarity
2 Autocovariance and Autocorrelation of Stationary Time Series
3 Estimating the ACF

4 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

Motivation

In time series analysis, we frequently would prefer to analyze a

stationary sequence. This allows us to
autocorrelation and other quantities of interest. One feature of
stationary sequences is that they are identically distributed–but
often not independent. (Though, certainly an iid sequence is
stationary.) There are two types of stationarity: strictly
stationary and weakly stationary.

5 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

1 Stationarity

2 Autocovariance and Autocorrelation of Stationary Time Series

3 Estimating the ACF

6 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

Strictly Stationary

A time series is strictly stationary if for a sequence of times

t1 , t2 , ..., tk
{xt1 , ..., xtk }
has the same distributions as

{xt1 +h , ..., xtk +h }

for every integer h. In other words,

P{xt1 ≤ c1 , ..., xtk ≤ ck } = P{xt1 +h ≤ c1 , ..., xtk +h ≤ ck }.

Location does not matter–ONLY distance.

7 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

Weakly Stationary

A time series {xt } is weakly stationary if µt is

, and γ(s, t)
.

From now on when we say stationary, we’ll mean weakly stationary.

All strongly stationary time series are also weakly stationary, but
the reverse may not be the case. Most of the time we are going to
be working with Gaussian time series, and in this case the two
concepts coincide.

8 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

1 Stationarity

2 Autocovariance and Autocorrelation of Stationary Time Series

3 Estimating the ACF

9 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

Autocovariance of Stationary Time Series

With a stationary time series, we have the following property:

γ(t + h, t) = γ(h, 0).

So, for stationary processes we write

γ(h) = E(xt+h − µ)(xt − µ). (1)

We simply use the rule γ(s, t) = γ(s − t). Another property of the
autocovariance function when the time series is stationary is
γ(h) = γ(−h).
10 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

Autocorrelation of Stationary Time Series

For the autocorrelation function, we have

γ(h)
ρ(h) = . (2)
γ(0)

11 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

White Noise

Question: Show that white noise is stationary.

12 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

Random Walk Process

Question: Is a random walk process {xt } stationary? Recall from

last unit I simulated three realizations of a random walk.
Five Random Walks
10
8
6
4
values

2
0
−2
−4

5 10 15 20

Time

13 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

MA(2) Process

Question: Show that the MA(2) process is stationary.

14 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

ACF of MA(2) Process

15 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

AR(1) Process

Question: Show that for the AR(1) process to be stationary, we

require that |φ1 | < 1.

16 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

ACF of AR(1) Process

17 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

1 Stationarity

2 Autocovariance and Autocorrelation of Stationary Time Series

3 Estimating the ACF

18 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

Recall Stationarity

Suppose {xt } is a stationary time series. Then

ˆ Its mean is .
ˆ Its autocovariance function is γ(h) = E(xt+h − µ)(xt − µ). It
depends only on h = |s − t|. This also means that the
variance, γ(0) is constant.
ˆ Its autocorrelation function is ρ(h) = γ(h)
γ(0) .

19 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

Estimating the ACF

Without stationarity, we have little hope of estimating the full

γ(s, t). With stationarity, we will have that
are h apart from one another (at least when h << n). We now
discuss how to estimate ρ(h) to produce ACF plots.

20 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

Estimating the ACF

With stationarity, the (true) mean is constant. We can therefore

estimate the mean using the
Pn
xt
x̄ = t=1 .
n
This converges to µ. In fact,
Pn n n
t=1 xt 1X 1X
E (x̄) = E ( )= E (xt ) = µ = µ.
n n n
t=1 t=1

21 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

Estimating the ACF

Consider the sample autocovariance function

Pn−h
t=1 (xt+h − x̄)(xt − x̄)
γ̂(h) = . (3)
n
for h = 0, 1, ..., n − 1.

For fixed h, all the random variables yt = (xt+h − x̄)(xt − P

x̄) have
n
t=1 yt
the same distribution ( ). Therefore, n−h
converges to γ(h) = E (xh − µ)(x0 − µ).

22 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

Estimating the ACF

To obtain the sample autocorrelation we simply scale by the

variance
γ̂(h)
ρ̂(h) = . (4)
γ̂(0)
One thing to notice in the sample autocovariance function (3) is
that we divide by n not n − h or n − 1. This ensures that if we
calculate variances, they are all positive.

23 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

Sample ACF

We can recognize the sample ACF of time series.

Time series ACF

White noise 0
Trend Slow decay
Periodic Periodic
MA(q) 0 for h > q
AR(p) Decays to 0 exponentially

24 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

Sample ACF for Gaussian White Noise

ACF for White Noise
1.0
0.8
0.6
ACF

0.4
0.2
0.0

0 5 10 15 20 25 30

Lag
25 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

Sample ACF for Gaussian White Noise

When the true model is white noise ρ̂(h) is approximately normally

√
distributed with zero mean and standard deviation of 1/ n.

26 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

Sample ACF: Marriages in Church of England

% marriages in Church of England ACF for Marriage

1.0
0.8
75

0.6
70
Percent

0.4
ACF

0.2
65

0.0
−0.2
60

1870 1890 1910 0 5 10 15

Time Lag
27 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

Sample ACF: Average Monthly Temperature in Dubuque,

IA
Avg monthly temp in Dubuque, IA ACF for avg temp

1.0
70
60

0.5
50
Temp (F)

ACF

0.0
40
30

−0.5
20
10

1960 2000 2040 2080 0 5 10 15 20

28 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

Sample ACF: MA(1) Process

ACF for MA(1) Process
1.0
0.8
0.6
ACF

0.4
0.2
0.0

0 5 10 15 20 25 30

Lag
29 / 30
Stationarity Autocovariance and Autocorrelation of Stationary Time Series Estimating the ACF

Sample ACF: AR(1) Process

ACF for AR(1) Process
1.0
0.8
0.6
ACF

0.4
0.2
0.0

0 5 10 15 20 25 30

Lag
30 / 30