AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

Unit 8: ARMA Models: AR(1)

Taylor R. Brown

Department of Statistics, University of Virginia

Spring 2020

1 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

Readings for Unit 8

Textbook chapter 3.1 (page 75 to 79).

2 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

So Far...

We have mainly discussed a number of techniques in exploratory

data analysis that will aid us in understanding the behavior of
various time series. These should be used as a guide to help us
decide what kind of model is appropriate.

3 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

This Unit

1 AR(p) process
2 AR(1) process
3 AR(1) in terms of backshift operator
4 AR(1) and causality

4 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

Motivation

Next, we discuss ARMA models which is a time series where the

current observation depends on a linear combination of past
observations plus a “noise” term which is possibly a moving
average process.

We will discuss the models and some of their properties. We will

discuss how to fit these models and how to do forecasting from the
models once we fit them. We will also discuss how to select among
competing models, and then we will discuss how to handle
seasonal components.

5 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

1 AR(p) Process

2 AR(1) Process

3 AR(1) in Terms of Backshift Operator

4 AR(1) and Causality

6 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

AR(p) Process

Recall the definition of an autoregressive process: xt is an

autoregressive model of order p, AR(p), if it can be written as

xt = φ1 xt−1 + φ2 xt−2 + ... + φp xt−p + wt (1)

where wt is a Gaussian white noise process with variance σw2 .

7 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

AR(p) Process

If we would like to include a nonzero mean we can generalize the

AR(p) model (1) as

xt − µ = φ1 (xt−1 − µ) + φ2 (xt−2 − µ) + · · · + φp (xt−p − µ) + wt .

You can think about this as having an AR(p) after transforming

the data by removing a mean.

8 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

AR(p) Process

Another way to write this is

xt = α + φ1 xt−1 + φ2 xt−2 + · · · + φp xt−p + wt

where α = µ(1 − φ1 − · · · − φp ). This is closer to how one writes a

regression model. Note that α will NOT be the mean of xt .

9 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

Backshift Operator

Recall the backshift operator. For any t,

Bxt = xt−1 .

As an example we can rewrite the differencing operator as

∇xt = xt − xt−1 = (1 − B)xt .

We can also use powers of B such as

B p xt = xt−p .

10 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

AR(p) Process

So, we may rewrite the AR(p) model as

=
=
=
φ(B)xt = wt (2)

where φ(B) =

This function φ(B) is called the autoregressive operator. This is

a polynomial in B. It will be important to determine what choices
of the coefficients will give us a stationary process.

11 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

1 AR(p) Process

2 AR(1) Process

3 AR(1) in Terms of Backshift Operator

4 AR(1) and Causality

12 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

AR(1) Process

We will now look more in depth at the simplest case the AR(1)
process. We will discuss

xt = φxt−1 + wt

for t = ..., −1, 0, 1, 2, .... We would like to clearly determine the

mean, variance, and autocovariance of this process.

13 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

AR(1) Process

Earlier we discussed that we may use recursion to hopefully

simplify this process.

=
=
=
=
=
...
k−1
X
k
xt = φ xt−k + φj wt−j . (3)
j=0

14 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

AR(1) Process

Notice that if |φ| < 1, then the first term in (3) becomes very
small. So, in fact, we can say that as k → ∞ that
∞
X
xt = φj wt−j
j=0
X∞
= φj B j w t . (4)
j=0

This is an MA(∞) process.

15 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

AR(1) Process

We next show that if |φ| < 1, then an AR(1) is stationary. Note

that in unit 3, we showed that if an AR(1) is stationary, then
|φ| < 1.

The representation in (4) should make doing things like calculating

the mean easier. First of all, it should be clear that

Ext = 0.

16 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

AR(1) Process

Next, we would like to calculate the variance.

So, the mean and variance are constant.

17 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

AR(1) Process

We now need to only verify that the covariance also does not
depend on time but only the lag. Since the mean is zero, we need
to find, for h > 0,

18 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

AR(1) Process

Note also, that the true ACF is

ρ(h) = γ(h)/γ(0) = φh

19 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

AR(1) Process

The ACF should show a geometric decline–but never truly go to

zero. However, after a number of lags the series will be essentially
zero. An ACF that does not fall off quickly but where the series
does not appear stationary–may not be indicative of an AR model.

Question: What will happen when φ is negative?

20 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

1 AR(p) Process

2 AR(1) Process

3 AR(1) in Terms of Backshift Operator

4 AR(1) and Causality

21 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

AR(1) in Terms of Backshift Operator

Let’s go back and think about the backshift operator

representation for an AR process. From (2), we have

φ(B)xt = wt

where φ(B) = 1 − φB for AR(1) process. From (4) we have

∞
X ∞
X
xt = φj wt−j = φj B j wt .
j=0 j=0

22 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

AR(1) in Terms of Backshift Operator

(4) can be written as

∞
X
xt = ψj wt−j = ψ(B)wt , (5)
j=0
P∞ j
where ψ(B) = j=0 ψj B and ψj = φj . ψj called psi-weights.

23 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

AR(1) in Terms of Backshift Operator

Using (2) and (5), we have

24 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

AR(1) in Terms of Backshift Operator

Another way to think about the operations:

25 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

AR(1) in Terms of Backshift Operator

Notice that manipulating operators like φ(B) and ψ(B) is like

manipulating polynomials φ(z) = 1 − φz, where z is complex and
|φ| < 1:
1
φ−1 (z) = = 1 + φz + φ2 z 2 + · · · + φj z j + · · ·
1 − φz
This representation works when φ(z) is not zero.

26 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

1 AR(p) Process

2 AR(1) Process

3 AR(1) in Terms of Backshift Operator

4 AR(1) and Causality

27 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

AR(1) and Causality

We have shownP that, for AR(1) process xt = φxt−1 + wt , if |φ| < 1,

then xt = ∞ j=0 φ jw
t−j , depends only on the present and past
wt , wt−1 , . . . . This property is called causality. What if |φ| > 1?

28 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

Explosive AR(1)

Recall from (4) that we can write an AR(1) process as

∞
X ∞
X
xt = φj wt−j = φj B j wt .
j=0 j=0

When |φ| > 1, we have an explosive process because the values in

the series become large in magnitude. This is due to |φ|j increasing
without a bound as j → ∞.

29 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

Stationary AR(1)
Time Series Plot of AR(1), phi=0.5 ACF Plot of AR(1), phi=0.5

1.0
2

0.8
1

0.6
0

ACF
x

0.4
−1

0.2
−2

0.0
−3

−0.2

0 50 100 150 200 0 5 10 15 20

Time Lag 30 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

Explosive AR(1)
Time Series Plot of AR(1), phi=1.1 ACF Plot of AR(1), phi=1.1
0.0e+00

1.0
0.8
−5.0e+07

0.6
ACF
x

0.4
−1.0e+08

0.2
0.0
−1.5e+08

0 50 100 150 200 0 5 10 15 20

Time Lag 31 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

AR(1) and Causality

We could rewrite the AR(1) process as

1 wt
xt = φxt−1 + wt ⇒ xt−1 = xt −
φ φ
and we have, by replacing the time index t with t + 1,
xt = φ−1 xt+1 − φ−1 wt+1 . Iterating this expression forward, we
have
Xk
xt = φ−k xt+k − φ−j wt+j
j=1

32 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

AR(1) and Causality

Notice that if |1/φ| < 1, and taking the limit we obtain

∞
X
xt = − φ−j wt+j .
j=1

Question: Why is such a model problematic?

33 / 34
AR(p) Process AR(1) Process AR(1) in Terms of Backshift Operator AR(1) and Causality

AR(1) and Causality

For the AR(1) process, we require |φ| < 1 for causality. We will
later extend and formalize a condition on the parameters
φ1 , φ2 , . . . , φp for an AR(p) process to be causal.

34 / 34