3.

2: Causality and Invertibility

While a moving average process of order q will always be stationary without conditions on the coefficients θ ,…,θ , 1 q

some deeper thoughts are required in the case of AR(p) and ARMA(p, q) processes. For simplicity, we start by
investigating the autoregressive process of order one, which is given by the equations X = ϕX + Z (writing t t−1 t

ϕ = ϕ ). Repeated iterations yield that

1

N −1

2 N j
Xt = ϕXt−1 + Zt = ϕ Xt−2 + Zt + ϕZt−1 = … = ϕ Xt−N + ∑ ϕ Zt−j . (3.2.1)

j=0

Letting N → ∞ , it could now be shown that, with probability one,

∞

j
Xt = ∑ ϕ Zt−j (3.2.2)

j=0

is the weakly stationary solution to the AR(1) equations, provided that |ϕ| < 1 . These calculations would indicate
moreover, that an autoregressive process of order one can be represented as linear process with coefficients ψ = ϕ . j
j

Example 3.2.1 : Mean and ACVF of an AR(1) process

Since an autoregressive process of order one has been identified as an example of a linear process, one can easily
determine its expected value as
∞

j
E[ Xt ] = ∑ ϕ E[ Zt−j ] = 0, t ∈ Z. (3.2.2)

j=0

For the ACVF, it is obtained that

γ(h) = Cov(Xt+h , Xt )

∞ ∞

j k
= E [ ∑ ϕ Zt+h−j ∑ ϕ Zt−k ]

j=0 k=0

∞ ∞ 2 h
2 k+h k 2 h 2k
σ ϕ
=σ ∑ϕ ϕ =σ ϕ ∑ϕ = ,
2
1 −ϕ
k=0 k=0

where h ≥ 0 . This determines the ACVF for all h using that γ(−h) = γ(h) . It is also immediate that the ACF
satisfies ρ(h) = ϕ . See also Example 3.1.1 for comparison.
h

Example 3.2.2 : Nonstationary AR(1) processes

In Example 1.2.3 we have introduced the random walk as a nonstationary time series. It can also be viewed as a
nonstationary AR(1) process with parameter ϕ = 1 . In general, autoregressive processes of order one with coefficients
|ϕ| > 1 are called {\it explosive}\/ for they do not admit a weakly stationary solution that could be expressed as a

linear process. However, one may proceed as follows. Rewrite the defining equations of an AR(1) process as
−1 −1
Xt = −ϕ Zt+1 + ϕ Xt+1 , t ∈ Z. (3.2.3)

Apply now the same iterations as before to arrive at

N

−N −j
Xt = ϕ Xt+N − ∑ ϕ Zt+j , t ∈ Z. (3.2.4)

j=1

Note that in the weakly stationary case, the present observation has been described in terms of past innovations. The
representation in the last equation however contains only future observations with time lags larger than the present time
t . From a statistical point of view this does not make much sense, even though by identical arguments as above we

may obtain
∞
−j
Xt = − ∑ ϕ Zt+j , t ∈ Z, (3.2.5)

j=1

Alexander Aue 5/19/2020 3.2.1 https://stats.libretexts.org/link?845

 as the weakly stationary solution in the explosive case.

The result of the previous example leads to the notion of causality which means that the process (X : t ∈ Z) has a t

representation in terms of the white noise (Z : s ≤ t) and that is hence uncorrelated with the future as given by
s

(Z : s > t) . We give the definition for the general ARMA case.

s

Definition: Causality
∞
An ARMA(p, q) process given by (3.1.1) is causal if there is a sequence (ψ j : j ∈ N0 ) such that ∑ j=0
| ψj | < ∞ and
∞

Xt = ∑ ψj Zt−j , t ∈ Z. (3.2.6)

j=0

Causality means that an ARMA time series can be represented as a linear process. It was seen earlier in this section how
an AR(1) process whose coefficient satisfies the condition |ϕ| < 1 can be converted into a linear process. It was also
shown that this is impossible if |ϕ| > 1 . The conditions on the autoregressive parameter ϕ can be restated in terms of the
corresponding autoregressive polynomial ϕ(z) = 1 − ϕz as follows. It holds that
|ϕ| < 1 if and only if ϕ(z) ≠ 0 for all |z| ≤ 1,

|ϕ| > 1 if and only if ϕ(z) ≠ 0 for all |z| ≥ 1 .

It turns out that the characterization in terms of the zeroes of the autoregressive polynomials carries over from the AR(1)
case to the general ARMA(p, q) case. Moreover, the ψ -weights of the resulting linear process have an easy representation
in terms of the polynomials ϕ(z) and θ(z) . The result is summarized in the next theorem.

Theorem 3.2.1
Let (X t be an ARMA(p, q) process such that the polynomials ϕ(z) and θ(z) have no common zeroes. Then
: t ∈ Z)

(Xt : t ∈ Z)is causal if and only if ϕ(z) ≠ 0 for all z ∈ C with |z| ≤ 1 . The coefficients (ψ : j ∈ N ) are determined j 0

by the power series expansion

∞
θ(z)
j
ψ(z) = ∑ ψj z = , |z| ≤ 1. (3.2.7)
ϕ(z)
j=0

A concept closely related to causality is invertibility. This notion is motivated with the following example that studies
properties of a moving average time series of order 1.

Example 3.2.3
Let (X t: t ∈ N) be an MA(1) process with parameter θ = θ . It is an easy exercise to compute the ACVF and the ACF
1

as
2 2
⎧ (1 + θ )σ ,
⎪
h = 0, ⎧1
⎪
h = 0.

2 −1
γ(h) = ⎨ θσ 2 , h =1 ρ(h) = ⎨ θ(1 + θ ) , h = 1. (3.2.8)
⎩
⎪ ⎩
⎪
0 h > 1, 0 h > 1.

These results lead to the conclusion that ρ(h) does not change if the parameter θ is replaced with θ . Moreover, there −1

exist pairs (θ, σ ) that lead to the same ACVF, for example (5, 1) and (1/5, 25). Consequently, we arrive at the fact
2

that the two MA(1) models

1
Xt = Zt + Zt−1 , t ∈ Z, (Zt : t ∈ Z) ∼ iid N (0, 25), (3.2.9)
5

and
~ ~ ~
Xt = Z t + 5 Z t−1 , t ∈ Z, (Z : t ∈ Z) ∼ iid N (0, 1), (3.2.10)

~
are indistinguishable because we only observe X but not the noise variables Z and
t t Zt .

For convenience, the statistician will pick the model which satisfies the invertibility criterion which is to be defined next.
It specifies that the noise sequence can be represented as a linear process in the observations.

Alexander Aue 5/19/2020 3.2.2 https://stats.libretexts.org/link?845

 Definition: Invertibility
An ARMA(p, q) process given by (3.1.1) is invertible if there is a sequence (π j: j ∈ N0 ) such that ∑ ∞

j=0
| πj | < ∞ and
∞

Zt = ∑ πj Xt−j , t ∈ Z. (3.2.11)

j=0

Theorem 3.2.2
Let (X t be an ARMA(p, q) process such that the polynomials ϕ(z) and θ(z) have no common zeroes. Then
: t ∈ Z)

(Xt : t ∈ Z)is invertible if and only if θ(z) ≠ 0 for all z ∈ C with |z| ≤ 1 . The coefficients (π ) are determined by j j∈N0

the power series expansion

∞
ϕ(z)
j
π(z) = ∑ πj z = , |z| ≤ 1. (3.2.12)
θ(z)
j=0

From now on it is assumed that all ARMA sequences specified in the sequel are causal and invertible unless explicitly
stated otherwise. The final example of this section highlights the usefulness of the established theory. It deals with
parameter redundancy and the calculation of the causality and invertibility sequences (ψ : j ∈ N ) and (π : j ∈ N ) . j 0 j 0

Example 3.2.4 : Parameter redundancy

Consider the ARMA equations

Xt = .4 Xt−1 + .21 Xt−2 + Zt + .6 Zt−1 + .09 Zt−2 , (3.2.13)

which seem to generate an ARMA(2,2) sequence. However, the autoregressive and moving average polynomials have
a common zero:
~ 2
ϕ (z) = 1 − .4z − .21 z = (1 − .7z)(1 + .3z),

~ 2 2
θ (z) = 1 + .6z + .09 z = (1 + .3z) .

Therefore, one can reset the ARMA equations to a sequence of order (1,1) and obtain
Xt = .7 Xt−1 + Zt + .3 Zt−1 . (3.2.14)

Now, the corresponding polynomials have no common roots. Note that the roots of ϕ(z) = 1 − .7z and
θ(z) = 1 + .3z are 10/7 > 1 and −10/3 < −1, respectively. Thus Theorems 3.2.1 and 3.2.2 imply that causal and

invertible solutions exist. In the following, the corresponding coefficients in the expansions
∞ ∞

Xt = ∑ ψj Zt−j and Zt = ∑ πj Xt−j , t ∈ Z, (3.2.15)

j=0 j=0

are calculated. Starting with the causality sequence (ψ j : j ∈ N0 ) . Writing, for |z| ≤ 1 ,
∞ ∞
θ(z) 1 + .3z
j j
∑ ψj z = ψ(z) = = = (1 + .3z) ∑(.7z) , (3.2.16)
ϕ(z) 1 − .7z
j=0 j=0

it can be obtained from a comparison of coefficients that

j−1 j−1
ψ0 = 1 and ψj = (.7 + .3)(.7 ) = (.7 ) , j ∈ N. (3.2.17)

Similarly one computes the invertibility coefficients (π j : j ∈ N0 ) from the equation

∞ ∞
ϕ(z) 1 − .7z
j j
∑ πj z = π(z) = = = (1 − .7z) ∑(−.3z) (3.2.18)
θ(z) 1 + .3z
j=0 j=0

(|z| ≤ 1 ) as
j j−1 j j−1
π0 = 1 and πj = (−1 ) (.3 + .7)(.3 ) = (−1 ) (.3 ) . (3.2.19)

Together, the previous calculations yield to the explicit representations

Alexander Aue 5/19/2020 3.2.3 https://stats.libretexts.org/link?845

 ∞ ∞

j−1 j j−1
Xt = Zt + ∑(.7 ) Zt−j and Zt = Xt + ∑(−1 ) (.3 ) Xt−j . (3.2.20)

j=1 j=1

In the remainder of this section, a general way is provided to determine the weights (ψ : j ≥ 1) for a causal ARMA(p, q)
j

process given by ϕ(B)X = θ(B)Z , where ϕ(z) ≠ 0 for all z ∈ C such that |z| ≤ 1 . Since ψ(z) = θ(z)/ϕ(z) for these
t t

z , the weight ψ can be computed by matching the corresponding coefficients in the equation ψ(z)ϕ(z) = θ(z) , that is,
j

2 p q
(ψ0 + ψ1 z + ψ2 z + …)(1 − ϕ1 z − … − ϕp z ) = 1 + θ1 z + … + θq z . (3.2.21)

Recursively solving for ψ 0, ψ1 , ψ2 , … gives

ψ0 = 1,

ψ1 − ϕ1 ψ0 = θ1 ,

ψ2 − ϕ1 ψ1 − ϕ2 ψ0 = θ2 ,

and so on as long as j < max{p, q + 1} . The general solution can be stated as

j

ψj − ∑ ϕk ψj−k = θj , 0 ≤ j < max{p, q + 1}, (3.2.1)

k=1

ψj − ∑ ϕk ψj−k = 0, j ≥ max{p, q + 1}, (3.2.2)

k=1

if we define ϕ = 0 if j > p and θ = 0 if j > q . To obtain the coefficients ψ one therefore has to solve the
j j j

homogeneous linear difference equation (3.2.2) subject to the initial conditions specified by (3.2.1). For more on this
subject, see Section 3.6 of Brockwell and Davis (1991) and Section 3.3 of Shumway and Stoffer (2006).

R calculations
In R, these computations can be performed using the command ARMAtoMA. For example, one can use the commands
>ARMAtoMA(ar=.7,ma=.3,25)
>plot(ARMAtoMA(ar=.7,ma=.3,25))
which will produce the output displayed in Figure 3.4. The plot shows nicely the exponential decay of the ψ -weights
which is typical for ARMA processes. The table shows row-wise the weights ψ , … , ψ . This is enabled by the
0 24

choice of 25 in the argument of the function ARMAtoMA.

1.0000000000 0.7000000000 0.4900000000 0.3430000000 0.2401000000

0.1680700000 0.1176490000 0.0823543000 0.0576480100 0.0403536070

0.0282475249 0.0197732674 0.0138412872 0.0096889010 0.0067822307
0.0047475615 0.0033232931 0.0023263051 0.0016284136 0.0011398895
0.0007979227 0.0005585459 0.0003909821 0.0002736875 0.0001915812

Alexander Aue 5/19/2020 3.2.4 https://stats.libretexts.org/link?845

 Figure 3.4: The R output for the ARMA(1,1) process of Example 3.2.4

Contributers
Alexander Aue (Department of Statistics, University of California, Davis)

Alexander Aue 5/19/2020 3.2.5 https://stats.libretexts.org/link?845