Spectral Properties of Non-Stationary Systems of Linear Stochastic Difference Equations

Author(s): Gregory C. Chow and Richard E. Levitan

Source: Journal of the American Statistical Association, Vol. 64, No. 326 (Jun., 1969), pp.
581-590
Published by: Taylor & Francis, Ltd. on behalf of the American Statistical Association
Stable URL: http://www.jstor.org/stable/2283640
Accessed: 27-06-2016 01:16 UTC

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
http://about.jstor.org/terms

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted
digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about
JSTOR, please contact [email protected].

American Statistical Association, Taylor & Francis, Ltd. are collaborating with JSTOR to digitize,
preserve and extend access to Journal of the American Statistical Association

This content downloaded from 128.163.2.206 on Mon, 27 Jun 2016 01:16:55 UTC
All use subject to http://about.jstor.org/terms
 SPECTRAL PROPERTIES OF NON-STATIONARY SYSTEMS
OF LINEAR STOCHASTIC DIFFERENCE EQUATIONS*

GREGORY C. CHOW AND RICHARD E. LEVITAN

International Business Machines Corporation

A method is proposed to eliminate trends from a sample from a non-

stationary system of linear stochastic difference equations. The auto-
covariance matrix and the spectral density matrix of the detrended
component of the sample are derived. The latter matrix turns out to
have the same form as the spectral density matrix for a stationary sys-
tem when expressed in terms of the roots of the system. Since the param-
eters of the system are assumed known throughout the paper, the
problem of statistical inference does not arise.

1. INTRODUCTION

IN an applied field such as economics, one is often interested in the dynamic

characteristics of a system of stochastic difference equations. For the pur-
pose of analyzing a theoretical model, e.g., business cycle theory [1], or an
empirical one, the parameters of the system are assumed to be known. In the
latter case, only estimates of the parameters are available, but the present
paper is concerned only with deducing certain measures of the dynamic proper-
ties of the system given the values of the parameters, rather than with sampling
errors of the measures induced by the errors of the estimates of the parameters.
The results of this paper, nevertheless, may be a step toward a solution of the
latter problem.'
If the system is linear, i.e., the vector time series yt satisfies

Yt Ayt-1 + ut( 1 )

where a higher-order system is already converted to first order, and where

Eu tu' = V and Eutu,_k = O for k#,0 and if it is stationary, i.e., the roots of A
are all smaller than one in absolute value, its second-order behav-ior can be
characterized by the autocovariance matrix or the spectral density matrix
with a well-known expression [4, p. 29].
However, if the system (1) is non-stationary, when some roots are greater
than one in absolute value, its behavior appears difficult to characterize since
the autocovariances do not converge and the spectral densities are not defined.
At the same time, it may be reasonable to represent a growing economy by
such a non-stationary system. The cyclical characteristics of many apparently
explosive economic time series have been studied, some by spectral analysis,
after applying certain methods to eliminate trends from the observed series.
The question then arises as to whether a method can be set forth to eliminate
trends from a realization of system (1), given its parameters, so that thec de-

* We would like to express our thanks to T. W. Anderson, R. K. Brayton, and B. B. Mandelbrot for helpful
discussions; and to E. P. Howrey, I. R. Savage, and two referees for suggestions to improve the exposition of the
original draft.
1 Given the sampling distribution of the estimates of the parameters, one can at least use simulation to obtain
the distribuition of the statistics given by (33).

581

This content downloaded from 128.163.2.206 on Mon, 27 Jun 2016 01:16:55 UTC
All use subject to http://about.jstor.org/terms
 582 AMERICAN STATISTICAL ASSOCIATION JOURNAL, JUNE 1969

trended component can be characterized by an appropriate autocovariance

matrix or spectral density matrix.
This paper proposes such a method for the case of an explosive or non-sta-
tionary system (1). The method is an extension of a suggestion made by
Quenouille [3] applicable to system (1) when A is a scalar. Our extension to
the multivariate case is fairly straightforward. However, Quenouille himself
did not make use of this suggestion but recommended a different procedure
for the multivariate situation.
In section 2, Quenouille's suggestion for the univariate case will be restated,
and a property of the detrended sample will be derived for the purpose of
adaptation to the multivariate situation. Section 3 deals with trend elimination
for a realization of system (1) and derives the autocovariance and spectral
properties of the detrended component. The Appendix raises some questions
concerning Quenouille's method for the multivariate system, which is supposed
to apply only when all roots of the system are real, and treats the case of com-
plex roots greater than one in absolute value.
It turns out that the spectral density matrix of the detrended component
of the sample from a non-stationary system of the form (1) will have the same
form as the spectral density matrix for a stationary system of the form (1) when
the density is expressed in terms of the roots of the system.

2. TREND ELIMINATION FOR A REALIZATION OF A NON-STATIONARY

FIRST-ORDER STOCHASTIC DIFFERENCE EQUATION

Consider a series Zt in discrete time generated by

Zt == xzt-i + 'et (t 1, I I * T) (2)

where Et are mutually independent random variables with mean zero and
variance w. We will be concerned with the non-stationary or explosive case
(X> 1), and will treat z0 as given. Quenouille [3, pp. 57-58 ] pointed out that
any particular realization (Zi, ' * * , ZT) can be written as the sum of an ex-
ponential trend and a realization (z4 * ' * T) of a stationary series (superscript
d for detrended).
Let (z4, * * *, Z) satisfy
d d. .(3

Zt - xzt-1 + C-t (t I }* **2T) (3)

where the Et are the same as in (2). The difference
d d
Xt -Zt - Zt X(Zt-1 - zt1) = XXt-i (4)

must be an exponential trend.

To find a vector (z4, * * * , 4T) satisfying (3) which is a sample from a sta-
tionary process, one first selects the end value Z4 from a distribution with zero
mean and independently of ET, and then works backward by equation (3), i.e.,

Zt-1 = X Zt \ ( (t = T) T - 1, ,1 (5)
The vector so chosen is a sample, going backward in time, from a stationary
first-order series with coefficient X-1 and independent residuals -X-1cEt. Since

This content downloaded from 128.163.2.206 on Mon, 27 Jun 2016 01:16:55 UTC
All use subject to http://about.jstor.org/terms
NON-STATIONARY LINEAR STOCHASTIC DIFFERENCE EQUATIONS 583

the variance of this series is

Var(-X-'et) w
= n~ ( 6)
1-X-2 X2 - (

Quenouille suggested choosing Z4 from a distribution with variance (6).

It is easy to show that a sample from such a stationary series, going back-
ward, can also be interpreted as a sample from the forward going scheme
d d

Zt = ?C1Zt-1 +tlt (t = 1, , T). (7)

By (5) and (7), one finds the residuals It of the forward scheme to be
d -1 d
t= Zt - X Zt-i

= (1 - X)Zt + 2 Et (8)

By repeated applications of (5), one obtains

d -1 =1 -(T-t) -(T-t) d
Zt-i -X (Et?+X Et+1? ?+x ET -x ZT). (9)
Hence, z4 , can be regarded as a linear function of future E's if the end point
Z4 is chosen from the distribution suggested by Quenouille, or is simply set
equal to zero for sufficiently large T. In the latter case, the trend value XT
equals ZT. When (9) is used to substitute for Zd in (8), we have

'it= - (1 X-2)X-1(Et+1 + ?c1Et+2 + X-2Et+3 + * * * ) + X-2t. (10)

From (10), it will follow that the residuals qt of the forward scheme (7)
have the same autocovariance function as the residuals - - 'Et of the backward
scheme (5). The expectation of qt is zero; its variance is
2 -2 2 -2 -2 -4 -4
En t = (I- X ) x (I + X+X + ***)W+X W (l
= X-2W

the same as the variance of - -lE. Similarly, the covariance between 77t and
7t1tk(k1>0) is

Ent7t-k = Et - (1 - 2X2) X1(Et+1 + ?X 1E+2 + * ) + X-2E]

X [-(1 - X-2)X l(Et-k+1 + X 1Et-k+2 + ) + X 2Et-k] (12)
= (1 - X-2)2X-2-k(l - X-2)-lW - (1 - X-2)X-2-kW = 0.

Equation (10) also provides all the information concerning the residuals 77t
required for our extension to the multivariate case. Any sample from the
non-stationary process (2), with Et as residuals, can be decomposed into an ex-
ponential trend plus a sample from the stationary series (7) whose residuals
,It are each the sum of X-2Et and a term independent of it. These residuals have
mean zero, variance X-2w. and are uncorrelated.

3. DYNAMIC PROPERTIES OF NON-STATIONARY LINEAR SYSTEMS

In this section, we will decompose a sample from the system (1) into a sam-
ple from a stationary system plus exponential trends, and derive the autoco-

This content downloaded from 128.163.2.206 on Mon, 27 Jun 2016 01:16:55 UTC
All use subject to http://about.jstor.org/terms
584 AMERICAN STATISTICAL ASSOCIATION JOURNAL, JUNE 1969

variance matrix and spectral density matrix of the stationary system. The
square matrix A in (1) is assumed to be real and non-singular. The covariance
matrix of ut is denoted by V, assumied to be positive semi-definite, with some
equations in (1) possibly being identities.
A new set of variables zt will be defined, which are linear combinations of
Yt, by diagonalizing the matrix A,
A = BDxB-1 (13)

where D, is the diagonal matrix consisting of the characteristic roots of A, and

the columns of B are the corresponding (right) characteristic vectors. Equations
(1) and (13) imply

B-lyt = DxB-lyt-, + B-lut (14)

or
Zt = Dxzt-1 + et.

The canonical variables zt are defined as B-lyt, and the residuals t = B-1ut in
the transformed system have a covariance matrix

EetE = B-'VB-l' = W -(wij). (15)

If some roots are greater than one in absolute value, the system will be non-
stationary. We will first treat the case of a single real root greater than one,
then the case of many real roots greater than one later on in this section,
leaving the case of complex roots greater than one in absolute value to the
Appendix. Let Xi> 1. By the result of section 2, a sample of the first canonical
variable

Zit = Xizi,t-i + ell (t = 1, * * ,T) (16)

can be decomposed into an exponential trend

xit= x zX1-l (t = 1, * * , T) (17)

plus a sample from a stationary series

d -l d
Zit = Xi Zl,t-l + nlt (t = 1, ... , T) (18)

where lit have zero mean and are mutually uncorrelated.

A sample from the multivariate time series (1) can be similarly decom-
posed. Observations on the first canonical variable are assumed to be decom-
posed as in the preceding paragraph. Observations on all other canonical vari-
ables will be unchanged. Let bij be the ijth element of B, and bi be the ith
row of B. The sample of the ith variable is partitioned as
d

Yit = biz1 + bi1x1t (t = 1, * * ,T) (19)

where Z4 denotes the vector Zt with the first component replaced by z'. We will
be concerned with the dynamic properties of the detrended observations

Yit = bizt (t = ,*** T) (20)

after the exponential trends ba1x1t (i = 1, * * * , p) have been eliminated.
Consider first the autocovariance matrix of the (detrended) canonical vari-
ables

This content downloaded from 128.163.2.206 on Mon, 27 Jun 2016 01:16:55 UTC
All use subject to http://about.jstor.org/terms
 NON-STATIONARY LINEAR STOCHASTIC DIFFERENCE EQUATIONS 585

rk = E(Zd=d-) = E(zd Zd = ('Y*d,). (21)

Here the superscript star indicates that the variables are canonical variables
rather than the original variables y; and the superscript d signifies "detrended."
Expectations here and elsewhere are taken under the assumption that the
correct value of X, and not an estimate of it, is used for detrending. The co-
variances *Yd7 for the three cases (i, j# 1), (i, j = 1) and (i=1, j# 1 or i H 1,
j= 1) are given respectively by expressions (22), (23), and (24).
*d *
7ij,k = 7ij,k = Ezitzj,t-k

- E(:it + Xjj,t-i Xii,t + (E,t-k + XjEj,t-k-1 + )

= k(l+XX X22
- Xk(1 + X Xj + X2dj + )EE,itEjt (22)
k

Xiij (i, j 1; k > 0).

*d1,k = xk2ll-k =) (k > 0). (23)

*d d
71yj,k = E(Z1, tZj, t-k)

- - E(Xi Ei,t+i + X1 Clst+2 + * (E, + Xjj,tk + * * ) (24a)

-0 = 7'Yl-k (j 7 1; k > 0)

and2

71j,-k= - E(X1 Cl,t+l + X1 El,t+2 + ) )(Ej,t+k + XjEj,t+k-1 + )

-k -k+l ~-k+ 2 2 - 1 k-i
(xi + X(i+ x + X-? X+ * X1 X-1 )w1 (24b)

(\ 1 )Wij *d
( = Yjl ,k (j - 1; k > 1).

Using these covariances, the corresponding spectral densities can be obtained

(except for a normalization constant3) by

*d *d -ik =v
fij (co) = fJ* i V-)(25)
k_ oo

Since the key term in the covariance functions is X`1k or Xj, the weighted sum
(25) can be performed by using

j X3e - 1- e (26)

2 Here we ignore the possibility that XlX; = 1, in which case (1 -X14) /(1 - XiX) should be replaced by k.
3 The purpose of normalization is to make the area under the spectral density function equal to one. Dividing
the auto covariance function _Y*d by the product of the standard deviations \I_Y*d and _ -*do yields the auto-
correlation function. If our discussion is confined to real time series in discrete time, the range of to is only from 0 to
7r; thus the area under the spectral density from 0 to 7r has to be one, as accomplished by a further division of (25)
by 7r. If complex time series in discrete time are considered, with X ranging from -or to 7r, then fuirther division
by 27r would make the area under the spectral density from -7r to 7r equal to one. When the apparatus is set up
for complex time series, with the spectral density defined accordingly, any plot of the spectral density for a real
time series only from 0 to 7r will require doubling the ordinate.

This content downloaded from 128.163.2.206 on Mon, 27 Jun 2016 01:16:55 UTC
All use subject to http://about.jstor.org/terms
586 AMERICAN STATISTICAL ASSOCIATION JOURNAL, JUNE 1969

The spectral densities for the three cases are (27), (28) and (29).

*d *d -icok + Y*d -ioxk *d

fii (co) = fij,ke + ij,-ke - Yij,o
k=O k=O

Wij/ 11
( 1 1 -1 (27)
l XiXi I - Xie- I 1- Xjei
Wi]
= Wij ~~~~~~(i, j 1)
(1 - Xie-i-)(I - X1ei-)

-2

*d Xl W11

(1- le-iw) (1 - X-leiw) (28)

Wll

(1- Xe-ie)(1 - Xleiw)

f*d iw7k
fji (CO) = E ldke
k--1

Wjj 1
w=~ X,e/ -i e
3e ________ (29a)
1- X Xi 1 - Xeie 1-iI -leiw
Wij

(1-Xe-ie)(I - Xjeiw)
and

fi w (C (29b)
(1 - Xje-iw)(I - Xieio)

The results (28) and (29) show that even when a characteristic root X1 is
greater than unity, or when a canonical variable zil is non-stationary, the
spectral or cross-spectral density functions f*d(co) of the canonical variables,
once detrended, will have the same form as the function (27) for the stationary
canonical variables. The autocovariance or cross-covariance functions (23) and
(24) involving the non-stationary (but detrended) canonical variable Z4t differ
in form from the function (22) for the stationary canonical variables, however,
the corresponding spectral density functions have the same form. This result
does not lead to mathematical inconsistency because the covariance functions
(23) and (24) are defined and valid only for XI> 1, whereas the covariance
functions (22) are defined for Xi< < and Xj < 1.
It is easy to show that the formula (27) applies to two or more real roots
greater than unity. For X1> 1 and X2> 1, say, one needs to check only f*d(w).
Using (9), one obtains

*d (Zd,d d
Y12,k =El 2
-1 -2' 1 -2

E(X -E 1 ? 2E1,t+2 + )(X2 E2,t-k+l + X2 E2,t-k+2 ? )

-k -i -1 (30)
X2 (Xl X2 W12) *d (k?O)
12 x- X- I 2l,-(;

This content downloaded from 128.163.2.206 on Mon, 27 Jun 2016 01:16:55 UTC
All use subject to http://about.jstor.org/terms
 NON-STATIONARY LINEAR STOCHASTIC DIFFERENCE EQUATIONS 587

The cross-spectral density function is

*d *d -i&k * * ick *
fl2(W) - 712,ke ? x 712,-ke 712,0
k=o k=O

-1 -1

Xl 12 W12 I
I X-'X;-1
= _ <1 -.letO)
+ 11-)Vl (31)
-31
W12

(X92 - e- i)(Xi- C)

W12

(1 - Xle-i0)(1- X26eiw)

which has the same form as (27).

In fact, the above analysis can be applied when there are complex roots.
Obviously, if the complex roots are smaller than one in absolute value, all of
our derivations in this section remain valid. For the case of complex roots greater
than one in absolute value, the reader is referred to the Appendix.
We have derived the autocovariance matrix r*d= [*yd] and the spectral
density matrix F*d(w)= [f( (w)] of the detrended canonical variables. The
corresponding matrices for the detrended original variables yd, defined by
equation (20), are simply

dh = Brk B (32)
and

Fd(wv) - BF*d(w)B' = B W-3 . (3l

(I- Xie-iw) (1 - Xjeiw)/(3
The main result of this paper is summarized by the following theorem.
Theorem: Given system (1), with A = BD;B-, DA being a diagonal matrix con-
sisting of the roots X of A, Eutu[ = V, (wj) =B-'VB-", (33) is the spectral
density matrix of
d d
Yit = bizt

where bi is the ith row of B, zt =B-'yt, Z4t=Zjt for I Xj <1, 4z-=zjt-t for
X Ai > 1 and for the exponential trend XSt = XjXj, t_14

APPENDIX

QUENOUILLE S METHOD OF DECOMPOSING A NON-STATIONARY

SYSTEM AND THE CASE OF COMPLEX ROOTS

Instead of following up his own suggestion for the univariate case, Quenouille
has recommended a different method for the multivariate system. In the para-
4 Strictly speaking, our method breaks down when a root is exactly one, but this rarely happens. Even when
certain theorizing, such as assuming linear equations for the first differences of some variables, could lead to roots
of unity if the theory were strictly adhered to, we would still recommend using equation (33) since the theory itself
is only an approximation. In an economic application [21, we have computed the spectral densities from a matrix A
with .99997 and .99991 as two of its roots. The expression (27) for each of these canonical variables implies extremely
high densities at very low frequencies, a phenomenon which many spectral analyses of economic time series have
revealed and which our calculations [21 confirm. For a discussion of the possible relevance of the present paper to
business cycle theory, the reader may refer to another paper [1], which has motivated the present paper.

This content downloaded from 128.163.2.206 on Mon, 27 Jun 2016 01:16:55 UTC
All use subject to http://about.jstor.org/terms
588 AMERICAN STATISTICAL ASSOCIATION JOURNAL, JUNE 1969

graph immediately after making his former suggestion [3, p. 58], Quenouille
wrote (italics ours; F-'(D) replacing F(D) for consistency with the rest of
his book),
"In general, any non-stationary autoregressive process F-'(D)xt=Et, with
real roots, may be partitioned in a similar manner [sic], the stochastic element
following a stationary scheme whose terms when operated upon by F-1(D) gives
series of random elements. To determine this stationary scheme, it is necessary
to use the results of ?3.2: the corresponding scheme has its ui and wi inter-
changed. "
Here xt and Et correspond respectively to our yt and ut in system (1). D is the
lag operator, and Fl7 (D) is a polynomial in D for an autoregressive process-
F-'(D) equals I-AD for the first order system (1). ui corresponds to the ith
column of the matrix B of right characteristic vectors. wi had been defined
[3, p. 10] as the vector

P 1 i V3.j

j=l1\-1_ly8j

where vi corresponds to the ith row of -D}'B-1 in our notations.

It is difficult to accept Quenouille's proposal for the multivariate case. His
rationale is the statement [3, ?3.2] that the 2n autoregressive schemes given by

F (D)- (i - X D),
i=l

where all permutations of the ? 1 are used, have the same autocorrelation
function. This statement is based on the relationship

co

r- E (EytYtAk)D = F (D)F (D').

k=-Xc

We question the validity of this relationship for a non-stationary time series

where EytUy' is a function of t and where F(D) may not converge. For exam-
ple, the scheme given by

F-1(D) = (1 - X1D)

will be non-stationary when X1> 1, and

F(D) = 1 + X1D + X12D2 +

does not converge.

Even if the above proposal were accepted, it could not deal with a process
with complex roots. We will now examine the applicability of our method to
the case of complex roots greater than one in absolute value. Since complex
roots come in conjugate pairs, let X1 and X2 be such a pair, and the corresponding
right characteristic vectors b1 and b2 (ul and u2 in Quenouille's notation) are

This content downloaded from 128.163.2.206 on Mon, 27 Jun 2016 01:16:55 UTC
All use subject to http://about.jstor.org/terms
NON-STATIONARY LINEAR STOCHASTIC DIFFERENCE EQUATIONS 589

also complex conjugates. So are the left characteristic vectors bl and b2, the
first and second rows of B-1. Therefore, z t=bly, and Z2t=b 2yt are conjugate
complex, as are Eit = blut and E2t = b2ut. From any sample of the pair of complex
canonical time series

Zit = Xizi,t-i + Elt (t = 1, , T)

and

Z2t = X2Z2,t-1 + f2t

we subtract the sample of the complex stationary series

d -1 d
Zit = Xi Z1,t-1 + fit (t = 1, ,T)

and
d -1 d
Z2t = X2 Z2,t-1 + '#2t

where, as before, Zd and Z4t are generated backward in time, with Z4 and Z2
selected at random from their conjugate bivariate stationary process, or set
equal to zero for sufficiently large T. The remaining "trends"

Xit =Zit -Zit = 'Xixio (t =1***,T)

and
d t
X2t = Z2t - Z2t = X2X20

will be conjugate complex.

For the partition of the sample of the original variables, we replace equation
(19) by

Yit = bizt + bi1x1t + bi2x2t (t = 1, , T). (Al)

The contribution of Xi and X2 to the stationary part bizd is real since the sum
bilzdt+bi2z2t is. The non-stationary part is

bi1x1oXt + b?2X20X2t

Letting r and 0 be the absolute value and the angle of the roots, and si and
pi be the absolute value and the angle of the coefficients, we can write the
non-stationary part as (superscript i= V-1)

t( i(Ot+p ) ? et(t?Pi)) 2sirt cos(Ot + p) (A2)

which is an explosive cosine function of time. Although our method of decom-
position is formally applicable to a pair of complex (canonical) time series, we
have to keep in mind the cyclical nature of the non-stationary part eliminated
from the original time series.

This content downloaded from 128.163.2.206 on Mon, 27 Jun 2016 01:16:55 UTC
All use subject to http://about.jstor.org/terms
590 AMERICAN STATISTICAL ASSOCIATION JOURNAL, JUNE 1969

REFERENCES

[11 Chow, G. C., "The acceleration principle and the nature of business cycles," IBM Re-
search Center Report, RC-1919, October 1967, Quarterly Journal of Economics, 82
(August, 1968) 403-418.
[21 Chow, G. C. and Levitan, R. E., "Nature of business cycles implicit in a linear economic
model," IBM Research Center Report, RC-2085, May 1968.
[31 Quenouille, M. H., The Analysis of Multiple Time-Series. London: Charles Griffin,
1957.
[4] Whittle, P., Prediction and Regulation by Linear Least-Square Methods. New York: D.
Van Nostrand Company, 1963.

This content downloaded from 128.163.2.206 on Mon, 27 Jun 2016 01:16:55 UTC
All use subject to http://about.jstor.org/terms