EIE3002/EII3002

Time Series Analysis

Lecture 7

Characterizing cycles
 1. Covariance Stationary Time
Series
• If the underlying probabilistic structure of the
series were changing over time, there would be
no way to predict the future accurately on the
basis of the past, because the laws governing the
future would differ from those governing the
past.
• If we want to forecast a series, at a minimum
we’d like its mean and its covariance structure
(i.e., the covariances between current and past
values) to be stable over time, in which case we
say that the series is covariance stationary.
• A covariance stationary series requires:
i. the mean of the series to be stable over time;
E ( yt ) =  , for all t, instead of E ( yt ) = t .
ii. its covariance structure be stable over time.
Quantifying stability of the covariance structure
can be done using the autocovariance function.
The autocovariance at displacement τ is just the
covariance between yt and yt-τ.
 (t , ) = cov( yt , yt − ) = E ( yt −  )( yt − −  )
In some books, “displacement” also called as
“Lag”.
• If the covariance structure is stable over time, as
required by covariance stationarity, then the
autocovariances depend only on displacement, τ,
not on time, t, and we write
 (t , ) =  ( )
• Example, see Excel file named “covariance”.
• The autocovariance function provides a basic
summary of cyclical dynamics in a covariance
stationarity series.
• Note that the autocovariance function is
symmetric; that is,
 ( ) =  (−  )
• Typically, we’ll consider only nonnegative values
of τ.
• Also note that
 (0) = cov( yt , yt ) = E ( yt −  )( yt −  ) = var ( yt ).

• Recall that the correlation between two random

variables x and y is defined by

corr(x, y ) =
cov( x, y )
,
 x y
where corr(x,y) Є [-1,1].
• It is simply the covariance, “normalized” or
“standardized” by the product of the standard
deviation of x and y.

Autocorrelation function (ACF).

• Similarly, we can translate autocovariance
function to autocorrelation function (ACF) by
dividing the autocovariance function by the
variance,
cov( yt , yt − )  ( )  ( )
 ( ) = = =
var ( yt ) var ( yt − )  (0)  (0)  (0)
 *by covariance stationarity,
var ( yt − ) =  (0).
• Note that we always have

 (0)
 (0) = = 1,
 (0)
because any series is perfectly correlated with itself.
• Thus, the autocorrelation at displacement 0 isn’t
of our interest.
• Only autocorrelation beyond displacement 0
inform us about a series’ dynamic structure.
Sample Autocorrelations:
• Suppose we have a sample of data on a time
series, and we don’t know the true model that
generated the data, or the mean, or
autocorrelation function associated with the true
model.
• Instead , we want to use the data to estimate the
mean and autocorrelation function, which we
might then use to help us learn about the
underlying dynamics and to decide on a suitable
model or set of models to fit to the data.
• The autocorrelation at displacement τ for the
covariance stationary series y is given by
E (( yt −  )( yt − −  ))
 ( ) =
(
E ( yt −  )
2
) .

• The estimator, viewed as a function of τ, is called

the sample autocorrelation function or
correlogram. Given a sample of size T, then
T

 (( y − y )( yt − − y )) 1 T
, where y =  yt .
t
ˆ ( ) = t = +1
T T t =1
 (y − y)
2
t
t =1
Partial Autocorrelation Function (PACF).
• The partial autocorrelation function (PACF), p( )
is sometimes useful.
• The autocorrelations and partial autocorrelations,
although related, differ in an important way.
• The autocorrelations are just the “simple” or
“regular” correlations between yt and yt-τ.
• The partial autocorrelations, on the other hand,
measure the association between yt and yt-τ after
controlling for the effects of yt-1,…., yt-τ+1.
• As with the autocorrelations, we often graph the
partial autocorrelations as a function of τ and
examine their qualitative shape.
• In general, a partial correlation is a conditional
correlation.
• It is the correlation between two variables under
the assumption that we know and take into
account the values of some other set of variables.
• For instance, consider a regression context in
which y is a response variable and x1, x2, and x3 are
predictor variables. The partial correlation
between y and x3 is the correlation between the
variables determined taking into account how
both y and x3 are related to x1 and x2.
• In regression, this partial correlation could be
found by correlating the residuals from two
different regressions: (1) regression in which we
predict y from x1 and x2, (2) regression in which we
predict x3 from x1 and x2. Basically, we correlate
the “parts” of y and x3 that are not predicted
by x1 and x2.
• More formally, we can define the partial
correlation as
Cov ( y, x3 | x1 , x2 )
Var ( y | x1 , x2 ) Var ( x3 | x1 , x2 )
• Note that this is also how the parameters of a
regression model are interpreted. Think about the
difference between interpreting the regression
models:
y =  0 + 1 x2 +  and y =  0 + 1 x1 +  2 x2 + 
• In the first model, β1 can be interpreted as the
linear dependency between x2 and y. In the
second model, β2 would be interpreted as the
linear dependency between x2 and y with the
dependency between x1 and y already accounted
for.
• For a time series, the partial autocorrelation
between yt and yt-τ is defined as the conditional
correlation between yt and yt-τ, conditional on yt-τ+1,
... , yt-1, the set of observations that come between
the time points t and t−τ.
• The 1st order partial autocorrelation will be defined
to equal the 1st order autocorrelation.
• The 2nd order (lag) partial autocorrelation is
Cov ( yt , yt − 2 | yt −1 )
Var ( yt | yt −1 ) Var ( yt − 2 | yt −1 )
• This is the correlation between values two time
periods apart conditional on knowledge of the value
in between. (By the way, the two variances in the
denominator will equal each other in a stationary
series.)
• The 3rd order (lag) partial autocorrelation is
Cov ( yt , yt −3 | yt −1 , yt − 2 )
Var ( yt | yt −1 , yt − 2 ) Var ( yt −3 | yt −1 , yt − 2 )

• And, so on, for any lag.

• Typically, matrix manipulations having to do with the
covariance matrix of a multivariate distribution are
used to determine estimates of the partial
autocorrelations.
• Given that  1  (1)   ( − 1) 
 
  (1) 1   ( − 2 )
P = 
    
 
  ( − 1)  ( − 2 )  
 1 
• Partial autocorrelation at displacement τ is given by
P*
p ( ) =
P
• Where 𝑃𝜏∗ is equal to the matrix 𝑃𝜏 , in which the
diagonal element of τ-th column is replaced by 𝜌 𝜏 .
• |.| indicates determinant.
• Since this can be applied for various orders of 𝜏, in
the end we obtain a partial autocorrelation function
(PACF).
• From definition, there is no difference between PACF
and ACF of order 1; 𝑝 1 = 𝜌 1 .
• For order 2, 1  (1)
 (1)  (2 )  (2 ) −  (1)2
p(2 ) = =
1  (1) 1 −  (1)
2

 (1) 1
Sample Partial Autocorrelations:
• Sample partial autocorrelation, 𝑝Ƹ 𝜏 can be
computed using sample autocorrelation, 𝜌ො 𝜏 .
• For instance, sample partial autocorrelation at
displacement 1 and 2 are:

pˆ (1) = ˆ (1)

ˆ (2) − ˆ (1)
2
pˆ (2) =
1 −  (1)
ˆ 2

* Note that the first partial autocorrelation is always

equal to the first autocorrelation.
 2. White noise
• The simplest of all time series process is the
fundamental building block from which all others
are constructed.
• Suppose that yt =  t
 t ~ (0,  ) ,
2

where y denotes the observed series of interest

and the “shock”, εt, is uncorrelated over time.
• We say that εt, and hence yt, is serially
uncorrelated.

• Such a process, with zero mean, constant

variance, and no serial correlation, is called zero-
mean white noise, or simply white noise.

• Sometimes we write
 t ~ WN (0,  2
)
and hence
(
yt ~ WN 0,  2 .)
• Because white noise is, by definition,
uncorrelated over time, all the autocovariances,
and hence all the autocorrelations, are 0 beyond
displacement 0.
• Formally, the autocovariance function for a white
noise process is
 2 ,  = 0
 ( ) = 
0,   1,
and the autocorrelation function for a white
noise is 1,  = 0
 ( ) = 
0,   1,
• Finally, consider the partial autocorrelation
function for a white noise series.
• Population regressions of yt on yt-1, or on yt-2 or
on other lags, produce nothing but 0 coefficients,
because the process is serially uncorrelated.
• Formally, the partial autocorrelation function of a
white process is
1,  = 0
p ( ) = 
0,   1.
 3. Inference for the Autocorrelation,
and Partial Autocorrelation Functions.
• It is often of interest to assess whether a series is
reasonably approximated as white noise, which is
to say whether all its autocorrelations are 0 in
population.
• If a series is white noise, then the distribution of
the sample autocorrelations in large sample is

 1
ˆ ( ) ~ N  0, .
 T
• Thus, if the series is white noise, then
approximately 95% of the sample autocorrelations
should fall in the interval  2 / T .
• In practice, when we plot the sample
autocorrelations for a sample of data, we typically
include the “two-standard-error bands”, which are
useful for making informal graphical assessments
of whether and how the series deviates from
white noise.
• The two-standard-error bands, although very
useful, only provide 95% bounds for the sample
autocorrelations taken one at a time.
• Ultimately, we’re often interested in whether a
series is white noise, that is whether all its
autocorrelations are jointly 0.
• This can be done using the Box-Piearce
Q-statistic, m
QBP ( ) = T   ( ) ,
ˆ 2

 =1

which is approximately distributed as a m
2

random variable under the null hypothesis that y

is white noise.
• A slight modification of the Box-Pierce Q-statistic,
known as Ljung-Box Q-statistic designed to
 2
follow more closely the distribution in small
samples, is given by
m 
ˆ ( ) 
 2
QLB ( ) = T (T + 2)  

 =1  T −  
• Under the null hypothesis that y is white noise,
QLB is approximately distributed as a  m random
2

variable.
• As m grows relative to T, the quality of the
distributional approximations deteriorates. Thus
in practice, focusing on m in the neighborhood of
T is often reasonable.
 4. Application: Characterizing
Canadian Employment Dynamics
Figure 7.9 Canadian Employment Index
115

110
Canadian Employment Index

105

100

95

90

85

80
65 70 75 80 85 90

Time
• Here, we examine a quarterly, seasonally adjusted
index of Canadian employment, Q1 1962 – Q4
1993.
• The series displays no trend and no seasonality.
• It, however, appear highly serially correlated. It
evolves in a slow, persistence fashion.
• To get a feel for the dynamics operating in the
employment series, we perform a correlogram
analysis.
• The results are given in Table 7.1.
Table 7.1 Canadian Employment Index, Correlogram.
• Q-statistic and p-value are calculated under the
null hypothesis of white noise for values of m.
• The p-value is consistently 0 to four decimal
places, so the null hypothesis of white noise is
decisively rejected.
• The sample autocorrelations are very large relative
to their standard errors and display slow one-sided
decay.
• The sample partial autocorrelations, in contrast,
are large relative to their standard errors at first
but statistically negligible beyond displacement 2.
• It’s clear that employment has a strong cyclical
component; all diagnostics reject the white noise
hypothesis immediately.