Ch.

9 Heteroscedasticity
(June 13, 2016)

1 Introduction
Regression disturbances whose variance are not constant across observations are het-
eroscedastic. There are several reasons why the disturbance of εi may be variable,
some of which are as follows.1

(a). Following the error-learning models, as people learn, their errors of behavior be-
come smaller over time.

(b). As income grows, people have more discretionary income and hence more scope
for choice about the disposition of their income. Hence, σi2 is likely to increase
with income.

(c). As data collecting techniques improve, σi2 is likely to decrease. etc. 

In the heteroscedastic model, the variances of the disturbances are

V ar(εi |x) = σi2 , i = 1, 2, ..., N.

We continue to assume that the disturbances are pairwise uncorrelated. Thus,

 2 
σ1 0 . . . 0
 0
 σ22 . . . 0  
 . . . . . . 
E(εε0 |x) = 
 .
.
 . . . . .  
 . . . . . . 
2
0 0 . . . σN
1
See Gujarati (2003), Basic Econometrics. p. 389.

1
Ch.9 Heteroscedasticity 1 INTRODUCTION

It will sometimes prove useful to write σi2 = σ 2 ωi . Hence

 
ω1 0 . . . 0
 0 ω2 . . . 0 
 
0 2 . . . . . . 

E(εε |x) = σ  ,
 . . . . . .  
 . . . . . . 
0 0 . . . ωN

= σ 2 Σ,

where
 
ω1 0 . . . 0

 0 ω2 . . . 0 

 . . . . . . 
Σ= . (9-1)

 . . . . . . 

 . . . . . . 
0 0 . . . ωN
This form is an arbitrary scaling which allows us to use a normalization,
N
X
tr(Σ) = ωi = N.
i=1
PN
σ2
(For example, σ 2 = i=1N
i
.) This makes the classical regression with homoscedastic
disturbances a simple special case with ωi = 1, i = 1, 2, ..., N .

Example.
See Figure 11.1 (p.216) of Greene 5th edition. 

r 2016 by Prof. Chingnun Lee 2 Ins.of Economics,NSYSU,Taiwan

Ch.9 Heteroscedasticity 2 TESTING FOR HETEROSCEDASTICITY

2 Testing for Heteroscedasticity

One can rarely be certain that the disturbances are heteroscedastic however, and un-
fortunately, what form the heteroscedasticity takes if they are. As such, it is useful to
be able to test for homoscedasticity and if necessary, modify our estimation procedure
accordingly.
Most of the test for heteroscedasticity are based on the following strategy. OLS
estimator is a consistent estimator of β even in the presence of heteroscedasticity. As
such, the OLS residuals will mimic, albeit imperfectly because of sampling variability,
the heteroscedasticity of the true disturbance. Therefore, tests designed to detect het-
eroscedasticity will, in most cases, be applied to the OLS residuals.

2.1 Nonspecific Tests for Heteroscedasticity

2.1.1 White’s General Test

White (1980) addressed the case where nothing is known about the structure of the
heteroscedasticity other than the heteroscedastic variance σi2 are uniformly bounded.
It would be desirable to be able to test a general hypothesis of the form:

H0 : σi2 = σ 2 f or all i,
H1 : N ot H0 .

If there is no heteroscedasticity (under H0 ), then s2 (X0 X) will give a consistent

estimator of variance β̂, where if there is, then it will not (see Ch. 8 sec.1). The
correct covariance matrix for the OLS is estimated by

V\
ar(β̂)HAC
T
!
−1 −1
X
= (X0 X) e2t xt x0t ) . (X0 X)
t=1

White derives a test for heteroscedasticity which consists of comparing the elements
of N S0 (= N
P 2 0 2 0 2
PN 0
i=1 et xt xt ) and s (X X)(= s t=1 xt xt ), thus indicating whether or not
the usual OLS formula s2 (X0 X) is a consistent covariance estimator. Large discrep-
ancies between N S0 and s2 (X0 X) support the contention of heteroscedasticity while
small discrepancies support homoscedasticity.

r 2016 by Prof. Chingnun Lee 3 Ins.of Economics,NSYSU,Taiwan

Ch.9 Heteroscedasticity 2 TESTING FOR HETEROSCEDASTICITY

A simple operational version of this test is carried out by obtaining N R2 in the

regression of e2i on a constant and all unique variables in x ⊗ x. This statistics is
asymptotically distributed as χ2p , where p is the number of regressors in the regression,
including the constant.

2.2 Specific Tests for Heteroscedasticity

If nothing is known a priori other than the heteroscedastic variance are uniformly
bounded, White (1980) general test is applicable.
There may be instance when the form of the heteroscedasticity is not known, but
nevertheless, it is known that the disturbance variance in monotonically related to the
size of a known exogenous variable Z by which observations on the dependent variable
Y can be ordered. One frequently used test in this instance is the Goldfeld-Quandt
test.
When it is believed that the broader class of heteroscedasticity is σi2 = h(z0i α),
where h(·) is a general function independent of i, is applicable (such as σi2 = z0i α,
σi2 = (z0i α)2 and σi2 = exp(z0i α)). If so, the Breush-Pagan test is appropriate.

2.2.1 The Goldfeld-Quandt Test

A very popular test for determining the presence of heteroscedasticity which is mono-
tonically related to an exogenous variable by which observations on the dependent
variable can be ordered is the Goldfeld-Quandt (1965) test.
For the GoldfeldQuandt test, we assume that the observations can be divided into
two groups in such a way that under the hypothesis of homoscedasticity, the disturbance
variances would be the same in the two groups, whereas under the alternative, the
disturbance variances would differ systematically. The steps of this test are as follow:

(a). Order the observations (from “supposed” large to small variance) by the values
of the variables Z.

(b). Choose p central observations and omit them, provides (N − p)/2 > k.

r 2016 by Prof. Chingnun Lee 4 Ins.of Economics,NSYSU,Taiwan

Ch.9 Heteroscedasticity 2 TESTING FOR HETEROSCEDASTICITY

(c). Fit separate regression by OLS to the two groups, with N1 and N2 observations,
respectively.

(d). Let SSE1 and SSE2 denote the sum of squared residuals based on the large vari-
ance (which you suppose they do) and the small variance group, respectively. 

e01 e1 e02 e2
Recall that σ12
∼ χ2[N1 −k] and σ22
∼ χ2[N2 −k] , then the statistics

e01 e1 /N1 − k SSE1 /N1 − k

F = 0
= ∼ F[N1 −k,N2 −k] ,
e2 e2 /N2 − k SSE2 /N2 − k
under the null hypothesis of homoscedasticity σ12 = σ22 = σ 2 ,

2.2.2 The Breush-Pagan Test

The Goldfeld-Quandt test has been found to be reasonably powerful when we are able
to identify correctly the variable to use in the sample separation. This requirement
does limit its generality, however. Breush-Pagan (1979) assume a border class of het-
eroscedasticity defined by

σi2 = σ 2 h(α0 + z0i α),

where zi is a (p × 1)vector of exogenous variables. This model is homoscedastic if

α = 0.
Breush and Pagen (1979) consider the general estimation equation
ê2i
= α0 + z0i α + vi ,
σ̄ 2
where êi represents the i − th OLS residual and σ̄ 2 = N 2
P
i=1 êi /N . The null hypothesis
α1 = 0 can be tested if the εi are normally distributed. Let SSR denote the sum of
squares obtained in an OLS estimation of
ê2i
= α̂0 + z0i α̂ + v̂i .
σ̄ 2
ê2
Denote Wi = σ̄i2 , W̄ = N
P 0
PN
i=1 Wi /N , and Ŵ i = α̂ 0 + zi α̂ 1 . Then SSR = i=1 (Ŵi −
2
W̄ ) . Breush and Pagan show, if α = 0, then
1
SSR ∼ χ2p .
2
r 2016 by Prof. Chingnun Lee 5 Ins.of Economics,NSYSU,Taiwan
Ch.9 Heteroscedasticity 2 TESTING FOR HETEROSCEDASTICITY

Exercise 1 .
Reproduce the results of Example 11.3 at p.224 of Greene 5th edition. 

r 2016 by Prof. Chingnun Lee 6 Ins.of Economics,NSYSU,Taiwan

Ch.9 Heteroscedasticity 3 OLS ESTIMATION

3 OLS Estimation
We showed in Section 8.2 that in the presence of heteroscedasticity, the OLS estimator
β̂ is unbiased and consistent. However it is inefficient relative to the GLS estimator.

3.1 Estimating the Appropriate Covariance Matrix for OLS Estimators

If the type of heteroscedasticity is known with certainty, then the OLS estimator is
undesirable; we should use the GLS instead. The precise form of the heteroscedasticity
is usually unknown, however. In that case, GLS is not usable, and we may need to
salvage what we can from the results of OLS estimators.
The conventional estimated covariance matrix for the OLS estimator σ 2 (X0 X)−1 is
inappropriate; the appropriate matrix is σ 2 (X0 X)−1 X0 ΣX(X0 X)−1 . White (1980) has
shown that it is still possible to obtain an appropriate covariance estimator of the OLS
estimators even the form of heteroscedasticity is unknown. What is actually required
is an estimate of
N
1 2 0 1 X 2
Ω = σ X ΣX = σi xi x0i .
N N i=1

White (1980) shows that under very general conditions, the matrix
N
1 X 2
S0 = e xi x0i ,
N i=1 i

where ei is the i − th OLS residual, is a consistent estimator of Ω. Therefore, the

White estimator,
−1 −1
V\
ar(β̂) = N (X0 X) S0 (X0 X) , (9-2)

can be used as an estimator of the true variance of the OLS estimator. Inference con-
cerning β is still possible by means of OLS estimator even when the specific structure
of Σ is not specified as β̂ is normally distributed asymptotically.
More generally, White shows that tests of the general linear hypothesis Rβ = q,
under the null hypothesis, the statistics
−1 −1 L
(Rβ̂ − q)0 [R(X0 X) N S0 (X0 X) R0 ]−1 (Rβ̂ − q) −→ χ2m ,

r 2016 by Prof. Chingnun Lee 7 Ins.of Economics,NSYSU,Taiwan

Ch.9 Heteroscedasticity 3 OLS ESTIMATION

where m denote the number of restrictions imposed.

Exercise 2 .
Reproduce the results at Table 11.1 on p. 221 of Greene 5th edition. 

r 2016 by Prof. Chingnun Lee 8 Ins.of Economics,NSYSU,Taiwan

Ch.9 Heteroscedasticity 4 GLS (WEIGHTED LEAST SQUARES)

4 GLS (Weighted Least Squares)

Having tested for and found evidence of heteroscedasticity, the logical next is to revise
the estimation technique to account for it if Σ is known. The GLS estimator is

β̃ = (X0 Σ−1 X)−1 X0 Σ−1 y. (9-3)

Consider the most general case, σi2 = σ 2 ωi . Then Σ−1 is a diagonal matrix whose
i − th diagonal element is 1/ωi (See Eq. (9-1)), that is
 1  1 
0 . . . 0
 √
ω1
0 . . . 0
ω1
 0 1 . . . 0  1
 0 √ . . . 0
 
 ω2  ω2 
. . . . . .   . . . . . .
 
Σ−1 = 
 
 =  ×

 . . . . . .  . . . . . . 
 
 . . . . . .   .

. . . . .


1
0 0 . . . ωN 0 0 . . . √1
ωN
 1 
√
ω1
0 . . . 0
 0 √1 . . . 0 
 ω2 
 . . . . . .
 

 . . . . . .
 

 . . . . . .
 

0 0 . . . √1
ωN

= P0 P.

The GLS is obtained by regressing (See Eq (8-18))

 √   0 √ 
Y1 / ω1 x 1 / ω1
 Y2 /√ω2   x0 2 /√ω2 
   
 .   . 
Py =   on PX =  .
 . 


 . 

 .   . 
√ 0 √
YN / ωN x N / ωN
Applying OLS to the transformed model, we obtain the GLS estimator, which is
also called weighted least squares (WLS) estimator,
" N #−1 " N #
X X
β̃ = wi xi x0i wi xi Yi ,
i=1 i=1

where wi = 1/ωi .
The logic of the computation is that observations with smaller variances receive a
large weight in the computations of the sums and therefore have greater influence in

r 2016 by Prof. Chingnun Lee 9 Ins.of Economics,NSYSU,Taiwan

Ch.9 Heteroscedasticity 4 GLS (WEIGHTED LEAST SQUARES)

the estimate obtained.

Example.
A common specification in linear regression model with heteroscedasticity is that the
variance of the disturbances is proportional to one of the regressors or its square. For
example, if the model is

Yi = β1 Xi1 + β2 Xi2 + · · · + βk Xik + εi , i = 1, 2, ..., N,

where

σi2 = σ 2 Xil2 ,

then
1
 
2
X1l
0 . . . 0
1

 0 2
X2l
. . . 0 

. . . . . .
 
Σ−1 = ,
 
 . . . . . . 
 
 . . . . . . 
1
0 0 . . . 2
XN l

and
1
0 . . . 0
 
X1l
1

 0 X2l
. . . 0 

 . . . . . . 
P= .

 . . . . . . 

 . . . . . . 
1
0 0 . . . XN l

Hence, the transformed regression model for the GLS is

       
Yi Xi1 Xi2 Xil Xik εi
= β1 + β2 + ... + βl + ... + βk +
Xil Xil Xil Xil Xil Xil
     
Xi1 Xi2 Xik εi
= βl + β1 + β2 + ... + βk + ,
Xil Xil Xil Xil
 2
σ2 X 2
where E Xεiil = X 2 il = σ 2 , ∀i.
il
If the variance σi2 is proportional to Xil instead of Xil2 , then the weight applied to
√
each observation is 1/ Xil instead of 1/Xil . 

r 2016 by Prof. Chingnun Lee 10 Ins.of Economics,NSYSU,Taiwan

Ch.9 Heteroscedasticity 5 ESTIMATION WHEN Σ IS UNKNOWN

5 Estimation When Σ is Unknown

The general form of the heteroscedastic regression model has too many parameters
to estimate by ordinary method as shown in Section 8.4. Typically, the model is re-
stricted by formulating σ 2 Σ as a function of a few parameters, such as σi2 = σ 2 Xilα or
σi2 = σ 2 [z0i α]2 . Write this as Σ(α), FGLS based on a consistent estimator of Σ(α) is
asymptotically equivalent to GLS. The new problem is that we must first find consis-
tent estimators of the unknown parameters in Σ(α). Two methods are typically used,
two step GLS and maximum likelihood.

5.1 FGLS, Two-Step Estimation

For the heteroscedastic model, the GLS estimator is
" N   #−1 " N   #
X 1 X 1
β̃ = 2
xi x0i xi Yi .
i=1
σi i=1
σi2

The two step estimators are computed by first obtaining estimators σ̂i2 , usually
using some function of the OLS residuals, then the FGLS will be
" N   #−1 " N   #
X 1 X 1
β̌ = 2
xi x0i xi Yi . (9-4)
i=1
σ̂i i=1
σ̂i2

The OLS estimator β̂, although inefficient, is still consistent. As such, statistics
computed using the OLS residual, ei = (Yi − x0i β̂), will have the same asymptotic
properties as those computed using the true disturbance, εi = (Yi − x0i β)
Let

ε2i = σi2 + vi ,

where vi is just the difference between the random variable ε2i and its expectation.
Since εi is unobservable, we would use the OLS residual, for which

ei = εi − x0i (β̂ − β) = εi + ui .
p
But in large sample, as β̂ −→ β, terms in ui will become negligible, so that at least
approximately,

e2i = σi2 + vi∗ .

r 2016 by Prof. Chingnun Lee 11 Ins.of Economics,NSYSU,Taiwan

Ch.9 Heteroscedasticity 5 ESTIMATION WHEN Σ IS UNKNOWN

The procedure suggested is to treat the variance function as a regression and use
the squares of the OLS residual as the dependent variable. For example, if σi2 = z0i α,
then a consistent estimator of α will be the OLS in the model2
e2i = z0i α + vi∗ , i = 1, 2, ..., N. (9-5)
Having obtained the estimated α̂ in the first step from (9-6), then we substitute
= z0i α̂ into Eq. (9-5), we finish the second step and the FGLS estimator is thus
σ̂i2
obtained.
The two-step estimator may be iterated by recomputing the residuals after com-
puting the FGLS estimate and then reentering the computation
Eq.(9−6) Eq.(9−5) Eq.(9−6) Eq.(9−5)
OLS → β̂ → e −→ α̂(1) −→ β̌ (1) → ě(1) −→ α̂(2) −→ β̌ (2) →
→ ě(2) → .....,
where
ě(1) = y − Xβ̌ (1) .

5.2 Maximum Likelihood Estimation

The log-likelihood function for a sample of normally distributed observations with
heteroscedastic variance is
N  
N 1X 2 1 0 2
ln L = − ln(2π) − ln σi + 2 (Yi − xi β) .
2 2 i=1 σi
For simplicity, let
σi2 = σ 2 fi (α),
where α is the vector of unknown parameters in Σ(α) and fi (α) is indexed by i to
indicate that is a function of zi . Assume as well that no elements of β appear in α.
The log-likelihood function is
N    
N 2 1X 1 1 0 2
ln L = − [ln(2π) + ln σ ] − ln fi (α) + 2 (Yi − xi β) .
2 2 i=1 σ fi (α)

In this model, vi∗ may be both heteroscedastic and autocorrelated, so α̂ is consistent but inefficient.
2

But, consistency is all that is required for asymptotically efficient estimation of β using Σ(α̂).

r 2016 by Prof. Chingnun Lee 12 Ins.of Economics,NSYSU,Taiwan

Ch.9 Heteroscedasticity 5 ESTIMATION WHEN Σ IS UNKNOWN

For convenience in what follows, substitute εi for (Yi − x0i β), denote fi (α) as simply
fi , and denote the vector of derivatives ∂fi (α)/∂α as gi . Then the derivatives of the
log-likelihood functions are
N
∂ ln L X εi
= xi 2 ,
∂β i=1
σ fi
N N 
1 X ε2i
 2 
∂ ln L N X 1 εi
= − 2+ 4 = −1 ,
∂σ 2 2σ 2σ i=1 fi i=1
2σ 2 σ 2 fi
N  
ε2i
 
∂ ln L X 1 1
= 2
−1 gi .
∂α i=1
2 σ fi fi

The maximum likelihood estimators are those values of β, σ 2 , and α that simultane-
ously equate these derivatives to zero. The likelihood equations are generally highly
nonlinear and will usually require an iteration solution.

Exercise 3
Reproduce the results at Table 11.2 on p.231 of Greene 5th edition. 

r 2016 by Prof. Chingnun Lee 13 Ins.of Economics,NSYSU,Taiwan

Ch.9 Heteroscedasticity 6 ARCH MODEL

6 ARCH Model
Heteroscedasticity is often associated with cross-sectional data, whereas time series are
usually studied in the context of homoscedastic processes. In analyses of macroeco-
nomic data, Engle (1982, 1983) and Cragg (1982) found evidence that for some kinds
of data, the disturbance variances in time-series models were less stable than usually
assumed.
With time-series data, it is not uncommon to see that the OLS residuals to be quite
small in absolute value for a number of successive periods of time, then much larger for
a while, then smaller again, and so on. This phenomenon of time-varying volatility (or
disturbances occur in clusters) is often encountered in models for stock returns, foreign
exchange rates, and other series that are determined in financial markets. Numerous
models for dealing with this phenomenon have been proposed. One very popular
approach is based on the concept of autoregressive, conditionally heteroscedastic, or
ARCH, that was introduced by Engle (1982). The basic idea of ARCH models is that
the variance of the disturbance at time t depends on the realized values of squared
disturbances in previous time periods.
A model which allows the conditional variance to depend on the past realization of
the series is considered in the following. Suppose that
p
ut = ht εt (9-6)
ht = α0 + α1 u2t−1 , (9-7)

with E(εt ) = 0 and V ar(εt ) = 1, then this is an example of what will be called an
autoregressive conditional heteroscedasticity (ARCH(1)) model.

Example.
See Figure 11.3 on p.239 of Greene 5th edition. 

r 2016 by Prof. Chingnun Lee 14 Ins.of Economics,NSYSU,Taiwan

Ch.9 Heteroscedasticity 6 ARCH MODEL

Figure (9-1). An Example of Volatility Clustering.

6.1 Population’s Properties of ARCH Models

6.1.1 The Conditional Mean and Variance

Let Ft−1 denote the information set available at time t − 1. The conditional mean of
ut is
p
E(ut |Ft−1 ) = ht · E(εt |Ft−1 ) = 0. (9-8)

From (9-9) it implies that the conditional variance of ut is

σt2 = V ar(ut |Ft−1 )

= E{[ut − E(ut |Ft−1 )]2 |Ft−1 }
= E(u2t |Ft−1 ) (since E(ut |Ft−1 ) = 0)
= E(ht ε2t |Ft−1 )
= E(ε2t )E(α0 + α1 u2t−1 |Ft−1 )
= α0 + α1 u2t−1
= ht ,

so ut is conditionally heteroscedastic. From the structure of the model, it is seen that

large past squared shocked shocks u2t−i , i = 1, .., m imply a large conditional variance
σt2 (= V ar(ut |Ft−1 )) for this variable ut . Consequently, ut tends to assume a large value.
This means that, under the ARCH framework, large shocks tend to be followed by
another large chock. This feature is similar to the volatility clustering observed in asset

r 2016 by Prof. Chingnun Lee 15 Ins.of Economics,NSYSU,Taiwan

Ch.9 Heteroscedasticity 6 ARCH MODEL

returns.

6.1.2 The Conditional Density

By assuming that εt is a Gaussian variate, the condition density of ut given all the
information update to t − 1 is
p p
f (ut |Ft−1 ) = ht f (εt |Ft−1 ) = ht · N (0, 1) ∼ N (0, ht ).

6.1.3 The Unconditional Mean and Variance

The unconditional mean of ut is

E(ut ) = E[E(ut |Ft−1 )] = E(0) = 0. (9-9)
While ut is conditional heteroscedastic, the unconditional variance of ut is
V ar(ut ) = V ar[E(ut |Ft−1 )] + E[V ar(ut |Ft−1 )]
= 0 + α0 + α1 E(u2t−1 )
= α0 + α1 V ar(ut−1 ).
If the process generating the disturbance ut is weakly stationary, then
V ar(ut ) = V ar(ut−1 )
= α0 α1 V ar(ut−1 )
α0
= . (9-10)
1 − α1
For this ratio to be finite and positive, we require that α0 > 0 and |α1 | < 1.
Moreover, since E(ut |Ft−j ) = 0, so
E(ut ut−j |Ft−j ) = ut−j E(ut |Ft−j ) = 0.
Hence
E(ut ut−j ) = E[E(ut ut−j |Ft−j )] = 0. (9-11)
Based on (9-10)–(9-12), ut follows ideal conditions.

r 2016 by Prof. Chingnun Lee 16 Ins.of Economics,NSYSU,Taiwan

Ch.9 Heteroscedasticity 6 ARCH MODEL

6.2 Linear Regression Model With ARCH(1) Disturbance

Suppose that we are interested in estimating the parameter of a regression model with
ARCH disturbances. Let the regression equation be
Yt = x0t β + ut .
Here xt denote a vector of predetermined explanatory variables, which could include
lagged value of Y . The disturbance term ut is assumed to satisfy (9-7) and (9-8).
Because as is shown in last section that ut is satisfied the classical assumptions,
the OLS estimator of β is most efficient linear estimator according to Gauss-Markov
theorem.
i.i.d.
But there is a more efficient nonlinear estimator. If εt ∼ N (0, 1), then conditioned
on starting value, the sample conditional log likelihood function is then
T T
T 1X 1 X (Yt − x0t β)2
l(θ) = − ln(2π) − ln(σt2 ) − , (9-12)
2 2 t=1 2 t=1 σt2
where
σt2 = α0 + α1 u2t−1
= α0 + α1 (Yt−1 − x0t−1 β)2 , (9-13)
and the vector of parameters to be estimated θ = (β 0 , α0 , α1 )0 . For a given numerical
value for the parameter vector θ, the sequence of conditional variances can be calculated
from (9-14) and used to evaluate the log likelihood function (9-13). This can then be
maximized numerically using the methods described in Chapter 3.

NSYSU from a Sea Looking.

End of this Chapter

r 2016 by Prof. Chingnun Lee 17 Ins.of Economics,NSYSU,Taiwan