Scribd
Upload
141 views

Lecture 10 Nonlinear Regression

This document provides an overview of estimating nonlinear regression models. It discusses: 1) Nonlinear regression models where the dependent variable is a nonlinear function of regressors plus an error term. Estimation is done using nonlinear least squares (NLS), maximum likelihood (ML), or generalized method of moments (GMM). 2) The NLS estimator solves a system of nonlinear equations numerically. Under certain conditions, it is consistent, asymptotically normal, and efficient. 3) GMM provides an alternative estimator that is robust to endogenous regressors and unknown forms of heteroskedasticity or autocorrelation. An example asset pricing model is presented to illustrate GMM.

Uploaded by

Martin Stojanovic
Copyright
© © All Rights Reserved
Available Formats
Download as DOC, PDF, TXT or read online on Scribd
141 views

Lecture 10 Nonlinear Regression

This document provides an overview of estimating nonlinear regression models. It discusses: 1) Nonlinear regression models where the dependent variable is a nonlinear function of regressors plus an error term. Estimation is done using nonlinear least squares (NLS), maximum likelihood (ML), or generalized method of moments (GMM). 2) The NLS estimator solves a system of nonlinear equations numerically. Under certain conditions, it is consistent, asymptotically normal, and efficient. 3) GMM provides an alternative estimator that is robust to endogenous regressors and unknown forms of heteroskedasticity or autocorrelation. An example asset pricing model is presented to illustrate GMM.

Uploaded by

Martin Stojanovic
Copyright
© © All Rights Reserved
Available Formats
Download as DOC, PDF, TXT or read online on Scribd
You are on page 1/ 10

Lecture 10

Estimating Nonlinear Regression Models

References:

Greene, Econometric Analysis, Chapter 10


Consider the following regression model:
yt = f(xt, ) + t t = 1,,T

xt is kx1 for each t, is an rx1constant vector, t is an


unobservable error process and f is a (sufficiently
well-behaved) function - f: RkxRr R. So, each y is
a (fixed) function of x and plus an additive error
term, .
Example: y x
t 1 t
2
t

The estimation problem: given f, y1,,yT, and


x1,,xT, estimate .

The solution: estimate by LS (NLS), ML, or GMM.

The problem? In contrast to the linear regression


case, the FOCs are nonlinear and so, in general,
numerical methods must be applied to obtain
(consistent) point estimates. Also, the avar matrice of
-hat will have a slightly more complicated form.

Nonlinear models are commonly encountered in


applied economics largely because advances in
computational mathematics and desktop/laptop
computer technology have made solving nonlinear
optimization problems more feasible and more
reliable.
Nonlinear Least Squares (NLS)
Choose -hat to minimize the SSR
T
SSR( ) ( yt f ( xt , )) 2
t 1

FOCs
T
f ( xt , )
g ( ) ( yt f ( xt , )) 0
t 1

which form a set of r nonlinear equations in the r


unknowns, ,..., . [In the case where f is linear in the
1 r

s, the derivative vector df/d = [ x1txrt], r = k.]

Example: yt 1 xt 2 t

T
g ( ) ( yt 1 xt 2 )[ xt 2 2 1 xt 2 1 ]' 0

t 1

Computing the NLS Estimator


In general, these FOCs must be solved numerically
to find the NLS estimator of , . (E.g., the Gauss-
NLS

Newton procedure described in Greene, 10.2.3.)

Some issues

- choice of algorithm
- selecting an initial value for -hat
- convergence criteria
- local vs. global min
Asymptotic Properties of the NLS Estimator

If
the xs are weakly exogenous
the errors are serially uncorrelated and
homoskedastic
the function f is sufficiently smooth
the {xt,t} process is sufficiently well-behaved
then
T 1 / 2 ( T , NLS ) N (0, 2 Q 1 )
D

where
2 = var(t)
T
Q p lim(1 / T )QT , QT [f ( xt , T ) / ][f ( xt , T ) / ' ]
t 1

The NLS estimator is (under appropriate conditions),


consistent, asymptotically normal and asymptotically
efficient.

Inference: For large samples act as though

NLS ~ N ( , 2QT )
1

T
2 (1 / T ) t2
1

If the disturbances are heteroskedastic and/or serially


correlated the NLS estimator will be consistent but
not asymptotically efficient. Also, the correct form of
the asymptotic variance matrix of the NLS estimator
requires a heteroskedasticity and/or autocorrelation
correction. Heteroskedasticity and HAC estimators of
the variance-covariance matrix of can be used if the
exact forms of the heteroskedasticity and
autorcorrelation are not know.

If the form of the heteroskedasticity and/or serial


correction is known up to a small number of
parameters (e.g., t is known to be an AR(1) process
with unknown ) then nonlinear GLS or (quasi)-
maximum likelihood will be asymptotically efficient
estimators.

Example GNLS

Suppose E() = . Then the GNLS estimator of is


the value of that minimizes the weighted SSR:

[y-f(x, )] -1[y-f(x, )]

If then it can be replaced with a consistent


estimator to obtain the FGNLS estimator. (What
consistent estimator of ?)
If the regressors are correlated with the errors, none
of these estimators is consistent (even if the errors are
homoskedastic and serially uncorrelated). A
consistent, semi-parametrically efficient estimator
that does not rely on knowledge of the form/existence
of heteroskedasticity/autocorrelation and allows for
endogenous regressors: Nonlinear GMM

In addition, GMM provides a semi-parametric


alternative to MLE for nonlinear models that do not
fit the nonlinear regression format.
GMM in the nonlinear regression model

Consider the population moment conditions:

E[wt(yt f(xt,))] = 0 for all t

where wt is an instrument vector.

The GMM estimator: choose to make the


corresponding sample moments
1 T
wt ( yt f ( xt , ))
T 1

close to zero. As in the linear case, this will


involve minimizing an optimally weighted
quadratic form in these moments.
GMM in a more general nonlinear setting -

Hansen and Singletons (Econometrica, 1982)


Consumption-Based Asset Pricing Model

At the start of each time period t, a representative


agent chooses consumption and saving to maximize
expected discounted utility:


ct 1
E[ iU (ct i ) t ] , U (ct ) ,0<<1
i 0

At the start of t, the agent can allocate income to


purchase the consumption good or N assets with
maturities 1,2,,N according to the sequence of
budget constraints
N N
ct p j , t q j ,t rj ,t q j , t j wt
j 1 j 1

where
pj,t = price of a unit of asset j (i.e., an asset that
matures in t+j) in period t
qj,t = units of asset j purchased in period t
rj,t = payoff in period t of asset j purchased in t-j
wt = labor income in period t

Unknown parameters in this model- ,


The optimal consumption path must satisfy the
sequence of Euler equations:
E[ j ( r j ,t j / p j ,t )(ct j / ct ) 1 t ] 0

Let zt be any vector in t. Then the Euler equations


imply the following set of moment conditions which
form the basis for estimating and by GMM

E[ j ( r j ,t j / p j ,t )(ct j / ct ) 1 zt ] 0

for all t and j=1,...N

GMM: Choose and to make the j sample


moments
T
1
j

T
(r
1
j ,t j / p j ,t )(ct j / ct ) 1 zt , j = 1,,N

close to zero.

The alternative MLE: specify the joint distribution


for {(rj,t+j/pj,t+j,ct+j/ct), j = 1,2,,N} then maximize the
corresponding likelihood function subject to the
Euler equations. (See, e.g., Hansen and Singleton,
JPE, 1983).

You might also like