Scribd
Upload
225 views

Chapter 14 - Nonlinear Regression Models

Uploaded by

lieschenlouw2002
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
225 views

Chapter 14 - Nonlinear Regression Models

Uploaded by

lieschenlouw2002
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 20

Chapter 14

NONLINEAR REGRESSION MODELS


Introduction
Majority of models considered in Chapter 1 to 13 – linear in the parameters (or can be
transformed to be)
We must consider nonlinear in the parameter models too
Linear in parameters vs linear in variables
◦ Parameters - (not linear in variables, but linear in the parameters)
◦ Variables - (not linear in parameters, but linear in the variables)

This chapter covers models that are nonlinear in parameters (whether the variables are linear or
not)
Intrinsically linear and intrinsically
nonlinear models
Some models appear nonlinear in the parameters but are inherently / intrinsically linear
because with a suitable transformation they can be converted to linear-in-the-parameters
models.
If nonlinear models cannot be linearized in their parameters, they are intrinsically nonlinear
regression models.
When referring to NLRM in this chapter = Intrinsically nonlinear
Examples 1 – Nonlinear in parameters?

Really?

What if we let ?

ln 𝑌 𝑖 =𝛼+ β 2 ln X i +ui
Model is linear in and
Examples 2 - Nonlinear in parameters?
Really??
ln both sides:
ln 𝑌 𝑖 =𝛽 1+ 𝛽 2 𝑋 𝑖 +𝑢𝑖

Now?

ln
( 1− 𝑌 𝑖
𝑌𝑖 )= 𝛽1 + 𝛽2 𝑋 𝑖 +𝑢𝑖
Estimation of linear and nonlinear
regressions
𝒀 𝒊= 𝜷𝟏 + 𝜷𝟐 𝑿 𝒊 +𝒖𝒊 𝒀 𝒊= 𝜷𝟏 𝒆 𝜷 𝟐 𝑿
+𝒖 𝒊
Estimate by using OLS (minimize RSS) Estimate by using OLS (minimize RSS)

Explicit solutions are obtained of the 2 Cannot obtain explicit solutions of unknowns
unknowns in terms of the data (unknowns on (unknowns both on left-hand and right-hand
the left-hand side and knowns on the right- side of equation)
hand side)
Unknowns are expressed in terms of
themselves and the data
Estimation of linear and nonlinear
regressions
We can therefore apply OLS to estimate NLRM parameters but we can’t obtain explicit solutions
of the unknowns.
OLS applied to NLRM is known as nonlinear least squares
What is the solution?
Estimating NLRM: trial and error
method
Example
◦ Relationship between management fees paid to investment advisors to manage assets and the net asset
value of the fund
Example (continued)
Example continued
Make use of trial and error / iterative process to estimate the model (exponential model)
Initially use and (pure guesses / prior experience)
We have enough info to calculate error sum of squares
With OLS objective is to find values of unknown parameters that will minimize error sum of
squares (this happens if estimated Y values are as close as possible to actual Y values).
How do we know that this is the minimum?
Repeat the process but use and
…..much smaller!
Example continued
Trial and error is an iterative and time-consuming process!
We need an algorithm to help with the process
Approaches to estimating NLRM
Direct search / trial and error
Direct optimization
Iterative linearization
Direct search / trial and error /
derivative-free method
As example covered – this method is generally not used
Disadvantages
◦ If NLRM involves several parameters, method becomes cumbersome and computationally expensive
◦ No guarantee that final set of parameter values selected, will necessarily give absolute minimum error
sum of squares
Direct optimization
Error sum of squares are differentiated with respect to each unknown coefficient, set resulting
equation to zero and solve resulting normal equations simultaneously
These equations cannot be solved explicitly / analytically
Therefore need an iterative (again!) routine
Possible routine: Method of steepest descent
◦ Very involved method
◦ Also requires initial values but proceeds more systematically than trial and error method
◦ Disadvantage: May converge to final values of parameters extremely slowly
Iterative linearization method
Nonlinear equation is linearized around some initial values of the parameters
Linearized equation is then estimated by using OLS and initially chosen values are adjusted
Adjusted values used to relinearize the model and estimated again by using OLS and readjust
estimated values
Process is continued until there is no substantial change in estimated values from last couple of
iterations
Main technique used in linearizing nonlinear equation – Taylor series expansion (calculus)
Estimating NLRM using Taylor series expansion systematized into 2 algorithms
◦ Gauss-Newton iterative method
◦ Newton-Raphson iterative method
Eviews
Quadratic hill climbing
Newton-Raphson
Berndt-Hall-Hall-Hausman

Estimating NLS Models in EViews


It is easy to tell EViews that you wish to estimate the parameters of a model using nonlinear
least squares. EViews automatically applies nonlinear least squares to any regression equation
that is nonlinear in its coefficients. Simply run a regression like you would normally do it and
click OK. EViews will do all of the work of estimating your model using an iterative algorithm.
Properties of nonlinear least squares
estimators
They are NOT normally distributed
They are NOT unbiased
They do NOT have minimum variance in small samples
Therefore
◦ Because we cannot obtain unbiased estimates of the error variance from estimated residuals
◦ We cannot use t-test to test for individual significance
◦ We cannot use F-stat to test for overall significance
◦ Also, because residuals do not necessarily sum to zero and ESS and RSS do not necessarily add up to TSS
◦ R2 may not be a meaningful statistic for such models
◦ But we can compute it as:
Properties of nonlinear least squares
estimators
Inferences about the regression parameters in nonlinear regression are usually based on large-
sample theory.

The least-squares estimators for nonlinear regression models with normal error terms, when the
sample size is large, are approximately normally distributed and almost unbiased and have
almost minimum variance. The large-sample theory also applies when the error terms are not
normally distributed.

Therefore t-statistics and the F-stat are meaningful only if interpreted in the large-sample
context.
Practical this week
No practical application on Chapter 14
Next week…..
Chapter 15 – Part 1.
Remember that you are writing a class test on Thursday 25 July 2024 at 13:00 in D1 lab 108
about the practical application of Chapters 1 to 13 of Gujarati (everything we covered in
Econometrics 3A).

You might also like