5.

NONLINEAR MODELS

[1] Nonlinear (NL) Regression Models

• General form of nonlinear or linear regression models:
yt = h(xt•,β) + εt, εt iid N(0,σ2).
• Assume that the xt• and εt stochastically independent.
→ This assumption implies that:
⎛ ∂h( xt i , β ) ⎞
E⎜ ε t ⎟ = 0k ×1 ;
⎝ ∂ β ⎠
⎛ ∂ 2h ( xt i , β ) ⎞
E⎜ ε t ⎟ = 0 k ×k .
⎝ ∂β ∂ β ′ ⎠

Example 1:

y t = xt • ′ β + ε t .

Example 2:
yt = β1 + β 2 e β3 xt + ε t , where xt is a scalar.

Example 3:
yt = Axtβ22 xtβ33 + ε t . (*)

NLLS-1
[2] Estimation of NL models
Definition:
Let S ( β ) = Σ t ( yt − h ( xt• , β ) ) . The NLLS (Nonlinear least squares)
2

estimator, βˆNL , minimizes S(β).

Assumptions: See Greene, Chapter 9.

Facts:
1) p lim βˆNL = β o (consistent). [No guarantee that E βˆNL = β o .] ( )
−1
⎛ ⎡ 1 ⎤ ⎞
2) T ( βˆNL − β o ) →d N ⎜ 0k ×1 , p lim σ ⎢ Σt H t ( β o ) H t ( β o )′⎥ ⎟ ,
2

⎝ ⎣T ⎦ ⎠

where H t ( β ) = ∂h ( xt• , β ) / ∂β .

⎛ H1 ( β )′ ⎞
⎜ H ( β )′ ⎟
If we define H ( β ) = ⎜ 2 ⎟ , H ( β )′H ( β ) = Σ t H t ( β ) H t ( β )′ .
⎜ : ⎟
⎜ H ( β )′ ⎟
⎝ T ⎠
2
3) Let sNL = S ( βˆNL ) / T , Then, p limT →∞ sNL
2
=σ 2.

(
4) βˆNL ≈ N β , sNL
2
⎡ H ( βˆNL )′H ( βˆNL ) ⎤
⎣ ⎦
−1
).
5) R2 = 1 - SSE/SST, where SSE = S ( βˆNL ) .

→ There is no guarantee that 0 ≤ R2 ≤ 1.

NLLS-2
Example:
yt = β1 + β 2 e β3 xt + ε t .
Then,
∂h ( xt• , β ) ∂h ( xt• , β ) ∂h ( xt• , β )
= 1; = e β3 xt ; = β 2 xt e β3 xt .
∂β1 ∂β 2 ∂β 3

⎛ 1 ⎞
Thus, H t ( β ) = ⎜ e β 3 xt ⎟.
⎜ ⎟
⎜ β x e β3 xt ⎟
⎝ 2 t ⎠

<Sketchy Proof of Consistency>

• From ∂S ( βˆNL ) / ∂β = 0k ×1 ,

ΣTt=1H t ( βˆNL )( yt − ht ( βˆNL )) = 0k ×1 .

• Taylor expansition around β = βo:

0k ×1 ≈ Σt H t ( β o )( yt − ht ( β o ))
⎛ ∂ 2 ht ( β o ) ⎞
+Σt ⎜
⎝ ∂β∂β ′ ⎠
(
( yt − ht ( βo ) ) − H t ( βo ) H t ( βo )′ ⎟ βˆNL − β o . )
→
−1
⎛ ⎛ ∂ 2 ht ( β o ) ⎞⎞
( βˆ − βo ) ≈ − ⎜ Σt ⎜ ( yt − ht ( βo ) ) − H t ( βo ) H t ( β o )′ ⎟ ⎟
⎝ ⎝ ∂β∂β ′
NL
⎠⎠
×Σt H t ( β o )( yt − ht ( β o )).
→

NLLS-3
 −1
⎛ 1 ⎛ ∂ 2 ht ( β o ) ⎞⎞
( βˆ − βo ) ≈ − ⎜ Σt ⎜ ( yt − ht ( βo ) ) − H t ( β o ) H t ( βo )′ ⎟ ⎟
⎝ T ⎝ ∂β∂β ′
NL
⎠⎠
1
× Σt H t ( β o )( yt − ht ( β o ))
T
−1
⎛ 1 ⎛ ∂ 2ht ( β o ) ⎞⎞ 1
≈ − ⎜ Σt ⎜ ε t − H t ( β o ) H t ( β o )′ ⎟ ⎟ Σ t H t ( β o )ε t
⎝ T ⎝ ∂ β ∂ β ′ ⎠⎠ T
−1
⎛1 ⎞
≈ ⎜ Σt H t ( β o ) H t ( β o )′ ⎟ × 0k ×1 = 0k ×1.
⎝T ⎠

<Sketchy Proof of Asymptotic Normality>

• Note that
−1
⎛ 1 ⎛ ∂ 2 ht ( β o ) ⎞⎞
(
T βˆNL − β o ) ≈ − ⎜ Σt ⎜
T ⎝ ∂ β ∂ β ′
( t t o ) t o t o ⎟⎟
y − h ( β ) − H ( β ) H ( β )
⎠⎠
⎝
1
× Σt H t ( β o )( yt − ht ( β o ))
T
−1
⎛1 ⎞ 1
≈ ⎜ Σt H t ( β o ) H t ( β o )′ ⎟ Σt H t ( β o )( yt − ht ( β o )).
⎝T ⎠ T
• But
1 1
Σ t H t ( β o )( yt − ht ( β o )) = Σ t H t ( β o )ε t
T T
⎛ 1 ⎞
→d N ⎜ 0k ×1 , p lim σ 2 Σ t H t ( β o ) H t ( β o )′ ⎟ .
⎝ T ⎠

NLLS-4
[3] SPECIFICATION TESTS

CASE A:
• Same dependent variables under both the null and the alternatives
Ho: yt = ho(xt•,β) + εt. (A)
Ha: yt = ha(wt•,γ) + εt. (B)

Example:
ho(xt•,β) = β1 + β2xt ; ha(wt•,γ) = γ1 + γ2ln(xt).
[Even if the models are linear, we still can perform J or P tests.]

(1) J Test: Davidson and Mackinnon (1981, Econometrica)

• Construct the following auxiliary model:
yt = (1-α)ho(xt•,β) + αha(wt•,γ) + εt. (C)
• If Ho is correct, α = 0.
• Let γˆNL be the NLLS estimator of γ from (B). Replace ha(wt•,γ)

by hˆta = h a ( wt• , γˆNL ) (fitted value of yt from (B)):

yt = (1 − α )h o ( xt• , β ) + α hˆta + error . (D)

• Do NLLS on (D), and estimate β and α jointly. Using the

estimates, we can perform a t-test for Ho′: α = 0. [In the sense
that we estimate β and α jointly, we call the test J-test.]

NLLS-5
(2) P-Test: An alternative to J-test.
• Get βˆNL and γˆNL by NLLS on both (A) and (B).

• Consider the following auxiliary regression:

yt − hˆto = Hˆ to′b + ( hˆta − hˆto )α + error , (E)

where hˆto = h o ( xt• , βˆNL ) , hˆta = h a ( wt• , γˆNL ) and

Hˆ to = H to ( xt• , βˆNL ) .

• Do OLS on (E) and estimate b and α. Then, perform t-test for

Ho′: α = 0.

CASE B:
• Different dependent variables
Ho: yt = ho(xt•,β) + εt.
Ha: g(yt) = ha(wt•,γ) + εt,
where g(yt) is a function of yt (e.g., g(yt) = ln(yt)).

Example:
Ho: yt = β1 + β2xt + εt
Ha: ln(yt) = γ1 + γ2ln(xt) + εt

NLLS-6
• P test:
Estimate both β and γ by NLLS (or OLS).
Construct the following auxiliary model:
yt − hˆto = Hˆ to b + (hˆta − g ( hˆto ))α + error . (F)

Do OLS on (F), and test Ho′: α = 0.

Example:
Ho: yt = xt•′β + εt
Ha: ln(yt) = wt•′γ + εt ,
where xt• = [1,xt2,...,xtk]′ and wt• = [1,ln(xt2),...,ln(xtk)].
Explain in detail how you would test Ho.

NLLS-7
[EXAMPLE]
• Data: (WAGE2.WF1 or WAGE2.TXT – from Wooldridge’s
website)
# of observations (T): 935
1. wage monthly earnings
2. hours average weekly hours
3. IQ IQ score
4. KWW knowledge of world work score
5. educ years of education
6. exper years of work experience
7. tenure years with current employer
8. age age in years
9. married =1 if married
10. black =1 if black
11. south =1 if live in south
12. urban =1 if live in SMSA
13. sibs number of siblings
14. brthord birth order
15. meduc mother's education
16. feduc father's education
17. lwage natural log of wage

Ho: lwaget = β1 + β 2 educt + β 3 exp ert + ε t .

Ha: lwaget = γ 1 + γ 2 ( eductγ 3 + exp ertγ 4 ) + ε t .

NLLS-8
<J-Test>
= STEP 1 =
Estimate the model under Ha:

Dependent Variable: LWAGE

Method: Least Squares
Sample: 1 935
Included observations: 935
Estimation settings: tol= 0.00010, derivs=analytic
Initial Values: C(1)=1.00000, C(2)=1.00000, C(3)=1.00000,
C(4)=1.00000
Convergence achieved after 80 iterations
LWAGE=C(1)+C(2)*(EDUC^C(3)+EXPER^C(4))

Coefficient Std. Error t-Statistic Prob.

C(1) 5.667911 0.444198 12.75986 0.0000

C(2) 0.035137 0.054108 0.649384 0.5163
C(3) 1.221412 0.446902 2.733068 0.0064
C(4) 0.832149 0.431536 1.928344 0.0541

R-squared 0.129527 Mean dependent var 6.779004

Adjusted R-squared 0.126723 S.D. dependent var 0.421144
S.E. of regression 0.393556 Akaike info criterion 0.977083
Sum squared resid 144.1993 Schwarz criterion 0.997791
Log likelihood -452.7862 Durbin-Watson stat 1.786041

Get fitya = lwage - resid

NLLS-9
= STEP 2 =
Estimate lwage = (1 − α )( β1 + β 2 educ + β 3exer ) + α fitya + error .

Dependent Variable: LWAGE

Method: Least Squares
Date: 04/09/02 Time: 12:26
Sample: 1 935
Included observations: 935
Estimation settings: tol= 0.00010, derivs=analytic
Initial Values: C(1)=5.66791, C(2)=0.03514, C(3)=1.22141,
C(4)=0.83215
Convergence achieved after 22 iterations
LWAGE=(1-C(1))*(C(2)+C(3)*EDUC+C(4)*EXPER)+C(1)*FITYA

Coefficient Std. Error t-Statistic Prob.

C(1) -4.692962 2.699983 -1.738145 0.0825

C(2) 5.507546 0.019665 280.0728 0.0000
C(3) 0.077409 0.001155 67.04214 0.0000
C(4) 0.019793 0.000579 34.15696 0.0000

R-squared 0.133671 Mean dependent var 6.779004

Adjusted R-squared 0.130879 S.D. dependent var 0.421144
S.E. of regression 0.392618 Akaike info criterion 0.972312
Sum squared resid 143.5129 Schwarz criterion 0.993020
Log likelihood -450.5558 Durbin-Watson stat 1.793672

Do not reject Ho: C(1) = 0 (α = 0).

NLLS-10
<P-Test>
= STEP 1 =
Estimate the model under Ho:

Dependent Variable: LWAGE

Method: Least Squares
Sample: 1 935
Included observations: 935
LWAGE=C(1)+C(2)*EDUC+C(3)*EXPER

Coefficient Std. Error t-Statistic Prob.

C(1) 5.502710 0.112037 49.11510 0.0000

C(2) 0.077782 0.006577 11.82660 0.0000
C(3) 0.019777 0.003303 5.988419 0.0000

R-squared 0.130859 Mean dependent var 6.779004

Adjusted R-squared 0.128994 S.D. dependent var 0.421144
S.E. of regression 0.393044 Akaike info criterion 0.973413
Sum squared resid 143.9786 Schwarz criterion 0.988944
Log likelihood -452.0704 Durbin-Watson stat 1.787824

Get fity0 = lwage – resid and res0 = resid.

NLLS-11
= STEP 2 =
Estimate the model under Ha:

Dependent Variable: LWAGE

Method: Least Squares
Sample: 1 935
Included observations: 935
Estimation settings: tol= 0.00010, derivs=analytic
Initial Values: C(1)=1.00000, C(2)=1.00000, C(3)=1.00000,
C(4)=1.00000
Convergence achieved after 80 iterations
LWAGE=C(1)+C(2)*(EDUC^C(3)+EXPER^C(4))

Coefficient Std. Error t-Statistic Prob.

C(1) 5.667911 0.444198 12.75986 0.0000

C(2) 0.035137 0.054108 0.649384 0.5163
C(3) 1.221412 0.446902 2.733068 0.0064
C(4) 0.832149 0.431536 1.928344 0.0541

R-squared 0.129527 Mean dependent var 6.779004

Adjusted R-squared 0.126723 S.D. dependent var 0.421144
S.E. of regression 0.393556 Akaike info criterion 0.977083
Sum squared resid 144.1993 Schwarz criterion 0.997791
Log likelihood -452.7862 Durbin-Watson stat 1.786041

Get fitya = lwage – resid.

NLLS-12
= STEP 3 =
Estimate res0 = β1 + β 2 educ + β 3 exp er + α ( fitya − fity 0) .

Dependent Variable: RES0

Method: Least Squares
Sample: 1 935
Included observations: 935
RES0=C(1)+C(2)*EDUC+C(3)*EXPER+C(4)*(FITYA-FITY0)

Coefficient Std. Error t-Statistic Prob.

C(1) 0.027535 0.113031 0.243602 0.8076

C(2) -0.002125 0.006682 -0.317934 0.7506
C(3) 9.34E-05 0.003299 0.028313 0.9774
C(4) -4.692962 2.699983 -1.738145 0.0825

R-squared 0.003235 Mean dependent var 3.38E-15

Adjusted R-squared 0.000023 S.D. dependent var 0.392623
S.E. of regression 0.392618 Akaike info criterion 0.972312
Sum squared resid 143.5129 Schwarz criterion 0.993020
Log likelihood -450.5558 Durbin-Watson stat 1.793672

Do not reject Ho at 5% of significance level.

NLLS-13
[4] NL models with unknown parameters in LHS

• General form: g(yt,δ) = h(xt•,β) + εt, εt iid N(0,σ2).

Example: Generalized Cobb-Douglas Function

ln(yt) + δyt = β1 + β2(1-β3)ln(Kt) + β2β3ln(Lt) + εt.
If δ = 0, the function becomes Cobb-Douglas.

Estimation:
1) NLLS could be inconsistent. Even if it is consistent, computation of
the covariance matrix of the NLLS estimator could be complicated.
2) Do MLE. (See Greene.)

Example: Box-Cox transformation

• Define: yt(δ) = [ytδ - 1]/δ ; xtj(λ) = [xtjλ - 1]/λ, j = 2, ... , k.
• Assume that λ is all the same for j, we can allow λ to vary over
different j.
• If λ = 1, xtj(1) = xtj - 1 (linear). If λ → 0, xtj(λ) → ln(xtj) (log).

NLLS-14
• Box-Cox Model:
• yt (δ ) = β1 + Σ kj =2 xtj (λ ) β j + ε t .

• yt(δ) = xt• (λ)′β + εt,

where xt• (λ) = [1,xt2(λ),...,xtk(λ)]′ and β = [β1,...,βk]′.
• Use MLE to estimate δ, λ and β.

NLLS-15