Econ 616 – Spring 2006

Qualitative Response Regression Models

Presented by Yan Hu

06/07/25 Econ 2062 1

 Outline
 Qualitative Response Regression Model

 Binary Response Regression Models

1. The Linear Probability Model (LPM)
2. The Logit Model
3. The Probit Model

06/07/25 Econ 2062 2

What is Qualitative Response
Regression Model?
 The dependent variable is qualitative
(or dummy) in nature.
--- The dependent variable is a
binary, or dichotomous variable: Y=1
if the person is in the labor force and
Y=0 if he or she is not.
--- Trichotomous response variable.
--- Poly-chotomous (or multiple-
category) response variable.
06/07/25 Econ 2062 3
Binary Response Regression Models

 E(Y) is related to the X’s through a

link function g( E(Y) ) = X.
 In binary regression, a link function
specifies a relationship between E(Y)
(the probability of Y=1, which is also
the expected value of Y) and a linear
composite score of X's.

06/07/25 Econ 2062 4

Three Binary Response Regression
Models

 The Linear Probability Model (LPM)

 The Logit Model
 The Probit Model

06/07/25 Econ 2062 5

What’s Linear Probability Model?
• The linear probability model is the regression model applied to
a binary dependent variable. To fix ideas, consider the
following simple model:

Y = β0 + β 1X1 + ui
Where X = family income
Y = 1 if the family owns a house
= 0 if the family does not own a house
Ui = the disturbance term
• The independent variable Xi can be discrete or continuous
variable

• E(Yi/Xi) the conditional expectation of Yi given Xi, can be

interpreted as the conditional probability that the event will
occur given Xi; that is, Pr(Yi = 1/Xi).
06/07/25 Econ 2062 6
Assuming E(Ui) = 0, as usual (to obtain unbiased estimators), we obtain
E(Yi/Xi) = β0 + β 1X1
•Let’s us denote Pi = probability that Yi = 1 (that is, that the event occurs)
and 1 – Pi = probability that Yi = 0 (that is, that the event does not
occur), the variable Yi has the following distributions:

Yi Prob.
1 Pi
0 1-Pi
Total 1

By the definition of mathematical expectation,

E(Yi) = 0 (1 – Pi) + 1(Pi) = Pi
we can equate it as follows
E(Yi/Xi) = Yi = β0 + β 1X1 = Pi
•That is, the conditional expectation of the model (1) can, in fact, be
interpreted as the conditional probability of Yi.
•Since the probability Pi must lie between 0 and 1, we have the restriction
0  E (Yi/Xi)  1
06/07/25 Econ 2062 7.7
Problems of LPM (1)
1. Non-normality of the disturbances:  i Yi  1   2 X i

 Ui follows the Bernoulli distribution :

ui Probability
Yi=1 1  1   2 X i Pi
Yi=0  1   2 X i (1-Pi)

 Problem may not be so critical. If the objective is point

estimation, the normality assumption of disturbance is
not necessary and the OLS still remain unbiased. As
the sample size increases indefinitely, the OLS
estimators tend to be normally distributed
06/07/25 Econ 2062 8
 Problems of LPM (2)
2. Heteroscedastic variances of the disturbances:While the interpretation of the

parameters is unaffected by having a binary outcome, several assumptions of the LPM are
necessarily violated.

Yi = β0 + β 1X1+ ui

When Yi=1, then Ui=1- β0 - β 1X1 , with Pi=1

Yi=0, then Ui=- β0 - β 1X1 , with Pi=0

Then, Var(Ui) = E(Ui2) = (β0 - β 1X1 i)2 (1-Pi) + (1- β0 - β 1X1 )2 (Pi)
•Hence, it is hetrosckedastic, because it depends on the value taken by X.
•Thus the OLS estimator of βi is inefficient and the standard errors are biased, resulting in
incorrect test.
Var(ui)=Pi(1-Pi), the variance is a function of the mean (Pi).
One way to solvewthe heteroskedasticity is to transform the model by dividing it by
i  p i (1 - p i )
the weights . Then, estimate the transformed equation by OLS.
Yi 1 Xi ui
  2 
wi wi wi wi

06/07/25 Econ 2062 9

Problems of LPM (3)
3. Nofulfillment of 0 E (Yi / X i ) 1

 Two ways of finding out whether the estimated Yˆ lie

i
between 0 and 1:
1. Estimate the LPM by the usual OLS method. If some Yˆi
are less than zero,Yˆi is assumed to be zero for
those cases; if they are greater than 1, they are
assumed to be 1.
2. Devise an estimating technique that will guarantee
that the estimated conditional probabilities Yˆi will lie
between 0 and 1, such as logit and probit models.
06/07/25 Econ 2062 10
Problems of LPM (4)
4. Questionable value of R2 as a measure of
goodness of fit.
 For a given X, the Y values will be either 0 or 1.
Therefore, all the Y values will either lie along the X-
axis or along the line corresponding to 1. Therefore,
generally no LPM is expected to fit such a scatter so
well. As a result, the conventionally computed R2 is
likely to be much lower than 1 for such models.

 Aldrich and Nelson contend that “use of the

coefficient of determination as a summary statistic
shoud be avoided in models with qualitative
dependent variable.”

06/07/25 Econ 2062 11

What is the Logit Model?

 The cumulative logistic distrubution:

P = E(Y=1|X) = 1/(1+e-βX)
P
1

0 X
06/07/25 Econ 2062 12
What is the Logit Model?
 From the logistic distribution,
1-P = e-βX / (1+e-βX)
P/(1-P) = eβX, odds ratio
log[p/(1-P)] = βX

 Link function: g=log[ p/(1-p) ], where p is

the probability of either Y=1 or Y=0,
depending on the software.

 Generally, log[ p/(1-p) ]=X.

06/07/25 Econ 2062 13
Two Types of Data

 To estimate the value of logit log[

p/(1-p) ]=X, we have to distinguish
two types of data:

--- Data at the individual, or micro, level

--- Grouped or replicated data

06/07/25 Econ 2062 14

 Data at the Individual Level
 X: family income,
FAMILY Y X
Y=1 if the family 1 0 8
owns a house and 0 2 1 16
if it does not own a 3 1 18
house. The 4 0 11
following table 5 0 12
gives data on 6 1 19
individual families. 7 1 20
8 0 13
9 0 9

06/07/25 Econ 2062 15

 Grouped or Replicated Data
 The following table Income N n
shows data on several 6 40 8
families grouped 8 50 12
according to income 10 60 18
level and the number of 13 80 28
families owning a house 15 100 45
at each income level.
20 70 36
Corresponding to each
25 65 39
income level Xi, there
30 50 33
are Ni families, ni among
35 40 30
whom are home owners.
40 25 20

06/07/25 Econ 2062 16

Steps in Estimating the Logit
Regression (Grouped Data)
 For each income level X, compute the probability of
owning a house as Pi^=ni/Ni.
 For each Xi, obtain the logit as Li^=log[Pi^/(1-Pi^)]
 To resolve the problem of heteroscedasticity,
Wi=NiPi^(1-Pi^)
(Wi)0.5Li = β1(Wi)0.5+ β2(Wi)0.5Xi+(Wi)0.5ui
or Li* = β1(Wi)0.5+ β2Xi*+vi
 Estimate above function by OLS on the transformed
data.
 Establish confidence intervals and/or test hypotheses
in the usual OLS framework.

06/07/25 Econ 2062 17

SAS Program
Proc Import Out= Work.incomes
Datafile= "c:\yan\econ616\DG-15.4.xls";
Run;

data incomes1;
set incomes;
phat=n1/n;
lhat=log(phat/(1-phat));
w=n*phat*(1-phat);
wsquar=sqrt(w);
lstar=round(lhat*wsquar, 0.0001);
xstar=round(income*wsquar, 0.0001);
run;

proc reg data=incomes1;

model lstar = wsquar xstar / NOINT;
run;

06/07/25 Econ 2062 18

SAS Output
Variable DF Paramete Standard t Value Pr > |t|
r Error
Estimator
wsquar 1 -1.59324 0.11150 -14.29 <.0001

xstar 1 0.07867 0.00545 14.44 <.0001

The estimated slope coefficient suggests

that for a unit ($1000) increase in weighted
income, the weighted log of odds in favor
of owning a house goes up by 0.08 units.

06/07/25 Econ 2062 19

Odds Interpretation
 The odds ratio:

Zi
Pi 1 e Zi  1.59324W *i 0.07867 X *i
  Zi
e e
1  Pi 1  e
 For a unit increase in weighted income, the
(weighted) odds in favor of owing a house
increase by 1.082 (e0.07867) or about 8.17%.

06/07/25 Econ 2062 20

 An Example of Individual Data
 In the following table,
Y=1 if a student’s final OBS GPA TUCE PSI GRADE LETTER

grade in an 1 2.66 20 0 0 C
intermediate 2 2.89 22 0 0 B
microeconomics
3 3.28 24 0 0 B
course was A and Y=0
if the final grade was B 4 2.92 12 0 0 B

or C. GPA, TUCE, and 5 4 21 0 1 A

Personalized System 6 2.86 17 0 0 B
of Instruction (PSI) are
7 2.76 17 0 0 B
grade predictors.
8 2.87 21 0 0 B

06/07/25 Econ 2062 21

SAS Program
Proc Import Out= Work.gpagrade
Datafile= "c:\yan\econ616\DG-15.7.xls";
Run;

proc print data=gpagrade;

run;

Proc Logistic data=gpagrade ;

Model grade (event='1') = gpa tuce psi;
run;
/* or */
proc probit data=gpagrade;
class grade;
model grade = gpa tuce psi / d=logistic itprint;
run;

06/07/25 Econ 2062 22

 Output
Standard Wald
Parameter DF Estimate Error Chi-Square Pr> ChiSq

Intercept 1 -13.0204 4.9310 6.9723 0.0083

GPA 1 2.8259 1.2629 5.0072 0.0252
TUCE 1 0.0951 0.1415 0.4518 0.5015
PSI 1 2.3785 1.0645 4.9925 0.0255

Testing Global Null Hypothesis: BETA=0

Test Chi-Square DF Pr > ChiSq
Likelihood Ratio 15.4042 3 0.0015
Score 13.3088 3 0.0040
Wald 8.3762 3 0.0388
06/07/25 Econ 2062 23
Interpretation
 Each slope coefficient is a partial slope and
measures the change in the estimated logit
for a unit change in the value of the given
regressor (holding other regressors
constant).
 Odds interpretation. For example, students
who are exposed to the new method of
teaching are more than 10.7887 (e 2.3785)
times to get an A than students who are not
exposed to it, other things remaining the
same.
06/07/25 Econ 2062 24
What’s the Probit Model
 Probit link: p= (h), where p is the
cumulative distribution function of a
standard normal variate.
 Pi=P(Y=1|X)=P(Ii*≤Ii)=P(Zi≤β1+β2Xi)=
(β1+β2Xi), where P(Y=1|X) means the
probability that an event occurs given the
values of the X, and where Zi~N(0,σ2).
 β1+β2Xi= -1(Pi), where -1 is the inverse of
the normal CDF.
06/07/25 Econ 2062 25
Use of Probit Model

 Probit model is used when Y is considered as

the “manifestation” of some unobservable
Gaussian-distributed latent variable in the data.
 For example, the decision of the family to own
a house or not depends on an unobservable
index I (latent variable), that is determined by
one or more explanatory variables, say income
X, in such a way that the larger the value of the
index I, the greater the probability of a family
owning a house.

06/07/25 Econ 2062 26

 Probit Estimation with Grouped
Data
 Method 1:
1. Calculate Pi^=N1/N.
2. Estimate Ii= -1(Pi^), where  is the
standard normal CDF.
3. Estimate β1 and β2 from Ii, i.e., β1+β2Xi= Ii.

 Method 2:
Use SAS or R program directly.

06/07/25 Econ 2062 27

 Program
SAS: R:
incomes <-
Proc Import Out= Work.incomes as.data.frame(matrix(scan(),ncol=3,
Datafile= "c:\yan\econ616\DG- byrow=T))
15.4.xls"; 6 40 8

Run; 8
10
50
60
12
18
13 80 28
15 100 45
proc genmod data=incomes; 20 70 36

class ; 25
30
65
50
39
33
model n1/n = income / dist = bin 35 40 30
40 25 20
Link =
probit names(incomes) <- c(“income”,”N”, “N1”)
lrci; N0 <- incomes$N- incomes$N1
run; glmA <- glm(cbind(N1, N0)~income,
incomes, family=binomial(link=”probit”))

06/07/25 Econ 2062 28

Output
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -0.988138 0.122144 -8.090 5.97e-16 ***
income 0.048587 0.005995 8.105 5.28e-16
***
---
Signif. codes: 0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1

(Dispersion parameter for binomial family taken to be 1)

Null deviance: 72.7581 on 9 degrees of freedom

Residual deviance: 2.3456 on 8 degrees of freedom
AIC: 49.002

Number of Fisher Scoring iterations: 3

06/07/25 Econ 2062 29
Interpretation
 We want to find out the effect of a unit change in X
(income) on the probability that Y=1, that is, a
family purchases a house.
1. The rate of change of the probability with respect to
income:
dP i
 f ( 1   2 X i )  2
dX i
2. If X=6 (thousand dollars), the normal density
function of f[-0.988138 + 0.048587(6)]=f(-
0.6966)=0.313.
3. 0.313*0.048587=0.0152. Starting with an income
level of $6000, if the income goes up by $1000, the
probability of a family purchasing a house goes up
by about 1.52%.
06/07/25 Econ 2062 30
Probit Model for Individual Data
 SAS program:

Proc Import Out= Work.gpagrade

Datafile= "c:\yan\econ616\DG-15.7.xls";
Run;
proc probit data=gpagrade;
class grade;
model grade = gpa tuce psi;
run;

06/07/25 Econ 2062 31

 Output

Analysis of Parameter Estimates

Standard 95% Confidence Chi-

Parameter DF Estimate Error Limits Square Pr > ChiSq

Intercept 1 7.4523 2.5425 2.4692 12.4355 8.59 0.0034

GPA 1 -1.6258 0.6939 -2.9858 -0.2658 5.49 0.0191
TUCE 1 -0.0517 0.0839 -0.2162 0.1127 0.38 0.5375
PSI 1 -1.4263 0.5950 -2.5926 -0.2601 5.75 0.0165

06/07/25 Econ 2062 32

Marginal Effect of Change in
Regressor
 Holding the effect of all other variables constant.
1. LPM: slope coefficient measures directly the
change in the probability of an event occurring as a
result of a unit change in the value of a regressor.
2. Logit model: the slope coefficient of a variable
gives the change in the log of the odds associated
with a unit change in that variable. The rate of
change in the probability of an event happening is
given by βjPi(1-Pi).
3. Probit model: the rate of change in the probability
is given by βj f(Xβ), where f is the density function
of the standard normal variable.

06/07/25 Econ 2062 33

Logit or Probit?
 In most applications, the models are quite similar, the
main difference being that the logistic distribution has
slightly fat tails.
 There is no compelling reason to choose one over the
other.
 In practice, many researchers choose the logit model
because of its comparative mathematical simplicity.
P
1 probit
logit

0
06/07/25 Econ 2062 34
Reading

 Damodar N. Gujarati,
Basic Econometrics,
P580-615

06/07/25 Econ 2062 35