Applied econometrics for

ManageMent (MgMt MSC 5411)

Chapter FOUr:
Discrete choice MoDels
(non-linear regression analysis)

Teklebirhan Alemnew (Assistant Professor)

[email protected]
AAU, 2024

By: Teklebirhan A. 1
 Outline
1. Introduction
2. Linear Regression Model (LPM)
3. Non-linear Regression models
3.1. The logit and Probit models
3.2. Multinomial logit and Probit models
3.3. Ordered logit and Probit models

By: Teklebirhan A. 2
1. Introduction

By: Teklebirhan A. 3
 Cont…
Determinates of employees satisfaction in AAU.
Factors affecting loan repayment in AAU.
Factors affecting Turnover Intention in CBE.
Determinants of women’s willingness to practice
family planning utilities in the case of AA town.
Determinants of Household’s Saving behavior in AA
town.

By: Teklebirhan A. 4
 Cont…
In the above examples, response (dependent)
variable is qualitative/ categorical/discrete.
Models dealing with such kind of binary responses
are called binary/ Discrete choice models.
Technically, it is possible to estimate the binary
choices using OLS.
If OLS is used to estimate qualitative response
variable, the resulting model is called linear
probability model (LPM).
By: Teklebirhan A. 5
 Cont…
Are you satisfied with your job?

By: Teklebirhan A. 6
 Cont…
Value of Yi Ui Probability of Yi Probability of Ui

1 p
0 1-p 1-p

Both Yi and Ui take only two values with

respective probabilities and such variables are
called Bernoulli variables and follow Bernoulli
distributions.
But, OLS assumes that both Yi and Ui are
normally distributed.
By: Teklebirhan A. 7
Cont…

By: Teklebirhan A. 8
Cont…

9
By: Teklebirhan A.
 Cont…
The very difference between linear regression
model (LRM) and binary regression model
(BRM) is that
In the case of LRM, regression analysis deals
with the prediction of the average value of the
response variable from the given values of
explanatory variables.
 This is because, the response variable is
continuous/quantitative in LRM.

By: Teklebirhan A. 10
Cont…

By: Teklebirhan A. 11
2. Linear Probability Model

By: Teklebirhan A. 12
Cont…

By: Teklebirhan A. 13
 Cont…
(Not Always)

e) The true relationship between a binary outcome and

a continuous explanatory variable is inherently
nonlinear
 This means the functional form of the LPM is
generally not correctly specified, which can lead to
biased estimates of some parameters of interest.
By: Teklebirhan A. 14
 Cont…
 Due to the above five basic limitations of using OLS on
qualitative response variable model, non-linear regression
models were developed.
 Example:

By: Teklebirhan A. 15
 Cont…
Violation of the axiom of Probability
reg grade gpa pc ase
predict yhat
scatter grade yhat||lfit grade yhat

Violation of the assumption of Heteroscedasticity

rvfplot, yline(0)
hettest

By: Teklebirhan A. 16
 Cont…
Violation of the assumption of normality
predict r, resid
pnorm r
qnorm r
mvtest norm r
kdensity r, normal
histogram r, kdensity normal
The coefficient of determination is not
dependable which is 0.4 2
By: Teklebirhan A. 17
 3. Nonlinear Regression Models
In the LPM, the heteroskedasticity problem is less
worrying as it can be easily handled.
We need to resort to other methods to account for
the other shortcomings.
In particular, we need a model which satisfies
0

By: Teklebirhan A. 18
 Cont…
Non-linear regression model includes
a) The Logit Model
b) The Probit Model
c) Multinomial Logit and Probit Model (MNL &
MNP)
d) Ordered Logit and Probit Model.

By: Teklebirhan A. 19
3.1. The Logit and Probit Models

Non-linearity
By: Teklebirhan A. 20
Cont…

By: Teklebirhan A. 21
Cont…

By: Teklebirhan A. 22
 Cont…

Satisfied
Not-Satisfied

By: Teklebirhan A. 23
 Cont…
Take the ratio of the probability of an event
occurring (Pi) to the probability of an event not
happening (1-Pi) and the resulting ratio is called
odds ratio.

By: Teklebirhan A. 24
 Cont…
Take the natural log of the above odds ratio and the
resulting equation is called Logit.

Where, Li is called Logit which is linearly related

with Xi is explanatory variables

By: Teklebirhan A. 25
 Cont…
Pi

Cumulative Normal Distribution Function

Pi =1

Logistic Distribution Function

By: Teklebirhan A. 26
Cont…

By: Teklebirhan A. 27
 Cont…
Example on Logit and Probit
Suppose that we want to examine the effect of routine
weekly exercises on the performance of students.
To this end, suppose we gave routine exercises to
MBA student and at the end of the semester, we
found average scores in exercise (ASE) for each
student.

By: Teklebirhan A. 28
Cont…

By: Teklebirhan A. 29
 Cont…
A) Logit Interpretation of Logit Model
logit grade gpa ase pc

By: Teklebirhan A. 30
 Cont…
Interpretation:
As GPA increases by one point, the log of the odds ratio
increases by 2.8 and statistically significant.
A student who owned PC, the log of the odds ratio
increases by 2.4 and statistically significant.

By: Teklebirhan A. 31
 Cont…
B) Odds Ratio Interpretation of Logit Model
logit grade gpa ase pc, or

By: Teklebirhan A. 32
 Cont…
Interpretation:
As GPA increases by one point, the odds of getting ‘A’ is
16.87 times the odds of getting other grades (B, C, D, F)
A student who owned PC, the odds of getting ‘A’ is 10.8
times the odds of getting other grades (B, C, D, F)

By: Teklebirhan A. 33
 Cont…
C) Probability (mfx) interpretation of the logit model

By: Teklebirhan A. 34
 Cont…
Interpretation:
As GPA increases by one point, the probability of getting
grade ‘A’ increases by 53%.
A student who owned PC, the probability of getting grade
‘A’ increases by 45.6%.

By: Teklebirhan A. 35
 Cont…
D) Probit Estimation
probit grade gpa ase pc

NB: the interpretation is similar with Logit estimation

By: Teklebirhan A. 36
 Cont…
E) Probability Interpretation of Probit Model

By: Teklebirhan A. 37
 Cont…
Logit/Probit Model Diagnostic Tests
Multicollinearity Test
 vif
 Heteroscedasticity Test
 hettest
Model specification/omitted variable Test
 linktest
NB: In any of the discrete choice models, we first run a linear
regression to test for multicollinearity using VIF command,
and then test for multicollinearity via the vif command.
By: Teklebirhan A. 38
 Cont…
Multicollinearity Test
. vif

Variable VIF 1/VIF

ase 1.19 0.840735

gpa 1.18 0.850226
pc 1.01 0.987262

Mean VIF 1.13

Heteroscedasticity Test
. hettest

Breusch-Pagan / Cook-Weisberg test for heteroskedasticity

Ho: Constant variance
Variables: fitted values of grade

chi2(1) = 2.53
Prob > chi2 = 0.1117

39
By: Teklebirhan A.
 Cont…
Link Test/Model specification test

grade Coef. Std. Err. z P>|z| [95% Conf. Interval]

_hat .9551764 .383456 2.49 0.013 .2036165 1.706736

_hatsq -.0453861 .1881828 -0.24 0.809 -.4142177 .3234455
_cons .0817277 .6074585 0.13 0.893 -1.108869 1.272324

 The insignificant hat square shows that the model

has no error on its formula and no omission of
significant variable.
40
By: Teklebirhan A.
 3.2 Multinomial Logit and Probit
Model
The probability that a particular consumer will choose a
particular alternative is given by the probability that the utility
of that alternative to that consumer is greater than the utility
to that consumer of all other alternatives.

Then the consumer picks the alternative that maximizes his or

her utility.

MNL model is a simple extension to the logit model

when the dependent variable can take more than two
categorical values.
By: Teklebirhan A. 41
 Cont…
For instance, in Addis Ababa, a person may have the
following choice of means of transportation to go to
work place.
 Car, Bus, Train
A person may have three voting options:
 Labor party
 Conservative party
 liberal democrat party
A respondent is provided with more than two
alternatives and s/he is expected to choose one.
By: Teklebirhan A. 42
 Cont…
There is no order within the categories of Y (any of a
chosen categories can be the baseline for comparison).

In Multinomial Logit Model (MLM), a response variable

with K categories will generate K-1 equations.

That means, in multinomial logit model we have K-1

equations instead of one equation.

That is why Multinomial logit models are called multi-

equation models.
By: Teklebirhan A. 43
 Cont…
Each of these K-1 equations is a binary logistic
regression comparing a group with the reference group.

The choice of reference /base category is arbitrary.

Example: if our dependent variable Y= 1, 2, 3 then

our reference category is 1, then we will have two logit
equations,
 First equation : Y=2 versus Y=1
 Second equation: Y=3 versus Y=1
By: Teklebirhan A. 44
 Cont…
The probabilities for all the categories of Y(all the possible
outcomes for our dependent variable) add to 1 or 100%.
 That means, P1 + P2 + P3=1

The multinomial logit is equivalent to running a series of

separate binary logit models to find the coefficients, but these
would not give us a single overall measures of our model.

Multinomial logistic regression simultaneously estimates the

K-1 logit coefficients through MLE.
Hence, if the first category is the reference, then for m= 2,
…, M.
By: Teklebirhan A. 45
 Cont…

Hence, for each case, there will be M-1 predicted log

odds, one for each category relative to the reference
category.

When there are more than 2 groups, computing

probabilities is a little more complicated than it was
in logistic regression. For j = 2, …, M,
By: Teklebirhan A. 46
Cont…

By: Teklebirhan A. 47
 Cont…
In other words, you take each of the M-1 log odds
you computed and exponentiate it.

Once you have done that the calculation of the

probabilities is straightforward.

Note that, when M=2, the mlogit and logistic

regression models (and for that matter the ordered
logit model) become one and the same.

By: Teklebirhan A. 48
Cont…

By: Teklebirhan A. 49
 Cont…
Multinomial Example
Suppose that we want to study the determinants of
rural households income diversification:
On-farm,
Local off farm &
Migration
Data on diversification, education, gender and age
were collected from a total of 500 households from
Kebele X using simple random sampling technique.

By: Teklebirhan A. 50
Cont…

By: Teklebirhan A. 51
 Cont…
Basic Commands for using MNL model:
 describe
 summarize
 tabulate divers
 mlogit divers edu age sex exp, baseoutcome(1)
 mlogit divers edu sex age exp, baseoutcome(1), rrr
 mfx, predict(outcome (1))
 mfx, predict(outcome (2))
 mfx, predict(outcome (3))
 predict plogit1 plogit2 plogit3
 summarize plogit1 plogit2 plogit3
By: Teklebirhan A. 52
 Cont…

. summarize

Variable Obs Mean Std. Dev. Min Max

divers 500 2.358 .7062816 1 3

edu 500 11.652 1.644477 8 17
sex 500 .89 .3132031 0 1
age 500 45.1 10.84275 22 88
exp 500 1.378 1.656685 0 7

By: Teklebirhan A. 53
 Cont…

. tabulate divers

income
diversificatio
n Freq. Percent Cum.

on farm 67 13.40 13.40

local off farm 187 37.40 50.80
migration 246 49.20 100.00

Total 500 100.00

By: Teklebirhan A. 54
 Cont…
Multinomial logistic regression Number of obs = 500
LR chi2(8) = 241.43
Prob > chi2 = 0.0000
Log likelihood = -372.34821 Pseudo R2 = 0.2448

divers Coef. Std. Err. z P>|z| [95% Conf. Interval]

on_farm (base outcome)

local_off_farm
edu -.5263692 .0975398 -5.40 0.000 -.7175437 -.3351948
age .0144084 .0146985 0.98 0.327 -.0144001 .0432169
sex -12.47745 527.1158 -0.02 0.981 -1045.605 1020.651
exp .8035401 .2313668 3.47 0.001 .3500694 1.257011
_cons 18.60435 527.1174 0.04 0.972 -1014.527 1051.736

migration
edu -.1060083 .0926298 -1.14 0.252 -.2875594 .0755427
age -.0015199 .0157332 -0.10 0.923 -.0323565 .0293166
sex -15.74113 527.1155 -0.03 0.976 -1048.869 1017.386
exp 1.40202 .2312467 6.06 0.000 .9487852 1.855256
_cons 16.90534 527.1172 0.03 0.974 -1016.225 1050.036

By: Teklebirhan A. 55
 Cont…
As education increases by one year, the log of the
ratio of the two probabilities, P(off farm=2)/P(on-
farm=1) will decrease by 0.52, and the log of the
ratio of the two probabilities P(migration=3)/P(on-
farm=1) will decrease by 0.11.

Therefore, the more the level of education the more

households will choose on-farm income
diversification than migration and local off farm
participation.
By: Teklebirhan A. 56
 Cont…
The ratio of the probability of choosing one category
over the probability of choosing the reference category
is often referred to as relative risk ratio (Odds
Ratio).
So another way of interpreting the multinomial
regression results is in terms of relative risk ratio
(odds ratio) ==see the following slide.

By: Teklebirhan A. 57
 Cont…
Multinomial logistic regression Number of obs = 500
LR chi2(8) = 241.43
Prob > chi2 = 0.0000
Log likelihood = -372.34821 Pseudo R2 = 0.2448

divers RRR Std. Err. z P>|z| [95% Conf. Interval]

on_farm (base outcome)

local_off_farm
edu .5907459 .0576212 -5.40 0.000 .4879493 .7151988
sex 3.81e-06 .0020092 -0.02 0.981 0 .
age 1.014513 .0149118 0.98 0.327 .9857031 1.044164
exp 2.233434 .5167425 3.47 0.001 1.419166 3.514899
_cons 1.20e+08 6.33e+10 0.04 0.972 0 .

migration
edu .8994171 .0833128 -1.14 0.252 .750092 1.078469
sex 1.46e-07 .0000768 -0.03 0.976 0 .
age .9984812 .0157093 -0.10 0.923 .9681614 1.029751
exp 4.063402 .9396483 6.06 0.000 2.58257 6.393333
_cons 2.20e+07 1.16e+10 0.03 0.974 0 .

Note: _cons estimates baseline relative risk for each outcome.

By: Teklebirhan A. 58
 Cont…
For one year increase in level of education, the probability of
choosing local off farm is 0.591 times the probability of on-
farm income diversification.
So, as age increases by one year, the probability of choosing
migration is 0.998 times the probability of on-farm
diversification.
For a dichotomous dummy explanatory variable such as male,
the ratio of the relative risks of choosing migration (2) over on
farm diversification(1) for male as compare to female is
0.00000381.
The log of the ratio of the two probabilities,
(migration=2)/P(on-farm=1), for male will be lower by
15.74 than female.
59
Thus, male is less probable to migrate. By: Teklebirhan A.
 Cont…
Marginal Effect Result

ME after MNL
ME after MNL for local off ME after MNL for
Categories for on-farm
farm diversification Migration
diversification

EDUC 0.0046787 -0.1012054*** 0.0965267

SEX 0 .0792842 *** 0.4155105 *** -0.4947947 ***
Age -0.0000766 0.0040408 -0.0039642

Exp -0.0198351 -0.1342864 0.1541215

By: Teklebirhan A. 60
 Cont…
ME after MNL for on-farm income diversification
As the above table shows, male are more probable (8%) to
participate in on farm income diversification than female, and
statistically significant.
ME after MNL for local off-farm income diversification
More educated person is less probable (10%) than less
educated person to participate in local off farm income
diversification, and statistically significant.
Male is more probable (41%) to participate in local off farm
income diversification than female, & statistically significant.
 Thus, it is more probable for male and less educated person
to participate in local off farm income diversification.
61
By: Teklebirhan A.
 Cont…
ME after MNL for migration

Male are 49.5% less probable than female to

participate in migration

By: Teklebirhan A. 62
 Cont…
We can also determine the probability that an
individual chooses each alternatives using the
following command.
predict plogit1 plogit2 plogit3, pr
Finally, we can also determine the average probability
for each category using the following Stata command.
summarize plogit1 plogit2 plogit3

By: Teklebirhan A. 63
 Cont…

. predict plogit1 plogit2 plogit3, pr

. summarize plogit1 plogit2 plogit3

Variable Obs Mean Std. Dev. Min Max

plogit1 500 .1342627 .1382494 1.55e-10 .6204082

plogit2 500 .3740613 .2271436 .0028397 .8532475
plogit3 500 .491676 .292386 .0880502 .9971603

By: Teklebirhan A. 64
 Cont…
The above summary statistics of the probability of
choosing on-farm diversification, local off farm
diversification and migration of rural households
showed that about half of the rural households have at
least one migrant family member and therefore, rural
households used migration as one type of income
diversification.

By: Teklebirhan A. 65
 3.3 Ordered Logit Model
The ordered logit model is also known as the proportional
odds model.
The terms parallel lines model and parallel regressions
model are also sometimes used.

In ordered logit model, there is observed ordinal variable, Y

which in turn, is a function of another variable, Y*, that is not
measured.

That means, in ordered logit model, there is a continuous,

unmeasured latent variable Y*, whose values determine the
value of ordinal variable Y.
By: Teklebirhan A. 66
 Cont…
The continuous latent variable Y* has various
threshold points (cut points).

Your value on the observed variable Y depends on

whether or not you have crossed a particular
threshold.

These cut points (thresholds) are represented by k

which is the Greek small letter kappa.

By: Teklebirhan A. 67
 Cont…
For example, when the number of categories are three
(M=3)

For example, it might be that if your score on the

unobserved latent variable Y* was 10 utils or less, your
score on Y would be low(1); if your Y* score was b/n
10 and 25 utils, Y would be medium (2); and if your
score on latent variable (Y*) was above 25, Y would be
high(3).
By: Teklebirhan A. 68
 Cont…
Put another way, you can think of Y as being a
collapsed version of Y*.
Example Y* can take on an infinite range of values
which might then be collapsed into 3 categories of Y.
So, what does Y* equal? How do you estimate this
model?
In the population, the continuous latent variable Y* is
equal to

By: Teklebirhan A. 69
 Cont…
Note that there is a random disturbance term, which,
in this case, has a standard logistic distribution (mean
of 0 and variance of 3.29).
This reflects the fact that relevant variables may
be left out of the equation, or variables may not
be perfectly measured.
The ordered logit model estimates part of the above:

By: Teklebirhan A. 70
Cont…

By: Teklebirhan A. 71
Cont…

By: Teklebirhan A. 72
Cont…

By: Teklebirhan A. 73
 Cont…
Example on Ordered Logit
Suppose that we want to study the level and determinants
of the service satisfaction of the employees of AAU.

To this end, data on the level of satisfaction (low, medium

and high), years of schooling, gender, age and experience
were collected from 500 respondents.

In this example, the response variable, level of satisfaction

has ordered categorical variables (low, medium & high) and
therefore, we can use Ordered Logit/Probit Model.
By: Teklebirhan A. 74
 Cont…
Three of the explanatory variables are quantitative
(experience, age and education) while the other one is
qualitative or categorical variable (gender).

By: Teklebirhan A. 75
 Cont…
Basic Commands
 summarize
 tabulate LEVEL
 ologit LEVEL EDUC MALE AGE EXPR
 ologit LEVEL EDUC MALE AGE EXPR, or
 mfx, predict(outcome (1))
 mfx, predict(outcome (2))
 mfx, predict(outcome (3))
 predict plogit1 plogit2 plogit3, pr
 summarize plogit1 plogit2 plogit3
 linktest
By: Teklebirhan A. 76
 Cont…
. describe

Contains data from D:\2010\Econometrics\Stata Training\Stata14\Ordered Logit.dta

obs: 500
vars: 5 29 Mar 2019 02:35
size: 2,500

storage display value

variable name type format label variable label

LEVEL byte %10.0g LEVEL LEVEL OF SATISFACTION

EDUC byte %10.0g EDUC
MALE byte %10.0g MALE
AGE byte %10.0g AGE
EXPR byte %10.0g EXPR

By: Teklebirhan A. 77
 Cont…
. sum

Variable Obs Mean Std. Dev. Min Max

LEVEL 498 2.35743 .7069325 1 3

EDUC 500 11.652 1.644477 8 17
MALE 500 .89 .3132031 0 1
AGE 500 45.1 10.84275 22 88
EXPR 500 1.378 1.656685 0 7

By: Teklebirhan A. 78
 Cont…
. tabulate LEVEL

LEVEL OF
SATISFACTIO
N Freq. Percent Cum.

Low 67 13.45 13.45

Moderate 186 37.35 50.80
High 245 49.20 100.00

Total 498 100.00

By: Teklebirhan A. 79
 Cont…
ologit LEVEL EDUC MALE AGE EXPR
Ordered logistic regression Number of obs = 498
LR chi2(4) = 185.97
Prob > chi2 = 0.0000
Log likelihood = -398.38453 Pseudo R2 = 0.1892

LEVEL Coef. Std. Err. z P>|z| [95% Conf. Interval]

EDUC .0798993 .0600279 1.33 0.183 -.0377533 .1975519

MALE -3.409955 .6115636 -5.58 0.000 -4.608597 -2.211312
AGE -.0082687 .0090129 -0.92 0.359 -.0259336 .0093962
EXPR .699957 .0748069 9.36 0.000 .5533381 .8465758

/cut1 -4.024487 1.017069 -6.017906 -2.031069

/cut2 -1.644877 1.013087 -3.630491 .340738

By: Teklebirhan A. 80
 Cont…
Ordered Logit Interpretation
The level of satisfaction is better (from low to medium
to high) with higher level of education and experience,
female and lower age.

Both gender and experience are statistically

significant; education and age are not.
For experience, a one year increases in experience, we
expect a 0.69957 increase in the log odds of being in a
higher level of satisfaction.
By: Teklebirhan A. 81
 Cont…
The coefficient of male is the ordered log-odds
estimate of comparing male to female on expected
level of satisfaction.

The ordered logit for male being in a higher level of

satisfaction category is 3. 40 less than female.

By: Teklebirhan A. 82
 Cont…
Odds Ratio Interpretation of the ordered logit model
ologit LEVEL EDUC MALE AGE EXPR, or

By: Teklebirhan A. 83
 Cont…
For gender, the odds of being in the higher level of
satisfaction of male is 0.0330 times that of female
staff.

This means that, there is greater probability for

female to be in the higher level of satisfaction or
female are more satisfied than male.

By: Teklebirhan A. 84
 Cont…
Probability Interpretation of ordered logit

By: Teklebirhan A. 85
 Cont…
A year increase in education is associated with 0.6%
less likely to be in the low level of satisfaction, 1.4%
less likely to be in the medium level of satisfaction and
2% more likely to be in the high level of satisfaction.
As years of schooling (education) increases by one year,
the probability of being in the higher level of
satisfaction increases.
NB: Sum of the probability of each category is Zero for each
explanatory variable.

By: Teklebirhan A. 86
 Cont…
Similarly, one year increases in experience is
associated with 4.8% less likely to be in the low level
of satisfaction, 12.6% less likely to be in the medium
level of satisfaction and 17.4% more likely to be in
the high level of satisfaction.

Male is 10.2% more likely, 41.8% more likely and

52% less likely to be in the low, medium and high
level of satisfaction, respectively.

By: Teklebirhan A. 87
 Cont…
Predict plogit1 plogit2 plogit3

By: Teklebirhan A. 88
 Cont…
As you can see, the predicted probability of being in
the lowest level of satisfaction is 13.3% and that of
the middle and highest level of satisfaction are 37.6%
and 49.2%, respectively.

We can infer that about half of the employees of the

university are highly satisfied with the services and the
main determinants of the level of satisfaction are
experience and gender.

By: Teklebirhan A. 89
 Cont…
Link test of model specification
The test result showed that there is no problem.

By: Teklebirhan A. 90
 Cont…
Measures of fit
The R2 don’t make sense for logit and probit model
rather we use pseudo-R2.
The pseudo-R2 measure the fit using the likelihood
function and measures the improvement in the value of
the log likelihood, relative to having no explanatory
variables (Xi).
Suppose that if pseudo-R2 = 0.189
This suggests that the log-likelihood value increases by
about 18.9% with the introduction of the set of regressors.
By: Teklebirhan A. 91
 Cont…
Note:

In any of the discrete choice models, we first run a

linear regression to test for multicollinearity using
VIF command, and then test for multicollinearity via
the vif command.

By: Teklebirhan A. 92
 End!

Thank you!
By: Teklebirhan A. 93