LPM, Logit and Probit Models
LPM, Logit and Probit Models
LPM, Logit and Probit Models
REGRESSION
ANALYSIS
WITH
LINEAR PROBABILITY MODEL, LOGIT AND PROBIT MODELS
HELLO!
I am Elijah Appiah from
Ghana.
I am an Economist by
profession.
I love everything about R!
1
12/6/2022
Lesson Goals
▸ Understand how to estimate
qualitative response models.
Lecture Series
2
12/6/2022
3
12/6/2022
LINEAR
PROBABILITY
MODEL
4
12/6/2022
10
5
12/6/2022
11
Bernoulli 0 𝟏 − 𝑷𝒊
Probability
Distribution 1 𝑷𝒊
Total 𝟏
12
6
12/6/2022
13
14
𝐏𝐫(𝒀 < 𝟎)
7
12/6/2022
15
16
8
12/6/2022
17
MODEL
18
9
12/6/2022
19
Logit Model
match match frequency What can we
Win
Win 4 do with this
Win
Lose 6 count data?
Win
Win
Lose
Lose What is the probability that a
Lose team wins the match?
Lose
Lose What are the odds of winning?
Lose
20
Logit Model
Probability is the number of events divided by total
outcome.
match frequency 𝟒
𝒑𝒓𝒐𝒃 = = 𝟎. 𝟒𝟎
Win 4 𝟏𝟎
Lose 6
10
12/6/2022
21
Logit Model
Probability and odds can be expressed in terms of each other.
𝒐𝒅𝒅𝒔 𝒑𝒓𝒐𝒃
𝒑𝒓𝒐𝒃 = 𝒐𝒅𝒅𝒔 =
𝟏 + 𝒐𝒅𝒅𝒔 𝟏 − 𝒑𝒓𝒐𝒃
𝟒 𝟒
𝒑𝒓𝒐𝒃 = = 𝟎. 𝟒𝟎 𝒐𝒅𝒅𝒔 = = 𝟎. 𝟔𝟕
𝟏𝟎 𝟔
𝟎. 𝟔𝟕 𝟎. 𝟒𝟎
𝒑𝒓𝒐𝒃 = = 𝟎. 𝟒𝟎 𝒐𝒅𝒅𝒔 = = 𝟎. 𝟔𝟕
𝟏 + 𝟎. 𝟔𝟕 𝟏 − 𝟎. 𝟒𝟎
22
Logit Model
Logistic regression (logit) model calculates the
log odds given known values of the predictors.
𝐥𝐨𝐠 𝒐𝒅𝒅𝒔 𝑬 𝒀𝒊 𝑿𝒊 = 𝜷𝟎 + 𝜷𝟏 𝑿𝒊
11
12/6/2022
23
Logit Model
𝒐𝒅𝒅𝒔
Recall: 𝒑𝒓𝒐𝒃 =
𝟏 + 𝒐𝒅𝒅𝒔
𝐥𝐨𝐠 𝒐𝒅𝒅𝒔 𝑬 𝒀𝒊 = 𝟏 𝑿𝒊 = 𝜷𝟎 + 𝜷𝟏 𝑿𝒊
𝒍𝒐𝒈(𝒐𝒅𝒅𝒔)
𝑷𝒓 𝒀𝒊 = 𝟏 𝑿𝒊 = Not Correct Yet!
𝟏 + 𝒍𝒐𝒈(𝒐𝒅𝒅𝒔)
24
Logit Model
𝒐𝒅𝒅𝒔
Recall: 𝒑𝒓𝒐𝒃 =
𝟏 + 𝒐𝒅𝒅𝒔
𝐥𝐨𝐠 𝒐𝒅𝒅𝒔 𝑬 𝒀𝒊 = 𝟏 𝑿𝒊 = 𝜷𝟎 + 𝜷𝟏 𝑿𝒊
𝒍𝒐𝒈(𝒐𝒅𝒅𝒔)
𝑷𝒓 𝒀𝒊 = 𝟏 𝑿𝒊 = Not Correct Yet!
𝟏 + 𝒍𝒐𝒈(𝒐𝒅𝒅𝒔)
𝒆𝒍𝒐𝒈(𝒐𝒅𝒅𝒔)
𝑷𝒓 𝒀𝒊 = 𝟏 𝑿𝒊 = Correct
𝟏 + 𝒆𝒍𝒐𝒈(𝒐𝒅𝒅𝒔)
12
12/6/2022
25
Logit Model
𝒆𝒍𝒐𝒈(𝒐𝒅𝒅𝒔)
𝑷𝒓 𝒀𝒊 = 𝟏 𝑿𝒊 = Correct
𝟏 + 𝒆𝒍𝒐𝒈(𝒐𝒅𝒅𝒔)
𝐥𝐨𝐠 𝒐𝒅𝒅𝒔 𝑬 𝒀𝒊 = 𝟏 𝑿𝒊 = 𝜷𝟎 + 𝜷𝟏 𝑿𝒊
𝒆 𝜷 𝟎 𝜷 𝟏 𝑿𝒊
𝑷𝒓 𝒀𝒊 = 𝟏 𝑿𝒊 =
𝟏 + 𝒆 𝜷 𝟎 𝜷 𝟏 𝑿𝒊
𝟏
𝑷𝒓 𝒀𝒊 = 𝟏 𝑿𝒊 = (𝜷𝟎 𝜷𝟏 𝑿𝒊 )
𝟏+𝒆
26
Logit Model
𝒆 𝜷 𝟎 𝜷 𝟏 𝑿𝒊
𝑷𝒓 𝒀𝒊 = 𝟏 𝑿𝒊 =
𝟏 + 𝒆 𝜷 𝟎 𝜷 𝟏 𝑿𝒊
𝜷𝟎 𝜷𝟏 𝑿𝒊
𝜷𝟎 𝜷𝟏 𝑿𝒊
𝒊 𝒊 𝜷𝟎 𝜷𝟏 𝑿𝒊
𝜷𝟎 𝜷𝟏 𝑿𝒊 𝜷𝟎 𝜷𝟏 𝑿𝒊
𝟏
𝑷𝒓 𝒀𝒊 = 𝟏 𝑿𝒊 = (𝜷𝟎 𝜷𝟏 𝑿𝒊 )
𝟏+𝒆
13
12/6/2022
27
Logit Model
𝐥𝐨𝐠 𝒐𝒅𝒅𝒔 𝑬 𝒀𝒊 = 𝟏 𝑿𝒊 = 𝜷𝟎 + 𝜷𝟏 𝑿𝒊
Probability is not
𝒁𝒊 = 𝜷𝟎 + 𝜷𝟏 𝑿𝒊 linear in 𝑿𝒊 or the
parameters in 𝒁𝒊 .
𝒆𝒁𝒊
𝑷𝒓 𝒀𝒊 = 𝟏 𝑿𝒊 = Logistic
𝟏 + 𝒆𝒁𝒊
Distribution
𝟏 Function
𝑷𝒓 𝒀𝒊 = 𝟏 𝑿𝒊 = 𝒁𝒊
𝟏+𝒆
28
14
12/6/2022
29
Logit Model
𝟏
𝑷𝒓 𝒀𝒊 = 𝟏 𝑿𝒊 = 𝒁𝒊
𝟏+𝒆
𝟏 𝟏 𝟏
𝑷𝒊 = 𝒁𝒊 𝟏 − 𝑷𝒊 = 𝟏 − 𝒁𝒊 =
𝟏+𝒆 𝟏+𝒆 𝟏 + 𝒆𝒁𝒊
𝒑𝒓𝒐𝒃
𝒐𝒅𝒅𝒔 =
𝟏 − 𝒑𝒓𝒐𝒃
𝒊 𝒁𝒊
𝒁𝒊
𝒊
𝒁𝒊 Odds Ratio
30
Logit Model
𝒑𝒓𝒐𝒃
𝒐𝒅𝒅𝒔 =
𝟏 − 𝒑𝒓𝒐𝒃
𝒊 𝒁𝒊
𝒁𝒊
𝒊
𝒁𝒊 Odds Ratio
If ,
The odds that the family
will own a house is 4 to 1.
15
12/6/2022
31
Logit Model
𝑷𝒊
𝑶𝒅𝒅𝒔 = = 𝒆𝒁𝒊
𝒙 𝟏 − 𝑷𝒊
𝑷𝒊 𝑷𝒊 𝒁
𝑶𝒅𝒅𝒔 = 𝒍𝒏
𝑳 = 𝒍𝒏(𝑶𝒅𝒅𝒔) = 𝒆 𝒊 = 𝒍𝒏 𝒆𝒁𝒊 = 𝒁𝒊
𝟏 − 𝑷𝒊 𝑷𝒊
𝟏 −
𝑳 = 𝒁𝒊 = 𝜷𝟎 + 𝜷𝟏 𝑿𝒊
where: 𝑳, is the log of the odds ratio, is not only
linear in 𝑿𝒊 but also the parameters. L is called
the Logit, hence the name Logit model.
32
Logit Model
Key Takeaways:
The logits are not bounded by 0 and 1, but the
probabilities will (so we need to calculate
probabilities for the logit model – MARGINAL
EFFECTS).
Although L is linear in X, but the probabilities are
not.
You can include many predictors.
16
12/6/2022
33
Logit Model
Key Takeaways:
If L, the logit, is positive, it means that when the
value of the regressor(s) increases, the odds that
the regressand equals 1 (meaning some event of
interest happens) increases. If L is negative, the
odds that the regressand equals 1 decreases as
the value of X increases.
34
MODEL
17
12/6/2022
35
Probit Model
The probit model uses a standardized normal CDF.
If a variable 𝑿 follows the normal distribution with
mean 𝝁 and standard deviation 𝝈, then the PDF is:
𝟏 𝟐 𝟐
𝒇 𝑿 = 𝒆 𝑿 𝝁 /𝟐𝝈
𝟐𝝈𝟐 𝝅
36
Probit Model
The probit model becomes:
𝑷𝒓 𝒀𝒊 = 𝟏 𝑿𝒊 = 𝑭(𝜷𝟎 + 𝜷𝟏 𝑿𝒊 )
𝜷𝟎 𝜷𝟏 𝑿𝒊
𝟏 𝒁𝟐 /𝟐
𝑭 𝑰 = 𝒆 𝒅𝒁
𝟐𝝅
18
12/6/2022
37
38
19
12/6/2022
39
stat::glm()
mfx:logitmfx()
mfx::probitmfx()
40
20
12/6/2022
41
42
THANKS!
Any questions?
Reach me anytime!
Email
[email protected]
LinkedIn
https://www.linkedin.com/in/appiah-elijah-383231123/
21