Binary, Ordinal, and Multinomial Logistic Regression
for Categorical Outcomes
Karen Grace-Martin
The Analysis Factor
What You’ll Learn
1. Linear Model: Quantitative Dependent Variable
2. Binary Logistic Model: Binary Dependent Variable
3. Multinomial Logistic Model: Unordered Multi-category Dependent
Variable
4. Proportional Odds Logistic Model: Ordered Multi-category
Dependent Variable
Karen Grace-Martin | https://TheAnalysisFactor.com 2
1. Linear Model:
Quantitative Dependent Variable
Karen Grace-Martin | https://TheAnalysisFactor.com 3
Linear Model
Dependent Variable is
• Continuous
• Unbounded
• Measured on an interval or ratio scale
Karen Grace-Martin | https://TheAnalysisFactor.com 4
Linear Regression
The Model:
Parameter Estimation: Ordinary Least Squares
Model Fit: R2
Assumptions: εi ~ i.i.d N(0, σ2)
Karen Grace-Martin | https://TheAnalysisFactor.com 5
Linear Regression 4.5
How to interpret coefficients: 4.0
3.5
3.0
2.5
College GPA Scores
2.0
1.5
1.0
300 400 500 600 700 800
SAT math
E(College GPA) = -.03 + .20*HSGPA + .003*SATV + .002*SATM -.15*Sports -.26*Male
Karen Grace-Martin | https://TheAnalysisFactor.com 6
2. Binary Logistic Model:
Binary Dependent Variable
Karen Grace-Martin | https://TheAnalysisFactor.com 7
Binary Dependent Variable
Two possible outcome values on
each trial:
Success/Failure
Academic Warning/Not
1/0
P is the probability of success for
any given student at any given
value of Xs
Karen Grace-Martin | https://TheAnalysisFactor.com 8
Binary Logistic Regression
P
The Model: Ln 0 1 X 1 2 X 2 ... k X k
1 P
Parameter Estimation: Maximum Likelihood
Model Fit: Deviance (-2LL)
Assumptions:
1. trials are independent
2. probability of success is the same on any given trial with the same values of X
Karen Grace-Martin | https://TheAnalysisFactor.com 9
Binary Logistic Regression
P
The Model: Ln 0 1 X 1 2 X 2 ... k X k
1 P
Odds(Y 1) e 0 1 X 1 2 X 2 ... k X k
How to interpret coefficients:
eβ, the odds ratio, is the ratio of the odds of two values of X one unit apart
http://thecraftofstatisticalanalysis.com/understanding-probability-odds-ratios-regression/
Karen Grace-Martin | https://TheAnalysisFactor.com 10
3. Multinomial Logistic Model:
Unordered Multi-category Dependent Variable
Karen Grace-Martin | https://TheAnalysisFactor.com 11
Unordered Multi-category Dependent Variable
More than two unordered possible outcome values on each trial:
1 = Academic Warning
2 = Passed classes and transferred out or withdrew
3 = Passed classes and remained in Good Standing
Karen Grace-Martin | https://TheAnalysisFactor.com 12
Multinomial Logistic Regression
Ph
The Model: ln 0 h 1h X 1 2 h X 2 ... kh X k
P
j
Where j is the number of categories
h=1 to j-1
k is the number of predictors
Appropriate when: Y is categorical, more than 2 categories
Parameter Estimation: Maximum Likelihood
Model Fit: Deviance (-2LL)
Karen Grace-Martin | https://TheAnalysisFactor.com 13
Multinomial Logistic Regression
Ph
The Model: ln 0 h 1h X 1 2 h X 2 ... kh X k
P
j
How to interpret coefficients:
Risk Ratios all compared to the reference outcome
P1
Academic warning vs.
ln 01 11 X 1 21 X 2 ... k1 X k
P3
Good Standing Entire set of coefficients is different
P
Transferred vs. Good Standing
ln 2 02 12 X 1 22 X 2 ... k 2 X k
P3
Karen Grace-Martin | https://TheAnalysisFactor.com 14
4. Proportional Odds Logistic Model:
Ordered Multi-category Dependent Variable
Karen Grace-Martin | https://TheAnalysisFactor.com 15
Ordered Multi-category Dependent Variable
More than two ordered possible outcome values on each trial:
1 = Academic Warning
2 = Good Standing
3 = Dean’s List
Karen Grace-Martin | https://TheAnalysisFactor.com 16
Ordinal Logistic Regression
Fij j
The Model: ln 0 j 1 X 1 2 X 2 ... k X k Fij Pim
1 F
ij m 1
Where j is the jth ordered category
k is the number of predictors
Appropriate when: Y is ordered categories
Parameter Estimation: Maximum Likelihood
Model Fit: Deviance (-2LL)
Assumptions: Proportional odds/Parallel Lines
Karen Grace-Martin | https://TheAnalysisFactor.com 17
Ordinal Logistic Regression
Fij j
The Model: ln 0 j 1 X 1 2 X 2 ... k X k Fij Pim
1 F
ij m 1
How to interpret coefficients: Each OR compares to
all higher categories
Academic Warning vs.
Pi1
Good Standing and Dean’s List ln 01 1 X 1 2 X 2 ... k X k
(1 vs. 2 and 3) 1 Pi1
Only the intercept is different
Academic Warning and
Good Standing ( Pi1 Pi 2 )
ln 02 1 X 1 2 X 2 ... k X k
vs. Dean’s List 1 ( Pi1 Pi 2 )
(1 and 2 vs. 3)
Karen Grace-Martin | https://TheAnalysisFactor.com 18
More Resources
Karen Grace-Martin | https://TheAnalysisFactor.com 19
More Resources
Logistic Regression for Count
Logistic Regression for and Proportion Data
Binary, Ordinal, and
Multinomial Outcomes ROC Curves
Understanding Probability, Multinomial Logistic
Odds, and Odds Ratios in Regression
Introduction to
Logistic Regression
Generalized Linear Explaining Logistic Regression
Mixed Models Results
Interactions in Logistic
Regression
https://www.theanalysisfactor.com/resources/by-topic/logistic-regression/
Karen Grace-Martin | https://TheAnalysisFactor.com 20
If you want to learn more: Online Workshop
Logistic Regression for Binary,
Ordinal, and Multinomial Outcomes
Instructor: Karen Grace-Martin
Stage: 3
Software: R, SPSS, SAS, Stata
Starts: November 8, 2023
Enrollment Close: November 7, 2023
Price: $397 $337
https://www.theanalysisfactor.com/LR15/
Karen Grace-Martin| https://TheAnalysisFactor.com 21
