Logistic Regression

Multivariate Analysis
 What is a log and an exponent?
 Log is the power to which a base of 10 must
be raised to produce a given number. The log
of 1000 is 3 as 103=1000.
 The log of an odds ratio of 1.0 is 0 as 100 = 1
 Exponent (e) or 2.718 raised to a certain
power is the antilog of that number. Thus,
(expβ) = antilog β
 Antilog of log odds 0 is 2.718o =1
 Exponential increases are curvilinear.
 Main questions with logistic regression
 How do the odds of a successful
outcome depend upon or change based
on each explanatory variable (X)?
 How does the probability that a
successful outcome occurs depend
upon or change based on each
explanatory variable (X)?

π
 Logistic regression
 Single binary response variable predicted by
categorical and interval variables
 Maximum likelihood model – the coefficients
that make sample observations most likely are
reported in the final model
 Binomial distribution that assumes a sigmoid
curve (non-linear)
 The probability of success falls between 0 and
1 for all possible values of X (s-curve bends)
Sigmoid curve for logistic regression
 Response variable
 Denote Y by 0 and 1 (dummy coding)
 0 and 1 are usually termed failure and
success of an outcome (by convention,
success is category 1)
 The sample mean of Y is the sum of the
number of successes divided by the
sample size (proportion of success)
Odds ratios in logistic regression
 Can be thought of as likelihood or odds success
based impact of predictors in model
 Interval : the odds of success for those who are a
unit apart in X, net of other predictors.
 For dummy coefficients: the odds of success for
those in the reference category of X (1) compared
with those in the omitted (0)
 Every unit increase in X has an exponential effect
on the odds of success so an odds ratio can be >1
Odds ratio
 π / 1- π is the odds ratio or the odds of
success
 When the probability of success or π is
½ or 50-50, odds for success equals .
5/1-.5 = 1.0. This means that success is
equally as likely as failure
 Thus, predicted probability of .5 and an
odds ratio of 1.0 are our points of
comparison when making inferences
Logistic transformation of odds ratio
 To model dichotomous outcomes, SPSS takes logistic transformation
of odds ratios:
Log (π / 1- π ) = α + βX1 + βX2 …
 To interpret, we take the exponent values of beta coefficient for each
predictor (can do for all in model)
 Odds ratio or the odds of success are:
π / 1- π = e α + βX = e α + (e β)X


Exponent 
We can also talk about the percentage
change in odds for interval and dummy
variables
 Thus, the exponential beta value in the
SPSS output can be calculated into a
percent by 100 (exp b –1) or the
percentage change in odds for each unit
increase in the independent variable.
 We don’t really talk about the intercept
here … betas for each predictor are our
concern
We can also talk about the probability of
success or π
 Can calculate point estimates by substituting
specific X values, thus it is good for
forecasting, given respondent characteristics
 Impact of X on π is interactive/non-constant
 π is the probability of success and this
probability varies as X changes and it is
expressed in a % form (ranges 0-1)
 π = e α + βX / 1 + e α + βX or odds / 1 + odds
Slope in logistic regression
models (FYI)
 Like the slope of a straight line, β refers to
whether the sigmoid curve (π or prob. of
success) increases β+ or decreases β- as the
values of the intervals increase or we move
from 0 to 1 for dummy
 Steepness of s-curve increases as absolute
value of β increases
 The rate at which the curve climbs or
descends changes according to the values of
the independent variable thus β (X)
Slope in logistic regression models (FYI)
 When β = 0, π does not change as X
increases (X has no bearing on probability or
odds of success ) so the curve is flat, there is
just a straight line
 For β > 0, π increases as X increases
(probability of success increases thus curve
increases)
 For β < 0, π decreases as X increases
(probability of success decreases thus curve
decreases)
 Mention .5 bit on next slide
Slope in logistic regression (FYI)
 Null hypothesis for predictors

 Ho: β = 0
for Log (π / 1- π ) = α + βX 1…I
 X has no effect on the likelihood that
[y =1] an outcome will occur
 Y is independent of X so the likelihood
of being successful is the same for all
income groups
 Wald Statistic
Null hypothesis test statistic for each predictor
in your model
 Wald statistic is the significance test for
each parameter in the model
 Null is that each β = 0
 Has df=1; Chi-square distribution
 It is the square of z statistic which
equals β/s.error
 -2 log likelihood as test of null
hypothesis for entire model
 A test of significance for model and is like the
F-ratio; chi-square distribution; df = pα + β - pα
 Does the observed likelihood or odds of
success differ from 1?
 Compares the model with the intercept alone
to intercept and predictors. Do your predictors
add to the predictive power of the model?
 Tests if the difference is 0 and is referred to
as the model chi-square
 Goodness of Fit Statistic – null
for residuals (FYI)
 Compares observed probabilities or what you
observed in the sample to the predicted
probabilities of an outcome occurring based on
model parameters in your equation
 Examines residuals – do the predictor
coefficients significantly minimize their squared
distances?
 Chi-square distribution; df = p
 Should be NS as observed and predicted are
anticipated to be quite similar
 Mean of our response variable
attending self-help group (FYI)
 The sample mean of Y is the sum of the
number of successes (yes to attend)
divided by the sample size, n
 The sample mean is the proportion of
successful outcomes
 Thus, 44 said yes and n = 400, thus
mean proportion of yes is .11 or 11%
 Odds ratio and % in odds change by age
 Age  =-.0586 and p<.01 (beta negative).
Thus, log odds of attending a self-help group
decrease as a person gets older
 Exp  = .9431 … the odds ratio [exp  <1] thus
odds decrease
 % change (in this case a reduction in) in odds
of attending for each additional year of age is

100(exp  - 1) = 100 (.9431 – 1) =

-5.69 % less likely each year one ages
Predicted probability of attending by age
 When  < .5, the probability of attending
declines and we would see a downward dip in
the sigmoid curve with increasing values of X
(keeping in mind probability ranges from 0-1)
 More meaningful with all predictors, however, a
point estimate for age 80 would be:
  = e(-.0586)(80) / 1 + e(-.0586)(80) = .009 The
probability of those 80 years of age attending a
group is 1%
 Odds ratio and % change in odds by gender
 Gender  =1.2540 and p<.05. Thus, odds
of attending a self-help group among
females is greater (referent category is
female and beta is positive)
 Exp  = 3.5045 … odds of attending are
3.5 times as large for females as they are
for males [exp  >1]
 % change (in this case an increase in) in
odds of attending when a person is female
is 100(exp  - 1)=100(3.50 – 1) = 250 %
 Predicted probability of attending by
gender
 e(1.254)(1) / 1 + e(1.254)(1) = .77
 Thus, the probability of attending
among females is 77%
 When  > .5, the probability of
attending increases and we would
see an upward trend in the sigmoid
curve with increasing values of X on
the horizontal axis (keeping in mind
probability ranges from 0-1)
 Wald statistic
 Coefficient for each independent
variable is 0
 Tells us which variables significantly
predictor the likelihood of attending a
self-help group
 Age = 7.2298** Gender 5.7723*
 Likelihood statistic for the model
 Likelihood or odds are 1.0 and predicted
probability is .5
 Constant alone minus constant and all
predictors 281.36838 - 240.518*
 All of our predictor variables have β = 0
 With 12 df model chi-square of 40.85 has a
p<.0001
 The predictors in model significantly add to
our capacity to predict attendance
 Goodness of Fit (FYI)

 371.093, 12 df, ns
 Our model parameters minimize the
squared distances [residual] between
actual sample observations of
attendance to that which the logistic
regression equation predicts (odds and
probabilities)
Logistic Regression References
 DeMaris, A. (1995). A tutorial in logistic regression.
Journal of Marriage and the Family, 57(10): 956-968

 Agresti, A. & Finlay, B. (1997). Logistic regression –

modeling categorical responses. Statistical methods
for social sciences (3rd ed., pp. 575-619). Prentice
Hall: New Jersey.

 Dwyer, J.H. (1983). Statistical methods for the social

and behavioral sciences (pp. 447-465). Oxford
University Press: New York.