Marketing Analytics

Based on First Principles :

Chapter 8

Using Logistic Regression for

Customer Selection

© Palmatier, Petersen, and Germann 1

 Agenda

 Learning Objectives

 Introduction
 Objectives of Logistic Regression Models
 Common Uses of Logistic Regression Models
 Logistic Regression Example

 Logistic Regression
 Logistic Regression Model
 Odds of an Event
 Model Fit and Testing for Significance of the Coefficients
 Odds Ratio in Logistic Regression
 Example
 Logistic regression with one predictor variable
 Logistic regression with multiple predictor variables

 Summary

 Takeaways

© Palmatier, Petersen, and Germann 2

 Learning Objectives

 Know when to use logistic regression as opposed to (for example) OLS

regression

 Understand what type of variables are used in logistic regression

 Know what form a logistic regression model takes

 Be able to estimate probabilities using logistic regression output

 Understand the difference between probabilities and odds

 Know how to convert probabilities into odds, and vice versa

© Palmatier, Petersen, and Germann 3

 Learning Objectives

 Know what the odds ratio is, how it is calculated and used

 Understand what type of model fit measures are used in logistic

regression models

 Know how the significance of logistic regression coefficients is

determined

 Know how to calculate Pseudo R2

 Understand and know how to create and understand a confusion matrix

based on logistic regression results

 Know how to use logistic regression to compute the profitability and

return on marketing investment (ROMI) of a marketing campaign

© Palmatier, Petersen, and Germann 4

 Agenda

 Learning Objectives

 Introduction
 Objectives of Logistic Regression Models
 Common Uses of Logistic Regression Models
 Logistic Regression Example

 Logistic Regression
 Logistic Regression Model
 Odds of an Event
 Model Fit and Testing for Significance of the Coefficients
 Odds Ratio in Logistic Regression
 Example
 Logistic regression with one predictor variable
 Logistic regression with multiple predictor variables

 Summary

 Takeaways

© Palmatier, Petersen, and Germann 5

 Objectives of Logistic Regression Models

 Logistic regression is similar to linear regression such as ordinary least

squares (OLS). It is used to estimate the effect of one or more independent
variables x on a dependent variable y.

 However, in contrast to (e.g.) OLS, where the dependent variable is

continuous in nature, logistic regression is used when the dependent
variable has only two possible categories, i.e., is binary.

 As such, the objective of logistic regression is to determine how one or

more independent variables x impact the probability that an observation
falls into one of the two categories captured by the y variable.

© Palmatier, Petersen, and Germann 6

 Common Uses of Logistic Regression Models

 Logistic regression is commonly used by Marketing researchers.

 It is a popular method used in customer acquisition. Will a certain customer buy
the company’s product (yes/no)? Note that the outcome (or dependent) variable
is binary, and hence logistic regression can be used to identify and use x variables
that predict whether or not a customer buys a product etc.
 It can be used to predict retention of current customers (yes/no). Which
customers are most likely to defect considering certain predictor (x) variables?
Once customers at risk of defecting have been identified, marketers can employ
interventions targeted at those customers they do not want to lose (but are likely
to defect otherwise).
 It can be used to estimate the likelihood that a person will respond favorably to a
mailing campaign (yes/no).

© Palmatier, Petersen, and Germann 7

 Logistic Regression Example

 MyFriend is a relatively new insurance company specializing in car

insurance policies.

 It prides itself in its effective use of marketing analytics. Using customer

data from the last 5 years, Max, MyFriend’s up and coming Analytics
Manager, recently developed a model that helps the company identify
current customers who are at risk of churning, that is, leaving the
company for a competitor.

 As part of his modeling approach, Max first pulled data from the local
company server on all of MyFriend’s current and former insurance policy
holders, including whether they had churned sometime during the last 5
years (captured by a binary variable: 1 = yes, 0 = no), the customer’s
complaint behavior, gender, age, higher education, annual income,
geographic information etc.

© Palmatier, Petersen, and Germann 8

 Logistic Regression Example

 Max estimated a logistic regression model where he used the respective

customer’s churn information as the dependent variable (i.e., customer
churned: yes, no) and the other customer variables as the independent
variables.

 The model indicated that churn probabilities of MyFriend’s current

customers ranged from as low as 9% to as high as 99%. He showed these
results to his manager, MyFriend’s Director of Marketing, who was
intrigued by the models’ strong fit.

 Together, they appended a variable they had previously calculated that

captured each customer’s worth to MyFriend. That variable was based on
how much MyFriend earned from each customer – the average annual
premium received per customer – minus the average yearly customer
costs due to insured events etc.

© Palmatier, Petersen, and Germann 9

 Logistic Regression Example

 They then sorted the customers by (1) customer worth and (2)
probability of churning. Many of MyFriend’s most valuable customers had
a high probability of churning.

 Armed with these insights, Max’s Manager assembled a team and tasked it
with identifying interventions they could target at MyFriend’s most
valuable customers with high probability of churning.

 She hoped these interventions would keep these customers happy and
thus not cancel their insurance policy.

© Palmatier, Petersen, and Germann 10

 Agenda

 Learning Objectives

 Introduction
 Objectives of Logistic Regression Models
 Common Uses of Logistic Regression Models
 Logistic Regression Example

 Logistic Regression
 Logistic Regression Model
 Odds of an Event
 Model Fit and Testing for Significance of the Coefficients
 Odds Ratio in Logistic Regression
 Example
 Logistic regression with one predictor variable
 Logistic regression with multiple predictor variables

 Summary

 Takeaways

© Palmatier, Petersen, and Germann 11

 Logistic Regression Model

 Logistic regression is frequently used when the dependent variable y can

only take on one of two categories, such as 1 or 0.

 The logistic regression model takes the following form

where e is the base of the natural logarithm, i.e., 2.71828, p is the probability
of the event of interest happening (e.g., probability of a customer to churn),
X are the predictor (or x) variables, and β are the coefficients to be
estimated.

 Thus, the model estimates the probability of the event happening as a

function of the X variables.

© Palmatier, Petersen, and Germann 12

 Logistic Regression Model

 Since they are probabilities, estimates of p will always be between 0 and 1.

 Logistic regression can be performed with one or more x variables that

can be either continuous of binary.

 With only one x variable, 1.2

the probability p of a focal 1.0

Probability of churning
event happening follows .8
an s-shape with .6
asymptotes at 0 and 1.
.4

 For example, see the figure .2

to the right. .0

1 2 3 4 5 6 7 8
x (number of complaints)

© Palmatier, Petersen, and Germann 13

 Odds of an Event

 This equation describes the odds of an event of interest happening (e.g., a

specific customer churning).

 Keep in mind that p is the probability of the event of interest happening,

1-p if the probability of the event of interest not happening, and captures
the odds of the event happening.

© Palmatier, Petersen, and Germann 14

 Odds of an Event

 Probability and odds are often used interchangeably. However, they are
not equivalent mathematically.
 Probability ranges between 0 and 1, where 0 is an impossible event and 1 an
inevitable event.
 Probability is usually reported as a percentage ranging from 0% to 100%
(which is equivalent to ranging from 0 and 1).
 Probability captures the ratio of how often an event of interest occurred (e.g.,
customers who churned) over all possible events (e.g., customers who did and
did not churn).
 Odds on the other hand can range from 0 to infinity. Odds capture the ratio of
how often an event of interest occurred (e.g., customers who churned) over how
often it did not occur (e.g., customer who did not churn).

© Palmatier, Petersen, and Germann 15

 Model Fit and Testing for Significance of the
Coefficients
 Instead of minimizing the squared deviations (i.e., least squares) like OLS
regression does, logistic regression uses maximum likelihood to estimate
the regression coefficients. That is, it maximizes the likelihood that an
event occurs using iterative computing methods. Thus, logistic regression
model fit is also assessed differently.

 A likelihood value (i.e., - 2 times the log of the likelihood value, or -2LL) is
used to determine how well a logistic regression model fits overall.
Models that fit well have a small -2LL value, and a -2LL value of 0
indicates perfect model fit.

© Palmatier, Petersen, and Germann 16

 Model Fit and Testing for Significance of the
Coefficients
 Several R2- like measures of overall model fit have also been developed for
logistic regression models. The Pseudo R2 is based on the improvement in
-2LL between the model being investigated and the “null” model. It is
calculated as follows:

 Other R2- like measures used in logistic regression include Cox and Snell’s
R2 as well as the Nagelkerke value.

© Palmatier, Petersen, and Germann 17

 Odds Ratio in Logistic Regression

 Logistic regression will also provide the odds ratio for each predictor
variable x. The odds ratio is calculated as follows:

 
where βi is the coefficient of predictor variable xi.

 The odds ratio captures by how much the predicted odds of the event of
interest happening (e.g., customer churning) increases as the focal
predictor variable x (e.g., number of customer complaints) increases by
one unit.

© Palmatier, Petersen, and Germann 18

 Logistic regression with one predictor variable

 Returning to the MyFriend example, the table below shows the logistic
regression results based on one of the predictor variables Max had pulled,
number of customer complaints.
Predictor Standard
Coefficient z-value p-value Odds Ratio
variable Error
Constant -2.57 .296 -8.66 .000 .07
Complaints 1.71 .181 9.43 .000 5.52

 Looking at the complaint coefficient, we can see that it is statistically

significant and positive. Thus, as one would expect, as the number of
complaints increases, so does the probability that the customers will churn.
Using the coefficient, we can estimate the probability of a customer churning
given how often s/he has complained.

 For example, a customer who complained once has a 30% probability of

churning: 
© Palmatier, Petersen, and Germann 19
 Logistic regression with one predictor variable

 We can also look at the estimated probability of churning for customers

who complained between 0 and 4 times:

# of times complained 0 1 2 3 4
Estimated probability of churning 7% 30% 70% 93% 98%
Estimated odds of churning .076 .42 2.3 12.9 71.5

 Knowing the probability of a customer churning based on the number of

complaints, we can also calculate the odds of a customer churning,
considering that odds = .
 The odds ratio of number of complaints is given in Table 8.1. However,
since we have the coefficient for complaints, we can also calculate the
odds ratio considering that odds ratio = . Thus, the odds ratio of number
of complaints is = 5.52.

© Palmatier, Petersen, and Germann 20

 Logistic regression with multiple predictor
variables

 Let us add two additional predictor variables to the logistic regression

model. That is, besides number of complaints, let us also include age and
gender as predictor variables. Gender is a binary variable coded as 0 =
female, 1 = male.

Predictor Standard
Coefficient z-value p-value Odds Ratio
variable Error
Constant 1.24 .69 1.85 .064 3.45
Complaints 1.74 .23 7.73 .000 5.69
Age -.103 .02 -5.72 .000 .902
Gender -.00 .45 -.000 1.000 .999
 Looking across the coefficients, we can see that two predictor variables
are statistically significant: The number of complaints and age. In
contrast, gender is not significant.

© Palmatier, Petersen, and Germann 21

 Logistic regression with multiple predictor
variables
 We could estimate the model again without gender included. When we do
this, we get:
Predictor Standard
Coefficient z-value p-value Odds Ratio
variable Error
Constant 1.24 .65 1.90 .057 3.45
Complaints 1.74 .22 8.03 .000 5.70
Age -.103 .02 -5.77 .000 .902

 Using the coefficients, we can again calculate the probability that a

specific customer will churn using equation (1). For example, for a
customer who has complained 2 times and is 50 years old, the probability
of churning is: 

 
 Likewise, the probability of a customer who has complained 1 time and is
25 years old churning is 60%.
© Palmatier, Petersen, and Germann 22
 Agenda

 Learning Objectives

 Introduction
 Objectives of Logistic Regression Models
 Common Uses of Logistic Regression Models
 Logistic Regression Example

 Logistic Regression
 Logistic Regression Model
 Odds of an Event
 Model Fit and Testing for Significance of the Coefficients
 Odds Ratio in Logistic Regression
 Example
 Logistic regression with one predictor variable
 Logistic regression with multiple predictor variables

 Summary

 Takeaways

© Palmatier, Petersen, and Germann 23

 Summary

 Logistic regression is used to estimate the effect of one or more predictor

variables on an outcome variable that is binary, that is, an outcome
variable that has only two possible outcomes such as, for example, 1 and 0
or yes and no.

 Marketers use logistic regression for a variety of purposes. For example,

akin to RFM analysis, researchers can use logistic regression to decide
which customers to target with a marketing campaign.

 Importantly, logistic regression tends to outperform RFM analysis in

identifying customers most likely to respond favorably to a marketing
campaign thus resulting in a higher ROMI than RFM analysis.

 Logistic regression is based on maximum likelihood methods, that is,

iterative computing methods, and can be easily estimated using programs
such as R or other statistical software programs.

© Palmatier, Petersen, and Germann 24

 Agenda

 Learning Objectives

 Introduction
 Objectives of Logistic Regression Models
 Common Uses of Logistic Regression Models
 Logistic Regression Example

 Logistic Regression
 Logistic Regression Model
 Odds of an Event
 Model Fit and Testing for Significance of the Coefficients
 Odds Ratio in Logistic Regression
 Example
 Logistic regression with one predictor variable
 Logistic regression with multiple predictor variables

 Summary

 Takeaways

© Palmatier, Petersen, and Germann 25

 Takeaways

 Logistic regression is akin to other regression models (e.g. OLS

regression) but is used when the dependent (i.e., outcome) variable is
binary instead of, for example, continuous.

 Logistic regression allows researchers to estimate the probability that a

binary outcome of interest occurs based on one or more predictor
variables.

 Logistic regression allows researchers to identify variables that

significantly predict the binary outcome variable.

 The probability with which a binary event of interest occurs based on

predictor variables can also be expressed in odds.

© Palmatier, Petersen, and Germann 26

 Takeaways

 Probability and odds are often used interchangeably, but they are not
equivalent mathematically. Probability captures the ratio of how often a
binary event of interest occurs (e.g., customers who churn) over all
possible events (e.g., customers who do and do not churn).

 In contrast, odds capture the ratio of how often an event of interest

occurs (e.g., customers who churn) over how often it does not occur (e.g.,
customer who do not churn).

 Logistic regression provides model fit measures that tell the researcher
how good the predictor variables predict the outcome variable.

© Palmatier, Petersen, and Germann 27