BIO 226: APPLIED LONGITUDINAL ANALYSIS

LECTURE 16

Review of Logistic and Poisson Regression Models

Generalized Linear Models

Generalized linear models are a class of regression models; they include the
standard linear regression model but also many other important models:

- Linear regression for continuous data

- Logistic regression for binary data

- Log-linear/Poisson regression models for count data

Generalized linear models extend the methods of regression analysis to

settings where the outcome variable can be categorical.

In the remainder of the course, we consider extensions of generalized linear

models to longitudinal data.

2
 Motivating Example

Oral Treatment of Toenail Infection

Randomized, double-blind, parallel-group, multicenter study of 294 patients

comparing 2 oral treatments (denoted A and B) for toe-nail infection.

Outcome variable: Binary variable indicating presence of onycholysis

(separation of the nail plate from the nail bed).

Patients evaluated for degree of onycholysis (separation of the nail plate

from the nail-bed) at baseline (week 0) and at weeks 4, 8, 12, 24, 36, and
48.

Interested in the rate of decline of the proportion of patients with

onycholysis over time and the effects of treatment on that rate.

Motivating Example

Clinical trial of anti-epileptic drug progabide

(Thall and Vail, Biometrics, 1990)

Randomized, placebo-controlled study of treatment of epileptic seizures

with progabide.

Patients were randomized to treatment with progabide, or to placebo in

addition to standard therapy.

Outcome variable: Count of number of seizures

Measurement schedule: Baseline measurement during 8 weeks prior to

randomization. Four measurements during consecutive two-week intervals.

Sample size: 28 epileptics on placebo; 31 epileptics on progabide

4
 Review of Generalized Linear Models for a Single
Response

So far, we have considered linear regression models for a continuous

response, Y , of the following form

Y = β1X1 + β2X2 + . . . + βpXp + e

The response variable, Y , is assumed to have a normal distribution with

mean

E(Y ) = β1X1 + β2X2 + . . . + βpXp

and with variance, σ 2.

Recall that the population intercept (for X1 = 1), β1, has interpretation as
the mean value of the response when all of the covariates take on the value
zero.

The population slope, say βk , has interpretation in terms of the expected

change in the mean response for a single-unit change in Xk given that all
of the other covariates remain constant.

In many studies, however, we are interested in a response variable that is

dichotomous/binary rather than continuous.

Next, we consider a regression model for a binary (or dichotomous)

response.

6
 Review: Logistic Regression

Let Y be a binary response, where

Y = 1 represents a “success”; Y = 0 represents a “failure”.

Then the mean of the binary response variable, denoted π, is the proportion
of successes or the probability that the response takes on the value 1.

That is,
π = E(Y ) = Pr(Y = 1) = Pr(“success”)

With a binary response, we are usually interested in estimating the

probability π, and relating it to a set of covariates.

To do this, we can use logistic regression.

A naive strategy for modeling a binary response is to consider a linear

regression model

π = E(Y ) = β1X1 + β2X2 + . . . + βpXp

However, in general, this model is not feasible since π is a probability and

is restricted to values between 0 and 1.

Also, the usual assumption of homogeneity of variance would be violated

since the variance of a binary response depends on the mean, i.e.

Var(Y ) = π (1 − π)

8
Instead, we can consider a logistic regression model where

ln [π/ (1 − π)] = β1X1 + β2X2 + . . . + βpXp

This model accommodates the constraint that π is restricted to values

between 0 and 1.

Recall that π/ (1 − π) is defined as the odds of success.

Therefore, modeling π with a logistic function can be considered equivalent

to a linear regression model where the mean of the continuous response has
been replaced by the logarithm of the odds of success.

Note that the relationship between π and the covariates is non-linear.

Figure 1: Plot of logistic response function.

10
Under the assumption that the binary responses are Bernoulli random
variables, we can use ML estimation to obtain estimates of the logistic
regression parameters.

Finally, recall the relationship between “odds” and “probabilities”.

π
Odds = ;
1−π

Odds
π= .
1 + Odds

11

Given the logistic regression model

ln [π/ (1 − π)] = β1X1 + β2X2 + . . . + βpXp

the population intercept, β1, has interpretation as the log odds of success
when all of the covariates take on the value zero.

The population slope, say βk , has interpretation in terms of the change in

log odds of success for a single-unit change in Xk given that all of the other
covariates remain constant.

When one of the covariates is dichotomous, say X2, then β2 has a special
interpretation:

exp (β2) is the odds ratio or ratio of odds of success for the two possible
levels of X2 (given that all of the other covariates remain constant).

12
Keep in mind that as:

π increases

⇒ odds of success increases

⇒ log odds of success increases

Similarly, as:

π decreases

⇒ odds of success decreases

⇒ log odds of success decreases

13

Example: Development of bronchopulmonary dysplasia (BPD) in a sample

of 223 low birth weight infants.

Binary Response: Y = 1 if BPD is present, Y = 0 otherwise.

Covariate: Birth weight of infant in grams.

Consider the following logistic regression model

ln [π/ (1 − π)] = β1 + β2Weight

where π = E(Y ) = Pr(Y = 1) = Pr(BPD)

14
For the 223 infants in the sample, the estimated logistic regression (obtained
using ML) is
π / (1 − π
ln [ )] = 4.0343 − 0.0042 Weight

The ML estimate of β2 implies that, for every 1 gram increase in birth

weight, the log odds of BPD decreases by 0.0042.

For example, the odds of BPD for an infant weighing 1200 grams is

exp (4.0343 − 1200 ∗ .0042) = exp (−1.0057)

= 0.3658

Thus the predicted probability of BPD is:

0.3658/ (1 + 0.3658) = 0.268

15

Figure 2: Plot of estimated logistic response function of BPD on birth

weight.

16
 Review: Poisson Regression

In Poisson regression, the response variable is a count (e.g. number of cases

of a disease in a given period of time).

The Poisson distribution provides the basis of likelihood-based inference.

Often the counts may be expressed as rates.

That is, the count or absolute number of events is often not satisfactory
because any comparison depends almost entirely on the sizes of the groups
(or the “time at risk”) that generated the observations.

17

Like a proportion or probability, a rate provides a basis for direct

comparison.

In either case, Poisson regression relates the expected counts or rates to a

set of covariates.

The Poisson regression model has two components:

1. The response variable is a count and is assumed to have a Poisson

distribution.
That is, the probability a specific number of events, y, occurs is

Pr(y events) = e−λλy /y!

Note that λ is the expected count or number of events and the expected
rate is given by λ/t, where t is a relevant baseline measure (e.g., t might
be the number of persons or the number of person-years of observation).

18
2. ln(λ/t) = β1X1 + β2X2 + . . . + βpXp

Note that since ln(λ/t) = ln(λ) − ln(t), the Poisson regression model can
also be considered as

ln(λ) = ln(t) + β1X1 + β2X2 + . . . + βpXp

where the ‘coefficient’ associated with ln(t) is fixed to be 1.

This adjustment term is known as an “offset”.

19

Therefore, modelling λ (or λ/t) with a log function can be considered

equivalent to a linear regression model where the mean of the continuous
response has been replaced by the logarithm of the expected count (or rate).

Note that the relationship between λ (or λ/t) and the covariates is non-
linear.

We can use ML estimation to obtain estimates of the Poisson regression

parameters, under the assumption that the responses are Poisson random
variables.

20
Given the Poisson regression model

ln(λ/t) = β1X1 + β2X2 + . . . + βpXp

the population intercept, β1, has interpretation as the log expected rate
when all the covariates take on the value zero.

The population slope, say βk , has interpretation in terms of the change in

log expected rate for a single-unit change in Xk given that all of the other
covariates remain constant.

When one of the covariates is dichotomous, say X2, then β2 has a special
interpretation:

exp (β2) is the (incidence) rate ratio for the two possible levels of X2 (given
that all of the other covariates remain constant).

21

Example: Prospective study of coronary heart disease (CHD).

The study observed 3154 men aged 40-50 for an average of 8 years and
recorded incidence of cases of CHD.

The risk factors considered include:

Smoking exposure: 0, 10, 20, 30 cigs per day;

Blood Pressure: 0 (< 140), 1 (≥ 140);
Behavior Type: 0 (type B), 1 (type A).

A simple Poisson regression model is:

ln (λ/t) = ln(rate of CHD) = β1 + β2 Smoke

or
ln (λ) = ln(t) + β1 + β2 Smoke

22
 Person - Blood
Years Smoking Pressure Behavior CHD

5268.2 0 0 0 20
2542.0 10 0 0 16
1140.7 20 0 0 13
614.6 30 0 0 3
4451.1 0 0 1 41
2243.5 10 0 1 24
1153.6 20 0 1 27
925.0 30 0 1 17
1366.8 0 1 0 8
497.0 10 1 0 9
238.1 20 1 0 3
146.3 30 1 0 7
1251.9 0 1 1 29
640.0 10 1 1 21
374.5 20 1 1 7
338.2 30 1 1 12

23

In this model the ML estimate of β2 is 0.0318. That is, the rate of CHD
increases by a factor of exp(0.0318) = 1.032 for every cigarette smoked.

Alternatively, the rate of CHD in smokers of one pack per day (20 cigs)
is estimated to be (1.032)20 = 1.88 times higher than the rate of CHD in
non-smokers.

We can include the additional risk factors in the following model:

ln (λ/t) = β1 + β2 Smoke + β3 Type + β4BP

Effect Estimate Std. Error

Intercept -5.420 0.130

Smoke 0.027 0.006
Type 0.753 0.136
BP 0.753 0.129

24
Now, adjusted rate of CHD (controlling for BP and behavior type) increases
by a factor of exp(0.027) = 1.028 for every cigarette smoked.

Adjusted rate of CHD in smokers of one pack per day (20 cigs) is estimated
to be (1.027)20 = 1.7 times higher than rate of CHD in non-smokers.

Finally, note that when a Poisson regression model is applied to data

consisting of very small rates (say, λ/t << 0.01), then the rate is
approximately equal to the corresponding probability, p, and

ln (rate) ≈ ln (p) ≈ ln [p/ (1 − p)]

Therefore, the parameters for Poisson regression and logistic regression

models are approximately equal when the event being studied is rare.

In that case, results from a Poisson and logistic regression will not give
discernibly different results.

25

Overdispersion

Count data (or counts of number of successes) often have variability that
far exceeds that predicted by Poisson (or binomial) distribution.

This phenomenon is referred to as overdispersion.

Although underdispersion can also arise, it is far less common.

Failure to account for overdispersion has negligible impact of the estimated

regression coefficients.

Neglecting overdispersion results in standard errors being underestimated

and potentially misleading inferences (e.g., confidence intervals that are too
narrow and p-values that are too small).

26
 Example: Clinical Trial of Antibiotics for Leprosy

Placebo-controlled clinical trial of 30 patients with leprosy at the Eversley

Childs Sanitorium in the Philippines.

Participants were randomized to either of two antibiotics (denoted

treatment drug A and B) or to a placebo (denoted treatment drug C).

Baseline data on number of leprosy bacilli at 6 sites of body were recorded.

After several months of treatment, number of bacilli were recorded a second

time.

Outcome: Total count of number of leprosy bacilli at 6 sites.

27

Table 1: Mean count of leprosy bacilli at six sites of the body (and variance)
post-treatment.

Treatment Group Post-Treatment

Drug A (Antibiotic) 5.3

(21.6)

Drug B (Antibiotic) 6.1

(37.9)

Drug C (Placebo) 12.3

(51.1)

28
Consider outcome (post-treatment) at end of study.

Variability is approximately 4 to 6 times larger than that predicted by

Poisson variation.

Adjustments to nominal standard errors to account for overdispersion can

be made either by including a scale factor φ in specification of the Poisson
variance,

Var(Yi) = φ μi,

or by basing standard errors on the so-called “sandwich” estimator of


Cov(β).

29