Download Ordered regression models parallel partial and non parallel alternatives 1st Edition Andrew S. Fullerton ebook All Chapters PDF
Download Ordered regression models parallel partial and non parallel alternatives 1st Edition Andrew S. Fullerton ebook All Chapters PDF
Download Ordered regression models parallel partial and non parallel alternatives 1st Edition Andrew S. Fullerton ebook All Chapters PDF
https://ebookgate.com
https://ebookgate.com/product/ordered-regression-
models-parallel-partial-and-non-parallel-
alternatives-1st-edition-andrew-s-fullerton/
https://ebookgate.com/product/plutarch-s-parallel-lives-1st-edition-
chrysanthos-s-chrysanthou/
ebookgate.com
https://ebookgate.com/product/professional-parallel-programming-with-
c-master-parallel-extensions-with-net-4-1st-edition-gaston-hillar/
ebookgate.com
https://ebookgate.com/product/king-lear-parallel-text-edition-rene-
weis/
ebookgate.com
https://ebookgate.com/product/high-performance-parallel-i-o-1st-
edition-i-foster/
ebookgate.com
https://ebookgate.com/product/parallel-computational-fluid-
dynamics-2000-1st-edition-c-b-jenssen/
ebookgate.com
Ordered
Regression
Models
Parallel, Partial, and
Non-Parallel Alternatives
Chapman & Hall/CRC
Statistics in the Social and Behavioral Sciences Series
Series Editors
Jeff Gill Steven Heeringa
Washington University, USA University of Michigan, USA
Tom Snijders
Oxford University, UK
University of Groningen, NL
Large and complex datasets are becoming prevalent in the social and behavioral
sciences and statistical methods are crucial for the analysis and interpretation of such
data. This series aims to capture new developments in statistical methodology with
particular relevance to applications in the social and behavioral sciences. It seeks to
promote appropriate use of statistical, econometric and psychometric methods in
these applied sciences by publishing a broad range of reference works, textbooks and
handbooks.
Ordered
Regression
Models
Parallel, Partial, and
Non-Parallel Alternatives
Andrew S. Fullerton
Oklahoma State University
Stillwater, Oklahoma
Jun Xu
Ball State University
Muncie, Indiana
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2016 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been
made to publish reliable data and information, but the author and publisher cannot assume responsibility for the valid-
ity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright
holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this
form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may
rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or uti-
lized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopy-
ing, microfilming, and recording, or in any information storage or retrieval system, without written permission from the
publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://
www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923,
978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For
organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for
identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
To my mother, Ede Steele, and the memory of my father, John Fullerton.
Andrew S. Fullerton
Preface............................................................................................................................................ xiii
Authors............................................................................................................................................xv
1. Introduction..............................................................................................................................1
1.1 Ordinal Variables versus Ordinal Models.................................................................1
1.2 Brief History of Binary and Ordered Regression Models........................................3
1.3 Three Approaches to Ordered Regression Models...................................................4
1.3.1 Cumulative Models..........................................................................................5
1.3.2 Stage Models......................................................................................................6
1.3.3 Adjacent Models................................................................................................7
1.4 The Parallel Regression Assumption..........................................................................8
1.5 A Typology of Ordered Regression Models...............................................................9
1.6 Link Functions.............................................................................................................. 11
1.7 Asymmetrical Relationships in Partial and Nonparallel Models......................... 12
1.8 Hypothesis Testing and Model Fit in Ordered Regression Models..................... 12
1.9 Datasets Used in the Empirical Examples................................................................ 14
1.9.1 Cumulative Models of Self-Rated Health.................................................... 14
1.9.2 Stage Models of Educational Attainment.................................................... 15
1.9.3 Adjacent Models of Welfare Attitudes......................................................... 16
1.10 Example: Education and Welfare Attitudes............................................................. 17
1.11 Organization of the Book............................................................................................ 19
2. Parallel Models...................................................................................................................... 21
2.1 Parallel Cumulative Model......................................................................................... 21
2.1.1 A Latent Variable Model................................................................................22
2.1.2 A Nonlinear Cumulative Probability Model.............................................. 23
2.1.3 Interpreting the Results from Ordered Regression Models..................... 24
2.1.4 Example: Parallel Cumulative Models of General Self-Rated Health..... 27
2.2 Parallel Continuation Ratio Model............................................................................ 31
2.2.1 A Latent Variable Model................................................................................ 32
2.2.2 A Nonlinear Conditional Probability Model.............................................. 33
2.2.3 Potential Sample Selection Bias and Scaling Effects in Stage Models....34
2.2.4 Example: Parallel Continuation Ratio Models of
Educational Attainment................................................................................. 35
2.3 Parallel Adjacent Category Model............................................................................. 39
2.3.1 A Nonlinear Adjacent Probability Model................................................... 40
2.3.2 Example: Parallel Adjacent Category Models of
Welfare Spending Attitudes..........................................................................42
2.4 Estimation..................................................................................................................... 45
2.4.1 Basics of Maximum Likelihood Estimation................................................ 45
2.4.2 Common Problems in the Use of MLE for Ordered Regression Models.... 46
2.4.3 Person-Level and Person/Threshold-Level Estimation............................ 47
2.5 Conclusions................................................................................................................... 50
ix
x Contents
2.6 Appendix....................................................................................................................... 50
2.6.1 Equivalence of Two Parallel Complementary Log Log Models............... 50
2.6.2 Stata Codes for Parallel Ordered Logit Models.......................................... 52
2.6.3 R Codes for Parallel Ordered Logit Models................................................ 52
3. Partial Models........................................................................................................................ 55
3.1 Unconstrained versus Constrained Partial Models................................................ 55
3.2 Partial Cumulative Models......................................................................................... 56
3.2.1 Unconstrained Partial Cumulative Model.................................................. 57
3.2.2 Constrained Partial Cumulative Model...................................................... 57
3.2.3 Example: Partial Cumulative Models of Self-Rated Health...................... 58
3.3 Partial Continuation Ratio Models............................................................................64
3.3.1 Unconstrained Partial Continuation Ratio Models...................................64
3.3.2 Constrained Partial Continuation Ratio Models........................................65
3.3.3 Example: Partial Continuation Ratio Models of
Educational Attainment.................................................................................65
3.4 Partial Adjacent Category Models............................................................................. 70
3.4.1 Unconstrained Partial Adjacent Category Models.................................... 71
3.4.2 Constrained Partial Adjacent Category Models......................................... 72
3.4.3 Example: Partial Adjacent Category Models of
Welfare Spending Attitudes.......................................................................... 72
3.5 Dimensionality in Partial Models.............................................................................77
3.6 Conclusions................................................................................................................... 78
3.6.1 Guidelines for Choosing a Partial or Parallel Ordered
Regression Model............................................................................................80
3.7 Appendix.......................................................................................................................80
3.7.1 Stata Codes for Partial Ordered Logit Models...........................................80
3.7.2 R Codes for Partial Ordered Logit Models................................................. 82
4. Nonparallel Models............................................................................................................... 85
4.1 The Nonparallel Cumulative Model......................................................................... 86
4.1.1 Example: Nonparallel Cumulative Models of Self-Rated Health............ 87
4.2 The Nonparallel Continuation Ratio Model............................................................ 92
4.2.1 Example: Nonparallel Continuation Ratio Models of
Educational Attainment................................................................................. 93
4.3 The Nonparallel Adjacent Category Model............................................................. 98
4.3.1 Example: Nonparallel Adjacent Category Models of
Welfare Spending Attitudes.......................................................................... 99
4.4 Practical Issues in the Estimation of Nonparallel Models................................... 103
4.5 Conclusions................................................................................................................. 104
4.5.1 Guidelines for Choosing a Parallel, Partial, or Nonparallel
Ordered Regression Model......................................................................... 105
4.6 Appendix..................................................................................................................... 106
4.6.1 Stata Codes for Nonparallel Ordered Logit Models................................ 106
4.6.2 R Codes for Nonparallel Ordered Logit Models...................................... 106
6. Extensions............................................................................................................................. 135
6.1 Heterogeneous Choice Models................................................................................ 135
6.2 Empirical Examples of Heterogeneous Choice Models....................................... 136
6.3 Group Comparisons Using Heterogeneous Choice Models................................ 139
6.4 Introduction to Multilevel Ordered Response Regression.................................. 141
6.4.1 Varying Intercepts........................................................................................ 142
6.4.2 Varying Slopes............................................................................................... 143
6.4.3 Intercepts and Slopes as Outcomes............................................................ 145
6.4.4 Multilevel Nonparallel Ordered Regression Models.............................. 147
6.4.5 Multilevel Ordered Regression Models Using Alternative
Link Functions.............................................................................................. 148
6.5 Bayesian Analysis of Ordered Response Regression........................................... 149
6.6 Empirical Examples of Bayesian Ordered Regression Models........................... 150
6.6.1 Bayesian Estimation of the Parallel Cumulative Model.......................... 150
6.6.2 Bayesian Estimation of the Nonparallel Cumulative Model.................. 151
6.6.3 Bayesian Estimation of the Random Cutpoint Multilevel
Stereotype Logit Model................................................................................ 155
6.7 Conclusion................................................................................................................... 158
References.................................................................................................................................... 159
Index.............................................................................................................................................. 167
This page intentionally left blank
Preface
This book is about regression models for ordinal outcomes, which are variables that have
ordered categories but unknown spacing between categories. Ordinal outcomes are very
common in the social and behavioral sciences. Self-rated health, educational attainment,
income measured in intervals of $25,000, and attitudes measured on a Likert scale are
just a few basic examples of ordinal variables that one may analyze using one or more
of the ordered regression models we present. Our focus in this book is narrower than
most books on the analysis of ordinal variables because we only consider regression-based
approaches. However, we provide arguably the most comprehensive coverage of the three
major classes of ordered regression models (cumulative, stage, and adjacent) and varia-
tions based on the application of the parallel regression assumption.
We assume that readers have a basic understanding of binary regression models, such
as logit and probit, and of the linear regression model. Although it is relatively easy to esti-
mate a wide variety of ordered regression models given recent advances in statistical soft-
ware programs, some researchers continue to analyze ordinal outcomes using the linear or
binary regression model on the grounds that results from an ordered regression model are
too difficult to interpret or present effectively. In this book, we show that ordered regres-
sion models are extensions of the more familiar binary regression model and that both
models rely on the same methods of estimation and interpretation. Therefore, researchers
that are familiar with the binary regression model should find it relatively straightfor-
ward to apply that knowledge to the estimation and interpretation of results from ordered
regression models. We illustrate the different ordered regression models using empirical
examples in order to highlight several ways to interpret and present the results.
We also show that ordered regression models have clear advantages over linear and
binary regression models for the analysis of ordinal outcomes. The linear regression
model requires the assumption of equal spacing among categories, which is not realistic
for ordinal outcomes that by definition have discrete categories with unknown spacing
among them. The binary regression model requires collapsing outcome categories in order
to construct a binary outcome, which results in a loss of information. Both approaches
can produce very misleading results. Ordered regression models avoid these problems
because they assume that the outcome is ordinal rather than continuous, and they do not
require one to recode the original ordinal variable into a single binary variable.
Ordered regression models that retain the parallel regression assumption also have
two advantages over the multinomial logit model, which is designed for nominal out-
comes but may also be used for ordinal outcomes. Ordered regression models with the
parallel assumption retain the ordinal nature of the relationships between the dependent
and independent variables, and they also require fewer parameters. Multinomial logit,
however, ignores the ordering of response categories and requires additional parameters,
which makes it less “efficient.”
The parallel regression assumption is the key assumption in traditional ordered regres-
sion models, and we devote an entire chapter of the book to this assumption (Chapter 5).
We distinguish between ordered regression models that apply the parallel regression
assumption to all, some, or none of the independent variables, and we refer to these as
parallel, partial, and nonparallel models, respectively. There are also several classes or
groups of ordered regression models based on different probabilities of interest, including
xiii
xiv Preface
cumulative, stage, and adjacent models. The so-called “ordered logit” and “ordered probit”
models are actually just the same type of ordered regression model with different link
functions: the cumulative model. The fact that the cumulative logit model is simply known
as the “ordinal logit” model is evidence of its popularity, but there are other ordinal logit
models, such as continuation ratio logit and adjacent category logit, that one should con-
sider. Building on our earlier work (Fullerton 2009; Fullerton and Xu 2012), we provide a
conceptual framework for understanding ordered regression models based on the prob-
ability of interest and on the application of the parallel regression assumption. The large
number of modeling alternatives is useful because it allows one to select the most appro-
priate model given the type of ordinal outcome and how restrictive the parallel assump-
tion is for each variable.
We provide brief examples in the appendices to Chapters 2, 3, and 4 that show how to
estimate the models in these chapters using R and Stata. More detailed examples are avail-
able at the following Web site: http://sociology.okstate.edu/faculty-staff-directory/faculty-
directory/dr-andrew-fullerton. The Web site contains JAGS, R, and Stata codes to estimate
the models we present in this book. It also contains detailed examples, syntax to reproduce
the results in the book, and Stata maximum likelihood estimation (MLE) routines that we
created to estimate the models in Chapters 2 through 4.
We benefitted from the comments and encouragement we received from many colleagues
during the process of writing this book. We thank Jeffrey Dixon, Andrew Gelman, Michael
Lacy, Michael Long, Scott Long, Ann O’Connell, and Richard Williams for their comments
on chapters. We appreciate their very helpful suggestions. We are indebted to Thomas Yee
for providing help with our R codes using his VGAM package, John Kruschke for shar-
ing his expertise in Bayesian statistics and for providing help with his BEST package, and
Jason Doll for many informal but informative discussions and e-mail exchanges about
JAGS and rjags. We also appreciate Riley Dunlap’s support for this project and our earlier
work on this topic. We thank Simon Cheng and David Weakliem for their feedback on
our earlier work on ordered regression models, which laid the groundwork for this book.
At Chapman & Hall/CRC Press, we thank our editor, John Kimmel, for his advice and sup-
port throughout this process. We also thank the anonymous reviewers for their valuable
critiques. Finally, we want to thank the students from our categorical data analysis courses
at Ball State University and Oklahoma State University over the last few years for their very
thoughtful comments and suggestions.
Authors
Jun Xu is an associate professor of sociology at Ball State University, Muncie, Indiana. His
primary research interests include Asia and Asian America, social epidemiology, and sta-
tistical modeling and programing. His work has been published in journals such as Social
Forces, Social Science & Medicine, Sociological Methods & Research, Social Science Research, and
The Stata Journal.
xv
This page intentionally left blank
1
Introduction
* Count variables, such as the number of promotions a worker has received, also have ordered categories with
known distances between them. However, count variables are considered discrete rather than continuous
because there are a finite number of values. For a discussion of the connections between count models and
ordinal models, see Clogg and Shihadeh (1994, p. 150).
1
2 Ordered Regression Models
a “stochastic ordering” (Agresti 2010, pp. 24–25), which means that the cumulative
distribution f unction (CDF) for one group of respondents is consistently below the CDF
for another group (McCullagh 1980, p. 115).* For example, if education increases support
for health care spending on a Likert scale from 1 (strong opposition) to 5 (strong support)
and there is a strict stochastic ordering, then respondents with 16 years of schooling
will have lower cumulative probabilities for each outcome category than respondents
with only 15 years of schooling. Respondents with higher levels of education will be
more likely to cluster at the upper end of the scale near strong support, which means
that at any given point in the distribution they will have lower cumulative probabilities
(i.e., Pr [y ≤ m]) compared to respondents with less education. The key aspect of this defi-
nition is that stochastic ordering is the result of model constraints. We will discuss one
such constraint, the parallel regression assumption, briefly at the end this chapter and
then in more detail in Chapter 5.
Using the strict definition, the “parallel” models that we introduce in Chapter 2 are the
only ordinal models we consider in this book.† However, we argue that it is useful to think
of the less constrained models as either partially ordinal or ordinal in a less strict sense.
The “partial” models that we discuss in Chapter 3 are partially ordinal in the sense that
they maintain a strict stochastic ordering for a subset of variables in the model through the
application of the parallel regression assumption, which constrains the effects of those
variables to remain constant across outcome levels. We may consider an even broader class
of models as ordinal if we only require that the models are not “permutation invariant”
(McCullagh 1980, p. 116). Models designed for nominal dependent variables, such as
multinomial logit, are permutation invariant, which means that arbitrary re-ordering of
the outcome categories has no effect on the coefficients or on model fit (McCullagh 1980,
p. 116). Permutation invariance is an important characteristic for nominal models because
there is no inherent ordering to the categories. Using these criteria, a categorical model is
“ordinal” if re-ordering the outcome categories leads to different results.
We refer to the models in this book as ordered regression models because they are varia-
tions on three ordinal models in the strictest sense (see Chapter 2). The variations are
based on different ways to relax the parallel regression assumption. The results are not
permutation invariant for almost all of the models we present in Chapters 2 through 4.
The nonparallel adjacent category model is the only one that is permutation invariant,
which means that it is ideally suited for nominal outcomes. The logit version of this model
is equivalent to the well-known multinomial logit model (McFadden 1973). Although it
is important to consider the defining characteristics of an ordinal model, the decision to
label a model as “ordinal,” “partially ordinal,” or “nonordinal” is not as important as other
modeling decisions, including the probability of interest and the application of the paral-
lel regression assumption. Throughout the book, we focus on how the type of probabil-
ity and application of the parallel regression assumption influence the results in ordered
regression models. We refer to the sets of ordered regression models based on different
probabilities of interest as “approaches to ordered regression models,” and we organize
Chapters 2 through 4 around these approaches.
The ordered regression models we consider are very useful for researchers examin-
ing ordinal outcomes for two main reasons. First, the models are appropriate for a wide
* The CDF describes the probability that a random variable, such as the error term, is less than or equal to a
particular value (see Gill 2006, p. 348).
† We also consider several model extensions in Chapter 6, and the ones that retain the parallel regression
variety of ordinal variables. Some ordinal outcomes have categories that represent inter-
vals on an underlying latent continuous scale. For example, an ordinal measure of income
with c ategories based on $25,000 intervals (e.g., <$25,000, $25,000 to <$50,000, $50,000 to
<$75,000) is essentially a continuous variable grouped into intervals on an ordinal scale
from lowest to highest income. As a result, scholars refer to some ordinal variables as
“grouped continuous” variables (Anderson 1984, p. 2). However, other ordinal variables,
such as educational attainment, are the result of a series of sequential decisions. The cat-
egories are ordered, but one must proceed through earlier stages in the sequence in order
to reach a higher category. For example, in order to attain a postgraduate degree, one must
first attain high school and college degrees. Finally, there are ordinal variables that have
categories that are of substantive interest, such as subjective social class (Fullerton and
Xu 2015), which are not the result of a sequential process and may or may not correspond
with an underlying latent continuous variable. The broad range of models covered in this
book is particularly important given the diversity of types of ordinal variables in the social
and behavioral sciences.
Second, most of the models in this book allow for asymmetrical effects. Researchers
often assume that the effects of independent variables are “symmetrical” (Jordan 1965;
Lieberson 1985). In the context of public opinion research, the assumption of symmetry
means that the effect of a variable on negative views toward an issue is equal in magnitude
to its effect on positive views in the opposite direction. For example, if political conserva-
tism is associated with increased opposition to same-sex marriage, then it should also be
associated with decreased support for same-sex marriage, and the relative strength of each
relationship should be equivalent.
However, the assumption of symmetry (Jordan 1965; Lieberson 1985) or bipolarity
(Cacioppo and Berntson 1994; Cacioppo et al. 1997) may be concealing an asymmetrical
relationship (Fullerton and Dixon 2010). Studies have shown that the racialization of wel-
fare attitudes is greater for opposition than for support (Fullerton and Dixon 2009), and
income reduces the incidence of low levels of life satisfaction but does not increase the
incidence of high levels of life satisfaction (Boes and Winkelmann 2010). The partial and
nonparallel models allow for asymmetrical relationships, whereas linear regression and
traditional ordered models impose symmetry on the relationships (for more details, see
Section 1.7).