This article was downloaded by: [Osaka University]

On: 19 November 2014, At: 21:26

Publisher: Taylor & Francis
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,
37-41 Mortimer Street, London W1T 3JH, UK

Journal of the American Statistical Association

Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/uasa20

Measurement Error Models

a
L. A. Stefanski
a
Department of Statistics , North Carolina State University , Raleigh , NC , 27695 , USA
Published online: 17 Feb 2012.

To cite this article: L. A. Stefanski (2000) Measurement Error Models, Journal of the American Statistical Association, 95:452,
1353-1358

To link to this article: http://dx.doi.org/10.1080/01621459.2000.10474347

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained
in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no
representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the
Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and
are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and
should be independently verified with primary sources of information. Taylor and Francis shall not be liable for
any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever
or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of
the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any
form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://
www.tandfonline.com/page/terms-and-conditions
 Theory and Methods of Statistics 1353

Bozdogan, H. (1994), Proceedings of the First U.S./Japan Conference of Lehmann, E. L. (1983), Theory of Point Estimation, New York Wiley.
the Frontiers of Statistical Modeling, Vols. 1-3, Amsterdam: Kluwer. Lindley, D. V. (1956), “On a Measure of Information Provided by an Ex-
Brockett, P. L. (1991), “Information Theoretic Approach to Actuarial Sci- periment,” Annals of Mathematical Statistics, 27, 986-1005.
ence: A Unification and Extension of Relevant Theory and Applica- (1961), “The Use of Prior Probability Distributions in Statistical
tions,” Transactions of Society of Actuaries, 43, 73-135. Inference and Decision,” in Proceedings of the Fourth Berkeley Sympo-
Brockett, P. L., Charnes, A,, Cooper, W. W., Learner, D., and Phillips, F. Y. sium, 1, Berkeley, C A University of California Press, pp. 436468.
(1995),“Information Theory as a Unifying Statistical Approach for Use Maasoumi, E. (1993), “A Compendium to Information Theory in Eco-
in Marketing Research,” European Journal of Operational Research, 84, nomics and Econometrics,” Econometric Reviews, 12(2), 137-181.
31&329. Pourahmadi, M., and Soofi, E. S. (2000), “Predictive Variance and Infor-
Burnham, K. P., and Anderson, D. R. (1998), Model Selection and Infer- mation Worth of Observations in Time Series,” Journal of Time Series
ence: A Practical Information-TheoreticApproach, New York: Springer- Analysis, 21, 413-434.
Verlag. Rao, C. R. (1973), Linear Statistical Inference and Its Applications (2nd
Cover, T. M., and Thomas, J. A. (1991), Elements of Information Theory, ed.), New York: Wiley.
New York: Wiley. Shannon, C. E. (1948), “A Mathematical Theory of Communication,” Bell
Csiszar, I. (19911, “Why Least Squares and Maximum Entropy? An Ax- System Technical Journal, 27, 379423.
iomatic Approach to Inference in Linear Inverse Problems,” The Annals Shore, J. E., and Johnson, R. W. (1980), “Axiomatic Derivation of the
of Statistics, 19, 2032-66. Principle of Maximum Entropy and the Principle of Minimum Cross-
Fisher, R. A. (1921),“On Mathematical Foundations of Theoretical Statis- Entropy,” IEEE Transactions on Informational Theory, IT-26, 26-37.
tics,” Philosophical Transactions of the Royal Society of London, Ser.
Soofi, E. S. (1994), “Capturing the Intangible Concept of Information,”
A, 222,309-368.
Journal of the American Statistical Association, 89, 1243-1254.
Fomby, T. B., and Hill, R. C. (1997),Advances in Econometrics: Applying
(1997), “Information Theoretic Regression Methods,” in Advances
Maximum Entropy to Econometric Problems, Vol. 12, Greenwich, CT:
in Econometrics:Applying Maximum Entropy to Econometric Problems,
JAI Press.
Gokhale, D. V., and Kullback, S. (1978), The Information in Contingency
12, eds. T. B. Fomby and R. C. Hill, Greenwich, C T JAI Press, pp.
25-83.
Downloaded by [Osaka University] at 21:26 19 November 2014

Tables, New York: Marcel Dekker.

Golan, A,, Judge, G. G., and Miller, D. (1996),Maximum Entropy Econo- Soofi, E. S., Ebrahimi, N., and Habibullah, M. (1995), “Information Dis-
metrics: Robust Estimation with Limited Data, New York: Wiley. tinguishability With Application to Analysis of Failure Data,” Journal
Good, I. J. (1950), Probability and Weighting of Evidence, London: Griffin. of the American Statistical Association, 90, 657468.
Gull, S. F. (1989), “Developments in Maximum Entropy Data Analysis,” Soofi, E. S., and Gokhale, D. V. (1997), “Information Theoretic Methods
in Maximum Entropy and Bayesian Methods, ed. J. Skilling, Boston: for Categorical Data,” in Advances in Econometrics:Applying Maximum
Kluwer. Entropy to Econometrics Problems, 12, eds. T. B. Fomby and R. C. Hill,
Jaynes, E. T. (1957), “Information Theory and Statistical Mechanics,” Greenwich, C T JAI Press, pp. 107-134.
Physical Review, 106, 62M30. Theil, H., and Fiebig, D. Z. (1984), Exploiting Continuity: Maximum
(1968), “Prior Probabilities,” IEEE TransactionsSystems in Science Entropy Estimation of Continuous Distributions, Cambridge, M A
and Cybernetics, SSC-4, 227-241. Ballinger.
Jeffreys, H. (1946),“An Invariant Form for the Prior Probability in Estima- Wyner, A. D., and Ziv, J. (1969), “On Communication of Analog Data
tion Problems,” Proceedings of the Royal Statistical Society (London), From Bounded Source Space,” Bell System Technical Journal, 48,3139-
Ser. A, 186,453461. 3172.
Kapur, J. N. (1989),Maximum Entropy Models in Science and Engineering, Yang, R., and Berger, J. (19971, “A Catalog of Noninformative Priors,”
New York: Wiley. ISDS Discussion Paper 97-42, Duke University.
Kullback, S. (1954), “Certain Inequalities in Information Theory and the Zellner, A. (1971), An Introduction to Bayesian Inference in Econometrics,
Cramer-Rao Inequality,” The Annals of Mathematical Statistics, 25, New York: Wiley.
745-75 1. (1988), “Optimal Information Processing and Bayes’s Theorem”
(1959),Information Theory and Statistics, New York Wiley. (with discussion), The American Statistician, 42, 278-284.
Kullback, S., and Leibler, R. A. (1951), “On Information and Sufficiency,” (1997),Bayesian Analysis in Econometrics and Statistics: The Zell-
Annals of Mathematical Statistics, 22, 79-86. ner View and Papers, Cheltenham, U.K.: Edward Elgar.

Measurement Error Models

L. A. STEFANSKI

1. INTRODUCTION the research on such models. The second was a com-

Following an invited paper session on errors-in-variables plaint that the scope of models studied was not keep-
models at the 1983 American Statistical Association An- ing pace with the complexity of problems encountered in
nual Meeting in Toronto, two comments foretold of change practice.
in the study of regression models in which predictor vari- With regard to names and key words, errors-in-variables
ables are measured with error. The first was an opin- models, regression with errors in x, and measurement error
ion that the negative connotation of the name “errors-in- models are still in use. However, publication of the book
variables models” was a hindrance to the impact of Measurement Error Models (Fuller 1987) established the
latter as the phrase associated with the study of regression
models in which the independent variables are measured
L. A. Stefanski is Professor, Department of Statistics, North Carolina
State University, Raleigh, NC 27695 (E-mail: [email protected]).The
author thanks JASA for the opportunity to write this vignette and acknowl-
edges the editor’s welcomed encouragement, the financial support of the @ 2000 American Statistical Association
National Science Foundation, and the career-long support as well as the Journal of the American Statistical Association
specific suggestions of R. J. Carroll and David Ruppert. December 2000, Vol. 95, No. 452, Vignettes
 1354 Journal of the American Statistical Association, December 2000

with error and explains why this vignette shares the same These four examples have in common the feature that for
title. certain statistical analyses, the variable X is preferred over
As for the second comment, the fact that statisticians are W : Methods for estimating animal abundance from line-
moving toward more complex models in general is clear transect data assume that distances are measured without
from reading any current issue of JASA. At the time of the error; for assessing the relationship between heart disease
Toronto meetings, the field of measurement error modeling and blood pressure, long-term average blood pressure is the
was already broadening to include more realistic models meaningful risk factor; personal exposure is the relevant
required for certain applications. In the period since then, risk factor in studies of the effects of NO2 exposure on res-
the breadth and complexity of the models studied under the piratory illness in children; and excess daily mortality due
heading of measurement error models has grown at an ever- to exposure to particulate matter is more directly related
increasing rate. This expansion is straining the terminology, to average personal exposure than to concentrations mea-
notation, and assumptions that were adequate for describing sured at central monitoring stations. These examples also
linear errors-in-variables models, and also is creating new have in common the feature that X is not observable for
problems for research. each observational unit, whereas W is. The problem is to fit
Despite the rapid developments in the field and the in- models described in terms of X (and other observed vari-
creasing use of measurement error models in practice over ables) given data on W (and the other observed variables).
the last 15-20 years, it is likely that many readers are not fa- Other, more detailed descriptions of measurement error
miliar with measurement error problems. The sections that problems have been given by Carroll, Ruppert, and Stefan-
follow provide an introduction to measurement error prob- ski (1995) and Fuller (1987).
Downloaded by [Osaka University] at 21:26 19 November 2014

lems and models and a necessarily cursory review of the

area, emphasizing certain key contributions and current and 2.1 Measurement Error in Simple Linear Regression
future research trends. Consider the usual simple linear regression model Y , =
a + p X , + ~ , , i= 1,. . . ,n.Suppose that the observed
W, X , + U,, where U, represents measurement error.
2. MEASUREMENT ERROR PROBLEMS Let pYlw denote the slope estimator from the least squares
In many areas of application, statistically meaningful regression of Y on W . Similarly let BYlxdenote the slope
models are defined in terms of variables X that for some estimator obtained by least squares regression of Y on X .
reason are not directly observable. In such situations, it is Then in the seemingly benign case in which the measure-
common for substitute variables W to be observed instead. ment errors U, have mean p~ = 0 and constant variance
The substitution of W for X complicates the statistical anal- 0; and are independent of the true predictors X , and the

ysis of the observed data when the purpose of the analysis is equation errors E,, it transpires that pylW = XpYlx + o p ( l ) ,
inference about a model defined in terms of X . Problems of where 1 = s$/(s$ + s;), where s2 denotes the sample
this nature are commonly called measurement error prob- variance of the subscript variable. Because 0 < 1 < 1, a
lems, and the statistical models and methods for analyzing consequence of ignoring measurement error is attenuation
such data are called measurement error models. (bias toward 0) in the slope estimator.
The terminology belies the generality of the class of This simple model illustrates the key features of measure-
problems and models connoted by the phrase “measurement ment error in regression models: The parameters of interest
error.” Its origins lie in the special case in which the substi- ( q p ) appear in the regression model for E ( Y I X ) , which
tute variable W is a measurement (in the usual sense of the depends on the unobserved variable X ; the second com-
word) of the true value of X . For example, the measured ponent of the model relates the observed substitute vari-
perpendicular distance W that an animal lies from a tran- able W to X (in this case via the additive error model
sect line can differ significantly from the true distance X W = X + U , note that in this case E ( W I X ) = X ) ; and the
depending on the method of measurement. Similarly, a per- model fit to the observed data results in biased estimators
son’s measured systolic blood pressure W differs from his of the parameters of interest. A less obvious feature that
or her long-term average systolic blood pressure X because it shares with other measurement error models is a lack of
of significant temporal variation as well as instrument and identifiability. The regression parameters ( a ,p) cannot be
reader error. These cases are fundamentally different from consistently estimated without additional data (e.g., repli-
the situation in which the ambient concentrations of NO2 in cate measurements) or additional information in the form
the bedroom and kitchen of a child’s home W = (Wl, W Z ) of distributional or moment restrictions on the error distri-
are substitutes for the child’s personal exposure to NO;! X butions or the distribution of X . An often-studied, early ver-
(which is directly measurable via a personal monitor but sion of the model known as the classical errors-in-variables
not done for reasons of cost or inconvenience). The latter model (Fuller 1987, sec. 1.3) incorporates the assumption
situation is similar to that in which airborne particulate mat- that the ratio of error variances 7 = a:/& is known. Under
ter concentration measured daily at a central location W is this identifiability assumption,the regression parameters are
used as a substitute for daily average (over a population consistently estimated using the method of orthogonal least
of individuals) personal exposure to particulate matter X squares, formulated and described more than 100 years ago
in time series studies of the relationship between daily air by R. J. Adcock (1877, 1878). However, the historical im-
pollution and daily mortality. portance of the model in which the ratio of error variances
 Theory and Methods of Statistics 1355

is assumed known is greater than its practical significance infinitely many nuisance parameters, for which the defi-
(Carroll et al. 1995; Carroll and Ruppert 1996). Finally, it ciencies of maximum likelihood estimation are now well
is important to distinguish between the classical errors-in- known. The pioneering work of Neyman and Scott (1948)
variables model and the classical error model. The latter and Kiefer and Wolfowitz (1956) on estimation in the pres-
phrase refers only to the measurement error component of ence of numerous nuisance parameters had significant im-
a model and describes the additive error model W = X + U , pact not only on the direction of research in the errors-in-
with U and X independent (or uncorrelated). variables literature, but also on the course of research in
statistics in general.
3. LINEAR ERRORS-IN-VARIABLES MODELS
Generalizations and variations of linear error-in-variables 4. MEASUREMENT ERROR IN NONLINEAR MODELS
models were studied extensively in the 100 years following In the 1980s, the research emphasis in measurement er-
Adcock’s description of orthogonal least squares. The ma- ror models shifted to problems in which the model of
jor generalizations include multiple and multivariate multi- interest defined in terms of X is something other than
ple linear regression, in which one or more predictor vari- a linear model. With increasing frequency, articles ap-
ables are measured with error and some predictor variables peared on nonlinear regression models with errors in both
are error free. The major variations depend on assumptions variables, generalized linear models with predictor vari-
about the unobserved predictors and the type of data or dis- ables measured with error, and nonparametric distribu-
tributional (moment) information available, ensuring iden- tion/density/regression estimation in the presence of mea-
tifiability of the regression parameters. surement error. But these are not the first investigations of
Downloaded by [Osaka University] at 21:26 19 November 2014

In functional errors-in-variables models, the unobserved measurement error in nonlinear models. Eddington ( 1913)
X i are modeled as unknown, nonrandom constants (pa- studied distribution estimation, and Trumpler and Weaver
rameters), whereas in structural errors-in-variables mod- (1953, sec. 1.56) described an interesting approach to
els, the observables and unobservables jointly vary in nonlinear regression errors-in-variables problems; however,
repeated sampling. For example, consider modeling the these researches were the exceptions, whereas by the late
relationship between aquatic species diversity Y and acid- 1980s, the focus of measurement error modeling research
neutralizing capacity X , given measurements of (Y,X ) was on nonlinear models.
from each of n lakes. If the only lakes of interest are those The early research in errors-in-variables models was
represented in the sample, then it is appropriate to model driven by applications in the physical sciences, especially
Xi, i = 1, . . . , n, as unknown constants. Alternatively, if the astronomy, and soon thereafter also by econometric appli-
lakes represented in the data are a random sample from a cations. Much of the current research in nonlinear measure-
large population of lakes, then it is appropriate to model ment error models is motivated by applications in the health
Xi,i = 1,.. . ,n, as iid random variables. sciences, this is especially true of research in generalized
Parameters are identified under a wide variety of combi- linear measurement error models. The article by Carroll,
nations of error distributional or moment restrictions and Spiegelman, Lan, Bailey, and Abbott (1984) on measure-
the existence of additional data (e.g., replicate measure- ment error in binary regression marks the shift in emphasis
ments), each giving rise to a unique and interesting infer- from linear to nonlinear measurement error models, and is
ence problem. Discussion of the numerous variations of noteworthy for breaking ground in the application of mea-
the model, the methods of analysis adapted to each, and surement error modeling in the health sciences and in the
the early contributions of many well-known statisticians to study of generalized linear models with measurement error.
the field were provided by Madansky (1959). Sprent (1989) Greater variety in the types of error model (the model re-
provided a more recent overview of the field, including ref- lating the observed W and unobserved X)studied also char-
erences to other key review articles. acterizes the recent research in measurement error model-
The traditional distinction in the literature between func- ing. The so-called Berkson error model, wherein the ob-
tional models and structural models is, for some applica- served values W are fixed in repeated sampling and the X
tions, not as relevant as the distinction between functional values vary (Berkson 1950), which is little studied in linear
modeling and structural modeling as defined by Carroll et models because of the lack of rectifiable problems that it
al. (1995, sec. 1.2), which facilitates classification of estima- causes, plays a more significant role in the study of non-
tion methods based on the strength of the assumptions made linear measurement error models. The distinction between
about the latent variables X I ,Xa, . . .. However, functional Berkson error and classical error models is subtle but im-
models played an important role in the study of measure- portant because of the very different problems associated
ment error models and in statistics more generally. with each (Carroll et al. 1995, sec. 1.3; Fuller 1987, sec.
In a functional errors-in-variables model, the unobserved 1.6.4). A frequently cited example due to Fuller (1987, sec.
latent variables are unknown parameters. Consequently, for 1.6.4) illustrates the key feature of Berkson error. Consider
a functional model with sample size n, and hence measure- an experiment in which the quality of cement, Y, is to be
ments on n latent variables, the unknown parameter vector studied as a function of the amount of water in the mix-
includes ( X I ,X z , . . . , X n ) and so has dimension increas- ture. The amount of water is controlled by setting a me-
ing linearly with sample size. Functional errors-in-variables tered valve to specified values. Because of fluctuations in
models are practical and classical examples of models with water pressure and inaccuracies in the metered valve, the
 1356 Journal of the American Statistical Association, December 2000

true amount of water in a mixture X , differs from the pre- that the error variance is known, or that it is independently
scribed amount W . Note that W is controlled as part of estimable, say from replicate measurements or validation
the experimental design and is not random. In replication data. The model is identified more generally, but in this
of the experiment at the same value of W , the true amount case identifiability does not imply the ability to obtain esti-
X will vary. If the valve is correctly calibrated, then a rea- mators with acceptable finite-sample properties (Carroll et
sonable model is X = W + U , where U is a mean-0 error al. 1995, sec. 7.1.1). A number of estimation methods have
regardless of the value of W . Thus for unbiased Berkson been studied for this model and were described by Carroll
error, E ( X I W ) = W , whereas for unbiased classical error, et al. (1995). One such method, chosen because of its link
E(WIX) = x. to the work of Neyman and Scott (19481, is described next.
There are experimental and sampling designs/studies in Consider the functional version of the logistic measure-
which the pure forms of error models (classical and Berk- ment error model with errors, U,, that are normally dis-
son) arise naturally, and hence their importance in ap- tributed with known variance &. In this case, the density
plied and theoretical work. However, there are applications of ( y Z , W,) is
where neither error model is directly applicable (measuring
f Y W ( Y ,WlPO, P z , XZ)
a child's exposure to NO2 and measuring average expo-
sure to particulate matter are two such examples), although
either may be appropriate after transformation of the raw = { H ( a + @ X i > } Y { l - H(a+@xi)}'-y-q3
OlJ
("iUXi),
~

measurements (error calibration and regression calibration;

Carroll et al. 1995, sec. 1.3).An incorrect assumption about where $( ) is the standard normal density function.
The functional maximum likelihood has n + 2 parameters
Downloaded by [Osaka University] at 21:26 19 November 2014

the error model has the potential of causing problems as

great as those created by ignoring measurement error com- X 1 , . . . , X,, a , P, and its maximization does not produce
pletely. Thus correct error model identification is crucial to consistent estimators of the logistic regression parameters
the successful use of measurement error models. (Stefanski and Carroll 1985).
However, examination of the foregoing density reveals
4.1 A Logistic Regression Measurement Error Model
that the parameter-dependent statistic Ai = W i+ x,a;/3 is
sufficient for the unknown X i in the sense that the condi-

models. or mean-variance function models (based on the conditional

The model discussed herein is not unlike that used by moments of yZ given A,) and quasi-likelihoodmethods. Es-
MacMahon et al. (1990) to investigate attenuation due to timating equations derived in these ways are called condi-
measurement in their study of blood pressure, stroke, and tional scores, and the corresponding estimators are called
coronary heart disease. MacMahon et al. (1990) described conditional score estimators.
the practical consequences of failing to adjust for attenu- Conditioning on sufficient statistics is a common strategy
ation (which they refer to as regression-dilution bias) and for eliminating nuisance parameters, and it applies more
emphasized the need to account for measurement in the generally to measurement error models. The key features
analysis of their data. of the logistic model with normal measurement error are
Consider the logistic regression model for the depen- that both component models are exponential family densi-
dence of the binary response Y on the scalar predictor ties. Thus there are a number of other generalized linear
X in which Pr(Y = 1 l X ) = H ( a + PX), where H ( t ) = measurement error models to which conditioning approach
(1 + exp(-t)}-'. Given data (X,,yZ),i = 1 , . . . , T I , max- applies. Details and examples of the method for logistic
imum likelihood estimation requires numerical maximiza- regression and generalizations to other generalized linear
tion. Now suppose that X , is not observed, but the mea- models with measurement error have been provided by Car-
+
surement W, = X , U, with additive independent error is roll et al. (1995, sec. 6.4) and Stefanski and Carroll (1987).
observed. This section closes with some numerical results providing
Because the maximum likelihood estimates have no an empirical illustration of attenuation and the conditional
closed-form expression, the effect of substituting W, for score estimation method described earlier. The data used
X, in logistic regression is not easily determined. Although are a subset of the data from the Framingham Heart Study
it is generally true that the estimate of P is attenuated as and consist of two measurements (from two exams 2 years
in the case of linear regression, and by approximately the apart) of systolic blood pressure (SBP) and an indicator of
same factor, this is not always the case (Stefanski and Car- coronary heart disease for each of 1,615 individuals. Blood
roll 1985).The logistic model is typical in this regard in that pressure measurements are transformed so that the variables
assessing the effects of measurement error on estimates in used in the analyses are Y,WI, and W2, where Y is the bi-
nonlinear models is intrinsically more difficult than in linear nary indicator and W, is the natural logarithm of the mea-
models (Carroll et al. 1995, sec. 2.5). sured SBP from the ith exam, i = l, 2. The model assumed
Estimation in the logistic model with normally distributed is that Wl and W2 are iid normal replicate measurements
measurement error usually proceeds under the assumption of X , defined here as an individual's long-term, mean log-
 Theory and Methods of Statistics 1357

transformed SBP. In other words W i= X + Ui, i = 1,2, 5. FUTURE RESEARCH TRENDS

where Uiis normally distributed with mean 0 and vari- Carroll et al. (1995) described a number of viable meth-
ance o$.The error variance estimated from the replicates ods of inference in nonlinear measurement error models,
is 8$ = .006. Define W = (W1 + W2)/2 and note that W ranging from simple moment-based methods for reduc-
is the preferred measurement of X with an error variance ing measurement error bias to fully specified parametric-
of o$/2 (with estimate .003). model maximum likelihood, including semiparametric and
The three unbiased measurements of X (W1, W2,W )per- flexible-parametric maximum likelihood. Further method-
mit an empirical demonstration of attenuation due to mea- ological developments are inevitable in light of the increas-
surement error. The measurement error variances of Wl and ing variety of problems tackled under the heading of mea-
W2 are equal and are twice as large the measurement error surement error modeling.
variance of their average W . Thus it would be expected that However, the long-term viability of the field will be de-
the attenuation due to measurement error in the regressions termined by how measurement error models and methods
of Y on W1 and Y on W2are equal, whereas the regression are integrated into the sciences from which they evolved.
of Y on W should manifest less attenuation. The recent methodological developments form a founda-
The estimates of slope from the logistic regressions of Y tion for inference in measurement error models, and what is
on Wl and Y on W2 are 2.98 and 2.86, with an average of needed now are strategies for implementing these methods
2.92. The estimate of slope from the logistic regression of in the commonly encountered, practical situations in which
Y on W is 3.30. In the absence of appropriate standard er- model formulation is a necessary part of the data analy-
sis. Some relevant areas of research are model selection,
Downloaded by [Osaka University] at 21:26 19 November 2014

rors, it is unreasonable to claim that these estimates provide

proof of attenuation due to measurement error; however, the model robustness, and variable selection in the presence of
measurement error.
relationships among the estimates, 2.98 M 2.86 < 3.30, are
Structural measurement error modeling is appealing be-
consistent with what is known about measurement error and
cause of its simplicity. Once a model is specified, maximum
attenuation-the larger the measurement error, the greater
likelihood estimation is an obvious alternative. However,
the attenuation. full specification of a structural measurement error model
The conditional score estimate of slope using W as the requires a model for the response variable given X and
measurement of X and assuming that the error variance is other observable predictors, a model relating W and X pos-
known and equal to .003 (a reasonable assumption in light sibly depending on other observables, and a model for the
of the large degrees of freedom associated with this vari- distribution of X. Thus model selection and robustness in
ance estimate) is 3.75. Note that this estimate exceeds 3.30. measurement error models are multifaceted problems that
This is as expected, because the latter estimate is attenuated, are further complicated by the fact that parameter identifi-
whereas the conditional score estimate completely corrects ability is often achieved through some combination of sup-
(at least in theory) for the attenuation due to measurement plementary data and error model assumptions. In the worst
error. of cases in which neither replicate measurements nor val-
The reasonableness of the conditional score estimate can idation data (observational units on which both X and W
be argued using a simple jackknife-like bias adjustment. are observed) are available, model parameter estimates are
Note that for the measurements with error variance of .006, largely determined by assumptions about the type of error
the average estimated slope is 2.92 (correspondingto the av- model and magnitude of the error variance. In the best of
erage of the leave-one-out estimates in a typical jackknife cases in which internal validation data are available (Xand
analysis-in the present case, one of the two measurements W observed on a subsample of the main-study observational
is left out). For the measurement with error variance of units-in which case the problem can also be viewed as a
.003, the estimated slope is 3.30 (corresponding to the full- missing-data problem; Carroll et al. 1995, sec. 1.8) there are
data estimate in the usual jackknife setting-in the present data for estimating error-model parameters, but seldom are
case, no measurements are left out). A linear, jackknife-like the validation data sufficient for error-model selection.
The fundamental inference problems on which model
bias correction extrapolates this trend to the case of zero
building and variable selection procedures depend are more
measurement error resulting in the jackknife, bias-adjusted
complicated in measurement error models, especially with
estimate 2(3.30) - 2.92 = 3.68, whose closeness to the con-
multiple correlated predictors measured with error. Known
ditional score estimate (3.75) is not unexpected. results on the adverse effects of measurement error on hy-
The reason that the jackknife-like estimate works is be- pothesis tests (Carroll et al. 1995, sec. 11.3) indicate the
cause bias (attenuation)due to measurement error is a func- need to account for the effects of measurement error when
tion of the measurement error variance in much the same using (W,Y )to select variables or build models for (X, Y ) .
way that for nonlinear estimators in general, finite-sample In recent investigations of the health effects of particulate
bias is a function of inverse sample size. This fact is the matter (Lipfert and Wyzga 1995; Schwartz, Dockery, and
basis of a jackknife-like method for reducing measurement Neas 1996), wherein both coarse and fine particulates are
error bias that uses simulated measurement error as an alter- measured with error, measurement error has been invoked
native to deleting measurements (Carroll et al. 1995; Cook to explain patterns of statistical (non) significance in the
and Stefanski 1994; Stefanski and Cook 1995). analysis of the observed data (coarse and fine particulate
 1358 Journal of the American Statistical Association. December 2000

measured at central sites). The fact that measurement er- lihood Estimator in the Presence of Infinitelv Manv Incidental Parame-
ror affects variable selection and model building is not in ters,” Annals of Mathematical Statistics, 27, 887-906.
Lipfert, F. W., and Wyzga, R. E. (1995), “Uncertainties in Identifying Re-
dispute, although the severity of its effect in multivariable sponsible Pollutants in Observational Epidemiology Studies,” Inhalation
models is not known. Model building and variable selec- Toxicology, 7, 671489.
tion in the presence of measurement error is a potentially MacMahon, S., Peto, R., Cutler, J., Collins, R., Sorlie, P., Neaton, J., Ab-
interesting and useful topic of study. bott, R., Godwin, J., Dyer, A,, and Stamler, J. (1990), “Blood Pressure,
Stroke and Coronary Heart Disease: Part 1, Prolonged Differences in
Blood Pressure: Prospective Observational Studies Corrected for the
REFERENCES Regression Dilution Bias,” Lancet, 335, 765-774.
Adcock, R. J. (1877), “Note on the Method of Least Squares,” Analyst, 4, Madansky, A. (1959), “The Fitting of Straight Lines When Both Variables
183-184. are Subject to Error,” Journal of the American Statistical Association,
-(1878), “A Problem in Least Squares,” Analyst, 5, 53-54. 54, 173-205.
Berkson, J. (1950), “Are There Two Regressions?,” Journal ofthe Ameri- Neyman, J., and Scott, E. L. (1948), “Consistent Estimates Based on Par-
can Statistical Association, 45, 164-180. tially Consistent Observations,” Econometrica, 16, 1-32.
Carroll, R. J., and Ruppert, D. (1996), “The Use and Misuse of Orthog- Schwartz, J., Dockery, D. W., and Neas, L. M. (1996), “Is Daily Mortality
onal Regression in Linear Errors-in-Variables Models,” The American Associated Specifically With Fine Particles?,” Journal of the Air and
Statistician, 50, 1 4 . Waste Management Association, 46, 927-939.
Carroll, R. J., Ruppert, D., and Stefanski, L. A. (1995), Measurement Error Sprent, P. (1989), “Some History of Functional and Structural Relation-
in Nonlinear Models, London: Chapman and Hall. ships,” in Statistical Analysis of Measurement Error Models and Appli-
Carroll, R. J., Spiegelman, C., Lan, K. K., Bailey, K. T., and Abbott, cations, eds. P. J. Brown and W. A. Fuller, Providence, RI: American
R. D. (1984), “On Errors-in-Variables for Binary Regression Models,” Mathematical Society.
Biometrika, 71, 19-26. Stefanski, L. A,, and Carroll, R. J. (1985), “Covariate Measurement Error
Cook, J., and Stefanski, L. A. (1994), “A Simulation Extrapolation Method in Logistic Regression,” The Annals of Statistics, 13, 1335-1351.
Downloaded by [Osaka University] at 21:26 19 November 2014

for Parametric Measurement Error Models,” Journal of the American (1987), “Conditional Scores and Optimal Scores in Generalized
Statistical Association, 89, 1314-1328. Linear Measurement Error Models,” Biometrika, 74, 703-716.
Eddington, A. S. (1913), “On a Formula for Correcting Statistics for the Stefanski, L. A., and Cook, J. (1995), “Simulation Extrapolation: The Mea-
Effects of a Known Probable Error of Observation,” Mon. Not. R. As- surement Error Jackknife,” Journal of the American Statistical Associ-
trom. Soc., 73, 359-360. ation, 90, 1247-1256.
Fuller, W. A. (1987), Measurement Error Models, New York: Wiley. Trumpler, R. J., and Weaver, H. F. (19531, Statistical Astronomy, Berkeley,
Kiefer, J., and Wolfowitz, J. (1956), “Consistency of the Maximum Like- CA. University of California Press.

Higher-Order Asymptotic Approximation: Laplace,

Saddlepoint, and Related Methods
Robert L. STRAWDERMAN

1. INTRODUCTION ing integrals known as Laplace’s method has been a popular

Approximation is ubiquitous in both statistical theory and tool in recent work in Bayesian inference and random- and
mixed-effects models.
practice. Many approximations routinely used in statistics
In the last 20 years or so, the statistical literature on
can be derived by approximating certain integrals and are
higher-order asymptotic approximation has grown signifi-
asymptotic in nature. That is, such approximations are typ-
cantly. As this article is not meant as a comprehensive re-
ically developed under the assumption that some quantity view of the area, many useful and interesting contributions
connected to the amount of information available (e.g., sam- have necessarily been left out. For example, the vast liter-
ple size) becomes infinitely large. In the case of approximat- ature on Edgeworth expansion is essentially ignored. The
ing density and distribution functions, one example here is significant recent growth has not been limited to applica-
“local linearization” tie., the delta method combined with tions of Laplace’s method and saddlepoint approximation.
the central limit theorem). This method of approximation However, much of the work reviewed in Section 3 is con-
has a long history, with roots that can be traced back to the nected to these basic methods of integral approximation at
time of Laplace and Gauss (i.e., early 1800s).In general, lo- some level, and these works have arguably had the most
cal linearization leads to approximations that are accurate significant influence on development in the area of higher-
to “first order.” Methods of “higher-order’’ asymptotic ap- order asymptotic approximation to date. A parallel motiva-
proximation, in principle more accurate than those of first tion for writing this article, emphasized primarily in Sec-
order, include Edgeworth expansion, tilted Edgeworth ex- tion 2, is that Laplace and saddlepoint approximations exist
pansion, and saddlepoint approximations for density and independently of statistics and have substantially broader
distribution functions. A related technique for approximat- applicability than is often recognized.

Robert L. Strawderman is Associate Professor, Department of Bio- @ 2000 American Statistical Association
metrics, Cornell University, 434 Warren Hall, Ithaca NY 14853 (E-mail: Journal of the American Statistical Association
rls54 @ cornell.edu). December 2000, Vol. 95, No. 452, Vignettes