Quality Engineering

ISSN: 0898-2112 (Print) 1532-4222 (Online) Journal homepage: http://www.tandfonline.com/loi/lqen20

Industrial statistics: The challenges and the

research

David M. Steinberg

To cite this article: David M. Steinberg (2016) Industrial statistics: The challenges and the
research, Quality Engineering, 28:1, 45-59, DOI: 10.1080/08982112.2015.1100453

To link to this article: http://dx.doi.org/10.1080/08982112.2015.1100453

Published online: 29 Jan 2016.

Submit your article to this journal

Article views: 45

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=lqen20

Download by: [University of Arizona] Date: 23 February 2016, At: 14:59

 QUALITY ENGINEERING
, VOL. , NO. , –
http://dx.doi.org/./..

Industrial statistics: The challenges and the research

David M. Steinberg
Department of Statistics and Operations Research, Tel Aviv University, Tel Aviv, Israel

ABSTRACT KEYWORDS
Industrial problems have stimulated an enormous amount of valuable statistical research, from the statistical practice; design of
t-test to advanced statistical tools for quality. Industry continues to generate challenging problems experiments; SPC; reliability;
for statistical design, modeling, and analysis. Useful articles are published in our journals, often stim- data science
ulated by industrial applications. Nonetheless, there is concern that research in industrial statistics is
falling well short of its potential for providing interesting problems, that some of the most exciting
problems are not getting space in our journals, and that few statisticians working in industry are pub-
lishing research. This article endeavors to map out the current state of research in industrial statistics,
Downloaded by [University of Arizona] at 14:59 23 February 2016

to describe major issues that need to be addressed, and to discuss whether the research is on target
to meet those challenges.

Historical perspective r Exploratory data analysis was stimulated by prob-

lems John Tukey encountered at Bell Laboratories.
There is good reason to expect a symbiotic link between r Response surface methodology resulted from
industry and statistical research. Industrial settings
George Box’s collaborations with chemists at
generate many challenging statistical problems. And as
Imperial Chemical Industries.
Box (1984) elegantly argued, real problems are often
the catalyst for innovative research. The accomplishments listed above, which span the
Many important statistical ideas and methods have first seven decades of the last century, are evidence of
their roots in industrial problems. To name just a few the impact that industrial problems have had on statis-
examples: tical research. Stu Hunter (1983) described the stim-
r The t-test grew out of Gosset’s desire to analyze ulating environment of statistics in the early 1950s:
small samples of data at the Guinness Brewery. “For someone interested in the applications of statis-
r The rank-sum test was developed by Frank tics in the sciences and engineering the excitement of
Wilcoxon from his need for distribution-free the period went far beyond the usual industrial applica-
methods to analyze data at American Cyanimid. tions of control charts and acceptance sampling plans.
r Statistical process control was Walter Shewhart’s … The desire to exchange ideas and experiences among
response to monitoring and improving produc- the industrial statisticians was intense” (pp. 3–4). Some
tion at Bell Telephone Company. of those statisticians were convinced that existing jour-
r The need for efficient munitions testing in World nals did not provide a satisfactory forum for their ideas
War II led Abraham Wald to invent the sequential and methods. This energy came to the fore at a meet-
probability ratio test. ing early in 1958. Hunter (1983, p. 4) quoted from
r The management philosophy of W. Edwards the notes of that meeting: “A new journal should be
Deming, and the role of statistics in management started to present applications of statistics in the physi-
and quality, arose from his close association with cal and engineering sciences.” The American Statisti-
industry. cal Association (ASA) and the American Society for
r Ridge regression addressed problems of correlated Quality Control agreed to jointly sponsor the journal
predictors faced by Arthur Hoerl and Robert Ken- and Technometrics was established, with Hunter as its
nard at Du Pont. editor.

CONTACT David M. Steinberg [email protected] Department of Statistics and Operations Research, Tel Aviv University, Tel Aviv , Israel.
Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/lqen.
©  Taylor & Francis
 46 D. M. STEINBERG

Prior to launching Technometrics, the ASA pub-

lished a single technical journal—the Journal of the
American Statistical Association. This initial decision
by the ASA to sponsor a more focused research jour-
nal was directed toward industrial and engineering
problems, providing further evidence of the impact of
industrial problems on statistical research.
The inaugural issue of Technometrics was published
in February 1959. Of 23 articles published in the first
volume, only 9 included authors from academia. The
remaining articles were written by researchers in indus-
try or at national laboratories. Those authors came
from a variety of sectors, including the chemical indus- Figure . Manufacturing and service jobs in the U.S. economy
try, telecommunications, and the National Bureau of (from the U.S. Bureau of Labor Statistics web site).
Standards.
Several subject areas dominated the initial research
Downloaded by [University of Arizona] at 14:59 23 February 2016

“a detailed study of a subject, especially in order to dis-

on industrial statistics: how to control and improve
cover (new) information or reach a (new) understand-
quality, accelerating development with designed exper-
ing” (http://dictionary.cambridge.org/dictionary/engl
iments, etc. The research proposed and refined many
ish/research). Thus, research includes new theory, new
of the tools that are now deeply ingrained in Six
methods, assessment of methods (including assess-
Sigma’s define–measure–analyze–improve–control
ment of their value for practice), and creative appli-
cycle: design of experiments (DOE), reliability and
cation of existing methods (for example, to new
metrology as fundamental to measurement and analy-
problems). It also embraces theoretical constructs for
sis, and statistical process control (SPC) and acceptance
enhancing thinking and application. Articles that illus-
sampling for control of processes and output.
trate existing methods are of great value to the profes-
sion as case studies, but if they do not include any inno-
Aims and scope vative ideas, I will not include them as research.

My goals in this article are to summarize the cur-

rent state of research in industrial statistics, to describe
Industry
some of the problems facing industrial statisticians that
need more research, and to comment on whether the More specifically, my focus is on research in “industrial
research is effectively aligned with the challenges. Four statistics.” The Cambridge Dictionary defines “indus-
basic questions will guide the presentation: try” as “the companies and activities involved in the
1. What is driving statistical research today? process of producing goods for sale, especially in a fac-
2. Are industrial challenges affecting the research tory or special area” (http://dictionary.cambridge.org/
agenda in statistics? dictionary/english/industry). So “industrial statistics”
3. Are statistical challenges in industry being bet- refers to the use of statistics to promote the develop-
ter addressed by other research communities? ment, testing, production, and marketing of goods and
4. Is the statistics research community missing the processes that support these efforts. I will take a lib-
important opportunities for impact in industry? eral view of what is industry so that “goods” will include
Some preliminary comments are in order to stake the production of services as well as physical items. My
out the playing field and impose some ground rules. choice reflects the marked shift of Western economies
from manufacturing to services during the last 30 years.
Figure 1, from the U.S. Bureau of Labor Statistics web
Research
site, shows the number of U.S. jobs in manufacturing
My focus is on statistical research, not on application of and services from 1970 through 2011. The ratio of ser-
statistical methods and thinking in industry. The Cam- vice to manufacturing jobs was about 2.5 to 1 in 1970
bridge Dictionary (online version) defines “research” as and since then has steadily increased, to about 7 to
 QUALITY ENGINEERING 47

1 in 2011. Given the dominant economic role of ser- expertise. Many companies still have strong statistical
vice production, it seems clear that this sector must be groups and some (especially in information technology
included in any discussion of industrial statistics. [IT] and Internet companies) have had strong growth,
but others have been downsized or eliminated.
Caveat
Big data, data science and industry
I will exclude two industrial sectors from this article:
biostatistics (even if the research was done within the “Big data” and “data science” are more than just buzz-
pharmaceutical industry) and financial engineering. words. Business, industry, science, and technology have
Although both are areas that currently provide many an unprecedented volume, velocity, and variety of data
jobs in the private sector, they are quite distinct in terms at their disposal and a matching need for skilled indi-
of research from what I regard as industrial statistics. viduals who can help them convert that data into com-
Moreover, both have their own publishing outlets and petitive advantage. The data deluge has led to many
they rarely publish in journals with an industrial statis- articles in the popular press (for example, “The Age of
tics focus. A closer look at the pharmaceutical industry Big Data” in the New York Times, February 11, 2012,
may help to make the distinction more precise: statis-
Downloaded by [University of Arizona] at 14:59 23 February 2016

or “Data Crunchers Now the Cool Kids on Campus” in

tical research on the design and analysis of Phase III the Wall Street Journal, March 2013), and it prompted a
clinical trials is excluded; statistical research related to special issue of the leading journal Science in February
identifying treatments and improving processes clearly 2011.
is within the scope of the article. The McKinsey Global Institute released a widely
cited report in June 2011 entitled Big Data: The Next
Frontier for Innovation, Competition and Productivity
Industrial statistics in 2015
(Manyika et al. 2011). The McKinsey report “suggests
The use of statistics in industry is widespread. Qual- that we are on the cusp of a tremendous wave of inno-
ity improvement programs, Six Sigma, etc., have been vation, productivity, and growth, as well as new modes
important contributing factors. The increasing avail- of competition and value capture—all driven by big
ability of data is also generating a need for statistics (see data as consumers, companies, and economic sectors
next section). The situation was aptly summarized by exploit its potential” (p. 2). The report points to strong
Gerry Hahn and Necip Doganaksoy in the May 2008 evidence of the major economic impact of big data and
Technometrics panel discussion on the future of indus- predicts that big data will make major contributions
trial statistics: “We live in a golden age of industrial to the world economy. To realize that potential, com-
statistics” (Steinberg 2008, p. 104), a sentiment shared panies will need to hire staff with an increasing level
by most of the other panelists. of sophistication in handling, analyzing, and making
As a counterpoint to the “golden age,” panelists also decisions based on data. The report cautions that “a sig-
noted that statisticians are not always involved. Sallie nificant constraint … will be a shortage of … people
Keller-McNulty stated that “the current status of statis- with deep expertise in statistics and machine learning
tics in industry is strong; however, the status of statis- … a talent gap of 140K–190K positions in 2018 [in the
ticians in industry is possibly at an all-time low. Statis- United States]” (p. 10).
tics has become an integral part of all of business and The McKinsey report describes statistics as one
industry, but statisticians have not. … Over the last 10– of the basic tools required to convert data into use-
20 years, we have seen a serious decline of major sta- ful information. That view is echoed by Provost and
tistical forces in industry” (p. 105). One reason is the Fawcett (2013), two of the leading figures in the
positive trend to democratization: easy-to-use software “big data” community: “With vast amounts of data
has enabled engineers, scientists, and others to handle now available, companies in almost every industry are
routine statistical work on their own. Another is that focused on exploiting data for competitive advantage.
individuals trained in other domains, including com- … We highlight data science as the connective tissue
puter science, informatics, and industrial engineering, between data-processing technologies (including those
may be the first to address problems that involve data for ‘big data’) and data-driven decision making. … A
and often end up being the primary source of statistical large portion of what has traditionally been studied
 48 D. M. STEINBERG

within the field of statistics is fundamental to data sci- provide is well aligned with their needs. Such a caustic
ence” (p. 51). tone in a professional editorial should set off warning
The London Workshop on the Future of Statistics bells.
devoted one chapter in its report to “big data.” The As I have already emphasized, the goal of this article
report noted a spectrum of opinion about big data, with is to examine the state of research in industrial statis-
some arguing that fundamental changes are needed to tics. I do so from a viewpoint that statistics is funda-
equip statisticians to work effectively today and oth- mental to converting data into knowledge, in line with
ers claiming that it is little more than a fad. In indus- the statement from Provost and Fawcett. As such, the
try, many major companies are recruiting data scien- widespread interest in big data and its potentially criti-
tists for their technical workforces. The term remains cal role in business and industry point to a ripe area for
somewhat amorphous, and although some knowledge statistical research of great relevance to industry.
of statistics is generally required, other skill sets and
training are often equally or more important in screen-
ing applicants. The focus on data science in industry, to Industrial statistics research in 2015
date, has only increased the extent of industrial statis-
In this section I will summarize the current research
tics carried out by people with limited training in statis-
Downloaded by [University of Arizona] at 14:59 23 February 2016

in industrial statistics. The summary is based pri-

tics.
marily on a survey of articles published in statistics
Jordan and Lin (2014) provided an insightful view
journals in 2014 with a pronounced interest in and
of big data and statistics. One of their primary points
a solid track record of relevance to industrial prac-
was that statisticians have often ignored the challenges
tice. These include Technometrics, Journal of Quality
of big data and have been largely absent from lead-
Technology, Quality and Reliability Engineering Interna-
ership roles. They write that statisticians have a great
tional, Applied Stochastic Models in Business and Indus-
deal to contribute to the utilization of big data. How-
try, Quality Technology and Quantitative Management,
ever, they add that this will require openness to address
and, of course, Quality Engineering. The above journals
large, unstructured problems. Like Keller-McNulty,
span a range from more theoretical (e.g., Technomet-
they point to statisticians being “on the outside look-
rics) to very applied and pedagogical (e.g., Quality Engi-
ing in” (Jordan and Lin 2014, p. 59). One of the telling
neering). I will also report on some engineering jour-
pieces of evidence noted in their article—when the U.S.
nals that encourage and publish statistical research of
National Science Foundation assembled a panel of 100
relevance to industry, like IIE Transactions and IEEE
experts in 2012 to formulate policy related to big data,
Transactions on Reliability, some general statistics jour-
not a single statistician was included.
nals that emphasize applications, and some venues with
The perception of statistics as less than relevant to
a strong industrial foothold.
big data is not limited to the industrial setting. Mal-
Many subject-matter journals in a variety of fields
ley and Moore (2013), the coeditors of the journal Bio-
also publish articles with interesting applications of
Data Mining, published an editorial entitled “The dis-
statistics in industry. A full survey of such periodi-
connect between classical biostatistics and the biolog-
cals is beyond the scope of this article. Moreover, most
ical data mining community.” They write that “statis-
of those articles illustrate known statistical methods
tics departments and journals still strongly emphasize
rather than developing new ones. How to highlight
a very narrow range of topics and methods and tech-
valuable articles that do breach new ground within the
niques, all driven by a tiny handful of results, many
statistics research community is a challenge that we
dating from the 1930s … the older zombie methods
should address.
persist in the statistics literature and teaching.” Malley
and Moore’s editorial is extreme. Many of their claims
are exaggerated and others, in my opinion, are wrong
(e.g., the implication that we should stop teaching What is being published?
regression and go straight to neural nets). But we also The research published in industrial statistics journals
need to take a closer look at the problems faced by continues to be dominated by the same topics that had
our colleagues (whether in biology or industry) and leading roles in the 1960s: DOE, SPC, and reliability.
to make sure that what we teach and the expertise we Statistical modeling is also a common theme, with the
 QUALITY ENGINEERING 49

nature of the model often simulated by specific applica- two-dimensional surface. Their work was motivated
tions. Within these primary areas, many new problems by the need to monitor the thickness of silicon wafers
are being studied—for example, designing experiments in a semiconductor fab. The proposed control chart
for computer simulators, SPC with complex data, and combines ideas from spatial statistics to model the
reliability for products with smart chips that provide a thickness data with methods from profile charting.
vast new array of field data. Vander Wiel et al. (2014) developed an approach for
My literature review found that interesting applica- monitoring an electric power grid using data from pha-
tions are driving an increasing number of published sor measurement units, a technological breakthrough
articles. However, few of these problems are from for collecting online data from the grid. These data
industry, and not many industrial statisticians are pub- can be used to detect problems in the grid, such as a
lishing research. See Hoerl (2000) for earlier comments line going down or a large load being dropped. The
on the need to enhance contact between industrial approach proposed here provides an estimate of the
practitioners and the research community. Table 1 gives current topology of a power grid and an algorithm for
a summary of published articles in 2014 for a number of the design problem of where to place the phasor mea-
journals. For each, I computed the number of industrial surement units in a grid. The estimation procedure
authors and the fraction of articles that were stimulated requires an understanding of the data signals that char-
Downloaded by [University of Arizona] at 14:59 23 February 2016

by industrial problems. (Two technical notes: (1) I clas- acterize problems that occur in the grid.
sified employees of national labs as “industrial” and (2)
I gave half credit to some articles that were “one step
removed” from a generating problem.) Design of experiments
The main stimulus to research is, not surprisingly, The two dominant topics in DOE in recent years have
other research. When we read articles, we often think been screening experiments and the design of com-
of an alternative, and perhaps better, way to attack the puter experiments.
problem, or we see that the ideas can be extended to Jones and Nachtsheim (2011) proposed a clever
address other questions. Thus, “threads” of research method for screening continuous input factors that
grow. Ideally (but not always), they retain roots to the gives the design sensitivity to pure linear effects and
applied concerns that led to the original research or pure quadratic effects. The linear effects are not biased
extend their applicability to additional problems. by any second-order effects and a full second-order
In the following subsections, I will give a brief sum- model can be fitted in any subset of three factors. The
mary of recent research in industrial statistics. In each appeal of these designs is that they achieve these wider
area, I will highlight several articles that I found partic- screening properties with only a few more runs than
ularly innovative and relevant to industry. would be required by a two-level screening design,
which has no ability at all to detect pure quadratic
effects. Subsequent work by Xiao, Lin, and Bai (2012)
Statistical process control
and Yang, Lin, and Liu (2014) showed that the designs
Recent work on SPC and monitoring has been stim- are related to the concept of conference matrices, which
ulated by new and interesting types of data that are provide a useful base for construction. See the article by
collected. Whereas classic SPC charts track a single Jones in this volume for up-to-date ideas on “definitive
characteristic (a mean value or a defect rate), today’s screening designs.”
quality data are often multivariate and have important Computer experiments involve special considera-
structural content. The monitoring of profiles is a good tions because they typically have no experimental
example. Woodall et al. (2004) coined the term and dis- error, so that blocking, randomization, and replica-
cussed an application in which the sequence of data tion become irrelevant. Most applications have rejected
from each object digitized its geometric shape. There standard factorial designs, preferring “space-filling”
has been much additional research in recent years, with designs that spread out the design points in a more-
extensions to general settings in which data describe a or-less uniform fashion. Joseph et al. (2015) proposed
relationship that characterizes an item. the concept of “minimum energy designs”; designs in
Wang, Wang, and Tsung (2014) described a this interesting and useful class spread out experimen-
method for monitoring a process that produces a tal sites in the design region and are also able to adapt
 50 D. M. STEINBERG

Table . Industrial collaboration in and direct stimulation of articles published in .

Authors Articles
Journal Total Industrial % Total Industrial %

Annals of Applied Statistics      

Applied Stochastic Models in Business and Industry      
IEEE Transactions on Reliability     . 
IIE Transactions      
Journal of the American Statistical Association: Case Studies and      
Applications
Journal of Quality Technology      
Journal of Machine Learning Research      
Quality Engineering     . 
Quality and Reliability Engineering International     . 
Statistical Analysis and Data Mining      
Technometrics      

Authors from national laboratories are included here as “industrial”authors. The count of “industrial articles”gave a full count for an article directly stimulated by an
industrial problem and a half count for a problem closely, but indirectly, stimulated by an industrial problem. The summary for Applied Stochastic Models in Business
and Industry does not include the special issue for George Box.The summary for IIE Transactions does not include special issues on scheduling and logistics, design
and manufacturing, public health, and health care surveillance; those issues had articles with minimal statistical content.
Downloaded by [University of Arizona] at 14:59 23 February 2016

to settings where some subregions prove more interest- failures. Hong and Meeker showed how to exploit those
ing than others. See the article by Joseph in this volume data to obtain substantially more accurate estimates of
for more details. warranty costs. See Meeker and Hong (2014) for a sum-
Of special note is the article by Albrecht et al. (2013), mary of other applications where new data collection
which grew out of an industrial application and led the methods have paved the way to improved reliability
authors to new and fertile ground. The article proposed analysis.
the use of dimensional analysis in the design of experi- Application of Bayesian methods is effectively
ments. The authors showed how exploiting dimension- illustrated by Alzbutas and Iešmantas (2014). Their
less quantities could lead to identification of the funda- research concerned how to incorporate the age and
mental factors driving a process. This led to analyses degradation status of components and systems in reli-
with a wider range of generalizability. Moreover, as the ability analysis; it was motivated by the study of elec-
fundamental factors are composites of the original fac- trical instrumentation and control systems in nuclear
tors, the dimensionality of the experimental space can plants. They discussed aspects of model building and
be reduced, with gains in cost and efficiency of exper- verification and the use of Bayesian model aver-
imentation. For some months preceding the writing of aging for making predictions. The method is also
this article, Albrecht et al. (2013) topped the “down- able to handle high-reliability components with few
load” list from Technometrics. It led to a useful sequel failures.
by Shen et al. (2014) illustrating the ideas on additional Zhu, Yashchin, and Hosking (2014) described inter-
examples. esting methodology that they developed to assess the
reliability of a municipal water system. Data on equip-
Reliability ment failures were available but only for a particu-
lar time window; failures prior to that time simply
A number of themes have been at the forefront of recent were not recorded. The resulting “window-censored”
research in reliability. These include life tests, degrada- data require special treatment for statistical analysis
tion models, use of field data, and Bayesian methods. and reliability predictions. In this article, the authors
The use of field data, and in particular the availabil- develop a useful model, apply it to reliability estima-
ity of field data, was the stimulus for work by Hong and tion, and study its properties.
Meeker (2010). The goal they addressed was to fore-
cast eventual warranty costs for a product. An impor-
tant predictor of warranty costs is actual product use,
Modeling
but traditionally this was known only for units that had
failed and been returned for repair. Modern sensor and “Modeling” refers to quite a diverse set of articles
smart chip technology now provide use levels for many where the primary innovation is how to represent a
units that were shipped to customers and never had any particular type of data. General areas covered include
 QUALITY ENGINEERING 51

Gaussian processes, dimensional analysis, computer Prediction sometimes was addressed in the context
experiments, spatiotemporal data fusion, unusual lon- of articles on regression (e.g., Mallows 1973) or time
gitudinal profiles, screening in drug discovery, online series (e.g., Box and Jenkins 1968) but was not a fea-
virtual market data, variable selection, uncertainty tured topic; classification was largely ignored. (For
quantification, and count data. Many more specific top- example, in the first 10 volumes of Technometrics, there
ics were studied in 2014: binary yield data, nanoparticle were only three articles on classification.) In recent
growth patterns and particle clustering, infrared trans- years there has been increased interest in these topics.
missivity data, transportation matrices, exposure to Prediction has been a major target for analytics and
health threats, traffic volume from mobile phone mon- classification has been at the forefront of research in
itoring, monitoring a nuclear accident, retail demand, data mining and machine learning.
price of a good, and return on assets. Almost all of this Recommender systems are algorithms that use data
research is driven, directly or indirectly, by applications to help match individuals to products, services, or other
that required the development of new tools. people. As an example, when I order a book from Ama-
Motivated by a problem in textural analysis of fab- zon.com, the web site “recommends” other books that
ric samples, Taylor, Eckley, and Nunes (2014) studied might interest me. Shi et al. (2014) address the prob-
a model for two-dimensional lattice processes based lem of constructing a data-based recommender sys-
Downloaded by [University of Arizona] at 14:59 23 February 2016

on a locally stationary wavelet process. Texture is both tem that requires fusing data from a variety of different
location-dependent and multiscale, making wavelets sources and coping with a high fraction of missing data.
a natural modeling choice. Of particular concern was Their work was motivated by the problem of construct-
whether a stationary model would be suitable for the ing social network graphs from a large demographic
application and a main contribution of the article is a database at AT&T in which there is a high fraction of
test of stationarity. missing data. The database is distributed and leads to
Chuang and Oliva (2014) developed a model for 90 different network graphs. The algorithm proposed
demand in a retail chain. Estimation of retail demand in the article was able to fuse the information from the
is critical to decisions about procuring, shipping, shelv- separate graphs and to effectively “fill in” the missing
ing, and inventory control. Their research was moti- data and predict which individuals are connected.
vated by fieldwork to improve sales performance in a Open markets have made the generation and sale
U.S. retail chain. A key concern was to estimate the of electricity a challenging business enterprise. Short-
probability that demand would outstrip inventory. The term forecasts of electricity demand are an important
authors noted that prior work modeled demand via the input when producers make production and pricing
Poisson distribution. Their data showed much greater decisions. Carpio, Juan, and López (2014) attacked the
dispersion, which led them to investigate alternative forecasting problem with a multivariate exponential
models for count data. smoothing scheme that exploits dimension reduction.
A major goal in semiconductor fabrication is to They illustrated the usefulness of the model by com-
improve yield—the fraction of manufactured chips that paring daily forecasts to actual demand in several day-
function successfully. Hwang and Lee (2014) addressed ahead markets.
the problem of how to set specification limits for
critical chip dimensions for high-yield production.
They explained why naïve modeling of yield versus Industrial research challenges
critical dimensions could give incorrect answers and What are the major challenges arising in industry
proposed a novel method in which yield probabil- today that require statistical research? The landscape is
ities also involve hidden (or latent) variables. The varied and so are the issues that we face. Some are
approach provides accurate target and specification technical and others are organizational. I cannot claim
limits. any special expertise in mapping out the challenges
faced by industry. So I polled a number of industrial
colleagues, representing diverse areas including micro-
Prediction and classification
processors, consumer products, IT, pharma, aerospace,
Traditionally the industrial statistics literature has not and defense. Most of what follows reflects the feedback
featured many articles on prediction or classification. I received from them.
 52 D. M. STEINBERG

Table . Technical challenges for modern industrial statistics.

Modeling: General Issues Prediction and predictive Model selection Model averaging Fully Bayesian Modeling “real” data
analytics analysis
Modeling: Big data and Analysis of large data sets Analysis of streaming data Making the most of small
small data samples
Modeling: Specific issues Computer models— Machine learning and Image analysis Clustering Data with complex error
tuning and validation data mining structure
DOE Computer experiments Fast, high-volume Bayesian methods Choice experiments Experiments with a
experiments nonconstant
background
Data science/IT Assessing data quality Quantifying uncertainty Anomaly detection Text mining
SPC Profile monitoring Image monitoring
Reliability Multilayer, multiscale Life tests with multiple
surveillance accelerating factors
Software Bayesian methods

I begin with several important nontechnical issues The list of topics is largely self-explanatory, but a few
that my colleagues cited as barriers to widespread appli- comments are in order to give perspective. The sub-
cation of statistical methods. They included issues of stantial interest in predictive analytics in recent years is
teaching and training for nonstatisticians, better soft- at the top of the list for general issues in modeling and
Downloaded by [University of Arizona] at 14:59 23 February 2016

ware (and specifically software for Bayesian analysis), is also reflected in additional issues like model selection
more involvement in decision making, avoiding deci- and model averaging.
sions based on limited data, “upgrading” statisticians The call for modeling “real data” reflects a con-
from consultant to collaborator, statistical input to cern that too many research papers continue to rely
industry manuals, and involvement in large, complex on assumptions that just do not characterize the data
problems. encountered in industry. This view was expressed
Some of the concerns above have close ties to inno- starkly by Bert Gunter in the 2008 panel discussion
vation, an essential component in industrial success in Technometrics on the future of industrial statistics
but an area where statistics has not been a major part (Steinberg 2008, p. 125):
of the dialog. Søren Bisgaard forcefully argued in his
I fear that Technometrics has evolved from primarily mak-
2005 Youden address (Bisgaard 2012) that quality man- ing connections to the real, hard, and complex questions
agement and methods should be regarded as provid- of scientific practice to primarily producing artificial for-
ing the framework and tools for systematic innovation. mulations of those questions suitable for compact “solu-
Box and Woodall (2012) provided further commen- tion” by mathematical characterization. … To under-
tary on the interplay of statistics, quality, and innova- stand what is useful and not merely wrong in industrial
statistical practice, we need to pay much more attention
tion, emphasizing lateral thinking (de Bono 1970) and
to the messy details that make up reality.
experimentation. Many constructive ideas were raised
in a panel discussion in Quality Engineering (Jensen I don’t share Gunter’s opinion that most of our
2012). The Harvard Business Review highlighted inno- published research (whether in Technometrics or other
vation in December 2014; the article by Thomke and journals) has become completely cut off from real prob-
Manzi (2014) in that issue is completely devoted to the lems. But I do share the concern that many of the most
use of statistical experiments for assessment of busi- challenging and exciting problems arising today are not
ness innovations. More research by statisticians on how getting space in our journals and that we need better
our unique understanding and skills support innova- theory to guide us in attacking such problems.
tion would be helpful to move this area forward. Applications in IT companies involve many simple
Now let me move on to technical challenges. Table 2 comparative experiments. Known as A/B tests, these
presents a list of topics where my colleagues would like experiments compare alternative landing pages at
to see more research. I have grouped them into six gen- Internet sites (see Scott [2015] and the ensuing dis-
eral categories. Traditional areas like DOE, SPC, and cussion by Agarwal [2015] for details). What is special
reliability continue to have high priority. Modeling is about these experiments is that they involve very large
also a major concern and relates to a wide variety of sample sizes, they are carried out sequentially, it is diffi-
problems; I have divided the modeling topics into three cult to know what characteristics might need blocking
groups. (hence the “nonconstant background”), and it is
 QUALITY ENGINEERING 53

desirable to let the data direct new subjects to more comments (e.g., involvement in large, complex prob-
successful landing pages during the course of the lems) clearly intersect with SE.
experiment.
The reference to “multi-layer, multi-scale surveil- Statistics and the service sector
lance” in reliability relates to the availability of comple-
I stressed early in this article the importance of includ-
mentary data sources that can be used for assessment.
ing the service sector as a component of industrial
The article by Hong and Meeker (2010) described ear-
statistics. In my survey of recent research and the “wish
lier is a good example of how to use such data.
list” in Table 2, the service sector has a very limited role.
Rodriguez (2000) pointed to the great opportunities for
SPC in working with service sector data. Yet research
Statistical engineering in industrial statistics remains focused on manufac-
turing, despite the dramatic changes that have taken
Hoerl and Snee (2010a, 2010b, 2012) adopted the term
place in the composition of the Western economies.
“statistical engineering” (SE) to describe “the study of
In his Youden address, Bisgaard (2012) took an espe-
how to best utilize statistical concepts, methods and
cially perceptive look at this trend and how it should
Downloaded by [University of Arizona] at 14:59 23 February 2016

tools and integrate them with information technol-

affect our research agenda. He concluded that “the field
ogy and other relevant sciences to generate improved
of quality and quality technology need to adapt to a
results” (Hoerl and Snee 2010b, p. 123). They argued
rapidly changing economic environment. We are in
that statistical engineering is essential to solving large,
a global knowledge economy dominated by services
challenging, unstructured problems and implored the
and high knowledge content products and processes”
statistical community to move into leadership roles
(pp. 35–36).
where they will have an impact on such problems. Their
Research specifically stimulated by and directed to
proposal touched off much discussion; see the special
service industries is breaking in to industrial statistics
issue of Quality Engineering devoted to statistical engi-
journals but at a slow pace. It is far from a sufficient
neering in 2012.
response to the sense of urgency described by Bisgaard
What is the interface between statistical engineering
or to the challenging statistical problems in the service
and statistical research? The panel discussion in Qual-
sector. The lack of research in these areas led Doug
ity Engineering (Anderson-Cook and Lu 2012) offers
Montgomery, the editor of Quality and Reliability Engi-
some ideas. Hoerl and Snee (p. 140) pointed to aca-
neering International, to begin 2014 with an appeal to
demic research as essential to give SE a needed theoret-
the quality community to be active participants in the
ical framework. Vining (p. 142) wrote about the need to
knowledge economy and to submit manuscripts based
develop a theory for the use of statistical thinking and
on challenges arising there.
methods to solve complex, unstructured problems. The
need for external funding of SE research was stressed by
Industry–Academia collaboration
Montgomery (p. 142) and Vining (p. 141) and the need
for publication venues by DeHart and Van Mullekom One of the most successful avenues from indus-
(p. 143, 144). trial problems to published research is collaboration
The paper by Kenett (2015) on a life cycle view of between industry and academia. For many involved,
statistics and its ensuing discussion are useful contri- such collaboration is a win–win arrangement: those in
butions to the theory of applied statistics and are rele- industry can benefit from the latest research and can
vant to the development of a theory of SE. DiBenedetto, often enjoy more complete solutions than they would
Hoerl, and Snee (2014) give a constructive discussion have time to develop and justify on their own; aca-
of problem solving in diverse areas. But they are out- demics are tuned in to interesting and relevant prob-
liers from the larger research cloud, which has yet to lems and have an opportunity to see their research
embrace the challenge set down by Hoerl and Snee. make a direct impact. In the Technometrics discus-
Establishing a coherent theory is still more a vision than sion, Bill Meeker encouraged students to spend time
an achievement. as industrial interns; his own career is a good example,
My colleagues did not mention SE per se as a with summers at Bell Labs providing many problems
major topic on the research agenda, but some of their for his research.
 54 D. M. STEINBERG

Most of the industrial colleagues who wrote to me the Joint Statistical Meetings, industrial topics com-
stated that they have colleagues in academia who are prise only a small fraction of the sessions (again exclud-
very happy to collaborate with them and work on prob- ing clinical trials in pharmaceuticals). General interest
lems that arise in industry. A few departments have journals like Annals of Applied Statistics or the “Case
industrial affiliate programs (e.g., Stanford and Virginia Studies and Applications” section of the Journal of the
Tech). American Statistical Association have few articles stim-
My summary of the 2014 literature suggests that the ulated by industrial problems. Major statistics depart-
extent of university–industry collaboration is not high. ments are making an effort to establish a foothold in
Bill Meeker went on to write that he has had only lim- machine learning and data mining, which are highly
ited success in getting his industrial partners to fund relevant to statistical problems in industry associated
research by students. That has also been my own expe- with big data, information technology, and the service
rience. Sometimes the barriers to industry–academia sector. However, they rarely teach courses with a direct
collaboration are legal: companies that insist on not focus on industrial topics or in the classical areas of
divulging proprietary information, even when it is fun- industrial statistics. Most graduates will complete their
damental to getting out good research on the prob- M.Sc. or Ph.D. studies with no exposure to the statis-
lems studied, and universities who regard themselves tical aspects of quality improvement or to many of the
Downloaded by [University of Arizona] at 14:59 23 February 2016

as full partners in any IP that results from the collabo- topics studied in a Six Sigma Black Belt course. (In fact,
ration. See Siegel, Waldman, and Link (2003) and Siegel I suspect that many will graduate never having heard
et al. (2004) for a critical view of the role of univer- about Six Sigma.) Even in departments that have been
sity technology transfer offices; they found that these leaders in the field, interest in industrial careers is wan-
technology transfer offices often discouraged collabo- ing. For example, Bill Woodall (private communica-
ration. The recent review article by Bozeman, Fay, and tion) related to me that there is less interest among stu-
Slade (2013) does not point to any dramatic improve- dents at Virginia Tech in pursuing industrial careers
ment since the works by Siegel et al. (with the exception of the financial and pharmaceutical
branches).

Is industrial statistics on the statistics agenda? Some bright spots

The London Workshop on the Future of Statistics In my review of industrial statistics journals, I found
convened in November 2013. The workshop brought that most of the articles were in one of the established
together about 100 leaders in the profession to discuss categories, such as DOE or reliability. There were also
the challenges in the years ahead. The London Work- articles that broke fresh ground and introduced inter-
shop Report, one of the fruits of that meeting, is aimed esting new problems. Invariably those articles showed
at a wide reader base, not just statisticians. It begins direct contact with a stimulating applied problem. I
with seven short “success stories” to help set the tone want to highlight a few of those articles.
for what statistics can contribute. These include the A second source for interesting problems and new
development and widespread use of randomized clin- territory is special issues that have appeared in some of
ical trials, the application of Bayesian statistics, and the journals. A good example is the section on anomaly
the successful use of statistical arguments in the court- detection published in Technometrics in 2009. One of
room. Oddly, the quality revolution and its statistical the 2015 issues will have a special section with arti-
underpinnings—to me one of the most influential suc- cles on system informatics. The discussions in Qual-
cess stories we can relate—are not mentioned at all. The ity Engineering on statistical engineering (Anderson-
final item is devoted to analytics—in sports and poli- Cook and Lu 2012) and innovation (Jenson 2012) were
tics. The strong business and industrial roots of analyt- also effective in publicizing new topics with impor-
ics are ignored. tant implications for industrial statistics. The January
The absence of industrial statistics from the Lon- 2015 issue of Applied Stochastic Models in Business and
don Workshop is not an isolated event. The statisti- Industry compiled articles on enhancing the impact
cal research community has become increasingly dis- of statistics. Effective use of special issues should cer-
connected from industry. At major conferences like tainly be encouraged. Journal space is limited and
 QUALITY ENGINEERING 55

editors cannot afford to allot too much to special and Hazen (2014) showed how to use ideas from statis-
issues, so it is important to choose special issue topics tical process control to monitor and improve the data
carefully. quality. This article extends well-established methods
Another stimulating source of new topics is the into a new and important arena.
annual knowledge discovery and data mining (KDD)
conference, in particular the “industry and govern-
ment” track. (Full information for 2014 is available Discussion
at http://www.kdd.org/kdd2014/.) Let me mention just It is time to take stock and draw conclusions: what is
a few of the interesting titles from the 2014 confer- the current status of research in industrial statistics?
ence: “Beating the news” with EMBERS: Forecasting Although there are many bright spots, there is much
civil unrest using open source indicators; modeling room for improvement and many missed opportuni-
mass protest adoption in social network communi- ties.
ties using geometric Brownian motion; activity ranking Among the bright spots: We are producing research
in LinkedIn feed; corporate residence fraud detection. at an ever increasing rate. There are many publications
There are many more. It is clear that the KDD confer- each year that are relevant and useful for advancing sta-
ence is attracting many of the interesting IT applica-
Downloaded by [University of Arizona] at 14:59 23 February 2016

tistical methods and practice in industry. My indus-

tions (and industrial writers on them) that the “main- trial colleagues wrote that the literature is addressing
stream” industrial statistics journals would like to pub- many of the topics on their priority list; most of them
lish. observed that they are finding useful articles for their
I will now proceed to a few of the sentinel articles needs. Many interesting statistical modeling problems
that I read. are being published.
Cyber security is a major concern in our A good example is the research on computer exper-
information-based global society. One of the methods iments. Scientists and engineers are making increas-
of cyber attack is “masquearading”—when a hacker ing use of computer models as an experimental plat-
succeeds in entering a network using a legitimate form. For many computer experiments, the results are
login–password combination. An important security deterministic and therefore might be regarded as out-
goal is to identify such an attack from the system side the realm of statistics. The realization that these
commands (mouse clicks and keyboard entries) of the experiments do have important statistical aspects but
phony user. Motivated by an application, Skudlarek also require different methods due to their unique fea-
and Yamomoto (2014) proposed an analysis method tures has spawned a great deal of useful research in the
for identifying anomalous user behavior. They demon- last 25 years. There is still much to learn. Several of my
strated a number of advantages for their algorithm. industrial colleagues reported on the added value that
An especially interesting aspect of this work is the research has had for them in applications.
nonstandard nature of the data (“variety” in the big Among the warning signs: Most of the research con-
data parlance) used to detect anomalies. tinues to be in SPC, DOE, and reliability. The industrial
Afful-Dadzie and Allen (2014) also wrote a pio- statistics journals have established a deserved reputa-
neering article on application of statistical methods to tion for promoting these areas, so it is not surprising
promote cyber security. Motivated by a live applica- that they continue to be major publication domains.
tion, their work exploits Markov decision processes to Nonetheless, the continued emphasis on traditional
study and improve maintenance policies for a computer topics may also be a sign of stagnation. The journals are
system that may be subjected to a cyber attack. The not attracting enough articles on interesting and excit-
method uses online data that are collected by the sys- ing problems that fall outside these traditional areas.
tem and makes a status evaluation that could result in Many challenging statistical questions are arising in
a clear decision as to the status of the system or a rec- the service sector and in IT companies; despite over-
ommendation to collect additional data. tures, the industrial statistics journals are receiving very
Data quality can be a problem in many applications. few submissions on these issues. Few industrial statisti-
Yet there are not many systematic tools for ensuring cians are contributing coauthors. Articles on methods
data quality. Motivated by an integrated case study of abound but articles devoted to principles of practice are
an aircraft maintenance database, Jones-Farmer, Ezell, unusual.
 56 D. M. STEINBERG

Let me return to the four questions that were posed doing so in alternative venues; the strong industrial
early in this article. statistics journals have published only a handful of
these articles. Research by academics is far more often a
response to previously published research (incremental
What is driving statistical research today?
innovation) than work addressing new and challeng-
Based on my summary of journal articles from 2014, ing problems (breakthrough innovation). The evidence
as well as my past experience as an editor, the primary from our publishing record is that most academics are
motivating force is the academic tenure and promo- not sufficiently in touch with real industrial problems.
tion system. A large fraction of the authors in our jour-
nals are in academics and they are publishing because
they are being paid (and promoted) to publish. This Are statistical challenges in industry being better
situation is neither new nor inherently wrong. How- addressed by other research communities?
ever, there are some pitfalls. One is the lack of publish-
I noted above that the traditional industrial statis-
ing activity by statisticians working in industry. They
tics journals are not attracting many articles on new
do not have the same incentive to publish and some-
and challenging industrial problems. An immediate
times are discouraged from publication due to con-
Downloaded by [University of Arizona] at 14:59 23 February 2016

reaction is that such articles are simply being pub-

cerns about sharing information or intellectual prop-
lished elsewhere. Moreover, it is possible that lack of
erty that has competitive value. A second is that most of
receptiveness by the existing journals led authors to
the academics (based on my survey) are not conducting
approach or even to establish other journals where they
research that was stimulated by real problems. A third
found more willing acceptance of their work. A classic
is that it is often easier to publish “standard” research,
example is machine learning—statistics journals were
so that the incentive system may discourage research
slow to publish papers in the area and it quickly became
on more innovative, but risky, topics.
the province of the computer science research commu-
The published statistical research is increasingly
nity, which established its own journals in the area. In
driven by the interface with challenging scientific and
the last 10 years there has been more research by statis-
technological problems. This is particularly evident in
ticians, but the field remains largely identified with
journals like the Annals of Applied Statistics, which has
computer science.
grown into a major publishing venue in less than 10
I began my review of 2014 journals convinced that
years. Most of the articles there grew out of actual prob-
I would find many interesting papers stimulated by
lems, with methods for better statistics on those prob-
industrial problems in other journals. That view proved
lems at the forefront of the article.
mistaken. None of the journals I reviewed stood out as
a venue for addressing industrial challenges. My review
Are industrial challenges affecting the statistical was, of course, limited in coverage. I will be interested
research agenda? to hear of good journals that I missed. The one excep-
tion, noted earlier, was the industry and government
The answer is largely “no.” Other areas appear much
section of the KDD conference. Much of that research is
more dominant in current statistical research. One
published in the conference proceedings but often not
of the striking features from my review of Annals of
in research journals.
Applied Statistics is the very small number of articles
with roots in industrial problems. The fact that indus-
trial statistics was almost totally ignored in the London
Is the statistics research community missing
Workshop reflects its status in the professional hierar-
important opportunities for impact in industry?
chy. What is remarkable is that industry is leading the
way in big data, streaming data, and new types of data The answer is an emphatic “yes.”
and should be a hotbed for statistical research. Thus far, Industry is blazing the big data trail. Analytics is
though, this has not paid many dividends in innovative a major topic on the statistical agenda in industry
statistical research. and business. Where is the accompanying statistical
As noted above, few industrial statisticians are pub- research? Statistical work in the service economy is still
lishing research. Those who are publishing are often only a thin stream in the research literature.
 QUALITY ENGINEERING 57

My industrial colleagues complained about the need problems and to enhance our participation in break-
for more research on statistical practice and the chal- through innovation.
lenges posed by realistic data. Research on statistical
practice will often be conceptual and messy; it does
About the author
not fit well into the precise mathematical templates
that govern most of our research and that promotion David M. Steinberg is Professor of Statistics at Tel Aviv
committees find easy to evaluate. The limited amount University. He has a Ph.D. in statistics from the University of
of foundational research on statistical engineering is a Wisconsin–Madison. He was awarded the 2013 Box Medal by
the European Network for Business and Industrial Statistics. He
good case in point.
was the editor of the leading journal Technometrics from 2008
In today’s world of big data and data science, a cru- to 2010. Professor Steinberg’s primary research area is experi-
cial issue is data quality. No one should have more to mental design, with an emphasis on the design and analysis of
say about data quality than the statistics community. industrial experiments and on experiments run on computer
Yet there is little published work in the area. The article simulators. He is involved in numerous applications in a wide
cited earlier by Jones-Farmer, Ezell, and Hazen (2014) variety of fields. Professor Steinberg was president of the Israel
Statistical Association from 2013 to 2015 and is a member of
is one of the exceptions that highlight the rule. Snee and
ASQ.
Hoerl (2012) give constructive suggestions in a Statis-
Downloaded by [University of Arizona] at 14:59 23 February 2016

tics Roundtable article. Kenett and Shmueli (2014) are

among the few who have begun the challenging task Acknowledgments
of building a conceptual framework for data quality; in
I want to thank the organizers of the Stu Hunter Research Con-
fact, they take a more encompassing view, assessing the ference for inviting me to take on the challenging task of writing
quality of the information. See also the discussion to about the current state of research in industrial statistics.
their article. Much more work is needed in this area. Many colleagues and friends contributed input and ideas that
helped me write this article: William Brenneman, Roger Hoerl,
Brad Jones, Ron Kenett, Bill Myers, Jose Ramirez, Phil Scinto,
Steve Scott, Winson Taam, Amit Teller, Xavier Tort-Martorell,
In summary Theresa Utlaut and Bob Wilkinson. I am indebted to them for
sharing their thoughts and experiences with me. This work
There is much good research being conducted and pub- would never have come to fruition without their help.
lished in industrial statistics. Most of that work pro-
vides incremental advances in areas that have long
References
been on the research front burner. Stu Hunter wrote
about the exciting environment of industrial statistics Afful-Dadzie, A., and T. T. Allen. 2014. Data-driven cyber-
in the 1950s. He and his colleagues then were learning, vulnerability maintenance policies. Journal of Quality Tech-
implementing, and advancing completely new ideas: nology 46:234–50.
Agarwal, D. 2015. Discussion of “Multi-armed bandit exper-
response surface methods, cumulative sum charts, new
iments in the online service economy.” Applied Stochastic
methods for analyzing reliability data, among others. Models in Business and Industry 31:46–47.
When I completed my studies in the early 1980s, the Albrecht, M. C., C. J. Nachtsheim, T. A. Albrecht, and R. D.
quality engineering ideas of Genichi Taguchi had a sim- Cook. 2013. Experimental design for engineering dimen-
ilar invigorating effect, as we attempted to understand sional analysis. Technometrics 55:257–70.
(and improve) the unique and innovative ways that Alzbutas, R., and T. Iešmantas. 2014. Application of Bayesian
methods for age-dependent reliability analysis. Quality and
Taguchi wed engineering, quality, and statistics. Today
Reliability Engineering International 30:121–32.
there are fascinating statistical challenges in industry. Anderson-Cook, C. M., and L. Lu. (eds.). 2012. Statistical
Products become increasingly complex and data for engineering—Forming the foundations. Quality Engineer-
monitoring, testing, and improvement become increas- ing 24:110–32.
ingly rich and varied. There is a need for novel think- Bisgaard, S. 2012. The future of quality technology: From a man-
ing and methods to exploit this environment. We have ufacturing to a knowledge economy and from defects to
innovations. Quality Engineering 24:30–36.
not been successful in getting those challenges into our
Box, G. E. P. 1984. The importance of practice in the develop-
research journals and high on our research agenda. As ment of statistics. Technometrics 26:1–8.
a community, we need to catalyze the process to keep Box, G. E. P., and G. M. Jenkins. 1968. Some recent advances in
our journals and our research focused on cutting-edge forecasting and control, I. Applied Statistics 17:91–109.
 58 D. M. STEINBERG

Box, G. E. P., and W. H. Woodall. 2012. Innovation, quality engi- The London Workshop Report. Statistics and Science: A report of
neering and statistics. Quality Engineering 24:20–29. the London workshop on the future of the statistical sciences.
Bozeman, B., D. Fay, and C. P. Slade. 2013. Research collabo- http://www.worldofstatistics.org/wos/pdfs/Statistics&Scie
ration in universities and academic entrepreneurship: The- nce-TheLondonWorkshopReport.pdf
state-of-the-art. Journal of Technology Transfer 38:1–67. Malley, J. D., and J. H. Moore. 2013. The disconnect between
Carpio, J., J. Juan, and D. López. 2014. Multivariate exponen- classical biostatistics and the biological data mining com-
tial smoothing and dynamic factor model applied to hourly munity. BioData Mining 6.
electricity price analysis. Technometrics 56:494–503. Mallows, C. L. 1973. Some comments on Cp . Technometrics
Chuang, H. H.-C., and R. Oliva. 2014. Estimating retail 15:661–75.
demand with Poisson mixtures and out-of-sample likeli- Manyika, J., M. Chui, B. Brown, J. Bughin, R. Dobbs, C. Rox-
hood. Applied Stochastic Models in Business and Industry bough, and A. H. Byers. 2011. Big data: The next frontier for
30:455–63. innovation, competition and productivity. McKinsey Global
de Bono, E. 1970. Lateral thinking: Creativity step by step. New Institute.
York, NY: Harper & Row. Meeker, W. Q., and Y. Hong. 2014. Reliability meets big data:
DiBenedetto, A., R. W. Hoerl, and R. D. Snee. 2014. Solving jig- Opportunities and challenges. Quality Engineering 26:102–
saw puzzles: Addressing large, complex and unstructured 16.
problems. Quality Progress 50–53. Montgomery, D. C. 2014. Lean six sigma and promoting innova-
Hoerl, R. W. 2000. Discussion. Journal of Quality Technology tion. Quality and Reliability Engineering International, 30:1.
Downloaded by [University of Arizona] at 14:59 23 February 2016

32:351–55. Provost, F., and T. Fawcett. 2013. Data science and its relation to
Hoerl, R. W., and R. D. Snee. 2010a. Closing the gap: Statistical big data and data-driven decision making. Big Data 1:51–59.
engineering links statistical thinking, methods, tools. Qual- Rodriguez, R. N. 2000. Discussion. Journal of Quality Technol-
ity Progress 43 (5):52–53. ogy 32:364–65.
Hoerl, R. W., and R. D. Snee. 2010b. Further explanation: Clar- Scott, S. 2015. Multi-armed bandit experiments in the online
ifying points about statistical engineering. Quality Progress service economy. Applied Stochastic Models in Business and
43 (12):68–72. Industry 31:37–45.
Hoerl, R. W., and R. D. Snee. 2012. Leadership—Essential for Shen, W., T. Davis, D. K. J. Lin, and C. J. Nachtsheim. 2014.
developing the discipline of statistical engineering. Quality Dimensional analysis and its applications in statistics. Jour-
Engineering 24:162–70. nal of Quality Technology 46:185–98.
Hong, Y., and W. Q. Meeker. 2010. Field-failure and warranty Shi, X., J.-F. Paiement, D. Grangier, and P. S. Yu. 2014. GBC:
prediction based on auxiliary use-rate information. Techno- Gradient boosting consensus model for heterogeneous data.
metrics 52:148–59. Statistical Analysis and Data Mining 7:161–74.
Hunter, J. S. 1983. The birth of a journal. Technometrics 25:3–7. Siegel, D. S., D. Waldman, L. E. Atwater, and A. N. Link. 2004.
Hwang, J. Y., and H. C. Lee. 2014. Parametric yield modeling Toward a model of the effective transfer of scientific knowl-
using hidden variable logistic regression. Journal of Quality edge from academicians to practitioners: Qualitative evi-
Technology 46:323–39. dence from the commercialization of university technolo-
Jensen, W. (ed.). 2012. Statistics to facilitate innovation: A panel gies. Journal of Engineering and Technology Management
discussion. Quality Engineering 24:2–19. 21:115–42.
Jones, B., and C. J. Nachtsheim. 2011. A class of three-level Siegel, D. S., D. Waldman, and A. N. Link. 2003. Assessing the
designs for definitive screening in the presence of second- impact of organizational practices on the relative productiv-
order effects. Journal of Quality Technology 43:1–15. ity of university technology transfer offices: An exploratory
Jones-Farmer, L. A., J. D. Ezell, and B. T. Hazen. 2014. Applying study. Research Policy 32:27–48.
control chart methods to enhance data quality. Technomet- Skudlarek, S. J., and H. Yamamoto. 2014. Unsupervised anomaly
rics 56:29–41. detection within non-numerical sequence data by average
Jordan, J. M., and D. K. J. Lin. 2014. Statistics for big data: Are index difference, with application to masquerade detection.
statisticians ready for big data? International Chinese Statis- Applied Stochastic Models in Business and Industry 30:632–
tical Association Bulletin 26:59–66. 56.
Joseph, V. R., T. Dasgupta, R. Tuo, and C. F. J. Wu. 2015. Sequen- Snee, R. D., and R. W. Hoerl. 2012. Inquiry on pedigree: Do you
tial exploration of complex surfaces using minimum energy know the quality and origin of your data? Quality Progress
designs. Technometrics 57:64–74. 65–68.
Kenett, R. S. 2015. Statistics: A life cycle view. Quality Engineer- Steinberg, D. M. (ed.). 2008. The future of industrial statistics:
ing 27:111–21. A panel discussion. Technometrics 50:103–27.
Kenett, R. S., and G. Shmueli. 2014. On information quality. Taylor, S. L., I. A. Eckley, and M. A. Nunes. 2014. A test of sta-
Journal of the Royal Statistical Society, Series A 177:3–27. tionarity for textured images. Technometrics 56:291–301.
Kenett, R. S. and Shmueli, G. (eds.). (2015). A special issue on: Thomke, S., and J. Manzi. 2014. The discipline of business
Actual impact and future perspectives on stochastic mod- experimentation. Harvard Business Review 92:71–79.
elling in business and industry. Applied Stochastic Models in Vander Wiel, S., R. Bent, E. Casleton, and E. Lawrence. 2014.
Business and Industry 31:1–2. Identification of topology changes in power grids using
 QUALITY ENGINEERING 59

phasor measurements. Applied Stochastic Models in Business Xiao, L., D. K. J. Lin, and F. Bai. 2012. Constructing defini-
and Industry 30:740–52. tive screening designs using conference matrices. Journal of
Wang, A., K. Wang, and F. Tsung. 2014. Statistical surface mon- Quality Technology 44:2–8.
itoring by spatial-structure modeling. Journal of Quality Yang, J., D. K. J. Lin, and M.-Q. Liu. 2014. Construction of
Technology 46:359–76. minimal-point mixed-level screening designs using confer-
Woodall, W. H., D. J. Spitzner, D. C. Montgomery, and S. ence matrices. Journal of Quality Technology 46:251–64.
Gupta. 2004. Using control charts to monitor process and Zhu, Y., E. Yashchin, and J. R. M. Hosking. 2014. Parametric
product quality profiles. Journal of Quality Technology 36: estimation for window censored recurrence data. Techno-
309–20. metrics 56:55–66.
Downloaded by [University of Arizona] at 14:59 23 February 2016